[WIP] Add umbrella optimization flags

[IanWood1]

This change adds support for optimization flags that can be used to broadly control compiler optimizations. Currently, iree-opt-level is the name of the flag that would control the overall optimization level for the compiler invocation and it would cascade down to more fine-grained flags (i.e. iree-opt-strip-assertions). This will give users an easy way to control the compilation process and reduce the number of flags required to get the desired optimizations.

Progress towards #19072

[ScottTodd]

Just dropping a reminder here: we have some existing docs for "optimization options" that are due to a rework, especially after these changes land: https://iree.dev/reference/optimization-options/

We could even add a snippet that we include on each page like https://iree.dev/guides/deployment-configurations/cpu/#compile-and-run-a-program and https://iree.dev/guides/deployment-configurations/gpu-vulkan/#compile-and-run-a-program that explains what to expect out of the box and how to optimize / tune / configure from there (linking to the optimization options reference page).

[IanWood1]

@benvanik I attempted to do similar to what you suggested but I've run into some problems.

As an example init_at_opt_level would be the function used to tell the binder to set stripAssertions to true when at O3 (and there wouldn't be a need for the manual applyOptimization):

 binder.opt<bool>(
      "iree-opt-strip-assertions",
      stripAssertions,
      init_at_opt_level(O3, true));

The problem I'm having is that iree-opt-strip-assertions should be set based upon the value iree-global-optimization-opt-level and I don't have a good way to let the binder know that.

There is a secondary problem, too. As you mentioned, the options shouldn't be directly set but rather a copy should be made. This is needed because we don't want setting optimization levels to affect the other flags for a session. This makes it hard (impossible?) to to make it so that GlobalOptimizationOptions::stripAssertions should be set based on GlobalOptimizationOptions::optLevel from inside of of bindOptions because the GlobalOptimizationOptions object we use to bind won't be the one that has the optimizations applied.

[benvanik]

This is a real good start!

There's a few super magic ways we could do things, but for v0 I'd have your applyOptimization above walk through the options registered on a binder and if there are registered init_at_opt_level entries perform the override. It's a bit of boilerplate we could remove with template magic but copy/pasting for now is fine. something like:

applyOptimization:
  binder.overrideDefault("...-opt-level", optLevel, ...);
  binder.overrideOptimizationDefaults(optLevel);
  // if there are nested binders, call applyOptimization on them

The fancier automatic way would be to have the pipeline opt level option be stashed by the binder - instead of binder.opt<OptimizationLevel>(...) could have binder.optimizationLevel("...-opt-level") to make it explicit what the primary scoped optimization level option is and then the applyOptimization default impl above can do everything itself: the top level gets called, it checks if it has a registered optimization level option, overrides it if needed, and then calls overrideOptimizationDefaults. Then we'd only need to override it if we had nested options (but even that could be improved, and for now we could ignore).

[IanWood1]

This is still a WIP, but I opened the PR and requested reviews to get some feedback on the general approach. There is definitely some cleanup needed + only 1 flag is setup to use the optimization levels. I would like to hear some thoughts on #20047 (comment) before flushing this out.

[benvanik]

This is a real good start!

There's a few super magic ways we could do things, but for v0 I'd have your applyOptimization above walk through the options registered on a binder and if there are registered init_at_opt_level entries perform the override. It's a bit of boilerplate we could remove with template magic but copy/pasting for now is fine. something like:

applyOptimization:
  binder.overrideDefault("...-opt-level", optLevel, ...);
  binder.overrideOptimizationDefaults(optLevel);
  // if there are nested binders, call applyOptimization on them

The fancier automatic way would be to have the pipeline opt level option be stashed by the binder - instead of binder.opt<OptimizationLevel>(...) could have binder.optimizationLevel("...-opt-level") to make it explicit what the primary scoped optimization level option is and then the applyOptimization default impl above can do everything itself: the top level gets called, it checks if it has a registered optimization level option, overrides it if needed, and then calls overrideOptimizationDefaults. Then we'd only need to override it if we had nested options (but even that could be improved, and for now we could ignore).

[IanWood1]

@benvanik replying to your comment above, I was trying to get this to work but I think there is a problem with having the binder automatically apply flag modifications specified with init_at_opt_level.

Maybe its a problem with how I'm trying to implement it, so I will share some details. I was thinking that init_at_opt_level would basically instruct the binder to store a callback that would take store reference to the member of the options struct (e.g. stripAssertions). That way it can get updated in applyOptimization. However, this is dangerous because the reference stored in the binder is not always the same object that applyOptimizations is called on. This is why I initially opted for manual updates in applyOptmizations.

For example:

GlobalOptimizationOptions options0;
{
  GlobalOptimizationOptions options1;
  // binds options and stores reference to `options1.stripAssertions` to be updated 
  // when `applyOptimizations` is called.
  options1.bindOptions(binder);
  options0 = optioins1;
}

// set from flags...

// Problematic because binder is storing references to options1's members.
// So, `options0` never gets updated and this is UB
options0.applyOptimizations(binder);

[benvanik]

I didn't think of using callbacks and instead just storing the data as a field on the options info that is cloneable - SmallVector or something of the optimization level -> value.

[IanWood1]

I didn't think of using callbacks and instead just storing the data as a field on the options info that is cloneable - SmallVector or something of the optimization level -> value.

Okay, so this would still require something that passes this field to the binder? Something roughly similar to

void applyOptimizations(...){
  binder.overrideDefault("...-opt-level", optLevel, ...);
  binder.overrideDefaultWith(optLevel, stripAssertions, stripAssertionsOptToDefaultMap);
}

And the benefit would be printability of default levels at each optimization level because they are declared during binding?

[benvanik]

I don't think so. You'd have a subclass of opt (or Option) - optimization_level_opt or something - that added an std::vector<std::tuple<OptimizationLevel, DataType>> of your map and an overload of setInitialValue that took an optimization level along with the value and added to that vector. Then you would have your element declaration struct init_at_opt_level (or whatever) ala init/initializer that calls your overloaded setInitialValue function. Your custom printOptionInfo could then walk the vector and print out the values. OptionsBinder::opt could then be changed to do a make_unique of your new opt type instead of the base one and you could stash in the LocalOptionInfo that it was something to apply optimizations to (I don't think there's dyn_cast unfortunately). Your top-level applyOptimizationLevel would walk through the LocalOptionInfo and if the option was your custom opt type cast and do whatever it needs with the std::vector on it.

A user should then be able to do something like

 binder.opt<bool>(
      "iree-opt-strip-assertions",
      stripAssertions,
      init_at_opt_level(O2, true));

(that make_unique'ing your optimization_level_opt, the init_at_opt_level calling the setInitialValue(O3, true) to add it to the opt vector, and your generic applyOptimizations walking over it to do the "if initial value then walk vector and find the max opt level value and use that" (e.g. the O2 option should be picked up above if -O3 was specified))

(rough idea, but it should work just as list works today with cloning and such for local scopes, still appear as an opt to most users and take all the same declarations, just also takes the new ones)