Setting HydraConfig parameters in structured configs

[NickGeneva]

🚀 Feature Request

Hello everyone,

I was curious if there was a good way to override hydra related settings inside of a structured config rather than in the main config.yaml. Namely items found in HydraConf. I haven't been able to find much on this in the docs and issues, please point me to related information if it exists. Thank you!

Motivation

The idea is that I want to tuck away some of the hyrda config settings away from the user defined .yaml config. This would allow the definition of different structured configs with specific hydra settings already set. A good example would be a default schema and a debug schema (with hydra.verbose on and a different hydra.job_logging).

Is your feature request related to a problem? Please describe.
I believe this may be related to issue #1830.

Pitch

Here would be a little example of the functionality I am looking to do using the docs SQL database configs. In conf_library.py with my structured configs:

@dataclass
class DBConfig:
    driver: str = MISSING
    host: str = "localhost"
    port: int = MISSING

@dataclass
class MySQLConfig(DBConfig):
    driver: str = "mysql"
    port: int = 3306
    user: str = MISSING
    password: str = MISSING

    # Does not work
    hydra: HydraConf = HydraConf()
    hydra.run.dir = "sql_outputs/${hydra.job.override_dirname}/${hydra.job.name}"

@dataclass
class MySQLiteConfig(DBConfig):
    driver: str = "mysqllite"
    port: int = 3306
    user: str = MISSING
    password: str = MISSING

    # Does not work
    hydra: HydraConf = HydraConf()
    hydra.run.dir = "sqllite_outputs/${hydra.job.override_dirname}/${hydra.job.name}"

def register_configs():
    cs = ConfigStore.instance()
    cs.store(name="base_config", node=DBConfig)
    cs.store(name="sql_config", node=MySQLConfig)
    cs.store(name="sqllite_config", node=MySQLiteConfig)

In main.py:

from conf_library import register_configs 

register_configs()

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    my_app()

In conf/config.yaml:

defaults:
  - sql_config # Can quickly change to sqllite_config with updated run dir
  - _self_

user: omry
password: secret

This currently does not work due to hydra saying hydra.searchpath is only supported from the primary config.

• Hydra Version : 1.1.1
• Python version : 3.8.12
• Virtual environment type and version : MiniConda environment
• Operating system : Ubuntu

Describe alternatives you've considered
I've managed to hack my way around some of the hydra settings, such as hydra.job_logging and hydra.help by defining configs using dictionaries and setting them through defaults in the schema. I can elaborate if needed, but I think this is a bit improper.

Are you willing to open a pull request? (See CONTRIBUTING)
Not right now since I do not know if this is a reasonable request.

Additional context

None

[Jasha10]

When figuring out how to write a structured config, it's sometimes productive to start with a yaml config and work backwards from there.
For example, suppose you want to produce a structured config that is equivalent to the following yaml:

hydra:
  run:
    dir: "sql_outputs/${hydra:job.override_dirname}/${hydra:job.name}"

The equivalent structured config should have a field called hydra with a default value given by the python dictionary {"run": {"dir": "sql_outputs/${hydra:job.override_dirname}/${hydra:job.name}"}}.
Thus, we have the following:

from dataclasses import dataclass, field

@dataclass
class Foo:
    hydra: Any = field(default_factory=lambda: {
        "run": {
            "dir": "sql_outputs/${hydra:job.override_dirname}/${hydra:job.name}"
        }
    })

I've managed to hack my way around ... setting them through defaults in the schema.

Using the defaults list is certainly a viable option. Once again starting with a yaml file:

defaults:
  - foo
  - bar: baz
  - override alpha/beta: gamma

The equivalent structured config should look something like this:

from dataclasses import dataclass, field

@dataclass
class MyConf:
    defaults: List[Any] = field(default_factory=lambda: [
        "foo",
        {"bar": "baz"},
        {"override alpha/beta": "gamma"},
    ])

You can check that the data stored in the yaml file is equivalent to the data from the structured config file using the following technique:

from omegaconf import OmegaConf
from dataclasses import dataclass, field

conf1 = OmegaConf.create("""
defaults:
  - foo
  - bar: baz
  - override alpha/beta: gamma
""")

@dataclass
class MyConf:
    defaults: List[Any] = field(default_factory=lambda: [
        "foo",
        {"bar": "baz"},
        {"override alpha/beta": "gamma"},
    ])

conf2 = OmegaConf.create(MyConf)

assert conf1 == conf2

In general, anything that can be done using yaml can also be done using python objects.

[NickGeneva]

Hi @Jasha10

I greatly appreciate the detailed answer and I'm glad that using dictionaries is a fine approach. Based on the documentation, I was thinking that one should attempt to use a data class for everything but this doesn't work for the hydra settings. I am not sure if this is actually present, but perhaps adding your explanation regarding the use of dictionaries being equal to yaml files would be a good addition to the structured config section.

Your suggestions work great, however, I do have a follow-up involving something that I was originally testing regarding setting hydra parameters through the defaults list. I cannot seem to understand why some items I can set through default groups and others I can't. Does this involve how they are initialized in the HydraConf?

Here is an example:

@dataclass
class DBConfig:
    driver: str = MISSING
    host: str = "localhost"
    port: int = MISSING


hydra_run = {"dir": "sql_outputs/${hydra.job.override_dirname}/${hydra.job.name}"}
hydra_help = {
    "app_name": "my_app",
    "header": "A Nice header",
    "footer": "A Nice footer",
    "template": "${hydra.help.header} \n\n This is the ${hydra.help.app_name} help page that is very useful. Wow. \n ${hydra.help.footer}",
}


@dataclass
class MySQLConfig(DBConfig):
    defaults: List[Any] = field(default_factory=lambda: [
        {"override hydra/run": "hydra_run"}, # This does not work
        {"override hydra/help": "hydra_help"}, # This does work
    ])

    driver: str = "mysql"
    port: int = 3306
    user: str = MISSING
    password: str = MISSING

    # This works fine
    # hydra: Any = field(default_factory=lambda: {
    #     "run": {"dir": "sql_outputs/${hydra.job.override_dirname}/${hydra.job.name}"}
    # })


def register_configs():
    cs = ConfigStore.instance()
    cs.store(group= "hydra/run", name="hydra_run", node=hydra_run)
    cs.store(group= "hydra/help", name="hydra_help", node=hydra_help)
    cs.store(name="base_config", node=DBConfig)
    cs.store(name="sql_config", node=MySQLConfig)

My best guess is that some of the Hydra parameters are instanced directly to be class objects rather through the defaults list.

Regardless thank you for the help! Cheers.

[Jasha10]

There are a few things going on here.
First, take a look at the definition of the HydraConf dataclass. This dataclass defines the schema for the top-level hydra config.

Looking at the defaults list defined by this dataclass, you will see that help appears as a key in this defaults list, whereas run does not. This explains why hydra/help can be overridden but hydra/run cannot; in the case of hydra/run, there is nothing to override, so setting the default {"override hydra/run": "hydra_run"} cannot succeed.

In theory, it should be possible to use the default setting {"hydra/run": "hydra_run"} (omitting the "override" keyword) as below:

@dataclass
class MySQLConfig(DBConfig):
    defaults: List[Any] = field(default_factory=lambda: [
        {"hydra/run": "hydra_run"},
        {"override hydra/help": "hydra_help"},
    ])
    ...

However, this does not work with Hydra 1.1 because of some deprecated legacy behavior. The legacy behavior is that defaults list settings starting with the prefix "hydra/" are automatically treated as if the "override" keyword was used. This means, in Hydra 1.1, writing {"hydra/run": "hydra_run"} is treated as equivalent to {"override hydra/run": "hydra_run"}, which brings us back to the issue mentioned in the first paragraph of this comment.

I have just submitted a PR (#2056) removing this deprecated legacy behavior; if/when PR is merged, it will be possible to use {"hydra/run": "hydra_run"} as above. Feel free to test-drive the PR and follow up if you have any issues with it.

For Hydra 1.1, the following workaround is available:

@dataclass
class MySQLConfig(DBConfig):
    defaults: List[Any] = field(default_factory=lambda: [
        {"[email protected]": "hydra_run"},
        {"override hydra/help": "hydra_help"},
    ])
    ...

def register_configs():
    cs = ConfigStore.instance()
    cs.store(group= "my_settings", name="hydra_run", node=hydra_run)
    ...

[Jasha10]

@NickGeneva feel free to follow up here if you have any additional questions.

This currently does not work due to hydra saying hydra.searchpath is only supported from the primary config.

Btw this restriction does hold for hydra.searchpath, but many other hydra settings can be configured from non-primary configs. Feel free to open another feature request related to the searchpath specifically.