kedro-airflow crashes with kedro<=0.18.3

[luizvbo]

Description

I was trying to use kedro-airflow with kedro==0.18.2 and was getting the error AttributeError: 'KedroContext' object has no attribute 'config_loader'.

Context

I tried different versions of kedro and the bug was solved by installing kedro==0.18.4.

I noticed "kedro>=0.17.5" in the requirements list for kedro-airflow. The requirement could be bumped or the bug could be fixed.

Steps to Reproduce

pip install kedro==0.18.3 kedro-airflow
kedro new --starter=spaceflights # Use default values
cd spaceflights
# Remove kedro-viz from src/requirements.txt (to avoid conflicts)
pip install -r src/requirements.txt
kedro airflow create

Actual Result

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /tmp/venv/bin/kedro:8 in <module>                                                                │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/kedro/framework/cli/cli.py:211 in main                     │
│                                                                                                  │
│   208 │   """                                                                                    │
│   209 │   _init_plugins()                                                                        │
│   210 │   cli_collection = KedroCLI(project_path=Path.cwd())                                     │
│ ❱ 211 │   cli_collection()                                                                       │
│   212                                                                                            │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:1157 in __call__                             │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/kedro/framework/cli/cli.py:139 in main                     │
│                                                                                                  │
│   136 │   │   )                                                                                  │
│   137 │   │                                                                                      │
│   138 │   │   try:                                                                               │
│ ❱ 139 │   │   │   super().main(                                                                  │
│   140 │   │   │   │   args=args,                                                                 │
│   141 │   │   │   │   prog_name=prog_name,                                                       │
│   142 │   │   │   │   complete_var=complete_var,                                                 │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:1078 in main                                 │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:1688 in invoke                               │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:1688 in invoke                               │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:1434 in invoke                               │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/core.py:783 in invoke                                │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/click/decorators.py:45 in new_func                         │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/kedro_airflow/plugin.py:104 in create                      │
│                                                                                                  │
│   101 │   bootstrap_project(project_path)                                                        │
│   102 │   with KedroSession.create(project_path=project_path, env=env) as session:               │
│   103 │   │   context = session.load_context()                                                   │
│ ❱ 104 │   │   dag_config = _load_config(context, pipeline_name)                                  │
│   105 │   │                                                                                      │
│   106 │   │   # Update with params if provided                                                   │
│   107 │   │   dag_config.update(params)                                                          │
│                                                                                                  │
│ /tmp/venv/lib/python3.8/site-packages/kedro_airflow/plugin.py:37 in _load_config                 │
│                                                                                                  │
│    34                                                                                            │
│    35 def _load_config(context: KedroContext, pipeline_name: str) -> dict[str, Any]:             │
│    36 │   # Set the default pattern for `airflow` if not provided in `settings.py`               │
│ ❱  37 │   if "airflow" not in context.config_loader.config_patterns.keys():                      │
│    38 │   │   context.config_loader.config_patterns.update(  # pragma: no cover                  │
│    39 │   │   │   {"airflow": ["airflow*", "airflow/**"]}                                        │
│    40 │   │   )                                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'ConfigLoader' object has no attribute 'config_patterns'```

Your Environment

Python 3.8.10 on Linux

[astrojuanlu]

Hi @luizvbo, thanks for reporting this. I contend that we should bump the version unless there's a strong push to support older ones, @merelcht @ankatiyar what do you think?

[ankatiyar]

This is because of this PR #233 which needs a Kedro session and the config loader. kedro-airflow<0.6.0 should work with older versions of Kedro but I think we must have added config_patterns argument to config loaders later(possibly after 0.18.3)?

[sbrugman]

Are you open to adding a simple fallback to the plugin when an older ConfigLoader is used?

It could be as simple as:

This adds very little overhead, and allows users stuck at <= 0.18.2 to use the plugin too.
They will not have the option to configure the configuration pattern unless they upgrade.