Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize airflow variables on docker creation. #31

Open
jimk-bdrc opened this issue Nov 25, 2024 · 0 comments
Open

Initialize airflow variables on docker creation. #31

jimk-bdrc opened this issue Nov 25, 2024 · 0 comments
Assignees

Comments

@jimk-bdrc
Copy link
Contributor

I want the list of directories that feed scans to be an airflow variable.

Github copilot starts with this approach.

To initialize Airflow variables when running in a Docker container, you can use the Airflow CLI within the container. Here is a step-by-step guide:

  1. Create a JSON file with the variables you want to initialize. For example, variables.json:

    {
      "key1": "value1",
      "key2": "value2"
    }
  2. Copy the JSON file into the Docker container. You can do this using the docker cp command:

    docker cp variables.json <container_id>:/path/to/variables.json
  3. Use the Airflow CLI to import the variables from the JSON file. Execute the following command inside the container:

    docker exec -it <container_id> airflow variables import /path/to/variables.json

Here is a complete example:

  1. Create variables.json:

    {
      "key1": "value1",
      "key2": "value2"
    }
  2. Copy the file into the container:

    docker cp variables.json <container_id>:/opt/airflow/variables.json
  3. Import the variables using the Airflow CLI:

    docker exec -it <container_id> airflow variables import /opt/airflow/variables.json

Replace <container_id> with the actual ID of your Airflow container. This will initialize the Airflow variables from the JSON file.

@jimk-bdrc jimk-bdrc self-assigned this Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant