# Add Params to Airflow DAGs

Yes. An agent turns the hard-coded names, buckets and images in your DAG files into typed Airflow params with defaults and validation, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-dag-params/airflow/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads every DAG file for values a person would want to change at trigger time. In astronomer/airflow-example-dags, the `Dockerfile` is one line, `FROM quay.io/astronomer/astro-runtime:6.0.0`, `requirements.txt` is empty, `packages.txt` adds `postgresql`, and `.astro-registry.yaml` lists five DAG files. `s3_upload_dag.py`, `s3_upload_copy_dag.py` and `s3_upload_copy_delete.py` all set `S3_CONN_ID='astro-s3-workshop'` and `BUCKET='astro-workshop-bucket'` as module constants, carry a `name` variable with the comment `#swap your name here` (`'workshop'` in two files, `"your_name_here"` in the third), and generate their upload tasks in a `for i in range(...)` loop whose `op_kwargs` and `S3CopyObjectOperator` keys embed that name. `kube_pod.py` hard-codes `image="hello-world"` and a `compute_resource` dict of 200m CPU and 1Gi memory, and `databricks_sample.py` carries a `job_info` list with `notebook_params` of `inPath` per job, a `PostgresOperator` query per job and a `S3KeySensor` on `globetelecom/copy_*`. No DAG declares `params`, no task reads `params`, and the files use `schedule_interval` and the deprecated `airflow.operators.dummy_operator` and `python_operator` import paths.

The change is to declare those values as params instead of constants. Each DAG gets a `params` dict of `Param` objects from `airflow.models.param`: `name` as `Param('workshop', type='string', minLength=1, description='Prefix for the generated files')`, `bucket` and `prefix` as strings, `file_count` as `Param(5, type='integer', minimum=1, maximum=50)` in place of the loop's literal, `image` and the resource requests in `kube_pod.py`, and the `inPath` values in the Databricks DAG as a `Param` with `type='array'`. A typed `Param` must have a valid default, and because these DAGs have a schedule, Airflow validates the defaults at parse time, so a wrong default shows up as an import error rather than at the first run. Values are read where the operators run, not where the file is imported: templated fields such as `S3CopyObjectOperator.source_bucket_key` get `{{ params.prefix }}/{{ params.name }}_file_{{ ti.map_index }}.txt` strings, and the `PythonOperator` callables take `**context` and read `context['params']['name']`, which carries a manual override, where `dag.params` carries only the defaults.

The loop that generates tasks is the one place params cannot reach directly, since the number of tasks is fixed when the file is parsed and a param is resolved per run. The agent rewrites each `for i in range(N)` into dynamic task mapping, `PythonOperator.partial(...).expand(op_kwargs=...)` fed by a small task that returns a list of indexes built from `params['file_count']`, so the count becomes a run-time value, and says plainly in the pull request that this is the one structural change. `render_template_as_native_obj=True` goes on each `DAG` so an integer param stays an integer inside a template instead of becoming the string `'5'`. Runtime 6.0.0 ships Airflow 2.4, where `Param` and its validation exist but the trigger form that draws a field per param arrived in 2.6; the pull request shows the `airflow dags trigger --conf '{"name": "alice"}'` form that works on 2.4 and names the Runtime upgrade that adds the form.

Checking happens in the sandbox without a scheduler. The agent installs the Airflow version the Runtime image carries with its constraints file, writes `tests/dags/test_dag_integrity.py` that loads a `DagBag` from `dags/` and asserts there are no import errors and that every DAG declares `params`, and runs it with `pytest`. It also runs `airflow dags list-import-errors` and renders one templated field with `airflow tasks render` for a chosen task to show the param substituted. The deprecated import paths are left as they are, since changing them is a separate upgrade, but the pull request lists them.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add Airflow params to the DAGs in this project. Read the Dockerfile, requirements.txt, packages.txt, .astro-registry.yaml and every file under dags/ first, and list each hard-coded value a person would change at trigger time: names, bucket names and prefixes, file counts, images, resource requests, notebook paths. Declare each as a typed Param from airflow.models.param with a sensible default, type and constraints in a params dict on the DAG, read them through templated fields or context['params'] rather than module constants, set render_template_as_native_obj on each DAG, and where a loop generates tasks from a constant replace it with dynamic task mapping so the count can be a param. Keep the schedules, connections and operators as they are. Install the Airflow version the Runtime image pins with its constraints, add a DagBag import test under tests/dags that also asserts every DAG declares params, run it, run airflow dags list-import-errors, and render one templated field to show a param substituted. Open a pull request with a table of params per DAG and the trigger command with --conf.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request changes the five DAG files under `dags/`, adds `tests/dags/test_dag_integrity.py`, and adds `pytest` to a `requirements-dev.txt` so the test is runnable outside the image. Its description has one table per DAG with each param's name, type, default and constraints, the `airflow dags trigger ... --conf` line for a run with overrides, the DagBag test output, the `list-import-errors` output, and the rendered template of one task showing the default substituted.

A reviewer checks the defaults against the values the DAGs ran with, because a default is what every scheduled run now uses; decides whether the bucket and connection id should be params or stay as constants an operator should not be able to change at trigger time; and reads the dynamic-mapping rewrite of the upload loops, which is the one change that alters the task ids in the grid view.

The `Dockerfile`, `packages.txt`, the schedules and the connections are untouched, and nothing deploys: the branch goes through the team's own Astro deploy.

## Run it on every Airflow project with hard-coded values

A platform team with several Astro projects can send this body once per repository from a short script on its own server: list what the GitHub App reaches with `GET /o/{org}/github`, keep the repositories whose default branch has a `dags/` directory, and give each request a `request_id` that is a UUID v5 of the repository's full name so a rerun returns the runs it already started. Each run opens one pull request, and the key's `webhook_url` receives `run.completed` for each.

After the first pass, an automation on the Project with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` can run read-only with a prompt that reads any changed file under `dags/` and comments when a new DAG introduces a module-level constant a `Param` should hold. The trigger adds the event after the prompt; it fills in no placeholder.

## Why run it on Rush?

The hard part of params is not the `Param` class but the rules around it: defaults are validated at parse time only when the DAG has a schedule, a templated string becomes a `str` unless the DAG renders native objects, `dag.params` does not see a manual override but `context['params']` does, and a loop that builds tasks from a constant has to become dynamic mapping before a count can vary. The agent applies each rule file by file, runs the DagBag test in its sandbox on the Airflow version the image pins, and shows one rendered template, so the pull request carries evidence rather than a promise.

Doing this across many DAG repositories is the same reading repeated, and a run per repository keeps every command and its output in that run's record.

## Frequently asked questions

### Will scheduled runs change?

No, as long as each default equals the value the file carried before. A scheduled run uses the defaults; only a manual trigger with `--conf` or the trigger form supplies other values. The pull request's tables are there so a reviewer can check every default against the old constant.

### Where is the trigger form?

The form that renders a field per `Param` arrived in Airflow 2.6. The Runtime 6.0.0 image in this project's `Dockerfile` ships Airflow 2.4, where params and their validation work but the UI takes raw JSON, so the pull request documents `airflow dags trigger --conf` and names the Runtime bump that adds the form.

### Why dynamic task mapping for the loops?

The `for i in range(5)` loops run when the file is parsed, so the task count is fixed before any run exists and a param cannot change it. `expand()` builds the mapped tasks per run from a list a small task returns, which is the only way a count can come from `params`. It changes the task ids, which the pull request calls out.

### Can a param be marked read-only?

Setting `core.dag_run_conf_overrides_params` to `False` turns every param into a constant for all DAGs. Per value, the agent keeps anything an operator should not change at trigger time, such as the connection id, as a module constant rather than a `Param`, and says so in the table.

## More Airflow jobs

- [Add Timeouts to Airflow DAGs](https://getrush.ai/run/add-dag-timeouts/airflow/)
- [Add Data-Aware Scheduling to Airflow DAGs](https://getrush.ai/run/add-data-aware-scheduling/airflow/)
- [Add Deferrable Operators to an Airflow Project](https://getrush.ai/run/add-deferrable-operators/airflow/)

## More jobs like this: Hand off a long task

- [Add Dark Mode to a Blazor Web App](https://getrush.ai/run/add-dark-mode/blazor/)
- [Add a Dark Mode Toggle to an Eleventy Site](https://getrush.ai/run/add-dark-mode/eleventy/)
- [Add Dark Mode to a Jekyll Site](https://getrush.ai/run/add-dark-mode/jekyll/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
