What the agent does
The agent reads every DAG file for values a person would want to change at trigger time. In astronomer/airflow-example-dags, the Dockerfile is one line, FROM quay.io/astronomer/astro-runtime:6.0.0, requirements.txt is empty, packages.txt adds postgresql, and .astro-registry.yaml lists five DAG files. s3_upload_dag.py, s3_upload_copy_dag.py and s3_upload_copy_delete.py all set S3_CONN_ID='astro-s3-workshop' and BUCKET='astro-workshop-bucket' as module constants, carry a name variable with the comment #swap your name here ('workshop' in two files, "your_name_here" in the third), and generate their upload tasks in a for i in range(...) loop whose op_kwargs and S3CopyObjectOperator keys embed that name. kube_pod.py hard-codes image="hello-world" and a compute_resource dict of 200m CPU and 1Gi memory, and databricks_sample.py carries a job_info list with notebook_params of inPath per job, a PostgresOperator query per job and a S3KeySensor on globetelecom/copy_*. No DAG declares params, no task reads params, and the files use schedule_interval and the deprecated airflow.operators.dummy_operator and python_operator import paths.
The change is to declare those values as params instead of constants. Each DAG gets a params dict of Param objects from airflow.models.param: name as Param('workshop', type='string', minLength=1, description='Prefix for the generated files'), bucket and prefix as strings, file_count as Param(5, type='integer', minimum=1, maximum=50) in place of the loop's literal, image and the resource requests in kube_pod.py, and the inPath values in the Databricks DAG as a Param with type='array'. A typed Param must have a valid default, and because these DAGs have a schedule, Airflow validates the defaults at parse time, so a wrong default shows up as an import error rather than at the first run. Values are read where the operators run, not where the file is imported: templated fields such as S3CopyObjectOperator.source_bucket_key get {{ params.prefix }}/{{ params.name }}_file_{{ ti.map_index }}.txt strings, and the PythonOperator callables take **context and read context['params']['name'], which carries a manual override, where dag.params carries only the defaults.
The loop that generates tasks is the one place params cannot reach directly, since the number of tasks is fixed when the file is parsed and a param is resolved per run. The agent rewrites each for i in range(N) into dynamic task mapping, PythonOperator.partial(...).expand(op_kwargs=...) fed by a small task that returns a list of indexes built from params['file_count'], so the count becomes a run-time value, and says plainly in the pull request that this is the one structural change. render_template_as_native_obj=True goes on each DAG so an integer param stays an integer inside a template instead of becoming the string '5'. Runtime 6.0.0 ships Airflow 2.4, where Param and its validation exist but the trigger form that draws a field per param arrived in 2.6; the pull request shows the airflow dags trigger --conf '{"name": "alice"}' form that works on 2.4 and names the Runtime upgrade that adds the form.
Checking happens in the sandbox without a scheduler. The agent installs the Airflow version the Runtime image carries with its constraints file, writes tests/dags/test_dag_integrity.py that loads a DagBag from dags/ and asserts there are no import errors and that every DAG declares params, and runs it with pytest. It also runs airflow dags list-import-errors and renders one templated field with airflow tasks render for a chosen task to show the param substituted. The deprecated import paths are left as they are, since changing them is a separate upgrade, but the pull request lists them.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add Airflow params to the DAGs in this project. Read the Dockerfile, requirements.txt, packages.txt, .astro-registry.yaml and every file under dags/ first, and list each hard-coded value a person would change at trigger time: names, bucket names and prefixes, file counts, images, resource requests, notebook paths. Declare each as a typed Param from airflow.models.param with a sensible default, type and constraints in a params dict on the DAG, read them through templated fields or context['params'] rather than module constants, set render_template_as_native_obj on each DAG, and where a loop generates tasks from a constant replace it with dynamic task mapping so the count can be a param. Keep the schedules, connections and operators as they are. Install the Airflow version the Runtime image pins with its constraints, add a DagBag import test under tests/dags that also asserts every DAG declares params, run it, run airflow dags list-import-errors, and render one templated field to show a param substituted. Open a pull request with a table of params per DAG and the trigger command with --conf.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
The pull request changes the five DAG files under dags/, adds tests/dags/test_dag_integrity.py, and adds pytest to a requirements-dev.txt so the test is runnable outside the image. Its description has one table per DAG with each param's name, type, default and constraints, the airflow dags trigger ... --conf line for a run with overrides, the DagBag test output, the list-import-errors output, and the rendered template of one task showing the default substituted.
A reviewer checks the defaults against the values the DAGs ran with, because a default is what every scheduled run now uses; decides whether the bucket and connection id should be params or stay as constants an operator should not be able to change at trigger time; and reads the dynamic-mapping rewrite of the upload loops, which is the one change that alters the task ids in the grid view.
The Dockerfile, packages.txt, the schedules and the connections are untouched, and nothing deploys: the branch goes through the team's own Astro deploy.
Run it on every Airflow project with hard-coded values
A platform team with several Astro projects can send this body once per repository from a short script on its own server: list what the GitHub App reaches with GET /o/{org}/github, keep the repositories whose default branch has a dags/ directory, and give each request a request_id that is a UUID v5 of the repository's full name so a rerun returns the runs it already started. Each run opens one pull request, and the key's webhook_url receives run.completed for each.
After the first pass, an automation on the Project with a github trigger on pull_request.opened and pull_request.synchronize can run read-only with a prompt that reads any changed file under dags/ and comments when a new DAG introduces a module-level constant a Param should hold. The trigger adds the event after the prompt; it fills in no placeholder.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The hard part of params is not the Param class but the rules around it: defaults are validated at parse time only when the DAG has a schedule, a templated string becomes a str unless the DAG renders native objects, dag.params does not see a manual override but context['params'] does, and a loop that builds tasks from a constant has to become dynamic mapping before a count can vary. The agent applies each rule file by file, runs the DagBag test in its sandbox on the Airflow version the image pins, and shows one rendered template, so the pull request carries evidence rather than a promise.
Doing this across many DAG repositories is the same reading repeated, and a run per repository keeps every command and its output in that run's record.
Frequently asked questions
Will scheduled runs change?
No, as long as each default equals the value the file carried before. A scheduled run uses the defaults; only a manual trigger with --conf or the trigger form supplies other values. The pull request's tables are there so a reviewer can check every default against the old constant.
Where is the trigger form?
The form that renders a field per Param arrived in Airflow 2.6. The Runtime 6.0.0 image in this project's Dockerfile ships Airflow 2.4, where params and their validation work but the UI takes raw JSON, so the pull request documents airflow dags trigger --conf and names the Runtime bump that adds the form.
Why dynamic task mapping for the loops?
The for i in range(5) loops run when the file is parsed, so the task count is fixed before any run exists and a param cannot change it. expand() builds the mapped tasks per run from a list a small task returns, which is the only way a count can come from params. It changes the task ids, which the pull request calls out.
Can a param be marked read-only?
Setting core.dag_run_conf_overrides_params to False turns every param into a constant for all DAGs. Per value, the agent keeps anything an operator should not change at trigger time, such as the connection id, as a module constant rather than a Param, and says so in the table.
More Airflow jobs
More jobs like this: Hand off a long task
- Add Dark Mode to a Blazor Web App (Blazor)
- Add a Dark Mode Toggle to an Eleventy Site (Eleventy)
- Add Dark Mode to a Jekyll Site (Jekyll)