Skip to content

Add Dag-level param presets to pre-fill the trigger form - #73909

Open
VinolFernandes wants to merge 5 commits into
apache:mainfrom
VinolFernandes:enable-presets-to-pre-load-params
Open

VinolFernandes wants to merge 5 commits into
apache:mainfrom
VinolFernandes:enable-presets-to-pre-load-params

Conversation

@VinolFernandes

@VinolFernandes VinolFernandes commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Dag authors can currently declare params and a single default value for each. When one Dag
serves several well-known configurations, everyone triggering it retypes the same sets of values
by hand, which is slow and easy to get wrong.

param_presets lets the Dag author name those configurations so they can be loaded in one click:

with DAG(
    dag_id="train_model",
    schedule=None,
    params={
        "dataset": Param("cifar10", type="string", enum=["cifar10", "imagenet", "mnist"]),
        "batch_size": Param(128, type="integer", minimum=1),
        "augmentations": Param({"flip": True, "rotate": 15}, type="object"),
    },
    param_presets={
        "ImageNet (heavy augmentation)": {
            "dataset": "imagenet",
            "batch_size": 512,
            "augmentations": {"flip": True, "rotate": 45},
        },
        "Smoke test": {"batch_size": 1},
    },
) as dag: ...

A Preset drop-down appears above the run parameters in the trigger form. It is rendered in
ConfigForm, so both the trigger form and the backfill form get it. A Dag that declares no preset
renders no drop-down and is otherwise untouched.

Design decisions

A preset loads on top of the Dag defaults, not on top of the current form state. Selecting the
same preset therefore always produces the same starting configuration, whatever was edited before.
Every field stays editable afterwards, which is the point: start from a known-good configuration
and change the one or two values that differ for this run. Carrying stale edits across a preset
switch would make runs non-reproducible between two people who picked the same preset.

Presets are validated while the Dag is parsed. A key that names no declared param, or a value
its schema rejects, fails Dag parsing rather than surfacing at trigger time. Validation resolves
each value against a copy of the Param, because Param.resolve() stores the value it validated
and must not overwrite the author's default.

Presets seed the form only. Param defaults are unchanged, and scheduled runs or API triggers
without conf behave exactly as before. A preset lists only the params it changes.

Stored verbatim in the serialized Dag. The generic field serializer applies nested type
encoding to dicts, so presets are assigned directly in serialize_dag, the same way edge_info
is. Dags with no presets get no key in the blob at all.

The drop-down has no clear button. Clearing would blank the label without restoring anything,
and resetting the form from an x would silently discard edits the user cannot recover.

Out of scope

The issue also mentions editing preset values from the UI. That needs a table, CRUD endpoints, a
permission model for who may publish an org-wide preset, and a story for presets that drift out of
sync with a Dag whose params were renamed. Dag-authored presets cannot drift, because parsing
fails first. That is worth a separate discussion and is deliberately not attempted here.

Generated artifacts

Regenerated rather than hand-edited: the REST API spec, the UI TypeScript client, the airflow-ctl
datamodels, and the TypeScript SDK's vendored Dag schema. The TypeScript SDK requires every new Dag
schema property to be classified, so param_presets is listed as unexpressible alongside params.

Screenshots

Testing

Suite Result
task-sdk/tests/task_sdk/definitions/test_dag.py 100 passed, 7 new
airflow-core/tests/unit/serialization/test_dag_serialization.py 227 passed, 2 new
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.py 155 passed
pnpm vitest run src/components/TriggerDag src/components/DagActions 3 files passed, 2 new tests
prek run --from-ref main --stage pre-commit and --stage manual passed
prek run mypy-task-sdk and prek run mypy-airflow-core passed

The new tests cover the parse-time validation errors, the serialization round trip including the
empty case, the field on the Dag details response, and the browser interaction that applies a
preset to the form fields and the configuration JSON.

closes: #71285


Screenshots

image image image
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

@eladkal eladkal added this to the Airflow 3.4.0 milestone Oct 2, 2026
@eladkal eladkal added the type:new-feature Changelog: New Features label Oct 2, 2026
@VinolFernandes
VinolFernandes force-pushed the enable-presets-to-pre-load-params branch from 7168e82 to 6787f5a Compare October 5, 2026 04:04

@bbovenzi bbovenzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UX makes sense to me. Eventually we should square this with selecting another dag run's config too like in Airflow 2. I will leave it to others to review the serialization code.

@VinolFernandes
VinolFernandes force-pushed the enable-presets-to-pre-load-params branch from 6787f5a to 8d890f0 Compare October 8, 2026 03:30
@VinolFernandes
VinolFernandes force-pushed the enable-presets-to-pre-load-params branch from 6847738 to bb04412 Compare October 8, 2026 05:10
@VinolFernandes
VinolFernandes force-pushed the enable-presets-to-pre-load-params branch 2 times, most recently from bcbe4f4 to 930e059 Compare October 15, 2026 08:53

@yuseok89 yuseok89 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding presets! I noticed a validation error that persists after applying a valid preset and left the details inline.

Comment thread airflow-core/src/airflow/ui/src/components/ParamPresetSelect.tsx
@VinolFernandes
VinolFernandes force-pushed the enable-presets-to-pre-load-params branch from 930e059 to e927213 Compare October 19, 2026 07:51
@uranusjr

Copy link
Copy Markdown
Member

Should presets live on ParamsDict instead of on DAG? They mean nothing without params.

Also, as Brent mentioned, there’s already a dropdown for recent configurations so we should do something better than simply adding another dropdown.

@VinolFernandes

Copy link
Copy Markdown
Contributor Author

thanks @bbovenzi and @uranusjr. Do you think giving the user an option to create a preset on the fly would be helpful in this scenario? Would include a lot of edge cases where a multiple users could edit the preset, Or maybe who should have access to edit the preset etc.

Having it in the dag code would make it immutable and versioned through git.

Maybe we can think around this. LMK your thoughts

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable Presets to pre-load parameters for the same DAG

6 participants