Repository navigation
Add Dag-level param presets to pre-fill the trigger form - #73909
VinolFernandes wants to merge 5 commits into
Conversation
7168e82 to
6787f5a
Compare
bbovenzi
left a comment
There was a problem hiding this comment.
UX makes sense to me. Eventually we should square this with selecting another dag run's config too like in Airflow 2. I will leave it to others to review the serialization code.
6787f5a to
8d890f0
Compare
6847738 to
bb04412
Compare
bcbe4f4 to
930e059
Compare
yuseok89
left a comment
There was a problem hiding this comment.
Thanks for adding presets! I noticed a validation error that persists after applying a valid preset and left the details inline.
930e059 to
e927213
Compare
|
Should presets live on ParamsDict instead of on DAG? They mean nothing without params. Also, as Brent mentioned, there’s already a dropdown for recent configurations so we should do something better than simply adding another dropdown. |
|
thanks @bbovenzi and @uranusjr. Do you think giving the user an option to create a preset on the fly would be helpful in this scenario? Would include a lot of edge cases where a multiple users could edit the preset, Or maybe who should have access to edit the preset etc. Having it in the dag code would make it immutable and versioned through git. Maybe we can think around this. LMK your thoughts |
Dag authors can currently declare
paramsand a single default value for each. When one Dagserves several well-known configurations, everyone triggering it retypes the same sets of values
by hand, which is slow and easy to get wrong.
param_presetslets the Dag author name those configurations so they can be loaded in one click:A
Presetdrop-down appears above the run parameters in the trigger form. It is rendered inConfigForm, so both the trigger form and the backfill form get it. A Dag that declares no presetrenders no drop-down and is otherwise untouched.
Design decisions
A preset loads on top of the Dag defaults, not on top of the current form state. Selecting the
same preset therefore always produces the same starting configuration, whatever was edited before.
Every field stays editable afterwards, which is the point: start from a known-good configuration
and change the one or two values that differ for this run. Carrying stale edits across a preset
switch would make runs non-reproducible between two people who picked the same preset.
Presets are validated while the Dag is parsed. A key that names no declared param, or a value
its schema rejects, fails Dag parsing rather than surfacing at trigger time. Validation resolves
each value against a copy of the
Param, becauseParam.resolve()stores the value it validatedand must not overwrite the author's default.
Presets seed the form only. Param defaults are unchanged, and scheduled runs or API triggers
without
confbehave exactly as before. A preset lists only the params it changes.Stored verbatim in the serialized Dag. The generic field serializer applies nested type
encoding to dicts, so presets are assigned directly in
serialize_dag, the same wayedge_infois. Dags with no presets get no key in the blob at all.
The drop-down has no clear button. Clearing would blank the label without restoring anything,
and resetting the form from an
xwould silently discard edits the user cannot recover.Out of scope
The issue also mentions editing preset values from the UI. That needs a table, CRUD endpoints, a
permission model for who may publish an org-wide preset, and a story for presets that drift out of
sync with a Dag whose params were renamed. Dag-authored presets cannot drift, because parsing
fails first. That is worth a separate discussion and is deliberately not attempted here.
Generated artifacts
Regenerated rather than hand-edited: the REST API spec, the UI TypeScript client, the airflow-ctl
datamodels, and the TypeScript SDK's vendored Dag schema. The TypeScript SDK requires every new Dag
schema property to be classified, so
param_presetsis listed as unexpressible alongsideparams.Screenshots
Testing
task-sdk/tests/task_sdk/definitions/test_dag.pyairflow-core/tests/unit/serialization/test_dag_serialization.pyairflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dags.pypnpm vitest run src/components/TriggerDag src/components/DagActionsprek run --from-ref main --stage pre-commitand--stage manualprek run mypy-task-sdkandprek run mypy-airflow-coreThe new tests cover the parse-time validation errors, the serialization round trip including the
empty case, the field on the Dag details response, and the browser interaction that applies a
preset to the form fields and the configuration JSON.
closes: #71285
Screenshots
Was generative AI tooling used to co-author this PR?