Skip to content

Add on_success_succeed_dagrun option to teardown tasks - #74916

Open
kadubhumika wants to merge 5 commits into
apache:mainfrom
kadubhumika:feature/on-success-succeed-dagrun
Open

kadubhumika wants to merge 5 commits into
apache:mainfrom
kadubhumika:feature/on-success-succeed-dagrun

Conversation

@kadubhumika

@kadubhumika kadubhumika commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Add on_success_succeed_dagrun to teardown tasks, so that a successful teardown
no longer hides a failed work task.

closes: #74727

Problem

With @teardown(on_failure_fail_dagrun=True) the teardown is the only task that
decides the Dag run state. If a work task fails but the teardown succeeds, the
Dag run is marked SUCCESS and the failure is hidden.

Fix

New teardown option on_success_succeed_dagrun (default True, so existing Dags
behave exactly as before). With on_failure_fail_dagrun=True and
on_success_succeed_dagrun=False:

Work Teardown Dag run
failed success FAILED
success failed FAILED
success success SUCCESS

What changed

  • airflow-core/.../models/dagrun.py: _tis_for_dagrun_state also evaluates the
    tasks before a teardown that sets on_success_succeed_dagrun=False.
  • task-sdk (@teardown, as_teardown(), _internal/abstractoperator.py,
    _internal/mixins.py, xcom_arg.py, bases/decorator.py, bases/operator.py,
    mappedoperator.py): new argument, property and validation, with support for
    mapped tasks and .override().
  • airflow-core/.../serialization/ (schema.json, serialized_objects.py,
    definitions/baseoperator.py, definitions/mappedoperator.py,
    dag_version_diff.py): the new field is serialized and deserialized.
  • Tests: test_dagrun.py covers all six flag and state combinations;
    test_dag_serialization.py registers the new serialized field.
  • Newsfragment added.

Testing

test_dagrun.py, test_dag_serialization.py and the task-sdk
`test_setup_t

Was generative AI tooling used to co-author this PR?
  • Yes (Chat Gpt)

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! I'd like to ask for the following:

  1. Add AI disclosure to the PR's body.
  2. Undo unrelated Dataproc commits
  3. Add documentation for using the new feature (howto/setup-and-teardown.rst)
  4. Add tests to task-sdk
  5. Fix the static checks
  6. Fix the title according to the guidelines

@kadubhumika
kadubhumika force-pushed the feature/on-success-succeed-dagrun branch from 6282a07 to 13d7f7f Compare October 17, 2026 11:30
@kadubhumika

kadubhumika commented Oct 17, 2026 via email

Copy link
Copy Markdown
Contributor Author

@shahar1
shahar1 dismissed their stale review October 17, 2026 20:27

Blocking concerns were addressed

@yuseok89 yuseok89 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one question inline about test coverage for mapped teardown tasks across a Dag serialization round trip.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a teardown declared with on_failure_fail_dagrun=True, has applying .as_teardown(on_success_succeed_dagrun=False) after .expand() also been checked?

If this path isn't covered yet, would it make sense to add a test checking that

  • the configured value is preserved after the Dag is serialized and deserialized
  • the restored value is reflected in the Dag run state evaluation

@shahar1 shahar1 changed the title Feature/on success succeed dagrun Add on_success_succeed_dagrun option to teardown tasks Oct 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add on_success_succeed_dagrun to teardown tasks

3 participants