Skip to content

Number task loop iterations from one instead of zero - #74773

Open
bbovenzi wants to merge 1 commit into
apache:mainfrom
astronomer:dev/loop-task-iteration-start-one-370470
Open

bbovenzi wants to merge 1 commit into
apache:mainfrom
astronomer:dev/loop-task-iteration-start-one-370470

Conversation

@bbovenzi

@bbovenzi bbovenzi commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Task Loop Iterations were starting at 0 which was confusing in the UI. Instead of remembering a UI-only "+1" change that we could easily forget to add. This PR changes it so that all region_indexes that are for loop iterations start at 1.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Claude Opus 5

Generated-by: Claude Opus 5 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:task-sdk area:UI Related to UI/UX. For Frontend Developers. kind:documentation labels Oct 14, 2026
@bbovenzi bbovenzi added the backport-to-v3-4-test Backport to v3-4-test label Oct 15, 2026
@bbovenzi bbovenzi added this to the Airflow 3.4.0 milestone Oct 15, 2026
@bbovenzi
bbovenzi force-pushed the dev/loop-task-iteration-start-one-370470 branch 2 times, most recently from c56e618 to 59eae68 Compare October 16, 2026 14:45
Comment thread airflow-core/docs/authoring-and-scheduling/loops.rst Outdated
Comment thread airflow-core/src/airflow/models/dynamic_region.py Outdated

@ashb ashb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, small nits though

@bbovenzi
bbovenzi force-pushed the dev/loop-task-iteration-start-one-370470 branch from 888cf42 to c79cd1f Compare October 19, 2026 21:55
("stop", State.SUCCESS, 1, True, False, [0]),
("continue", State.SUCCESS, 1, True, True, [0]),
(None, State.FAILED, 1, True, False, [0]),
("continue", State.SUCCESS, 3, False, False, [1, 2]),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this row now [1, 2], the if mapped and passes == [0, 1]: block further down (L2584) no longer matches any row, so the check that the next pass's mapped task starts as a -1 placeholder and then expands silently stopped running. It needs passes == [1, 2] and parent_region_index == 2 there.

) == [0, 1]
) == [1, 2]

def test_loop_gate_completion_uses_its_pinned_limit(self, client, session, running_loop_gate, dag_maker):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ti.region_index == 1 filter in this test now picks up the first pass (gate and terminal) instead of the successor, so len(successors) == 2 and the version check pass whatever the server does with the pinned limit. Should be == 2.

The loop iteration and mapped position are separate "coordinates". In this
example, each iteration has mapped instances 0 and 1, so mapped instance 1 in
iteration 0 is distinct from mapped instance 1 in iteration 1.
The loop iteration and mapped position are separate "coordinates", and they do

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Rerun part of an iteration" further down (L389 and L396) still counts from zero: "iterations 0 through 4" and "Iterations 0 and 1 remain untouched". With this change that becomes "1 through 4" and "Iteration 1 remains untouched".

assert client.head(url, params=params).headers["Content-Range"] == "map_indexes 1"


def test_previous_iteration_of_the_first_pass_resolves_to_nothing(client, loop_xcoms, authenticate_as):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also passes with the old < 0 guard: pass 1 then looks up pass 0, which never exists, so it 404s either way. To pin "stops before looking for one", assert the producer query never runs (capture_orm_selects, as elsewhere in this file), or trim the docstring to the 404.

run_id: Mapped[str] = mapped_column(StringID(), nullable=False)
# The slot this instance fills in its region, so its meaning follows the region's node: a loop
# region numbers its iterations from one, a mapped expansion numbers its slots from zero, and -1
# is the placeholder for a mapped task that has not been expanded yet. Code that reads the column

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: plain tasks outside any region also store -1 (they get (-1,) in dagrun.py), and that's the most common case. Can we mention it here too?

@bbovenzi
bbovenzi force-pushed the dev/loop-task-iteration-start-one-370470 branch from c79cd1f to 3258cc3 Compare October 20, 2026 03:50
A loop iteration is a number people read: it shows up in the loop filter,
on the Gantt tooltip, in the task instance list and in the `loop.index` a
Dag author branches on. Starting it at zero meant each of those readers had
to translate between the number Airflow stored and the number a person
would say, and that translation was easy to apply in one place and forget
in another.

Numbering from one in storage removes the translation instead of hiding it
behind a display-time adjustment, so the stored index, the API field and
the UI agree without anyone adding to it. Mapped expansion indexes share
the column and stay zero-based: a map index is a position, not a count, so
code reading the column has to know whether the region executes a loop or
an expansion.

Task loops have not shipped in a release, so no stored run and no client
depends on the old numbering, and the execution API version that
introduces the loop context is still unreleased.

Co-Authored-By: Claude <noreply@anthropic.com>
@bbovenzi
bbovenzi force-pushed the dev/loop-task-iteration-start-one-370470 branch from 3258cc3 to e9da72d Compare October 20, 2026 14:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk area:UI Related to UI/UX. For Frontend Developers. backport-to-v3-4-test Backport to v3-4-test kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants