Skip to content

Stop running OpenLineage e2e compat tests when full tests are derived - #75262

Merged
potiuk merged 1 commit into
apache:mainfrom
FrankYang0529:fix-openlineage-compat-derived-full-tests
Oct 28, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
FrankYang0529:fix-openlineage-compat-derived-full-tests

Conversation

@FrankYang0529

@FrankYang0529 FrankYang0529 commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Why

  • Consolidate OpenLineage e2e tests into airflow-e2e-tests #69671 kept the OpenLineage e2e compat matrix off derived full tests on purpose, like run_ui_e2e_tests. Run OpenLineage e2e compat tests when the compat setup changes #69866 kept a comment saying so, but made run_openlineage_e2e_compat_tests fall back to _should_be_run(...), which returns True whenever full_tests_needed is True. So selective checks turn the matrix on (five jobs on main today) in every PR run where they derive full tests.
  • On 2026-10-15, 29 of the 30 ci-amd.yml PR runs that ran the compat jobs did so only because full tests were derived. In those 29 runs, the compat matrix ran 145 jobs for 1,008 runner-minutes, about 1% of the runner-minutes used by ci-amd.yml PR runs that day.
The 29 runs from 2026-10-15

The table lists every ci-amd.yml pull_request run created on 2026-10-15 (UTC) that ran the OpenLineage e2e compat jobs. Minutes are completed_at - started_at, summed over the "OpenLineage e2e compat" jobs and counting every attempt. All 235 PR runs that day used 100,144 runner-minutes.

Run PR Reason Compat jobs Minutes
38667409803 74773 API spec 5 35.6
38671723924 74528 standard provider 5 34.9
38688821872 74778 environment files 5 35.2
38689484797 73964 environment files 5 34.4
38690075328 74560 standard provider 5 34.5
38693473514 74696 root pyproject.toml 5 34.5
38695331396 73909 API spec 5 35.0
38695868768 74780 standard provider 5 34.6
38701744585 69859 git provider 5 34.2
38701864371 74694 environment files 5 36.1
38701951577 74715 environment files 5 33.2
38701966240 74488 test helpers 5 34.0
38708907221 73909 API spec 5 32.2
38719396314 73964 environment files 5 35.2
38722622917 73822 root pyproject.toml 5 34.0
38734857648 73662 git provider 5 34.6
38734949416 74778 environment files 5 35.8
38737237625 74768 environment files 5 37.1
38739808340 69809 test helpers 5 34.0
38751817368 74773 API spec 5 34.5
38753535968 74803 environment files 5 33.3
38758184899 74046 environment files 5 33.4
38771473424 74803 environment files 5 34.5
38774598147 74773 API spec 5 36.9
38784345099 74046 environment files 5 32.9
38794135276 73038 environment files 5 37.5
38798813157 74806 API spec 5 36.1
38808639276 74046 environment files 5 35.0
38811990881 69859 git provider 10 (2 attempts) 70.3

How


Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

@FrankYang0529
FrankYang0529 force-pushed the fix-openlineage-compat-derived-full-tests branch from ce777c4 to df73418 Compare October 27, 2026 02:06
@potiuk
potiuk merged commit e31f52e into apache:main Oct 28, 2026
144 checks passed
@FrankYang0529
FrankYang0529 deleted the fix-openlineage-compat-derived-full-tests branch October 28, 2026 02:36
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-4-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Oct 28, 2026
… are derived (apache#75262)

(cherry picked from commit e31f52e)

Co-authored-by: PoAn Yang <payang@apache.org>
Signed-off-by: PoAn Yang <payang@apache.org>
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Oct 28, 2026
… are derived (apache#75262)

(cherry picked from commit e31f52e)

Co-authored-by: PoAn Yang <payang@apache.org>
Signed-off-by: PoAn Yang <payang@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants