[zozo123]

Source-only changes currently invalidate CI image dependency installation. Extract workspace pyproject.toml files and uv.lock into a separate build stage and preinstall locked third-party dependencies before copying the full checkout. The existing full sync still installs workspace packages and keeps its resolution and validation path.

Split from #74173 at reviewer request so this Dockerfile optimization can be reviewed and reverted independently from the PR-only snapshot transport.

How the preinstall works:

• It runs install_airflow_when_building_images.sh with INSTALL_LOCKED_THIRD_PARTY_ONLY=true. That mode adds --no-install-workspace to the same frozen uv sync that the full installation runs, so the two layers cannot drift apart.
• A frozen dry run decides whether to preinstall. When uv.lock does not match the manifests (a workspace member missing from the lock, or no lock at all), the warm-up is skipped and the full sync after COPY re-resolves as before.
• Any other failure, such as a network error or an lxml/xmlsec build error, fails the build, because it would fail the full sync too. A partial dependency layer therefore cannot be published through the registry's mode=max cache.
• It uses the existing architecture-specific uv cache. It is skipped for upgrade builds, both in Dockerfile.ci and in the script itself.

Validation:

• CI image build on eb931fa102: the frozen dry run reported 801 packages to install, and the preinstall installed them in a 34.2 s step. After COPY ., the full sync installed only the 141 workspace packages in 15.8 s.
• I ran the Dockerfile.ci preinstall step in the CI image against a manifest-only tree (every pyproject.toml plus uv.lock):
  • a lock in sync installs;
  • a workspace member missing from the lock and a missing lock both skip, with exit 0;
  • an install that fails after a passing dry run exits 1;
  • upgrade builds run no uv sync.
• A local BuildKit experiment with the manifest-extraction stage confirmed the cache boundary. Changing an Airflow Python source file reruns extraction but keeps the downstream manifest COPY and RUN cached, while changing the root pyproject.toml or uv.lock invalidates both. This checks the cache boundary, not full-image equivalence or registry performance.
• In the last 30 days, 66 of 994 main commits touched a manifest or uv.lock, so most commits would reuse the layer.

Actual reuse and net savings can only be measured on a source-only build after main publishes this layer to the registry.

---

Was generative AI tooling used to co-author this PR?

• Yes — Codex (GPT-6) and Claude Code (Opus 5.5)

Generated-by: Codex (GPT-6) and Claude Code (Opus 5.5), following the Airflow guidelines.

[potiuk]

Nice idea — caching the third-party layer on the manifests alone should save a lot of CI image rebuild time. We actually had a similar mechanism implemented before we switched to uv, so this is bringing back something that worked well for us. One thing needs fixing before it can go in, plus a few smaller suggestions.

Needs fixing: the pre-sync has no fallback

The new uv sync --frozen ... --no-install-workspace step hard-fails whenever the lock does not match the workspace, while the full sync after it (install_from_sources) falls back to re-resolving without --frozen (AIRFLOW_FALLBACK_NO_CONSTRAINTS_INSTALLATION=true by default). I reproduced this with the exact command from the PR against a manifest-only tree built from this branch (--dry-run):

• lock in sync: OK
• new workspace member added without regenerating uv.lock: error: The lockfile at uv.lock needs to be updated, but --frozen was provided: Missing workspace member ...
• dependency added to an existing member without regenerating uv.lock: OK (stale set installed, the full sync then fixes it up)

So a PR that adds a new provider/distribution and doesn't refresh uv.lock used to get an image via the fallback, and now the image build aborts at the pre-sync step. Since the pre-sync is purely a cache warm-up and the full sync covers everything anyway, it should never fail the build — e.g. let it fail soft:

uv sync ... || echo "Pre-sync of locked dependencies failed - the full sync below will install them."

The comment above the step ("retains its existing resolution fallback") should then describe that the pre-sync is best-effort, since as written it suggests the stale-lock path still degrades gracefully.

Suggestions

1. The manifest stage treats uv.lock as optional (if [ -f uv.lock ]), but the pre-sync always requires it (Unable to find lockfile at uv.lock, but --frozen was provided). With the fail-soft change above that becomes harmless; otherwise either drop the guard or skip the pre-sync when there is no lock.
2. The pre-sync duplicates the uv sync flags from install_from_sources in scripts/docker/install_airflow_when_building_images.sh (--group ci-image, the lxml / xmlsec --no-binary-package workaround, …). If they drift, the two layers install different builds of the same packages and the full sync has to reinstall them. Consider adding a "locked third-party only" mode to install_from_sources (e.g. an env var that adds --no-install-workspace) and calling the script instead.
3. Optional: the cache key is the full content of every pyproject.toml. On main over the last 30 days, 66 of 994 commits touched a manifest or uv.lock, and 20 of those changed only pyproject.toml files — so the cache hit rate is already good, but keying on uv.lock plus the dependency sections only would avoid a few more rebuilds.
4. Optional: UPGRADE_RANDOM_INDICATOR_STRING now feeds the pre-sync RUN, so every upgrade-to-newer-dependencies build re-executes it only to skip the install.

---

Drafted-by: Claude Code (Opus 5.5); reviewed by @potiuk before posting

[zozo123]

Thanks for the reviews. Here is what changed:

• Must-fix: the pre-sync had no fallback (4bb265f, refined in 9f2b0c8). A stale or missing uv.lock no longer fails the build: a frozen dry run of the same uv sync decides whether to preinstall. When the lock does not match the manifests, the warm-up is skipped and the full sync after COPY re-resolves as before.
• Suggestion 1: no longer applies. A missing uv.lock is one of the cases the dry run skips.
• Suggestion 2 (ad2ceda): done in this PR, instead of the follow-up I mentioned earlier.
  • install_airflow_when_building_images.sh has an INSTALL_LOCKED_THIRD_PARTY_ONLY=true mode that adds --no-install-workspace to the existing frozen uv sync.
  • The pre-sync RUN calls the script in that mode, so both layers run the same uv sync line.
• @shahar1's nits (9f2b0c8):
  • Only a lock that doesn't fit is skipped now. Any other failure, such as a network error or an lxml/xmlsec build error, fails the build instead of leaving an empty layer in the mode=max cache. The full sync would fail the same way.
  • The script returns early for upgrade builds itself, instead of relying on the Dockerfile.ci guard.
• Suggestions 3 and 4: I'm leaving these as they are.
  • In the last 30 days, 66 of 994 main commits touched a manifest or uv.lock, so keying on the full manifests already hits well.
  • In upgrade builds, the re-executed RUN is a no-op.

How I tested the pre-sync step. I ran the RUN body from Dockerfile.ci at 9f2b0c8 in the CI image, with the RUN shell options. The source tree held every pyproject.toml plus uv.lock from this branch, and a uv wrapper turned the real install into a dry run.

case	frozen dry run	pre-sync RUN
lock in sync	OK	installs, exit 0
new workspace member missing from uv.lock	Missing workspace member	skipped, exit 0
no uv.lock	Unable to find lockfile	skipped, exit 0
install fails after the dry run passes	OK	fails, exit 1
upgrade build	–	no uv sync, exit 0

In the real CI image build on eb931fa (job), the frozen dry run reported "Would install 801 packages" in 0.6 s, and the pre-sync then installed all 801 locked third-party packages, in a 34.2 s step. After COPY ., the full sync installed only the 141 workspace packages, in 15.8 s. When only workspace sources change, the pre-sync layer stays cached.

The install script is inlined into Dockerfile too, so that file changes as well. The new mode is only enabled by the pre-sync step in Dockerfile.ci, so the PROD build behaves as before.

The 13 checks that failed earlier came from the Oct 14 base: mypy-airflow-core and test_api_client.py. #74731 fixed them on main, and main is merged in.

CI on eb931fa: green, with 132 checks passed and none failing. While this was open, main briefly broke its own asset-permission tests; #75126 and #75138 fixed them, and merging main in picked up those fixes.

[shahar1]

Correct and verified on the current head's CI image build (801 locked packages pre-installed, then only 141 workspace packages, pip check clean); potiuk's must-fix and suggestion 2 are addressed, and what remains is a stale PR description plus two optional nits.

Smaller observations

• general — The PR description is stale after 4bb265f. It still says that installation errors fail the build, but the pre-install now ends in || echo "Skipping the locked dependency pre-install...", so every pre-install failure is swallowed and the (empty or partial) layer is committed and can be exported to the registry cache. The Validation section also quotes the pre-rebase run (54.8 s, 791 packages); the current head's CI image build shows 801 packages in 29.6 s and then 141 workspace packages in 13.4 s. Please refresh the description from your 2026-10-18 comment so that it matches the code that will be merged.
• See inline comments on Dockerfile.ci:1985, scripts/docker/install_airflow_when_building_images.sh:49.

@potiuk WDYT?

---

This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The findings
below are observations, not blockers; an Apache Airflow
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Airflow handles maintainer review:
Contributing guide.