Skip to content

Cache the fixture adapters' Cargo registry and build tree from main - #799

Merged
rmcdaniel merged 1 commit into
mainfrom
rust-325-adapter-cache
Oct 27, 2026
Merged

rmcdaniel merged 1 commit into
mainfrom
rust-325-adapter-cache

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Part of #325. CI speed: fix CI before adding more PRs.

Every Shared Server fixtures matrix job waits for published-sdk-adapters. That job spent about four of its 6.4 minutes building the same pinned, --locked adapter dependencies on every run (PR run 40414928426).

Change

The job restores the public Cargo registry data (sdk-cargo/registry, sdk-cargo/git) and the build tree (sdk-target), never the rest of CARGO_HOME, under one exact key:

  • No restore keys. A run restores this exact key's tree or nothing.
  • Only a successful push to main saves it. The if: is a top-level && with github.event_name == 'push' and github.ref == 'refs/heads/main', which the organization policy accepts since Allow an exact shared cache key written only by protected pushes .github#269. No pull request or merge group writes a cache that a later run reads.
  • Every run still builds, stamps and hashes its own adapters. The cache only makes those builds incremental. The final adapter directories and their revision checks are unchanged.

scripts/ci/sdk-adapter-cache-key.sh computes the key from:

  • the runner OS and architecture;
  • every pinned Rust image among the inputs: the build scripts' 300ec56… and the query-authority profile's ba81bc3…;
  • a digest of every tracked adapter input's name and content: the five adapter crates, their build scripts, the pip wrapper and the SDK profiles.

It fails unless all five crates and build scripts are among the inputs. The helper and the pip wrapper join the push paths, so a change to either saves a fresh cache on main.

tests/Unit/ServerParityAdapterCacheTest.mjs pins the boundary:

  • only this job caches, and only these paths;
  • there are no restore keys, and only push to main saves;
  • the key comes from the helper, and the helper's inputs and guards are pinned.

Checks run

Effect

This PR's own run cannot hit the cache, because nothing is saved until the first push to main after merge. I'll post the adapter job's time from the first PR run after that.

🤖 Generated with Claude Code

Every Shared Server fixtures matrix job waits for published-sdk-adapters,
which spent about four of its 6.4 minutes building the same pinned,
--locked adapter dependencies each run (PR run 40414928426).

The job now restores the public Cargo registry data (registry, git) and
the build tree, never the rest of CARGO_HOME, under one exact key:
- there are no restore keys, so a run restores this key's tree or none;
- only a successful push to main saves it (the top-level && guard the
  organization policy accepts since durable-workflow/.github#269), so no
  pull request or merge group writes what a later run reads;
- every run still builds, stamps and hashes its own adapters, and the
  final adapter directories and revision checks are unchanged.

scripts/ci/sdk-adapter-cache-key.sh computes the key from the runner OS
and architecture, every pinned Rust image among the inputs (the build
scripts' and the query-authority profile's) and a digest of every tracked
adapter input: the five adapter crates, their build scripts, the pip
wrapper and the SDK profiles. It fails unless all five crates and build
scripts are inputs. The helper and the pip wrapper join the push paths
so a change to either saves a fresh cache on main. A unit test pins the
boundary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security accepted this change at 682312d79d8e9f68133473b16f6cb877fa14a3e5 (advisory, no gate). This head, 3ff3606a781b089d32db17e2b1e560f8c5cc622f, is that commit rebased onto main after #797 merged. The commit's added and removed lines are identical; only main's surrounding php2113 cohort differs.

rmcdaniel added a commit that referenced this pull request Oct 27, 2026
#797 merged; #799, this PR and the native image are next. External payload
takeover stays refused: PHP keeps references in about 30 locations in
their URI form, so it is its own project, and the release-candidate scope
question now includes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security revalidation at exact PR head 3ff3606a781b089d32db17e2b1e560f8c5cc622f:

  • scripts/ci/sdk-adapter-cache-key.sh and tests/Unit/ServerParityAdapterCacheTest.mjs have the same Git blob identities as the previously reviewed 682312d79d8e9f68133473b16f6cb877fa14a3e5 (ae9fe5b8… and 59d152fe…).
  • The server-parity.yml commit patch is the same 31-addition/1-deletion cache change; only surrounding line offsets changed after rebasing onto newer main.
  • The accepted boundary is unchanged: one exact key covers runner OS/architecture, every selected Rust toolchain digest and every tracked adapter input; there are no restore prefixes; only public Cargo registry/git data and the build tree are cached; final adapters and credentials are excluded; saves require success, a push to main, and a cache miss.
  • Current CodeQL, public-boundary, central policy, repository-contract, Sonar and feature-suite checks are successful. The Shared Server fixture matrix is still running and retains its ordinary merge authority.

The prior Security acceptance therefore applies to this exact head. No new source request, release hold, or additional Security gate is added; re-review only if the cache boundary or key/save semantics change.

@rmcdaniel
rmcdaniel added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 1cab1fa Oct 27, 2026
57 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged through the merge queue at 2026-10-27T02:57:10Z, 24 minutes after enqueue (02:33:15Z, pinned to head 3ff3606a781b089d32db17e2b1e560f8c5cc622f).

  • Merge commit: 1cab1fa6238c87181a91b8e1b7758fcb8bd0fb1a
  • Tree: 7669db75fa19eaaae327923c4dd83a2ba1806ef4, identical to the head's tree
  • Parents: 8b84670c3b59a54b989a31cf298abd3a52c766f2 (main) and 3ff3606a781b089d32db17e2b1e560f8c5cc622f (head)

Baseline for the before/after comparison: this PR's own run (40435889275) had no cache to restore. published-sdk-adapters ran 5m28s, and the Shared Server fixtures wall time was 32.5 min, with matrix jobs waiting 5 to 10 min for runners. Main's push run saves the cache next; I'll post the first PR run that restores it.

@rmcdaniel
rmcdaniel deleted the rust-325-adapter-cache branch October 27, 2026 02:58
@rmcdaniel

Copy link
Copy Markdown
Member Author

Measured after merge.

Most of that wall-time drop is runner queueing, not the cache. Matrix jobs waited a median of 0 min (max 2.3) instead of 5 to 10, because fewer runs overlapped. The cache's own share is the roughly 2 minutes on the adapter job, which is on every run's critical path.

rmcdaniel added a commit that referenced this pull request Oct 27, 2026
The checkpoint names the verified 2.11.4 artifacts, the Core 2.14
expectation changes native now follows, the poll_transport position and
the Workflow #931 boundary, with what was checked. The Rust port's
current state lists #799 and #801 as merged, #803 and this move as
next, and 2.11.4 as the compared reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant