Repository navigation
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are recorded for the Azure Pipelines vcpkg caching changes.
No open issues 🎉
What changed in this PR
Extends ONNX Runtime’s existing GitHub Actions vcpkg binary caching approach to Azure Pipelines, reducing repeated dependency compilation across native and Docker-based builds.
Changes:
- Adds a shared cache template with dependency-aware, per-commit keys and fallback restores disabled for release branches.
- Integrates caching across Windows, macOS, Linux, and Android jobs, forwarding configured cache paths into Docker.
- Enables vcpkg for Windows Node.js packaging and adds regression tests for cache keys, restore ordering, and Docker forwarding.
| File | Description |
|---|---|
tools/ci_build/test_vcpkg_pipeline_cache.py |
Tests cache configuration, job keys, build ordering, and Docker forwarding. |
tools/ci_build/github/linux/run_python_dockerbuild.sh |
Conditionally mounts the binary cache for Python builds. |
tools/ci_build/github/linux/build_webgpu_plugin_package.sh |
Conditionally forwards caching to WebGPU plugin builds. |
tools/ci_build/github/linux/build_tensorrt_c_api_package.sh |
Conditionally forwards caching to TensorRT packaging. |
tools/ci_build/github/linux/build_nodejs_package.sh |
Conditionally forwards caching to Node.js packaging. |
tools/ci_build/github/linux/build_cuda_plugin_package.sh |
Conditionally forwards caching to CUDA plugin builds. |
tools/ci_build/github/linux/build_cuda_c_api_package.sh |
Conditionally forwards caching to CUDA C API packaging. |
tools/ci_build/github/azure-pipelines/templates/win-ci.yml |
Adds cache setup to Windows CI builds. |
tools/ci_build/github/azure-pipelines/templates/setup-vcpkg-cache.yml |
Defines shared cache setup, keys, and release-branch restore policy. |
tools/ci_build/github/azure-pipelines/templates/py-win-cpu.yml |
Adds caching to Windows CPU Python builds. |
tools/ci_build/github/azure-pipelines/templates/py-macos.yml |
Adds caching to macOS Python builds. |
tools/ci_build/github/azure-pipelines/templates/py-linux.yml |
Adds cache setup before Linux Python builds. |
tools/ci_build/github/azure-pipelines/templates/mac-cpu-packaging-steps.yml |
Adds caching to macOS CPU packaging. |
tools/ci_build/github/azure-pipelines/templates/c-api-linux-cpu.yml |
Restores and mounts the cache for Linux CPU packaging. |
tools/ci_build/github/azure-pipelines/templates/android-java-api-aar.yml |
Restores and mounts the cache for Android AAR builds. |
tools/ci_build/github/azure-pipelines/stages/py-win-gpu-unified-stage.yml |
Adds caching before unified Windows GPU wheel builds. |
tools/ci_build/github/azure-pipelines/stages/py-win-gpu-stage.yml |
Adds caching to Windows GPU Python builds. |
tools/ci_build/github/azure-pipelines/stages/py-linux-gpu-unified-stage.yml |
Adds caching to unified Linux GPU wheel builds. |
tools/ci_build/github/azure-pipelines/stages/py-linux-gpu-stage.yml |
Adds caching to Linux GPU Python builds. |
tools/ci_build/github/azure-pipelines/stages/plugin-win-webgpu-stage.yml |
Adds caching to Windows WebGPU plugin builds. |
tools/ci_build/github/azure-pipelines/stages/plugin-win-cuda-stage.yml |
Adds caching to Windows CUDA plugin builds. |
tools/ci_build/github/azure-pipelines/stages/plugin-mac-webgpu-stage.yml |
Adds caching to macOS WebGPU plugin builds. |
tools/ci_build/github/azure-pipelines/stages/plugin-linux-webgpu-stage.yml |
Adds cache setup before Linux WebGPU plugin builds. |
tools/ci_build/github/azure-pipelines/stages/plugin-linux-cuda-stage.yml |
Adds cache setup before Linux CUDA plugin builds. |
tools/ci_build/github/azure-pipelines/stages/nuget-linux-cuda-packaging-stage.yml |
Uses distinct CUDA and TensorRT cache keys to avoid collisions. |
tools/ci_build/github/azure-pipelines/stages/nodejs-win-packaging-stage.yml |
Enables vcpkg and asset caching with target-specific binary cache keys. |
tools/ci_build/github/azure-pipelines/stages/nodejs-linux-packaging-stage.yml |
Adds caching to Linux Node.js packaging. |
tools/ci_build/github/azure-pipelines/stages/nightly-webgpu-shader-validation-stage.yml |
Adds caching before shader-validation builds. |
tools/ci_build/github/azure-pipelines/linux-gpu-tensorrt-cuda-minimal-ci-pipeline.yml |
Restores and mounts the cache for minimal TensorRT/CUDA CI. |
Review effort: Balanced
Edward Chen (edgchen1)
previously approved these changes
Nov 2, 2026
Edward Chen (edgchen1)
left a comment
Edward Chen (edgchen1)
left a comment
Contributor
There was a problem hiding this comment.
good with the changes, just had a comment about the new test file
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Edward Chen (edgchen1)
previously approved these changes
Nov 2, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a shared
Cache@2template for vcpkg binary archives and integrate it into the Azure Pipelines Windows, macOS, Linux, and Android jobs that use vcpkg. Forward the configured cache into Linux Docker builds without changing wrapper behavior when caching is not configured.Cache keys separate pipeline/job/platform variants, hash the vcpkg dependency inputs, and include
Build.SourceVersionso new commits can save additional ABI archives. Non-release branches can restore earlier snapshots via a prefix;refs/heads/rel-*branches have no fallbackrestoreKeys.Use explicit keys for the Linux CUDA and TensorRT jobs: observed 1ES-expanded runs resolve
System.JobNameto__defaultfor both jobs in the same stage, which otherwise causes cache-save collisions.Warm versus cold build-step reductions were 6m 9s (x64) and 5m 58s (ARM64). Relative to the previous non-vcpkg sample, x64 was 1m 35s faster and ARM64 was essentially unchanged (3 seconds slower). Warm Azure restore tasks additionally took 17.3/34.7 seconds for x64/ARM64, plus about 1 second each for the post-job exact-hit check; these are outside the build-step table. Thus the results demonstrate effective binary reuse, but do not establish a several-minute net migration benefit over the previous non-vcpkg builds. These are single samples on different agents, not controlled performance trials. The cold run failed only the separate Web E2E task; the warm run had no failed tasks and elapsed 2h 37m 15s.
Local validation:
python -m pytest tools\ci_build\test_vcpkg_pipeline_cache.py -q— passed (7 tests, 45 subtests).python -m ruff check tools\ci_build\test_vcpkg_pipeline_cache.py— passed.python -m ruff format --check tools\ci_build\test_vcpkg_pipeline_cache.py— passed.git --no-pager diff --check— passed before commit.Two sequential Pipeline 2138 (
Nuget - Packaging - CUDA13) runs on commit32d1476238completed: run 1 and run 2. Run 2 had Azure cache hits AND actual vcpkg binary restores in 11/12 cache-enabled jobs: 30 packages per Windows build, 27 per Linux C API build, 25 on macOS, and 29 per Android ABI.These are full build-step durations, not isolated dependency time. Reported vcpkg installation time dropped from 6.4–6.8 minutes to 2.2–2.3 seconds on Windows CPU x64/CUDA/TensorRT; package archive restoration adds roughly 3–4 seconds. Warm Azure cache tasks took about 6–33 seconds. End-to-end pipeline elapsed was 2h 56m 8s vs 2h 49m 16s (6m 52s shorter). Different agents/scheduling and the single paired sample limit attribution. Both pipelines failed in the separate Web E2E package-consuming test; no cache task reported failure.
Enable vcpkg and its internal asset cache in the shared Windows Node.js packaging template, covering both x64 and ARM64 while retaining DirectML/WebGPU, LTO, cross-compilation, and packaging settings. Restore binary caches before building, with explicit stage/target/configuration keys independent of 1ES job-name rewriting.
Windows Node.js verification on
1003f5587bis complete: cold run 1511846 built and packaged both architectures successfully and explicitly saved both caches. Warm run 1512042 succeeded end-to-end, with exact same-commit cache hits and 31 vcpkg packages restored per architecture in 3.6 seconds. Reported subsequent installation was 2.1 seconds for x64 and 2.2 seconds for ARM64, versus 6.4 and 6.3 minutes cold. Both retained the intended x64/ARM64 target triplets, DirectML/WebGPU, LTO, and the x64-to-ARM64 tablegen artifact handoff. Neither Node.js post-job log containedFailed to cache item.Verification of corrected commit
d2f776368eis complete:Cache item created.Cache service reliability caveat: run 1510925 also logged
Failed to cache itemfor the Linux CPU ARM64 and Windows CPU ARM64 post-job saves despite green tasks. Those jobs successfully restored older snapshots, and run 1511108 successfully saved both missing same-commit entries plus Android's entry (the prior run skipped Android save after a separate GPG signing failure). No run 1511108 cache log containedFailed to cache item. Evidence points to intermittent cache artifact creation failures, not a demonstrated repository-side defect; directory cache task success alone is not proof of either save success or vcpkg package reuse.Motivation and Context
Extend the vcpkg binary caching used in GitHub Actions (#32682, #32692, and #32933) to Azure Pipelines, reducing repeated dependency compilation while retaining vcpkg ABI validation and Azure's branch-isolated cache scopes.