Skip to content

ci: reduce queueing and duplicated builds without dropping coverage - #1675

Merged
1lck merged 11 commits into
previewfrom
codex/actions-performance
Nov 3, 2026
Merged

1lck merged 11 commits into
previewfrom
codex/actions-performance

Conversation

@1lck

@1lck 1lck commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Problem and changes

Recent Actions data (50 workflow runs / 142 jobs) shows macOS runner queueing and compilation dominate, while Windows serializes shared Core and host tests and repeats ordinary Core tests in the real-JDT lane. This change preserves test coverage and native checks while reducing duplicated work.

  • Cancel superseded PR runs with workflow concurrency. Reuse only completed successful single-job results; the planner no longer has write permission or preserves active older runs. Every main push receives full CI and a unique concurrency group, protecting pending commits as well as running commits. Selected jobs explicitly reject cancellation and failed classification; reports also stop on cancellation.
  • Retain the required macOS CI gate for docs-only PRs, skip unrelated tooling/native jobs, and filter documentation from the other native workflows. Move CI tooling/build-script tests and Rust static checks to parallel Ubuntu execution; remove duplicated cache/relay tests from Windows classification. Move platform-independent structured logging checks to Ubuntu while preserving native i18n packaging coverage.
  • Split Windows Rust into two exhaustive Core shards, an independent shared host/MCP job and a Windows host job, with unchanged per-test/suite deadlines and fail-fast disabled. Verify both shard reports against their discovered test inventories in the gate. Real-JDT runs only the native integration prefix instead of repeating all ordinary Core tests.
  • Use Swatinem/rust-cache for pruned Rust dependency outputs, share Swift/plugin Rust cache identities, allow only one Core shard writer, and restore the Core cache read-only for real-JDT. macOS PR download caches are restore-only to avoid duplicate entries against the 10 GB quota. Frontend jobs do not load Rust caches; Node-only scripts have no npm dependencies to cache.
  • Run plugin-focused Swift tests and official package verification in the main Swift job, reusing its complete compilation. Preserve the standalone plugin workflow for manual diagnosis, route plugin-only changes to the shared lane, and remove overlapping language-service suite execution. Plugin checks still execute after an ordinary Swift test failure once compilation succeeds, preserving their previous independence.
  • Reuse the dylib already produced alongside Swift's Core archive for the Git execution bridge test, avoiding a second target-directory compilation. Give the two DMG test reports distinct names so one cannot overwrite the other.
  • Skip Markdown-only i18n/logging/matrix workflows, and cancel superseded plugin release-plan PR checks while preserving publishing runs.
  • Document scheduling/caching decisions and explicitly reject CI compilation caches in local cross-worktree resource copying.
  • Stabilize the existing native arrow boundary fixture without changing product code or assertions: exercise both 1× and 2× geometry and establish the synthetic click's NSApp.currentEvent during synchronous window dispatch. A controlled 1× stale mouse-move context reproduced the exact four hosted failures before the fix; the test restores the original AppKit getter after every scope.

Validation

Hosted measurements

Final head c4fbba23189ae5c5f4712461319d44fba964d46b passes every selected check. macOS run and Windows run are successful, including both gates. The PR is ready for review and has not been merged.

The baseline covers the latest 50 workflow runs / 142 jobs. Queue time is separate from runner execution; preparation includes checkout, downloads, compilation, upload and step overhead. Known build durations are subtracted from test steps. Historical mixed steps without independent timing remain explicitly unresolved in the CSV. Samples differ in source revision, cache warmth and runner load; these are observations, not a controlled benchmark.

Lane Successful baseline median Final successful runner duration Change
Windows Rust critical path 24m 07s 11m 14s (slowest of four parallel jobs) −53.4%
Windows real JDT 16m 02s 4m 49s (validated reused result) −70.0%
Windows frontend 6m 32s 6m 30s −0.5%
macOS native Rust 12m 19s 10m 16s −16.6%
Swift + plugins cumulative runner occupancy 43m 11s across two parallel jobs 29m 51s in one job −30.9%
macOS arm64 package 25m 23s 26m 29s +4.3%

The Swift comparison measures aggregate resource use, not workflow latency: the old jobs could run in parallel. The final combined job is slower than the old Swift-only median, and package compilation did not improve. Final macOS workflow wall time is 30m 38s, Windows is 11m 48s. Windows Rust splitting also duplicates some compilation; critical-path savings are not equivalent to runner-minute savings.

Final Windows reports account for all 1,586 Core cases exactly once (the synchronized preview added cases since the earlier 1,570-case runs). Shared host/MCP and Windows project/terminal/host checks remain intact. The final gate verifies the unchanged JDT lane's completed successful result; skipping its new job is reuse, not zero-cost execution or missing coverage.

Hosted macOS passes 3,729 ordinary Swift tests, 549 plugin tests, C ABI/Git/Swift bridge checks, Git performance/observation checks, rollback disk images and official plugin package validation. The package job also passes WebKit, runtime-bundle immutability and complete package verification. Ubuntu tooling, remote workspace, all three i18n platforms, logging, feature matrix and Agent Notes pass on the final head.

Limits and verification details

  • 156 focused CI regression tests pass with HTML/JUnit per-test durations (slowest 357 ms); 188 CI/resource cases pass after synchronizing preview. The final ordinary Swift suite passes all 3,729 tests in 127.661 s, with 549 plugin tests, bridge verification and official plugin package validation also passing locally. The added-lines stability gate passes; an additional whole-repository scan still identifies 34 existing sleep/timer findings, which this PR neither suppresses nor rewrites.
  • Two hosted Swift runs exposed the arrow fixture's stale-event/1× issue. The deterministic local reproducer failed with the same four assertions; the corrected four geometry/input combinations pass in 113 ms. The complete Git graph suite plus both Markdown appearance cases pass (33 tests, 3.567 s). One hosted dark Markdown template load exceeded its unchanged 5-second deadline; focused local runs have not reproduced that failure. A separate local non-UTC run exposed an existing Git date-test timezone assumption; UTC is used to match CI. No assertions or budgets were weakened.
  • Repeated pushes initially exposed cache key collisions (shared-key overrides key), shard artifacts being deleted by checkout, duplicated Swift compilation, and cancellation taking five minutes. These were corrected before the final run. A cancellation-condition validation run released its queued native jobs immediately and its running tooling/gate within 46 seconds; that sample does not establish a bound for cancelling an active native compiler.
  • Removed 14 closed-PR caches, 3 obsolete experiment keys, 3 caches after fix(macos): 恢复隐藏工具入口并修复中文动作搜索 #1677 merged, and 4 superseded lockfile keys belonging to this PR: 7,728,171,896 bytes deleted cumulatively. Other workflows created caches during validation; the final snapshot is 26 entries / 10,125,264,500 bytes (9.43 GiB). Other active branches were not deleted; cumulative deleted data is not net free-space growth.
  • Fixed a second reproducible test-only failure in local history capacity checking: Swift Testing mis-resolved the optional Int64 / compound-literal equality despite both sides printing 524288000. A diagnostic confirmed typed equality passes while the original macro fails. The assertion now requires a non-nil response and compares explicit Int64 values, retaining the original 500 MiB requirement (Swift #91221).
  • The sole product-source change is a byte-identical module-directory move required by the existing layout guard; public APIs and logic are unchanged. No relaxed assertions/timeouts, release publication or merge.

@ghfind-review ghfind-review Bot added the review: high ghfind author score; see https://ghfind.com label Nov 3, 2026
@1lck
1lck marked this pull request as ready for review November 3, 2026 08:09
@lithe-code-review

lithe-code-review Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Lithe Review

结论: ⛔ 有阻塞
范围: preview ← codex/actions-performance · head: 6095a22

Findings

  1. [P1] 在下载分片报告之前完成首次 checkout
    .github/workflows/ci-windows.yml:892

    • 实际影响:Core 分片全部成功后,Windows gate 仍会因报告文件被删除而失败,影响所有选择 Core 测试的运行。
    • 触发条件:在全新托管 runner 上,rust_core == 'true' 且 Rust 矩阵成功。
    • 复现步骤:修改 Core 文件并让两个分片通过。gate 先将报告下载到工作区的 .artifacts/rust-shards,随后才首次执行 checkout。此时目标目录没有 .git,checkout 初始化仓库前会删除已有内容;clean: false 只关闭已有仓库的 Git 清理,不能保护这种情况。第 902 行因此收到不存在的报告路径并失败。
    • 来源:本次引入;base 中没有这组下载及覆盖检查步骤。

Scope Check

  • 关联需求:未找到可验证的关联 Issue;按 PR 描述检查减少排队、重复构建及保留覆盖的目标。
  • 已完成范围:审查清单完整,已检查全部 28 个变更文件及直接相关调用链。
  • 未完成范围:托管 CI 验收尚未完成,当前不能确认优化目标全部达成。
  • 能力复用:使用现有 Cargo、计时框架和缓存 action,未引入产品领域的第二套实现。
  • Finding 属于本次新增 gate 流程,处于本次变更职责范围。

Suggestions

  1. 将分片校验器的 checkout 移到报告下载之前,或把报告下载到 checkout 目录之外;补充覆盖首次 checkout 与报告保留顺序的回归验证。

Summary

分片、复用和缓存调整总体符合现有边界,但新增报告处理顺序会导致 Windows gate 系统性失败,需先修复。

Verification

  • 静态检查范围:1ffabf1...6095a22,包含工作流、分类与复用规划、Rust 分片/报告、桥接验证、资源注册表及相关文档。
  • 已提供结果:Windows 三个 Rust 分片、前端及 real-JDT 成功;Windows gate、CI tooling 失败;部分 macOS/插件检查仍在运行或已取消。
  • 作者声明的本地验证未由本次审查复验。未运行仓库代码或测试,未访问网络,未修改文件。
  • 尚缺失败 job 日志及完整托管运行结果;不能确认 CI tooling 的失败原因,也不能将所有失败归因于正文所述的历史格式问题。

@lithe-code-review

Copy link
Copy Markdown

Lithe 运行验收

版本:f71c6b9448d05b80ca9f5ff38fd61598773f3d3e;状态:partial。
本报告独立于代码审查;只报告实际执行的场景,不代表所有问题均已修复。

对照:共同祖先 fec1497ef5c8b957e16c9d9339716a5c1338f182 → PR head;当前目标分支合并适用性:未运行。

  • 条件不足,未验证:The Rust timing runner must partition discovered test identities across two Core shards, preserve a full unsharded run, filter real-JDT execution to its native integration prefix, and have the shard verifier reject incomplete aggregate coverage.
    • Probe audit: The rust-runner-shards-and-prefix probe changes its invocation semantics on the baseline: when parseArguments rejects the new flags, it reparses legacy arguments and manually assigns shardIndex, shardCount, and testPrefix. The baseline runner ignores those fields and runs unfiltered suites, while head parses and applies the filters. This violates the identical-semantics requirement. The resource-isolation probe uses the same request on both revisions.
  • 条件不足,未验证:Local cross-worktree resource reuse must explicitly classify CI Rust compilation outputs as isolated and refuse them before copy or worktree resolution, while retaining the existing verified Cargo download-cache path.
    • Probe audit: The rust-runner-shards-and-prefix probe changes its invocation semantics on the baseline: when parseArguments rejects the new flags, it reparses legacy arguments and manually assigns shardIndex, shardCount, and testPrefix. The baseline runner ignores those fields and runs unfiltered suites, while head parses and applies the filters. This violates the identical-semantics requirement. The resource-isolation probe uses the same request on both revisions.
  • 条件不足,未验证:Validate that the CI scheduling and cache changes reduce queueing and compilation cost without dropping required coverage, using matched baseline/head hosted runs and a prespecified statistical protocol.
    • This requires matched hosted GitHub Actions baseline/head runs and their queue, build, cache, and coverage evidence. The PR body does not specify a statistical protocol (sample count, measurement window, metrics, controls, or acceptance thresholds), and this planning environment has no hosted Actions runner or GitHub credentials. A single local run cannot establish an intermittent queue-time or performance improvement.

未覆盖:

  • GitHub concurrency behavior: superseded PR cancellation, unique non-cancelling main run groups, read-only planner permissions, and reuse limited to completed successful jobs across real workflow history.
  • Workflow trigger and gate behavior: docs-only macOS gate retention, documentation path filters, unrelated lane skipping, Ubuntu tooling/static-check split, Linux structured logging with native i18n packaging, and full main-push coverage.
  • Hosted Windows matrix behavior: independent host job, two exhaustive Core shards, fail-fast disabled, preserved per-test/suite deadlines, real-JDT prefix on an actual JDK, and end-to-end gate selection.
  • Artifact service behavior: checkout/download ordering and report retention through GitHub artifact transfer. The inspected head places the shard-verifier checkout before report download, but local probes cannot establish hosted runner behavior.
  • Actual Cargo-discovered Core and host tests are not exercised by the synthetic runner adapter; Java is unavailable, so no real-JDT integration can be verified here.
  • Rust cache action identity and effects: shared Swift/plugin cache identity, single Core-shard writer, read-only restore for real-JDT, restore-only macOS download caches, and ensuring frontend jobs do not load Rust caches.
  • macOS-only checks: Swift/native i18n packaging, dylib reuse for the Git bridge, and distinct DMG test reports.
  • Queue-time, build-time, cache-hit, and overall cost improvements remain unverified pending a prespecified statistical protocol and hosted baseline/head data.

工作副本与运行状态已清理;可复用编译缓存按 12 GiB / 7 天预算保留。
详细步骤和原始证据保存在服务器受限目录(14 天、总计不超过 1 GiB);当前没有公开下载链接。

@lithe-code-review

Copy link
Copy Markdown

Lithe 运行验收

版本:9e2734e03a8b921ab7d21d64183b29bae6e8d83e;状态:blocked。
本报告独立于代码审查;只报告实际执行的场景,不代表所有问题均已修复。

执行未完成:Review timed out

本次未形成完整验收结果,详见执行原因。
详细步骤和原始证据保存在服务器受限目录(14 天、总计不超过 1 GiB);当前没有公开下载链接。

@lithe-code-review

Copy link
Copy Markdown

Lithe 运行验收

版本:c4fbba23189ae5c5f4712461319d44fba964d46b;状态:partial。
本报告独立于代码审查;只报告实际执行的场景,不代表所有问题均已修复。

对照:共同祖先 7cf6a9f8fec88b14e64a071e0672a3ac500f3188 → PR head;当前目标分支合并适用性:未运行。

  • 条件不足,未验证:Windows Rust CI must divide discovered Core test identities into two exhaustive shards, retain a full unsharded runner path, filter real-JDT execution to its native integration prefix, and reject missing aggregate shard coverage without changing the configured per-test or suite deadlines.
    • Probe audit: Static audit found blockers. The Rust probe requires inventory/selection fields absent on baseline before emitting its mandatory full-run guard, so successful baseline execution cannot satisfy that guard. Generated tests contain tautological assertions or empty bodies, with no independent execution oracle. Isolation is also unproven: Cargo inherits host cache/target configuration, and fixture Git setup leaves inherited hooks/configuration active. Neither probe was run.
  • 条件不足,未验证:CI Rust compilation outputs must be explicitly classified as non-reusable and rejected before worktree resolution or copying, while verified Cargo download archives remain eligible for cross-worktree reuse.
    • Probe audit: Static audit found blockers. The Rust probe requires inventory/selection fields absent on baseline before emitting its mandatory full-run guard, so successful baseline execution cannot satisfy that guard. Generated tests contain tautological assertions or empty bodies, with no independent execution oracle. Isolation is also unproven: Cargo inherits host cache/target configuration, and fixture Git setup leaves inherited hooks/configuration active. Neither probe was run.
  • 条件不足,未验证:Verify that PR supersession cancellation, unique non-cancelling main-run groups, completed-success-only lane reuse, artifact transfer, and Rust cache policies reduce queueing and compilation cost without dropping required coverage, using matched hosted evidence and a prespecified statistical protocol.
    • The supplied environment has no hosted GitHub Actions runner or Actions cache/artifact telemetry interface, and this planning pass prohibits external service contact. The PR's timing summary is observational, with no prespecified matched-run protocol, sample window, controls, or acceptance thresholds, so local assertions cannot establish queue-time or compilation-cost improvement.

未覆盖:

  • Actual GitHub workflow concurrency behavior: superseded PR cancellation, unique main commit groups including pending runs, planner permissions, completed-success-only reuse against real run history, cancellation/failure classification, and docs-only gate retention and trigger filters.
  • Hosted workflow selection and coverage: Ubuntu tooling/static checks, Linux structured logging, native i18n packaging, plugin routing, Windows host/MCP jobs, exhaustive real Core and host test inventories, fail-fast behavior, actual per-test and suite deadlines, and the real-JDT workflow on a JDK.
  • GitHub artifact service and gate lifecycle behavior, including checkout/report ordering, report retention across artifact upload/download, and aggregate coverage verification on actual shard reports.
  • Actual Swatinem/rust-cache identity and restore/save behavior: shared Swift/plugin cache identity, single Core shard writer, read-only real-JDT restore, macOS restore-only download caches, and frontend jobs avoiding Rust caches.
  • macOS-only changes and checks: AppKit arrow interactions at 1x and 2x with currentEvent restoration, Swift local-history assertion, Swift/plugin test execution, dylib reuse, DMG report naming, and native i18n packaging. No macOS or Swift toolchain is available here.
  • The shard scenario uses generated tests and verifies the production runner and verifier, not the actual 1,570-case Windows Core inventory, Windows host adapters, MCP protocol, Java/JDT discovery, or product end-to-end behavior. Java is unavailable in the supplied tool inventory.
  • The resource scenario validates a synthetic Cargo archive payload against a deterministic Cargo.lock checksum and verifies actual copy bytes; it does not exercise GitHub cache action storage or Cargo unpacking/compilation from that archive.
  • Queue time, build time, cache-hit rates, runner cost, and total CI cost improvements need matched hosted baseline/head runs plus a prespecified sample count, measurement window, controls, metrics, and thresholds; the supplied PR body does not define that protocol.
  • Linux tooling checks, Agent Notes, platform feature matrix, logging, i18n, Rust static checks, and other claimed PR validations were not executed during this planning pass.

工作副本与运行状态已清理;可复用编译缓存按 12 GiB / 7 天预算保留。
详细步骤和原始证据保存在服务器受限目录(14 天、总计不超过 1 GiB);当前没有公开下载链接。

@1lck
1lck merged commit 1713611 into preview Nov 3, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review: high ghfind author score; see https://ghfind.com

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant