Repository navigation
Description
Finding
The e2e-coverage report is the artifact everyone reads its region percentages
out of — it is written to $GITHUB_STEP_SUMMARY on every run and uploaded as
e2e-coverage with retention-days: 90. It stamps revision, so a reader can
always tell which code the line numbers address.
It does not stamp the runtime, and for this report the runtime is the other half
of the provenance. The region counts are a V8 artifact: the same commit, suite
and fixture builds split into a different number of ranges on a different
runtime, so the denominator moves while the code does not. The repository
already knows this and says so in two places:
CONTRIBUTING.mdwarns that a local render on a newer runtime "reports
headroom the gate does not have", and measures it —DirectoryFreshness.jsat
11/15 (73.33%) on the pinned major vs 9/10 (90.00%) on v26.10.0.tests/e2e-coverage-gate.test.mjscarriesMEASURED_ON_NODE_MAJOR = '22'and
fails if the job moves off it.
Both of those guard the job. Neither reaches the artifact. So a report read
out of the 90-day upload months later, or pasted into a pull request from a
local render, carries percentages with nothing in them to say which runtime
chose the denominator — and the two are indistinguishable.
Evidence
A full local render on this repository at 3bbac0d (init → build:e2e:coverage
→ test:e2e:coverage, 362 specs passed → seal → report:e2e:coverage, exactly
the recipe in CONTRIBUTING.md) reports:
E2E coverage: ok (run local-1; status passed)
Revision: 3bbac0da081df644da541a2fbde6b9908bae4abf
src files | 100.00 | 97.05 | 657/677 regions
That 97.05% was produced on Node v26.12.0, not the pinned major 22. Nothing in
the report says so. Read beside a CI artifact it looks like a 0.24-point
regression against the 97.29% CONTRIBUTING.md records for the pinned major;
it is runtime drift. This is precisely the mistake the prose exists to prevent,
and the artifact gives a reader no way to avoid it.
To be clear about what this is not: there is no coverage gap behind it. Unit
src coverage is 100.00% lines and 100.00% regions (10892/10892, 2995/2995),
and every one of the 20 residual e2e regions is the documented cross-script
region-union artifact, not a missing test.
Recommendation
Record the runtime in the report envelope beside revision, and render it on
its own line so the step summary and the .txt artifact carry it too:
- add
runtime: { node: process.version, v8: process.versions.v8 }to both the
success envelope (emptyReport) and the failure envelope inmain's
catch— a tooling error that only reproduces on one runtime is the case
where knowing it matters most, and that envelope is the only record such a
run leaves; - render
Runtime: node <v> (v8 <v>)beneathRevision:, with the same
unknownfallbackrevisionuses so the line is always present; v8and not justnode, because V8 is what splits the ranges and it moves
between patch releases of one Node major — which is the difference
CONTRIBUTING.mdwarns is "runtime drift, not a code change".
Additive and backward compatible: no existing key changes meaning, no threshold
moves, and no workflow edit is needed — the e2e-coverage job already tees the
rendered text into its summary and uploads the JSON.
-
runtimepresent on the success envelope -
runtimepresent on the failure envelope -
Runtime:line in the text render, with anunknownfallback - tests covering all three
Priority
- Impact: medium
- Effort: low
🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 3bbac0d
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88
Activity