Skip to content

test: isolate allocation regressions and bound smoke execution - #60

Merged
AGiorgetti merged 3 commits into
developfrom
feat/codex-allocation-test-isolation
Oct 15, 2026
Merged

AGiorgetti merged 3 commits into
developfrom
feat/codex-allocation-test-isolation

Conversation

@AGiorgetti

@AGiorgetti AGiorgetti commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Change

Exact allocation regressions can report phantom bytes under .NET 10 background GC accounting (dotnet/runtime#134724, proposed fix dotnet/runtime#134855). Run only allocation-sensitive methods in a bounded test-only child with background GC disabled, effective-configuration verification and failure propagation. Ordinary correctness keeps default GC. Exact allocation assertion bodies are unchanged; there are no retries, tolerances or discarded samples.

The user also approved a scoped follow-up after the Windows benchmark smoke exceeded its old 60-second process limit:

  • Only the BenchmarkDotNet smoke receives 300 seconds; sample calls retain their 60-second default. Its command, success assertions, benchmark algorithms, workflow settings and controlled performance gates are unchanged.
  • Packaging process timeouts preserve the last 4,096 characters of each redirected stream, configured/elapsed time, termination action and cleanup status. Successful output remains complete.
  • Process exit and both pipes share the execution deadline. A separate one-second cleanup wait follows a termination request, then local reads are canceled. If the direct child already exited while descendants hold pipes open, cleanup is bounded and explicitly unconfirmed.

Current head and scope

HEAD a6faf309edfabd1180cdb28fa8c9aef12e483013 includes the reviewed allocation isolation rebased onto develop c3ef53ec3762d825c2945cbc6394af310bda1933 containing merged symbols PR #59, the approved timeout follow-up, and a Linux fixture ownership correction.

Linux fixture ownership uses PID plus the exact kernel start token from procfs field 22. Cross-process Process.StartTime timestamps can differ because .NET estimates boot time separately per process (dotnet/runtime#108959). Other platforms retain exact UTC ticks. PID reuse protection and the orphan-alive assertion remain intact; no tolerance was added.

Both solution projects and both guide additions are retained. Symbols validator/configuration, product code, allocation isolation and workflow files are unchanged by the timeout follow-up. Independent read-only static review has no remaining material findings.

Current local validation

  • Three new process regressions fail against the old helper; all four existing process controls pass.
  • Final process regression set: 14/14 across three complete runs (42 cases), zero failures/skips. Covers full successful 1 MiB stderr, bounded timeout tails, direct/tree termination, exited-parent/inherited pipes, exact ownership and procfs names containing spaces/parentheses. Seven new parser cases fail before implementation. Earlier timeout coverage passed 21 final cases before the Linux correction.
  • Broader process/release/publication safeguards after the identity correction: 255/255; existing allocation isolation: 8/8 before this fixture-only correction, zero failures/skips. Packaging-test project Release build has zero warnings/errors.
  • Release solution build passes with zero errors and two existing transitive System.Security.Cryptography.Pkcs 9.0.6 audit warnings (NU1902/NU1903).

Prior CI and limits

Run 38739919081 tested prior head 07a7dd2: Linux full suite passed 1,390 cases and all six Roslyn jobs passed 1,091 each. Windows passed generator/runtime/integration and 292 packaging cases; only the existing benchmark smoke exceeded its 60-second liveness limit. This is prior-head evidence, not current CI.

Run 38746844875 tested b9ae890: Windows full suite passed 1,393 cases and all six Roslyn jobs passed 1,091 each. Linux passed 1,392 and failed only the new orphan fixture's timestamp-based ownership assertion. The current head corrects that fixture identity without changing the timeout helper or its assertions.

No local timing benchmarks or nested real-build/package-launching cases were run for the timeout follow-up. Current exact-head CI supplies full Windows/Linux, Roslyn, packaging, trimming, required Native AOT and unchanged smoke coverage. The 300-second limit is a provisional liveness budget, not a measured runtime percentile or throughput gate. Historical failures lack detailed progress traces.

Current exact-head CI

Run 38749497578 completed successfully on exact head a6faf309edfabd1180cdb28fa8c9aef12e483013 (attempt 1):

  • Windows and Linux full suites each passed 1,400 tests: generator 1,091, runtime 5, integration 1, packaging/process 303. Zero failures/skips, including the Linux fixture correction and required Native AOT.
  • All six Roslyn jobs passed 1,091 tests each on 4.8.0, 4.14.0 and 5.9.0 across both operating systems, zero failures/skips.
  • Linux sample and existing benchmark smoke steps, plus both solution-path validations, passed. These are execution/correctness results, not a performance acceptance claim.
  • Pack & Validate, Verify Release Artifacts and Publish NuGet Packages were conditionally skipped for the PR trigger. No release-artifact publication evidence is claimed.

PR remains open and ready for human review; no merge, closure, tag, publication or machine/system settings changes. Independent static review has no remaining material findings. Working tree is clean.

Fixes #58

@AGiorgetti
AGiorgetti force-pushed the feat/codex-allocation-test-isolation branch from 2d01eee to 07a7dd2 Compare October 15, 2026 13:06
@AGiorgetti AGiorgetti changed the title test: isolate exact allocation regressions from background GC test: isolate allocation regressions and bound smoke execution Oct 15, 2026
@AGiorgetti
AGiorgetti merged commit 1489dad into develop Oct 15, 2026
11 checks passed
@AGiorgetti
AGiorgetti deleted the feat/codex-allocation-test-isolation branch October 15, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Isolate allocation regression tests from background GC accounting

1 participant