Skip to content

codec-native-build's 4 GiB cap is within 6% of the release compile's peak #781

Closed

Description

@rmcdaniel

Problem

codec-native-build (in scripts/conformance/server-parity/codec-operations-build.sh) compiles the release server in a container capped at 4 GiB with no swap: --cpus=1 --memory=4g --memory-swap=4g, CARGO_BUILD_JOBS=1. The script's comment says the compile "peaks near 2 GiB". It now peaks much closer to the cap.

On PR #776's exact head, run 40321678938 attempt 1 lost this job: the final durable-workflow-server rustc was killed by signal 9 with no compiler diagnostic. The single rerun passed. Security accepted the classification as a resource kill and asked for the CI memory bound itself to be treated as the defect if it recurs near the cap.

Measurement

Local release builds with CI's settings (--release --locked, jobs 1, incremental off, release debug 0) in a container capped at exactly 4 GiB, no swap, on the same compiler image. The peak is the container cgroup's memory.peak, which includes page cache.

Tree Result Peak memory Headroom to 4 GiB
main before #776 (6419fdb8) built 3,994,550,272 B about 300 MB
#776 head (7ce33fe5, now main 47f4c544) built 4,060,192,768 B about 235 MB

The same #776 build with an 8 GiB cap peaked at 4,746,670,080 B, because page cache grows when nothing reclaims it.

So both trees fit, but within about 6%. One feature PR added about 66 MB, and the compile will keep growing. A hosted runner under memory pressure can tip it over, as attempt 1 shows.

Proposed fix

Raise the container cap to 6 GiB (--memory=6g --memory-swap=6g) and record the new bound in build.json. GitHub's standard hosted Linux runners have 16 GB, and this job runs only this one build container, so the runner has room.

Update the script's comment to the measured peak, and keep CARGO_BUILD_JOBS=1 so the bound still means one rustc at a time.

The job's purpose, a bounded optimized artifact for deadline health, is unchanged; only the bound's headroom changes.

This is a CI-only change, with no product source.

🤖 Generated with Claude Code

Activity

  1. added a commit that references this issue on Oct 26, 2026
  2. rmcdaniel commented on Oct 26, 2026

    @rmcdaniel
    MemberAuthor

    Fixed by #782, merged as 0ef7c93 (tree f486727e = cleared head 7e3b1dd). codec-native-build now runs under a 6 GiB container bound, and it passed in run 40329690090. Receipt: #782. Closed by hand after the merge receipt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions