Skip to content

ci: run CI, cloud tests, conformance and CodeQL on the CodeBuild runner - #75

Closed
wangyb-A wants to merge 2 commits into
mainfrom
ci/codebuild-runner
Closed

wangyb-A wants to merge 2 commits into
mainfrom
ci/codebuild-runner

Conversation

@wangyb-A

@wangyb-A wangyb-A commented Oct 7, 2026 •

Copy link
Copy Markdown

Summary

Goal: run this repository's CI, cloud tests, conformance tests and CodeQL on the CodeBuild-hosted GitHub Actions runner in the Rust testing account instead of the shared ubuntu-latest pool, so a PR's checks start in seconds rather than waiting 0–74 minutes for a shared runner.

Same change the other SDKs made (aws/aws-durable-execution-sdk-js#937, aws/aws-durable-execution-sdk-python#747, aws/aws-durable-execution-sdk-java#751, aws/aws-durable-execution-sdk-go#135). The runner is the github-actions-runner CodeBuild project (EC2 standard:7.0, Ubuntu 22.04, 60-minute job limit, one fresh container per job); it has no access to test resources, and the jobs keep getting their AWS credentials through OIDC and TEST_ROLE_ARN exactly as before.

What changed

Workflow Jobs Runner
ci.yml check, msrv, dependencies, external-types, semver, feature-powerset, workspaces (2) CodeBuild
cloud-tests.yml example-tests CodeBuild
conformance-tests.yml discover_suites, build_handlers, conformance (12 suites) CodeBuild
codeql.yml analyze CodeBuild

Each runs-on is now

runs-on: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && 'ubuntu-latest' || format('codebuild-github-actions-runner-{0}-{1}', github.run_id, github.run_attempt) }}

Pull requests from forks are routed to ubuntu-latest and behave exactly as today. This is routing, not a trust boundary: a pull_request event runs the PR's own workflow file, so the fork boundary is the repository's "Approval for running fork pull request workflows" setting. No job gating was added or changed.

rustup is not on the CodeBuild image. GitHub-hosted runners preinstall it; aws/codebuild/standard:7.0 has no rustup, cargo or rustc. The workflows call rustup toolchain install … at ten sites, so a new composite action, .github/actions/ensure-rustup, runs before each of them: a no-op when rustup is present, otherwise it downloads rustup-init 1.29.1 for x86_64 from static.rust-lang.org, verifies its published sha256, installs with --default-toolchain none --profile minimal, and adds ~/.cargo/bin to GITHUB_PATH. The existing rustup toolchain install lines still decide which toolchain each job uses, so hosted and CodeBuild runs build with the same toolchains.

The free-disk-space step in the workspaces job now runs only when runner.environment == 'github-hosted': it exists because the hosted image is short on disk, and the CodeBuild image has a 317 GB volume (126 GB free) and a different package set.

Left on GitHub-hosted runners: notify.yml (a thin shell over aws-durable-execution-ci; its runs-on lives in the reusable workflow and is handled with the other SDKs' shells in a follow-up).

Verification

  • All four workflows run on pull_request, so this PR's own checks are the end-to-end test: every job should show a codebuild-github-actions-runner-<run_id>-<attempt> label, and the ensure-rustup step should report an install on CodeBuild.
  • Job durations on recent main runs fit the 60-minute limit: ci.yml jobs 0–3 min, example-tests 15 min, build conformance handlers 13 min, conformance suites 1–3 min, Analyze Rust 9 min.
  • Toolchain facts come from a probe on the same CodeBuild project (aws-durable-execution-sdk-js run 37551910779 and its follow-up on 168a1d92): no rustup/cargo/rustc, Python 3.9–3.14 via pyenv (the actions/setup-python step still provisions its own), Docker 28.5, 317 GB volume.

no linked issue: CI infrastructure change.

Also in this PR: clippy on Rust 1.99

main has been red since Rust 1.99.0 (2026-09-28): rustup toolchain install stable now resolves to 1.99, which deprecates AtomicUsize::fetch_update and adds the assert_is_empty lint, and -D warnings turns both into errors in check (sdk) and feature powerset (same failure on ubuntu-latest in run 37515503745). The second commit fixes that so this PR's CI can go green:

  • src/test_util.rs: the two fetch_update calls stay (try_update is only stable from 1.95; MSRV is 1.94.1) under #[allow(deprecated)] with a note to switch once the MSRV moves.
  • Six assert!(x.is_empty()) / assert!(!x.is_empty()) test assertions become assert_eq! / assert_ne! against an empty Vec, as the lint asks.

Checked locally: cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings and cargo test --lib (538 passed) on 1.99.0; cargo check --all-targets --all-features on 1.94.1.

Alex Wang added 2 commits October 7, 2026 01:04
Route the pull_request and push jobs of ci.yml, cloud-tests.yml, conformance-tests.yml and codeql.yml to the CodeBuild-hosted GitHub Actions runner in the Rust testing account; pull requests from forks keep using ubuntu-latest.

The CodeBuild image has no rustup, so a new composite action .github/actions/ensure-rustup installs a pinned rustup-init (1.29.1, sha256-verified) with no default toolchain when rustup is absent, and is a no-op on hosted runners. The free-disk-space step in the workspaces job is limited to GitHub-hosted runners.
Rust 1.99.0 (2026-09-28) deprecates AtomicUsize::fetch_update in favour of try_update and adds the assert_is_empty lint; with -D warnings both fail `cargo clippy` on stable, so CI has been red since the 1.99 release.

try_update is only stable from 1.95 and the MSRV is 1.94.1, so the two fetch_update calls in test_util.rs keep fetch_update under #[allow(deprecated)] with a note to switch once the MSRV moves. The six assert!(x.is_empty()) / assert!(!x.is_empty()) test assertions become assert_eq!/assert_ne! against an empty Vec, as the lint asks, so the offending value is printed on failure.

Verified: cargo fmt --check, cargo clippy --all-targets --all-features -D warnings and cargo test --lib (538 passed) on 1.99.0; cargo check --all-targets --all-features on 1.94.1.
@yaythomas yaythomas added the changes-requested Reviewer requested changes; waiting on the author label Oct 23, 2026
Comment thread .github/workflows/ci.yml
# must match the CodeBuild project name. Pull requests from forks are routed
# to GitHub-hosted runners. This is routing, not a trust boundary: the fork
# boundary is the repository's approval setting for fork workflows.
runs-on: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && 'ubuntu-latest' || format('codebuild-github-actions-runner-{0}-{1}', github.run_id, github.run_attempt) }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include Dependabot in the hosted-runner condition. Dependabot branches belong to this repository, so the current expression runs updated dependency build scripts and procedural macros inside the CodeBuild account before maintainer review. Python PR 747 routes Dependabot builds to ubuntu-latest. Please add github.actor == 'dependabot[bot]' to this condition and the repeated conditions. The cloud and conformance workflows should instead skip fork and Dependabot PRs because those jobs require unavailable secrets.

# must match the CodeBuild project name. Pull requests from forks are routed
# to GitHub-hosted runners. This is routing, not a trust boundary: the fork
# boundary is the repository's approval setting for fork workflows.
runs-on: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && 'ubuntu-latest' || format('codebuild-github-actions-runner-{0}-{1}', github.run_id, github.run_attempt) }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This moves the job to a CodeBuild project with a stated 60-minute limit, but the next line retains timeout-minutes: 90. CodeBuild can terminate the runner before GitHub cancels the job. The if: always() teardown then cannot run, which can leave the three test stacks deployed. Please set the GitHub timeout below 60 minutes with enough cleanup time, move cleanup into a separate if: always() job, or raise the CodeBuild timeout above 90 minutes.

yaythomas added a commit that referenced this pull request Nov 6, 2026
On 2026-11-05, jobs waited up to 94 minutes for a GitHub-hosted runner,
while the JS and Python SDKs' jobs on the CodeBuild-hosted runner
started in under two minutes. This builds on #75, which moved the four
workflows to that runner, rebases it onto main and addresses its review
comments:

- ci.yml and codeql.yml run fork and Dependabot pull requests on
  ubuntu-latest, so a dependency update's build scripts and procedural
  macros do not run in the CodeBuild account before review. Everything
  else runs on the CodeBuild runner.
- cloud-tests.yml and conformance-tests.yml skip fork and Dependabot
  pull requests, which cannot read the test secrets.
- Every job times out before the CodeBuild build limit, the jobs that
  deploy stacks soonest, so that their own teardown step can finish.
  A teardown job in each AWS workflow deletes the stacks by name on a
  fresh runner, also when CodeBuild stops a test runner. The
  cloud-tests concurrency group moves to the workflow, so a run deletes
  its stacks before the next run deploys stacks of the same names.
- check (sdk) asks for the large instance: on the default one, the
  kernel killed the rustc that compiles the unit tests.
- The action ensure-rustup installs a pinned, checksummed rustup-init
  where rustup is missing. free-disk-space runs only on GitHub-hosted
  runners.

Co-authored-by: Alex Wang <wangyb@amazon.com>
yaythomas added a commit that referenced this pull request Nov 6, 2026
On 2026-11-05, jobs waited up to 94 minutes for a GitHub-hosted runner,
while the JS and Python SDKs' jobs on the CodeBuild-hosted runner
started in under two minutes. This builds on #75, which moved the four
workflows to that runner, rebases it onto main and addresses its review
comments:

- ci.yml and codeql.yml run fork and Dependabot pull requests on
  ubuntu-latest, so a dependency update's build scripts and procedural
  macros do not run in the CodeBuild account before review. Everything
  else runs on the CodeBuild runner.
- cloud-tests.yml and conformance-tests.yml skip fork and Dependabot
  pull requests, which cannot read the test secrets.
- Every job times out before the CodeBuild build limit, the jobs that
  deploy stacks soonest, so that their own teardown step can finish.
  A teardown job in each AWS workflow deletes the stacks by name on a
  fresh runner, also when CodeBuild stops a test runner. The
  cloud-tests concurrency group moves to the workflow, so a run deletes
  its stacks before the next run deploys stacks of the same names.
- check (sdk) asks for the large instance: on the default one, the
  kernel killed the rustc that compiles the unit tests.
- The action ensure-rustup installs a pinned, checksummed rustup-init
  where rustup is missing. free-disk-space runs only on GitHub-hosted
  runners.

Co-authored-by: Alex Wang <wangyb@amazon.com>
@yaythomas

Copy link
Copy Markdown
Contributor

Thank you, @wangyb-A! Your work landed in #158: ci: run workflows on the CodeBuild runner, with you as co-author. It carries your workflow changes and the ensure-rustup action onto the current main, and adds the follow-ups from this review: Dependabot runs go to ubuntu-latest, fork and Dependabot pull requests skip the AWS workflows, stack teardown runs in its own job, and every timeout stays under CodeBuild's 60-minute limit. Closing this one in favor of #158.

@yaythomas yaythomas closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changes-requested Reviewer requested changes; waiting on the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants