Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
98fe573
ci: Run the Actor dev-loop e2e on the self-hosted macOS arm64 runner
Pijukatel Sep 17, 2026
f891f92
ci: Bring a Docker engine to the clean macOS runner and remove it after
Pijukatel Sep 17, 2026
9c42b0b
fix(ci): Stop the macOS engine script recursing into its own `install`
Pijukatel Sep 17, 2026
1f4491c
fix(ci): Verify the buildx darwin binary against `checksums-signed.txt`
Pijukatel Sep 17, 2026
f48ef54
fix(ci): Keep the macOS engine's state dir short enough for Lima's so…
Pijukatel Sep 17, 2026
479440b
ci: Run the whole e2e suite on the macOS runner, against Docker and P…
Pijukatel Sep 17, 2026
ccbf98b
test(e2e): Mount the engine socket with label=disable on SELinux Podm…
Pijukatel Sep 17, 2026
63705e6
fix(ci): Keep the macOS engine smoke test bash-3.2-safe
Pijukatel Sep 17, 2026
fb013ae
test(e2e): Count dangling volumes through the suite's engine CLI
Pijukatel Sep 17, 2026
e3aa6a5
ci: Log the macOS engines into Docker Hub to stay under the pull rate…
Pijukatel Sep 17, 2026
040e0d2
ci: Tear down a leftover engine before installing a fresh one on the Mac
Pijukatel Sep 17, 2026
c889f25
Merge master into claude/fervent-sagan-3b11vx
Pijukatel Oct 13, 2026
e03a55a
test(e2e): mirror the CLI's SELinux security option in the runtime st…
claude Oct 13, 2026
49202a5
ci: drive the e2e suite with apify-cli built from its actor-runtime b…
claude Oct 13, 2026
3f1aa3e
ci(macos): run the Podman legs on the runner's installed podman, root…
claude Oct 13, 2026
6952444
test(e2e): start sample runs with 1 GB, 8 GB on the macOS runner, ins…
claude Oct 13, 2026
a8d0890
ci(macos): run only the files that prove the hard features on the sel…
claude Oct 13, 2026
6e9c3f8
ci(macos): drop `--rosetta` from `podman machine init`, gone in Podman 6
claude Oct 13, 2026
0b3af66
ci(macos): make Rosetta, not the image's QEMU handler, serve amd64 in…
claude Oct 13, 2026
4ee128c
ci(macos): create the podman machine on applehv, the one provider wit…
claude Oct 13, 2026
0390b70
ci(macos): keep the Rosetta setting in force through `podman machine …
claude Oct 13, 2026
651fe1f
docs: say what the tests and the Mac leg cover, not how
claude Oct 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions .github/scripts/e2e-host-cleanup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#!/usr/bin/env bash
# Removes everything the e2e suite leaves on the host it ran on, so a persistent runner - the
# self-hosted macOS arm64 one in `ci.yml` - starts each job clean and leaves it clean. The hosted
# runners are ephemeral and never need this; there `stopRuntimeContainer`'s `afterAll` is enough,
# and the "Dump runtime container logs on failure" step relies on the container still being there.
# On the macOS leg the engine itself is per-job (`macos-container-engine.sh`), so the engine-side part
# below finds little to do there; the temp dirs and the port check are what it is for on that runner.
#
# Scoped to what the suite and the runtime it drives create, nothing else on the machine:
# - the runtime container(s) `actor-runtime-e2e*` and their `<name>-data` volumes
# (`test/e2e/helpers/docker.ts`), which hold the fixed host ports 3333/3000
# - Actor run, dev-folder probe and browser-viewer containers/volumes the runtime creates, found by
# the labels `docker-driver.ts` puts on them (a runtime killed mid-run cannot clean them itself)
# - images the runtime builds (`actor-runtime/<actor>:<buildId>`, `localhost/actor-runtime/*`) and
# the runtime image the suite builds (`actor-runtime:e2e`), plus whatever dangling layers retagging
# left behind. Pre-pulled base images stay: on a persistent host they are the cache that keeps the
# job fast, and the suite re-pulls them anyway.
# - the suite's temp dirs (`actor-runtime-e2e-*` under `$TMPDIR`, where Node's `os.tmpdir()` puts them)
#
# Usage: e2e-host-cleanup.sh [--require-free-ports]
# --require-free-ports fail once cleanup is done if 3333 or 3000 is still held by something this
# script does not own - meant for the pre-test step, where a busy port would
# otherwise fail `docker run` deep inside the suite with a far less clear error.
#
# Honors the suite's own engine selection: `CONTAINER_CLI=podman` drives Podman, `docker` otherwise.
set -euo pipefail

cli=${CONTAINER_CLI:-docker}
[ "$cli" = podman ] || cli=docker

require_free_ports=0
for arg in "$@"; do
case "$arg" in
--require-free-ports) require_free_ports=1 ;;
*)
echo "Unknown argument: $arg" >&2
exit 2
;;
esac
done

# Whitespace-separated ids -> `$cli <cmd> -f <ids>`; a no-op on an empty list.
remove() {
local kind=$1
shift
local ids
ids=$(printf '%s\n' "$@" | sed '/^$/d' | sort -u)
[ -n "$ids" ] || return 0
echo "Removing $kind:"
printf ' %s\n' $ids
# shellcheck disable=SC2086
"$cli" "$kind" rm -f $ids >/dev/null || true
}

if ! "$cli" info >/dev/null 2>&1; then
echo "::warning::$cli daemon is not reachable; nothing to clean up on the engine side (the suite itself will fail on this)."
else
remove container \
"$("$cli" ps -aq --filter name=actor-runtime-e2e)" \
"$("$cli" ps -aq --filter label=actor-runtime.runId)" \
"$("$cli" ps -aq --filter label=actor-runtime.devFolderProbe)" \
"$("$cli" ps -aq --filter label=actor-runtime.browserViewer)"

remove volume \
"$("$cli" volume ls -q --filter name=actor-runtime-e2e)" \
"$("$cli" volume ls -q --filter label=actor-runtime.browserViewer)"

remove image \
"$("$cli" images -q --filter reference='actor-runtime:e2e')" \
"$("$cli" images -q --filter reference='actor-runtime/*')" \
"$("$cli" images -q --filter reference='localhost/actor-runtime/*')"

# Only dangling (untagged, unreferenced) layers - retagging `actor-runtime:e2e` on every run leaves
# the previous image behind as one, and they add up on a machine that is never recreated.
"$cli" image prune -f >/dev/null || true
fi

# `os.tmpdir()` in Node: `$TMPDIR` if set (macOS sets it per user, to a `/var/folders/...` path),
# `/tmp` otherwise - the same rule as here, so the glob lands where the suite's `mkdtemp` calls did.
tmp=${TMPDIR:-/tmp}
tmp=${tmp%/}
shopt -s nullglob
leftovers=("$tmp"/actor-runtime-e2e-*)
shopt -u nullglob
if [ ${#leftovers[@]} -gt 0 ]; then
echo "Removing temp dirs:"
printf ' %s\n' "${leftovers[@]}"
rm -rf "${leftovers[@]}"
fi

if [ "$require_free_ports" -eq 1 ]; then
if ! command -v lsof >/dev/null 2>&1; then
echo "::warning::lsof not found; cannot verify that ports 3333 and 3000 are free."
exit 0
fi
busy=0
for port in 3333 3000; do
# `docker run -p` binds these on the host; anything still listening now is not ours to remove.
if holders=$(lsof -nP -iTCP:"$port" -sTCP:LISTEN 2>/dev/null) && [ -n "$holders" ]; then
echo "::error::Port $port is still in use after cleanup - the e2e runtime container needs it:"
echo "$holders"
busy=1
fi
done
[ "$busy" -eq 0 ] || exit 1
fi

echo "e2e host cleanup done."
35 changes: 35 additions & 0 deletions .github/scripts/install-apify-cli-package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Installs a packed `apify-cli` tarball - `ci.yml`'s `apify-cli` job builds one from the CLI's
# `actor-runtime` branch, the unreleased CLI the runtime is developed against - into a directory of
# its own and points the e2e suite at it (`ACTOR_RUNTIME_E2E_APIFY_CLI`, `test/e2e/helpers/apify-cli.ts`),
# so every e2e job drives the runtime with that CLI rather than the one published on npm: the two are
# developed together, and this is what keeps them compatible. Local use: the same, with the export
# line printed at the end for the shell that runs the suite.
#
# Usage: install-apify-cli-package.sh <apify-cli-*.tgz> [install-dir]
# install-dir defaults to `$RUNNER_TEMP/apify-cli` (or `$TMPDIR/apify-cli` outside Actions).
set -euo pipefail

tgz=${1:?usage: install-apify-cli-package.sh <apify-cli-*.tgz> [install-dir]}
dir=${2:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}/apify-cli}
[ -f "$tgz" ] || { echo "No such file: $tgz" >&2; exit 2; }
tgz=$(cd "$(dirname "$tgz")" && pwd)/$(basename "$tgz")

# A plain npm install of the tarball as a dependency of a throwaway package: the suite runs the entry
# script straight from `node_modules/apify-cli`, exactly the files `npm install -g apify-cli` would put
# there (the package's `files`, with its dependencies resolved from npm).
rm -rf "$dir"
mkdir -p "$dir"
cd "$dir"
npm init -y >/dev/null
npm install --no-audit --no-fund --loglevel=error "$tgz"

entry=$dir/node_modules/apify-cli/dist/apify.js
[ -f "$entry" ] || { echo "The package did not install an entry script at $entry" >&2; exit 1; }
echo "Installed apify-cli $(node "$entry" --version) from $tgz"

if [ -n "${GITHUB_ENV:-}" ]; then
echo "ACTOR_RUNTIME_E2E_APIFY_CLI=$entry" >> "$GITHUB_ENV"
else
echo "export ACTOR_RUNTIME_E2E_APIFY_CLI=$entry"
fi
Loading
Loading