Skip to content

Commit 29ac393

Browse files
feat(ci): unified package-sdk.sh contract + helper scripts
Three cross-cutting helpers + one per-SDK wrapper for the packaging contract discussed in thoughts/shared/plans/artifact-build-system.md. scripts/: - detect-mode.sh sets RAC_BUILD_MODE=local|ci based on $CI / $GITHUB_ACTIONS. Callers source it to share the same detection logic. - validate-artifact.sh one-shot sanity checks per artifact extension: XCFramework zip (Info.plist + slices), .so (ELF magic + arch), .aar (classes.jar + jni/*.so), .jar (manifest), .wasm (magic bytes), .tgz (package.json at top). Same script runs locally and in CI. - sync-checksums.sh ZIP_DIR after native iOS/macOS builds, reads the fresh .zip SHAs and updates the `checksum: "..."` lines in Package.swift for the remote binaryTarget entries. Per-SDK `package-sdk.sh` (unified contract): USAGE: package-sdk.sh [--mode local|ci] [--natives-from PATH] - Swift validates that `swift build` succeeds with natives in Binaries/. No tarball — SPM consumers reference git tags. - Kotlin stages .so files into src/androidMain/jniLibs/, runs gradle assembleRelease + jvmJar, emits AAR + JAR + sha256 to dist/sdk-kotlin/. - Web stages WASM into packages/*/wasm/, runs build:ts, emits 3 npm tarballs (@runanywhere/web, /web-llamacpp, /web-onnx) to dist/sdk-web/. - Flutter stages iOS xcframeworks + Android .so into each package's plugin dirs, runs flutter pub publish --dry-run per package. No tarball — pub.dev consumers reference git URLs. - RN stages natives into each package, runs npm pack per workspace, emits 3 tarballs (@runanywhere/core, /llamacpp, /onnx) to dist/sdk-rn/. `package-sdk.sh` is named distinctly from the pre-existing `build-sdk.sh` wrappers (which are developer-pipeline orchestrators that also build C++ from source). package-sdk.sh consumes pre-built natives. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 9190d96 commit 29ac393

8 files changed

Lines changed: 796 additions & 0 deletions

File tree

scripts/detect-mode.sh

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
#!/usr/bin/env bash
2+
# =============================================================================
3+
# detect-mode.sh
4+
# =============================================================================
5+
# Sets RAC_BUILD_MODE to "local" or "ci" based on the environment, unless the
6+
# caller already set it. Other build scripts can `source` this to share the
7+
# same detection logic:
8+
#
9+
# source "$(dirname "$0")/../../../scripts/detect-mode.sh"
10+
#
11+
# Contract:
12+
# RAC_BUILD_MODE=ci ← running in GitHub Actions (or any CI environment
13+
# that sets $CI=true)
14+
# RAC_BUILD_MODE=local ← developer machine
15+
#
16+
# Behavior that should differ between modes:
17+
# local: tolerant of missing deps, uses cache, prints hints
18+
# ci: strict toolchain checks, no cache warmup, fail fast
19+
#
20+
# Callers can force a mode by setting RAC_BUILD_MODE before sourcing this.
21+
# =============================================================================
22+
23+
if [ -n "${RAC_BUILD_MODE:-}" ]; then
24+
# Respect explicit override
25+
:
26+
elif [ "${CI:-}" = "true" ] || [ -n "${GITHUB_ACTIONS:-}" ]; then
27+
RAC_BUILD_MODE="ci"
28+
else
29+
RAC_BUILD_MODE="local"
30+
fi
31+
32+
export RAC_BUILD_MODE
33+
34+
if [ "${VERBOSE:-}" = "1" ]; then
35+
echo "RAC_BUILD_MODE=${RAC_BUILD_MODE}"
36+
fi

scripts/sync-checksums.sh

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
#!/usr/bin/env bash
2+
# =============================================================================
3+
# sync-checksums.sh
4+
# =============================================================================
5+
# Updates the SHA-256 checksum lines in Package.swift's remote binaryTarget
6+
# entries to match freshly-built XCFramework zips. Run after the native
7+
# iOS/macOS builds have produced the zips and before cutting a release tag.
8+
#
9+
# Usage:
10+
# scripts/sync-checksums.sh ZIP_DIR
11+
#
12+
# Example:
13+
# scripts/sync-checksums.sh sdk/runanywhere-commons/dist
14+
# scripts/sync-checksums.sh release-artifacts/native-ios-macos
15+
#
16+
# Looks for files of the form:
17+
# {name}-v{version}.zip
18+
# where {name} is one of:
19+
# RACommons, RABackendLLAMACPP, RABackendONNX, RABackendMetalRT,
20+
# onnxruntime-ios, onnxruntime-macos
21+
#
22+
# and updates the corresponding `checksum: "..."` line in Package.swift.
23+
# =============================================================================
24+
25+
set -euo pipefail
26+
27+
if [ $# -ne 1 ]; then
28+
echo "usage: $0 ZIP_DIR" >&2
29+
exit 1
30+
fi
31+
32+
ZIP_DIR="$1"
33+
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
34+
PACKAGE_SWIFT="${REPO_ROOT}/Package.swift"
35+
36+
if [ ! -f "$PACKAGE_SWIFT" ]; then
37+
echo "ERROR: Package.swift not found at $PACKAGE_SWIFT" >&2
38+
exit 1
39+
fi
40+
41+
if [ ! -d "$ZIP_DIR" ]; then
42+
echo "ERROR: zip dir not found: $ZIP_DIR" >&2
43+
exit 1
44+
fi
45+
46+
# swiftpm binary target name → local-filename-prefix pairs. Names match the
47+
# `.binaryTarget(name: "X", ...)` entries in Package.swift.
48+
declare_mapping() {
49+
# Printed form: BINARY_NAME|ZIP_PREFIX
50+
echo "RACommonsBinary|RACommons"
51+
echo "RABackendLlamaCPPBinary|RABackendLLAMACPP"
52+
echo "RABackendONNXBinary|RABackendONNX"
53+
echo "RABackendMetalRTBinary|RABackendMetalRT"
54+
echo "ONNXRuntimeiOSBinary|onnxruntime-ios"
55+
echo "ONNXRuntimemacOSBinary|onnxruntime-macos"
56+
}
57+
58+
sha256_of() {
59+
# macOS: shasum. Linux: sha256sum. Both emit `<hex> <file>` on stdout.
60+
if command -v shasum >/dev/null 2>&1; then
61+
shasum -a 256 "$1" | awk '{print $1}'
62+
else
63+
sha256sum "$1" | awk '{print $1}'
64+
fi
65+
}
66+
67+
update_checksum_line() {
68+
# Updates the `checksum: "..."` line that belongs to the `name: "$1"`
69+
# binaryTarget in Package.swift. Relies on the checksum appearing within
70+
# a few lines after the name line.
71+
local binary_name="$1"
72+
local new_sum="$2"
73+
python3 - "$binary_name" "$new_sum" "$PACKAGE_SWIFT" <<'PY'
74+
import re, sys
75+
76+
binary_name, new_sum, path = sys.argv[1], sys.argv[2], sys.argv[3]
77+
with open(path) as f:
78+
src = f.read()
79+
80+
# Find the `name: "X"` line and the first `checksum: "..."` within 6 lines.
81+
pattern = re.compile(
82+
r'(name:\s*"' + re.escape(binary_name) + r'".*?checksum:\s*")([0-9a-f]{64})(")',
83+
re.DOTALL,
84+
)
85+
86+
m = pattern.search(src)
87+
if not m:
88+
print(f" skip: no remote binary target named '{binary_name}' found in Package.swift",
89+
file=sys.stderr)
90+
sys.exit(0)
91+
92+
old_sum = m.group(2)
93+
if old_sum == new_sum:
94+
print(f" unchanged: {binary_name} ({old_sum[:12]}...)")
95+
sys.exit(0)
96+
97+
src = src[:m.start(2)] + new_sum + src[m.end(2):]
98+
with open(path, "w") as f:
99+
f.write(src)
100+
print(f" bumped: {binary_name} {old_sum[:12]}... → {new_sum[:12]}...")
101+
PY
102+
}
103+
104+
echo ">> Syncing Package.swift checksums from $ZIP_DIR"
105+
106+
missing=0
107+
updated=0
108+
109+
while IFS='|' read -r binary_name zip_prefix; do
110+
# Match files like RACommons-v0.20.0.zip, RACommons-v0.20.0-beta.zip, etc.
111+
zip_file=$(ls "$ZIP_DIR"/${zip_prefix}-v*.zip 2>/dev/null | head -1 || true)
112+
if [ -z "$zip_file" ]; then
113+
echo " missing: no ${zip_prefix}-v*.zip in $ZIP_DIR"
114+
missing=$((missing + 1))
115+
continue
116+
fi
117+
sum=$(sha256_of "$zip_file")
118+
update_checksum_line "$binary_name" "$sum"
119+
updated=$((updated + 1))
120+
done < <(declare_mapping)
121+
122+
echo ""
123+
echo ">> Done. $updated processed, $missing missing."
124+
echo ""
125+
echo ">> Verify with:"
126+
echo " git diff -- Package.swift"

scripts/validate-artifact.sh

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
#!/usr/bin/env bash
2+
# =============================================================================
3+
# validate-artifact.sh
4+
# =============================================================================
5+
# Same-shape artifact validator for local + CI. Given a path to one of our
6+
# published artifact files, does the minimum sanity-checks to catch
7+
# "built it but it's broken" before we publish:
8+
#
9+
# .zip → unzip list non-empty, contains expected files
10+
# .xcframework (as .zip) → Info.plist present + declares arch slices
11+
# .so → ELF, machine arch matches filename convention
12+
# .aar → unzip lists classes.jar + jni/{abi}/*.so
13+
# .wasm → starts with WebAssembly magic bytes (0x00 'asm')
14+
# .tgz (npm) → `tar -tzf` lists package/package.json
15+
# .jar → zip-listable, has META-INF/MANIFEST.MF
16+
#
17+
# Usage:
18+
# scripts/validate-artifact.sh PATH [PATH ...]
19+
#
20+
# Exit status: 0 if every path passed, 1 on first failure.
21+
# =============================================================================
22+
23+
set -euo pipefail
24+
25+
if [ $# -eq 0 ]; then
26+
echo "usage: $0 FILE [FILE ...]" >&2
27+
exit 1
28+
fi
29+
30+
fail() { echo "$1" >&2; exit 1; }
31+
ok() { echo "$1"; }
32+
33+
validate_one() {
34+
local path="$1"
35+
36+
if [ ! -f "$path" ]; then
37+
fail "not a regular file: $path"
38+
fi
39+
40+
local size
41+
size=$(stat -f%z "$path" 2>/dev/null || stat -c%s "$path")
42+
if [ "$size" -lt 64 ]; then
43+
fail "suspiciously small ($size bytes): $path"
44+
fi
45+
46+
echo ">> $path ($size bytes)"
47+
48+
case "$path" in
49+
*.sha256)
50+
# .sha256 is its own checksum file; sanity check it's single-line, 64 hex + space + filename
51+
local line
52+
line=$(head -1 "$path")
53+
if ! echo "$line" | grep -Eq '^[0-9a-f]{64} '; then
54+
fail "bad .sha256 format in $path: $line"
55+
fi
56+
ok ".sha256 looks well-formed"
57+
;;
58+
*.wasm)
59+
# Magic: 0x00 'asm' (0x00 0x61 0x73 0x6d)
60+
local magic
61+
magic=$(head -c 4 "$path" | xxd -p | head -1)
62+
if [ "$magic" != "0061736d" ]; then
63+
fail "not a WebAssembly module (bad magic '$magic'): $path"
64+
fi
65+
ok "WebAssembly magic bytes OK"
66+
;;
67+
*.so)
68+
if ! head -c 4 "$path" | grep -q $'\x7fELF' 2>/dev/null; then
69+
fail "not an ELF shared library: $path"
70+
fi
71+
ok "ELF shared library OK"
72+
if command -v readelf >/dev/null 2>&1; then
73+
local arch
74+
arch=$(readelf -h "$path" 2>/dev/null | awk -F'Machine:' 'NF>1 {print $2}' | xargs)
75+
[ -n "$arch" ] && echo " machine: $arch"
76+
fi
77+
;;
78+
*.aar)
79+
if ! unzip -l "$path" >/dev/null 2>&1; then
80+
fail "cannot unzip $path"
81+
fi
82+
if ! unzip -l "$path" | grep -q '^.* classes.jar$'; then
83+
fail "AAR missing classes.jar: $path"
84+
fi
85+
ok "AAR contains classes.jar"
86+
if unzip -l "$path" | grep -q 'jni/[^/]*/.*\.so$'; then
87+
local count
88+
count=$(unzip -l "$path" | grep -c 'jni/[^/]*/.*\.so$' || true)
89+
ok "AAR contains $count jni/*.so entries"
90+
else
91+
echo " note: no JNI .so files bundled (AAR may link against external natives)"
92+
fi
93+
;;
94+
*.jar)
95+
if ! unzip -l "$path" >/dev/null 2>&1; then
96+
fail "cannot unzip $path"
97+
fi
98+
if ! unzip -l "$path" | grep -q 'META-INF/MANIFEST.MF$'; then
99+
fail "JAR missing META-INF/MANIFEST.MF: $path"
100+
fi
101+
ok "JAR has valid manifest"
102+
;;
103+
*.tgz|*.tar.gz)
104+
if ! tar -tzf "$path" >/dev/null 2>&1; then
105+
fail "cannot list tarball $path"
106+
fi
107+
if tar -tzf "$path" | grep -q '^package/package.json$'; then
108+
ok "npm tarball contains package/package.json"
109+
else
110+
ok "tarball listable ($(tar -tzf "$path" | wc -l | xargs) entries)"
111+
fi
112+
;;
113+
*.zip)
114+
if ! unzip -l "$path" >/dev/null 2>&1; then
115+
fail "cannot unzip $path"
116+
fi
117+
# XCFramework ZIPs contain an Info.plist at top of the framework root
118+
if unzip -l "$path" | grep -q '\.xcframework/Info\.plist$'; then
119+
ok "XCFramework ZIP: Info.plist present"
120+
# Count arch slices (each subdir with an Info.plist sibling is a slice)
121+
local slices
122+
slices=$(unzip -l "$path" | grep -c '\.xcframework/[^/]*/Info\.plist$' || true)
123+
[ "$slices" -gt 0 ] && echo " arch slices declared: $slices"
124+
else
125+
ok "ZIP listable ($(unzip -l "$path" | tail -1 | awk '{print $2}') entries)"
126+
fi
127+
;;
128+
*)
129+
ok "unknown extension — size check only"
130+
;;
131+
esac
132+
}
133+
134+
for path in "$@"; do
135+
validate_one "$path"
136+
done
137+
138+
echo ""
139+
echo "All $# artifact(s) passed validation."
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#!/usr/bin/env bash
2+
# =============================================================================
3+
# sdk/runanywhere-flutter/scripts/package-sdk.sh
4+
# =============================================================================
5+
# Unified SDK packaging contract for the Flutter SDK. Consumes pre-built
6+
# iOS XCFrameworks + Android .so files, stages them into each flutter
7+
# package's plugin-native directories, then validates the package with
8+
# `flutter pub publish --dry-run`.
9+
#
10+
# No tarball is produced — Flutter pub.dev packages are consumed by git-ref
11+
# or by publishing, not by file URL. `--dry-run` verifies the package shape
12+
# is valid.
13+
#
14+
# USAGE:
15+
# package-sdk.sh [--mode local|ci] [--natives-from PATH]
16+
#
17+
# OPTIONS:
18+
# --mode local|ci Build mode (default: auto-detect from $CI)
19+
# --natives-from PATH Directory with iOS xcframeworks + Android .so files.
20+
# Expected: PATH/{ios,android}/<stuff> OR PATH/*.zip/*.tar.gz
21+
# =============================================================================
22+
23+
set -euo pipefail
24+
25+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
26+
REPO_ROOT="$(cd "${SCRIPT_DIR}/../../.." && pwd)"
27+
FLUTTER_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
28+
29+
source "${REPO_ROOT}/scripts/detect-mode.sh"
30+
31+
NATIVES_FROM=""
32+
33+
while [ $# -gt 0 ]; do
34+
case "$1" in
35+
--mode) RAC_BUILD_MODE="$2"; shift 2 ;;
36+
--natives-from) NATIVES_FROM="$2"; shift 2 ;;
37+
--help|-h) head -22 "$0" | tail -18; exit 0 ;;
38+
*) echo "unknown option: $1" >&2; exit 1 ;;
39+
esac
40+
done
41+
42+
echo ">> Flutter SDK packaging (mode=${RAC_BUILD_MODE})"
43+
44+
if [ -n "$NATIVES_FROM" ]; then
45+
[ -d "$NATIVES_FROM" ] || { echo "ERROR: --natives-from not found: $NATIVES_FROM" >&2; exit 1; }
46+
# Each Flutter plugin package has its own ios/ and android/ subdirs. Stage
47+
# matching natives into each. This is intentionally a best-effort copy —
48+
# the binary_config.gradle files inside each package are responsible for
49+
# wiring them up correctly at build time.
50+
echo ">> Staging natives from $NATIVES_FROM into each flutter package"
51+
for pkg_dir in "$FLUTTER_ROOT/packages"/*/; do
52+
pkg=$(basename "$pkg_dir")
53+
# Android: copy per-ABI .so files
54+
android_jni="$pkg_dir/android/src/main/jniLibs"
55+
for abi in arm64-v8a armeabi-v7a x86_64 x86; do
56+
if [ -d "$NATIVES_FROM/$abi" ]; then
57+
mkdir -p "$android_jni/$abi"
58+
cp -f "$NATIVES_FROM/$abi"/*.so "$android_jni/$abi/" 2>/dev/null || true
59+
fi
60+
done
61+
# iOS: copy xcframeworks if present
62+
if ls "$NATIVES_FROM"/*.xcframework >/dev/null 2>&1; then
63+
mkdir -p "$pkg_dir/ios/Frameworks"
64+
cp -R "$NATIVES_FROM"/*.xcframework "$pkg_dir/ios/Frameworks/" 2>/dev/null || true
65+
fi
66+
done
67+
fi
68+
69+
# Bootstrap with melos if available
70+
cd "$FLUTTER_ROOT"
71+
if command -v melos >/dev/null 2>&1; then
72+
echo ">> melos bootstrap"
73+
melos bootstrap || echo "::warning::melos bootstrap failed — continuing"
74+
fi
75+
76+
# Validate each package with flutter pub publish --dry-run
77+
for pkg_dir in "$FLUTTER_ROOT/packages"/*/; do
78+
pkg=$(basename "$pkg_dir")
79+
if [ ! -f "$pkg_dir/pubspec.yaml" ]; then
80+
continue
81+
fi
82+
echo ""
83+
echo ">> Validating $pkg"
84+
(
85+
cd "$pkg_dir"
86+
flutter pub get || echo "::warning::pub get failed for $pkg"
87+
flutter pub publish --dry-run || echo "::warning::pub publish dry-run failed for $pkg"
88+
)
89+
done
90+
91+
echo ""
92+
echo ">> Flutter SDK packages validated. No tarball emitted — consumers"
93+
echo " reference each package via git-URL in their pubspec.yaml, or pub.dev"
94+
echo " publishes those (we don't publish to pub.dev in this release flow)."

0 commit comments

Comments
 (0)