Skip to content

Add android e2e setup + simulator run iOS e2e#546

Merged
dewabisma merged 16 commits into
mainfrom
beast/add-android-e2e-setup
Jul 2, 2026
Merged

Add android e2e setup + simulator run iOS e2e#546
dewabisma merged 16 commits into
mainfrom
beast/add-android-e2e-setup

Conversation

@dewabisma

@dewabisma dewabisma commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Added android emu and ios simu e2e test scripts.

The commit history shown here is buggy because I extend from other PR that already merged to main then I pull main to this branch. But the files changed should be correct. We can ignore and it will resolve it properly. Like how sometimes we have it before.


Note

Low Risk
Changes are limited to test automation (Gradle test runner settings, androidTest stub, and dev scripts); production app behavior is unaffected aside from standard instrumentation test dependencies.

Overview
Adds Patrol-native Android E2E wiring so patrol_test/ Dart tests run as instrumentation tests: PatrolJUnitRunner, clearPackageData between runs, AndroidX Test Orchestrator, and a parameterized MainActivityTest that lists and runs each Dart test.

Introduces patrol_android_emulator.sh and patrol_ios_simulator.sh for local runs—auto-pick or boot an emulator/simulator, optional single-file or full-suite targets, debug/release builds, and test secrets via .env.test or TEST_IMPORT_MNEMONIC (--dart-define, not bundled assets). Both invoke patrol test in one step (unlike the existing physical-iOS device script).

Reviewed by Cursor Bugbot for commit 7c38e21. Configure here.

@dewabisma dewabisma requested a review from n13 July 1, 2026 05:05

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 7c38e21. Configure here.

xcrun simctl list devices available 2>/dev/null \
| grep -F " $name (" \
| tail -1 \
| sed -E 's/^[[:space:]]+[^(]+ \(([A-F0-9-]+)\).*/\1/'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong simulator runtime selected

Medium Severity

When several runtimes define the same simulator name, simulator_udid_for_name uses tail -1 on matching lines, which picks the last runtime block in simctl output. That usually maps to an older iOS runtime, not the newest one the comment describes, so -s can boot the wrong UDID and E2E runs on an unintended OS version.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7c38e21. Configure here.

exit 1
fi
start_emulator "$AVD_NAME"
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reused emulator skips boot wait

Medium Severity

If a running emulator is auto-selected or chosen with -d, the script never calls wait_for_emulator_boot, unlike the path that starts a new AVD. Patrol can start while sys.boot_completed is still unset, leading to flaky or failed instrumentation runs.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 7c38e21. Configure here.

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Solid, well-documented E2E addition. The Android native Patrol wiring is correct (PatrolJUnitRunner + clearPackageData + AndroidX Orchestrator + parameterized MainActivityTest), package placement matches applicationId/patrol android.package_name (com.quantus.wallet), MainActivity import from the namespace package is right, .env.test is already gitignored, and the empty-array expansions correctly guard set -u on macOS Bash 3.2.

Requesting changes for two confirmed correctness bugs + a DRY concern before merge.

Correctness (agree with Bugbot)

  1. Android reused emulator skips boot wait (patrol_android_emulator.sh). wait_for_emulator_boot only runs when a fresh AVD is booted. When an emulator is auto-selected or passed with -d, it's skipped. adb devices reports device before sys.boot_completed=1, so Patrol can start against a not-fully-booted emulator → flaky runs. Fix (also a DRY win): call wait_for_emulator_boot "$DEVICE_SERIAL" once after the serial is resolved, on every path.

  2. iOS wrong runtime selected (patrol_ios_simulator.sh). simulator_udid_for_name uses tail -1, but the comment claims "prefer the newest runtime" — simctl list devices section ordering isn't guaranteed, so -s can boot the wrong OS version (same non-determinism in default_simulator_name). More robust: parse xcrun simctl list devices available -j and pick the highest runtime version.

DRY (repo rule)

The three patrol scripts now triplicate large verbatim blocks: the .env.test/TEST_IMPORT_MNEMONICDART_DEFINES block (~16 lines), the TARGET_ARGS builder, SCRIPT_DIR/APP_ROOT resolution, and the common arg-parse loop. Recommend extracting scripts/lib/patrol_common.sh (e.g. patrol_resolve_app_root, patrol_collect_dart_defines, patrol_build_target_args) sourced by all three scripts.

Minor nits

  • Stale comment in MainActivityTest.java: "...run/filtered individually by xcodebuild's Android equivalent (the orchestrator)" — "xcodebuild" is an iOS copy-paste leftover.
  • patrol_ios_simulator.sh passes PATROL_DEVICE="${DEVICE_LABEL:-$DEVICE_ID}", preferring the possibly-ambiguous name over the already-resolved UDID; passing $DEVICE_ID is unambiguous when duplicate names exist.

Verdict: Request changes — prioritize the Android boot-wait fix and the DRY extraction; the iOS runtime selection and nits are quick follow-ups.

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review (commits 2456ccf4c0f52b)

The three items from the previous review were all addressed — thank you. Unfortunately the DRY refactor introduced a critical regression that breaks every platform-specific flag in all three scripts, so requesting changes one more time. Reviewed in a clean worktree with the bugs reproduced by actually running the scripts.

Critical: platform option callbacks run in a subshell — -d, -a, -s are all silently ignored

In lib/patrol_common.sh, patrol_parse_runner_args invokes the platform callback via command substitution:

consumed="$("$platform_option_fn" "$@")" || consumed=0

Command substitution runs the callback in a subshell, so assignments like DEVICE_SERIAL=…, AVD_NAME=…, DEVICE_ID=…, SIMULATOR_NAME=…, DEVICE_UDID=… are lost when the subshell exits. The echo 2 consumed-count still reaches the parent, so the flag and its value are shifted away without error — a textbook silent failure. Reproduced on all three scripts:

  • patrol_android_emulator.sh -d emulator-9999 → serial discarded, falls into auto-select. With another emulator running it would silently test the wrong device; with none it errors with "No running emulator found" despite an explicit serial.
  • patrol_ios_simulator.sh -d <udid> → UDID discarded; the script booted a different simulator (the default-name fallback) and would have run tests on it.
  • patrol_ios_device.sh -d <udid> → UDID discarded → "ERROR: No iOS device found. Connect a device or pass its UDID."
  • -s "iPhone 16 Pro" and -a <avd> are equally ignored.

The pre-refactor inline parsers (7c38e21) handled these flags correctly, so this is a pure regression from 2456ccf.

Suggested fix: don't use command substitution for the callback. Call it directly and pass the consumed count through a global, e.g.:

# patrol_common.sh
if [[ "$platform_option_fn" != "-" ]] && "$platform_option_fn" "$@"; then
  shift "$PLATFORM_CONSUMED"
  continue
fi

# callers
patrol_android_emulator_platform_option() {
  case "$1" in
    -d|--device) DEVICE_SERIAL="${2:?--device requires a serial}"; PLATFORM_CONSUMED=2; return 0 ;;
    -a|--avd)    AVD_NAME="${2:?--avd requires a name}";           PLATFORM_CONSUMED=2; return 0 ;;
  esac
  return 1
}

Please add a quick smoke test of each flag before pushing (-d, -a, -s, --release, a target file) — none of the flag paths currently work.

Bug: first_booted_simulator breaks on parenthesized simulator names

The sed in patrol_ios_simulator.sh:

sed -E 's/^[[:space:]]+([^(]+) \(([A-F0-9-]+)\).*/\1\t\2/'

[^(]+ cannot cross a (, so for stock names like iPhone SE (3rd generation) (UDID) (Booted) the regex never reaches the UDID group, sed passes the line through unmodified, and DEVICE_LABEL/DEVICE_ID both become the raw untrimmed line — which is then handed to patrol test --device. Verified with a synthetic line: plain names produce name\tudid, parenthesized names produce garbage. Since the auto-select path is currently the only working path (see bug above), anyone with an SE-class simulator booted hits this. Note simulator_udid_for_name doesn't have this problem (its awk uses literal index() and matches the UDID pattern directly) — consider reusing that approach for the booted case too.

Previous review items — confirmed fixed

  • Android boot wait on every pathwait_for_emulator_boot "$DEVICE_SERIAL" now runs once after the serial is resolved, covering auto-select, -d, and fresh-boot paths.
  • iOS newest-runtime selection — the awk version-key approach in simulator_udid_for_name / default_simulator_name correctly parses the -- iOS X.Y -- section headers and picks the highest version; verified the logic against real simctl output shape. (JSON would still be more robust, but this works.)
  • DRY extractionlib/patrol_common.sh with patrol_resolve_app_root / patrol_collect_dart_defines / patrol_build_target_args is exactly the right structure; the shared-flag parsing is the right idea too, it just needs the subshell fix above.

Also re-verified: Gradle wiring (PatrolJUnitRunner + clearPackageData + orchestrator 1.5.1) is standard and correct, MainActivityTest package placement matches applicationId/patrol package_name (com.quantus.wallet) with MainActivity imported from the namespace package, .env.test is gitignored, and the ${ARR[@]+"${ARR[@]}"} expansions behave under macOS Bash 3.2 set -u (tested).

Nits (carried over, still open)

  • MainActivityTest.java still has the iOS copy-paste leftover: "run/filtered individually by xcodebuild's Android equivalent (the orchestrator)".
  • PATROL_DEVICE="${DEVICE_LABEL:-$DEVICE_ID}" still prefers the possibly-ambiguous name over the resolved UDID.
  • Tiny: in simulator_udid_for_name, ver is never reset when a non-iOS section header (tvOS/watchOS) starts; harmless for iPhone names but worth a guard.

Verdict: request changes — the flag-parsing regression must be fixed (and the flags actually exercised) before merge; the sed name bug should go in the same pass.

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review (commits ebdc0c22e563c2)

Both blocking issues from the last review are fixed. Verified by executing all three scripts against stubbed adb/emulator/xcrun/patrol/xcodebuild binaries under macOS stock Bash 3.2, exercising every flag path.

✅ Flag-parsing regression fixed

The platform callback is now invoked directly instead of via command substitution, with the consumed count passed through PLATFORM_CONSUMED — so DEVICE_SERIAL/AVD_NAME/DEVICE_ID/SIMULATOR_NAME/DEVICE_UDID assignments persist in the parent shell. Confirmed by running:

  • patrol_android_emulator.sh -d emulator-9999 (with a different emulator running) → tests correctly target emulator-9999, and the boot wait runs on that path too. -a My_Custom_AVD boots the named AVD; --release and positional targets are consumed correctly; the auto-select path still works.
  • patrol_ios_simulator.sh -d <udid> → the UDID reaches patrol test --device <udid> unchanged. -s "iPhone 16 Pro" resolves the newest runtime when two runtimes have the same device name (picked the iOS 26.5 UDID over 18.2 in my fixture).
  • patrol_ios_device.sh -d <udid> → flows through to xcodebuild -destination "platform=iOS,id=<udid>"; --release is correctly rejected there since that script doesn't take build-mode flags.

first_booted_simulator name parsing fixed

The awk match(/\([A-F0-9-]+\)/) approach handles parenthesized device names: a booted iPhone SE (3rd generation) (<udid>) (Booted) now yields the correct name+UDID pair, and -s "iPhone SE (3rd generation)" resolves correctly too (both verified against synthetic simctl output).

Footnote, not blocking: awk takes the first regex match, so a custom simulator name containing a parenthesized run of only [A-F0-9-] characters (e.g. Test (ABC)) would mis-parse. All stock Apple names are safe — "(3rd generation)" contains lowercase — so this is theoretical.

Nits carried over (non-blocking, fine as a follow-up)

  • MainActivityTest.java still has the "xcodebuild's Android equivalent" copy-paste leftover in the class comment.
  • PATROL_DEVICE="${DEVICE_LABEL:-$DEVICE_ID}" still prefers the possibly-ambiguous name over the resolved UDID on the auto-select and -s paths (the -d path correctly uses the UDID).
  • ver in simulator_udid_for_name isn't reset when a non-iOS runtime section (tvOS/watchOS) starts; harmless for iPhone names.

Verdict: Approve. The two blockers are properly fixed, all documented flag paths now work as advertised, and the shared patrol_common.sh structure is clean.

@n13 n13 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review (head still 2e563c2)

Fresh independent verification pass — head commit is unchanged since the previous approval, so this re-confirms that verdict from scratch rather than reviewing new code. All three scripts were re-executed against stubbed adb/emulator/xcrun/patrol/idevice_id/xcodebuild binaries under macOS stock Bash 3.2 (3.2.57), exercising every documented path.

Verified by execution

patrol_android_emulator.sh

  • Auto-select running emulator → boot wait → patrol test --device emulator-5554 --debug
  • -d emulator-9999 with a different emulator running → targets emulator-9999, boot wait included ✅
  • -a My_Custom_AVD with nothing running → boots exactly that AVD ✅; with no -a/env → falls back to first listed AVD ✅
  • --release + positional target → --release -t <file> reaches patrol ✅; -- separator with multiple targets → each gets its own -t ✅; unknown flag → error + usage, exit 1 ✅

patrol_ios_simulator.sh

  • Booted iPhone SE (3rd generation) (<udid>) (Booted) auto-selected with correct name/UDID split (the parenthesized-name fix holds) ✅
  • -d <udid> → UDID passed through unchanged ✅
  • -s "iPhone 16 Pro" with the name present under both iOS 18.2 and 26.5 → boots the 26.5 UDID (newest runtime) ✅
  • No booted sim, no -sdefault_simulator_name picks an iPhone from the newest runtime section and correctly stops at the following -- tvOS -- header ✅
  • -s "iPhone 99" → clean error with the simctl list hint, exit 1 ✅

patrol_ios_device.sh

  • -d <udid> persists through the shared parser (PLATFORM_CONSUMED mechanism) ✅; --release correctly rejected since that script takes no build-mode flag ✅

Static re-checks

  • Gradle wiring (PatrolJUnitRunner + clearPackageData + orchestrator 1.5.1) matches Patrol's documented Android setup; MainActivityTest package com.quantus.wallet matches applicationId/patrol android.package_name, with MainActivity imported from the namespace package.
  • CI is green (Analyze passed on 2e563c2).
  • shellcheck: only false-positive SC2034s (cross-file globals) and informational notes; nothing actionable.

Nits carried over (non-blocking, same as before)

  • MainActivityTest.java: "xcodebuild's Android equivalent" copy-paste leftover in the class comment.
  • PATROL_DEVICE="${DEVICE_LABEL:-$DEVICE_ID}" passes the (possibly ambiguous) name to patrol test --device on the auto-select and -s paths; the -d path uses the UDID.
  • ver in simulator_udid_for_name isn't reset at non-iOS runtime headers; harmless for iPhone names.

Verdict: Approve. Re-verified end to end — every flag path behaves as documented, both previously-fixed blockers stay fixed, and no new issues surfaced.

@dewabisma dewabisma merged commit f70e942 into main Jul 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants