Repository navigation
Conversation
meta-cla
Bot
added
the
CLA Signed
|
@cortinico has exported this pull request. If you are a Meta employee, you can view the originating Diff in D126723208. |
cortinico
added a commit
to cortinico/react-native
that referenced
this pull request
Oct 22, 2026
Summary: Pull Request resolved: react#59338 Add a Maestro visual suite for `Touchable` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/touchable.md` for the example and checkpoint inventory. Changelog: [Internal] Differential Revision: D126723208
meta-codesync
Bot
changed the title
cortinico
force-pushed
the
export-D126723208
branch
from
October 22, 2026 13:42
5ec4356 to
55e3584
Compare
Summary: RNTester's rendering coverage, particularly text, currently lives in Meta's internal jest-e2e suite. To move it into public CI we need screenshot comparison that works for fork PRs and doesn't add PNGs to the repo. This adds that, as a separate set of visual flows alongside the existing Maestro E2E. ### How it works Visual tests are Maestro flows in `packages/rn-tester/visual-tests/flows/`. Each one deep-links to an RNTester example, waits for it to be ready and captures a screenshot cropped to a `testID`. Maestro only captures - it never decides pass or fail. The work is split across a trust boundary, so a fork PR never runs with credentials: - **Visual capture** runs as two jobs in Test All, reusing its RNTester Release builds: `test_visual_android_rntester` after `build_android` and `test_visual_ios_rntester` after `test_ios_rntester`. Each runs every flow in one Maestro invocation, on a Pixel 7 API 35 emulator (1080x2400, 420dpi) or an iPhone 17 Pro on iOS 26.2, and uploads the PNGs and a manifest recording the commit and device environment as an artifact. They have no secrets. - **Visual Compare RNTester** (`workflow_run` on Test All, from `main`) treats that artifact as data - it's validated before any image is decoded and nothing in it is executed. It finds the baseline, compares with pixelmatch, posts a check run per platform and a single PR comment with expected, captured and diff images, and on pushes to `main` publishes the captures as a new baseline. It runs once Test All completes, including when unrelated jobs fail, so the report arrives with the rest of CI rather than as soon as the captures are ready. - **Visual Approve RNTester** (`pull_request_target`, on the `visual-change-approved` label) records a maintainer's approval of an intentional change. The approval is tied to the exact set of differences, so a later push that changes them removes the label and needs a new review. Comparison is strict by default: any changed pixel is a difference. A test can be loosened in `visual-tests/config.json`, with evidence. Captures have been byte-identical across separate builds and runners so far, so I'd expect that to be rare. ### What defines the baseline A baseline is the full set of captures from a commit on `main`, keyed by commit and by device environment. Every push to `main` publishes one, and nothing is ever overwritten. A PR is compared with the baseline for the commit it's based on (the first parent of the merge commit CI tests), or the nearest earlier commit on `main` that has one, and the report says which. So a PR based on last week's `main` is compared with last week's rendering, not with whatever has landed since. Approving a PR doesn't promote its images - `main` is captured again after merge and that becomes the baseline. The environment key covers everything that affects rendering (OS runtime, device, screen size, density, font scale, locale, appearance, animations, system image and, on Android, the emulator version, which is pinned). If the CI image drifts, the key changes and a fresh baseline is published rather than every PR showing a spurious diff. ### Settings Storage is any GCS bucket, configured through repository variables. Writes use GitHub's OIDC token, so there's no stored secret, and only the two trusted workflows on `main` can authenticate. They need create-only access (`roles/storage.objectCreator`). Reads are public. | Variable | | |---|---| | `VISUAL_BUCKET` | Bucket to upload to | | `VISUAL_PUBLIC_BASE_URL` | Public URL for reads and report images - the bucket, or a CDN in front of it | | `VISUAL_GCP_PROJECT` | GCP project for authentication | | `VISUAL_WORKLOAD_IDENTITY_PROVIDER` | Workload identity provider trusting this repo's `visual-compare-rntester.yml` and `visual-approve-rntester.yml` on `main` | Expo can host the bucket and a CDN for this. Reports and approvals expire after 30 days by lifecycle rule. Baselines are small (8-175 KB per image), and at the current rate of pushes to `main` the cost is cents a month. `packages/rn-tester/visual-tests/README.md` covers adding a test and handling a failure. This starts with two text tests and runs non-blocking. The plan is to port high-value jest-e2e text and rendering tests, then make the checks required once they've proven reliable. ## Changelog: [INTERNAL] [ADDED] - Visual regression testing for RNTester in CI Pull Request resolved: react#59115 Test Plan: Ran the full flow in a fork (https://github.com/robhogan/react-native), with Test All trimmed to the jobs this depends on, against a personal GCS bucket: - A PR with no visual change passes on both platforms (robhogan#3). - A deliberate text layout change fails both platforms, with expected, captured and diff images on the checks page and in the PR comment (robhogan#4). - Applying `visual-change-approved` passes the approval check. Pushing a further, different change removed the label with a comment and failed the check again. - Captures were byte-identical across 5 Android and 4 iOS runs, from 3 separate builds on fresh runners. - When an unrelated Test All job fails, the comparison still runs. Run against a Test All run whose Android capture had failed, it reports Android as having no capture and compares iOS as usual. - Pixel Launcher raised an ANR dialog during emulator boot in 3 of 8 probe runs. After the capture's setup step, none of the 8 had a dialog showing. - Running all flows in one Maestro invocation brought the iOS capture step to 2m20s for two tests, from 3m49s for one test when each flow started Maestro separately. Failed flows are retried individually. - New unit tests for artifact validation (path traversal, symlinks, hash and size checks), comparison, storage keys, approval and report rendering. Not yet tested: a PR from a third-party fork, to confirm it can't reach storage credentials. ### Internal CI repair after the V11 import - Rebased on `remote/master` (`de108a6bc86ad106de94b84aa819e58b23cfc797`). Removed the dependency on abandoned D126750229; no Gauntlet framework files are included. All ten child suite patches are unchanged. Rob's PR react#59115 remains linked. - `xplat/yarn-validation/bin/yarn-validation`: passed for the root and RN OSS workspace, including the offline mirror check. - ARVR `HelixHTestImages-test`, `HelixHTest-test`, and `xplat-js-versions-aligned-test`: 3 suites and 486 tests passed. - `js1 test` selecting `rn-oss`: all 3 visual workflow suites and 51 tests passed. - OSS ESLint with `--no-ignore --max-warnings=0` on all 15 workflow JavaScript files: passed. `arc f` applied formatting; `arc lint` found no new issues. The Record helper missing-test warning also occurs on the unchanged `remote/master` file. - The exact four-root ARVR installer check exits 1 solely for existing Gauntlet `electron-chromedriver/METADATA.bzl` package URL drift. Restored that unrelated file and excluded the abandoned prerequisite change. - Device/Maestro captures were not repeated for this dependency and lint repair; Rob's Android/iOS evidence above and the child suite test plans are retained. Fresh CI results remain pending. Reviewed By: Abbondanzo, cipolleschi Differential Revision: D126719236 Pulled By: cortinico
Summary: Pull Request resolved: react#59340 Add a Maestro visual suite for `Text` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/text.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723199
Summary: Pull Request resolved: react#59344 Add a Maestro visual suite for `TextInput` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/text-input.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723200
Summary: Pull Request resolved: react#59339 Add a Maestro visual suite for `View` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/view.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723201
Summary: Pull Request resolved: react#59343 Add a Maestro visual suite for `Border` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/border.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723202
Summary: Pull Request resolved: react#59347 Add a Maestro visual suite for `Transform` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/transform.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723203
Summary: Pull Request resolved: react#59341 Add a Maestro visual suite for `Image` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/image.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723204
Summary: Pull Request resolved: react#59342 Add a Maestro visual suite for `ScrollView` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/scroll-view.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723205
Summary: Pull Request resolved: react#59346 Add a Maestro visual suite for `Switch` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/switch.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723206
Summary: Pull Request resolved: react#59345 Add a Maestro visual suite for `Button` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/button.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723207
Summary: Pull Request resolved: react#59338 Add a Maestro visual suite for `Touchable` using existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations. See `packages/rn-tester/visual-tests/suites/touchable.md` for the example and checkpoint inventory. Changelog: [Internal] Reviewed By: Abbondanzo Differential Revision: D126723208
cortinico
force-pushed
the
export-D126723208
branch
from
October 22, 2026 14:49
55e3584 to
e4bc470
Compare
|
This pull request has been merged in e9443ba. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Add a Maestro visual suite for
Touchableusing existing RNTester examples. Capture stable cropped checkpoints after opening each example and settling animations.See
packages/rn-tester/visual-tests/suites/touchable.mdfor the example and checkpoint inventory.Changelog: [Internal]
Reviewed By: Abbondanzo
Differential Revision: D126723208