diff --git a/endform-lambda-playwright-programmatic-runner-plan.md b/endform-lambda-playwright-programmatic-runner-plan.md new file mode 100644 index 0000000000000..b8dfa65ed4edf --- /dev/null +++ b/endform-lambda-playwright-programmatic-runner-plan.md @@ -0,0 +1,488 @@ +# Endform Lambda Playwright Programmatic Runner Plan + +## Summary + +Endform runs Playwright tests remotely inside AWS Lambda. The current Lambda Runner invokes Playwright through the normal CLI after preparing scratch files, dependencies, proxy state, config rewrites, reporter shims, and environment variables. This works, but it serializes too much startup work and expresses Endform policy through fragile command-line arguments and temporary edits to the user's Playwright config file. + +The new target architecture is a per-invocation Node host process that starts as early as possible, imports Playwright, preforks Playwright workers, and later receives a structured `runTests` request from Rust over IPC. The Playwright fork should expose generic programmatic runner primitives. Endform-specific config policy, reporter pipeline construction, generated reporters, and Rust IPC protocol should stay in the Lambda Runner repository, ideally inside an Endform-owned Node host script. + +The Playwright fork should not expose Endform-shaped APIs. It should expose a small, stable, Playwright-shaped API that lets an external host load a user config, mutate that config in normal JavaScript, and run selected tests with preforked workers and native Playwright reporting. + +## Goals + +- Start the Node Playwright host as early as possible in each Lambda invocation. +- Import Playwright and prefork one or more Playwright workers while Rust downloads dependencies, prepares scratch space, starts proxy infrastructure, and computes run parameters. +- Replace CLI argument construction with a typed programmatic `runTests` call. +- Replace temporary mutation of `playwright.config.*` with direct mutation of the raw user config object in the Endform Node host. +- Replace `--test-list=` with structured in-memory test selection. +- Keep reporter execution in the Playwright host process, preserving native Playwright reporter lifecycle and object identity. +- Keep Endform-specific policy out of the Playwright fork where practical. +- Keep the fork surface small enough to carry across multiple Playwright version branches. +- Design the generic pieces so some can plausibly be upstreamed later without bringing Endform-specific behavior with them. + +## Non-Goals + +- Do not move Playwright native reporters into worker processes. +- Do not make the Playwright fork understand Endform concepts such as suite-control, completion assets, trace upload policy, remote reporters, or test attempt IDs. +- Do not introduce a Playwright-fork-owned config mutation DSL such as `deleteTopLevel`, `mergeUse`, or `reporterPipeline`. +- Do not require the Playwright fork to own the Rust-to-Node IPC protocol. +- Do not remove Chrome launch monkey patching in this phase. Endform can continue using `NODE_OPTIONS` and spawn/fork monkey patches for Chrome flags and related process behavior. +- Do not optimize away runner-side test loading or prewarm browser contexts until the generic prefork/programmatic runner path is measured. + +## Current Endform Flow + +The current Lambda Runner flow is roughly: + +1. Rust receives the Lambda invocation. +2. Rust builds a `PlaywrightController` with middlewares. +3. Controller setup downloads dependent files, creates symlinks, writes env vars, starts proxy infrastructure, creates generated JS reporters, and prepares config overrides. +4. The controller writes a generated config override shim. +5. The controller temporarily edits the user's `playwright.config.*` file so it imports/requires that shim. +6. The controller spawns `node test ...`. +7. Playwright CLI loads the modified config file. +8. Playwright runs tests normally. +9. Generated reporters send observer events back to Rust through a file descriptor transport. +10. Rust waits for Playwright exit, restores the config file, runs teardowns, uploads artifacts, and reports results. + +This means Playwright startup begins late, after much of setup has already completed. It also means config policy is expressed partly as CLI flags and partly as generated JavaScript injected into the user's config file. + +## Current CLI Arguments To Replace + +Endform currently constructs Playwright CLI arguments equivalent to: + +```text +node test +--config= +--retries=0 +--workers=1 +--max-failures=0 +--no-deps +--timeout= +--update-snapshots= +--test-list= +``` + +These should become structured parameters to `runTests`: + +```ts +await runTests({ + config, + configLocation, + ignoreProjectDependencies: true, + testSelection, + workerEnv, + preforkedWorkers, +}); +``` + +The exact field names can follow Playwright naming, but the important boundary is that these are programmatic run options, not CLI strings. + +## Current Config File Mutations To Move Out Of The Fork + +Endform currently mutates the loaded config through a generated shim. Examples include: + +- Delete top-level `webServer`. +- Delete top-level `globalSetup`. +- Delete top-level `globalTeardown`. +- Delete `use.launchOptions.executablePath` from top-level `use`. +- Delete `use.launchOptions.executablePath` from every project `use`. +- Set top-level and project `outputDir`. +- Set top-level and project `use.trace`. +- Set top-level and project `use.browserName = "chromium"`. +- Set top-level and project `use.defaultBrowserType = "chromium"`. +- Set top-level and project `use.channel = "chromium"`. +- Merge Endform-provided `extraHTTPHeaders` into top-level and project `use.extraHTTPHeaders` with user config taking precedence. +- Rewrite `reporter` to include generated Endform reporters, selected pre-existing user reporters, OTEL reporter, and blob reporter in the correct order. + +The Playwright fork should not own a schema for these mutations. Instead, the Lambda Runner's Node host should load the raw user config object and mutate it directly with ordinary JavaScript before passing it to Playwright's new `runTests` export. + +Example Endform-owned host logic: + +```ts +const config = await loadUserConfig(configLocation); + +delete config.webServer; +delete config.globalSetup; +delete config.globalTeardown; + +deleteNestedUse(config.use, ['launchOptions', 'executablePath']); +for (const project of config.projects ?? []) + deleteNestedUse(project.use, ['launchOptions', 'executablePath']); + +setOutputDirEverywhere(config, request.outputDir); +setUseEverywhere(config, { + trace: request.trace, + browserName: 'chromium', + defaultBrowserType: 'chromium', + channel: 'chromium', +}); +mergeUseEverywhere(config, 'extraHTTPHeaders', request.extraHTTPHeaders, 'lower'); + +config.reporter = buildEndformReporterList(config.reporter, request); + +await runTests({ + configLocation, + config, + ignoreProjectDependencies: true, + testSelection, + preforkedWorkers, + workerEnv, +}); +``` + +This keeps Endform's policy close to Endform and avoids coupling the fork to Lambda Runner implementation details. + +## Target Architecture + +```text +Rust Lambda Runner + starts Node host early + continues dependency/scratch/proxy setup + sends structured run request over IPC + receives lifecycle/result/timing data + +Endform Node Host, owned by Lambda Runner repo + imports generic Playwright fork exports + owns Rust-to-Node IPC protocol + owns Endform config mutation policy + owns reporter pipeline generation + owns generated reporter files/modules + owns mapping Endform test attempts to Playwright structured test selection + +Playwright Fork + exposes generic runTests/loadConfig/prefork APIs + loads user configs without forcing immediate CLI execution + accepts an already-mutated config object + normalizes config into FullConfigInternal + runs selected tests through normal Playwright dispatcher/worker/reporter flow + supports preforked delayed-init workers +``` + +## Proposed Playwright Export + +Add a deliberate package export such as: + +```json +{ + "exports": { + "./programmatic-runner": { + "types": "./programmatic-runner.d.ts", + "import": "./programmatic-runner.mjs", + "require": "./programmatic-runner.js", + "default": "./programmatic-runner.js" + } + } +} +``` + +The exact export name can change. The important part is that consumers do not import `playwright/lib/runner`, `WorkerHost`, `Dispatcher`, or other private internals directly. + +The primary exported function can simply be named `runTests`: + +```ts +export async function runTests(params: RunTestsParams): Promise; +``` + +Supporting exports should be generic and minimal: + +```ts +export async function loadUserConfig(location: ConfigLocation): Promise; + +export async function createPreforkedWorkers(params: { + workers: number; +}): Promise; + +export async function disposePreforkedWorkers(workers: PreforkedWorkers): Promise; +``` + +If a class is more ergonomic than separate functions, expose a generic host-like object, but keep method names Playwright-shaped: + +```ts +const runner = await createRunnerHost({ workers: 1 }); +await runner.ready(); +await runner.runTests(params); +await runner.stop(); +``` + +Do not expose `WorkerHost` itself. + +## Proposed Generic `runTests` Parameters + +The run parameters should use Playwright concepts and avoid Endform-specific names: + +```ts +type RunTestsParams = { + configLocation: ConfigLocation; + config: Config; + ignoreProjectDependencies: boolean; + testSelection: StructuredTestSelection; + preforkedWorkers: PreforkedWorkers; + workerEnv: Record; +}; +``` + +Notes: + +- `config` is the raw user config object, possibly mutated by the caller before `runTests` receives it, and is the single source of truth for normal Playwright configuration such as workers, retries, timeouts, snapshot policy, metadata, and reporters. +- `ignoreProjectDependencies` is the programmatic equivalent of `--no-deps`. +- Reporters are owned by the caller through the mutated `config.reporter`; the programmatic runner should not expose a parallel reporter assembly path. +- `preforkedWorkers` should be an opaque handle, not an array of internal `WorkerHost` instances. + +## Structured Test Selection + +Endform should move away from generating a `--test-list` file. The Node host should send an in-memory structured test selection to Playwright. + +Use whichever shape best matches Playwright's existing test-list implementation and title filtering model. A likely shape is: + +```ts +type StructuredTestSelection = { + tests: StructuredSelectedTest[]; +}; + +type StructuredSelectedTest = { + projectName: string; + file: string; + titlePath: string[]; +}; +``` + +For Endform, `titlePath` should be the test's describe path plus case name: + +```ts +{ + projectName: request.projectName, + file: test.fileName, + titlePath: [...test.describes, test.caseName], +} +``` + +The Playwright implementation should internally convert this to the same filtering behavior as the existing test-list code: + +- Filter files to only the selected files. +- Filter tests by project and title path. +- Preserve behavior for duplicate titles as strictly as Playwright's current test-list behavior allows. +- Surface useful errors for selected tests that are not found. + +This should live as a generic in-memory sibling of the current `testList` file support, not as an Endform-specific selector. + +Potential implementation path: + +1. Keep existing `loadTestList` file parsing unchanged. +2. Add a new helper that builds the same `{ testFilter, fileFilter }` pair from structured entries. +3. Add `structuredTestSelection?: StructuredTestSelection` to `TestRunOptions`. +4. Apply it in `createLoadTask` at the same point as `testList`. +5. Thread it through the new exported `runTests` API. + +## Preforked Worker Support + +The current smoke implementation proved this internal flow: + +```ts +const worker = new WorkerHost(0); +await worker.prefork(); + +await testRunner.runTests(reporter, { + locations: [selectedTestFile], + projects: [selectedProjectName], + preforkedWorkers: [worker], + workerEnv, +}); +``` + +The maintained implementation should hide this behind an opaque prefork handle: + +```ts +const preforkedWorkers = await createPreforkedWorkers({ workers: 1 }); + +await runTests({ + configLocation, + config, + preforkedWorkers, + testSelection, + workerEnv, + ignoreProjectDependencies: true, +}); +``` + +Required internal Playwright changes remain: + +- Split process startup from runner initialization in `ProcessHost`. +- Allow `WorkerHost` to start without a `TestGroup` and initialize later. +- Allow `Dispatcher` to consume preforked delayed-init workers. +- Ensure preforked workers are leased safely across phases and not reused incompatibly. + +Avoid leaking these classes across the public export boundary. + +## Reporter Pipeline Ownership + +Reporter pipeline construction should stay in the Endform Node host. + +The Playwright fork should only need to support normal Playwright reporter declarations and optional direct reporter objects passed to `runTests`. Endform can continue to generate reporter modules on disk where that is the lowest-risk approach. + +The Endform Node host owns logic such as: + +- Preserve selected user reporters from the original user config. +- Drop unapproved user reporters. +- Inject a synchronous test-property reporter before blob reporter. +- Inject observer reporters that write events to the Rust-side reporter runtime transport. +- Inject `playwright-opentelemetry` with options when configured. +- Inject blob reporter for completion assets. + +This can be implemented by setting the mutated raw config's `reporter` field before calling Playwright: + +```ts +config.reporter = [ + [testPropertyReporterPath], + ...preservedUserReporters, + [testOutcomeObserverReporterPath], + [traceObserverReporterPath], + ...maybeOtelReporter, + ['blob'], +]; +``` + +The generated observer reporter transport can continue using file descriptor `3` initially. The Endform Node host, not the Playwright fork, should be responsible for opening/owning whatever transport is used between reporter JS and Rust or between reporter JS and the Node host. + +## Chrome Flags And Process Monkey Patching + +Do not include Chrome flag support in the Playwright fork API for this phase. + +Endform can continue using its current generated `NODE_OPTIONS` scripts to monkey patch: + +- `child_process.spawn` for Chromium launch flags. +- `child_process.fork` for propagation into Playwright worker processes. +- `worker_threads.Worker` for propagation into worker threads. + +This keeps the fork focused on runner orchestration and avoids opening another browser-launch-specific API surface before measurement proves it is needed. + +## Endform Node Host Responsibilities + +The Lambda Runner repository should add a Node host script/package that imports the Playwright fork export. + +Responsibilities: + +- Start immediately when Rust receives an invocation. +- Import Playwright and create preforked workers before scratch setup is complete. +- Expose a simple Rust-to-Node IPC protocol, likely over stdio or a Unix domain socket. +- Receive structured run parameters from Rust. +- Load the raw user Playwright config through the Playwright export. +- Mutate the config according to Endform policy. +- Generate any Endform reporter modules needed for the run. +- Build the final Playwright reporter list. +- Build structured test selection from Endform test attempt data. +- Call the Playwright fork's generic `runTests` export. +- Return status, timing data, stdout/stderr if captured, and any structured failure data Rust needs. +- Stop preforked workers and cleanup host resources on cancellation or invocation completion. + +This keeps rapidly changing Endform behavior in the Endform repository and keeps the fork as a lower-level runner API provider. + +## Rust Lambda Runner Changes + +The Rust controller should be reorganized so process execution can be backed either by the existing CLI path or by the new Node host path. + +Suggested phases: + +1. Start Node host early. +2. Run existing middleware setup. +3. Instead of collecting CLI args/config shim, collect structured run inputs for the Node host. +4. Send a `runTests` IPC request to the Node host. +5. Wait for completion/cancellation. +6. Run existing middleware teardown. +7. Keep the CLI backend as a fallback until parity is proven. + +The current `PlaywrightConfigMiddleware` should eventually split into more explicit responsibilities: + +- Lambda browser/runtime defaults. +- Endform config policy inputs. +- Test selection construction. +- Programmatic run options. +- Legacy CLI arg generation for fallback only. + +## Cancellation And Failure Handling + +The Node host should support cancellation explicitly: + +```text +Rust -> Node: cancel current run +Node -> Playwright: stop current run +Node -> Rust: cancelled/finished +``` + +The host must also handle: + +- Preforked worker exits before run starts. +- Worker crashes during initialization. +- Selected test not found. +- Config load errors. +- Reporter startup errors. +- Observer transport failure. +- Rust disconnecting or killing the invocation. + +For safety, the first implementation can fall back to starting a fresh worker when a preforked worker is unavailable or incompatible. + +## Measurement + +Add timings at the Playwright host level and return them to Rust: + +- host process start to ready +- Playwright import complete +- worker prefork start to ready +- run request received +- user config load +- Endform config mutation time, measured in the Endform Node host +- config normalization +- test collection/filtering +- runner-side test file load +- worker initialization +- worker-side test file load +- test execution +- reporter finalization +- total run time + +Only after these timings are available should we decide whether to optimize discovery, avoid double test-file load, or prewarm browsers. + +## Implementation Order + +1. Add the new Playwright package export with no Endform-specific types. +2. Export `loadUserConfig` or equivalent raw config loading helper. +3. Export generic `runTests({ configLocation, config, ... })` that can run from an already-loaded raw config object. +4. Add structured in-memory test selection and wire it into the same filtering phase as current test-list support. +5. Hide prefork support behind an opaque `PreforkedWorkers` handle. +6. Update the existing smoke test to consume the new export instead of importing `playwright/lib/runner` internals. +7. Add Playwright-side tests for config-object execution, structured test selection, preforked worker execution, late worker env, reporter lifecycle, attachments, stdout/stderr attribution, and selected-test-not-found behavior. +8. Add the Endform Node host in the Lambda Runner repository. +9. Implement config mutation and reporter pipeline construction inside the Endform Node host. +10. Add a Rust execution backend flag to switch between legacy CLI and new host backend. +11. Add parity tests comparing legacy CLI/shim output with the new host backend. +12. Enable the new backend gradually and retain CLI fallback until production metrics are stable. + +## Maintained Fork Strategy + +Keep the fork patch set focused on generic seams: + +- Programmatic `runTests` from a raw config object. +- Structured in-memory test selection. +- Process fork/init split. +- Delayed worker initialization. +- Opaque preforked worker pool consumed by dispatcher. + +Avoid placing these in the fork: + +- Endform config mutation policy. +- Endform reporter pipeline semantics. +- Rust-to-Node IPC protocol. +- Suite-control reporting details. +- Completion asset and trace upload details. +- Chrome flag policy. + +Maintain one fork branch per supported Playwright version line, with this file as the intended architecture reference. Each branch should carry the smallest possible version-specific adaptation of the same generic public export. + +## Success Criteria + +- Lambda Runner can start a Node host and prefork a worker before dependency setup completes. +- Endform can run the selected tests without invoking the Playwright CLI. +- Endform can run without modifying the user's Playwright config file on disk. +- Endform can select tests through structured in-memory data, not a test-list file. +- Existing reporter lifecycle remains intact: `onConfigure`, `onBegin`, `onTestBegin`, step hooks, stdio attribution, attachments, `onTestEnd`, `onError`, `onEnd`, and `onExit`. +- Endform-specific config and reporter behavior can evolve in the Lambda Runner repository without changing the Playwright fork API. +- The Playwright fork surface is small enough to rebase across Playwright versions with predictable conflicts. diff --git a/packages/playwright/package.json b/packages/playwright/package.json index fe19790cdaee9..8f058e43da7c5 100644 --- a/packages/playwright/package.json +++ b/packages/playwright/package.json @@ -19,6 +19,12 @@ "default": "./index.js" }, "./package.json": "./package.json", + "./programmatic-runner": { + "types": "./programmatic-runner.d.ts", + "import": "./programmatic-runner.mjs", + "require": "./programmatic-runner.js", + "default": "./programmatic-runner.js" + }, "./lib/common": "./lib/common/index.js", "./lib/fsWatcher": "./lib/fsWatcher.js", "./lib/mcp/index": "./lib/mcp/index.js", diff --git a/packages/playwright/programmatic-runner.d.ts b/packages/playwright/programmatic-runner.d.ts new file mode 100644 index 0000000000000..bc359db09d4d6 --- /dev/null +++ b/packages/playwright/programmatic-runner.d.ts @@ -0,0 +1,55 @@ +/** + * Copyright (c) Microsoft Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import type { Config } from './types/test'; +import type { FullResult } from './types/testReporter'; + +export type ConfigLocation = string | { + resolvedConfigFile?: string; + configDir: string; +}; + +export type StructuredTestSelection = { + tests: StructuredSelectedTest[]; +}; + +export type StructuredSelectedTest = { + projectName: string; + file: string; + titlePath: string[]; +}; + +export type PreforkedWorkers = { + readonly __brand: unique symbol; +}; + +export type RunTestsParams = { + configLocation: ConfigLocation; + config: Config; + ignoreProjectDependencies: boolean; + testSelection: StructuredTestSelection; + preforkedWorkers: PreforkedWorkers; + workerEnv: Record; +}; + +export type RunTestsResult = { + status: FullResult['status']; +}; + +export function loadUserConfig(location: ConfigLocation): Promise; +export function runTests(params: RunTestsParams): Promise; +export function createPreforkedWorkers(params: { workers: number }): Promise; +export function disposePreforkedWorkers(workers: PreforkedWorkers): Promise; diff --git a/packages/playwright/programmatic-runner.js b/packages/playwright/programmatic-runner.js new file mode 100644 index 0000000000000..ec62430d35eda --- /dev/null +++ b/packages/playwright/programmatic-runner.js @@ -0,0 +1,17 @@ +/** + * Copyright (c) Microsoft Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +module.exports = require('./lib/programmaticRunner'); diff --git a/packages/playwright/programmatic-runner.mjs b/packages/playwright/programmatic-runner.mjs new file mode 100644 index 0000000000000..40f0cbf416943 --- /dev/null +++ b/packages/playwright/programmatic-runner.mjs @@ -0,0 +1,22 @@ +/** + * Copyright (c) Microsoft Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import programmaticRunner from './programmatic-runner.js'; + +export const loadUserConfig = programmaticRunner.loadUserConfig; +export const runTests = programmaticRunner.runTests; +export const createPreforkedWorkers = programmaticRunner.createPreforkedWorkers; +export const disposePreforkedWorkers = programmaticRunner.disposePreforkedWorkers; diff --git a/packages/playwright/src/DEPS.list b/packages/playwright/src/DEPS.list index a67d9689d0619..be6b095535bbf 100644 --- a/packages/playwright/src/DEPS.list +++ b/packages/playwright/src/DEPS.list @@ -10,6 +10,9 @@ node_modules/minimatch [program.ts] ** +[programmaticRunner.ts] +** + [testActions.ts] ** diff --git a/packages/playwright/src/common/configLoader.ts b/packages/playwright/src/common/configLoader.ts index 0af93470ee5aa..d4cd9a5262e94 100644 --- a/packages/playwright/src/common/configLoader.ts +++ b/packages/playwright/src/common/configLoader.ts @@ -92,7 +92,8 @@ export async function deserializeConfig(data: SerializedConfig): Promise { +export async function loadUserConfig(location: ConfigLocation, overrides?: ConfigCLIOverrides): Promise { + await setSingleTSConfig(overrides?.tsconfig); let object = location.resolvedConfigFile ? await requireOrImport(location.resolvedConfigFile) : {}; if (object && typeof object === 'object' && ('default' in object)) object = object['default']; @@ -105,6 +106,10 @@ export async function loadConfig(location: ConfigLocation, overrides?: ConfigCLI // 2. Load and validate playwright config. const userConfig = await loadUserConfig(location); + return await loadConfigFromObject(location, userConfig, overrides, ignoreProjectDependencies, metadata); +} + +export async function loadConfigFromObject(location: ConfigLocation, userConfig: Config, overrides?: ConfigCLIOverrides, ignoreProjectDependencies = false, metadata?: Config['metadata']): Promise { validateConfig(location.resolvedConfigFile || '', userConfig); const fullConfig = new FullConfigInternal(location, userConfig, overrides || {}, metadata); fullConfig.defineConfigWasUsed = !!(userConfig as any)[kDefineConfigWasUsed]; diff --git a/packages/playwright/src/common/ipc.ts b/packages/playwright/src/common/ipc.ts index 3fdaeccfcb6d5..6f6ac2c38d8dd 100644 --- a/packages/playwright/src/common/ipc.ts +++ b/packages/playwright/src/common/ipc.ts @@ -66,6 +66,7 @@ export type WorkerInitParams = { projectId: string; config: SerializedConfig; artifactsDir: string; + extraEnv?: Record; pauseOnError: boolean; pauseAtEnd: boolean; }; diff --git a/packages/playwright/src/programmaticRunner.ts b/packages/playwright/src/programmaticRunner.ts new file mode 100644 index 0000000000000..bdb772e5636f9 --- /dev/null +++ b/packages/playwright/src/programmaticRunner.ts @@ -0,0 +1,107 @@ +/** + * Copyright (c) Microsoft Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { configLoader } from './common'; +import { testRunner, workerHost } from './runner'; + +import type { ConfigLocation } from './common'; +import type { StructuredTestSelection } from './runner/loadUtils'; +import type { Config } from '../types/test'; +import type { FullResult } from '../types/testReporter'; + +type ConfigLocationInput = string | ConfigLocation; + +type RunTestsParams = { + configLocation: ConfigLocationInput; + config: Config; + ignoreProjectDependencies: boolean; + testSelection: StructuredTestSelection; + preforkedWorkers: PreforkedWorkers; + workerEnv: Record; +}; + +type RunTestsResult = { + status: FullResult['status']; +}; + +export class PreforkedWorkers { + readonly workers: workerHost.WorkerHost[]; + + constructor(workers: workerHost.WorkerHost[]) { + this.workers = workers; + } +} + +export async function loadUserConfig(location: ConfigLocationInput): Promise { + return await configLoader.loadUserConfig(resolveLocation(location)); +} + +export async function runTests(params: RunTestsParams): Promise { + const location = resolveLocation(params.configLocation); + const config = await configLoader.loadConfigFromObject(location, params.config, {}, params.ignoreProjectDependencies); + validateTestSelection(params.testSelection); + const status = await testRunner.runAllTestsWithConfig(config, { + projectFilter: projectFilterFromSelection(params.testSelection), + testSelection: params.testSelection, + preforkedWorkers: params.preforkedWorkers.workers, + workerEnv: params.workerEnv, + }); + return { status }; +} + +function validateTestSelection(selection: StructuredTestSelection) { + if (!selection || !selection.tests.length) + throw new Error('Programmatic runner requires at least one selected test'); + for (const test of selection.tests) { + if (test.projectName === undefined) + throw new Error('Programmatic runner selected test must specify projectName'); + if (!test.file) + throw new Error('Programmatic runner selected test must specify file'); + if (!test.titlePath?.length) + throw new Error('Programmatic runner selected test must specify non-empty titlePath'); + } +} + +function projectFilterFromSelection(selection: StructuredTestSelection): string[] { + return [...new Set(selection.tests.map(test => test.projectName))]; +} + +export async function createPreforkedWorkers(params: { workers: number }): Promise { + const workers: workerHost.WorkerHost[] = []; + try { + for (let i = 0; i < params.workers; i++) { + const worker = new workerHost.WorkerHost(i); + workers.push(worker); + const error = await worker.prefork(); + if (error) + throw new Error(`Worker process exited before it was ready (code=${error.code}, signal=${error.signal})`); + } + } catch (e) { + await Promise.all(workers.map(worker => worker.stop().catch(() => {}))); + throw e; + } + return new PreforkedWorkers(workers); +} + +export async function disposePreforkedWorkers(workers: PreforkedWorkers): Promise { + await Promise.all(workers.workers.map(worker => worker.stop().catch(() => {}))); +} + +function resolveLocation(location: ConfigLocationInput): ConfigLocation { + if (typeof location === 'string') + return configLoader.resolveConfigLocation(location); + return location; +} diff --git a/packages/playwright/src/runner/dispatcher.ts b/packages/playwright/src/runner/dispatcher.ts index 29f4f4e84eb1a..10e98fbfc9e20 100644 --- a/packages/playwright/src/runner/dispatcher.ts +++ b/packages/playwright/src/runner/dispatcher.ts @@ -46,8 +46,9 @@ export class Dispatcher { private _extraEnvByProjectId: EnvByProjectId = new Map(); private _producedEnvByProjectId: EnvByProjectId = new Map(); - constructor(testRun: TestRun) { + constructor(testRun: TestRun, preforkedWorkers: WorkerHost[] = []) { this._testRun = testRun; + this._workerSlots = preforkedWorkers.map(worker => ({ worker })); for (const project of testRun.config.projects) { if (project.workers) this._workerLimitPerProjectId.set(project.id, project.workers); @@ -117,7 +118,7 @@ export class Dispatcher { let worker = this._workerSlots[index].worker; // 1. Restart the worker if it has the wrong hash or is being stopped already. - if (worker && (worker.hash() !== job.workerHash || worker.didSendStop())) { + if (worker && (worker.didSendStop() || (worker.isInitialized() && worker.hash() !== job.workerHash))) { await worker.stop(); worker = undefined; if (this._isStopped) // Check stopped signal after async hop. @@ -133,6 +134,12 @@ export class Dispatcher { startError = await worker.start(); if (this._isStopped) // Check stopped signal after async hop. return; + } else if (!worker.isInitialized()) { + this._initializeWorker(worker, job, index, ipc.serializeConfig(this._testRun.config, true)); + worker.on('exit', () => this._workerSlots[index].worker = undefined); + startError = await worker.start(); + if (this._isStopped) // Check stopped signal after async hop. + return; } // 3. Finally, run some tests in the worker! Or fail all of them because of startup error... @@ -173,12 +180,15 @@ export class Dispatcher { } private _isWorkerRedundant(worker: WorkerHost) { + const hash = worker.hash(); + if (!hash) + return false; let workersWithSameHash = 0; for (const slot of this._workerSlots) { - if (slot.worker && !slot.worker.didSendStop() && slot.worker.hash() === worker.hash()) + if (slot.worker && !slot.worker.didSendStop() && slot.worker.hash() === hash) workersWithSameHash++; } - return workersWithSameHash > this._queuedOrRunningHashCount.get(worker.hash())!; + return workersWithSameHash > this._queuedOrRunningHashCount.get(hash)!; } private _updateCounterForWorkerHash(hash: string, delta: number) { @@ -191,12 +201,15 @@ export class Dispatcher { for (const group of testGroups) this._updateCounterForWorkerHash(group.workerHash, +1); this._isStopped = false; - this._workerSlots = []; + const preforkedSlots = this._workerSlots + .filter(slot => slot.worker && !slot.worker.isInitialized() && !slot.worker.didSendStop()) + .slice(0, this._testRun.config.config.workers); + this._workerSlots = preforkedSlots; // 0. Stop right away if we have reached max failures. if (this._testRun.hasReachedMaxFailures()) void this.stop(); // 1. Allocate workers. - for (let i = 0; i < this._testRun.config.config.workers; i++) + for (let i = this._workerSlots.length; i < this._testRun.config.config.workers; i++) this._workerSlots.push({}); // 2. Schedule enough jobs. for (let i = 0; i < this._workerSlots.length; i++) @@ -207,13 +220,14 @@ export class Dispatcher { await this._finished; } - _createWorker(testGroup: TestGroup, parallelIndex: number, loaderData: ipc.SerializedConfig) { + _initializeWorker(worker: WorkerHost, testGroup: TestGroup, parallelIndex: number, loaderData: ipc.SerializedConfig) { const project = this._testRun.config.projects.find(p => p.id === testGroup.projectId)!; const pauseAtEnd = this._testRun.topLevelProjects.includes(project) && !!this._testRun.options.pauseAtEnd; - const worker = new WorkerHost(testGroup, { + worker.initialize({ + testGroup, parallelIndex, config: loaderData, - extraEnv: this._extraEnvByProjectId.get(testGroup.projectId) || {}, + extraEnv: { ...this._testRun.options.workerEnv, ...this._extraEnvByProjectId.get(testGroup.projectId) }, outputDir: project.project.outputDir, pauseOnError: !!this._testRun.options.pauseOnError, pauseAtEnd, @@ -260,6 +274,11 @@ export class Dispatcher { const producedEnv = this._producedEnvByProjectId.get(testGroup.projectId) || {}; this._producedEnvByProjectId.set(testGroup.projectId, { ...producedEnv, ...worker.producedEnv() }); }); + } + + _createWorker(testGroup: TestGroup, parallelIndex: number, loaderData: ipc.SerializedConfig) { + const worker = new WorkerHost(parallelIndex); + this._initializeWorker(worker, testGroup, parallelIndex, loaderData); return worker; } diff --git a/packages/playwright/src/runner/index.ts b/packages/playwright/src/runner/index.ts index deca5ea33718e..9881886fba753 100644 --- a/packages/playwright/src/runner/index.ts +++ b/packages/playwright/src/runner/index.ts @@ -20,6 +20,7 @@ // imports. export * as testRunner from './testRunner'; +export * as workerHost from './workerHost'; export * as testServer from './testServer'; export * as watchMode from './watchMode'; export * as projectUtils from './projectUtils'; diff --git a/packages/playwright/src/runner/loadUtils.ts b/packages/playwright/src/runner/loadUtils.ts index a2cce3b386459..be0fd95da7637 100644 --- a/packages/playwright/src/runner/loadUtils.ts +++ b/packages/playwright/src/runner/loadUtils.ts @@ -32,6 +32,22 @@ import type { TestGroup } from './testGroups'; import type { FullConfig, Reporter, TestError } from '../../types/testReporter'; import type { Matcher, TestCaseFilter } from '../util'; +export type StructuredTestSelection = { + tests: StructuredSelectedTest[]; +}; + +export type StructuredSelectedTest = { + projectName: string; + file: string; + titlePath: string[]; +}; + +type TestDescription = { + project?: string; + file: string; + titlePath: string[]; +}; + export async function collectProjectsAndTestFiles(testRun: TestRun, doNotRunTestsOutsideProjectFilter: boolean) { const fsCache = new Map(); @@ -347,22 +363,67 @@ export async function loadTestList(config: FullConfigInternal, filePath: string) } return { project, file: toPosixPath(parseLocationArg(tokens[0]).file), titlePath: tokens.slice(1) }; }); - const testFilter = (test: testNs.TestCase) => descriptions.some(d => { - // Note: there is no root yet at the time of filtering. - const [projectName, , ...titles] = test.titlePath(); - if (d.project !== undefined && d.project !== projectName) - return false; - const relativeFile = toPosixPath(path.relative(config.config.rootDir, test.location.file)); - if (relativeFile !== d.file) - return false; - return d.titlePath.length <= titles.length && d.titlePath.every((_, index) => titles[index] === d.titlePath[index]); - }); - const fileFilter = (file: string) => { - const relativeFile = toPosixPath(path.relative(config.config.rootDir, file)); - return descriptions.some(d => d.file === relativeFile); - }; - return { testFilter, fileFilter }; + return createFilters(config, descriptions).filters; } catch (e) { throw errorWithFile(filePath, 'Cannot read test list file: ' + e.message); } } + +export function createStructuredTestSelectionFilters(config: FullConfigInternal, selection: StructuredTestSelection): { testFilter: TestCaseFilter, fileFilter: Matcher, unmatchedErrors: () => TestError[] } { + const descriptions = selection.tests.map(test => ({ + project: test.projectName, + file: normalizeSelectedFile(config, test.file), + titlePath: test.titlePath, + })); + const { filters, unmatchedErrors } = createFilters(config, descriptions); + return { ...filters, unmatchedErrors }; +} + +function createFilters(config: FullConfigInternal, descriptions: TestDescription[]): { filters: { testFilter: TestCaseFilter, fileFilter: Matcher }, unmatchedErrors: () => TestError[] } { + const matched = new Array(descriptions.length).fill(false); + const testFilter = (test: testNs.TestCase) => descriptions.some((d, index) => { + // Note: there is no root yet at the time of filtering. + const [projectName, , ...titles] = test.titlePath(); + if (d.project !== undefined && d.project !== projectName) + return false; + if (!matchesSelectedFile(config, test.location.file, d.file)) + return false; + const result = d.titlePath.length <= titles.length && d.titlePath.every((_, index) => titles[index] === d.titlePath[index]); + matched[index] ||= result; + return result; + }); + const fileFilter = (file: string) => { + return descriptions.some(d => matchesSelectedFile(config, file, d.file)); + }; + const unmatchedErrors = () => descriptions.filter((_, index) => !matched[index]).map(d => ({ message: `Error: selected test not found: ${formatTestDescription(d)}` })); + return { filters: { testFilter, fileFilter }, unmatchedErrors }; +} + +function normalizeSelectedFile(config: FullConfigInternal, file: string): string { + if (path.isAbsolute(file)) + return toPosixPath(path.relative(config.config.rootDir, file)); + return toPosixPath(parseLocationArg(file).file); +} + +function matchesSelectedFile(config: FullConfigInternal, file: string, selectedFile: string): boolean { + const relativeFile = toPosixPath(path.relative(config.config.rootDir, file)); + if (relativeFile === selectedFile) + return true; + const absoluteFile = path.resolve(file); + const absoluteSelectedFile = path.resolve(config.config.rootDir, selectedFile); + return toPosixPath(absoluteFile) === toPosixPath(absoluteSelectedFile) || toPosixPath(realpath(absoluteFile)) === toPosixPath(realpath(absoluteSelectedFile)); +} + +function realpath(file: string): string { + try { + return fs.realpathSync(file); + } catch { + return file; + } +} + +function formatTestDescription(description: TestDescription): string { + const tokens = [description.file, ...description.titlePath]; + const prefix = description.project === undefined ? '' : `[${description.project}] › `; + return prefix + tokens.join(' › '); +} diff --git a/packages/playwright/src/runner/processHost.ts b/packages/playwright/src/runner/processHost.ts index 5c0257933f8a8..e3d00ea537e57 100644 --- a/packages/playwright/src/runner/processHost.ts +++ b/packages/playwright/src/runner/processHost.ts @@ -32,6 +32,7 @@ export type ProcessExitData = { export class ProcessHost extends EventEmitter { private process: child_process.ChildProcess | undefined; + private _didSendInit = false; private _didSendStop = false; private _processDidExit = false; private _didExitAndRanOnExit = false; @@ -50,7 +51,7 @@ export class ProcessHost extends EventEmitter { this._extraEnv = env; } - async startRunner(runnerParams: any, options: { onStdOut?: (chunk: Buffer | string) => void, onStdErr?: (chunk: Buffer | string) => void } = {}): Promise { + async preforkRunner(options: { onStdOut?: (chunk: Buffer | string) => void, onStdErr?: (chunk: Buffer | string) => void } = {}): Promise { assert(!this.process, 'Internal error: starting the same process twice'); this.process = child_process.fork(this._entryScript, { // Note: we pass detached:false, so that workers are in the same process group. @@ -125,6 +126,12 @@ export class ProcessHost extends EventEmitter { if (error) return error; + } + + initRunner(runnerParams: any) { + assert(this.process, 'Internal error: initializing a process before it starts'); + assert(!this._didSendInit, 'Internal error: initializing the same process twice'); + this._didSendInit = true; const processParams: ipc.ProcessInitParams = { processName: this._processName, @@ -139,6 +146,17 @@ export class ProcessHost extends EventEmitter { }); } + async startRunner(runnerParams: any, options: { onStdOut?: (chunk: Buffer | string) => void, onStdErr?: (chunk: Buffer | string) => void } = {}): Promise { + const error = await this.preforkRunner(options); + if (error) + return error; + this.initRunner(runnerParams); + } + + hasProcess() { + return !!this.process; + } + sendMessage(message: { method: string, params?: any }) { const id = ++this._lastMessageId; this.send({ diff --git a/packages/playwright/src/runner/tasks.ts b/packages/playwright/src/runner/tasks.ts index 50b6951177ffd..d93a6ee195e23 100644 --- a/packages/playwright/src/runner/tasks.ts +++ b/packages/playwright/src/runner/tasks.ts @@ -24,7 +24,7 @@ import { monotonicTime } from '@isomorphic/time'; import { removeFolders } from '@utils/fileUtils'; import { Dispatcher } from './dispatcher'; -import { collectProjectsAndTestFiles, createRootSuite, loadFileSuites, loadGlobalHook, loadTestList } from './loadUtils'; +import { collectProjectsAndTestFiles, createRootSuite, createStructuredTestSelectionFilters, loadFileSuites, loadGlobalHook, loadTestList } from './loadUtils'; import { buildDependentProjects, buildProjectsClosure, buildTeardownToSetupsMap, filterProjects } from './projectUtils'; import { applySuggestedRebaselines, clearSuggestedRebaselines } from './rebase'; import { TaskRunner } from './taskRunner'; @@ -35,12 +35,15 @@ import { createTitleMatcher, forceRegExp, removeDirAndLogToConsole } from '../ut import type { TestGroup } from '../runner/testGroups'; import type { EnvByProjectId } from './dispatcher'; +import type { StructuredTestSelection } from './loadUtils'; +import type { WorkerHost } from './workerHost'; import type { TestRunnerPluginRegistration } from '../plugins'; import type { Task } from './taskRunner'; import type { ReporterDescription } from '../../types/test'; import type { FullResult, TestError } from '../../types/testReporter'; import type { Matcher, TestCaseFilter } from '../util'; import type { InternalReporter } from '../reporters/internalReporter'; +import type { AnyReporter } from '../reporters/reporterV2'; const readDirAsync = promisify(fs.readdir); @@ -67,13 +70,18 @@ export type TestRunOptions = { lastFailedFile?: string; testList?: string; testListInvert?: string; + testSelection?: StructuredTestSelection; lastFailedTestIds?: string[]; pauseOnError?: boolean; pauseAtEnd?: boolean; onTestPaused?: (params: TestPausedParams) => void; preserveOutputDir?: boolean; additionalReporters?: ReporterDescription[]; + additionalReporterObjects?: AnyReporter[]; + disableConfigReporters?: boolean; shardWeights?: number[]; + preforkedWorkers?: WorkerHost[]; + workerEnv?: Record; }; export type TestPausedParams = { @@ -310,12 +318,20 @@ export function createLoadTask(mode: 'out-of-process' | 'in-process', options: { return { title: 'load tests', setup: async (testRun, errors, softErrors) => { + let unmatchedSelectionErrors: (() => TestError[]) | undefined; if (testRun.options.locations?.length) { const { testFilter, fileFilter } = suiteUtils.createFiltersFromArguments(testRun.options.locations); testRun.loadFileFilters.push(fileFilter); testRun.preOnlyTestFilters.push(testFilter); } + if (testRun.options.testSelection) { + const { testFilter, fileFilter, unmatchedErrors } = createStructuredTestSelectionFilters(testRun.config, testRun.options.testSelection); + testRun.preOnlyTestFilters.push(testFilter); + testRun.loadFileFilters.push(fileFilter); + unmatchedSelectionErrors = unmatchedErrors; + } + if (testRun.options.testList) { const { testFilter, fileFilter } = await loadTestList(testRun.config, testRun.options.testList); testRun.preOnlyTestFilters.push(testFilter); @@ -359,11 +375,13 @@ export function createLoadTask(mode: 'out-of-process' | 'in-process', options: { } await createRootSuite(testRun, options.failOnLoadErrors ? errors : softErrors, !!options.filterOnly); + if (unmatchedSelectionErrors) + (options.failOnLoadErrors ? errors : softErrors).push(...unmatchedSelectionErrors()); // Fail when no tests. if (options.failOnLoadErrors && !testRun.rootSuite?.allTests().length && !testRun.options.passWithNoTests && !testRun.config.config.shard && !testRun.options.onlyChanged - && !testRun.options.testList && !testRun.options.testListInvert) { + && !testRun.options.testList && !testRun.options.testListInvert && !testRun.options.testSelection) { if (testRun.options.locations?.length) { throw new Error([ `No tests found.`, @@ -423,7 +441,7 @@ function createPhasesTask(): Task { processed.add(project); if (phaseProjects.length) { let testGroupsInPhase = 0; - const phase: Phase = { dispatcher: new Dispatcher(testRun), projects: [] }; + const phase: Phase = { dispatcher: new Dispatcher(testRun, testRun.options.preforkedWorkers), projects: [] }; testRun.phases.push(phase); for (const project of phaseProjects) { const projectSuite = projectToSuite.get(project)!; diff --git a/packages/playwright/src/runner/testRunner.ts b/packages/playwright/src/runner/testRunner.ts index 3f1f250dee0e6..4db7127db4a27 100644 --- a/packages/playwright/src/runner/testRunner.ts +++ b/packages/playwright/src/runner/testRunner.ts @@ -445,7 +445,7 @@ export async function runAllTestsWithConfig(config: FullConfigInternal, options: config.plugins.push(...webServerPluginsForConfig(config)); const filteredProjects = filterProjects(config.projects, options.projectFilter); - const reporters = await createReporters(config, options.listMode ? 'list' : 'test', undefined, options); + const reporters = options.disableConfigReporters ? [] : await createReporters(config, options.listMode ? 'list' : 'test', undefined, options); const lastRun = new LastRunReporter(filteredProjects, options.listMode, options.lastFailedFile); if (options.lastFailed) { const lastFailedTestIds = await lastRun.filterLastFailed(); @@ -453,7 +453,7 @@ export async function runAllTestsWithConfig(config: FullConfigInternal, options: options = { ...options, lastFailedTestIds }; } - const reporter = new InternalReporter([...reporters, lastRun]); + const reporter = new InternalReporter([...reporters, ...(options.additionalReporterObjects || []), lastRun]); const tasks = options.listMode ? [ createLoadTask('in-process', { failOnLoadErrors: true, filterOnly: false }), createReportBeginTask(), diff --git a/packages/playwright/src/runner/workerHost.ts b/packages/playwright/src/runner/workerHost.ts index 2e1781c2d0762..3b9d6236a7f2a 100644 --- a/packages/playwright/src/runner/workerHost.ts +++ b/packages/playwright/src/runner/workerHost.ts @@ -17,6 +17,7 @@ import fs from 'fs'; import path from 'path'; +import { assert } from '@isomorphic/assert'; import { removeFolders } from '@utils/fileUtils'; import { ProcessHost } from './processHost'; @@ -37,46 +38,81 @@ type WorkerHostOptions = { pauseAtEnd: boolean; }; +type WorkerHostInitOptions = WorkerHostOptions & { + testGroup: TestGroup; +}; + export class WorkerHost extends ProcessHost { - readonly parallelIndex: number; readonly workerIndex: number; - private _hash: string; - private _params: ipc.WorkerInitParams; + private _parallelIndex: number; + private _hash: string | undefined; + private _params: ipc.WorkerInitParams | undefined; private _didFail = false; - constructor(testGroup: TestGroup, options: WorkerHostOptions) { + constructor(testGroup: TestGroup, options: WorkerHostOptions); + constructor(parallelIndex: number); + constructor(testGroupOrParallelIndex: TestGroup | number, options?: WorkerHostOptions) { const workerIndex = lastWorkerIndex++; + const extraEnv = options?.extraEnv || {}; super(require.resolve('../worker/workerProcessEntry.js'), `worker-${workerIndex}`, { - ...options.extraEnv, + ...extraEnv, FORCE_COLOR: '1', DEBUG_COLORS: process.env.DEBUG_COLORS === undefined ? '1' : process.env.DEBUG_COLORS, }); this.workerIndex = workerIndex; - this.parallelIndex = options.parallelIndex; - this._hash = testGroup.workerHash; + this._parallelIndex = typeof testGroupOrParallelIndex === 'number' ? testGroupOrParallelIndex : options!.parallelIndex; + + if (typeof testGroupOrParallelIndex !== 'number') + this.initialize({ ...options!, testGroup: testGroupOrParallelIndex }); + } + + get parallelIndex() { + return this._parallelIndex; + } + + initialize(options: WorkerHostInitOptions) { + if (this._params) + return; + + this._parallelIndex = options.parallelIndex; + this._hash = options.testGroup.workerHash; this._params = { workerIndex: this.workerIndex, parallelIndex: options.parallelIndex, - repeatEachIndex: testGroup.repeatEachIndex, - projectId: testGroup.projectId, + repeatEachIndex: options.testGroup.repeatEachIndex, + projectId: options.testGroup.projectId, config: options.config, - artifactsDir: path.join(options.outputDir, artifactsFolderName(workerIndex)), + artifactsDir: path.join(options.outputDir, artifactsFolderName(this.workerIndex)), + extraEnv: options.extraEnv, pauseOnError: options.pauseOnError, pauseAtEnd: options.pauseAtEnd, }; } async start() { + assert(this._params, 'Internal error: starting a worker before it is initialized'); await fs.promises.mkdir(this._params.artifactsDir, { recursive: true }); + if (this.hasProcess()) { + this.initRunner(this._params); + return; + } return await this.startRunner(this._params, { onStdOut: chunk => this.emit('stdOut', ipc.stdioChunkToParams(chunk)), onStdErr: chunk => this.emit('stdErr', ipc.stdioChunkToParams(chunk)), }); } + async prefork() { + return await this.preforkRunner({ + onStdOut: chunk => this.emit('stdOut', ipc.stdioChunkToParams(chunk)), + onStdErr: chunk => this.emit('stdErr', ipc.stdioChunkToParams(chunk)), + }); + } + override async onExit() { - await removeFolders([this._params.artifactsDir]); + if (this._params) + await removeFolders([this._params.artifactsDir]); } override async stop(didFail?: boolean) { @@ -101,8 +137,12 @@ export class WorkerHost extends ProcessHost { return this._hash; } + isInitialized() { + return !!this._params; + } + projectId() { - return this._params.projectId; + return this._params?.projectId; } didFail() { diff --git a/packages/playwright/src/worker/workerMain.ts b/packages/playwright/src/worker/workerMain.ts index 22e8f46451639..409f94c2264c8 100644 --- a/packages/playwright/src/worker/workerMain.ts +++ b/packages/playwright/src/worker/workerMain.ts @@ -64,6 +64,12 @@ export class WorkerMain extends ProcessRunner { constructor(params: ipc.WorkerInitParams) { super(); + for (const [key, value] of Object.entries(params.extraEnv || {})) { + if (value === undefined) + delete process.env[key]; + else + process.env[key] = value; + } process.env.TEST_WORKER_INDEX = String(params.workerIndex); process.env.TEST_PARALLEL_INDEX = String(params.parallelIndex); globals.setIsWorkerProcess(); diff --git a/tests/playwright-test/programmatic-runner.spec.ts b/tests/playwright-test/programmatic-runner.spec.ts new file mode 100644 index 0000000000000..0b95924db8943 --- /dev/null +++ b/tests/playwright-test/programmatic-runner.spec.ts @@ -0,0 +1,31 @@ +/** + * Copyright Microsoft Corporation. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import path from 'path'; + +import { test, expect } from './playwright-test-fixtures'; + +test('programmatic runner public export can run selected test with preforked worker', async ({ childProcess }, testInfo) => { + const repoRoot = path.resolve(__dirname, '..', '..'); + const script = path.join(repoRoot, 'utils', 'endform', 'programmatic-runner-smoke.js'); + const proc = childProcess({ + command: ['node', script, '--playwright-root', repoRoot], + cwd: testInfo.outputPath(), + }); + const { exitCode } = await proc.exited; + expect(proc.output).toContain('PROGRAMMATIC_RUNNER_SMOKE_OK'); + expect(exitCode).toBe(0); +}); diff --git a/utils/endform/README.md b/utils/endform/README.md new file mode 100644 index 0000000000000..1841c6e5771fe --- /dev/null +++ b/utils/endform/README.md @@ -0,0 +1,159 @@ +# Endform Playwright Packages + +This directory contains Endform-only tooling for publishing this Playwright fork under scoped npm package names while preserving canonical Playwright module names at runtime. + +## Published Packages + +The fork is published as three scoped packages: + +| Package | Source package | Purpose | +| --- | --- | --- | +| `@endform/playwright-core` | `packages/playwright-core` | Browser automation engine | +| `@endform/playwright` | `packages/playwright` | Public Playwright package plus Endform's `playwright/programmatic-runner` export | +| `@endform/playwright-test` | `packages/playwright-test` | Wrapper equivalent of upstream `@playwright/test` | + +The source workspace package names stay unchanged: `playwright-core`, `playwright`, and `@playwright/test`. This keeps normal Playwright build tooling, tests, generated files, and internal imports working without a fork-wide rename. + +## Why Runtime Names Stay Canonical + +Playwright packages intentionally import each other by canonical package name. Examples: + +```js +require('playwright-core') +require('playwright/test') +``` + +The Endform packages therefore use npm alias dependencies in their staged `package.json` files: + +```json +{ + "name": "@endform/playwright", + "dependencies": { + "playwright-core": "npm:@endform/playwright-core@1.60.0-beta.1" + } +} +``` + +```json +{ + "name": "@endform/playwright-test", + "dependencies": { + "playwright": "npm:@endform/playwright@1.60.0-beta.1" + } +} +``` + +This means code can keep using canonical imports when a project installs the fork through aliases: + +```json +{ + "dependencies": { + "playwright-core": "npm:@endform/playwright-core@1.60.0-beta.1", + "playwright": "npm:@endform/playwright@1.60.0-beta.1", + "@playwright/test": "npm:@endform/playwright-test@1.60.0-beta.1" + } +} +``` + +With that install shape, these imports all resolve to the Endform fork: + +```js +require('playwright') +require('playwright/test') +require('playwright/programmatic-runner') +require('@playwright/test') +``` + +## LambdaRunner Symlink Strategy + +LambdaRunner can install both upstream Playwright and Endform Playwright into isolated roots in the Docker image: + +```text +/opt/playwright-upstream/node_modules/ + playwright + playwright-core + @playwright/test + +/opt/playwright-endform/node_modules/ + playwright + playwright-core + @playwright/test +``` + +The Endform root should be installed with canonical alias names that point at the scoped packages: + +```json +{ + "dependencies": { + "playwright-core": "npm:@endform/playwright-core@1.60.0-beta.1", + "playwright": "npm:@endform/playwright@1.60.0-beta.1", + "@playwright/test": "npm:@endform/playwright-test@1.60.0-beta.1" + } +} +``` + +At invocation time, LambdaRunner should select a matched package set and create execution-directory symlinks such as: + +```text +/tmp/node_modules/playwright -> selected root/node_modules/playwright +/tmp/node_modules/playwright-core -> selected root/node_modules/playwright-core +/tmp/node_modules/@playwright/test -> selected root/node_modules/@playwright/test +``` + +Do not symlink only one package. `@playwright/test`, `playwright`, and `playwright-core` must be selected as a matched set so that all canonical package imports bind to the same implementation. + +The switch must happen before the Node host imports Playwright. Once Node loads `playwright`, `playwright-core`, or `@playwright/test`, the module cache makes in-process switching unsafe. + +Avoid using `NODE_OPTIONS=--preserve-symlinks` as the main solution. It changes module identity globally and can create duplicate-module problems. Prefer isolated install roots plus canonical symlinks. + +## Staging Packages + +Build Playwright first: + +```bash +npm run build +``` + +Stage and validate Endform tarballs: + +```bash +node utils/endform/stage_endform_packages.js +``` + +By default, tarballs are written to `endform-packages/`: + +```text +endform-packages/endform-playwright-core-.tgz +endform-packages/endform-playwright-.tgz +endform-packages/endform-playwright-test-.tgz +``` + +The staging script validates canonical alias installation, which is the install shape LambdaRunner needs for symlink selection. Direct scoped installation requires the packages to already exist in the npm registry because the staged packages intentionally depend on each other through npm aliases. + +## Publishing + +For a beta release: + +```bash +node utils/workspace.js --set-version 1.60.0-beta.1 +npm run build +utils/endform/publish_endform_packages.sh --beta --dry-run +utils/endform/publish_endform_packages.sh --beta +``` + +For a stable release: + +```bash +node utils/workspace.js --set-version 1.60.0 +npm run build +utils/endform/publish_endform_packages.sh --release --dry-run +utils/endform/publish_endform_packages.sh --release +``` + +`--beta` publishes with the `beta` dist-tag and requires a version containing `-beta`. `--release` publishes with the `latest` dist-tag and rejects pre-release versions. + +The publish order is dependency-safe: + +1. `@endform/playwright-core` +2. `@endform/playwright` +3. `@endform/playwright-test` diff --git a/utils/endform/programmatic-runner-smoke.js b/utils/endform/programmatic-runner-smoke.js new file mode 100644 index 0000000000000..f7724ec7789f5 --- /dev/null +++ b/utils/endform/programmatic-runner-smoke.js @@ -0,0 +1,262 @@ +#!/usr/bin/env node +/** + * Copyright Microsoft Corporation. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +function parseArgs() { + const result = { playwrightRoot: process.cwd() }; + for (let i = 2; i < process.argv.length; i++) { + const arg = process.argv[i]; + if (arg === '--playwright-root') + result.playwrightRoot = path.resolve(process.argv[++i]); + else if (arg.startsWith('--playwright-root=')) + result.playwrightRoot = path.resolve(arg.substring('--playwright-root='.length)); + else + throw new Error(`Unknown argument: ${arg}`); + } + return result; +} + +function requireFirst(candidates, label) { + const errors = []; + for (const candidate of candidates) { + try { + if (fs.existsSync(candidate)) + return require(candidate); + } catch (e) { + errors.push(`${candidate}: ${e.message}`); + } + } + throw new Error([ + `Could not load ${label}.`, + `Tried:`, + ...candidates.map(candidate => ` ${candidate}`), + `Build Playwright first, or pass --playwright-root to a built patched checkout/package.`, + ...errors.map(error => ` ${error}`), + ].join('\n')); +} + +function publicRunner(root) { + return requireFirst(publicRunnerCandidates(root, '.js'), 'playwright/programmatic-runner'); +} + +function publicRunnerCandidates(root, extension) { + return [ + path.join(root, 'packages', 'playwright', `programmatic-runner${extension}`), + path.join(root, `programmatic-runner${extension}`), + path.join(root, 'node_modules', 'playwright', `programmatic-runner${extension}`), + ]; +} + +function firstExisting(candidates, label) { + const result = candidates.find(candidate => fs.existsSync(candidate)); + if (!result) + throw new Error(`Could not find ${label}. Tried:\n${candidates.map(candidate => ` ${candidate}`).join('\n')}`); + return result; +} + +function publicRunnerESM(root) { + return firstExisting(publicRunnerCandidates(root, '.mjs'), 'playwright/programmatic-runner ESM entry'); +} + +function playwrightTestEntry(root) { + return firstExisting([ + path.join(root, 'packages', 'playwright', 'test.js'), + path.join(root, 'test.js'), + path.join(root, 'node_modules', 'playwright', 'test.js'), + ], 'playwright/test entry'); +} + +async function writeFile(filePath, text) { + await fs.promises.mkdir(path.dirname(filePath), { recursive: true }); + await fs.promises.writeFile(filePath, text); +} + +function assert(condition, message) { + if (!condition) + throw new Error(message); +} + +class SmokeReporter { + constructor(options) { + this.eventsFile = options.eventsFile; + this.events = []; + this.stdoutWithTest = false; + this.stderrWithTest = false; + this.attachments = []; + } + + version() { + return 'v2'; + } + + onConfigure(config) { + this.events.push('onConfigure'); + this.configMetadata = config.metadata; + } + + onBegin(suite) { + this.events.push('onBegin'); + this.testCount = suite.allTests().length; + } + + onTestBegin(test, result) { + this.events.push('onTestBegin'); + this.testTitle = test.title; + this.workerIndex = result.workerIndex; + } + + onStepBegin(test, result, step) { + this.events.push('onStepBegin:' + step.title); + } + + onStepEnd(test, result, step) { + this.events.push('onStepEnd:' + step.title); + } + + onStdOut(chunk, test, result) { + if (String(chunk).includes('stdout-from-programmatic-test')) { + this.events.push('onStdOut'); + this.stdoutWithTest = !!test && !!result; + } + } + + onStdErr(chunk, test, result) { + if (String(chunk).includes('stderr-from-programmatic-test')) { + this.events.push('onStdErr'); + this.stderrWithTest = !!test && !!result; + } + } + + onTestEnd(test, result) { + this.events.push('onTestEnd:' + result.status); + this.attachments = result.attachments.map(a => ({ name: a.name, contentType: a.contentType, body: a.body && a.body.toString() })); + } + + onError(error) { + this.events.push('onError'); + this.errors = this.errors || []; + this.errors.push(error.message); + } + + onEnd(result) { + this.events.push('onEnd:' + result.status); + this.finalStatus = result.status; + } + + onExit() { + this.events.push('onExit'); + fs.writeFileSync(this.eventsFile, JSON.stringify({ + events: this.events, + configMetadata: this.configMetadata, + finalStatus: this.finalStatus, + testCount: this.testCount, + testTitle: this.testTitle, + stdoutWithTest: this.stdoutWithTest, + stderrWithTest: this.stderrWithTest, + attachments: this.attachments, + errors: this.errors || [], + })); + } +} + +async function main() { + const { playwrightRoot } = parseArgs(); + const runner = publicRunner(playwrightRoot); + const esmRunner = await import(publicRunnerESM(playwrightRoot)); + assert(typeof esmRunner.runTests === 'function', 'Expected ESM entry to export runTests'); + const tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'pw-programmatic-runner-smoke-')); + const configFile = path.join(tmpDir, 'playwright.config.js'); + const reporterFile = path.join(tmpDir, 'smoke-reporter.js'); + const eventsFile = path.join(tmpDir, 'smoke-events.json'); + const testFile = path.join(tmpDir, 'programmatic-smoke.spec.js'); + const testEntry = playwrightTestEntry(playwrightRoot); + const preforkedWorkers = await runner.createPreforkedWorkers({ workers: 1 }); + + await writeFile(path.join(tmpDir, 'node_modules', '@playwright', 'test', 'index.js'), `module.exports = require(${JSON.stringify(testEntry)});\n`); + await writeFile(reporterFile, `const fs = require('fs');\nmodule.exports = ${SmokeReporter.toString()};\n`); + await writeFile(configFile, ` +module.exports = { + testDir: ${JSON.stringify(tmpDir)}, + workers: 4, + metadata: { fromConfigFile: true }, + reporter: 'line', +}; +`); + await writeFile(testFile, ` +const { test, expect } = require('@playwright/test'); + +test('programmatic smoke', async ({}, testInfo) => { + expect(process.env.ENDFORM_LATE_ENV).toBe('from-init'); + console.log('stdout-from-programmatic-test'); + console.error('stderr-from-programmatic-test'); + await test.step('programmatic step', async () => { + expect(1 + 1).toBe(2); + }); + await testInfo.attach('programmatic-attachment', { + body: Buffer.from('attachment-body'), + contentType: 'text/plain', + }); +}); + +test('not selected', async () => { + throw new Error('This test should not run'); +}); +`); + + try { + const config = await runner.loadUserConfig(configFile); + config.workers = 1; + config.reporter = [[reporterFile, { eventsFile }]]; + config.metadata = { fromMutatedConfig: true }; + + const result = await runner.runTests({ + configLocation: configFile, + config, + ignoreProjectDependencies: true, + testSelection: { tests: [{ projectName: '', file: testFile, titlePath: ['programmatic smoke'] }] }, + preforkedWorkers, + workerEnv: { ENDFORM_LATE_ENV: 'from-init' }, + }); + const reporter = JSON.parse(await fs.promises.readFile(eventsFile, 'utf-8')); + + assert(result.status === 'passed', `Expected run status passed, got ${result.status}. Test count: ${reporter.testCount}. Events: ${reporter.events.join(', ')}. Errors: ${JSON.stringify(reporter.errors || [])}`); + assert(reporter.finalStatus === 'passed', `Expected reporter final status passed, got ${reporter.finalStatus}`); + assert(reporter.testCount === 1, `Expected one test in onBegin, got ${reporter.testCount}`); + assert(reporter.testTitle === 'programmatic smoke', `Expected programmatic smoke test, got ${reporter.testTitle}`); + assert(reporter.configMetadata.fromMutatedConfig, `Expected mutated metadata, got ${JSON.stringify(reporter.configMetadata)}`); + assert(reporter.stdoutWithTest, 'Expected stdout to be attributed to test/result'); + assert(reporter.stderrWithTest, 'Expected stderr to be attributed to test/result'); + assert(reporter.events.includes('onStepBegin:programmatic step'), `Missing step begin. Events: ${reporter.events.join(', ')}`); + assert(reporter.events.includes('onStepEnd:programmatic step'), `Missing step end. Events: ${reporter.events.join(', ')}`); + assert(reporter.attachments.some(a => a.name === 'programmatic-attachment' && a.body === 'attachment-body'), `Missing attachment. Attachments: ${JSON.stringify(reporter.attachments)}`); + for (const event of ['onConfigure', 'onBegin', 'onTestBegin', 'onStdOut', 'onStdErr', 'onTestEnd:passed', 'onEnd:passed', 'onExit']) + assert(reporter.events.includes(event), `Missing reporter event ${event}. Events: ${reporter.events.join(', ')}`); + + console.log('PROGRAMMATIC_RUNNER_SMOKE_OK'); + console.log(JSON.stringify({ events: reporter.events })); + } finally { + await runner.disposePreforkedWorkers(preforkedWorkers); + } +} + +main().catch(e => { + console.error(e.stack || e.message || String(e)); + process.exit(1); +}); diff --git a/utils/endform/publish_endform_packages.sh b/utils/endform/publish_endform_packages.sh new file mode 100755 index 0000000000000..b95c004d0edff --- /dev/null +++ b/utils/endform/publish_endform_packages.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +set -euo pipefail + +function usage { + echo "usage: $(basename "$0") [--beta|--release] [--dry-run] [--out-dir ] [--skip-smoke]" + echo + echo "Stages and publishes @endform Playwright packages." + echo + echo "--beta publish a pre-release version under the beta dist-tag" + echo "--release publish a stable version under the latest dist-tag" + echo "--dry-run stage and validate tarballs without publishing" +} + +MODE="" +DRY_RUN=0 +OUT_DIR="" +SKIP_SMOKE=0 + +while [[ $# -gt 0 ]]; do + case "$1" in + --beta|--release) + MODE="$1" + shift + ;; + --dry-run) + DRY_RUN=1 + shift + ;; + --out-dir) + OUT_DIR="$2" + shift 2 + ;; + --out-dir=*) + OUT_DIR="${1#--out-dir=}" + shift + ;; + --skip-smoke) + SKIP_SMOKE=1 + shift + ;; + --help) + usage + exit 0 + ;; + *) + echo "Unknown argument: $1" >&2 + usage >&2 + exit 1 + ;; + esac +done + +if [[ -z "${MODE}" ]]; then + echo "Please specify --beta or --release" >&2 + usage >&2 + exit 1 +fi + +if ! command -v npm >/dev/null; then + echo "ERROR: npm is not found" >&2 + exit 1 +fi + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" +ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd -P)" +cd "${ROOT_DIR}" + +VERSION="$(node -e 'console.log(require("./package.json").version)')" +NPM_TAG="" + +if [[ "${MODE}" == "--release" ]]; then + if [[ "${VERSION}" == *-* ]]; then + echo "ERROR: cannot publish pre-release version ${VERSION} with --release" >&2 + exit 1 + fi + NPM_TAG="latest" +else + if [[ "${VERSION}" != *-beta* ]]; then + echo "ERROR: --beta requires a beta version, got ${VERSION}" >&2 + exit 1 + fi + NPM_TAG="beta" +fi + +STAGE_ARGS=() +if [[ -n "${OUT_DIR}" ]]; then + STAGE_ARGS+=("--out-dir" "${OUT_DIR}") +else + OUT_DIR="${ROOT_DIR}/endform-packages" +fi +if [[ "${SKIP_SMOKE}" == "1" ]]; then + STAGE_ARGS+=("--skip-smoke") +fi + +if [[ ${#STAGE_ARGS[@]} -eq 0 ]]; then + node "${SCRIPT_DIR}/stage_endform_packages.js" +else + node "${SCRIPT_DIR}/stage_endform_packages.js" "${STAGE_ARGS[@]}" +fi + +CORE_TGZ="${OUT_DIR}/endform-playwright-core-${VERSION}.tgz" +PLAYWRIGHT_TGZ="${OUT_DIR}/endform-playwright-${VERSION}.tgz" +TEST_TGZ="${OUT_DIR}/endform-playwright-test-${VERSION}.tgz" + +if [[ "${DRY_RUN}" == "1" ]]; then + echo "Dry run complete. Tarballs are ready in ${OUT_DIR}:" + echo " ${CORE_TGZ}" + echo " ${PLAYWRIGHT_TGZ}" + echo " ${TEST_TGZ}" + exit 0 +fi + +npm publish --access=public --tag="${NPM_TAG}" "${CORE_TGZ}" +npm publish --access=public --tag="${NPM_TAG}" "${PLAYWRIGHT_TGZ}" +npm publish --access=public --tag="${NPM_TAG}" "${TEST_TGZ}" + +echo "Published @endform Playwright ${VERSION} with dist-tag ${NPM_TAG}." diff --git a/utils/endform/stage_endform_packages.js b/utils/endform/stage_endform_packages.js new file mode 100755 index 0000000000000..8c0d48ab56d7c --- /dev/null +++ b/utils/endform/stage_endform_packages.js @@ -0,0 +1,212 @@ +#!/usr/bin/env node +/** + * Copyright Microsoft Corporation. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +const childProcess = require('child_process'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const ROOT = path.resolve(__dirname, '..', '..'); +const DEFAULT_OUT_DIR = path.join(ROOT, 'endform-packages'); + +const packages = [ + { + sourceDir: path.join(ROOT, 'packages', 'playwright-core'), + stagedName: '@endform/playwright-core', + }, + { + sourceDir: path.join(ROOT, 'packages', 'playwright'), + stagedName: '@endform/playwright', + dependencies: version => ({ + 'playwright-core': `npm:@endform/playwright-core@${version}`, + }), + }, + { + sourceDir: path.join(ROOT, 'packages', 'playwright-test'), + stagedName: '@endform/playwright-test', + dependencies: version => ({ + 'playwright': `npm:@endform/playwright@${version}`, + }), + }, +]; + +function parseArgs() { + const result = { + outDir: DEFAULT_OUT_DIR, + skipSmoke: false, + keepTemp: false, + }; + for (let i = 2; i < process.argv.length; i++) { + const arg = process.argv[i]; + if (arg === '--out-dir') + result.outDir = path.resolve(process.argv[++i]); + else if (arg.startsWith('--out-dir=')) + result.outDir = path.resolve(arg.substring('--out-dir='.length)); + else if (arg === '--skip-smoke') + result.skipSmoke = true; + else if (arg === '--keep-temp') + result.keepTemp = true; + else if (arg === '--help') { + usage(); + process.exit(0); + } else { + throw new Error(`Unknown argument: ${arg}`); + } + } + return result; +} + +function usage() { + console.log(`usage: stage_endform_packages.js [--out-dir ] [--skip-smoke] [--keep-temp]\n\nStages @endform Playwright packages by packing the canonical workspace packages, rewriting only staged package metadata, and producing final tarballs.`); +} + +function run(command, args, options = {}) { + const result = childProcess.spawnSync(command, args, { + cwd: options.cwd || ROOT, + env: { ...process.env, PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1', ...options.env }, + encoding: 'utf8', + stdio: options.capture ? 'pipe' : 'inherit', + shell: process.platform === 'win32', + }); + if (result.status !== 0) { + if (options.capture) { + process.stdout.write(result.stdout || ''); + process.stderr.write(result.stderr || ''); + } + throw new Error(`${command} ${args.join(' ')} failed with exit code ${result.status}`); + } + return result.stdout ? result.stdout.trim() : ''; +} + +async function rm(dir) { + await fs.promises.rm(dir, { recursive: true, force: true }); +} + +async function readJSON(file) { + return JSON.parse(await fs.promises.readFile(file, 'utf8')); +} + +async function writeJSON(file, object) { + await fs.promises.writeFile(file, JSON.stringify(object, null, 2) + '\n'); +} + +function tarballBaseName(name, version) { + const unscoped = name.replace(/^@/, '').replace('/', '-'); + return `${unscoped}-${version}.tgz`; +} + +async function packSourcePackage(sourceDir, tempDir) { + const packDir = path.join(tempDir, 'source-packs'); + await fs.promises.mkdir(packDir, { recursive: true }); + const output = run('npm', ['pack', sourceDir, '--pack-destination', packDir], { capture: true }); + const tgzName = output.split('\n').filter(Boolean).pop(); + return path.join(packDir, tgzName); +} + +async function extractPackage(tgzPath, destination) { + await fs.promises.mkdir(destination, { recursive: true }); + run('tar', ['-xzf', tgzPath, '-C', destination]); + return path.join(destination, 'package'); +} + +async function rewritePackageJSON(packageDir, descriptor, version) { + const packageJSONPath = path.join(packageDir, 'package.json'); + const packageJSON = await readJSON(packageJSONPath); + packageJSON.name = descriptor.stagedName; + packageJSON.version = version; + if (descriptor.dependencies) + packageJSON.dependencies = descriptor.dependencies(version); + await writeJSON(packageJSONPath, packageJSON); +} + +async function packStagedPackage(packageDir, outDir, expectedName) { + const output = run('npm', ['pack', packageDir, '--pack-destination', outDir], { capture: true }); + const tgzName = output.split('\n').filter(Boolean).pop(); + const tgzPath = path.join(outDir, tgzName); + const expectedPath = path.join(outDir, expectedName); + if (tgzPath !== expectedPath) { + await fs.promises.rename(tgzPath, expectedPath); + return expectedPath; + } + return tgzPath; +} + +async function smokeInstall(tarballs, version) { + const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'endform-playwright-smoke-')); + try { + // Local pre-publish validation uses the same canonical alias shape LambdaRunner + // installs in its isolated Endform root. Direct scoped installation requires + // the scoped packages to already exist in the npm registry because staged + // package dependencies intentionally use npm aliases. + await writeJSON(path.join(tempDir, 'package.json'), { + private: true, + dependencies: { + 'playwright-core': tarballs['@endform/playwright-core'], + 'playwright': tarballs['@endform/playwright'], + '@playwright/test': tarballs['@endform/playwright-test'], + }, + }); + run('npm', ['install', '--ignore-scripts'], { cwd: tempDir }); + run('node', ['-e', [ + `const core = require('playwright-core/package.json');`, + `const pw = require('playwright/package.json');`, + `const pwt = require('@playwright/test/package.json');`, + `if (core.version !== ${JSON.stringify(version)} || pw.version !== ${JSON.stringify(version)} || pwt.version !== ${JSON.stringify(version)}) throw new Error('version mismatch');`, + `if (pw.dependencies['playwright-core'] !== ${JSON.stringify(`npm:@endform/playwright-core@${version}`)}) throw new Error('bad playwright-core alias');`, + `if (pwt.dependencies.playwright !== ${JSON.stringify(`npm:@endform/playwright@${version}`)}) throw new Error('bad playwright alias');`, + `require('playwright');`, + `require('playwright/test');`, + `require('playwright/programmatic-runner');`, + `require('@playwright/test');`, + ].join('')], { cwd: tempDir }); + } finally { + await rm(tempDir); + } +} + +async function main() { + const options = parseArgs(); + const version = (await readJSON(path.join(ROOT, 'package.json'))).version; + const tempDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'endform-playwright-stage-')); + const tarballs = {}; + try { + await rm(options.outDir); + await fs.promises.mkdir(options.outDir, { recursive: true }); + for (const descriptor of packages) { + const sourceTgz = await packSourcePackage(descriptor.sourceDir, tempDir); + const packageDir = await extractPackage(sourceTgz, path.join(tempDir, descriptor.stagedName.replace('/', '-').replace('@', ''))); + await rewritePackageJSON(packageDir, descriptor, version); + const outputName = tarballBaseName(descriptor.stagedName, version); + tarballs[descriptor.stagedName] = await packStagedPackage(packageDir, options.outDir, outputName); + } + + if (!options.skipSmoke) + await smokeInstall(tarballs, version); + + console.log(JSON.stringify({ version, outDir: options.outDir, tarballs }, null, 2)); + } finally { + if (options.keepTemp) + console.error(`Kept temp staging directory: ${tempDir}`); + else + await rm(tempDir); + } +} + +main().catch(error => { + console.error(error.stack || error.message || String(error)); + process.exit(1); +});