Repository navigation
Expose app dev session files for agents - #8926
isaacroldan wants to merge 14 commits into
Conversation
|
/snapit |
|
🫰✨ Thanks @isaacroldan! Your snapshot has been published to npm. Built from Test the snapshot by installing your package globally: pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20261015153801Caution After installing, validate the version by running |
dee7821 to
2b3b7dd
Compare
|
/snapit |
|
🫰✨ Thanks @isaacroldan! Your snapshot has been published to npm. Built from Test the snapshot by installing your package globally: pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20261019115812Caution After installing, validate the version by running |
cfd91fb to
b06cf68
Compare
|
/snapit |
|
🫰✨ Thanks @isaacroldan! Your snapshot has been published to npm. Built from Test the snapshot by installing your package globally: pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20261019155212Caution After installing, validate the version by running |
b06cf68 to
42ae66e
Compare
Differences in type declarationsWe detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:
New type declarationspackages/cli-kit/dist/private/node/dev-instance-output.d.tsimport { zod } from '../../public/node/schema.js';
import { Writable, type WritableOptions } from 'stream';
export declare const devInstanceLogOutputSchema: import("../../public/node/json-output-schema.js").JsonOutputSchema<zod.ZodObject<{
timestamp: zod.ZodString;
source: zod.ZodString;
level: zod.ZodOptional<zod.ZodEnum<["debug", "info", "warning", "error"]>>;
message: zod.ZodString;
}, "strict", zod.ZodTypeAny, {
source: string;
message: string;
timestamp: string;
level?: "info" | "error" | "debug" | "warning" | undefined;
}, {
source: string;
message: string;
timestamp: string;
level?: "info" | "error" | "debug" | "warning" | undefined;
}>>;
/** One line of a dev instance's JSON Lines log. */
export type DevInstanceLogEntry = zod.infer<typeof devInstanceLogOutputSchema.schema>;
interface DevInstanceOutputOptions {
logFile: string;
logOutput?: Writable;
isActive: () => boolean;
onError: (error: unknown) => void;
}
export declare function createDevInstanceOutput({ logFile, logOutput, isActive, onError }: DevInstanceOutputOptions): {
opened: Promise<any[]>;
log: (entry: Omit<DevInstanceLogEntry, "timestamp"> & {
timestamp?: string;
}) => Promise<void>;
tee: (target: Writable, { source, stringWritesAreMessages }: {
source: string;
stringWritesAreMessages?: boolean;
}) => DevInstanceOutput;
flush: () => Promise<PromiseSettledResult<void>[]>;
close: () => Promise<void>;
};
declare class DevInstanceOutput extends Writable {
private readonly contexts;
constructor(options: WritableOptions & {
write: NonNullable<WritableOptions['write']>;
});
write(chunk: string | Uint8Array, encoding?: BufferEncoding | ((error?: Error | null) => void), callback?: (error?: Error | null) => void): boolean;
end(chunk?: string | Uint8Array | null | (() => void), encoding?: BufferEncoding | (() => void), callback?: () => void): this;
}
export {};
packages/cli-kit/dist/private/node/output-context.d.tsexport interface ConcurrentOutputContext {
outputPrefix?: string;
/** Controls ANSI stripping for terminal output. JSON diagnostics are always unstyled. */
stripAnsi?: boolean;
}
export declare function useConcurrentOutputContext<T>(context: ConcurrentOutputContext, callback: () => T): T;
/** Returns the source and styling context for the current output write. */
export declare function getConcurrentOutputContext(): ConcurrentOutputContext | undefined;
packages/cli-kit/dist/public/node/dev-instance.d.tsimport { type DevInstanceLogEntry } from '../../private/node/dev-instance-output.js';
import { Writable } from 'stream';
/**
* A stable error code for JSON consumers when a long-running command refuses to start because another instance of
* it is already running for the same project.
*/
export declare const DEV_INSTANCE_ALREADY_RUNNING_CODE = "dev_instance_already_running";
/**
* Describes a running instance of a long-running command such as `app dev` or `theme dev`.
* It's written to `.shopify/dev/<slug>.json` so agents and other tools can discover the instance.
*/
export interface DevInstanceInfo {
schemaVersion: 1;
/** Identifies this run, including when the OS reuses a PID. */
sessionId: string;
/** The command that owns the instance, for example `app dev`. */
command: string;
/** Distinguishes instances of the same command in the same directory, for example the app configuration file. */
key?: string;
pid: number;
status: 'starting' | 'running';
startedAt: string;
updatedAt: string;
/** The project directory the instance runs for. */
directory: string;
/** The JSON Lines file the instance appends its output to. */
logFile: string;
/** Command-specific, JSON-serializable details such as URLs and ports. */
details: {
[key: string]: unknown;
};
}
export { devInstanceLogOutputSchema, type DevInstanceLogEntry } from '../../private/node/dev-instance-output.js';
export interface DevInstancePaths {
/** The folder that holds every dev instance of the project: `.shopify/dev`. */
folder: string;
/** The instance description. */
infoFile: string;
/** The lock directory, owned by `proper-lockfile`. */
lockFile: string;
/** The JSON Lines log. */
logFile: string;
}
export interface DevInstanceIdentity {
/** The command that owns the instance, for example `app dev`. */
command: string;
/** The project directory. */
directory: string;
/** Distinguishes several instances of the same command in the same directory. */
key?: string;
}
export interface AcquireDevInstanceOptions extends DevInstanceIdentity {
/** Initial command-specific details. */
details?: {
[key: string]: unknown;
};
/** Receives the exact JSONL bytes written to the session log. The caller owns this stream. */
logOutput?: Writable;
}
export interface DevInstanceUpdate {
status?: DevInstanceInfo['status'];
/** Merged into the existing details. */
details?: {
[key: string]: unknown;
};
}
export interface DevInstance {
readonly info: DevInstanceInfo;
readonly paths: DevInstancePaths;
/** Aborts when this session loses its lock or output fails. Consumers must stop their owned work. */
readonly abortSignal: AbortSignal;
/** Updates and persists the instance description. */
update: (changes: DevInstanceUpdate) => Promise<void>;
/** Appends one entry to the JSON Lines log. */
log: (entry: Omit<DevInstanceLogEntry, 'timestamp'> & {
timestamp?: string;
}) => Promise<void>;
/**
* Wraps a stream so every line written to it is also appended to the log, without ANSI styling.
* The returned stream forwards the original bytes to `target` unchanged.
*/
tee: (target: Writable, options: {
source: string;
/** Treat each string write as a complete message. Byte writes remain buffered into lines. */
stringWritesAreMessages?: boolean;
}) => Writable;
/** Closes the log, removes the instance description and releases the lock. Safe to call more than once. */
release: () => Promise<void>;
}
/**
* Returns the files used by a dev instance. Every instance of a project shares `.shopify/dev`,
* with one `<slug>.json`, `<slug>.lock` and `<slug>.jsonl` per command and key.
*
* @param identity - The command, directory and optional key of the instance.
* @returns The paths of the instance files.
*/
export declare function devInstancePaths(identity: DevInstanceIdentity): DevInstancePaths;
/**
* Marks this process as the running instance of a long-running command for a project.
*
* Mutual exclusion uses `proper-lockfile`: an atomic `mkdir` lock whose mtime this process refreshes as a heartbeat.
* An exclusive startup directory serializes abandoned-lock recovery. An interrupted startup requires explicit
* cleanup of that directory. A live owner blocks takeover, including when its heartbeat is stale.
*
* @param options - The instance identity, initial details.
* @returns The acquired instance.
* @throws AbortError with code {@link DEV_INSTANCE_ALREADY_RUNNING_CODE} when another live instance holds the lock.
*/
export declare function acquireDevInstance(options: AcquireDevInstanceOptions): Promise<DevInstance>;
/**
* Acquires a dev instance and releases it after the callback completes or throws.
*
* @param options - The command, project and output options for the instance.
* @param execute - The work to run while the instance is owned.
* @returns The callback's result after instance cleanup completes.
*/
export declare function withDevInstance<T>(options: AcquireDevInstanceOptions, execute: (instance: DevInstance) => T | Promise<T>): Promise<T>;
/**
* Returns the running instance of a command for a project, if there is one.
*
* @param identity - The command, directory and optional key of the instance.
* @returns The instance description, or undefined when no live instance holds the lock.
*/
export declare function readDevInstance(identity: DevInstanceIdentity): Promise<DevInstanceInfo | undefined>;
/**
* Lists every running dev instance of a project, across commands and keys.
*
* @param directory - The project directory.
* @returns The descriptions of the live instances.
*/
export declare function listDevInstances(directory: string): Promise<DevInstanceInfo[]>;
/**
* Checks whether a process exists. A process owned by another user counts as alive.
*
* @param pid - The process ID.
* @returns True when the process exists.
*/
export declare function isProcessAlive(pid: number): boolean;
Existing type declarationspackages/cli-kit/dist/private/node/ui/components/ConcurrentOutput.d.ts@@ -1,3 +1,4 @@
+import { ConcurrentOutputContext, getConcurrentOutputContext, useConcurrentOutputContext } from '../../output-context.js';
import { type OutputProcess } from '../../../../public/node/output.js';
import { AbortSignal } from '../../../../public/node/abort.js';
import { FunctionComponent } from 'react';
@@ -13,12 +14,6 @@ export interface ConcurrentOutputProps {
keepRunningAfterProcessesResolve?: boolean;
useAlternativeColorPalette?: boolean;
}
-interface ConcurrentOutputContext {
- outputPrefix?: string;
- /** Controls ANSI stripping for terminal output. JSON diagnostics are always unstyled. */
- stripAnsi?: boolean;
-}
-declare function useConcurrentOutputContext<T>(context: ConcurrentOutputContext, callback: () => T): T;
/** Runs finite processes concurrently and routes their output through the shared diagnostic context. */
export declare function runConcurrentProcessesForJson({ processes, abortSignal, }: Pick<ConcurrentOutputProps, 'processes' | 'abortSignal'>): Promise<void>;
/**
@@ -55,4 +50,4 @@ export declare function runConcurrentProcessesForJson({ processes, abortSignal,
* ```
*/
declare const ConcurrentOutput: FunctionComponent<ConcurrentOutputProps>;
-export { ConcurrentOutput, ConcurrentOutputContext, useConcurrentOutputContext };
\ No newline at end of file
+export { ConcurrentOutput, ConcurrentOutputContext, getConcurrentOutputContext, useConcurrentOutputContext };
\ No newline at end of file
|
1ae5360 to
a3cebb6
Compare
WHY are these changes introduced?
Agents need to find an existing
app devsession and read its current status and output from another terminal. This PR builds on the reusable CLI Kit API in #8952.WHAT is this pull request doing?
Register one session per app configuration, capture process output in JSONL, and save the CLI's current preview status and errors. Non-interactive sessions remain registered until they stop, and dev work stops if session ownership or output fails. Active dev sessions handle signal cancellation and drain their logs before exit. Function compilation uses the supplied streams so Javy errors appear in both terminal output and the session log.
The
withDevSessionservice maps app metadata to the sharedwithDevInstancehelper in #8952. The command passes app work to that service; there is no session class or command-owned acquisition logic. The app integration uses the shared tee API from #8952. The session files include the PID, session ID, URLs, log path, anddetails.preview. Preview status reflects the CLI's reported state. The agent workflow and integration tests are included here; file ownership, logging, and their unit tests are in #8952.How to manually test your changes?
shopify app devfor a configured app and development store. From another terminal, read.shopify/dev/*.jsonand the reportedlogFile.details.preview.stateanderrors, then correct the import and check the new output and status.--no-input. Confirm the session ID changes, the log starts fresh, and the session remains registered while dev runs. Press Ctrl+C once and confirm that the process exits and the instance record is removed. Repeat with SIGTERM.See
docs/cli/app-dev-agent-workflow.mdfor the file-reading workflow.Validation
CI repair: dev tees and their tests use the merged CLI Kit API without the removed
streamoption. Source-loaded shutdown subprocesses have a 60-second overall allowance, a 20-second deadline from signal dispatch, and the existing 5-second postrun exit check. An unbuilt-checkout probe with a simulated 22-second startup fails before the change and passes afterward; a deliberately stuck shutdown still fails through the child watchdog. The 59-test focused suite and the full workspace build pass locally.After adopting the scoped helper, all 59 focused tests for this PR pass. The updated stack passes 158 tests, including output-context and preview-process coverage. SIGINT subprocess tests in text and JSON mode also pass on Node 22.12.0.
Non-interactive text output now adds a newline to complete CLI string messages. Existing LF/CRLF endings and subprocess byte chunks are preserved. The regression reproduced joined polling warnings before the fix and covers both output streams, empty writes, explicit string encodings, split UTF-8, and write completion. All 16 focused UI/service/shutdown tests for this fix pass on this PR.
This PR independently passes 59 focused tests covering app-dev command/service/UI behavior, real-bootstrap shutdown, and function builds. App and CLI type checks and focused lint pass. Neither this PR nor #8957 changes
dev-instance.tsor its unit tests.runJavyand the dev tee streams. Its diagnostic reaches both the terminal sink and session JSONL. The signal and compiler regressions failed before their fixes.The shutdown subprocess tests also pass from a clean checkout with no generated workspace modules on Node 22.12.0, 24.1.0, and 26.1.0. Their source loader and test TypeScript config resolve workspace packages directly. This reproduces the unit-test CI setup and keeps normal type checking enabled in the child processes. The 58-test focused suite passes with CI environment variables and forked workers.
Stress testing
The larger local session-API checks used real temporary files and separate child processes:
pnpm shopify app dev --help, and probed the generated chunk loaded byapp dev. Complete messages stayed separate, split UTF-8 was preserved, and JSONL records omitted session IDs.The ownership races, SIGKILL recovery, 33-second pause/resume, 3,000-update pressure test, 100,001-record burst, and filesystem-error matrix are detailed in the base PR, #8952. Regression cases are covered by committed tests; the larger stress loops were local, ad hoc scripts and are not part of CI.
These checks ran on macOS arm64 with Node v26.1.0. The stress harnesses did not exercise Windows/Linux or remote Shopify update behavior.
Checklist