fix(appkit): non-blocking typegen on Analytics#406
Merged
Conversation
Type generation threw an uncaught error whenever the SQL warehouse was down. Every DESCRIBE QUERY failed, all queries degraded to an unknown result, and generateFromEntryPoint unconditionally threw an aggregate "Type generation failed" error that escaped uncaught at the Vite plugin (un-awaited generate()) and the CLI (sync cmd.parse()) call sites. Distinguish connectivity failures from genuine SQL errors: - Connectivity (executeStatement rejects): degrade silently. Reuse the last-known-good cached type if present, otherwise emit an unknown result. Never fatal, so a transient outage no longer fails a build. - SQL error (reachable warehouse, DESCRIBE FAILED): surface via a typed TypegenSyntaxError so the existing prod-throws / dev-warns gate applies. Eligible to fail prod builds only. Also stop caching unknown results: only successful describes with a result schema are persisted, so a transient outage never poisons the cache and a fixed query recovers on the next run. PR1 of 2 (user-visible behavior). PR2 will await the Vite buildStart/watcher, use parseAsync().catch() in the CLI, and add degrade/throw regression tests. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
Add the regression coverage the warehouse-down crash slipped through: the prior suite tested rejection->unknown as graceful but never connected it to the aggregate throw in generateFromEntryPoint. query-registry (generate-queries.test.ts): - connectivity reuses the last-known-good cached type - empty result (described, no columns) -> unknown, not syntax, not cached - syntax (FAILED) -> recorded in syntaxErrors, not cached - cache HIT serves the stored type without a warehouse call - legacy retry-flagged entry is re-described, not reused - mixed run records only the syntax failure; failures are not persisted generateFromEntryPoint (index.test.ts): - syntax errors throw TypegenSyntaxError - connectivity-only failures do NOT throw (the warehouse-down regression) - the .d.ts is written before the throw Layers 1+2 of the test plan; Layer 3 (analytics vite-plugin) and Layer 4 (CLI exit codes) land in PR2 with their await/parseAsync refactors. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
Signed-off-by: Atila Fassina <atila@fassina.eu>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR prevents analytics type generation from crashing when the SQL warehouse is unreachable by distinguishing connectivity failures (non-fatal; reuse cached types or emit unknown) from genuine SQL errors (warehouse reachable but DESCRIBE QUERY returns FAILED, which can be treated as fatal by callers).
Changes:
- Classifies per-query
DESCRIBE QUERYoutcomes into success, syntax error (FAILED), connectivity error (request rejected), and empty result; only successful results are cached. - Updates the typegen entrypoint to throw a typed
TypegenSyntaxErroronly when genuine SQL errors occur, and only after writing the.d.tsoutput. - Adds/updates tests to cover “warehouse down doesn’t crash” and caching behavior during connectivity vs syntax failures.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/appkit/src/type-generator/types.ts | Adds QuerySyntaxError and QueryGenerationResult types to represent non-fatal vs fatal describe outcomes. |
| packages/appkit/src/type-generator/query-registry.ts | Implements failure classification, prevents caching unknown, reuses prior cached types on connectivity failures, and returns {schemas, syntaxErrors}. |
| packages/appkit/src/type-generator/index.ts | Introduces TypegenSyntaxError and gates throwing on syntaxErrors after emitting .d.ts. |
| packages/appkit/src/type-generator/tests/index.test.ts | Adds tests asserting syntax errors throw (after writing output) while connectivity failures do not throw. |
| packages/appkit/src/type-generator/tests/generate-queries.test.ts | Expands coverage for caching rules, offline behavior, empty result handling, and syntax error reporting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Atila Fassina <atila@fassina.eu>
Add a warehouse-status pre-flight to typegen and rework error handling so a stopped, starting, or unreachable warehouse degrades gracefully instead of crashing or emitting EMPTY types. - Classify connectivity (incl. the SDK's "Can't connect to <url>"/code-500 DNS wrapper) as OFFLINE, and non-terminal describe states (PENDING/RUNNING) as degraded rather than EMPTY. - Surface typegen errors as their actionable message (no internal stack trace); format and de-duplicate the failure output. - Add a warehouse status probe + pure pre-flight policy; block in the CLI/build, roll forward (degrade) in dev. - Dev: regenerate types in the background once the warehouse reaches RUNNING (single-flight guarded); auto-start a stopped warehouse in dev. - CLI: --no-block degrades instead of describing so postinstall never blocks. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
c073494 to
97c94a1
Compare
…locking Replace the dev/blocking/degrade modes with two: non-blocking (default) and blocking. non-blocking always degrades (skip probe + DESCRIBE, write cache-or- unknown instantly); blocking keeps the current probe+wait flow. The CLI default flips to non-blocking and --no-block becomes a positive --block flag. The dev plugin runs the foreground in non-blocking (instant degrade) while its background warehouse-watch regenerates in blocking so real types still land. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
In blocking mode a STOPPED/STOPPING warehouse is now started and waited on (startWaitProceed: startWarehouse -> waitUntilRunning -> describe) instead of failing fast. Only DELETED/DELETING is fatal. STARTING still waits; RUNNING describes. The write-the-.d.ts-then-throw invariant is preserved for the fatal and wait-timeout paths. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
The dev background lifecycle returned early for RUNNING, a leftover from when the foreground described it synchronously. Now that the foreground always degrades instantly (non-blocking), RUNNING must also background-describe or a running warehouse never gets real types in dev. Only DELETED/DELETING leaves degraded; single-flight coalescing and abort-on-shutdown are preserved. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
The default (non-blocking) generate-types now writes degraded types, then spawns a detached `generate-types --block` worker behind a single-flight lock and exits 0 -- so postinstall/predev never block on warehouse state. The worker does the full blocking lifecycle in the background, refreshes real types, and releases the lock on exit (process-exit guard covers a hard fail). A stale lock from a crashed worker is stolen after 6 min; spawn failure is non-fatal to the foreground. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
…block Template postinstall/predev now run the non-blocking default `appkit generate-types` (instant degrade + background refresh) instead of a dedicated no-block script; prebuild keeps `--block` for accurate CI types. Removed the now redundant `typegen:no-block` script. Documented the non-blocking default and the `--block` flag in the type-generation guide. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
A failed DESCRIBE logged the raw SQL error message per query during the describe loop, and the aggregated TypegenSyntaxError (printed by the Vite plugin / CLI) carries the same message in its formatted block -- so every SQL syntax error showed up twice in dev. Drop the per-query warn; the formatted block and the summary table already surface it. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
MarioCadenas
reviewed
Jun 5, 2026
…er tsx The non-blocking CLI spawned its background worker as `node <argv[1]>` without the parent's node/loader flags. Run from source via tsx, argv[1] is a .ts file that plain node can't parse, so the worker died silently (detached + stdio ignore) and the degraded types never refreshed -- the queries appeared to never run. Forward process.execArgv, which carries tsx's --require/--import loader flags (and is empty for the built bin, so production is unaffected), so the worker runs under the same runtime as the parent. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
0de804c to
2947a69
Compare
Rename the user-facing CLI flag --block to --wait (commander option property block -> wait), including the detached worker's self-spawn arg, the --help example, the template prebuild script, and the type-generation guide. The internal "blocking"/"non-blocking" PreflightMode names are unchanged -- they describe runtime behaviour and aren't user-facing. The flag only ever existed on this branch (unreleased), so no deprecation alias is needed. Co-authored-by: Isaac Signed-off-by: Atila Fassina <atila@fassina.eu>
MarioCadenas
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes type generation warehouse-aware and non-blocking by default.
Previously typegen ran
DESCRIBE QUERYblind to the warehouse's lifecycle state and to the SDK's real error shapes, producing two wrong outcomes — a STOPPED warehouse was misread as "empty" (every query emittedresult: unknownand the known-good types were discarded), and an unreachable host was treated as a fatal build failure. On top of that it blockednpm install/npm devon a cold warehouse and could crash uncaught.After this PR:
appkit generate-types, the dev server,postinstall/predev). Typegen writes the best types it can immediately — the last-known-good cached type where the SQL is unchanged, otherwiseresult: unknown— and never blocks on, or fails because of, the warehouse. Real types then refresh in the background.--block(CI /prebuild) waits for readiness and produces accurate types, failing fast only when the warehouse genuinely can't serve them (deleted) or the SQL is wrong.Behavior
A pure policy function maps (warehouse state × mode) to an action; the orchestration executes it.
non-blocking(default)blocking(--block)unknown)DESCRIBE … FAILED)unknown; reportedresult: unknown. Degraded types are never persisted, so a transient outage can't poison the cache and a fixed query recovers on the next run..d.tsis always written before any throw.--blockworker behind a single-flight lock (stale-stealable after 6 min), sopostinstallreturns instantly and types refresh once the worker finishes.Key pieces
preflight.ts— puredecidePreflight(state, mode)policy; fully table-tested, no I/O.warehouse-status.ts—getWarehouseState,startWarehouse, and a bounded, abortablewaitUntilRunning(exp backoff;treatStoppedAsTransientso it polls through the STOPPED → STARTING transition right after a start).query-registry.ts— pre-flight orchestration + per-query classification (ok/syntax/connectivity/ non-terminal → degrade), with a per-query backstop so a mid-run state change can't resurface the "empty" misclassification.vite-plugin.ts— dev: instant degrade in the foreground, background lifecycle (including a RUNNING warehouse) with single-flight coalescing and abort-on-shutdown.cli/commands/generate-types.ts+spawn-lock.ts— non-blocking CLI degrades then spawns the detached worker behind the lock; positive--blockflag (replaces--no-block); hidden internal--worker-lock."Can't connect to <url>"carrying numeric code 500) → degrade, not fatal.template/package.json—postinstall/predev→ non-blocking default;prebuild→--block; dropped the redundanttypegen:no-blockscript.type-generation.mddocuments the non-blocking default and--block.Testing
.d.tsis written; the per-query non-terminal backstop.build:package+ publint clean.This pull request and its description were written by Isaac.