Commit 2f93205
authored
fix(execution): run pptx/docx/pdf generation inside isolated-vm sandbox (#4217)
* fix(execution): run pptx/docx/pdf generation inside isolated-vm sandbox
Retires the legacy doc-worker.cjs / pptx-worker.cjs pipeline that ran user
DSL via node:vm + full require() in the same UID/PID namespace as the main
Next.js process. User code now runs inside the existing isolated-vm pool
(V8 isolate, no process / require / fs, no /proc/1/environ reachability).
Introduces a first-class SandboxTask abstraction under apps/sim/sandbox-tasks/
that mirrors apps/sim/background/ — one file per task, central typed
registry, kebab-case ids. Adding a new thing that runs in the isolate is
one file plus one registry entry.
Runtime additions in lib/execution/:
- task-mode execution in isolated-vm-worker.cjs: load pre-built library
bundles, run task bootstrap, run user code, run finalize, transfer
Uint8Array result as base64 via IPC
- named broker IPC bridge (generalizes the existing fetch bridge) with
args size, result size, and per-execution call caps
- cooperative AbortSignal support: cancel IPC disposes the isolate, pool
slot is freed, pending broker-call timers are swept
- compiled scripts + references explicitly released per execution
- isolate.isDisposed used for cancellation detection (no error-string
substring matching)
Library bundles (pptxgenjs, docx, pdf-lib) are built into isolate-safe
IIFE bundles by apps/sim/lib/execution/sandbox/bundles/build.ts and
committed; next.config.ts / trigger.config.ts / Dockerfile updated to
ship them instead of the deleted dist/*-worker.cjs artifacts.
Call sites migrated:
- app/api/workspaces/[id]/pptx/preview/route.ts
- app/api/files/serve/[...path]/route.ts (+ test mock)
- lib/copilot/tools/server/files/{workspace-file,edit-content}.ts
All pass owner key user:<userId> for per-user pool fairness + distributed
lease accounting.
Made-with: Cursor
* improvement(sandbox): delegate timers to Node, add phase timings + saturation logs
Follow-ups on top of the isolated-vm migration (da14027):
Timer delegation (laverdet/isolated-vm#136 recommended pattern):
- setTimeout / setInterval / clearTimeout / clearImmediate delegate to
Node's real timer heap via ivm.Reference. Real delays are honored;
clearTimeout actually cancels; ms is clamped to the script timeout
so callbacks can't fire after the isolate is disposed.
- Per-execution timer tracking + dispose-sweep in finally. Zero stale
callbacks post-dispose.
- unwrapPrimitive helper normalizes ivm.Reference-wrapped primitives
(arguments: { reference: true } applies uniformly to all args).
- _polyfills.ts shrinks from ~130 lines to the global->globalThis alias.
Timers / TextEncoder / TextDecoder / console all install per-execution
from the worker via ivm bridges.
AbortSignal race fix (pre-existing bug surfaced by the timer smoke):
- Listener is registered after await tryAcquireDistributedLease. If the
signal aborted during that ~200ms window (Redis down), AbortSignal
doesn't fire listeners registered after the fact — the abort was
silently missed. Now re-checks signal.aborted synchronously after
addEventListener.
Observability:
- executeTask returns IsolatedVMTaskTimings (setup, runtimeBootstrap,
bundles, brokerInstall, taskBootstrap, harden, userCode, finalize,
total) in every success + error path. run-task.ts logs these with
workspaceId + queueMs so 'which tenant is slow' is queryable.
- Pool saturation events now emit structured logger.warn with reason
codes: queue_full_global, queue_full_owner, queue_wait_timeout,
distributed_lease_limit. Matches the existing broker reject pattern.
Security policy:
- New .cursor/rules/sim-sandbox.mdc codifies the hard rules for the
worker process: no app credentials, all credentialed work goes
through host-side brokers, every broker scopes by workspaceId.
Pre-merge checklist for future changes to isolated-vm-worker.cjs.
Measured phase breakdown (local smoke, Redis down): pptx wall=~310ms
with bundles=~16ms, finalize=~83ms; docx ~290ms / 17ms / 70ms; pdf
~235ms / 17ms / 5ms. Bundle compilation is not the bottleneck —
library finalize is.
Made-with: Cursor
* fix(sandbox): thread AbortSignal into runSandboxTask at every call site
Three remaining callers of runSandboxTask were not threading a
cancellation signal, so a client disconnect mid-compile left the pool
slot occupied for the full 60s task timeout. Matching the pattern the
pptx-preview route already uses.
- apps/sim/app/api/files/serve/[...path]/route.ts — GET forwards
`request.signal` into handleLocalFile / handleCloudProxy, which
forward into compileDocumentIfNeeded, which forwards into
runSandboxTask.
- apps/sim/lib/copilot/tools/server/files/workspace-file.ts — passes
`context.abortSignal` (transport/user stop) into runSandboxTask.
- apps/sim/lib/copilot/tools/server/files/edit-content.ts — same.
Smoke: simulated client disconnect at t=1000ms during a task that would
otherwise have waited 10s. The pool slot unwinds at t=1002ms with
AbortError; previously would have sat 60s until the task-level timeout.
Made-with: Cursor
* chore(build): raise node heap to 8GB for next build type-check
Next.js's type-check worker OOMs at the default 4GB heap on Node 23 for
this project's type graph size. Bumps the heap to 8GB only for the
`next build` invocation inside `bun run build`.
Docker builds are unaffected — `next.config.ts` sets
`typescript.ignoreBuildErrors: true` when DOCKER_BUILD=1, which skips
the type-check pass entirely. This only fixes local `bun run build`.
No functional code changes.
Made-with: Cursor
* fix lint
* refactor(copilot): dedup getDocumentFormatInfo across copilot file tools
The same extension -> { formatName, sourceMime, taskId } mapping was
duplicated in workspace-file.ts and edit-content.ts. Any future format
or task-id change had to happen in two places.
Exports getDocumentFormatInfo + DocumentFormatInfo from workspace-file.ts
(which already owned the PPTX/DOCX/PDF source MIME constants) and
imports it in edit-content.ts. Same source-of-truth pattern the file
already uses for inferContentType.
Made-with: Cursor
* fix(sandbox): propagate empty-message broker/fetch errors
Both bridges in the isolate used truthiness to detect host-side errors:
if (response.error) throw new Error(response.error); // broker
if (result.error) throw new Error(result.error); // fetch
If a host handler ever threw `new Error('')`, err.message would be ''
(falsy), so { error: '' } was silently swallowed and the isolate saw
a successful null result. Existing call sites don't throw empty-message
errors, but the pattern was structurally unsafe.
Switch both to typeof check === 'string' and fall back to a default
message if the string is empty, so all host-reported errors propagate
into the isolate regardless of message content.
Made-with: Cursor1 parent 319e0db commit 2f93205
File tree
32 files changed
+1888
-784
lines changed- .cursor/rules
- apps/sim
- app/api
- files/serve/[...path]
- workspaces/[id]/pptx/preview
- lib
- copilot/tools/server/files
- core/config
- execution
- sandbox
- brokers
- bundles
- sandbox-tasks
- docker
32 files changed
+1888
-784
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
82 | 84 | | |
83 | 85 | | |
84 | 86 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
| 7 | + | |
12 | 8 | | |
13 | 9 | | |
14 | 10 | | |
| |||
22 | 18 | | |
23 | 19 | | |
24 | 20 | | |
| 21 | + | |
25 | 22 | | |
26 | 23 | | |
27 | 24 | | |
| |||
30 | 27 | | |
31 | 28 | | |
32 | 29 | | |
33 | | - | |
| 30 | + | |
34 | 31 | | |
35 | 32 | | |
36 | 33 | | |
37 | 34 | | |
38 | 35 | | |
39 | 36 | | |
40 | | - | |
| 37 | + | |
41 | 38 | | |
42 | 39 | | |
43 | 40 | | |
44 | 41 | | |
45 | | - | |
| 42 | + | |
46 | 43 | | |
47 | 44 | | |
48 | 45 | | |
49 | 46 | | |
50 | | - | |
| 47 | + | |
51 | 48 | | |
52 | 49 | | |
53 | 50 | | |
| |||
65 | 62 | | |
66 | 63 | | |
67 | 64 | | |
68 | | - | |
69 | | - | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
70 | 69 | | |
71 | 70 | | |
72 | 71 | | |
| |||
90 | 89 | | |
91 | 90 | | |
92 | 91 | | |
93 | | - | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
94 | 97 | | |
95 | 98 | | |
96 | 99 | | |
| |||
153 | 156 | | |
154 | 157 | | |
155 | 158 | | |
156 | | - | |
| 159 | + | |
157 | 160 | | |
158 | 161 | | |
159 | | - | |
| 162 | + | |
160 | 163 | | |
161 | 164 | | |
162 | 165 | | |
| |||
171 | 174 | | |
172 | 175 | | |
173 | 176 | | |
174 | | - | |
| 177 | + | |
| 178 | + | |
175 | 179 | | |
| 180 | + | |
176 | 181 | | |
177 | 182 | | |
178 | 183 | | |
| |||
205 | 210 | | |
206 | 211 | | |
207 | 212 | | |
208 | | - | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
209 | 216 | | |
210 | 217 | | |
211 | 218 | | |
| |||
225 | 232 | | |
226 | 233 | | |
227 | 234 | | |
228 | | - | |
| 235 | + | |
| 236 | + | |
229 | 237 | | |
| 238 | + | |
230 | 239 | | |
231 | 240 | | |
232 | 241 | | |
| |||
262 | 271 | | |
263 | 272 | | |
264 | 273 | | |
265 | | - | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
266 | 277 | | |
267 | 278 | | |
268 | 279 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
48 | 52 | | |
49 | 53 | | |
50 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | | - | |
| 8 | + | |
13 | 9 | | |
14 | 10 | | |
15 | | - | |
| 11 | + | |
16 | 12 | | |
17 | 13 | | |
18 | 14 | | |
| |||
26 | 22 | | |
27 | 23 | | |
28 | 24 | | |
29 | | - | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | 25 | | |
64 | 26 | | |
65 | 27 | | |
| |||
241 | 203 | | |
242 | 204 | | |
243 | 205 | | |
244 | | - | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
245 | 211 | | |
246 | 212 | | |
247 | 213 | | |
| |||
0 commit comments