[nicolas-abril]

Draft: comp.ts is 64,306 ttok, 306 over the 64,000 cap. Opened to decide whether this is worth the tokens.

Follows #1288, which made calls with a known callback skip the closure by inlining the callee at each call site. That needed a fuel budget and couldn't handle recursive callees, so it was closed. This does the same thing in comp.ts only, with no fuel and no language change. The JS lane is unchanged.

What it does

A call that passes a lambda written in the source, in a declared live parameter position, runs a specialized def callee$cbN instead of building a closure:

• The specialized def: its body is the callee's with the lambda substituted for that parameter, and the lambda's captured variables become its leading parameters.
• Sharing: specializations are keyed by the lambda's code with captures abstracted and types dropped, plus its parameters' layouts, so the same lambda code at different call sites shares one def.
• When they're created: only while the book is read. During emission, calls only look them up.
• Beta reduction: anf reduces the callback's applications, binding each argument.

Termination, by construction. A substituted lambda carries a marker span (CB). A lambda that contains one is never used as a key, except the substituted lambda itself passed on unchanged. So every key is a pair of a callee and a lambda written in the source, and both sets are finite. A callback that a recursion wraps, as in go(n - 1, y => f(g(y))), keeps its closure.

Results

The fixtures from #1288 (by @mizchi) are included unchanged. Each prints 42 in the interpreter, JS and native (1 and default threads), and builds no closures beyond the runtime's baseline (main builds 9 and 2). known_callback_shapes.bend covers:

• mutual recursion passing the callback unchanged
• a callee that ignores its callback
• a callback before a tuple-destructured parameter
• Array and List captures
• two lambdas in one call
• a recursion that wraps its callback

Speed (M4, 10⁸ steps, median of 7, run alone; the same at 1 and default threads):

program	main	this	hand-written
helper called each step with a fresh capturing lambda	0.25 s	0.07 s	0.07 s
callback passed down 10⁸ steps, called once	0.07 s	0.07 s	0.07 s

Regression (1,644 programs in tests/, demos/ and bench/, main vs this):

• Unchanged: every JS file is identical, no build fails, and no bench's C changes.
• Changed C: 47 programs (40 tests, 7 demos). All 40 tests give main's output natively and match their #| lines, and all 7 demos build.
• Closures: across the changed programs, closure builds go from 1,820 to 1,548. Their C grows 1.3%.

Compile time (median of 3): app_slash_boss_3d 1.13 → 1.15 s, netcode walkers_test 0.32 → 0.35 s; others unchanged.

Not covered

• IO continuations: IO.bind applies its callback to more arguments than the lambda has, which would need let-floating. Maybe.bind, Result.bind and String.word are covered.
• Only one lambda per call is specialized.
• A callback in a raised (undeclared) parameter position is skipped.

Cost and how it could fit the cap

+1,174 ttok:

• cb_spine: 757 in total (rejection checks 122, key and captures 278, building the def 275, redirecting the call 82)
• def_inst, which instantiates the callee's body and type: 203
• anf beta reduction: 146
• hooks and File fields: ~130

A review pass on cb_spine found only about 6 safe tokens to cut. Getting under the cap would need one of these:

1. Key by the lambda's source position instead of its code. This saves about 100–150 tokens. Identical lambdas at different call sites would no longer share a def. A source lambda that ends up in two type contexts would then need its own guard.
2. Drop callbacks after a matched parameter (def_inst's Mat case). This loses String.word.go-style matchers that take a continuation.
3. Free tokens elsewhere in comp.ts.

Open risks

• Stale annotations: annotations above the substituted parameter still mention it. Emission reads inner annotations, so this hasn't mattered, but a U32 match table could read the stale type.
• Capture identity: captures are identified by probe index plus name, because term_higher copies probe variables. A collision cancels the specialization; it doesn't miscompile.

Not run: the cluster gates and CUDA.

🤖 Generated with Claude Code

[mizchi]

Thanks for this. I rewrote my particle sample to fit this change and confirmed performance comparable to the optimized version from #1288.

With the original source, this PR reduced the update closure segments from 14 to 12. After adapting the sample, they're down to 0.

[nicolas-abril]

Rebased on main 7896791 and pushed three commits on top of the original two.

Bugs fixed

1. Wrong output in C: the spec key had no types, so the same lambda code capturing a U32 in one call and a String in another shared one def, typed by the first caller. ap with y => (y, b) printed (2, "") instead of (2, "hi"). The key now includes the lambda's types. Covered by tests/run/known_callback_layouts.bend, which fails on the old head.
2. Build never finished: an erased type argument inside the lambda (go(A, p, f(x), y => id(A, y))) went into the key as a probe number that changed on every walk, so specs were minted forever. Free variables are now keyed by position, and a spec is only made when all of the lambda's free variables are runtime captures with known types. Covered by tests/run/known_callback_edges.bend, along with the String.word cycle, swapped positions, two-level pass-through, local and returned closures, and polymorphic and directly applied lambdas.
3. Leak: compile_book left the callee$cbN defs in the caller's book. file_new now works on a copy.

Simplifications: the key is built in one pass, specs register themselves (no file_book hook), and an unreachable branch in anf's beta step plus the nested ternaries in def_inst are gone.

Not fixed: annotations above the replaced parameter still mention it. I found no case where this matters.

Numbers (macOS)

• Gates: repo 53/54, ping 55/55, test 1618/1618, perf 124/124. The only failure is the cap: comp.ts is 64,400 ttok (main 63,311, +1,089, cap 64,000).
• All 17 bench/runtime programs give the same output on main and this branch, in both C and JS.
• M4 target: 250.7 → 69.6 ms.
• walkers_test compile: 323 → 374 ms.

This stays a draft until the cap question is settled: either a cap raise, or about 400 ttok cut elsewhere.

Unsupervised AI review: written and checked by an AI agent without line-by-line human review.

[nicolas-abril]

Superseded by #1866, which gets the same speed by inlining instead of specializing defs, at +889 ttok instead of +1,136.