Repository navigation
feat(terminal): draw herdr-core panes with libghostty, Kitty graphics pictures included, behind a Settings toggle - #2908
Conversation
… included, behind a Settings toggle
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 4 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…esize, frees its module objects and places wide cells per grapheme
|
Review round (omo + Codex Terra 6.1 fast, xhigh) on 0486e36: no HIGH from either. Resolved in the next commit:
|
…ts features, as a herdr-core server does
…rm does, and hand the pane back to xterm when libghostty fails mid-frame
…-renderer # Conflicts: # DESIGN.md # scripts/ci-browser.sh # server/static.test.ts # server/static.ts # src/components/PaneTerminal.tsx # src/components/SettingsDialog.tsx # src/lib/i18n/settings.ja.ts # src/lib/i18n/settings.ko.ts # src/lib/i18n/settings.zh.ts # src/lib/settings.legacy-fixture.ts # src/lib/settings.test.ts # src/lib/settings.ts
|
Second review round (omo, with its Codex Terra pass) on 14f0036, resolved in 8189fde and the merge of origin/main (d9bd543):
|
Deploying ovis with
|
| Latest commit: |
d9bd543
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://2ef89c75.ovis.pages.dev |
| Branch Preview URL: | https://feat-terminal-ghostty-render.ovis.pages.dev |
|
Review at d9bd543: omo CLEAR (with its Codex Terra xhigh pass), no HIGH. Remaining findings, left for follow-ups after two fix rounds:
|
|
Review at 312eb94: omo CLEAR (with its Codex Terra xhigh pass), no HIGH. Non-blocking notes for the follow-ups:
|
|
Merge note: GitHub required checks green at this head, omo CLEAR at this head, threads resolved. Opt-in renderer (Settings > Terminal > Draw with libghostty), off by default, herdr-core mode only; xterm stays the input, selection and search engine. The wasm build patch enabling kitty graphics is documented in the PR. |
…lication/wasm (#2953) ## What The terminal's WebAssembly module (Settings > Terminal > Draw with libghostty, #2908) was sent uncompressed and as `application/octet-stream`: 1,110,716 bytes on every first load. The build already writes `.br`/`.gz` copies next to it (#2908 added `wasm` to `scripts/compressed-assets.ts`), but `serveStatic` only negotiates precompressed copies for `/assets/*.js|css`. - `server/static.ts`: `/assets/*.wasm` is negotiated like js/css (br, gzip, identity, 406 when none is acceptable), immutable cache, and served as `application/wasm`. ## Validation - `server/static-encoding.test.ts`: new case serves a wasm fixture as br and gzip with `application/wasm`, and identity without encoding; it fails on main (`content-encoding` null). - `bun test server/static-encoding.test.ts server/static.test.ts`: pass. - `bun run check fast`: pass (exit 0) on this head. - Manual, isolated hc 0.1.17 core + this branch's server on loopback, setting on, headless Chromium with a fresh profile: - `curl -H 'Accept-Encoding: br'`: 200, 344,901 bytes, `application/wasm` (was 1,110,716). - first open: the service worker fetches the module once, 344,901 bytes br; it lands in the build's cache (`ovis-v4-sheep-<build>`). - re-open: no network request for the module at all; the page gets it from the service worker.
…2948) Fixes #2934 Follow-up to #2908. ## What was wrong With Settings > Terminal > Draw with libghostty on, libghostty gave each code point of an emoji sequence its own wide cell. For `ABC👨👩👧XYZ` it drew `XYZ` at column 9, while xterm (cursor, selection, links, and herdr's grid via `terminalWidths.ts`) has it at column 5. Skin tones, flags, keycaps, subdivision flags and `⚠️ ` (emoji-presentation selector) drifted the same way. ## Fix `src/lib/ghostty/vt.ts`, `write()`: - libghostty's grapheme clustering (DEC mode 2027) stays off. Without it libghostty lays out everything except emoji sequences as herdr and xterm do. Turning it on everywhere also clusters what they keep apart: Bengali and Devanagari vowel signs, Indic conjuncts (`क्ष`), Thai SARA AM. Review caught that. - Each code point that joins an emoji is printed with mode 2027 on: ZWJ and the pictograph after it, VS16, skin tones, the keycap mark, tags and regional indicators. The scan is linear in the bytes written: 1 MB of Hangul in 17 ms, 1 MB of ASCII in 10 ms. - xterm ignores mode 2027, so a program's own 2027 on (`CSI ? … 2027 … h`, or `r` restoring a saved one) is turned off again right after. A small CSI state machine finds it across writes, through C0 controls or DEL inside the sequence, and across `reset()`. - A grapheme longer than the 120-byte read buffer is read again with the size libghostty asks for, instead of being drawn as a blank. ## Validation - `bun test src/lib/ghostty/vt.test.ts src/lib/terminalWidths.test.ts`: 39 pass. With `main`'s `vt.ts`, 9 of the new tests fail. - `bun scripts/terminal-ghostty-demo-regression.ts`: a new assertion on the real DOM checks that the family is one `.ghostty-wide` cell and that the text after it starts at xterm's column 5. It fails on `main` (`[ "👨", "👩", "👧" ]`) and passes here. - `bun run check fast`: pass (typecheck, build, unit tests) - Screenshots (demo fixture, herdr-core, libghostty on; 390 and 1280 px, light and dark) were checked by eye. Before, the family showed as three emoji and the flag as `K R`; after, each sequence is one two-column cell. - No changelog fragment: this fixes the unreleased #2908 feature, whose own fragment covers it. ## Not in this PR - An emoji followed by U+FE0E (`⚽︎`): herdr 0.9.5 and hc 0.1.17 give it one cell, while xterm's widths and this PR give it two. #2954's lane handles it after this PR merges and will flip the `text-presentation selector` test in `vt.test.ts`. ## Known remaining differences (LOW) - Each case below was found in review, and on `main` each is either the same or worse: - A style change inside an emoji sequence (`👨 SGR 👩`): libghostty still joins it, even with mode 2027 off. - A lone regional indicator. - A ZWJ sequence past libghostty's grapheme storage limit (about 34 people). - Byte-array writes that split a joiner's UTF-8 between two calls. The app writes strings.
…let screen readers read them (#2959) ## What Settings > Terminal > Draw with libghostty (#2908) cost more than plain xterm on a busy pane, and screen readers read nothing from it. - **xterm stops redrawing the rows libghostty covers.** xterm keeps parsing every byte (its buffer still serves input, selection, search and the input row) and keeps firing its render events, but its DOM renderer's `renderRows` is a no-op while the layer draws (restored, with a full refresh, on dispose). Those hidden rows were about a third of the extra cost. - **Unchanged rows are not rewritten**: a row libghostty marks dirty that comes out with the same HTML is skipped. Row writes were already batched per animation frame. - **Screen readers**: the libghostty rows are no longer `aria-hidden` (pictures and cursor still are), and a visually hidden polite live region announces the lines a program writes, as xterm's screen reader mode does: the latest frame's new lines, at most 20, never the screen a pane opens on. xterm.js has no public way to stop its renderer, so this reaches `_core._renderService._renderer` the way `lib/terminalReplies.ts` reaches `_core.coreService`; without it, xterm simply keeps drawing as before. ## Measurements Isolated hc 0.1.17 core + this branch's server, headless Chromium, 390x844 touch, CPU throttled 6x, `seq 1 200000` in a herdr-core pane, runs alternating off/on (ABBA): | | finish | stalls > 50 ms | p95 frame gap | |---|---|---|---| | main, off | 7.5 / 8.4 / 9.7 s | 1 / 2 / 9 | 50-83 ms | | main, on | 10.9 / 11.3 / 12.6 s | 11 / 11 / 26 | 83-100 ms | | this PR, off | 9.9 / 10.8 / 12.4 s | 2 / 7 / 23 | 67-83 ms | | this PR, on | 9.6 / 10.5 / 11.1 s | 1 / 4 / 8 | 67-83 ms | | this PR final build, off | 12.1 / 16.4 s | 29 / 57 | 100-117 ms | | this PR final build, on | 10.1 / 12.9 s | 9 / 6 | 83 ms | (the machine was shared and loaded, so absolute numbers move between batches; within a batch on is now at or below off.) Real NVDA 2026.2 in the Windows 11 test VM, Edge 156, the pane printing two marker lines: with the setting on NVDA spoke `ECHOLINE-1 ECHOLINE-2 $`; with it off (xterm, screen reader mode off) it spoke nothing for the same output. ## Validation - `bun scripts/terminal-ghostty-demo-regression.ts`: new assertions: the screen a pane opens on is not announced, a line the program writes is, and xterm's hidden rows no longer receive it. They fail on main's overlay (nothing announced). - `bun run check fast`: pass (exit 0) on this head (merged with origin/main).
…#2976) ## What Settings > Terminal > Draw with libghostty (#2908) is on by default. - It still only draws on a herdr-core server (`terminal-history`, as the desktop app runs); on herdr and mirrored panes xterm.js draws exactly as before. - The setting stays, per device. A device that turned it off stays off: a stored `false` is kept, only a missing or invalid value means on. Note: since #2908 every settings save writes this key, so a device that saved any setting on a main build after #2908 also has `false` stored and stays off; released versions never had the key, so their users get the new default. ## Gates (step 1 checklist, measured on isolated hc 0.1.17 + this stack) | Gate | Result | |---|---| | Phone first load (slow 4G 1.6 Mbps/150 ms, CPU x4, fresh profile, median of 5) | off 7,510 ms / on 7,918 ms: +408 ms, within +0.5 s (wasm served br since #2953: 1.11 MB -> 345 KB, re-open fetches no wasm) | | Low-end phone frames (CPU x6, `seq 1 200000`, ABBA) | after #2959 on is at or below off: median finish 10.5 s vs 10.8 s, stalls > 50 ms 4 vs 7 | | Screen reader (real NVDA 2026.2, Windows 11 VM, Edge 156) | on: NVDA reads new output; off (xterm, screen reader mode off): reads nothing. Known follow-ups in #2963 | | iOS long-press copy (iOS 27 Simulator, Safari) | **provisionally equal**: neither xterm nor libghostty shows a selection/copy menu on a 1.6 s long-press (driven with simctl + AXe touch). Pending one manual check once Simulator components are installed on the test Mac. Copy keeps working through the key bar's copy button and OSC 52 (#2925) | ## Validation - `bun test src/lib/settings`: pass (the two tests that pinned the old default now pin the new one: missing/invalid means on, a stored `false` stays off, the key stays out of the whole-record save). - `bun scripts/terminal-ghostty-demo-regression.ts`: pass (herdr-core pane drawn with libghostty, herdr's own and off stay xterm). - `bun run check fast`: pass (exit 0) on this head
… itself (#3035) ## What libghostty now draws every pane: the Settings > Terminal > Draw with libghostty switch is gone, and a device that had turned it off is drawn by libghostty too. xterm.js stays the engine underneath (input, IME, selection, links, search, scrolling) and still draws by itself only where the layer cannot, so a pane never goes blank: - the libghostty module does not load or fails (old browser, CSP, out of memory): `capture.ts` / `overlay.ts` hand the pane back to xterm, as before; - an iTerm2 or sixel picture (Pictures in the terminal), which libghostty does not take (#2980): xterm draws that pane, as before. Changes: `terminalGhostty` removed from `Settings` (type, default, scope, sanitize, its `ovis:terminal-ghostty` key, telemetry projection), the Settings row and its en/ko/ja/zh strings, the `ghosttyRef` plumbing in `PaneTerminal.tsx`/`useTerminalMount.ts` (the capture always starts), and the docs (DESIGN.md, development.md, comments). An old tab's whole-record save that still carries the field is dropped by `sanitizeSettings` as any unknown field. Design follows Ghostty's own split (libghostty-vt as the terminal core, the app's own renderer), as in #2908/#2976/#3025; keeping xterm's renderer only as an automatic fallback matches how browsers keep a software path when a GPU path fails. ## Gates (herdr mode, isolated herdr test session, loopback, telemetry off) See the Validation comment below for the numbers of this head (phone first load vs xterm, re-open wasm bytes, CPU x6 runaway output, announcements). ## Validation - Failing first: in `scripts/terminal-ghostty-demo-regression.ts` a device with the old switch stored off is drawn by libghostty; on main it stays xterm (`waiting for .pane-terminal[data-renderer='ghostty']` times out), on this branch it passes. - `bun run check fast`: see below. - Browser regressions: see below. - winvm (native Windows, main 0df1012): `pr-check.ps1` incl. `windows-terminal-browser-qa.ts` PASS before this change; this change does not touch the mirror path.
What
Settings > Terminal > Draw with libghostty (off by default, this device only). On a PC that runs herdr-core, as the desktop app does, a pane is drawn by libghostty-vt (Ghostty's terminal core, MIT) instead of xterm.js's own text layer, and the pictures a program sends with the Kitty graphics protocol are drawn:
chafa -f kitty,timg -pk,kitten icat --transfer-mode=stream.xterm stays the engine underneath: it still takes every key, IME syllable, paste, click, selection, link, search and scroll, and still answers the program's queries. Its text and cursor layer is hidden; libghostty draws the same bytes, in the same order, as plain DOM rows over xterm's screen at xterm's cell size, plus a canvas per Kitty placement (below or above the text, as Kitty's z says). libghostty answers nothing: a reply it made to a replayed stream would reach the program as typing (herdr-core already answers size queries itself).
This complements #2871 (sixel and iTerm2 through
@xterm/addon-image, which has no Kitty graphics): libghostty has no sixel, the addon has no Kitty, and the two layers do not overlap.How
src/lib/ghostty/ghostty-vt.wasm: libghostty-vt at ghostty e95bed6 for wasm32, rebuilt byscripts/ghostty-vt/build.shwithkitty-on-wasm.patch. Upstream turns Kitty graphics off on freestanding wasm; the patch turns it on, refuses the file / temp-file / shared-memory media (no filesystem in a browser), and links ghostty's bundled wuffs PNG decoder. 1.11 MB (345 KB brotli). The build now writes .br/.gz copies beside it, but the server still serves precompressed copies for js/css only, so today the module goes out uncompressed (follow-up below). Notices for ghostty, wuffs and uucode in THIRD_PARTY_NOTICES.md.src/lib/ghostty/capture.ts: from the terminal's mount (setting on), every byte xterm is given is held; once the server lists its features and saysterminal-history(herdr-core), the overlay module is loaded and starts from those bytes. herdr's own server, a module that fails to load, or more than 32 MB held leave the terminal to xterm and never fetch the module.src/lib/ghostty/vt.ts,overlay.ts: the wasm wrapper (rows as styled runs, the render state's Kitty image list, palette from the xterm theme, scroll sync with xterm's own herdr-core history) and the DOM layer. Each cell carries its own grapheme and width, so a combining mark never shifts a wide cell.script-src 'self' 'wasm-unsafe-eval'(the same line feat(terminal): draw sixel and iTerm2 inline pictures behind a Settings toggle #2871 adds), so the page may compile WebAssembly; no string eval.Design reference: Ghostty's own split of a terminal core (libghostty-vt) from the renderer an app draws with, rather than a second terminal widget; everything already built on xterm keeps working while the drawing moves. Clean-room from ghostty's public MIT source.
Validation
bun run check fast: pass (exit 0) on 312eb94 (merged with origin/main 22a4b33, feat(terminal): draw sixel and iTerm2 inline pictures behind a Settings toggle #2871 and fix(test): read the header crumb apart from the session line in the reconnect regression #2937 included).check fullnot run here; CI runs the browser lane.bun scripts/terminal-ghostty-demo-regression.ts(new, added toscripts/ci-browser.sh): a herdr-core pane is drawn from its first byte (printed before the module loaded), a Kitty picture is drawn at its row and goes ona=d, a wide cell after a combining accent stays the CJK character (fails on the first head), ANSI red under a palette that names no ANSI colours is xterm's red (fails before 8189fde), typing still reaches the pane through xterm; on herdr's own server and with the setting off nothing is drawn by libghostty and the.wasmis never fetched.bun test src/lib/settings.test.ts server/static.test.ts server/static-encoding.test.ts scripts/compressed-assets.test.ts scripts/changelog-fold.test.ts: pass.lessalternate screen and back.kitten icatin its default mode gives up because herdr-core answers the DA1 query on the server before the browser's Kitty reply arrives, and hc gives the pty no pixel size; those are herdr-core-side.Follow-ups (this PR is step 0)
The end state is one rendering path: libghostty draws every terminal, on herdr-core and on herdr, and xterm's drawing code goes.
/assets/cache-first and carries a same-named file into the new build's cache);Then input, selection and search move to libghostty's key encoder, selection and search APIs, and xterm's parser and renderer are removed.