Repository navigation
feat(chat): show the picture attached to a Hermes prompt - #2482
Merged
Merged
Conversation
Hermes v0.21.5 keeps no picture bytes in state.db: a prompt with a picture is stored as its caption, a `[Image attached at: <path>]` hint per picture and a `[screenshot]` stand-in for each. Chat now reads that block as Hermes's own history does (tui_gateway/session_history.py) and shows each hint as a Markdown image of that local file, so the chat's local Markdown image path (#2435: /api/fs/stat then /api/fs/file, raster and 20 MiB checks, a link otherwise) draws it. No new route. Any other hint (a URL, a relative or Windows path, a malformed block) stays as written.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 18 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
devswha
added a commit
that referenced
this pull request
Oct 31, 2026
…window (#2483) ## Change Closes #2478. Copilot CLI interns every picture of a turn in its session file as base64 (`session.binary_asset`), so one turn with a few pictures can pass the 16 MiB page window (`TRANSCRIPT_WINDOW_BYTES`). With no turn start in that window, the newest page began mid-turn and showed only the answer. The older page was also capped at one window (`MAX_PAGE_BYTES`), so it held the prompt and its tool calls with **empty output**: their results sat on the other side. - A Copilot page now widens back to its turn's start, the newest page included, up to four windows (`COPILOT_PAGE_BYTES`, 64 MiB). Past that it starts mid-turn, as before. - Widening searches only each chunk it adds, plus the rest of the line that chunk cuts (read forward 1 MiB at a time, stopping on an empty read), then reads the page once from the start it finds. - Other agents' pages are unchanged: their cap is still one window, so they never widen. Own PR from `main`, not stacked on #2479. `git merge-tree` against #2479, #2481 and #2482 is clean in either order. ## Validation - Failing first: the new `server/copilot.test.ts` case uses the real recorded `copilot-recorded.jsonl` turn, after an earlier turn. Two interned pictures go between its tool call and the call's result (over one window together). On `main`, the newest page held only the answer. With the fix, the newest page (`prompts: 1`) equals the whole-file parse of the pictured turn, tool output included, and the older page holds the earlier turn. - `bun run check fast` on 893d55c: all steps ok, unit **9828 pass, 0 fail, 41 skip**. - Cost per poll (merge-pass finding, fixed in 9bfb005): the first fix re-widened the newest page on every append. The newest Copilot turn that has outgrown its window is now kept between polls: - It keeps the turn's start, the read point, the 64 bytes before it, and the complete lines. Each picture record is checked whole once, then kept only as its byte length behind a per-process mark, so later records keep their exact offsets. - A poll reads only the appended bytes. - It reads afresh on another history, a changed tail, a new prompt, or a turn grown a window past the cap. - It is released with the pane's other per-path caches, and bounded by bytes as well as count: least-recently-read entries are evicted while the kept lines pass one capped turn (`COPILOT_PAGE_BYTES` + a window). Picture records count only as their length. - Measured on a 55 MB file (5 pictures of 11 MiB of base64, 5 small appends): | Code | First read | Read per append | Time per append | Turns on newest page | |---|---|---|---|---| | `main` | 87.1 MiB | 16.1–16.1 MiB | 22–52 ms | 1 (turn split: answer only) | | 0b5ce2f (first fix) | 200.0 MiB | 129.0–129.0 MiB | 247–341 ms | 2 | | 9bfb005 (this round) | 200.0 MiB | 741 bytes | 0–1 ms | 2 | The new test counts bytes read with a `readSync` spy (no timing asserts): under 64 KiB per append. It read 45 MB against 0b5ce2f's code. - Between the cap and the cap plus one window, a warm cache may hold up to one window more of the same turn than a fresh read would. This hysteresis rescans once per window of growth instead of on every poll. - Known limit, like the existing `liveScans` cache: the cache trusts Copilot to append. An earlier line rewritten in place, with the 64 bytes before the read point unchanged and the file grown, is not noticed. ## Reviews - omo-native-code-reviewer (cache round, e59939ff): REQUEST_CHANGES on H1, M1, M2 and L1. All four are the same findings as Codex's and are fixed in 9bfb005 (the reviewer read that diff and found them addressed). It also verified that tool-output refs resolve across set-aside lines, that partial and UTF-8-split lines and held cursors match a fresh read, and that malformed asset lines still make the strict reader throw. - omo-native-code-reviewer (first fix): **PASS / APPROVE**. A page walk over 27, 40 and 72 MiB turns, and sub-agent pictures, matched the whole-file parse. Other agents' paging is unchanged. - MEDIUM (per-poll `Buffer.concat` cost): fixed. The page is now read once. - MEDIUM (the test never checked the older page): fixed. An earlier turn is first, and both pages are asserted. - Codex Terra 6.1 fast xhigh (cache round, 0064d2ac): HIGH (a new prompt plus more than a window appended between polls was missed) fixed; HIGH (unbounded, never-released cache) fixed with the cap rescan, 4 entries and pane release; LOW (forged marker) fixed with a per-process nonce; HIGH (rewrite-and-grow) kept as the documented known limit above. Final round on 9bfb005: **PASS**. - Codex Terra 6.1 fast xhigh (first fix): - Round 1: MEDIUM (a `session.binary_asset` skip in `copilotRecords` bypassed strict validation). The skip is removed; it is not needed for the cost. - Delta round: HIGH (the forward scan could loop on an empty read after truncation). Fixed. - Final round: **PASS**. - Known limit, unchanged: a single record longer than the cap still starts a page mid-line. Codex found this pre-existing at the old 16 MiB cap; A 64 MiB single record is far beyond a Copilot picture as seen here, since Copilot resizes images before interning them.
Owner
Author
|
Merging ae5912f on the maintainer's OK of the screenshots (brought into 0.4.22 at his request): a Hermes prompt's attached picture ('[Image attached at: ]' with '[screenshot]') now shows above the words in Chat, as a normal Markdown image through #2435's existing local image rules and file API (no new route). Strict review by the merge pass (APPROVE, own repros through the real parser): only absolute paths, accepted less widely than a typed Markdown image; |
devswha
added a commit
that referenced
this pull request
Oct 31, 2026
Merge main and fold the changelog fragments of every PR merged since the last refresh into 0.4.22: - #2490 (Fixed): Chat keeps messages mounted when the pane's skill names load, by @nocstah - #2483 (Fixed): a Copilot turn stays whole when its pictures pass the page window - #2479 (Added): Chat shows the pictures in a Copilot CLI session - #2482 (Added): Chat shows the picture attached to a Hermes prompt #2480 (site only) had no fragment.
devswha
added a commit
that referenced
this pull request
Oct 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
Part of #992 (pictures in Chat), after #2479 (Copilot) and #2481 (Grok). Hermes v0.21.5 keeps no picture bytes in its store (checked on a real recording:
state.dbmessageshold no base64 or data URL, andvision_analyze's result is text only). A prompt with a picture is stored as its caption, an[Image attached at: <path>]hint per picture, and one[screenshot]stand-in per picture (agent/image_routing.py,agent/session_persistence.py).What changes: the chat reads that block the way Hermes's own history does (
tui_gateway/session_history.py) and shows each hint as a Markdown image of that local file. The chat's existing local Markdown image path then draws it: #2435'smarkdownImagePreview, which stats via/api/fs/stat, requireskind: image, caps at 20 MiB, fetches via/api/fs/file, and falls back to a link otherwise. No new route, no server file-serving change, no new UI. This is the same reach a user-typedin that bubble already has.[screenshot]. Hermes writes that stand-in for each picture it actually attached, so typed look-alike text stays text. URL hints, relative or Windows paths, malformed blocks and mismatched counts stay as written.localImageTargetreads back the exact path.Validation
server/fixtures/hermes-pictures-0.21.5-rows.json, the rows of a session recorded on this PC withhermes chat -q ... --image red.png(the vision tool then also readblue.png). Paths are redacted.server/hermes.test.tscase failed (the raw hint text was returned), then passed. It also covers URL, relative, Windows, trailing-text, placeholder-mismatch, no-placeholder, open-fence and[-in-name inputs, and a real skill-invocation fixture with a picture.bun run check faston ae5912f: all steps ok, unit 9828 pass, 0 fail, 41 skip./api/fs/statand/api/fs/fileanswered for the recorded path): the picture decodes in the prompt bubble on desktop and phone, with 0 page errors.Reviews