fix(js): handle bun test --coverage and --bail

[HenriqueSchroeder]

Follow-up to #1374, closed in favour of #988. Picks up what was agreed in #1374 (comment): bun test --coverage and --bail handling, plus the bun 1.3.14 fixtures develop's bun_* set does not reach yet.

All output below is a real bun 1.3.14 run through a build of this branch, compared against a build of develop at c356374.

--bail reads as a complete run today

bun prints no count lines when it aborts, and the notice was being dropped, so what is left reads as a finished two test run:

develop

SUMMARY:
  Ran 2 tests across 1 file. [17.00ms]

this branch

SUMMARY:
  Ran 2 tests across 1 file. [18.00ms]
  Bailed out after 1 failure

--coverage silently does nothing today

The table was dropped whole, and the snapshot line with it:

develop

SUMMARY:
   2 pass
   1 skip
   1 todo
   1 fail
  Ran 5 tests across 1 file. [15.00ms]

this branch

SUMMARY:
  File         | % Funcs | % Lines | Uncovered Line #s
  -------------|---------|---------|-------------------
  All files    |  100.00 |  100.00 |
   src/math.ts |  100.00 |  100.00 |
  -------------|---------|---------|-------------------
   2 pass
   1 skip
   1 todo
   1 fail
   1 snapshots, 3 expect() calls
  Ran 5 tests across 1 file. [14.00ms]

Both snapshot shapes are covered: snapshots: +1 added on the run that writes one, and the N snapshots, M expect() calls variant bun prints once the snapshot exists.

How the table is tracked

From its header (File | % Funcs | ...) to the next blank line, not by counting column separators. Counting | false-positives on any source frame holding a || b, since bun prints frames as NN | source, and that eats the failure's own context. test_bun_source_frame_with_pipes_is_not_coverage pins that shape.

Rows are capped at CAP_LIST. One row per file makes the table the only part of the summary that grows with the repo, and the aggregate row comes first, so a capped table still carries the repo-wide numbers:

  All files   |  100.00 |  100.00 |
   src/f0.ts  |  100.00 |  100.00 |
   ...
  ... +14 more coverage rows
   30 pass
   0 fail
  Ran 30 tests across 1 file. [15.00ms]

Measured on green --coverage runs, raw bun against this branch:

project	raw	develop	this branch
1 file	975B	71B (93%)	622B (36%)
30 files	1398B	71B (95%)	874B (38%)
200 files	7556B	73B (99%)	897B (88%)

develop's numbers are higher because it throws the answer away. The cap is what keeps this bounded: filtered size is flat at roughly 900B whatever the repo size, so the reduction grows with the project and the small-project case is small only because there was little to cut.

Fixtures

Real bun 1.3.14 captures, named after what each one carries:

• bun_test_bail_raw.txt, the aborted run with no counts
• bun_test_coverage_raw.txt, the table plus the N snapshots, count variant
• bun_test_snapshot_raw.txt, a mixed run with skip, todo and snapshots: +1 added
• bun_test_crash_raw.txt, a run killed by SIGKILL, which prints the banner and nothing else
• bun_test_many_failures_raw.txt, 40 failures in one run, enough to reach MAX_RUNNER_FAILURES

One note on the SIGKILL fixture, since it is narrower than it sounds: exit code propagation does not run through extract_test_summary, so the fixture pins the other half of that case, which is that the raw tail fallback surfaces the banner instead of a summary that reads clean. The exit code itself I checked end to end, 137 from both raw bun and rtk on the same killed run.

Testing

cargo fmt --all --check, cargo clippy --all-targets with zero warnings, cargo test --all at 4120 passing, cargo test --all -- --ignored at 8 passing. Both commits are green on their own.

Real runs against a bun 1.3.14 project, with a release build: plain, --coverage and --bail on red and green runs, exit codes 1 and 0 where expected, 137 on the SIGKILLed run, the 30 file and 200 file tables for the cap, and bun run, bun install and bunx for regressions. Startup is 2.6ms average over 20 runs of the release binary.

[rtk-wshm-sync-bot]

wshm · Automated triage by AI

📊 Automated PR Analysis

🐛 Type	bug-fix
🟢 Risk	low

Summary

Fixes bun test output summarization to properly surface --coverage tables and --bail abort notices, which were previously silently dropped from the summary. Adds row capping for the coverage table (bounded growth regardless of repo size) and handles both snapshot reporting line variants, backed by real bun 1.3.14 fixture captures.

Review Checklist

• Tests present
• Breaking change
• Docs updated

---

Analyzed automatically by wshm · This is an automated analysis, not a human review.

[HenriqueSchroeder]

@KuSh the follow-up from #1374 is up: coverage, bail, and the fixtures you listed, on top of 36788f6.

[KuSh]

Round 1 — CHANGES REQUESTED

Claim: rtk bun test keeps bun's --coverage table (capped), the --bail notice and the snapshot count lines, which develop drops.
Scope: accept.
Ran: real bun 1.2.20, 1.3.14 and 1.4.2 × 12 cases (green, red, --coverage red and green, snapshot first write and second run, --bail, --bail=2, --bail --coverage, 200-file --coverage, coverageThreshold miss, no matching test, --coverage-reporter=lcov) × raw / develop c356374 / this head, plus a 41-file project with one partially covered file and two unhandled-error shapes on 1.3.14; invariants: under the hook (rtk rewrite → rtk bun test …, and rtk test bun test --coverage), over-cap + accounting, split streams (stdout and stderr captured apart), every decision; savings 26% (1-file green --coverage), 47% (red --coverage), 88% (200 files) vs 20% floor (CONTRIBUTING.md).
Pins: 12 decisions, 14 mutations, 4 survive on reachable input (cap and overflow notice, below).
Matrix: executed this round; pin check: executed this round.
CI @7d8420c4: not run yet (the workflow is waiting for maintainer approval).

Blocks merge (2)

1. src/core/runner.rs:971 — the capped coverage table has no way back to the rows it hides. On a green run nothing is stored for recovery, so on any project past 17 files the rows past the cap are gone. Bun sorts the table by path, so what survives is whichever files come first alphabetically, not the ones that need attention. Real 1.2.20 run, 40 fully covered files plus src/zeta.ts at 50% funcs / 83% lines: the summary shows src/f10.ts to src/f26.ts, all at 100%, then ... +25 more coverage rows, and zeta.ts, the only row the agent ran --coverage for, appears nowhere and cannot be retrieved. The 200-file run hides 184 lines the same way, with no hint in either recovery mode (tee files or rtk recall). The other capped lists in the tree (gh, glab, eslint, rubocop, pytest) print a hint to the hidden part through crate::core::tee::force_tee_tail_hint. A correct fix makes the hidden rows reachable: on the 41-file project, src/zeta.ts is either shown or reachable through a hint printed in the output, on the 200-file run the 184 hidden rows are reachable the same way, and the shown rows plus the announced count still equal the table. Whether rows below 100% are also kept ahead of fully covered ones when over the cap is your call.
2. src/core/runner.rs:1547 — unpinned behaviour (the row cap and its overflow notice). Each of these mutations passes 35/35 module tests: showing one row fewer than the cap (<= → < at :971), showing three more (<= MAX_BUN_COVERAGE_ROWS + 3), announcing +0 more on a table of exactly 20 lines (> → >= at :1114), and dropping *rows = 0, which prints the notice twice on every capped run. test_bun_coverage_table_is_capped checks only that f0 is shown and f39 is not. The test must take real captured bun output over the cap and assert that exactly MAX_BUN_COVERAGE_ROWS table lines are shown, the notice appears once with N = table lines − cap, and a table of exactly MAX_BUN_COVERAGE_ROWS lines prints no notice. Each of the four mutations must turn it red.

Optional — will not hold merge

• src/core/runner.rs:1106 — push_coverage_overflow and its doc landed between is_bun_count_line's doc comment and the function, so rustdoc merges both comments onto push_coverage_overflow and is_bun_count_line has none.
• The notice counts the header and rulers as "coverage rows": the 200-file run hides 183 file rows and says +184, and the header and rulers take 3 of the 20 lines. Counting files only, or saying "lines", would match what the reader expects.
• The end-of-input push_coverage_overflow call covers a table with nothing after it. Bun always prints a blank line and the counts after the table, so no real run reaches it.

Checked and correct — no need to re-verify

• Exit codes equal raw bun's in all 36 runs (0, 1, and 1 on the coverageThreshold miss).
• Bailed out after 1 failure and after 2 failures survive on all three versions, with and without --coverage (bun prints no table after a bail).
• snapshots: +1 added on the first run and 1 snapshots, 7 expect() calls on the second, on all three versions.
• A console.log line containing || stays out of the table, and so do the source frames echoed with a failure.
• --coverage-reporter=lcov (no table) is unchanged from develop.
• On a coverageThreshold miss, bun exits 1 with 0 fail and no message. The table is the only thing that explains it, and it now comes through.
• The PR's 200-file numbers reproduce exactly: 7556 B raw → 897 B.

Next

Make the hidden coverage rows recoverable (1) and pin the cap and its notice with a test on real bun output (2).
Threads: 2 open (waiting on the author), 0 resolved this round.

[HenriqueSchroeder]

Both blockers are fixed in 6f79d51.

1, hidden rows. The table is buffered and filled in priority order: its structure, then the rows reporting a gap, then whatever still fits, each in bun's own order. To your open question: yes, rows below 100% now keep their place ahead of fully covered ones. What the cap drops goes through force_tee_tail_hint with offset 1, so what is written is exactly the dropped rows rather than the whole table, and "see remaining" is literal in both recovery modes.

Your 41-file case, real 1.3.14 run:

   src/zeta.ts |   50.00 |   33.33 | 5-8
  -------------|---------|---------|-------------------
  ... +25 more files
  [+25 hidden: rtk recall 6d78ce2c2e4a]

rtk recall returns those 25 rows, all fully covered, zeta not among them. The 200-file run shows 20 table lines, says +184, and recall returns 184.

2, pinning. bun_test_coverage_wide_raw.txt is that 41-file run, and the tests assert exactly MAX_BUN_COVERAGE_ROWS lines shown, the notice once with N = table lines - cap, and no notice at exactly the cap. All four mutations turn red: <= to < fails the at-cap test, cap-1 and cap+3 fail the equality, and not clearing the buffer fails the "once" assert. I also ran the inverse of the gap priority, which fails the zeta assert. One note: 1 => false does not compile under -D warnings, since it leaves coverage_row_has_gap dead, so the inverse is the form I used.

Optional, all three. The helper no longer sits between is_bun_count_line's doc and its signature. The notice counts files, not table lines: only file rows can be hidden now, so the 200-file run says +184 for 184 file rows. The end-of-input call stays, since it is what flushes the buffer, with a comment that only a truncated capture reaches it.

Savings on green --coverage: 36% (1 file), 48% (41 files), 87% (200 files).

[KuSh]

Round 2 — CHANGES REQUESTED

Claim: rtk bun test keeps bun's --coverage table (capped, rows below 100% first, the rest recoverable), the --bail notice and the snapshot count lines, which develop drops.
Scope: accept (round 1, frozen).
Ran: real bun 1.2.20, 1.3.14 and 1.4.2 × 14 cases × raw / round-1 head 7d8420c / this head 6f79d51 / this head merged onto develop f16beca (merge on 1.2.20 only: develop now links glibc 2.43, which the bun images lack). New cases: 41 files with one below 100%, 30 files all below 100%. Invariants: over-cap + accounting, the recovery hint followed under the hook in both recovery modes (rtk recall <id>, and tail -n +1 ~/…, which rtk rewrite leaves alone), split streams, every decision. Savings on 1.3.14: 39% (1-file green --coverage) to 91% (200 files), vs 20% floor (CONTRIBUTING.md).
Pins: 18 mutations of this push's decisions: 7 caught, 6 survive on input bun prints (below), 5 change nothing bun can print.
Matrix: executed this round; pin check: executed this round.
CI @6f79d518: not run yet (the workflow is waiting for maintainer approval). Locally, the merge with develop passes fmt and clippy, and every test except go_test_help_is_not_replayed_through_the_injected_json, which fails the same way on develop alone here.

Blocks merge (2, both inside round 1's set)

Round 1's first item is fixed (see "Checked and correct"). Round 1's four surviving mutations now fail. What remains is the new code that answers them, which no test pins yet:

1. src/core/runner.rs:1158 — unpinned behaviour (which rows the cap keeps when the rows below 100% alone exceed it, and in what order). Each of these passes 37/37: skipping the structure pass (0 => false && is_coverage_structure(line)), running the gap pass before the structure pass, not counting rule lines as structure, and emitting the kept rows reversed. On a real 30-file run with every file below 100%, the first three drop the closing rule, and the fourth prints the table upside down. The only capped fixture has a single row below 100%, so structure never competes for the budget. The test must take real bun output with more rows below 100% than the cap (30 files at 50%/66.67% gives a 34-line table) and assert that the table shown is exactly bun's first 19 lines plus its closing rule, followed once by ... +14 more files. All four mutations must turn it red.
2. src/core/runner.rs:1711 — unpinned behaviour (what the hint stores). Storing hidden[1..] instead of hidden passes, and so does offset 2 instead of 1. Either way, the first hidden row can no longer be recovered. test_bun_hidden_coverage_rows_are_written_out checks the stored rows only when the hint is a tail -n +1 path, but a test build runs with the default recovery mode, which is recall. So only the hint's presence is checked. The tee branch also reads the hinted path as written, and tee mode prints it with a leading ~. The test must read back what the hint names, the recall entry or the tee file, on the 41-file fixture, and assert it is exactly the 25 hidden rows, starting with the first one. Both mutations must turn it red.

Optional — will not hold merge

• trim_coverage_table_to (runner.rs:1597) does not build the table its doc describes: on the 41-file fixture it returns the header, rules, All files and src/f0.ts to src/f23.ts, with no closing rule and no src/zeta.ts. The at-cap test still catches <= → <. A real capture of a 16-file project, which bun prints as exactly 20 table lines, would be a table bun actually prints.
• test_bun_coverage_table_survives (runner.rs:1654) still asserts !out.contains("more coverage rows"), the old notice text, so it can no longer fail. "more files" is the current one.

Filed as follow-ups

• #3431 (tee files named by the second, so two runs in the same second share one) also reaches bun-coverage: two --coverage runs within one second leave the first run's [see remaining: …] hint pointing at the second run's rows. This is pre-existing in the tee helper and #3431 already covers every caller that passes a fixed name, so it is not this PR's to fix.

Checked and correct — no need to re-verify

• Round 1's first item is fixed. On the 41-file project, src/zeta.ts keeps its place above ... +25 more files. Following the hint returns exactly the 25 hidden rows, in recall mode (bun 1.3.14) and in tee mode (bun 1.2.20). The same holds for 200 files (184 rows) and for 30 files below 100% (14 rows).
• Accounting: 200 files show 20 table lines (16 files) and announce +184 files.
• With 30 files below 100%, the header, both rules and All files survive, in bun's order.
• Round 1's optional items: the doc comment is back on is_bun_count_line, the notice counts files, and the end-of-input flush is commented.
• Exit codes equal raw bun's in all runs; the merge with develop prints the same as this head on every case.
• The table is buffered once and drained, so a second call cannot repeat it.

Next

Pin the cap's choice when the rows below 100% alone exceed it (1) and what the hint stores (2). With those two tests in, this merges.
Threads: 2 open (waiting on the author), 2 resolved this round.

[HenriqueSchroeder]

Both blockers were test gaps, no behavior change. Fixed in 2d91fa73, on top of a merge with current develop (f16becac).

Blocker 1 (runner.rs:1158). You were right that nothing pinned the choice when the rows below 100% alone overflow the cap: my only over-cap fixture had a single row with a gap, so structure never competed for the budget. Captured a real run of 30 files all at 50.00/33.33 (bun_test_coverage_all_partial_raw.txt, 34 table rows) and test_bun_coverage_cap_keeps_bun_order_when_every_row_has_a_gap now asserts the shown table is exactly bun's first 19 rows plus the closing ruler, followed once by ... +14 more files.

Blocker 2 (runner.rs:1711). The old test only checked the hint was present. test_bun_hidden_coverage_rows_are_recoverable now follows it the way a caller would: it resolves rtk recall <hash> through a #[cfg(test)] helper in retriever.rs and compares the result with the rows that were dropped. Worth flagging how the offset mutation died: my first read-back decoded the stored blob and offset 2 still survived, because rtk recall prints slice_from_line(&full, row.shown_upto), which is exactly what the offset decides. Once the helper applied the same slice, the mutation went red. One gap left: the test build runs in recall mode, so the tail -n +N branch of the hint is written but not exercised.

All six mutations you listed are red now, each one built and run on its own:

mutation	test that fails
skip the structure pass	..._cap_keeps_bun_order_when_every_row_has_a_gap
gap pass before structure	same
rulers not counted as structure	same
emit kept rows reversed	same + ..._hidden_coverage_rows_are_recoverable
store hidden[1..]	..._hidden_coverage_rows_are_recoverable
offset 2 instead of 1	..._hidden_coverage_rows_are_recoverable

Both optionals are in. trim_coverage_table_to built a table bun never prints (no closing ruler, no partial row), so it is gone, replaced by a real 16 file capture that bun prints as exactly 20 rows. The stale !out.contains("more coverage rows") assert now checks more files.

Gate on the merged branch: fmt clean (root and proxy/), clippy --all-targets with zero warnings, cargo test --all at 4964 passing. The go_test_help_is_not_replayed_through_the_injected_json failure you hit does not reproduce here.

Real runs, bun 1.3.14: with 41 files, rtk recall returns the 25 dropped rows, first src/f22.ts, last src/f9.ts, zeta not among them; with 30 partial files the header, both rulers and All files survive in bun's order; with 16 files nothing is announced; exit codes stay 1 red, 0 green, 0 green with --coverage.

One number I want on the record rather than buried. Savings on green runs with --coverage:

project	raw	filtered	cut
16 files (exactly at the cap)	908B	862B	5%
30 partial files	1488B	971B	35%
41 files	1832B	946B	48%
200 files	7556B	947B	87%
red run with --coverage	932B	606B	35%

The 16 file case sits under the 20% floor in CONTRIBUTING. It follows directly from the shape you validated: when the table fits the cap it is kept whole, and at that size the table is nearly the whole output. I did not change it, since that behavior is now pinned by a test you asked for. If you want it under the floor there too, the option I see is collapsing the per file rows into All files when no row reports a gap and the table is below the cap. Your call.

[KuSh]

Round 3 — APPROVED

Claim: rtk bun test keeps bun's --coverage table (capped, rows below 100% first, the rest recoverable), the --bail notice and the snapshot count lines, which develop drops.
Scope: accept (round 1, frozen).
Ran: real bun 1.4.3 (plain output, and the quieter agent output under CLAUDECODE=1) and 1.2.20 × 15 cases × raw / round-2 head 6f79d51 / this head 2d91fa7 / this head merged onto develop 2ba9956, in both recovery modes (recall and tee), with every hint followed under the hook. The bun-specific code is byte-identical to the round-2 head, so the 1.3.14 and 1.4.2 runs from round 2 still apply. Invariants: over-cap + accounting, under the hook, every decision. Savings: 26% (1-file green --coverage) to 87% (200 files) vs the 20% floor (CONTRIBUTING.md); at exactly the cap, 22% in plain output and 5% in agent output, see your question below.
Pins: round 2's 18 mutations rerun against your tests: the 13 that real bun output can reach all fail now; the other 5 change nothing bun prints.
Matrix: executed this round; pin check: executed this round.
CI @ed174025: all green, 14/14 jobs (Linux, macOS, Windows) (ed17402 is 2d91fa7 merged with develop 1b47098, which I pushed through "update branch". The run on 2d91fa7 failed only a_clustered_h_is_answered_by_git_once on the macOS and Windows git 2.56 runners: it was merged with a develop from before #4578, the fix for that test.)

Checked and correct — no need to re-verify

• Round 2's two blockers are closed. Skipping the structure pass, running the gap pass first, not counting rules as structure, and reversing the kept rows each fail test_bun_coverage_cap_keeps_bun_order_when_every_row_has_a_gap. Storing hidden[1..], or using offset 2, fails test_bun_hidden_coverage_rows_are_recoverable.
• recalled_content takes the same path as rtk recall <hash> with no flags: load_by_hash, then slice_from_line(&full, row.shown_upto).
• The new fixtures have bun's own shape: path-sorted rows, column widths, a 34-line table for 30 files and a 20-line one for 16.
• Following the hint returns exactly the hidden rows, in bun's order, in recall and in tee mode, on this head and on the merge with develop: 25 rows on 41 files, 184 on 200 and 14 on 30 files below 100%. At 16 files the table is whole, with no notice and no hint.
• On bun 1.4.3, both bail notices, both snapshot lines, the table and the overflow notices come through in plain and agent output alike.
• Exit codes equal raw bun's in every run. The merge with develop builds and passes fmt (root and proxy/), clippy and the bun tests.

Your questions

• "If you want it under the floor there too, the option I see is collapsing the per file rows into All files when no row reports a gap and the table is below the cap. Your call." → No change. The floor is about what a filter cuts from output nobody asked to read. With --coverage, the table is the answer that was asked for, and at 16 files it is most of the output, so keeping it whole is right. Thanks for putting the number on the record.
• The go_test_help_is_not_replayed_through_the_injected_json failure is specific to my machine; nothing for this PR.

Next

Nothing — approving.
Threads: 0 open, 2 resolved this round.