Skip to content

SB3a: facts and roles on tokens; no string notes - #570

Open
sf624 wants to merge 4 commits into
mainfrom
sb3a-token-facts
Open

sf624 wants to merge 4 commits into
mainfrom
sb3a-token-facts

Conversation

@sf624

@sf624 sf624 commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

SB3a of the "one source per fact" plan (docs/design/sources.md, stage SB, item 3; rows 6, 14, 18, 27; M3). SB3c (the row record per token) and SB3d (the ui texts out of render.ts) follow in other PRs.

What changed

  • Token.role: the colour role of the token's kind, from one table (src/roles.ts, KIND_ROLES), which is what media/panel.css's .kind-* rules write out (a test checks them against it). ROLES/Role moved there from src/ui/themeTokens.ts (re-exported). render.ts KIND_ROLE and readingSteps' ROLES are untouched (SC10).
  • Token.facts: the ids of the catalog's facts said at the token's node, in the order said (markTokens, src/tokenmarks.ts, from the anchors cx.say/cx.note already record; a fact at a node with no token of its own goes to the innermost token around it; one about the line to none). Over the corpus: 904 anchors match a token exactly, 32 fall back to the innermost, 42 have no token on the line.
  • No string notes: the string overload of cx.note, plainNote and the continuationNotes string list are gone; marked is private to src/messages.ts. test/fixtures/note-strings-known.json is empty.
    • New catalog facts: redirections-in-order, stderr-to-original-stdout, heredoc-end-line, heredoc-body-line, statement-continued, read-ifs-separators, yq-flavour, yq-compat, host-substitution, github-file, the host region notes (dockerfile-*, jenkinsfile-groovy-interpolation, python-*, yaml-folded-block), and one entry per idiom (src/messages-idioms.ts, with the manual sections each is about).
    • A style check's and a dialect issue's notes are made by the catalog at their tables' levels (styleNote from STYLE_LEVELS, problemNote from PROBLEM_LEVELS, which gains parse: 'info' for the zsh-syntax note, the severity it always had as a problem); their texts stay in their modules, as the design keeps them.
    • yq-flavour, yq-compat, host-substitution, github-file have real ids, but their wording is still built outside the catalog (recorded in sources.md; SB3c).
  • 2>&1 >file: redirectOrder (src/idioms.ts) gives the descriptor table as data; the catalog says it. cmd 2>&1 >file is a caveat (Info) with the rewrite >file 2>&1 as quick-fix data (a P3 fix), checked in bash by test/rewrites-corpus.test.ts (new states in test/support/rewrite-states.ts). No caveat where the command's original stdout is a pipe or a command substitution (stdoutTo, src/semantics/env.ts, from the env chain: a stage before |, $( ), <( ), a coprocess, a group or loop piped on): the table says "only the error messages go into the pipe" / "make the substitution's value". I widened M3's "/dev/null before a pipe" to any file there: the order is the only way to send stderr alone. |&'s implicit 2>&1 is applied after the stage's redirections (3.2.3, stderrPiped), so f 2>&1 >o |& x says stderr → o. Fixed on the way: cmd 2>&1 >o 2>e and cmd 2>&1 >o |& x no longer claim stderr goes to the original stdout.
  • "In short": problemSections is now problemCodes (fact ids; a host's dockerPipefail counts as its fact), and each catalog entry names the sentence clauses it says (says, read by clausesOf). A test sorts every problem citing 3.2.3 or 4.3.1.
  • The panel's yq dropdown reads the yq-flavour fact instead of matching the note's text yq:.
  • Small moves for room/one source: src/messages-values.ts (value texts, re-exported), spanWithin/sameSpan in src/noteanchors.ts (reading.ts and hoverParts.ts had copies).

Explanation changes (snapshot of the corpus, both languages, with sentences)

  • 388 lines: the notes that were plain text now start with ℹ (idioms, here-document lines, continued statements, yq, the redirection table, host notes). No other text change in them.
  • 8 lines × 2 languages: the 2>&1 >file sentence moved out of the table note into its own caveat: "ⓘ Caveat: 2>&1 comes before >build.log, so stderr goes to the original stdout, not to the file. To send both to the file, write >build.log 2>&1".
  • 9 lines × 2 languages: "In short" now keeps "its status is that of …" where the line's problem is pipeline-subshell-variable.
  • No token text changed.

Tests

  • Full npm test locally (bash 5.3, bash-ast, browsers): all pass but the two ps layout tests of test/examples-layouts.test.ts, which fail here because of a stray process on this machine (bash … /tmp/explain-bash-argv-5kqVAQ/0/script.sh, running for a day, whose TIME field 1-00:09:37 the layout regex does not expect); unrelated.
  • New: test/token-facts.test.ts (roles, facts on tokens, the panel.css roles, the redirection caveat and the pipe case, bash runs of the four orders); test/linesentence.test.ts (the 3.2.3 clause kept for a subshell problem; CLAUSE_FACTS classification).

Open

  • The ~130 inline token texts carry no fact id; the 17 PROSE_MARK glyphs in them stay until a renderer can draw a token's level from its facts (SB3c).

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

sf624 and others added 4 commits November 1, 2026 12:58
Each token says its colour role (`Token.role`, from its kind: src/roles.ts,
the table panel.css writes out) and the ids of the catalog's facts said at
its node (`Token.facts`, `markTokens`). Every note is a catalog fact, or a
style check's or a dialect issue's made by the catalog at its table's level
(`styleNote`, `problemNote`): the string overload of `cx.note`, `plainNote`
and the `continuationNotes` string list went, and SA5's known list is empty.

The order of redirections is data (`redirectOrder`) said by the catalog
(`redirections-in-order`); `cmd 2>&1 >file` is a caveat at Info with the
rewrite `>file 2>&1` (`stderr-to-original-stdout`), none where the command
writes into a pipe, which then gets only the errors. The sentence of a line
compares the problems' facts (`CLAUSE_FACTS`, `problemCodes`), not their
manual sections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t around a range); the PR number in the docs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…`|&` after the redirections, `says` on the catalog entries, zsh syntax at info)

- No `2>&1 >file` caveat where the command's original stdout is a pipe or a
  command substitution (`stdoutTo`, src/semantics/env.ts, from the chain's
  links): `$( )`, `<( )`, a group or a loop piped on. Before, only a stage
  of a pipeline itself; `err=$(f 2>&1 >/dev/null)` got a fix that breaks it.
- `|&`'s implicit `2>&1` comes after the stage's redirections (3.2.3,
  `stderrPiped`): `f 2>&1 >o |& x` sends stderr to `o`.
- The clauses a fact says are a field of its catalog entry (`says`,
  `clausesOf`), not a list beside the sentence's texts.
- zsh syntax a line cannot be read with is information, as the Problems
  list always showed it.
- The rewrite's bash check takes `/dev/null` too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
github-pages — 5be582cb Deployed Nov 1, 2026 by sf624 via Deploy to GitHub Pages #2005
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant