Skip to content

Replace parser tests with executable specs - #1359

Open
clauderic wants to merge 1 commit into
executable-specsfrom
parser-specs
Open

clauderic wants to merge 1 commit into
executable-specsfrom
parser-specs

Conversation

@clauderic

@clauderic clauderic commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Stacked on #1347, which is stacked on #1357. Review those first; this PR's base moves down the stack as they merge.

What are you adding in this PR?

The parser tests become executable, language-neutral specs, like the check specs in #1347: YAML files in specs/parser/, at the repo root. Each case is a template, the parses it's for, and the whole tree it parses to:

cases:
  - name: parses a drop
    source: "{{ x }}"
    parse: [html, liquid]
    ast:
      type: Document
      name: "#document"
      position: { start: [0, 0], text: "{{ x }}" }
      children:
        - type: LiquidVariableOutput
          whitespaceStart: ""
          whitespaceEnd: ""
          markupPosition: { start: [0, 2], text: " x " }
          position: { start: [0, 0], text: "{{ x }}" }
          markup:
            type: LiquidVariable
            rawSource: x
            position: { start: [0, 3], text: "x " }
            expression:
              type: VariableLookup
              name: x
              position: { start: [0, 3], text: x }
              lookups: []
              lookupModes: []
            filters: []
  • parse names the parses, not TypeScript functions: html is Liquid and HTML (toLiquidHtmlAST), and liquid is Liquid with HTML as text (toLiquidAST). When they give different results for a template, they're two cases.
  • ast is every node with every property. A case fails on a missing, extra or different one, so there's no partial matching and no snapshot.
  • A case whose parse throws has error instead: its message, and where it is.
  • A position is where its range starts, a 0-based [line, character] in UTF-16, and the text it covers, cut out of the template at the range. That's how the check specs give offenses.
  • src/specs.test.ts runs them with the rest of the suite. With UPDATE_SPECS=1, it writes each case's tree or error from this parser instead, so writing a case is writing its name, source and parse.
  • specs/parser/README.md covers the format, what the trees leave out (the template on every node, and the links between nodes, which the parser doesn't set), and errors.

The specs cover what the parser builds, not how. Another implementation can tokenize however it likes, as long as it builds the same trees.

The specs replace the tests they cover:

  • Before: liquid-html-parser had 982 tests.
  • After: it has 669 TypeScript tests and 415 spec cases, in 9 spec files.
  • 6 test files are deleted, and 3 more get smaller.

How the conversion was done, and checked

  • Every toLiquidHtmlAST and toLiquidAST call in the parser tests was recorded from the running suite: 591 parses, which became 412 cases, since many tests parse the same template.
  • Each case was re-run from its YAML and had to give the same tree, or the same error, as the recorded call.
  • A test was deleted only when every parse it made became a case, and every expect in it was about what those parses returned. As in Add executable check specs #1347, a script checks this with the TypeScript compiler API.
  • Statement, branch and function coverage of liquid-html-parser is the same before and after, except one branch of deepGet, which only tests use.
  • Three cases are new, for what no test checked: a description glued to @description, as in @descriptionText; a position after an emoji, which is two UTF-16 code units; and an error after emoji, on the second line. Every other error case is on the first line.

What stays in TypeScript

  • Tests of the tokenizers, the node factories, tolerant parsing and the tree builder.
  • Tests that parse with allowUnclosedDocumentNode other than its default (11 calls).
  • The completion-mode tests, which are skipped.

What's next? Any followup issues?

  • Add a Rust port of theme-check #1356, the Rust port, is stacked on this PR, and its parser passes every case.
  • LiquidHTMLASTParsingError has an unclosed property, the node left open, and the language server's HtmlElementAutoclosingOnTypeFormattingProvider reads it. The parser never sets it, so the specs leave it out, and the runner fails if an error has one.

What did you learn?

A position that ran past the end of the template couldn't be cut out of it, which is how #1357 turned up. An existing case recorded the same bug without anyone noticing: its unclosed if happened to end inside the template, on \n .

Tophatting

  • npx vitest run packages/liquid-html-parser/src/specs.test.ts runs just the 415 cases.
  • To see a spec fail, change any property or text in a spec file.
  • UPDATE_SPECS=1 npx vitest run packages/liquid-html-parser/src/specs.test.ts rewrites the specs, and changes nothing.

Before you deploy

  • My feature is backward compatible

🤖 Generated with Claude Code

@clauderic
clauderic requested a review from a team as a code owner October 21, 2026 00:47
@clauderic
clauderic added this pull request to stack #1360 October 21, 2026 00:48
@clauderic
clauderic force-pushed the parser-specs branch 2 times, most recently from a1ef941 to 134ae37 Compare October 21, 2026 12:47
@clauderic clauderic mentioned this pull request Oct 21, 2026
2 tasks done
The Liquid HTML parser gets language-neutral specs in specs/parser: a
YAML list of cases, each a template, the parses it's for (html, liquid
or both) and the whole tree it parses to, or the error it throws.
Positions are a 0-based line and UTF-16 character, and the text they
cover. specs/parser/README.md describes the format.

packages/liquid-html-parser/src/specs.test.ts runs them with the rest
of the suite. With UPDATE_SPECS=1, it writes each case's tree or error
from this parser instead.

Every parse the parser tests made became a case, recorded from the
suite, re-run from the YAML and compared: 591 parses, 412 cases. A test
was deleted when all of its parses became cases and everything it
asserted was a function of what those parses returned. Coverage of
liquid-html-parser is unchanged, except one branch of deepGet, which
only tests use. Three more cases cover what no test did: a description
glued to @description (@descriptionText), a position after an emoji,
which is two UTF-16 code units, and an error after emoji and past the
first line.

These stay in TypeScript:
- tests of the tokenizers, the node factories, tolerant parsing and the
  tree builder;
- tests that parse with allowUnclosedDocumentNode other than its
  default;
- the completion-mode tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant