Repository navigation
Fix quadratic parsing of deeply nested markup - #800
Open
josefguenther wants to merge 1 commit into
Open
josefguenther wants to merge 1 commit into
josefguenther wants to merge 1 commit into
Conversation
Many tree builder steps search the stack of open elements. "Has an element in scope" runs on most start and end tags in body, and a few other steps look for an element by name or for the nearest element of some kind. Each search walks down from the top of the stack, and when nothing matches it walks all the way to the bottom. So on deeply nested markup every token costs time in proportion to the nesting depth, and parsing takes quadratic time. A few hundred kilobytes of nested <ul><li> can keep the parser busy for minutes. Keep the stack in a new OpenElems type that also records, for each element name and for each set of elements these steps look for, the positions in the stack where those elements are open. Pushing and popping update the positions in constant time. "Is X in scope" then compares the topmost X with the topmost element that bounds the scope, and the other steps read the positions directly. Parse results are unchanged. The new test counts how many elements the tree builder looks at for each token on nested input, so it fails quickly if a step goes back to walking the stack. See servo#788.
jaredlynskey
added a commit
to jaredlynskey/pyhtml2text
that referenced
this pull request
Oct 18, 2026
Our patch only made the "p in button scope" check O(1). Unmatched end tags and end tags in foreign content still walked the whole stack of open elements, so 32,000 nested <span> followed by unmatched </em> took 3.4s and SVG <g> followed by unmatched </x> took 4.5s. servo/html5ever#800 by Josef Guenther (67c7e7c, not merged yet) keeps the positions of open elements by name and by tag set, so every scope check and stack lookup is O(1). Its src/ changes apply to 0.39.0 with only line offsets. Checked against stock 0.39.0: html5ever's tests and the 3,515 html5lib tree-construction tests pass in release and debug builds, and the tree, quirks mode and exact parse errors are identical on every html5lib input, 300,000 random tag-soup documents and fragments, and 28 real pages. pyhtml2text's output is unchanged on the benchmark corpus. Costs about 0.4us per call and 0-2% on real pages. The deep-nesting tests now cover both new shapes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PwkfVNzHx1JafFtb559mx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Deeply nested markup makes tree building quadratic in the nesting depth, so a small document can keep the parser busy for minutes. This is the parse time reported in #788. A depth cap, which that issue also asks about, would be a separate change, and it would not remove this cost on its own, because the stack of open elements still grows with the input.
Many tree builder steps search the stack of open elements. "Has an element in scope" runs on most start and end tags in body, and some other steps look for an element by name or for the nearest element of some kind: resetting the insertion mode,
</option>, end tags in foreign content, the "any other end tag" steps and the adoption agency. Each search walks down from the top of the stack, and when nothing matches it walks all the way to the bottom.This change keeps the stack in a new
OpenElemstype intree_builder/open_elems.rs. Next to the elements, it records the stack positions of the open elements for each name and for each set of elements these steps look for. Push and pop update the positions in constant time. "Is X in scope" then compares the topmost X with the topmost element that bounds the scope, and the other steps read the positions directly.One parse with a release build, at 64,000 levels of nesting:
<div>nested, then</div><ul><li>nested<div>nested, then</p></p><svg>with<g>nested, then</x>Ordinary documents parse at the same speed. On four large real pages the instruction count went up 1.6% and the cycle count did not change. While parsing, the stack uses about 30 more bytes per open element. Name lookups use a small hasher built on the hash each atom already carries; with the default hasher the same pages were about 7% slower.
Two cases are still quadratic and are not changed here: the adoption agency removing elements from the middle of the stack, and searches of the list of active formatting elements.
Parse results do not change. The html5lib tests pass as before, and the tree and the parse errors match
mainon every html5lib input and on 45,000 random documents. The new test intests/tree_builder.rscounts how many elements the tree builder looks at for each token on 19 nested inputs. It sees 1 to 7 with this change and 250 to 1,008 onmain.