Skip to content

Avoid building a MatchData in two string checks - #814

Merged
nesquena merged 1 commit into
masterfrom
faster-process-source
Nov 1, 2026
Merged

nesquena merged 1 commit into
masterfrom
faster-process-source

Conversation

@tagliala

@tagliala tagliala commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Two guards only ask a yes/no question about a string, and both asked it with =~, which builds a MatchData neither of them reads and sets $~:

  • Renderer#process_source: does a source given as a string already contain a newline, so a template string isn't looked up as a file? → String#include?
  • Engine's JSONP callback check: is the request's callback parameter a valid name? → Regexp#match?, which runs the same match without keeping the result.

Benchmark

Apple M5 Pro, macOS 27.2, ruby 4.0.7 arm64-darwin23, YJIT off. Same script against master and this branch, 5–7 alternating runs, medians; the two variants' output is compared on every run to check they do the same work.

Renderer.new (multi-line source) [same output]
  before:  1995372.1 i/s      10 objects/call
  after:   2329402.8 i/s       7 objects/call    1.17x
render (multi-line source) [same output]
  before:   115514.5 i/s      56 objects/call
  after:    117811.2 i/s      53 objects/call    1.02x
render with a JSON callback [same output]
  before:   160495.3 i/s      43 objects/call
  after:    167166.6 i/s      40 objects/call    1.04x, 1.00x on a second series

The steady part is the allocations: 3 objects fewer per compile, per render, and per render with callbacks. The timings move clearly only on Renderer.new; the full render's 2% and the callback path's 1.04x are inside the noise.

Behaviour

match? changes nothing: ascii_only? still runs first, the same names match, and nothing reads $~ after either check.

include? differs for one input: a source that isn't valid in its own encoding. =~ raised from the regexp, include? compares bytes, so the failure moves rather than disappears — the same class and message from a different frame:

"object @obj\xFF\nattributes :name".force_encoding("UTF-8")
  before: ArgumentError: invalid byte sequence in UTF-8
            rabl/renderer.rb:66 in 'Rabl::Renderer#process_source'
  after:  ArgumentError: invalid byte sequence in UTF-8
            active_support/core_ext/object/blank.rb:159 in 'Regexp#match?'   (Engine#eval_source)

"object @obj\xFF".force_encoding("UTF-8")   (no newline, so it reaches the template lookup)
  after:  ArgumentError: invalid byte sequence in UTF-8
            rabl/sources.rb:321 in 'String#split'                             (Partials#partialized)

rabl rescues ArgumentError nowhere, so nothing that caught it stops catching it. test/renderer_test.rb now records both cases, and the CHANGELOG entry says where the error comes from.

Two further cases, from @nesquena's review. Nothing renders from either kind of source before or after, and both are verified on this branch:

  • A source in a non-ASCII-compatible encoding, such as UTF-16LE, still raises Encoding::CompatibilityError at the same place, Renderer#process_source, and with or without a newline; only the message differs, incompatible character encodings: UTF-16LE and UTF-8 instead of incompatible encoding regexp match (US-ASCII regexp with UTF-16LE string).
  • An invalid source that contains a newline now raises from #render instead of from Rabl::Renderer.new: new succeeds and the ArgumentError comes from Engine#eval_source on render.

The CHANGELOG entry records both the ArgumentError frames and the Encoding::CompatibilityError message change.

Where this came from

rubocop-performance's Performance/StringInclude.

  • It found process_source, which is where this started, and finds nothing in the tree after this change.
  • It does not find the callback check, nor the last =~ left, test/configuration_test.rb:33 (to_s =~ /oj/i). The cop only rewrites a regexp written inline with a metacharacter-free pattern and no option flags; RABL_JSON_CALLBACK_NAME is a named anchored constant and that test's pattern is case-insensitive. Neither was reported by any cop, and include? could not express them anyway; both were found by reading every =~ in the tree.
  • Its third safe report in lib/, [:mode, :nan] → a constant in Configuration#oj_dump_options, is not taken: 0.99x, and slower in isolation (5.10M → 4.41M i/s), because CRuby compiles that literal to opt_duparray_send already.

No RuboCop configuration is added here; that's a separate decision.

Attribution

  • Change written by DeepSeek Flash (the agent that made it), with the benchmarks above.
  • Reviewed by Claude Opus 5.5 and Claude Fable 5.1, independently, on a frozen brief holding only the two diffs, the claims and the numbers. Both caught the same two errors in the first draft, before it was pushed: that RuboCop reported neither site (it did report the first), and that the invalid-encoding failure always moved to eval_source (without a newline it moves to the template lookup). Those sentences and the test above are the result.
  • Bench scripts are not part of this PR; they load the two checkouts and reproduce the table.

CI is green on the current head: Ruby 2.7, 3.0-3.4, 4.0 and head, plus the generated Rails 7.2, Rails 8.1 and Sinatra apps.

@tagliala
tagliala marked this pull request as draft November 1, 2026 10:08
@tagliala
tagliala force-pushed the faster-process-source branch 3 times, most recently from 0fdc426 to a30f8c1 Compare November 1, 2026 10:23
@tagliala tagliala changed the title Use String#include? instead of a regex match in process_source Avoid building a MatchData in two string checks Nov 1, 2026
@tagliala
tagliala force-pushed the faster-process-source branch 2 times, most recently from 42c5cfd to d2e7244 Compare November 1, 2026 10:24
@tagliala
tagliala marked this pull request as ready for review November 1, 2026 14:09

nesquena commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Thanks @tagliala. This looks good on review. The match?/include? swaps behave the same as =~ for every input I tried (including binary, Shift_JIS and UTF-16 sources, and the callback name edge cases), and the suite and the generated Rails/Sinatra apps pass.

The description ends with "Draft." Is it ready to merge as it is, or is there more you want to add? It would go into 0.19.0, which we're about to cut.

Two small notes for the description, not blockers:

  • A source in a non-ASCII-compatible encoding (UTF-16LE, say) still raises Encoding::CompatibilityError at the same place, but the message is now "incompatible character encodings" rather than "incompatible encoding regexp match".
  • An invalid source that contains a newline now raises from #render instead of Rabl::Renderer.new.

Nothing renders from either kind of source, before or after.


Generated by Claude Code

Two checks only ask whether a pattern occurs in a string, and both asked
it with `=~`, which builds a MatchData neither of them reads and sets
$~:

- Renderer#process_source: whether a source given as a string already
  contains a newline, so that a template string is not looked up as a
  file. String#include? answers it without allocating.
- Engine's JSONP callback check: whether the request's callback
  parameter is a valid name. Regexp#match? runs the same match and drops
  the result.

Measured on an Apple M5 Pro, ruby 4.0.7 arm64-darwin23, median of 5 to
7 alternating runs of the same script against master and this branch,
with the two variants' output compared to check the work is the same.
The allocation counts are steady: Renderer.new with a multi-line source
10 -> 7 objects per call, a full render 56 -> 53, a render with
enable_json_callbacks 43 -> 40. The timings move less clearly, and only
in the first: Renderer.new 1.99M -> 2.33M i/s (1.17x), a full render
115.5k -> 117.8k i/s (1.02x, inside the noise), the callback render
1.00x.

One behaviour difference, only in the first check and only for a source
that is not valid in its own encoding: `=~` raised ArgumentError from
the regexp, include? compares bytes. The same ArgumentError ("invalid
byte sequence in UTF-8") still comes out of the render, from a
different frame: with a newline from String#present? in
Engine#eval_source, without one from String#split in
Partials#partialized during the template lookup. A source in an
encoding that can't be compared with the pattern, such as UTF-16LE,
still raises Encoding::CompatibilityError from the same check, now
with Ruby's "incompatible character encodings" message rather than
"incompatible encoding regexp match". rabl rescues ArgumentError
nowhere, so no caller loses the error it caught, and
test/renderer_test.rb records both cases. match? changes no behaviour:
the same names match, and nothing reads $~ after either check.

Written by DeepSeek Flash. Reviewed by Claude Opus 5.5 and Claude
Fable 5.1.

Reviewed-by: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Claude Fable 5.1 <noreply@anthropic.com>
@tagliala
tagliala force-pushed the faster-process-source branch from d2e7244 to c1488e1 Compare November 1, 2026 18:07
@tagliala

tagliala commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Draft was only there because CI hadn't finished on the first push, and the last line of the description said so. It went stale: the PR is ready.

Since then:

  • Both notes from your review are now in the description, checked on this branch rather than copied: a UTF-16LE source keeps raising Encoding::CompatibilityError in Renderer#process_source with Ruby's incompatible character encodings message, and an invalid source that contains a newline now raises from #render while Rabl::Renderer.new succeeds.
  • The CHANGELOG.md entry records the message change too, next to the ArgumentError frames.

Current head c1488e1. CI is green on it: Ruby 2.7, 3.0-3.4, 4.0 and head, plus the generated Rails 7.2, Rails 8.1 and Sinatra apps.

@tagliala
tagliala requested a review from nesquena November 1, 2026 18:11
@nesquena
nesquena merged commit 7d14138 into master Nov 1, 2026
11 checks passed

nesquena commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Merged. Thanks @tagliala, and thanks for folding those notes into the description and the CHANGELOG. It goes out in 0.19.0.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants