Skip to content

Reduce allocations in Luhn, Redact, and CLI sanitizer paths - #13

Merged
dgageot merged 4 commits into
mainfrom
worktree-board-c6922d7e023b5db8
Sep 24, 2026
Merged

dgageot merged 4 commits into
mainfrom
worktree-board-c6922d7e023b5db8

Conversation

@dgageot

@dgageot dgageot commented Sep 24, 2026

Copy link
Copy Markdown
Member

Profiling the hot paths in card/SSN validation, redaction, and the CLI's output sanitizer surfaced avoidable allocations and wasted CPU: validLuhn built a fresh []int per call, Redact grew its strings.Builder to the full input length regardless of whether the output would shrink or expand, sanitizeValue allocated a new strings.Replacer on every multi-line match, and the digit-run prefilter walked every byte of long non-digit prose while maintaining run/afterDigit state it never used.

This change lands four independent, behavior-preserving optimizations:

  • validLuhn scans digits in place from the end of the string instead of allocating a slice, and bails out once more than 19 digits are seen.
  • Redact precomputes the exact output size (falling back to the input length on int overflow) so strings.Builder.Grow allocates once instead of resizing as it writes.
  • sanitizeValue reuses a single package-level strings.Replacer instead of constructing one per call; sharing it is safe since Replacer.Replace holds no mutable state.
  • hasSeparatedDigitRun skips runs of non-digit bytes with a tight inner loop instead of tracking separator state on every byte.

Each commit is checked against a reference implementation via fuzzing (over 2.3M differential cases) plus unit tests, and the branch passes go test -race -count=3 ./..., a GOTOOLCHAIN=go1.26.5 go test -race ./... run, go vet ./..., golangci-lint run ./... and golangci-lint fmt --diff, and go mod tidy -diff. A temporary baseline-vs-current harness additionally compared 18,157 inputs across Contains/Redact/Find/ContainsBytes/FindBytes under the race detector, and 19 CLI scenarios at 1/2/8/32 workers for exact stdout/stderr/exit-status parity, with no mismatches.

On one Apple M4 Max machine running Go 1.27, interleaved benchmarks show the clean-input Redact/Contains paths roughly 25% faster, validLuhn dropping from 192 B and 2 allocations to zero, 64 KiB secret redaction dropping from about 74 KB to 248 B allocated, and the multiline sanitizer dropping from 10 to 3 allocations and running about 19% faster. These numbers are specific to that run and hardware, and are not a guarantee against regressions on other platforms. An earlier draft also buffered CLI output in a bytes.Buffer for fewer syscalls, but that was dropped after review because it could retain more queued-output memory, so it is not part of this PR. The existing Unicode gate mismatch and the unbounded CLI reorder buffer are known, unrelated issues left out of scope here.

Replaces the per-byte state machine with a loop that jumps straight
past non-digit runs, avoiding redundant checks on text-heavy input.

Assisted-By: cagent
@dgageot
dgageot requested a review from a team as a code owner September 24, 2026 14:14

@docker-agent docker-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

Optimizes validation, redaction sizing, and CLI value sanitization.

@dgageot
dgageot merged commit a2eb85d into main Sep 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant