feat: typed normalizer pipeline (ordered step list + JS snippet + smart quotes) by SoundMindsAI · Pull Request #509 · SoundMindsAI/relyloop

SoundMindsAI · 2026-06-09T19:29:25Z

Summary

Extends Phase 1's fixed-bundle query_normalizer into a typed pipeline: a template can declare query_normalizer as {type: "normalizer_pipeline", steps: [...]} over six atomic NormalizerSteps, and the Optuna loop searches the powerset of the declared steps (2^N labels). Phase 1 bundles keep working unchanged — a bundle is just a label whose tokens are a subset of the step vocabulary, and both paths share one execution engine.

Implements feat_query_normalizer_typed_pipeline. The design-ahead gate cleared: Phase 1 (feat_query_normalization_tuning) is merged, and the two open questions were locked — Q-1: include expand_contractions_custom as an inert reserved step (6 steps) (operator decision); Q-2: JS-snippet parity via a frontend vitest fixture (engineering decision).

What shipped (by epic/story)

Story 1.1 — NormalizerStep enum (6 steps), normalize_pipeline (canonical STEP_ORDER, permutation-invariant), _pipeline_labels (powerset), steps_for_label, decoupled STEP_ORDER/LABEL_ORDER; FR-3 smart-quote (U+2019) expansion; normalize() reimplemented over normalize_pipeline (byte-parity guard). Broke the normalizers↔search_space import cycle via a function-local back-edge.
Story 1.2 — NormalizerPipelineParam as the 4th ParamSpec member; estimate_cardinality 2**N; apply_search_space samples powerset labels.
Story 1.3 — reservation accepts the pipeline under query_normalizer, rejects it elsewhere (NormalizerPipelineMisplacedError → INVALID_SEARCH_SPACE, D-8 no new code); broadened shape message.
Story 1.4 — both adapter pre-render hooks resolve bundle or pipeline label via steps_for_label → normalize_pipeline (closes the bundle-only-dispatch gap).
Story 2.1 — label-driven build_python_snippet/build_js_snippet; PR body emits ### Python + ### JavaScript / TypeScript blocks; three-way output parity (runtime ≡ Python ≡ JS) over a committed shared corpus.
Story 3.1 — <RowNormalizerPipeline> builder row (6-step checkbox multi-select), enum/glossary mirrors, cardinality 2^N, digest advisory broadened to the lowercase-token test (AC-13).
Story 3.2 — real-backend Playwright spec (smoke job).
Story 4.1 — optimization.md / adapters.md / local-dev.md docs.

Test coverage

Unit (backend): 8 new files — pipeline engine (AC-1/4/6/I-1), bundle byte-parity (AC-5/I-3), NormalizerPipelineParam (AC-2/3/4), template-defaults (AC-10), PR-snippet Python parity, JS-fixture freshness, elastic + solr render (bundle + non-bundle label, immutability) — plus consumer-fix regressions.
Integration: test_trial_runner_normalizer_pipeline.py (I-5, CI-only).
Contract: test_studies_normalizer_pipeline_contract.py (AC-12 misplaced, duplicate, wrong-shape, valid→201; CI-only).
E2E: query-normalizer-pipeline.spec.ts (smoke job).
Vitest: row (AC-9), advisory (AC-13), JS↔runtime parity (110 checks). Full suite 1274 green.

Test plan

make test-unit (2615 passed)
ruff format --check + ruff check + mypy backend/ (612 files, clean)
pnpm test (1274) + pnpm typecheck + pnpm build + pnpm lint
Regenerated ui/openapi.json + ui/src/lib/types.ts (NormalizerPipelineParam in the SearchSpace union)
Integration + contract layers run in CI (need Postgres service container)
E2E runs in the opt-in smoke job

Notes / documented deviations

Cross-model review: Opus self-review (GPT-5.5 unreachable in the Claude Code remote sandbox — no OPENAI_API_KEY / egress). Gemini Code Assist is the live cross-family gate at the code stage and will be adjudicated per the four-quadrant rubric.
The plan's "reproduce Phase 1's 4 snippets byte-for-byte" is superseded by FR-3 — the generated snippet now includes U+2019 handling, so it intentionally differs from Phase 1's pre-FR-3 text. The generator is the single source; guarantee is three-way output parity.
The builder row uses a checkbox multi-select (no new shadcn primitive), so the <SelectItem>-focused form-select-discipline guard is N/A; options are still .map()-sourced from NORMALIZER_STEP_VALUES.

🤖 Generated with Claude Code

Generated by Claude Code

- Add NormalizerStep StrEnum (6 members incl. inert expand_contractions_custom per Q-1) + STEP_ORDER/LABEL_ORDER/STEP_LABEL_TOKEN/_BUNDLE_TO_STEPS - normalize_pipeline (canonical-order, permutation-invariant), _label_for_subset, _pipeline_labels (powerset), steps_for_label (bundle + pipeline labels) - FR-3 smart-quote (U+2019) pre-normalization in the contraction step - Reimplement normalize() over normalize_pipeline (D-6) with byte-parity guard - Break the normalizers<->search_space cycle: move the search_space import in validate_normalizer_reservation to function-local so search_space can import NormalizerStep at module top (Story 1.2 enabler) - Lock plan gates: Phase 1 merged, Q-1 (6 steps inert) + Q-2 (vitest fixture) Tests: test_normalizers_pipeline.py (AC-1/AC-4/AC-6/I-1 + inert custom), test_normalizers_bundle_compat.py (AC-5/I-3 byte-parity); updated Phase 1 smart-quote test to the new FR-3 expand behavior. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…ory 1.2) - Add NormalizerPipelineParam (type='normalizer_pipeline', steps: list[NormalizerStep] min_length=1, extra=forbid, duplicate-step validator) as the 4th ParamSpec union member - estimate_cardinality: 2**len(steps) branch (powerset) - apply_search_space: trial.suggest_categorical(name, _pipeline_labels(steps)) - Import NormalizerStep/_pipeline_labels from normalizers at module top (cycle stays broken via the Story 1.1 function-local back-edge) - FR-7/AC-10: compute_default_params already returns "none" for the query_normalizer key (Phase 1) — pinned by a new test for the pipeline declaration shape Tests: test_search_space_normalizer_pipeline.py (AC-2/AC-3/AC-4 + categorical regression), test_template_defaults_normalizer_pipeline.py (AC-10). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…(Story 1.3) - validate_normalizer_reservation: accept normalizer_pipeline under query_normalizer (discriminator-string check, no cycle); add NormalizerPipelineMisplacedError when a pipeline param sits under any other key; broaden NORMALIZER_PARAM_SHAPE message to name both accepted shapes - studies router catches NormalizerPipelineMisplacedError -> INVALID_SEARCH_SPACE (D-8, no new code) - Update Phase 1 reservation test for the broadened shape message Tests: test_studies_normalizer_pipeline_contract.py (AC-12 misplaced, duplicate-step, wrong-shape, valid->201; CI-only, needs Postgres); no-DB unit coverage of misplaced/accept/wrong-shape in test_search_space_normalizer_pipeline.py. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…ory 1.4) - ElasticAdapter.render + SolrAdapter.render now resolve query_normalizer via steps_for_label -> normalize_pipeline, so a winning non-bundle powerset label (e.g. "lowercase+strip_punctuation") applies instead of raising. Bundles route through the same path (back-compat); caller-dict immutability + absent->none default preserved. Tests: test_elastic_render_normalizer_pipeline.py + test_solr_render_normalizer_pipeline.py (bundle + non-bundle label, smart-quote, immutability, unknown-token raise); test_trial_runner_normalizer_pipeline.py (I-5 trial->adapter->normalize_pipeline end-to-end; CI-only, needs Postgres). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…Story 2.1) - Add build_python_snippet/build_js_snippet to normalizers.py — generate a self-contained normalize_query/normalizeQuery for ANY powerset label (not just the 4 bundles), faithful to normalize_pipeline incl. FR-3 smart-quote handling and the inert custom-step no-op - _render_normalizer_requirement now emits ### Python AND ### JavaScript/TypeScript blocks, resolving bundle OR pipeline labels via steps_for_label (closes the four-bundle-only-dispatch gap; AC-13) - Three-way OUTPUT parity (FR-5): committed shared corpus fixture (ui/.../normalizer_snippet_parity.json); backend test_normalizers_pr_snippets_js.py keeps it in sync with normalize_pipeline + build_js_snippet; vitest normalizer-snippet-parity.test.ts runs each JS snippet and asserts == runtime (Q-2: each runtime in its own suite) Deviation (documented): the plan's "reproduce Phase 1's 4 snippets byte-for-byte" is superseded by FR-3 — the generated expand snippet now includes U+2019 pre-normalization, so it intentionally differs from Phase 1's pre-FR-3 snippet. The generator is now the single source; legacy _PR_BODY_NORMALIZER_SNIPPETS constants removed. Guarantee is output parity, verified across 11 label cases × 10-input corpus (110 JS checks). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…ory (Story 3.1) - enums.ts: NORMALIZER_STEP_VALUES (STEP_ORDER) + NORMALIZER_STEP_GLOSSARY_KEYS; glossary.ts: 6 step keys + 1 row key - search-space-defaults.ts: ParamSpec gains normalizer_pipeline; estimateParamCardinality returns 2**steps.length - row-type-selector adds 'normalizer_pipeline' to TYPE_VALUES; stash default {steps:[]}; param-row dispatch; cardinality detail; NEW <RowNormalizerPipeline> (ordered 6-step checkbox multi-select from NORMALIZER_STEP_VALUES, glossary labels, empty-steps incomplete helper paralleling the categorical __placeholder__ sentinel) - digest-panel advisory broadened to the lowercase-token test (AC-13) - discriminator parity test expects the 4th member; regenerated openapi.json + types.ts (NormalizerPipelineParam now in the SearchSpace union schema) Note: the row uses a checkbox multi-select (no new shadcn primitive) rather than <Select>, so the SelectItem-focused form-select-discipline guard is N/A; options are still .map()-sourced from NORMALIZER_STEP_VALUES. Tests: row-normalizer-pipeline.test.tsx (AC-9 a-d), advisory AC-13 cases, parity test updated. Full vitest 1274 green; tsc + next build + eslint clean. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…C-11) Real-backend Playwright spec: seed a query_normalizer-declaring template, walk the create-study wizard to Step 4, switch the reserved row to the normalizer_pipeline type, pick lowercase+trim via checkboxes, assert the incomplete helper clears + the live 2^N cardinality preview reads 4, and assert the persisted study carries {type:"normalizer_pipeline", steps:["lowercase","trim"]}. Scope mirrors the sibling query-normalization.spec.ts (wizard + persistence; engine application / PR body / advisory covered at their own layers). Runs in the opt-in smoke CI job (no local stack in the sandbox). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

- optimization.md: typed normalizer-pipeline subsection — powerset sampling, STEP_ORDER vs LABEL_ORDER decoupling, bundle⊂label-space, bilingual PR snippet - adapters.md: pre-render hook now accepts a bundle string OR a pipeline label under query_normalizer (steps_for_label -> normalize_pipeline) - local-dev.md: normalizer_pipeline search_space example diff + the 6 steps + INVALID_SEARCH_SPACE failure modes No llm-data-flow.md change (no LLM call introduced — FR-9). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…nsumers The full `mypy backend/` pre-push gate caught two downstream consumers that didn't handle the 4th ParamSpec union member added in Story 1.2 (per-file mypy during the story missed them): - search_space_defaults.estimate_param_cardinality: add the 2**len(steps) branch (mirrors search_space.estimate_cardinality) - baseline_resolver._midpoint: a normalizer_pipeline baseline resolves to the "none" label (always in the param's powerset; consistent with compute_default_params / FR-7); signature widened to the 4-member union Also clean up test-side mypy nits (unused type:ignore, BaseDistribution.choices cast, dict type-arg). Regression tests for both consumer branches. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…guard Two CI failures from the Story 3.1 push: - prettier --check (both frontend jobs) flagged row-normalizer-pipeline.test.tsx — reformatted (my local `pnpm lint` runs eslint, not prettier). - verify_enum_source_of_truth.sh couldn't resolve the NormalizerStep StrEnum cited by the new NORMALIZER_STEP_VALUES source-of-truth comment — the helper only handled Literal/frozenset/tuple/list. Extended resolve_values to read member .value for any enum.Enum subclass (first StrEnum-backed enums.ts citation). Verifier now reports 42 allowlists clean. Sanity test added. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

gemini-code-assist

Code Review

This pull request implements the typed normalizer pipeline feature (MVP2), allowing templates to declare a normalizer_pipeline parameter under the reserved query_normalizer key. The search space now supports searching over the powerset of six atomic normalization steps, with canonical application ordering and backward compatibility for Phase 1 bundles. It also introduces dynamic Python and JavaScript snippet generators to provide operators with accurate reference implementations of the winning pipeline. The review feedback highlights opportunities to optimize the generated Python and JavaScript snippets by replacing inefficient character-by-character punctuation stripping with regex-based replacements, matching the efficiency of the runtime implementation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

gemini-code-assist · 2026-06-09T19:43:01Z

+        if step is NormalizerStep.lowercase:
+            body.append("    q = q.lower()")
+        elif step is NormalizerStep.strip_punctuation:
+            body.append('    q = "".join(c for c in q if c not in _PUNCTUATION)')


The implementation for strip_punctuation in the generated Python snippet is inefficient and inconsistent with the runtime implementation in _apply_step. It uses "".join(c for c in q if c not in _PUNCTUATION), which performs a linear scan of the punctuation string for each character of the input query.

To improve performance and consistency, I recommend using re.sub in the generated snippet. This would involve:

Updating the needs_re flag on line 350 to be set when strip_punctuation is present.

Generating a compiled regex pattern for punctuation in the snippet's header on lines 357-358.

Using _PUNCTUATION_PATTERN.sub("", q) in the snippet's body, as suggested.

Suggested change

body.append(' q = "".join(c for c in q if c not in _PUNCTUATION)')

body.append(' q = _PUNCTUATION_PATTERN.sub("", q)')

gemini-code-assist · 2026-06-09T19:43:02Z

+        if step is NormalizerStep.lowercase:
+            lines.append("  q = q.toLowerCase();")
+        elif step is NormalizerStep.strip_punctuation:
+            lines.append('  q = q.split("").filter((c) => !PUNCTUATION.includes(c)).join("");')


The JavaScript snippet for stripping punctuation uses q.split("").filter(...).join(""), which can be inefficient for long query strings. A regular expression-based replacement would be more performant.

I suggest generating a regex for punctuation and using q.replace().

This would involve:

On lines 401-402, instead of creating punct_js, create a string for the regex character class.

On line 406, create the RegExp object.

Use replace with this regex as suggested below.

Suggested change

lines.append(' q = q.split("").filter((c) => !PUNCTUATION.includes(c)).join("");')

lines.append(' q = q.replace(PUNCTUATION_REGEX, "");')

CI surfaced the only failure: test_valid_pipeline_creates_study hit the gate-3c preflight overlap probe (INSUFFICIENT_JUDGMENT_OVERLAP, 422) because the seed has no judgments. Monkeypatch probe_judgment_overlap to return sufficient overlap so the test exercises the typed-pipeline create path through to 201 — the same pattern test_studies_api.py uses for happy-path creates. The other 3 cases fail before the probe (model_validate / reservation) and are unaffected. Coverage was green (81.64%). https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

…zer-typed-pipeline

…ts (Gemini) Adjudicates Gemini Code Assist's 2 Medium findings (accepted): the generated Python/JS reference snippets stripped punctuation with a membership-filter (`"".join(...)` / `split.filter.join`) instead of a regex, diverging from the runtime `_PUNCTUATION_PATTERN.sub`. Now both snippets mirror the runtime — Python emits `_PUNCTUATION_PATTERN = re.compile(...)` + `.sub("", q)`; JS emits a backslash-escaped char-class `PUNCTUATION_REGEX` + `.replace(...)`. Output unchanged (three-way parity re-verified: 11 Python label cases + 110 JS↔runtime node checks). Fixture regenerated. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com>

SoundMindsAI · 2026-06-09T20:12:45Z

Review adjudication (Gemini Code Assist)

Commit landing fixes: 7047190

Gemini Code Assist (2 findings)

#	Sev	Location	Verdict	Notes
1	Medium	`normalizers.py` `build_python_snippet` (strip_punctuation)	Accepted	Snippet stripped punctuation with `"".join(c for c in q if c not in _PUNCTUATION)` instead of the runtime's regex. Now mirrors `_PUNCTUATION_PATTERN.sub("", q)` — consistency with the runtime + idiomatic for the operator's copy-pasted reference.
2	Medium	`normalizers.py` `build_js_snippet` (strip_punctuation)	Accepted	Same theme on the JS side — replaced `split("").filter(...).join("")` with a backslash-escaped char-class `PUNCTUATION_REGEX` + `q.replace(...)`.

Outcomes

Applied fixes (2): both strip_punctuation snippet branches now use a regex, mirroring the runtime.
Output unchanged: three-way parity re-verified — 11 Python label cases (exec ≡ normalize_pipeline) + 110 JS↔runtime checks executed under Node against the regenerated shared corpus fixture. The vitest normalizer-snippet-parity.test.ts + the backend test_normalizers_pr_snippets_js.py freshness guard both pass, so any future escaping drift is caught.
Rejected / Deferred: none.

GPT-5.5 final review: not run — unreachable in the Claude Code remote sandbox (no OPENAI_API_KEY / api.openai.com egress); Opus self-review + Gemini (this adjudication) are the cross-family gates per CLAUDE.md.

Ready for merge once CI is green on 7047190.

Generated by Claude Code

Move the feature folder planned_features/02_mvp2 -> implemented_features/2026_06_09_; stamp pipeline_status Complete (+ Release: mvp2 marker) and implementation_plan status; add the merge one-liner to state.md (drop the 6th) + full narrative to state_history.md; regenerate MVP2/DASHBOARD + public roadmap. https://claude.ai/code/session_012vWN7bUoy74xvuqVR7S2H8 Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>

claude added 10 commits June 9, 2026 18:49

gemini-code-assist Bot reviewed Jun 9, 2026

View reviewed changes

claude added 3 commits June 9, 2026 19:52

Merge remote-tracking branch 'origin/main' into feature/query-normali…

75d72f5

…zer-typed-pipeline

SoundMindsAI merged commit 7a24849 into main Jun 9, 2026
19 checks passed

SoundMindsAI deleted the feature/query-normalizer-typed-pipeline branch June 9, 2026 20:22

SoundMindsAI mentioned this pull request Jun 9, 2026

docs: finalize feat_query_normalizer_typed_pipeline (PR #509) #513

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: typed normalizer pipeline (ordered step list + JS snippet + smart quotes)#509

feat: typed normalizer pipeline (ordered step list + JS snippet + smart quotes)#509
SoundMindsAI merged 13 commits into
mainfrom
feature/query-normalizer-typed-pipeline

SoundMindsAI commented Jun 9, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jun 9, 2026

Uh oh!

gemini-code-assist Bot Jun 9, 2026

Uh oh!

SoundMindsAI commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	body.append(' q = "".join(c for c in q if c not in _PUNCTUATION)')
	body.append(' q = _PUNCTUATION_PATTERN.sub("", q)')

	lines.append(' q = q.split("").filter((c) => !PUNCTUATION.includes(c)).join("");')
	lines.append(' q = q.replace(PUNCTUATION_REGEX, "");')

Conversation

SoundMindsAI commented Jun 9, 2026

Summary

What shipped (by epic/story)

Test coverage

Test plan

Notes / documented deviations

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

SoundMindsAI commented Jun 9, 2026

Review adjudication (Gemini Code Assist)

Gemini Code Assist (2 findings)

Outcomes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants