Add missing .replace, .upper, .lower string methods to vendored minja

[samuel100]

Summary

Adds the three missing string-method handlers — .replace, .upper, .lower — to the vendored minja parser at shared/api/minja.hpp. These are standard Jinja2 string methods that Hugging Face transformers chat templates rely on, and the most impactful gap (.replace) currently breaks any model whose chat_template.jinja uses it — most visibly smollm3-3b, where OrtxApplyChatTemplate throws:

Unknown method: replace at row 23, column 48:
{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}
                                               ^

Fixes #1081
Refs microsoft/Foundry-Local#800

Diff against upstream

Comparing shared/api/minja.hpp against current upstream google/minja:

String method	Upstream minja	Before this PR	After this PR
.strip / .rstrip / .lstrip	✅	✅	✅
.split	✅	✅	✅
.title	✅	✅	✅
.endswith / .startswith / .capitalize	✅	✅	✅
.replace	✅	❌	✅
.upper	✅	❌	✅
.lower	✅	❌	✅

(Note: lower was already available as a global filter — {{ x \| lower }} — but not as the method form x.lower() that HF templates actually use.)

Implementation notes

Handlers were added in MethodCallExpr::do_evaluate next to the existing string-method dispatch:

• .replace(before, after[, count]) — left-to-right substring replacement matching Python's str.replace and upstream minja. Defaults to all occurrences when count is omitted. Empty before returns the string unchanged (matches upstream behavior; avoids an infinite loop).
• .upper() / .lower() — std::transform with an unsigned char-typed lambda around std::toupper / std::tolower to avoid undefined behavior on signed-char platforms.

I deliberately limited the scope of this PR to the methods needed to unblock real HF templates today; a full re-sync of minja.hpp against current upstream would also be reasonable but is out of scope here.

Tests

Adds two regression tests in test/pp_api_test/test_tokenizer_chat.cc, following the inline-template_str pattern introduced in #1070 (MinjaStringSliceOOBClamped):

• MinjaStringReplace — single replace, chained replace + rstrip (the exact smollm3-3b pattern), three-arg count, empty before, no-match.
• MinjaStringUpperLower — .upper() and .lower() on mixed-case input, plus idempotent cases.

Local verification

I verified the fix end-to-end with a small standalone harness that compiles shared/api/minja.hpp directly and renders the exact failing line from smollm3-3b's chat_template.jinja:

template:  {% set custom_instructions = system_message.replace('/no_think', '').replace('/think', '').rstrip() %}{{ custom_instructions }}
context:   { "system_message": "You are helpful /no_think.   " }

before this PR: ERROR: Unknown method: replace at row 1, column 6
after  this PR: OK: "You are helpful ."

I did not build the full pp_api_test binary locally (would require ORT + sentencepiece + onnxruntime-extensions deps setup on macOS); the new gtest cases are written to existing conventions and will run in CI.

Follow-up (out of scope here)

It would be worth adding a periodic re-sync of shared/api/minja.hpp with current upstream google/minja, and/or a regression suite that renders the chat templates of a handful of popular HF models (SmolLM3, Qwen3, Llama-3.x, Phi-4) against OrtxApplyChatTemplate, to catch this kind of drift earlier. I'm happy to follow up with that in a separate PR if maintainers want.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds missing string methods to the vendored minja parser to support common HF chat templates and introduces regression tests to prevent future breakage.

Changes:

• Implemented upper(), lower(), and replace() string methods in minja.
• Added tokenizer chat-template regression tests covering these methods.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
test/pp_api_test/test_tokenizer_chat.cc	Adds regression tests ensuring templates using .replace()/.upper()/.lower() render successfully.
shared/api/minja.hpp	Implements the missing minja string methods used by popular chat templates.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[baijumeswani]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[samuel100]

Closing because this PR originates from a fork and the required pipeline credentials/ACR access are unavailable for fork-based runs. Please recreate from a branch in microsoft/onnxruntime-extensions once write access is available.