[Jiajia Qin (qjia7)]

Description

Alternative to #33901 for QLoRA QKV/MLP fusion. Extend the existing MatMulNBitsMlp and MatMulNBitsQkv operators with optional runtime adapter inputs, and route adapter-off calls through their existing base decode shaders. No new contrib-operator names are added. The portable Muffin model is unchanged.

The optimizer fuses QLoRA only when the WebGPU kernel advertises the TAdapter type constraint. An older plugin that exposes only the base kernels keeps the unfused graph.

Performance

RTX 5080, built-in D3D12 WebGPU, graph capture on, batch 1, prompt length 16, 128 generated tokens; median of four runs after two warmups, adapter off:

Model/path	Decode tokens/s	Decode ms/token
Portable Muffin, fusion disabled	227.1	4.404
Portable Muffin, fusion enabled	253.5	3.945
Pruned AION base-only graph	254.6	3.928

Fusion improves portable Muffin decode by about 11.6% and brings it within 0.5% of the pruned base-only graph in this run window. The separate-operator PR #33901 reported the same performance relationship.

Validation

• Windows Release builds of onnxruntime, onnxruntime_test_all, and onnxruntime_provider_test passed.
• 37 focused fusion/switching tests and 3 WebGPU operator tests passed.
• One Muffin session ran adapter off, on, then off again; the final off run returned to the original token result.

Tradeoff for review

This approach extends the version-1 schemas of existing contrib operators. Automatic fusion is guarded for older plugins, but a serialized optimized graph containing the new adapter inputs requires a plugin that understands them. #33901 uses distinct operator names to keep that contract explicit.

[Copilot]

🟡 Changes recommended

Adding TAdapter breaks initialization of previously saved base-only ORT-format models, and the capability probe lacks registration-level regression coverage.

2 open issues

• Preserve legacy resolver compatibility for adapter kernels · New
• Test adapter capability probing in transformer registration · New

What changed in this PR

This PR extends the existing WebGPU MatMulNBitsMlp and MatMulNBitsQkv operators to support runtime QLoRA adapters, rather than adding separate operators as proposed in #33901.

Changes:

• Adds optional adapter inputs and reuses base decode shaders when adapters are off.
• Extends MLP/QKV fusion with a TAdapter capability check intended to protect older plugins.
• Updates operator documentation and adds numerical, adapter-switching, schema, and malformed-weight tests.

File	Description
onnxruntime/​test/​optimizer/​matmul_nbits_qlora_fused_switch_test.cc	Tests adapter switching and weight updates within one session.
onnxruntime/​test/​optimizer/​matmul_nbits_qkv_fusion_test.cc	Adds QLoRA QKV fusion, numerical, and optional-output tests.
onnxruntime/​test/​optimizer/​matmul_nbits_mlp_fusion_test.cc	Adds QLoRA MLP fusion and numerical tests.
onnxruntime/​test/​contrib_ops/​matmul_nbits_qlora_test.cc	Tests malformed base weights with empty adapters.
onnxruntime/​core/​optimizer/​matmul_nbits_qkv_fusion.h	Adds the QLoRA fusion enable flag.
onnxruntime/​core/​optimizer/​matmul_nbits_qkv_fusion.cc	Matches QLoRA projections and preserves adapter input edges.
onnxruntime/​core/​optimizer/​matmul_nbits_mlp_fusion.h	Adds the QLoRA fusion enable flag.
onnxruntime/​core/​optimizer/​matmul_nbits_mlp_fusion.cc	Extends MLP fusion to QLoRA projections.
onnxruntime/​core/​optimizer/​graph_transformer_utils.cc	Probes adapter kernel support before enabling fusion.
onnxruntime/​core/​graph/​contrib_ops/​contrib_defs.cc	Extends schemas and adjusts optional QKV output inference.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_qlora.h	Declares shared QLoRA projection and validation helpers.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_qlora.cc	Extracts reusable projection logic and base-weight validation.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_qkv.h	Adds adapter configuration and execution methods.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_qkv.cc	Implements adapter-aware QKV execution and base-path reuse.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_mlp.h	Adds adapter configuration and execution methods.
onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_mlp.cc	Implements adapter-aware MLP execution and base-path reuse.
docs/​OperatorKernels.md	Documents adapter inputs and WebGPU type constraints.
docs/​ContribOperators.md	Documents the extended operator schemas.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Both WebGPU kernels mistake empty optional-input placeholders for adapter inputs, causing schema-valid base-only nodes to fail initialization or dereference null tensors.

2 open issues

• Empty adapter placeholders trigger null dereferences in base-only nodes · New
• Empty adapter placeholders trigger null dereferences in base-only QKV nodes · New

2 resolved since last review

• Preserve legacy resolver compatibility for adapter kernels
• Test adapter capability probing in transformer registration

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The MLP and QKV all-empty adapter shortcuts bypass required shape validation, while fused adapter tests lack FP32 and multi-row projection coverage.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Validate adapter dimensions before the all-empty shortcut

onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_mlp.cc:564

The all-empty shortcut accepts a dynamically shaped adapter-up tensor such as [0, 1, 32] and returns the base result, whereas the standalone projection rejects it because the first dimension must be N. Validate each group's packed adapter ranks and dimensions before taking the shortcut, preferably using shared validation, and add a malformed-empty-adapter regression. The same issue affects onnxruntime/contrib_ops/webgpu/quantization/matmul_nbits_qkv.cc:409–411.

Add FP32 and multi-row fused adapter coverage

onnxruntime/​contrib_ops/​webgpu/​quantization/​matmul_nbits_qlora.cc:236

The new fused adapter numerical and switching tests cover only FP16 with M=1 and K=32, leaving integration with the FP32 and multi-row projection paths untested. Add unfused-versus-fused numerical cases for float and M>1, with active and mixed empty adapters, for both MLP and QKV. Choose dimensions and provider capabilities that exercise the alternative projection dispatches together with fused normalization and MLP activation.

2 resolved since last review

• Empty adapter placeholders trigger null dereferences in base-only QKV nodes
• Empty adapter placeholders trigger null dereferences in base-only nodes

_{Review effort: Balanced}