Skip to content

Skip QDQ to QLinear fusions when Q/DQ scales are not float32 - #33915

Open
Oke Iyanuoluwa Enoch (Iyanuoluwa007) wants to merge 1 commit into
microsoft:mainfrom
Iyanuoluwa007:fix-qdq-fp16-scale-qlinear
Open

Oke Iyanuoluwa Enoch (Iyanuoluwa007) wants to merge 1 commit into
microsoft:mainfrom
Iyanuoluwa007:fix-qdq-fp16-scale-qlinear

Conversation

@Iyanuoluwa007

Copy link
Copy Markdown

Description

The QDQ selector/action transformer rewrites Q/DQ node groups into QLinear* contrib ops (QLinearAdd, QLinearMul, QLinearConcat, the unary QLinear ops, QLinearWhere), QLinearConv and QGemm. These replacement ops only accept float32 scales, but the selectors never checked the scale type. A valid model whose Q/DQ nodes use float16 scales was therefore rewritten into an invalid node, and session creation failed with INVALID_GRAPH.

This change:

  • Adds an opt-in require_float_scales flag to BaseSelector. When set, BaseSelector::Select returns no selection if any Q or DQ node in the group has a non-float32 scale.
  • Enables the flag for UnarySelector, BinarySelector, InputVariadicSelector, ConvSelector, WhereSelector and GemmSelector.
  • Leaves MatMulSelector unchanged, since QLinearMatMul (opset 21) and MatMulIntegerToFloat accept float16 scales. The DropQDQ, DropDQ, Split and DQ MatMulNBits selectors are also unchanged, as they do not produce ops with scale type constraints.
  • Puts the check in BaseSelector::Select only, so the node unit grouping that EPs obtain through GetQDQSelection (e.g. QNN) is not affected.
  • Adds QDQTransformerTests.ConcatFloat16ScalesAreNotFused, which fails on main with the same INVALID_GRAPH error as the issue and passes with this change.

Testing (Windows, Release build):

  • The repro script from the issue now creates the session and runs.
  • Compared outputs with graph optimizations disabled vs. enabled for float16-scale Q/DQ models around Add, Mul, Concat, Conv, Gemm, Where, MatMul and several unary ops. All load and match.
  • onnxruntime_test_all --gtest_filter=*QDQ*:*Qdq*:GraphTransformationTests.*:*TransposeOptimizer*:*NhwcTransformer*:*SelectorManager* passes (884 tests).

Motivation and Context

Fixes #33788

The QLinear* contrib ops (QLinearAdd, QLinearMul, QLinearConcat, the unary
QLinear ops, QLinearWhere), QLinearConv and QGemm only accept float32
scales. The QDQ selector/action transformer did not check the scale type,
so a valid Q/DQ group with float16 scales was rewritten into an invalid
node and session creation failed with INVALID_GRAPH.

Add an opt-in check to BaseSelector::Select and enable it for the
selectors whose replacement requires float32 scales. MatMul is left out
because QLinearMatMul (opset 21) and MatMulIntegerToFloat accept float16
scales. The check is in BaseSelector::Select, so the node unit grouping
that EPs get through GetQDQSelection is unchanged.

Fixes microsoft#33788
Copilot AI balanced review requested due to automatic review settings November 5, 2026 07:58
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the change to prevent QDQ fusion with incompatible quantization scale types.

No open issues 🎉

What changed in this PR

Prevents QDQ fusion from creating invalid operators when quantization scales are not float32, addressing the session-creation failure reported in #33788.

Changes:

  • Adds an opt-in float32-scale check for selectors whose replacement operators require it.
  • Leaves MatMul selection and execution-provider node grouping unchanged.
  • Adds a regression test verifying that Concat with float16 Q/DQ scales is not fused.
File Description
onnxruntime/​test/​optimizer/​qdq_transformer_test.cc Adds the float16-scale Concat regression test.
onnxruntime/​core/​optimizer/​qdq_transformer/​selectors_actions/​qdq_selectors.h Defines the scale-check flag and enables it for affected selectors.
onnxruntime/​core/​optimizer/​qdq_transformer/​selectors_actions/​qdq_selectors.cc Checks Q/DQ scale types and skips incompatible fusions.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU EP: QDQ fusion creates QLinearConcat / QLinearAdd with float16 scales -> INVALID_GRAPH at session creation

2 participants