You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support native file handles for external initializers - #33660
A sandboxed process may have permission to use model files without permission to open their filesystem paths. This change adds SessionOptionsSetExternalInitializerFileHandles and its C++ SessionOptions wrapper to the stable ONNX Runtime 1.32 API so a host can register named, pre-opened external initializer files before loading an ONNX graph.
ONNX Runtime duplicates native Windows file handles or read-only POSIX file descriptors and retains them across option clones and sessions. Initializer ranges are loaded or mapped from those handles without resolving or canonicalizing filesystem paths. On Windows WebGPU, the D3D12 accelerated external data loader reopens registered handles directly before falling back to CPU-mapped initializers. Missing sources and invalid ranges fail explicitly rather than falling back to disk access, and nested-graph validation inspects initializer metadata without copying mapped weight payloads.
The API accepts flat relative external-data names and supports graph initializers, including subgraphs. External sparse tensors, tensor attributes, model-local function data, and model packages are rejected. Minimal builds and unsupported platforms return ORT_NOT_IMPLEMENTED. Path-based loading, legacy ORT-format loading, and execution-provider selection are unchanged.
Validated on Linux and Windows. All 31 shared-library session-options and file-source tests pass in the full build and 12 capability tests pass in the minimal build, alongside end-to-end Linux CPU and Windows WebGPU D3D12 model-loading benchmarks.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
External sparse tensors and model-local function constants can bypass descriptor-registry validation and trigger filesystem reads even when the registry is empty.
Adds an experimental Linux API for loading ONNX external initializers from pre-opened file descriptors, allowing hosts to authorize model assets before filesystem lockdown without changing the stable API table.
Changes:
Registers named, read-only descriptors with atomic replacement and ownership retained across option clones and sessions.
Maps registered initializer ranges into CPU tensors, copying small or unaligned data and rejecting unsupported loading modes.
Adds API validation and lifetime tests, minimal-build capability checks, and selected WebGPU adapter logging.
Register owned read-only descriptors through the experimental API, map bounded initializer ranges before graph resolution, and reject missing sources without reopening paths. Preserve existing loading when no source registry is supplied. Include selected-adapter diagnostics for hardware validation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Convert the external-data PathString before formatting diagnostics so the descriptor loader compiles with Windows wide-character paths. Assert that missing-source diagnostics retain the model's external-data filename.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise inference parity after closing the original descriptor and releasing session options, preserve valid registrations after a failed replacement, reject path-only descriptors, and cover unsupported builds. Clarify empty-registry and unsupported-build semantics in the experimental API contract.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The descriptor API is unavailable in minimal builds, so do not retain its session/model state or descriptor-duplication virtual implementation there. Preserve the existing three-argument ModelOptions constructor and keep the experimental API's explicit unsupported status. Validate both the full runtime and a real no-operators, exceptions-disabled minimal shared library.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Omit the unavailable descriptor entry and its stub from minimal experimental lookup instead of retaining unsupported functionality in the minimal binary. Keep explicit NotImplemented status on unsupported non-minimal builds and test capability absence against a real minimal runtime. Full Linux descriptor inference is unchanged; do not relax the Android size budget.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Avoid MSVC C4127 when the descriptor initializer loader checks the host byte order. Preserve the same unsupported-endianness error without disabling warnings or changing initializer loading.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Append default-initialized file-source state after existing SessionOptions members instead of shifting positional aggregate fields. The unchanged training-runner initializer now compiles with missing-field warnings treated as errors. Minimal layouts remain unchanged and the full runtime retains descriptor ownership.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend named initializer sources to Windows, Linux, Android, macOS and iOS. Duplicate native Windows handles with read-only rights and reuse copy-on-write range mappings; retain CLOEXEC read-only descriptor ownership on POSIX. Run the existing API ownership and inference tests across supported platforms.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mustapha Jaber (mustjab)
changed the title
Add experimental Linux external initializer file-descriptor sources
Support native file handles for external initializers
Oct 28, 2026
Apply the single-space alignment requested by the existing clang-format CI check. No code tokens or runtime behavior change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Create the writable Windows test file with a native delete-on-close handle in the per-user temporary directory, and use secure wide-character CRT opening for the read-only fixture. Avoid C4996 without suppressing warnings or weakening handle ownership and inference checks. All11 full and8 minimal API cases pass locally; Windows CI remains required.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reject external sparse tensors and external function data before schema inference or Graph construction in both Model paths. Validate zero-byte offsets against source length. Add sparse/default-function/nested-graph regressions and256-byte aligned/unaligned initializer lifetime inference tests to the existing shared-library suite.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Duplicate initializer names can bypass registry checks, allowing an external tensor’s location to reach filesystem validation despite an empty registry and violating the no-filesystem-fallback contract.
Validate initializer-name uniqueness before Graph maps can hide protobuf entries. Cover external/embedded duplicates in both orders with an empty registry, and document the strict contract. All16 official API cases and8 minimal capability checks pass locally.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
BOOL initializers larger than 127 bytes bypass normalization, allowing noncanonical bytes to reach boolean consumers and produce inconsistent results.
No open issues 🎉
4 previously missed issues
In code that hasn't changed since last review
Normalize large BOOL external initializers
onnxruntime/core/graph/graph.cc:4502
BOOL initializers larger than 127 bytes retain their file bytes in a graph-owned OrtValue, which is reused or uploaded without the usual boolean normalization. Noncanonical bytes such as 0x02 and 0xFF can therefore reach boolean consumers and produce inconsistent results. Normalize the tensor before conversion and add a regression above the inline-size threshold.
Subgraph inference still validates unavailable external paths
onnxruntime/core/graph/graph.cc:4460
Subgraph injection updates copied node attributes but leaves the original NodeProto used by the first Resolve() unchanged. ONNX checking therefore still validates external initializer pathnames in If/Loop subgraphs, so a registered handle does not suffice when the original path is unavailable. Make checking consume an updated representation with mapped data safely inlined, and add a successful subgraph-inference regression with the weight path unavailable.
Detect external initializer and Constant name collisions
onnxruntime/core/graph/model.cc:90
An external initializer X plus an embedded Constant producing X passes this preflight, but Graph construction replaces the lookup entry with the Constant's initializer. File-source injection then misses the external tensor, and later validation accesses its pathname even with an empty registry. Include Constant outputs in the duplicate-name check and add this collision to the regression tests.
With ORT_LOAD_CONFIG_FROM_MODEL=1, FinalizeSessionOptions() replaces the caller's options and discards external_initializer_file_sources. This load path then skips preflight and injection and falls back to pathname loading, even when the caller installed an empty registry to prohibit fallback. Preserve the registry when finalizing model-supplied options, or reject this combination before external-data access.
Normalize BOOL initializers in owned storage, check updated subgraphs with safely inlined data, reject Constant outputs hiding external initializers, and retain source capabilities when model metadata supplies session options. Each of the four new public API regressions failed against the previous runtime and passes with these fixes. Also verified 11 legacy external-data cases, minimal capability omission, and the 285-case GenAI native suite.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve the existing checker representation for legacy ORT-format models. Only graphs initialized through registered file sources materialize updated subgraphs, avoiding invalid ONNX name checks on ORT-format Loop bodies while keeping unavailable-path file-source inference working.
Added public API tests first and confirmed both fail against the previous runtime with the exact Loop graph-name error. Cover path/buffer/no-copy/mmap inference and ONNX-to-ORT serialization/reload, verifying123-to125 outputs. After the fix,38 official API/ORT-format/legacy cases,8 checks against an actual minimal rebuild, and32 focused GenAI memory/file-source cases all pass without skips.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Validation of file-source models with If/Loop subgraphs copies entire initializer payloads into temporary buffers, with nested control flow repeating the copies and risking memory exhaustion.
Use a checker-only representation for file-source graphs. Rebind already-validated mapped initializer names as typed inputs in the temporary graph while preserving the live graph, inference buffers, ordinary initializer checks and legacy ORT behavior. Recurse through current If/Loop subgraphs without inlining mapped tensor payloads.
Add a fresh-process Linux RSS regression using32MiB external weights: the previous implementation grew by130MiB at depth1 and514MiB at depth3, exceeding a16MiB budget. The corrected checker stays below11MiB in repeated runs. Keep this unsanitized RSS budget separate from sanitizer shadow/quarantine accounting; portable nested If, Loop and malformed-graph tests remain enabled. All40 related official ORT cases pass three times, plus8 checks against a minimal rebuild and32 focused GenAI cases including13 native-handle cases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because graph-list attributes can bypass registry-backed external-data handling, compounded by duplicate-attribute validation gaps and size-dependent initializer override behavior.
…ndles
Retain registered RandomAccessFile provenance on Graph for aligned numeric external initializers and route eligible device-allocated weights through IExternalDataLoader before falling back to CPU-mapped OrtValues. Extend D3D12AcceleratedExternalDataLoader and D3D12FileBufferLoader to reopen pre-opened Windows file handles with overlapped unbuffered and tail-buffered flags without path lookup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Reviewed this head with bounded domain specialists, a separate integration reviewer, and independent verification. Posting seven verified findings: three medium correctness findings, two low internal/validation correctness findings, and two low missing-test findings. Existing matching-head Windows GPU and Linux CPU test logs were inspected; local validation was limited to static proofs and native Windows primitive probes, not ORT/GPU session execution.
Expose OrtFileHandle and SessionOptionsSetExternalInitializerFileHandles directly on OrtApi and Ort::SessionOptionsImpl in ORT 1.32, remove the experimental C API lookup entries, and move the shared-library file-source tests to test_session_options.cc.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Convert logical file-source names to UTF-8 via PathToUTF8String in D3D12
external data loader diagnostics while retaining filesystem::path only for
normalized batch keys.
- Reject non-empty embedded TensorProto data fields on EXTERNAL initializers
before replacing them in Graph::InjectExternalInitializersFromFileSources.
- Preserve ExternalInitializerFileSource provenance across Graph::InlineIfSubgraph,
MemcpyTransformer initializer duplication (including outer-scope lookup with
local-shadowing semantics), and graph_utils initializer copy helpers.
- Reject dense external initializers inside unsupported GRAPHS attributes and
their nested GRAPH descendants during ValidateFileSourceModel prevalidation.
- Satisfy WindowsRandomAccessFile::Read via read-only mapped views for handles
duplicated by DuplicateReadOnlyFileHandle so shared synchronous FILE_OBJECT
cursors are never modified.
- Expand unit test coverage across test_session_options.cc,
external_data_loader_test.cc, and webgpu_context_test.cc.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Model-package loading fails to reject external-initializer registries, including empty ones, and materializing small external weights causes subsequent initializer overrides to fail.
…zer overrides
- Reject non-null external_initializer_file_sources registries (including
empty registries) in CreateModelPackageOptionsFromSessionOptions,
ModelPackageOptions, OrtModelPackageAPI::CreateSession, and
CreateSessionForModelPackage.
- Retain external provenance for small (<= 127B), BOOL, and unaligned
file-source initializers so subsequent AddExternalInitializers overrides
succeed while still rejecting non-external initializers and withholding
file-source loader eligibility when unmapped.
- Add unit tests in test_session_options.cc covering model package rejection
and AddExternalInitializers overrides for both small and large weights.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because an unpublished critical finding indicates that exception-dependent test code prevents full ORT_NO_EXCEPTIONS builds from compiling.
…ONS builds
Replace C++ try/catch/throw and iostream exception masks in
FileSourcesSessionOptionsTest with C API OrtStatus checks and ORT_ENFORCE so
full builds with onnxruntime_DISABLE_EXCEPTIONS=ON (-DORT_NO_EXCEPTIONS
-fno-exceptions) compile and run all file-source tests cleanly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Iterate directly over graph_proto_->mutable_initializer() in
Graph::InjectExternalInitializersFromFileSources and pass the mutable
TensorProto* to ReplaceInitializedTensorImpl so each replacement verifies map
consistency in O(1) without scanning mutable_initializers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because a critical initializer-copying issue can break mixed-provider control-flow compilation, while invalid-name tests fail to detect name-validation regressions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Edward Chen (edgchen1)
left a comment
A sandboxed process may have permission to use model files without permission to open their filesystem paths. This change adds SessionOptionsSetExternalInitializerFileHandles and its C++ SessionOptions wrapper to the stable ONNX Runtime 1.32 API so a host can register named, pre-opened external initializer files before loading an ONNX graph.
ONNX Runtime duplicates native Windows file handles or read-only POSIX file descriptors and retains them across option clones and sessions. Initializer ranges are loaded or mapped from those handles without resolving or canonicalizing filesystem paths. On Windows WebGPU, the D3D12 accelerated external data loader reopens registered handles directly before falling back to CPU-mapped initializers. Missing sources and invalid ranges fail explicitly rather than falling back to disk access, and nested-graph validation inspects initializer metadata without copying mapped weight payloads.
The API accepts flat relative external-data names and supports graph initializers, including subgraphs. External sparse tensors, tensor attributes, model-local function data, and model packages are rejected. Minimal builds and unsupported platforms return ORT_NOT_IMPLEMENTED. Path-based loading, legacy ORT-format loading, and execution-provider selection are unchanged.
Validated on Linux and Windows. All 31 shared-library session-options and file-source tests pass in the full build and 12 capability tests pass in the minimal build, alongside end-to-end Linux CPU and Windows WebGPU D3D12 model-loading benchmarks.