Repository navigation
[Framework] Add opt-in fixed-placement session memory diagnostics - #33657
Draft
Chi Lo (chilo-ms) wants to merge 1 commit into
Draft
Chi Lo (chilo-ms) wants to merge 1 commit into
Chi Lo (chilo-ms) wants to merge 1 commit into
Conversation
Capture local-graph initializer snapshots and alias-aware tensor/workspace lifetimes. Keep allocator-wide observations separate from modeled payloads and physical peaks unavailable until uncovered backing terms are resolved. Add opt-in logging, documentation, and focused CPU regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d876416-81ec-4982-a9b0-38b8b28b5e4a
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add opt-in fixed-placement memory diagnostics with
session.enable_memory_backing_report=1(default off).This is the first diagnostic adapter, not complete session backing accounting or a no-OOM guarantee. It changes no allocation, placement, contract-selection, or budget policy and adds no device allocation or explicit synchronization. Control-flow/subgraph reports are not composed into a session peak.
Validation
Windows CPU Release, Ninja/MSVC 2022, using the existing isolated incremental build with the Windows memory leak checker disabled.
From the configured CPU Release build directory:
Passed.
Passed: 22 tests, no skips. Eight new cases cover sequential/parallel scratch, input/initializer aliases, source release and packed destinations, dynamic shapes, known-zero versus unavailable workspace, reporting disabled, and a real report-enabled CPU Run with and without an arena. The runtime case verifies allocator-retained bytes are not added to the modeled payload peak and unsupported allocator statistics stay unavailable.
From the source worktree:
Both passed. No new CUDA, training-enabled, minimal-build, or cross-stream runtime validation was performed for this slice.
Motivation and Context
Implements the first fixed-placement reporting slice from #32944, stacked on #33240's L1/L2 reconciliation. The stack is #33316 -> #33240 -> this PR.
The report deliberately separates conditional logical lifetime estimates from physical backing observations. Retained pools, alignment/fragmentation, packed ownership/sharing, constructor/preparation scratch, caller I/O/KV capacities, private library/EP allocations, capture, retained outputs, subgraph residency, and concurrency still prevent a complete physical peak.
The remaining sequence is to complete those backing adapters, introduce enforceable pre-allocation session budgets, then add generic contract selection and CPU-region placement. CUDA workspace/activation-pattern integration remains in #32071, WebGPU region integration in #32720, and the placement design in #33372; this PR does not duplicate them.