Skip to content

Fix NamedArrayPartition Vector{Int} / UnitRange indexing (#583)#584

Merged
ChrisRackauckas merged 2 commits intoSciML:masterfrom
ChrisRackauckas-Claude:fix-vector-int-indexing-583
Apr 28, 2026
Merged

Fix NamedArrayPartition Vector{Int} / UnitRange indexing (#583)#584
ChrisRackauckas merged 2 commits intoSciML:masterfrom
ChrisRackauckas-Claude:fix-vector-int-indexing-583

Conversation

@ChrisRackauckas-Claude
Copy link
Copy Markdown
Contributor

Fixes #583.

Root cause

05faa730b3 (Eliminate all invalidation trees) correctly narrowed getindex(::NamedArrayPartition, ...) from a catch-all args... form to just Int. Anything else now falls through AbstractArray's generic indexing, which routes through similar(x, T, dims).

That exposed a latent bug in NamedArrayPartition's similar overloads. When dims != size(A), the underlying similar(::ArrayPartition, T, dims) already degrades to a plain Vector — a partition layout no longer fits the requested shape (see src/array_partition.jl:72-78). The NamedArrayPartition overloads then tried to wrap that Vector via the inner constructor

NamedArrayPartition{T, A<:ArrayPartition{T}, NT}(::A, ::NT)

which raised the MethodError reported in #583. The pre-05faa730 catch-all args... getindex bypassed similar entirely, so the bug stayed hidden — getindex(::NamedArrayPartition, 1:2) short-circuited to getindex(ArrayPartition(x), 1:2) and never went through the broken similar path.

Fix

Make similar(::NamedArrayPartition, dims) and similar(::NamedArrayPartition, T, dims) mirror ArrayPartition's own degradation: wrap when the inner similar returns an ArrayPartition, otherwise propagate the fallback array as-is. No new method shadows a wider signature than before — using RecursiveArrayTools still produces 0 invalidation trees (verified locally with SnoopCompile.invalidation_trees).

function Base.similar(A::NamedArrayPartition, ::Type{T}, dims::NTuple{N, Int}) where {T, N}
    inner = similar(getfield(A, :array_partition), T, dims)
    inner isa ArrayPartition || return inner
    return NamedArrayPartition(inner, getfield(A, :names_to_indices))
end

x[1:end] keeps returning NamedArrayPartition (full-range slice → dims == size(A)similar(::ArrayPartition, T, dims) returns an ArrayPartition). x[1:2] and x[[1, 4]] now return a Vector{Float64} (the same fallback ArrayPartition exposes for non-matching dims).

Test plan

Added a regression @testset in test/named_array_partition_tests.jl:

  • Issue MWE: NamedArrayPartition(a=ones(2), b=2*ones(3))[1:2] succeeds and returns the right values.
  • Vector{Int} indexing: x[[1, 2]], x[[1, 4]].
  • Partial UnitRange: x[2:4].
  • Full-range slice still preserves type: x[1:end] isa NamedArrayPartition.
  • similar(x, Float64, (2,)) isa Vector{Float64}; similar(x, Float64, size(x)) isa NamedArrayPartition.
  • Scalar getindex / setindex! untouched.

Local results:

  • Pkg.test() passes end-to-end (NamedArrayPartition Tests | 35 35, all other suites green).
  • SnoopCompile.invalidation_trees(@snoop_invalidations using RecursiveArrayTools) reports 0 trees — same as the post-05faa730 baseline.

🤖 Generated with Claude Code

The invalidation-eliminating change in 05faa73 narrowed
`getindex(::NamedArrayPartition, ...)` from a catch-all `args...` form
to just `Int`. Anything else now falls through `AbstractArray`'s
generic indexing, which routes via `similar(x, T, dims)`.

That exposed a latent bug in the `similar` overloads: when
`dims != size(A)`, `similar(::ArrayPartition, T, dims)` intentionally
degrades to a plain `Vector` (a partition layout no longer fits the
requested shape). The `NamedArrayPartition` overloads then tried to wrap
that `Vector` via the inner constructor
`NamedArrayPartition{T, A<:ArrayPartition{T}, NT}(::A, ::NT)`, which
raised a MethodError. The catch-all `args...` getindex used to bypass
`similar` entirely, so the bug stayed hidden until 05faa73.

Make `similar(::NAP, dims)` and `similar(::NAP, T, dims)` mirror
`ArrayPartition`'s own degradation: wrap when the inner result is an
`ArrayPartition`, otherwise return the fallback array as-is. This
restores `x[1:2]` and `x[[1, 4]]` for partial slices, keeps `x[1:end]`
returning a `NamedArrayPartition` (the full-range slice still hits the
ArrayPartition path), and reintroduces no method that catches a wider
signature than necessary — `using RecursiveArrayTools` still produces
0 invalidation trees, verified with `SnoopCompile.invalidation_trees`.

Closes SciML#583.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The first commit on this branch fixed `x[1:2]` and `x[[1, 4]]` by making
`similar(::NAP, T, dims)` degrade to a Vector when `dims != size(A)`,
mirroring ArrayPartition. That worked, but it left the indexing path
inferring as `Union{NamedArrayPartition, Vector{Float64}}` because
`similar(::ArrayPartition, T, dims)` itself is a Union (the
`dims == size(A)` branch is a runtime check).

Add a `_unsafe_getindex(::IndexStyle, ::NAP, I::Vararg{Union{Real, AbstractArray}})`
shortcut that mirrors the one at array_partition.jl:317. Allocate the
destination directly off the underlying first array and fill it with
`Base._unsafe_getindex!`. The shortcut bypasses `similar` entirely for
the indexing path, so `x[1:2]`, `x[[1, 4]]`, `x[1:length(x)]` all infer
to a clean `Vector{Float64}`.

Trade-off: this regresses the post-05faa730 test
`typeof(x .+ x[1:end]) <: NamedArrayPartition` back to `<: Vector` (the
v3 behavior). That test was added in 05faa73 alongside the
invalidation cleanup, but its only effect was to mask the unstable
small-Union return; restoring v3 semantics here gives full type
stability and matches what ArrayPartition already does. Use
`similar(x)` / `copy(x)` if you want a NamedArrayPartition back.

The `similar(::NAP, dims)` and `similar(::NAP, T, dims)` overloads
keep the graceful-degrade-to-Vector behavior from the previous commit,
so direct `similar(x, T, (2,))` calls (e.g. from downstream library
code) still work.

`SnoopCompile.invalidation_trees(@snoop_invalidations using
RecursiveArrayTools)` still reports 0 trees, full `Pkg.test()` passes,
and the new regression test asserts type stability via `@inferred`.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude
Copy link
Copy Markdown
Contributor Author

Pushed 9cbf6d0 switching to the always-Vector / fully type-stable approach (option 2 from the analysis in #583).

What changed. Added an _unsafe_getindex(::IndexStyle, ::NamedArrayPartition, I::Vararg{Union{Real, AbstractArray}, N}) shortcut mirroring array_partition.jl:317. The indexing path now bypasses similar entirely:

Base.@propagate_inbounds function Base._unsafe_getindex(
        ::IndexStyle, A::NamedArrayPartition,
        I::Vararg{Union{Real, AbstractArray}, N}
    ) where {N}
    shape = Base.index_shape(I...)
    dest = similar(getfield(A, :array_partition).x[1], shape)
    Base._unsafe_getindex!(dest, A, I...)
    return dest
end

The similar(::NAP, dims) / similar(::NAP, T, dims) graceful-degrade fix from the previous commit stays in place for direct callers.

Inferred return types now:

x[1]              :: Float64        (stable)
x[1:2]            :: Vector{Float64} (stable)
x[1:length(x)]    :: Vector{Float64} (stable)
x[[1, 4]]         :: Vector{Float64} (stable)
ap[1:2]           :: Vector{Float64} (matches AP exactly)

Trade-off, called out in the commit message and the test diff. The post-05faa730 assertion typeof(x .+ x[1:end]) <: NamedArrayPartition regresses to <: Vector, which is the pre-05faa730 (v3) behavior. Users who want a NamedArrayPartition back should use similar(x) / copy(x).

Verified:

  • Pkg.test() — all suites green; NamedArrayPartition Tests | 40 40.
  • @inferred x[1:2] / @inferred x[1:length(x)] / @inferred x[[1, 4]] all return Vector{Float64} (asserted in the regression test).
  • SnoopCompile.invalidation_trees(@snoop_invalidations using RecursiveArrayTools) reports 0 trees, same as the post-05faa730 baseline.

@ChrisRackauckas ChrisRackauckas merged commit 951f7da into SciML:master Apr 28, 2026
30 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vector{Int} indexing of NamedArrayPartition broken for v4+

2 participants