Repository navigation
Two defects in vendored nanoarrow 0.9.0 to report upstream #17
Description
Found while building Stages 2 and 3 against the vendored nanoarrow 0.9.0 (src/vendor/nanoarrow/). zuarrow avoids both paths, so neither affects it today; both belong upstream in apache/arrow-nanoarrow.
1. ArrowHalfFloatToFloat() mishandles subnormals. In nanoarrow.h, an exponent of 0 sets the float exponent field to 0 but keeps the shifted fraction, which yields fraction * 2^-136 rather than the binary16 value fraction * 2^-24. For example 0x0001 should widen to 2^-24 (about 5.96e-8). ArrowArrayViewGetDoubleUnsafe() uses it for half_float arrays, so every subnormal half float reads wrong. zuarrow widens half floats itself (zar_half_to_double() in src/zar_to_r.c), as design §6.2 asks, and test-to-r.R checks subnormals.
2. ArrowBasicArrayStreamInit() on its second allocation failure. When ArrowMalloc() for the arrays fails, the function calls ArrowBasicArrayStreamRelease(array_stream) before it has stored private_data on the stream, so the release reads an unset private_data. The schema it has already moved into private_data also leaks. Only an out-of-memory path reaches it.
Next step: open the upstream issues (and a pull request for the first, which is small), then link them here and in src/vendor/PROVENANCE if a patch is ever vendored.
Activity
pedrobtz commentedon Oct 21, 2026
More nanoarrow 0.9.0 defects to report upstream, all found by fuzzing zuarrow's IPC reader (Stage 5). Each is fixed by a vendored patch listed in
src/vendor/PROVENANCE:ArrowIpcDecoderPeekFooter()formats%sfrom one past the end of the unterminated input.ArrowIpcDecoderReadHeaderPrefix()overflowsint32_tadding the prefix to a length nearINT32_MAX.DictionaryEncoding.indexType, a message header, a field type, the footer schema) are read through NULL.strlen(). That truncates binary values at a NUL and reads an absent key through NULL.Two gaps zuarrow covers itself rather than patching:
pedrobtz commentedon Oct 21, 2026
Another nanoarrow 0.9.0 encoder defect, found in Stage 6 (#22):
ArrowIpcEncoderFinalizeBuffer()writes the footer'sBlockstructs without zeroing their four bytes of padding. Files are then not byte-for-byte reproducible, and the padding leaks heap bytes. Vendored patch 0013 zeroes the padding along with addingArrowIpcEncoderEncodeBatch(). Both belong in the upstream pull request.pedrobtz commentedon Oct 21, 2026
Found by the fuzz job on #22: nanoarrow 0.9.0's IPC decoder lets two fields share a dictionary id with different value types. The dictionary batch is decoded with one field's type, and
ArrowIpcDecoderWalkSetArrayView()then views it with the other's, which reads past the buffers (heap-buffer-overflow inArrowArrayViewValidateDefault(), large-string offsets). zuarrow's reader now refuses such a schema before setting up the decoder, without a vendored patch. Regression input:tests/testthat/data/crash-shared-dictionary.arrow. Upstream should check this inArrowIpcDecoderSetSchemaWithDictionaries().pedrobtz commentedon Oct 21, 2026
Re-tested every regression input against nanoarrow
mainat efe4274 (2026-10-21). The inputs went through nanoarrow's ownArrowIpcArrayStreamReaderat full validation andArrowIpcDecoderVerifyFooter()/DecodeFooter(), built with ASan and UBSan.Already fixed on main (unreleased):
indexTypeStill present on main, now with upstream pull requests:
int32_tBlockpadding taken from the stackEach pull request has a test that fails without its fix, and passes
ctestin a plain build and under ASan and UBSan. Upstream CI is waiting for a maintainer to approve workflows for a first-time contributor.Not sent: patch 0004 (decoding views, list views and run-end encoding with variadic buffers). It is a feature, not a defect fix, since main's IPC reader refuses those types cleanly.
Once a release has these, the re-vendor can drop patches 0003 and 0005 to 0012 and most of 0013 (design §15 Q1).
pedrobtz commentedon Nov 3, 2026
Another nanoarrow 0.9.0 defect for the upstream report, found by the nightly fuzz run (#85):
ArrowArrayViewValidateFull()bounds a large list view element asoffset + size > child_len, which overflowsint64_tfor two large non-negative values, so a wrapped sum passes the check. Vendored patch 0014 comparessize > child_len - offset. Regression input:tests/testthat/data/crash-list-view-overflow.arrow.pedrobtz commentedon Nov 5, 2026
Every defect here is now reported upstream or already fixed on apache/arrow-nanoarrow
main(checked at 9bf2d88).New upstream PRs. Each has a test that fails without its fix.
ctestpasses in a plain build and under ASan and UBSan.ArrowBasicArrayStreamInit()leaks the schema and releases an unset stream when its second allocation failsoffset + sizeoverflowsint64_tAlready fixed on main:
zar_half_to_double()Earlier, still open upstream: apache/arrow-nanoarrow#984 to #990, listed in the comment above. Patch 0004 (decoding views, list views and run-end encoding) was not sent, because it is a feature and main refuses those types cleanly.
At the next re-vendor (design §15 Q1), drop each patch whose upstream fix has been released: 0003, 0005–0012, 0014–0016 and most of 0013.