Repository navigation
Fix overflow in the max_array_size check of array pad - #1218
Merged
Merged
Conversation
Collaborator
|
Great find. I wonder would we need to do this for maps as well? Perhaps the |
Collaborator
|
Ah seems that maps don't have a padding function so it doesn't apply. |
Contributor
Author
|
Agreed on maps. On check_data_size, I don't think it needs the same treatment: it and the calc_*_sizes helpers only add up data that already exists, element by element, so there's no script-supplied count multiplied in and the totals hit a limit long before they could get near usize::MAX. When I ran the other built-ins with huge length arguments, pad was the only one that got past the check. Thanks for merging this. |
Collaborator
|
I'm wondering whether we should also change |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With
max_array_sizeset, padding an array to a huge length gets past the size check and takes down the host:array_functions::padprojects the padded size asa + pad + ax * pad(andm + mx * pad,s + sx * padfor maps and strings) in plainusizearithmetic, andpadcomes straight from the script. In a release build the totals wrap: each copy of the 63-element item counts 64 towards the array total, 64 × 2^58 = 2^64 wraps to 0,throw_on_sizesees an empty array, andVec::resizethen asks for 4 EiB. The allocation failure aborts the process, so neithercatch_unwindnor any engine limit can stop it. A smaller item gives a panic instead, e.g.a.pad(4611686018427387904, [1, 2, 3])hits "capacity overflow", and debug builds panic on the overflowing arithmetic itself. The totals are now saturated, so an oversized pad returnsErrorDataTooLargelike any other, and pads that fit compute the same totals as before.padis the only place where a script-supplied count is multiplied into a size check; the string and BLOBpadandblob(len, value)compare the length directly.Added an assertion to
test_max_array_sizeintests/data_size.rs, gated to 64-bit builds withoutonly_i32where the length fits. Without the fix it panics; with it, it passes.cargo test --testspasses with default features and with theno_object,no_index,only_i32,f32_float,only_i32,unchecked,sync,decimal,compact_strandno_float,only_i32,no_object,no_index,no_module,no_function,syncrows of the CI matrix.cargo fmt --check, the CI clippy invocation andcargo +1.66.0 checkagainstCargo.msrv.lockare clean. CHANGELOG line under 1.27.1's bug fixes.