Describe the bug, including details regarding any error messages, version, and platform.
Background
When manually creating an encoded vector, with values referencing indices in a Dictionary, it is possible to have values equal to valueCount of the Dictionary vector i.e. one index out of bounds. This is interpreted as a null value in the Dictionary.
Example
Dictionary
| Index |
Original Value |
| 0 |
Foo |
Encoded Vector - 1
| Index |
Encoded Value |
Expected Decode Outcome |
Actual Outcome |
| 0 |
0 |
Return the original value |
Returns the original value |
Encoded Vector - 2
| Index |
Encoded Value |
Expected Decode Outcome |
Actual Outcome |
| 0 |
1 |
Raise IllegalArgumentException |
Returns null |
Encoded Vector - 3
| Index |
Encoded Value |
Expected Decode Outcome |
Actual Outcome |
| 0 |
2 |
Raise IllegalArgumentException |
Raises IllegalArgumentException |
Test to reproduce the error
@Test
public void testReferencingIndexOutOfBounds() {
// Index at which the original value will be stored at in the dictionary
var encodedIndex = 0;
// The encoded value that references an index in the dictionary
var indexReferenced = 1;
try (final IntVector encodedVector = new IntVector("encodings", allocator);
final VarCharVector dictionaryVector = newVarCharVector("dict", allocator); ) {
var originalValue = "Foo";
dictionaryVector.allocateNew(1);
dictionaryVector.setValueCount(1);
dictionaryVector.set(encodedIndex, originalValue.getBytes(StandardCharsets.UTF_8));
encodedVector.allocateNew(1);
encodedVector.setValueCount(1);
encodedVector.set(0, indexReferenced);
Dictionary dictionary =
new Dictionary(dictionaryVector, new DictionaryEncoding(1L, false, null));
try (ValueVector decoded = DictionaryEncoder.decode(encodedVector, dictionary)) {
fail("There should be an exception when decoding index outside dictionary's range.");
} catch (Exception e) {
assertEquals("Provided dictionary does not contain value for index " + indexReferenced, e.getMessage());
}
}
}
Describe the bug, including details regarding any error messages, version, and platform.
Background
When manually creating an encoded vector, with values referencing indices in a Dictionary, it is possible to have values equal to
valueCountof the Dictionary vector i.e. one index out of bounds. This is interpreted as a null value in the Dictionary.Example
Dictionary
Encoded Vector - 1
Encoded Vector - 2
IllegalArgumentExceptionEncoded Vector - 3
IllegalArgumentExceptionIllegalArgumentExceptionTest to reproduce the error