Skip to content

[Java] DictionaryEncoder doesn't crash when decoding index outside of Dictionary #1158

@jakobhok-neo4j

Description

@jakobhok-neo4j

Describe the bug, including details regarding any error messages, version, and platform.

Background

When manually creating an encoded vector, with values referencing indices in a Dictionary, it is possible to have values equal to valueCount of the Dictionary vector i.e. one index out of bounds. This is interpreted as a null value in the Dictionary.


Example

Dictionary

Index Original Value
0 Foo

Encoded Vector - 1

Index Encoded Value Expected Decode Outcome Actual Outcome
0 0 Return the original value Returns the original value

Encoded Vector - 2

Index Encoded Value Expected Decode Outcome Actual Outcome
0 1 Raise IllegalArgumentException Returns null

Encoded Vector - 3

Index Encoded Value Expected Decode Outcome Actual Outcome
0 2 Raise IllegalArgumentException Raises IllegalArgumentException

Test to reproduce the error

    @Test
    public void testReferencingIndexOutOfBounds() {
        // Index at which the original value will be stored at in the dictionary
        var encodedIndex = 0;
        // The encoded value that references an index in the dictionary
        var indexReferenced = 1;

        try (final IntVector encodedVector = new IntVector("encodings", allocator);
             final VarCharVector dictionaryVector = newVarCharVector("dict", allocator); ) {
            var originalValue = "Foo";
            dictionaryVector.allocateNew(1);
            dictionaryVector.setValueCount(1);
            dictionaryVector.set(encodedIndex, originalValue.getBytes(StandardCharsets.UTF_8));

            encodedVector.allocateNew(1);
            encodedVector.setValueCount(1);
            encodedVector.set(0, indexReferenced);

            Dictionary dictionary =
                    new Dictionary(dictionaryVector, new DictionaryEncoding(1L, false, null));

            try (ValueVector decoded = DictionaryEncoder.decode(encodedVector, dictionary)) {
                fail("There should be an exception when decoding index outside dictionary's range.");
            } catch (Exception e) {
                assertEquals("Provided dictionary does not contain value for index " + indexReferenced, e.getMessage());
            }
        }
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions