Voyager: Corrupted or unsupported index after saving.

[uelca]

Issue Description:

I encountered an error when trying to load an index created using Voyager as the backend on a Windows system. The error message displayed is: "Corrupted or unsupported index."

Steps to Reproduce:

Create an index using Voyager as the backend.
Save the index.
Attempt to load the index on a Windows system.

Expected Behavior:
The index should load without any errors.

Actual Behavior:
The system throws an error stating "Corrupted or unsupported index."

Reference:

This issue is known and has been documented in the Voyager repository: GitHub Issue #40.

Resolution:

To resolve the issue, you can modify the load method in the voyager.py file. Follow these steps:

Navigate to the Vicinity directory within your virtual environment.
Locate the file vicinity/backend/voyager.py.
Modify the load method as follows:

with open(path, 'rb') as f:
index = Index.load(f)

This will fix the error.

[Pringled]

Hi @uelca, thanks for flagging and for the informative issue! I've just created a PR here: #69

Could you let me know if this resolves the issue? I don't have a Windows system to test this on, but if you could try installing the branch with pip install git+https://github.com/MinishLab/vicinity.git@fix-voyager and let me know if everything works as expected I can include this in the next release.

[uelca]

Hi @Pringled,

Thanks for the quick turnaround! I appreciate you looking into this.
the provided fix is not working. You currently use the following to load:

with open(path, "rb"):
     index = Index.load(str(path))

this will still cause the error, instead it should be the following:

with open(path, 'rb') as f:
    index = Index.load(f)

I assume that the Index.load() method expects to be given an active stream (f) to read from, not just the name of the file (str(path)).

[Pringled]

Hi @uelca, you're right, my bad 🤦 . Just pushed the fix you proposed, let me know if this works and I'll merge and do a release soon!

[uelca]

@Pringled thanks for the fix, works now!

[Pringled]

Great, thanks for creating an issue! I've just made a new release with the fix, see https://github.com/MinishLab/vicinity/releases/tag/v0.4.2. Closing this, let me know if you encounter any other problems!