Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there any resources of insight about memory cosumption of Voyager with around 100milions embeddings? #98

Open
Lorenzoncina opened this issue Nov 8, 2024 · 1 comment
Labels
question Further information is requested

Comments

@Lorenzoncina
Copy link

No description provided.

@stephen29xie stephen29xie added the question Further information is requested label Dec 16, 2024
@stephen29xie
Copy link
Contributor

Hi @Lorenzoncina. The number of embeddings in the index is just one factor that affects the memory usage so it is difficult to know based on that alone. Other significant factors are the dimensionality of the vectors, the StorageDataType of the vector values, and M value used for index construction. You can use a reduced precision StorageDataType and smaller M value to reduce memory usage, but at the cost of lower precision and recall.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants