Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recsplit don't store MPH when keys are monotonically increasing integers. #13748

Open
sudeepdino008 opened this issue Feb 8, 2025 · 3 comments
Labels

Comments

@sudeepdino008
Copy link
Member

sudeepdino008 commented Feb 8, 2025

get rid of MPHT for cases when keys are incrementing integers. These can be stored simply as ef.
Discussion

@wmitsuda
Copy link
Member

wmitsuda commented Feb 9, 2025

FYI: read the discussion on Discord about baseId + ef, I also had this requirement in my non-merged branch, maybe we can dedup some code in the future when it is merged: https://github.com/erigontech/erigon/blob/wmitsuda/ef-optimization-experiments/erigon-lib/recsplit/eliasfano32/rebased_elias_fano.go

@sudeepdino008
Copy link
Member Author

FYI: read the discussion on Discord about baseId + ef, I also had this requirement in my non-merged branch, maybe we can dedup some code in the future when it is merged: https://github.com/erigontech/erigon/blob/wmitsuda/ef-optimization-experiments/erigon-lib/recsplit/eliasfano32/rebased_elias_fano.go

So it seems like we will already have the no-MPH thingy with your optimizations @wmitsuda?

@wmitsuda
Copy link
Member

actually we don't, I think. I did only better encoding in the .ef contents, but I didn't changed other places that use the Elias-fano code like other snapshot files (I saw the code briefly, but it is out of scope for this change + simple encoding doesn't apply in those cases; but maybe the rebased elias fano does because I saw your comment in Discord)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants