You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dict compression does a very naive iteration of the array which constructs rust strings from the array data. Since we only need to hash the bytes we can avoid constructing those strings and we can avoid wrapping them in options.
The text was updated successfully, but these errors were encountered:
robert3005
changed the title
Dict compression of VarBinView can be optimized
Avoid constructing rust strings when performing varbin/varbinview dict compression
Jan 15, 2025
Dict compression does a very naive iteration of the array which constructs rust strings from the array data. Since we only need to hash the bytes we can avoid constructing those strings and we can avoid wrapping them in options.
https://github.com/spiraldb/vortex/blob/develop/encodings/dict/src/compress.rs#L88-L107
related to #1453
The text was updated successfully, but these errors were encountered: