You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the unmapping and re-mmapping take ~400ms for 256 shards. Could be mitigated with a "predictive growth plus trim at the end" strategy.
This may seem like a lot, however for real data, unlike in a test, where building the whole filter (for hibp for example) takes 2-3minutes for 1billion entries, the 400ms is not significant.
The primary use case for sharded_filter is for such large datasets, so this would probably be a microptimisation.
The text was updated successfully, but these errors were encountered:
oschonrock
changed the title
performance during repeated binfuse::sharded_filter::add()
performance during repeated binfuse::sharded_filter::add_shard()Dec 11, 2024
the unmapping and re-mmapping take ~400ms for 256 shards. Could be mitigated with a "predictive growth plus trim at the end" strategy.
This may seem like a lot, however for real data, unlike in a test, where building the whole filter (for hibp for example) takes 2-3minutes for 1billion entries, the 400ms is not significant.
The primary use case for
sharded_filter
is for such large datasets, so this would probably be a microptimisation.The text was updated successfully, but these errors were encountered: