Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consider further optimization/exploitation of FrozenMinHash #2133

Open
ctb opened this issue Jul 20, 2022 · 1 comment
Open

consider further optimization/exploitation of FrozenMinHash #2133

ctb opened this issue Jul 20, 2022 · 1 comment

Comments

@ctb
Copy link
Contributor

ctb commented Jul 20, 2022

Over in #2128, I wrote

Also while I am at it, any speed/memory advantage using FrozenHashes?

Not a big one. The main goal of FrozenMinHash is to enable future optimizations where we don't make new copies of FrozenMinHash objects when we don't need to. It also helps deal with object ownership issues that complicated the Rust-and-Python layers. You can read more #1616 if you're really interested ;).

...and then when debugging #2132, I immediately found a bunch of situations where we could cache various computations done in FrozenMinHash such as __len__.

So.

I think we could do some judicious profiling to guide the addition of cached properties to the FrozenMinHash class based on immutability of the objects.

@ctb
Copy link
Contributor Author

ctb commented Aug 15, 2022

(being explored in #2135)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant