Fix memory leak on exception caching #594
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
Comment on lines
+32
to
+33
| @pytest.mark.skipif( | ||
| reason="Memory leak is not fixed for PyPy3.9", |
There was a problem hiding this comment.
This suggests that it should be fixed at some point, in which case we probably want an xfail instead of a skip.
| @@ -222,6 +222,7 @@ async def __call__(self, /, *fn_args: Any, **fn_kwargs: Any) -> _R: | |||
| if self.__maxsize is not None and len(self.__cache) > self.__maxsize: | |||
| dropped_key, cache_item = self.__cache.popitem(last=False) | |||
| cache_item.cancel() | |||
| del cache_item | |||
There was a problem hiding this comment.
I'm not clear how this relates to exceptions specifically, but here's my understanding of the issue, which should be documented:
Suggested change
| del cache_item | |
| # While waiting for future below, we want dropped cache_item to be garbage | |
| # collected. Without the del, it is possible to end up with a chain of cache_item | |
| # references stopping them from being garbage collected at all. | |
| del cache_item |
There was a problem hiding this comment.
Should we be doing dropped_key too?
|
I think #595 seems more sensible. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What do these changes do?
When exception is "evicted" from cache, if following call raises an exception, both of them will be kept in memory introducing a memory leak.
Are there changes in behavior for the user?
No
Related issue number
199 is probably related
Checklist