You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is effectively a follow-up on #109 which is closed for some reason.
We have experienced a bug in production service when hard limits were removed and OOM killed the app. The app has to hold some data in cache which is then repeatedly re-read from DB and re-set in a fixed interval of ~30 min. Keys for saving data in memory are always the same. What we observed is that after few hours memory consumption in our service has grown above any limit.
I slightly modified code snipped from #109 to experiment and reproduce Bigcache behavior:
The memory usage growth depends only on evictionInterval.
So, for instance, on my linux machine, when evictionInterval set to
1 minute, RSS is ~877M
2 minutes, RSS is ~1680M
etc
So if evictionInterval is big enough and we keep setting data with same key, we would end up with OOM killer.
It doesn't matter whether GODEBUG=madvdontneed=1 is set or not. I run with this param, but it just seems to not affect anything.
Please note commented // cacheCfg.CleanWindow = time.Second line - I tried setting this param to different values starting from 1 sec, and it didn't help.
As a result the only way to limit memory consumption and prevent OOM is to set HardMaxCacheSize.
The text was updated successfully, but these errors were encountered:
Hello, I think the way the code works is intentional.
// Config for BigCachetypeConfigstruct {
...//Default value is 0 which means unlimited size. When the limit is higher than 0 and reached thenHardMaxCacheSizeint...
}
@Hojun-Cho , How is that intentional? Why should setting the same key increase the memory usage? Shouldn't it over write the same key with new value? If that's intentional, it's a bug. HardMaxCacheSize should not be used as workaround for this.
This is effectively a follow-up on #109 which is closed for some reason.
We have experienced a bug in production service when hard limits were removed and OOM killed the app. The app has to hold some data in cache which is then repeatedly re-read from DB and re-set in a fixed interval of ~30 min. Keys for saving data in memory are always the same. What we observed is that after few hours memory consumption in our service has grown above any limit.
I slightly modified code snipped from #109 to experiment and reproduce Bigcache behavior:
The memory usage growth depends only on
evictionInterval
.So, for instance, on my linux machine, when
evictionInterval
set toSo if
evictionInterval
is big enough and we keep setting data with same key, we would end up with OOM killer.It doesn't matter whether
GODEBUG=madvdontneed=1
is set or not. I run with this param, but it just seems to not affect anything.Please note commented
// cacheCfg.CleanWindow = time.Second
line - I tried setting this param to different values starting from 1 sec, and it didn't help.As a result the only way to limit memory consumption and prevent OOM is to set
HardMaxCacheSize
.The text was updated successfully, but these errors were encountered: