You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In addition to being affected by the memory-inefficiency of chunk uploading (#2291), sourcemap uploads are affected by even more severe memory usage issues.
From investigating a sourcemap upload with a memory profiler, I observed three large memory allocations which we should attempt to fix. Note that these tests were performed with --no-rewrite (more info on this later):
The first (and longest) allocation reads all of the sourcemaps into memory. This allocation lasts for pretty much the entire program run. We perform the allocation on this line, which we call when collecting sourcemaps for upload.
We can probably address this allocation by memory-mapping the sourcemap files.
However, we perform some processing/validation with the sourcemap crate. The crate's documentation states that it loads the entire sourcemap into memory, so we might end up still needing at least to allocate one sourcemap at a time into memory, even after addressing this problem.
Lastly, without --no-rewrite, memory usage seems to be even worse. We still need to investigate the case where --no-rewrite is not passed to Sentry CLI.
The text was updated successfully, but these errors were encountered:
Symbolic version `12.13.3` includes [a change](getsentry/symbolic#890), which will reduce, in some cases significantly, the memory usage of sourcemap uploads.
ref #2344
In addition to being affected by the memory-inefficiency of chunk uploading (#2291), sourcemap uploads are affected by even more severe memory usage issues.
From investigating a sourcemap upload with a memory profiler, I observed three large memory allocations which we should attempt to fix. Note that these tests were performed with
--no-rewrite
(more info on this later):sourcemap
crate. The crate's documentation states that it loads the entire sourcemap into memory, so we might end up still needing at least to allocate one sourcemap at a time into memory, even after addressing this problem.symbolic
when we validate that the sourcemap is valid UTF-8. Fix is here: ref(sourcebundle): Check UTF-8 validity memory efficiently symbolic#890 and build: Bumpsymbolic
to12.13.3
#2346Lastly, without
--no-rewrite
, memory usage seems to be even worse. We still need to investigate the case where--no-rewrite
is not passed to Sentry CLI.The text was updated successfully, but these errors were encountered: