You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #1274 while reading the image tarball from stdin, it is first saved to file, because export algorithm uses random file access as described here #1274 (comment)
To makes the export more efficient, the algorithm could cache image bits in memory until they are no longer needed. So, for example, if manifest.json is parsed, keep the parsed structure in memory and discard cached bytes that contained it. For well-aligned images it will save both speed and memory. For badly aligned the performance will be the same as with current temp file, because temp file is still written into memory tmpfs on Linux.
bufio can potentially help https://pkg.go.dev/bufio but it looks like a lib for helping with string scanning. Not sure it can handle several GB of memory cache efficiently.
This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.
TarBuffered scans stream (`io.Reader`) once for filename and saves
unused sections in memory for later access. This should speedup
parsing a bit, because right now tarball is scanned several times,
and should save resources and speed for parsing well-formed images
from network.
See google#1339.
TarBuffered scans stream (`io.Reader`) once for filename and saves
unused sections in memory for later access. This should speedup
parsing a bit, because right now tarball is scanned several times,
and should save resources and speed for parsing well-formed images
from network.
See google#1339.
In #1274 while reading the image tarball from stdin, it is first saved to file, because export algorithm uses random file access as described here #1274 (comment)
To makes the export more efficient, the algorithm could cache image bits in memory until they are no longer needed. So, for example, if
manifest.json
is parsed, keep the parsed structure in memory and discard cached bytes that contained it. For well-aligned images it will save both speed and memory. For badly aligned the performance will be the same as with current temp file, because temp file is still written into memorytmpfs
on Linux.bufio
can potentially help https://pkg.go.dev/bufio but it looks like a lib for helping with string scanning. Not sure it can handle several GB of memory cache efficiently.And here is a good article with code that allows to see random tar access in debug mode - https://blog.gopheracademy.com/advent-2017/seekable-http/
The text was updated successfully, but these errors were encountered: