Multi-file plot.bin wrapper #3274
Labels
farmer
Farming library/app
good first issue
Good for newcomers
improvement
it is already working, but can be better
performance
Related to performance measurement or improvement
refactor
Right now we have a problem on Windows: due to NTFS file system that triggers full write of the file we disable plot cache on large farms and even for small farms writing of a couple terabytes takes a really long time.
We can do an annoying thing and split
plot.bin
into multiple files, let's say 100G each and create a wrapper data structure similar toDirectIoFile
that will allow to transparently read and write contents as if it was a single file.This way we constrain writes to just one file worth of space at a time and improve UX for Windows users.
The text was updated successfully, but these errors were encountered: