-
Notifications
You must be signed in to change notification settings - Fork 182
mv cache object warming
- merged to master -
- code complete - July 30, 2015
- development started - July 24, 2015
leveldb's performance is highly dependent upon its caches: file cache, block cache, and operating system block cache. A restart of leveldb flushes at least the file cache and block cache. A server restart flushes all three. An application that depends heavily upon leveldb for fast access is weakened right after a leveldb starts due to two or three of its caches lacking good content.
The file cache is the most performance critical cache. A miss against the file cache requires an .sst table file to be opened, index read, index decompressed, bloom filter read, and bloom filter decompressed. All this happens before attempting to read the requested data. Large datasets can easily open 500 or more .sst table files per second while trying to properly populate the file cache. Each file open is not only slow, but also interferes with other leveldb file operations such as reads for an already open file.
The user application can suffer for minutes during a server restart. With Riak, the performance of an entire server cluster is impacted by the restart of just one server. This makes the problem even more critical since individual servers regularly need maintenance and restarts for software updates and security patches.