-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve the RTC process of Read/Write model #2542
Comments
Which PikiwiDB functionalities are relevant/related to the feature request?Pika has implemented hybrid storage: it caches hot data in the in-memory RedisCache and stores full data on disk. Now, it is necessary to further optimize the read and write processes on this basis to increase throughput and reduce latency. Descriptionfrom https://tech.meituan.com/2024/03/15/kv-squirrel-cellar.html Proposed solutionprocess:
Result: The new thread model has been measured to increase service read throughput by 30%+ in a memory engine hit rate scenario of 80%. Although the new thread queue model only implements RTC for read cache hit requests, in fact, most of the online traffic is in scenarios where there are more reads and less writes, obvious hotspot data, and a relatively high memory engine hit rate. Therefore, after the new model goes online, most of the business clusters have achieved significant performance improvements. Alternatives considerednone |
1 Pikiwidb 将是内存和磁盘混合存储模型 |
1 Pikiwidb will be a hybrid memory and disk storage model |
RTC 对同一个连接读写顺序性的保证:假设 X = 2,有请求 "set X 3; get X“,原来顺序能够保证”get X“ 结果是 3。 RTC 改造后,redis client 能够保证在 "set X 3; " 这条指令没得到结果之前,不会执行 ”get X“,所以仍然能保证结果是 3。 |
RTC guarantees the order of reading and writing on the same connection: assuming X = 2, there is a request for "set X 3; get X", the original order can guarantee that the result of "get X" is 3. After the RTC transformation, the redis client can guarantee that "get X" will not be executed before the "set |
related doc http://mysql.taobao.org/monthly/2019/08/09/ |
related doc http://mysql.taobao.org/monthly/2019/08/09/ |
@cheniujh 跟进 |
@cheniujh follow up |
Which PikiwiDB functionalities are relevant/related to the feature request?
Pika 已经实现了混合存储:在内存 RedisCache 中缓存热数据,在磁盘中存储全量数据。现需要在这个基础之上,继续优化读写流程,提高吞吐,降低延时。
Description
from https://tech.meituan.com/2024/03/15/kv-squirrel-cellar.html
Proposed solution
process:
Result:
新的线程模型,经实测在 80% 内存引擎命中率场景下,服务读吞吐可以提升 30%+。虽然新的线程队列模型只实现了读缓存命中请求的 RTC,但其实在线流量大多都是读多写少且热点数据明显、内存引擎命中率比较高的场景,所以,新模型上线后在大多数的业务集群都取得了明显的性能提升。
Alternatives considered
none
The text was updated successfully, but these errors were encountered: