[init] persistent feature #43
Conversation
|
yulong.gyl seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
src/log_ring_file.c
Outdated
| { | ||
| return -1; | ||
| } | ||
| if (fileOffset + buffer_size - nowOffset > file->maxFileSize) |
There was a problem hiding this comment.
每次写的数据量 write_size = min(buffer_size - nowOffset , file->maxFileSize - fileOffset); 这样可以把if else 这两段同样的代码逻辑精简到一起
src/log_ring_file.c
Outdated
| return -1; | ||
| } | ||
| int rst = 0; | ||
| if (fileOffset + buffer_size - nowOffset > file->maxFileSize) |
src/log_persistent_manager.c
Outdated
| totalOffset += manager->in_buffer_log_sizes[id % manager->config->maxPersistentLogCount]; | ||
| } | ||
|
|
||
| manager->checkpoint.start_file_offset += totalOffset; |
There was a problem hiding this comment.
最终的start_file_offset值，最好直接传入， 如果 endId - startId > manager->config->maxPersistentLogCount的话， 最终计算出来的totalOffset 是不准确的。
另外， 只要任意一个地方发生错误， 整体计算的offset 都会错误。 这个容错性看上去有点差。
There was a problem hiding this comment.
写入的时候会判断是否到达 maxPersistentLogCount
没有传入offset的原因是对之前的producer改动太大，所以没有加这个功能
src/log_persistent_manager.c
Outdated
| log_persistent_checkpoint * checkpoint = &(manager->checkpoint); | ||
| checkpoint->check_sum = checkpoint->start_log_uuid + checkpoint->now_log_uuid + | ||
| checkpoint->start_file_offset + checkpoint->now_file_offset; | ||
| if (manager->checkpoint_file_size > MAX_CHECKPOINT_FILE_SIZE) |
There was a problem hiding this comment.
manager->checkpoint_file_size >= MAX_CHECKPOINT_FILE_SIZE ， 看语义应该是这样
| return -3; | ||
| } | ||
| // set empty log uuid len 0 | ||
| for (int64_t emptyUUID = logUUID + 1; emptyUUID < header.log_uuid; ++emptyUUID) |
There was a problem hiding this comment.
最大循环 manager->config->maxPersistentLogCount 次就好了， 避免出现reset 等情况， logUUID和 header.log_uuid 值相差巨大的情况。 所有 对于uuid的循环都可以这样处理
There was a problem hiding this comment.
所有使用这种for之前，都会去判断Size相差多少，超过1024*1024这种invalid的都会直接返回出错
[add] support time auto fix
[modify] refine delay log policy
…-sdk into persistent
[modify] always retry once when send error is discard data
This reverts commit cb679aa.
[modify] check persistent checkpoint before add
fix network panic log
* add windows * update flags * feature: use _WIN32 and stdint * use export api * use bundle curl * fix vla error * fix c2306 * fix open file sync * fix gettime in win * fix sample sleep in windows * fix file sync, time * support set curl * stash save cmake * add option for use curl * fix file write * rm extra set * refine cmake * refine * add windows pkg * open file in utf8, fix log print, refine func to utils * add ut * update ut * chore: update comment * chore: reformat * fix compile in linux * fix linux warn strict * add linux cmake support * chore: add indent * chore: remove space
support persistent log send