Request history

[jakekgrog]

We would like to be able to create a log of recent request history for the node (the last hour for example). These logs would contain information about what commands were executed, who the request came from (what client). Whether to request failed or succeeded etc. It is open for discussion here so please contribute if you've any good ideas!

[Adm28]

Hi @jakekgrog
I believe we can have a thread in the background which logs the history. I think so this feature will be a win when we look at it from the perspective of fault tolerance. We can replay the logs to rebuild the cache in event of node failure.
What is your use case behind this ?

[jakekgrog]

Hi @Adm28, we already have subsystems that provide on-disk persistence. These can be point-in-time snapshots or an append only file. These only take a snapshot of the cache state, we don't have a way of storing any information related to the actual http requests.

[Adm28]

@jakegrog I wasn't aware of it.
We can create a queue of struct. Each struct can have timestamp, request_id , command , client, response status etc as member variables. After every certain interval , all the elements of the queue until the element whose response status isn't known yet can be persisted and popped out from the queue.
How are you planning to go about this ?

[jakekgrog]

@Adm28 That is a similar approach to the one I'd have taken. Add each "request log struct" to a buffer and flush the buffer at regular intervals.