Skip to content

Commit

Permalink
[-] updated README with more info & examples (#6)
Browse files Browse the repository at this point in the history
[-] added example for custom store
[-] added info on how thundering herd problem is avoided
[-] added info on how Pocache combines different strategies to keep the cache fresh all the time
  • Loading branch information
bnkamalesh authored Oct 13, 2024
1 parent b6ad05e commit d8c4b00
Showing 1 changed file with 59 additions and 10 deletions.
69 changes: 59 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,35 +10,84 @@

Pocache (`poh-cash (/poʊ kæʃ/)`), **P**reemptive **o**ptimistic cache, is a lightweight in-app caching package. It introduces preemptive cache updates, optimizing performance in concurrent environments by reducing redundant database calls while maintaining fresh data. It uses [Hashicorp's Go LRU package](https://github.com/hashicorp/golang-lru) as the default storage.

Yet another _elegant_ solution for the infamous [Thundering herd problem](https://en.wikipedia.org/wiki/Thundering_herd_problem), save your database(s)!

## Key Features

1. **Preemptive Cache Updates:** Automatically updates cache entries _nearing_ expiration.
2. **Threshold Window:** Configurable time window before cache expiration to trigger updates.
3. **Serve stale**: Opt-in configuration to serve even expired cache and do a background refresh.
4. **Debounced Updates:** Prevents excessive I/O calls by debouncing concurrent requests for the same key.
5. **Custom store**: customizable underlying storage to extend in-app cache to external database
3. **Serve stale**: Opt-in configuration to serve expired cache and do a background refresh.
4. **Debounced Updates:** Prevents excessive I/O calls by debouncing concurrent update requests for the same key.
5. **Custom store**: customizable underlying storage to extend/replace in-app cache or use external cache database.

## Why use Pocache?

In highly concurrent environments (e.g., web servers), multiple requests try to access the same cache entry simultaneously. Without query call suppression / call debouncing, the app would query the underlying database multiple times until the cache is refreshed. While trying to solve the thundering herd problem, most applications serve stale/expired cache until the update is completed.

Pocache solves these scenarios by combining debounce mechanism along with optimistic updates during the threshold window, keeping the cache up to date all the time and never having to serve stale cache!

## How does it work?

Given a cache expiration time and a threshold window, Pocache triggers a preemptive cache update when a value is accessed within the threshold window.

Example:

- Cache expiration: 10 minutes
- Threshold window: 1 minute

```
|______________________ __threshold window__________ ______________|
0 min 9 mins 10 mins
Add key here Get key within window Key expires
|______________________ ____threshold window__________ ______________|
0 min 9 mins 10 mins
Add key here Get key within window Key expires
```

When a key is fetched between 9-10 minutes (within the threshold window), Pocache initiates an update for that key (_preemptive_). This ensures fresh data availability, anticipating future usage (_optimistic_).
When a key is fetched within the threshold window (between 9-10 minutes), Pocache initiates a background update for that key (_preemptive_). This ensures fresh data availability, anticipating future usage (_optimistic_).

## Custom store

## Why use preemptive updates?
Pocache defines the following interface for its underlying storage. You can configure storage of your choice as long as it implements this simple interface, and is provided as a configuration.

In highly concurrent environments (e.g., web servers), multiple requests might try to access the same cache entry simultaneously. Without preemptive updates, the system would query the underlying database multiple times until the cache is refreshed.
```golang
type store[K comparable, T any] interface {
Add(key K, value *Payload[T]) (evicted bool)
Get(key K) (value *Payload[T], found bool)
Remove(key K) (present bool)
}
```

Additionally by debouncing these requests, Pocache ensures only a single update is triggered, reducing load on both the underlying storage and the application itself.
Below is an example(not for production use) of setting a custom store.

```golang
type mystore[Key comparable, T any] struct{
data sync.Map
}

func (ms *mystore[K,T]) Add(key K, value *Payload[T]) (evicted bool) {
ms.data.Store(key, value)
}

func (ms *mystore[K,T]) Get(key K) (value *Payload[T], found bool) {
v, found := ms.data.Load(key)
if !found {
return nil, found
}

value, _ := v.(*Payload[T])
return value, true
}

func (ms *mystore[K,T]) Remove(key K) (present bool) {
_, found := ms.data.Load(key)
ms.data.Delete(key)
return found
}

func foo() {
cache, err := pocache.New(pocache.Config[string, string]{
Store: mystore{data: sync.Map{}}
})
}
```

## Full example

Expand Down

0 comments on commit d8c4b00

Please sign in to comment.