Skip to content

Distributed KV data store with tunable consistency, synchronous replication

License

Notifications You must be signed in to change notification settings

flipkart-incubator/dkv

Folders and files

NameName
Last commit message
Last commit date

Latest commit

db32783 · May 13, 2024
Apr 11, 2023
Mar 29, 2023
Mar 29, 2023
Sep 2, 2022
May 13, 2024
Mar 31, 2023
Jan 5, 2023
Feb 6, 2020
Feb 6, 2024
Sep 6, 2022
Nov 4, 2019
Feb 6, 2024
Sep 11, 2022
Apr 7, 2022
Sep 11, 2022
Feb 15, 2023
Feb 15, 2023
Jun 11, 2021

Repository files navigation

dkv logo

DKV CI

DKV is a distributed key value store server written in Go. It exposes all its functionality over gRPC & Protocol Buffers.

Features

  • Data Sharding
  • Tunable consistency
  • Data replication over WANs

Supported APIs

  • Put(Key, Value)
  • MultiPut([]{Key, Value})
  • Get(Key, Consistency)
  • MultiGet([]Keys, Consistency)
  • Delete([]Keys)
  • CompareAndSet(Key, Value, OldValue)
  • Scan(KeyPrefix, StartKey)

Design

Design Diagram

Dependencies

  • Go version 1.16+
  • RocksDB v6.22.1 as a storage engine
  • GoRocksDB provides the CGo bindings with RocksDB
  • Badger v1.6 as a storage engine
  • Nexus for sync replication over Raft consensus

Running

Launching the DKV server in standalone mode

A single DKV instance can be launched using the following docker command:

docker run -it -p 8080:8080 ghcr.io/flipkart-incubator/dkv:latest dkvsrv

or while using native binaries using :

$ ./bin/dkvsrv --config dkvsrv.yaml  --db-folder <folder_name>  --listen-addr <host:port>

Any operations can be done using the dkvctl cli, or using the clients:

$ ./bin/dkvctl -a <host:port> --set <key> <value>
$ ./bin/dkvctl -a <host:port> --get <key>

Example session:

$ ./bin/dkvsrv --config dkvsrv.yaml --db-folder /tmp/db --listen-addr 127.0.0.1:8080
$ ./bin/dkvctl -a 127.0.0.1:8080 --set foo bar
$ ./bin/dkvctl -a 127.0.0.1:8080 --get foo
bar
$ ./bin/dkvctl -a 127.0.0.1:8080 --set hello world
$ ./bin/dkvctl -a 127.0.0.1:8080 --get hello
world
$ ./bin/dkvctl -a 127.0.0.1:8080 --del foo
$ ./bin/dkvctl -a 127.0.0.1:8080 --iter "*"
hello => world

Launching the DKV server in cluster mode

Please refer to the wiki instructions on how to run DKV in cluster mode.

Documentation

Detailed documentation on specific features, design principles, data guarantees etc. can be found in the dkv Wiki

Support

dkv is undergoing active development. Consider joining the dkv-interest Google group for updates, design discussions, roadmap etc. in the initial stages of this project.