-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
List API for the state store building block #61
base: main
Are you sure you want to change the base?
Changes from 1 commit
6bf9e88
5a1cae5
dc407ca
df5edde
6e29898
529e302
ab04f50
648440d
ce3199a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
Signed-off-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -164,19 +164,19 @@ Here's a list of the relevant capabilities of all the stable state stores: | |
|
||
| Store | Cursor listing | Offset listing | Sorting | Number of Items per Page | Prefix Search | Comments | | ||
| --- | --- | --- | --- | --- | --- | --- | | ||
| aws dynamodb | Yes | No | Yes, with a GSI | Yes | Yes, with an additional sortKey and a GSI | In order to be able to use prefix search, users will need to have a Global Search Index(GSI) where the partition key will be a single fixed string (for ex. the `_` character) and the sort key will be the key name. There are some drawbacks to this that can be discussed in detail elsewhere. | | ||
| azure blob store | Yes (continuation token) | No | Always sorted in ASC order. Desc, or unsorted is not possible. | Yes | Yes | Results are always sorted by key name in ascending order. | | ||
| azure cosmos db | Yes | Yes | Yes | Yes | Yes | | | ||
| azure table storage | Yes | No | Yes, just ASC | Yes, with $top | Yes, with range search | Partition key is the application id. | | ||
| cassandraYes | No | No | No | No | Can’t prefix search and sort across all partitions. We could consider maintaining a new table containing all keys, and mirroring the original key’s ttl. | | | ||
| cockroachdbYes, if sorting is required | Yes | Yes | No | Yes | Need to create an index on the search column | | | ||
| gcp firestore | Yes | | | | | | | ||
| in-memory | No | No | No | No | No | We can implement all the features, but it’s not trivial to aggregate data across multiple instances | | ||
| memcached | No | No | No | No | No | | | ||
| mongodbYes | Yes | Yes | | Yes | | | | ||
| mysqlYes | Yes | | | Yes | Need to create an index on the id column. MySql supports specialized prefix indices, but you would have to know the exact length of the prefix you’ll be searching on, also sorting will not use the index. | | | ||
| postgresqlYes | Yes | Yes | No | Yes | Need to create an index on the key column. We can use the varchar\_pattern\_ops operator class, optimised for prefix search. | | | ||
| redisYes | No | No | No | Yes | Number of record per page is not guaranteed, but best effort. | | | ||
| **aws dynamodb** | Yes | No | Yes, with a GSI | Yes | Yes, with an additional sortKey and a GSI | In order to be able to use prefix search, users will need to have a Global Search Index(GSI) where the partition key will be a single fixed string (for ex. the `_` character) and the sort key will be the key name. There are some drawbacks to this that can be discussed in detail elsewhere. | | ||
| **azure blob store** | Yes (continuation token) | No | Always sorted in ASC order. Desc, or unsorted is not possible. | Yes | Yes | Results are always sorted by key name in ascending order. | | ||
| **azure cosmos db** | Yes | Yes | Yes | Yes | Yes | | | ||
| **azure table storage** | Yes | No | Yes, just ASC | Yes, with $top | Yes, with range search | Partition key is the application id. | | ||
| **cassandra** | Yes | No | No | Yes | No | Can’t prefix search and sort across all partitions. We could consider maintaining a new table containing all keys, and mirroring the original key’s ttl. | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can cassandra to filtering within the same partition? It might be enough to begin with. It is a common thing in CosmosDb too for transaction API, for example. |
||
| **cockroachdb** | Yes, if sorting is required | Yes | Yes | Yes | Yes | Need to create an index on the search column | | ||
| **gcp firestore** | Yes | | | | | | | ||
| **in-memory** | No | No | No | No | No | We can implement all the features, but it’s not trivial to aggregate data across multiple instances | | ||
elena-kolevska marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| **memcached** | No | No | No | No | No | | | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This does not belong to state store IMO, so we should not dismiss list API just because of this one. |
||
| **mongodb** | Yes | Yes | Yes | Yes | Yes | | | ||
| **mysql** | Yes | Yes | | Yes | Yes | Need to create an index on the id column. MySql supports specialized prefix indices, but you would have to know the exact length of the prefix you’ll be searching on, also sorting will not use the index. | | ||
| **postgresql** | Yes | Yes | Yes | Yes | Yes | Need to create an index on the key column. We can use the varchar\_pattern\_ops operator class, optimised for prefix search. | | ||
| **redis** | Yes | No | No | Yes (Best effort) | Yes | Number of record per page is not guaranteed, but best effort. | | ||
| | | | | | | | | ||
| sqliteYes, if sorting is required | Yes | Yes | No | Yes | Need to create an index on the key column. It’s a standard b-tree index.We could maintain an index of all keys in a hash | | | ||
| sqlserver | Yes, if sorting is required | Yes | Yes | Yes | Yes | need to create a non-clustered index on the “key” column | | ||
| **sqlite** | Yes, if sorting is required | Yes | Yes | Yes | Yes | Need to create an index on the key column. It’s a standard b-tree index.We could maintain an index of all keys in a hash | | ||
| **sqlserver** | Yes, if sorting is required | Yes | Yes | Yes | Yes | need to create a non-clustered index on the “key” column | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From this list, we can clearly see that with cursor listing, page limit and prefix search, we will have plenty of coverage.