Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure the composed store adaptors #2637

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

MathieuDutSik
Copy link
Contributor

@MathieuDutSik MathieuDutSik commented Oct 16, 2024

Motivation

We have the Journaling, ValueSplitting, LruCaching and MeteredStore but the potential is not fully exploited with a relatively large quantity of boilerplate code.

Proposal

The following was done:

  • The get_name() function is introduced to the AdminKeyValueStore. Unfortunately, it was not possible to have a &'static str since we want to have composed types and so composed names.
  • The introduction of the get_name() allows removing all the KeyValueStoreMetrics static variables and instead having a global store that contains the KeyValueStoreMeytrics.
  • That global variable allows to put metrics for the AdminKeyValueStore trait function.
  • The AdminKeyValueStore is added to all the Adaptor.
  • For the LruCaching this requires the introduction of the LruSplittingConfig that contains the configurations in a nested way.
  • The CommonStoreConfig is split with the cache_size being removed from it.
  • The pub store are systematically changed into store in the adaptors.

Possible criticism of the design:

  • There is no adaptor for the error types in the JournalingKeyValueStore. The rust difficulty was in the type Keys = K::Keys since this requires a nested key type and I think it was a little too complicated at this time.
  • The KeyValueStoreMetrics were previously as &'static LazyLock<KeyValueStoreMetrics>. Now they are as Arc<KeyValueStoreMetrics>.
  • There is no systematic adaptor for the error and config type, like e.g. LruSplittingError. But those would be pure boilerplate since for LRU, there is no additional error occurring.
  • The input has not changed. The next step is in using YAML file for the input.

Test Plan

CI. Nothing should be affected by the refactoring.

Release Plan

Not relevant.

Links

None.

@MathieuDutSik MathieuDutSik marked this pull request as ready for review October 17, 2024 10:54
@ma2bd ma2bd changed the title Restructure the composed store adaptor. Restructure the composed store adaptors Oct 18, 2024
};
let config = DynamoDbStoreConfig {
inner_config,
cache_size: common_config.cache_size,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cache_size belongs to a LruCacheConfig.

Copy link
Contributor

@ma2bd ma2bd Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So compared to this PR, my idea was to replace DynamoDbStoreConfig by LruCacheConfig<DynamoDbStoreConfig> and rename DynamoDbStoreInternalConfig into DynamoDbStoreConfig.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. Indeed this PR does not implement that idea. However, that can be done next.

Thus the next step would be to remove the "Internal" attribute to the RockDb, ScyllaDb, DynamoDb and others. And they could then be exposed to users so that users could run, e.g. with ScyllaDb with cache or without.

But I think this can be done in a subsequent work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants