Support dynamic arbitrary filtering on NoSQL databases #1682

jnmt · 2024-04-18T09:56:30Z

Description

This PR adds cross-partition scan filtering (i.e., ScanAll with a where clause`) for non-JDBC databases. It filters results based on the specified arbitrary conditions in the ScalarDB layer. It also limits results in the ScalarDB layer if it's a cross-partition scan with filtering to count the results correctly.

Sorry for the big PR, but the basic concept is the same for all three databases. See inline comments for details.

Related issues and/or PRs

N/A

Changes made

Add dynamic filtering in Cassandra, Cosmos DB, and DynamoDB.
- Add filterable (and limitable) Scanner implementations.
- Move the function to evaluate conditional expressions from Snapshot to ScalarDbUtils.
- Add integration tests with conditions and limits for cross-partition scan.
Fix warning for incorrectly specified projections when getting keys in results.

Checklist

I have commented my code, particularly in hard-to-understand areas.
I have updated the documentation to reflect the changes.
Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
Tests (unit, integration, etc.) have been added for the changes.
My changes generate no new warnings.
Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

Dynamic filtering will also be applied for Get, GetWithIndex, Scan, and ScanWithIndex in this PR, but the builders to actually run those queries (and the integration tests) will be added in other PRs since this PR is already very big.

Release notes

Added dynamic arbitrary filtering for non-JDBC databases.

jnmt · 2024-04-18T10:53:41Z

...on-test/java/com/scalar/db/storage/cassandra/CassandraCrossPartitionScanIntegrationTest.java

+import org.junit.jupiter.api.Test;
+
+public class CassandraCrossPartitionScanIntegrationTest
+    extends DistributedStorageCrossPartitionScanIntegrationTestBase {


I only added storage-level tests for each database since transaction-level tests would be tested enough using JDBC databases and they would increase CI time. But please feel free to comment if more tests should be added.

I changed my mind and added integration tests with consensus commit since it's good to create a chance to detect bugs like the one I fixed in 9729ce8.

jnmt · 2024-04-18T10:53:57Z

core/src/main/java/com/scalar/db/storage/cassandra/FilterableScannerImpl.java

+      Scan scan, ResultSet resultSet, ResultInterpreter resultInterpreter) {
+    super(resultSet, resultInterpreter);
+    this.conjunctions = scan.getConjunctions();
+    this.left = scan.getLimit() > 0 ? scan.getLimit() : -1;


We need to manage left items that can be returned before reaching the limit.

jnmt · 2024-04-18T10:54:42Z

core/src/main/java/com/scalar/db/storage/cassandra/Cassandra.java

+    if (scan.getConjunctions().isEmpty()) {
+      return new ScannerImpl(handlers.select().handle(scan), resultInterpreter);
+    } else {
+      // Ignore limit to control it in FilterableScannerImpl
+      Scan scanAll = Scan.newBuilder(scan).limit(0).build();
+      return new FilterableScannerImpl(scan, handlers.select().handle(scanAll), resultInterpreter);
+    }


I introduce a new filterable scanner implementation so that the regular ScanAll (w/o filtering) does not have to check the existence of filters every time when iterating. It's also nice to keep the logic simple.

jnmt · 2024-04-18T10:59:26Z

...src/main/java/com/scalar/db/api/DistributedStorageCrossPartitionScanIntegrationTestBase.java

@@ -560,7 +560,6 @@ private Scan prepareScanWithLike(boolean isLike, String pattern) {
        .table(CONDITION_TEST_TABLE)
        .all()
        .where(condition)
-        .ordering(Ordering.asc(PARTITION_KEY_NAME))


I removed orderings (including at other places) to commonly use these tests in non-JDBC databases except for order-specifc tests.

jnmt · 2024-04-18T23:44:56Z

Sorry, I faced an issue when using this with consensus commit. Let me investigate and maybe fix it.

jnmt · 2024-04-19T09:23:54Z

...alar/db/storage/cassandra/ConsensusCommitCrossPartitionScanIntegrationTestWithCassandra.java

+  @Override
+  protected Properties getProps(String testName) {
+    Properties properties = CassandraEnv.getProperties(testName);
+    properties.setProperty(ConsensusCommitConfig.ISOLATION_LEVEL, "SERIALIZABLE");


Even though serializable isolation is not recommended in non-JDBC databases, as we warn when using it, I think it's better to use serializable for testing purposes so as not to miss bugs regarding extra-read.

It might be better to use serializable for other consensus commit integration tests if no concerns. Any thoughts?

Even though serializable isolation is not recommended in non-JDBC databases

Does this mean that we don't recommend using SERIALIZABLE with the cross-partition scan in non-JDBC databases?

Yes. I mean this warning: https://scalardb.scalar-labs.com/docs/latest/configurations/#cross-partition-scan-configurations. Even so, I thought these tests should be done with extra-read just in case.

jnmt · 2024-04-19T09:29:02Z

core/src/main/java/com/scalar/db/transaction/consensuscommit/Snapshot.java

+              // For JDBC databases or non-JDBC databases without filtering, we only get tx_id,
+              // tx_version, and primary key columns because we use only them to compare. For
+              // non-JDBC databases with filtering, we also need filtering columns to compare.
+              projectRequiredColumns(storage, scan);


This is the fix for a bug with the serializable consensus commit. We need non-primary key columns to perform filtering correctly in the ScalarDB layer if they are used in the conditions, even for the extra-read purpose.

[minor] Does it mean that the values of scanSet doesn't contain filtering columns in the case of JDBC (thanks to the condition pushdown?), while the values of scanSet contain filtering columns in the case of non-JDBC?

If so, I guess it would be a bit more helpful to understand if Scan tells whether filtering columns are contained with a method like scan.containsFilteringColumnsInKey() so that we can revise the condition in projectRequiredColumns like this?

if (scan.containsFilteringColumnsInKey()) { scan.getConjunctions() .forEach( conjunction -> conjunction ...

I might not get your point perfectly, but it doesn't mean whether the user's original Scan has filtering columns in projections or not. Do you think this condition if (!(storage instanceof JdbcDatabase)) is hard to understand the background? For JDBC databases, we don't have to care about the existence of filtering columns in projections, so I didn't want to check it excessively.

By the way, thanks to your question, I might find an issue regarding projections...

Thanks for the reply!

What I thought was as follows:

Why filtering columns were needed for non-JDBC (w/ filtering) ?

The results of this scan are put to currentReadMap as a part of the key

Then, the map is looked-up with the values of scanSet (TransactionResult latestResult = currentReadMap.get(key))

So, including filtering columns in projection is for consistency of the key types related to the equals()'s logic?

But I think I had the wrong idea... Can you elaborate a bit more why filtering columns are needed for projection in the case of non-JDBC with filtering?

Ahh, the reason is not related to the key consistency of currentReadMap since the validation is always handled with the primary key basis.

We originally project columns except for tx_id, tx_version, and primary key columns for efficiency since they are enough for the re-scan for the validation phase (extra-read). We now want to filter results in the ScalarDB layer for non-JDBC databases. So, we need to additionally have filtering columns here since they are used even for the re-scan, the same as the first scan.

Yeah, I misread the code around Key class... Sorry for bothering you.

We originally project columns except for tx_id, tx_version, and primary key columns

s/except for/only for/ ?

But, I think I don't still get why columns for filtering need to be projected. My understanding is as follows:

Scan instances stored in scanSet contain the original conditions

FilterableScannerImpl#getResultWithFiltering filters records based on the conditions. Whether the columns for filtering are projected isn't related to the filtering. (The columns for filtering included in the conditions are automatically scanned at the underlying storage layer, right?)

The columns for filtering of the result in currentReadMap are not used

So, the columns for filtering don't need to be projected?

I think I have some wrong ideas of them. Can you point out it?

Yeah, I misread the code around Key class... Sorry for bothering you.

No no, it's OK. I think this part is so complicated and worth while to share the background.

We originally project columns except for tx_id, tx_version, and primary key columns

s/except for/only for/ ?

Sorry, "only for".

But, I think I don't still get why columns for filtering need to be projected. My understanding is as follows:

Scan instances stored in scanSet contain the original conditions

Right. Precisely, the list of projections is basically cleared in CrudHandler for the sake of transaction processing. In that sense, it's a bit different from the original one specified by users.

FilterableScannerImpl#getResultWithFiltering filters records based on the conditions. Whether the columns for filtering are projected isn't related to the filtering. (The columns for filtering included in the conditions are automatically scanned at the underlying storage layer, right?)

Right. We always need columns specified in the where clause to filter results. (*)

The columns for filtering of the result in currentReadMap are not used

Right. currentReadMap is for the read set verification. So, primary key columns are enough, but, of course, the map should be based on the result of the scan with the filtering conditions in the current database state.

So, the columns for filtering don't need to be projected?

No. We used them to get the current results above.

Let's chat a bit if you have further questions : )

Thanks for the detailed answers!

I found my misunderstanding...

I thought Cassandra.scan() implicitly projected the columns for filtering when there were any conditions in the scan. What I expected was like this:

--- a/core/src/main/java/com/scalar/db/storage/cassandra/Cassandra.java +++ b/core/src/main/java/com/scalar/db/storage/cassandra/Cassandra.java @@ -14,6 +14,7 @@ import com.scalar.db.api.Mutation; import com.scalar.db.api.Put; import com.scalar.db.api.Result; import com.scalar.db.api.Scan; +import com.scalar.db.api.ScanBuilder.BuildableScanOrScanAllFromExisting; import com.scalar.db.api.Scanner; import com.scalar.db.common.AbstractDistributedStorage; import com.scalar.db.common.TableMetadataManager; @@ -125,7 +126,9 @@ public class Cassandra extends AbstractDistributedStorage { return new ScannerImpl(handlers.select().handle(scan), resultInterpreter); } else { // Ignore limit to control it in FilterableScannerImpl - Scan scanAll = Scan.newBuilder(scan).limit(0).build(); + BuildableScanOrScanAllFromExisting builder = Scan.newBuilder(scan); + withProjectionsForFilteringIfMissing(builder); + Scan scanAll = builder.limit(0).build(); return new FilterableScannerImpl(scan, handlers.select().handle(scanAll), resultInterpreter); } }

BTW, assuming the above example works, it is a kind of provisioning the columns for filtering at storage layer while the change of this PR is provisioning the columns for filtering at an upper layer. Which do you think better?

Right. As you thought, we needed to handle projections properly in the storage layer, and it would be better to hide such a storage matter from the upper layer. So, your question got to the core of the matter from the beginning. Thanks again. Fixed in 6b1e92a.

Thank you for the consideration!

jnmt · 2024-04-19T09:31:49Z

Now ready for review! PTAL, when you get a chance since the issue was fixed in 9729ce8.

komamitsu · 2024-04-22T05:45:18Z

core/src/main/java/com/scalar/db/storage/cosmos/FilterableScannerImpl.java

+        }
+      } else {
+        // Otherwise, advance to the next page
+        currentPageRecords = recordsPages.next().getResults().iterator();


[trivial] I'm not sure if recordsPages.next() fetches the next page data lazily, but checking if left is zero before calling next() might be always good?

Like this

while (currentPageRecords.hasNext() || recordsPages.hasNext()) { if (left == 0) { return Optional.empty(); } // Return the next record of the current page if there is one if (currentPageRecords.hasNext()) { Optional<Result> result = getResultWithFiltering(currentPageRecords.next()); if (result.isPresent()) { return result; }

Good point. Thank you. Yeah, we don't have to call next() if there are no results left that can be returned. Moreover, we don't have to check left several times in one() since decreasing left only happens in getResultWithFiltering at most once. So, the following should be possible (the same as for all other scanners):

public Optional<Result> one() { if (left == 0) { return Optional.empty(); } while (currentPageRecords.hasNext() || recordsPages.hasNext()) { // Return the next record of the current page if there is one if (currentPageRecords.hasNext()) { Optional<Result> result = getResultWithFiltering(currentPageRecords.next()); if (result.isPresent()) { return result; }

Fixed in new FilterableScanner of 6b1e92a.

komamitsu · 2024-04-22T05:56:40Z

core/src/main/java/com/scalar/db/storage/dynamo/FilterableQueryScanner.java

+import java.util.Set;
+import javax.annotation.Nonnull;
+
+public class FilterableQueryScanner extends QueryScanner implements Scanner {


@NotThreadSafe?

Fixed in new FilterableScanner of 6b1e92a.

jnmt · 2024-04-24T09:59:14Z

@komamitsu Based on the feedback, I updated the following in 6b1e92a. PTAL again when you get a chance.

Fixed a bug where both projections and conditions are given and the projections do not include the columns specified in conditions.
Refactored filtering and limiting in the storage layer using a wrapper scanner.

brfrn169

Did my first review for this PR and left several comments. Please take a look when you have time! I'll review it again later.

Also, I have a question: it seems that for Cassandra, Cosmos DB, and DynamoDB, we always execute dynamic filtering for the conjunctions. Are there any cases where we can push down the conjunctions to the databases?

Additionally, I think we could support conjunctions for the normal Scan and ScanWithIndex. What are your thoughts on this?

brfrn169 · 2024-04-24T13:49:11Z

...alar/db/storage/cassandra/ConsensusCommitCrossPartitionScanIntegrationTestWithCassandra.java

+
+  @Test
+  @Override
+  @Disabled


We can document the reason why this test is disabled as follows:

Suggested change

@Disabled

@Disabled("<REASON>")

Adding some documentation would be beneficial in this case.

The same applies to other parts in this PR.

Thank you! Fixed in 580fae5 and edd250c.

brfrn169 · 2024-04-24T13:53:04Z

core/src/main/java/com/scalar/db/common/FilterableScanner.java

+  }
+
+  @Override
+  public void close() {}


I think we should close the scanner here:

Suggested change

public void close() {}

public void close() throws IOException {

scanner.close();

}

Good catch! Thank you!

Fixed in 580fae5.

brfrn169 · 2024-04-24T13:54:41Z

core/src/main/java/com/scalar/db/storage/cassandra/Cassandra.java

      throw new IllegalArgumentException(
-          CoreError.CASSANDRA_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED


We no longer use CASSANDRA_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED, so we can remove it from CoreError.

Ahh, we do not maintain old ones for reference, do we? OK, then I will remove them.

I think we can remove old error messages.

@feeblefakie @josh-wong What do you think?

Since this PR is for version 3.13 and later, I think we can remove the error message since it won't appear in version 3.13 or later.

Note

We should keep the error message in the doc for version 3.12 since the error message will still exist in that version.

Fixed in 580fae5. Note that, as discussed with Toshi, we need to maintain deleted messages to avoid reusing already deleted message IDs somehow, especially the most recent ones.

brfrn169 · 2024-04-24T13:55:24Z

core/src/main/java/com/scalar/db/storage/cosmos/Cosmos.java

      throw new IllegalArgumentException(
-          CoreError.COSMOS_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED


Ditto. We can remove CoreError. COSMOS_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED.

Fixed in 580fae5.

brfrn169 · 2024-04-24T13:55:52Z

core/src/main/java/com/scalar/db/storage/dynamo/Dynamo.java

      throw new IllegalArgumentException(
-          CoreError.DYNAMO_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED


Ditto. We can remove CoreError. DYNAMO_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED.

Fixed in 580fae5.

brfrn169 · 2024-04-24T13:56:41Z

core/src/main/java/com/scalar/db/storage/cassandra/ScannerImpl.java

It seems like this change of this file is not needed?

Sorry, my bad. Will fix.

Fixed in 580fae5.

brfrn169 · 2024-04-24T13:58:10Z

core/src/main/java/com/scalar/db/storage/cosmos/ScannerImpl.java

Ditto. It seems like this change of this file is not needed?

Fixed in 580fae5.

brfrn169 · 2024-04-24T13:58:24Z

core/src/main/java/com/scalar/db/storage/dynamo/QueryScanner.java

Ditto. It seems like this change of this file is not needed?

Fixed in 580fae5.

jnmt · 2024-04-25T00:51:02Z

@brfrn169 Thanks for the comments. Those are very good points.

We can push down the conditions with some limitations. For example, Cosmos DB and DynamoDB do not support a LIKE operator. Cassandra has it, but only for index columns. Since all those databases scan all partitions in any case, I guess the performance benefit could not be so significant in spite of the effort for per-storage implementation. But, if it’s highly prioritized, we can improve one by one (in the future, IMHO).

For Scan and ScanWithIndex, my first thought was that supporting them was a bit complicated for the benefits and use cases, especially when it comes to considering ScalarDB SQL, as I mentioned in a Demo session. But after reconsideration, it’s very nice to have for non-JDBC database users.

Since we additionally need the following, and they will get a bit big changes to some extent, I think it’s good to separate PRs. What do you think?

API changes so that we can call where after specifying keys
JDBC database changes for better query building when mixing partition-scan conditions
Non-JDBC database changes with dynamic filtering (maybe not so big deal)
(ScalarDB SQL changes for a kind of optimization when mixing partition-scan conditions)

komamitsu · 2024-04-25T02:21:06Z

core/src/main/java/com/scalar/db/common/ProjectedResult.java

+    }
+    for (Value<?> value : key.get()) {
+      if (!containedColumnNames.contains(value.getName())) {
+        logger.warn("Full key doesn't seem to be projected into the result");


In this case, it means Cassandra.scan() or equivalents didn't setup proper projections? If so, it's not an expected situation and throwing an AssertionError or IllegalStateException would be good, I think.

Honestly, I just followed the same manner as FilteredResult in the consensus commit, but IllegalStateException with a message like CoreError.COLUMN_NOT_FOUND would be better as you mentioned.

@brfrn169 Do you remember why you chose a warning rather than an exception?

@komamitsu Thank you for the suggestion. I fixed it in 580fae5, including other similar classes in this PR based on the discussion with @brfrn169.

core/src/main/java/com/scalar/db/storage/dynamo/SelectStatementHandler.java

josh-wong

I've left one minor suggestion. Other than that, LGTM. Thank you!🙇🏻‍♂️

docs/configurations.md

jnmt · 2024-05-01T05:46:03Z

I will close this PR because we discussed and decided to first refactor Scan.Conjunction, ScanBuilder.ConditionSetBuilder, and so on so that we can support the where clause and dynamic filtering in Get. I will re-open once the refactoring PR is merged.

Co-authored-by: Josh Wong <[email protected]>

jnmt · 2024-05-17T09:17:28Z

@feeblefakie @komamitsu @brfrn169 @Torch3333
I revised and reopened this PR since it's now ready for review. One of the major changes is to make it handle not only cross-partition scans but also partition and index scans and get operations in the same manner. However, the builder part (e.g., get with where) and integration tests for get and partition/index scan will be added in other PRs since this PR is already big.

komamitsu · 2024-05-17T10:32:52Z

core/src/main/java/com/scalar/db/common/FilterableScanner.java

+  private final Scanner scanner;
+  private final List<String> projections;
+  private final Set<Conjunction> conjunctions;
+  private int left;


[trivial] How about using Integer so that we can clearly represent N/A with null ?

Yeah, that's better than the unclear value like -1. Fixed in 74ab4fb. Note that I needed to add @SuppressFBWarnings to suppress a warning regarding left--, although I guess it is due to the false positive since left is used in the condition for returning the empty.

komamitsu · 2024-05-17T10:33:46Z

core/src/main/java/com/scalar/db/common/FilterableScanner.java

+  private final Set<Conjunction> conjunctions;
+  private int left;
+
+  private ScannerIterator scannerIterator;


[trivial] @Nullable or @LazyInit might be good?

Fixed in 74ab4fb. Thank you! I didn't know the @LazyInit annotation :P

brfrn169

Left several comments. Please take a look when you have time!

brfrn169 · 2024-05-20T03:36:49Z

...alar/db/storage/cassandra/ConsensusCommitCrossPartitionScanIntegrationTestWithCassandra.java

+  @Override
+  protected Properties getProps(String testName) {
+    Properties properties = CassandraEnv.getProperties(testName);
+    properties.setProperty(ConsensusCommitConfig.ISOLATION_LEVEL, "SERIALIZABLE");


Even though serializable isolation is not recommended in non-JDBC databases

Does this mean that we don't recommend using SERIALIZABLE with the cross-partition scan in non-JDBC databases?

brfrn169 · 2024-05-20T03:50:43Z

core/src/main/java/com/scalar/db/storage/cassandra/Cassandra.java

+      if (get.getConjunctions().isEmpty()) {
+        scanner = getInternal(get);
+      } else {
+        scanner = new FilterableScanner(get, getInternal(copyAndPrepareForDynamicFiltering(get)));


Actually since we already copied the get object here (https://github.com/scalar-labs/scalardb/pull/1682/files#diff-660ca76c9b64d94000e0854e2a18fdc5e6a69c028a74b8cacfa4ef733d4604c2L100), I don't think we need to copy the get object here with copyAndPrepareForDynamicFiltering() again.

Thank you for the comment. I wanted to pass the original get for FilterableScanner and the different modified get for getInternal() since we need the original limit value and projections in FilterableScanner. It might be possible to avoid copying the whole get operation somehow, but I think the current one is not so bad from the readability perspective. What do you think?

That makes sense. Thanks.

Torch3333

LGTM, thank you!

feeblefakie

LGTM! Thank you!

komamitsu

LGTM, thank you!

brfrn169

LGTM! Thank you!

Support cross-partition scan with filtering on NoSQL databases

05435c8

jnmt commented Apr 18, 2024

View reviewed changes

jnmt self-assigned this Apr 18, 2024

jnmt added the enhancement New feature or request label Apr 18, 2024

jnmt requested review from komamitsu, brfrn169, feeblefakie, Torch3333 and josh-wong April 18, 2024 11:01

jnmt marked this pull request as draft April 18, 2024 23:45

Fix issue when using serializable consensus commit

9729ce8

jnmt commented Apr 19, 2024

View reviewed changes

jnmt marked this pull request as ready for review April 19, 2024 09:31

komamitsu reviewed Apr 22, 2024

View reviewed changes

komamitsu self-requested a review April 23, 2024 01:01

Fixes based on feedback

6b1e92a

brfrn169 reviewed Apr 24, 2024

View reviewed changes

komamitsu reviewed Apr 25, 2024

View reviewed changes

josh-wong approved these changes Apr 25, 2024

View reviewed changes

docs/configurations.md Outdated Show resolved Hide resolved

jnmt closed this May 1, 2024

jnmt mentioned this pull request May 8, 2024

Refactor scan with filtering #1715

Merged

6 tasks

jnmt added 3 commits May 15, 2024 11:33

Merge branch 'master' into support-cross-partition-filtering-on-nosql

e202e68

Merge branch 'master' into support-cross-partition-filtering-on-nosql

fc7a869

Fix based on feedback and follow recent changes in the base branch

580fae5

jnmt reopened this May 17, 2024

jnmt and others added 3 commits May 17, 2024 14:49

Fix disabled reason

edd250c

Fix to remove unnecessary changes

05e3087

Update docs/configurations.md

2f47375

Co-authored-by: Josh Wong <[email protected]>

jnmt changed the title ~~Support cross-partition scan with filtering on NoSQL databases~~ Support dynamic arbitrary filtering on NoSQL databases May 17, 2024

jnmt requested review from komamitsu and brfrn169 May 17, 2024 09:17

komamitsu reviewed May 17, 2024

View reviewed changes

brfrn169 reviewed May 20, 2024

View reviewed changes

Torch3333 approved these changes May 20, 2024

View reviewed changes

feeblefakie approved these changes May 21, 2024

View reviewed changes

Revise based on feedback

74ab4fb

jnmt requested a review from komamitsu May 21, 2024 08:24

komamitsu approved these changes May 21, 2024

View reviewed changes

brfrn169 approved these changes May 22, 2024

View reviewed changes

Merge branch 'master' into support-cross-partition-filtering-on-nosql

2f0b323

brfrn169 merged commit 693388b into master May 22, 2024
24 checks passed

brfrn169 deleted the support-cross-partition-filtering-on-nosql branch May 22, 2024 14:46

feeblefakie mentioned this pull request May 22, 2024

Backport to branch(3) failed: Support dynamic arbitrary filtering on NoSQL databases #1760

Closed

jnmt added a commit that referenced this pull request May 23, 2024

Support dynamic arbitrary filtering on NoSQL databases (#1682)

f7a0979

jnmt mentioned this pull request May 23, 2024

Backport to branch(3) : Support dynamic arbitrary filtering on NoSQL databases #1761

Merged

Torch3333 mentioned this pull request May 23, 2024

Fix integration test failure because of conflicting use of table names #1762

Merged

6 tasks

jnmt mentioned this pull request May 23, 2024

Add arbitrary filtering for partition scan and index scan #1763

Merged

6 tasks

brfrn169 mentioned this pull request May 24, 2024

Should add prefix to namespaces for Cosmos DB integration tests #1767

Merged

6 tasks

jnmt mentioned this pull request Jun 4, 2024

Add arbitrary filtering for get operations #1834

Merged

6 tasks

-  public void close() {}
+  public void close() throws IOException {
+    scanner.close();
+  }

		throw new IllegalArgumentException(
		CoreError.CASSANDRA_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED

		throw new IllegalArgumentException(
		CoreError.COSMOS_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED

		throw new IllegalArgumentException(
		CoreError.DYNAMO_CROSS_PARTITION_SCAN_WITH_FILTERING_OR_ORDERING_NOT_SUPPORTED

Support dynamic arbitrary filtering on NoSQL databases #1682

Support dynamic arbitrary filtering on NoSQL databases #1682

Conversation

jnmt commented Apr 18, 2024 • edited Loading

Description

Related issues and/or PRs

Changes made

Checklist

Additional notes (optional)

Release notes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt commented Apr 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

komamitsu Apr 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt commented Apr 19, 2024

Choose a reason for hiding this comment

jnmt Apr 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt commented Apr 24, 2024

brfrn169 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt commented Apr 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt May 17, 2024 • edited Loading

Choose a reason for hiding this comment

josh-wong left a comment

Choose a reason for hiding this comment

jnmt commented May 1, 2024

jnmt commented May 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brfrn169 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnmt commented Apr 18, 2024 •

edited

Loading

komamitsu Apr 24, 2024 •

edited

Loading

jnmt Apr 22, 2024 •

edited

Loading

jnmt May 17, 2024 •

edited

Loading