Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust): implement a ListingCatalog #3300

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

yanghua
Copy link
Collaborator

@yanghua yanghua commented Dec 26, 2024

No description provided.

@github-actions github-actions bot added the enhancement New feature or request label Dec 26, 2024
@yanghua yanghua force-pushed the 3290-catalog-rust-interface branch from 6a32543 to f5347b5 Compare December 26, 2024 09:30
@codecov-commenter
Copy link

codecov-commenter commented Dec 26, 2024

Codecov Report

Attention: Patch coverage is 88.38710% with 18 lines in your changes missing coverage. Please review.

Project coverage is 78.85%. Comparing base (58c5e27) to head (cb98b79).

Files with missing lines Patch % Lines
rust/lance/src/catalog/catalog_trait.rs 88.38% 9 Missing and 9 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3300      +/-   ##
==========================================
+ Coverage   78.81%   78.85%   +0.03%     
==========================================
  Files         250      251       +1     
  Lines       91306    91461     +155     
  Branches    91306    91461     +155     
==========================================
+ Hits        71963    72121     +158     
+ Misses      16390    16376      -14     
- Partials     2953     2964      +11     
Flag Coverage Δ
unittests 78.85% <88.38%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yanghua yanghua force-pushed the 3290-catalog-rust-interface branch from f5347b5 to 1d308e0 Compare December 26, 2024 09:56
@yanghua
Copy link
Collaborator Author

yanghua commented Dec 26, 2024

Hi @westonpace I drafted a PR for discussion.

About naming, some thoughts:

  • xxx_dataset or xxx_table API in Catalog which one do you prefer?
  • xxx_namespace or xxx_database in Catalog which one do you prefer?

fn list_datasets(&self, namespace: &Namespace) -> Vec<DatasetIdentifier>;

/// Create a new dataset in the catalog.
fn create_dataset(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using register_dataset interface to register an existed lance dataset is also needed.

fn invalidate_dataset(&self, identifier: &DatasetIdentifier) -> Result<(), String>;

/// Register a dataset in the catalog.
fn register_dataset(&self, identifier: &DatasetIdentifier) -> Result<Dataset, String>;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SaintBacchus it's here

@yanghua yanghua changed the title feat(rust): introduce catalog interface for rust module feat(rust): implement a ListingCatalog Jan 4, 2025
@yanghua yanghua force-pushed the 3290-catalog-rust-interface branch from 8f2bec4 to a1f3359 Compare January 27, 2025 07:14
@yanghua yanghua self-assigned this Jan 27, 2025
}

#[async_trait::async_trait]
impl Catalog for ListingCatalog {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@westonpace I implemented a simple version for discussion. Please take a look when you have time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants