Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrepancy between setting save_version via catalog constructor and when passing datasets #4327

Open
ElenaKhaustova opened this issue Nov 13, 2024 · 0 comments
Labels
Component: Framework Issue/PR that addresses core framework functionality Component: IO Issue/PR addresses data loading/saving/versioning and validation, the DataCatalog and DataSets

Comments

@ElenaKhaustova
Copy link
Contributor

Description

In the catalog constructor

save_version: str | None = None,
and from_config
save_version: str | None = None,
method we require passing one save version for all datasets. We further use this save version to instantiate all datasets objects.

This constraint is bypassed when passing instantiated dataset objects to catalog constructor.

Context

  1. It's not clear whether this constraint is a requirement.
  2. It's not clear what save version should we save and load back when serialization/deserialization [DataCatalog]: Spike - Catalog serialization and deserialization support #3932

Possible Implementation

  1. Clarify the requirement of having single load_version per catalog.
  2. Based on the first point either remove the requirement or apply it for instantiated datasets passed to the catalog.

Possible Alternatives

@ElenaKhaustova ElenaKhaustova added Component: Framework Issue/PR that addresses core framework functionality Component: IO Issue/PR addresses data loading/saving/versioning and validation, the DataCatalog and DataSets labels Nov 13, 2024
@ElenaKhaustova ElenaKhaustova added this to the Dataset Versioning milestone Nov 13, 2024
@ElenaKhaustova ElenaKhaustova changed the title Discrepancy when setting save_version via catalog constructor and when passing datasets Discrepancy when setting save_version via catalog constructor and when passing datasets Nov 13, 2024
@ElenaKhaustova ElenaKhaustova changed the title Discrepancy when setting save_version via catalog constructor and when passing datasets Discrepancy between setting save_version via catalog constructor and when passing datasets Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Framework Issue/PR that addresses core framework functionality Component: IO Issue/PR addresses data loading/saving/versioning and validation, the DataCatalog and DataSets
Projects
Status: No status
Development

No branches or pull requests

1 participant