Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to add/remove columns and recompute checksums #25

Open
RasmusSkytte opened this issue Sep 4, 2023 · 1 comment
Open

Add function to add/remove columns and recompute checksums #25

RasmusSkytte opened this issue Sep 4, 2023 · 1 comment
Labels
backlog This will be worked on in the future enhancement New feature or request

Comments

@RasmusSkytte
Copy link
Contributor

Presently, SCDB can handle data sets which do not add or remove new columns over time. In practice, the data the user wants to store will (for some data sets) get new columns as time goes by. It is also conceivable, that the user will want to remove certain columns from their datasets in some rare cases.
SCDB should have functionality to at least add (and conceivably also remove) columns from the dataset.
Adding the functionality to add columns should be relatively easy to implement since it "only" needs re-computation of the checksums after the addition of the column. If we add the option to remove columns, we should then also check that the resulting table is minimized after the operation. That is, if a records has only a change is in the column being removed, it will leave two consecutive, identical records that can be reduced to a single record with the combined validity from_ts/until_ts.

@RasmusSkytte RasmusSkytte added the enhancement New feature or request label Sep 4, 2023
@marcusmunch
Copy link
Collaborator

This is definitely tricky, and should be done very carefully. Additionally, it becomes very tricky if you want these columns to disappear/reappear when retrieving old data, as you would possibly require some sort of column metadata table to refer to.

If you instead simply ("simply") want to retrieve the values of current columns at a previous timestamp, I would be tempted to simply adding/removing the columns in the target table and consequently have all rows deactivated and (re)inserted, as one could argue that the previous values technically aren't the same anymore (a lack of information is still information).

Taking Postgres as a reference, adding/dropping columns require either table ownership, ALTER privileges on the table, or (of course) superuser status and is therefore arguably reserved for a small group of users.

Because of this, I would argue in favor of sending this to the backlog for now. It's definitely neat to have everything at the user's fingertips, but it might be a long time until this becomes relevant for SCDB - if ever.

@RasmusSkytte RasmusSkytte added the backlog This will be worked on in the future label Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog This will be worked on in the future enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants