Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove topics and associated ACLs on deployment if topic removed. #58

Closed
big-andy-coates opened this issue Feb 2, 2023 · 0 comments
Closed
Assignees
Labels
enhancement New feature or request

Comments

@big-andy-coates
Copy link
Member

big-andy-coates commented Feb 2, 2023

When deploying later versions of a spec, it may be the case that a topic has been removed. Leaving this lying around would result in some of the issues SpecMesh is trying to avoid, e.g. topics that no one knows if they're still important. Hence we should remove them, and their associated ACLs.

The tricky bit here is that applications may create their own internal topics not in the spec: e.g. a streams app might create repartition and changelog topics. I'm not sure we want to have to define all of these in the spec. However, I think it's ok to limit such creations to _private topics.

Our clean up rules are then simple:

  1. remove any _public and _protected topics, and their ACLs, that aren't in the spec.
  2. leave any _private topics that are not in the spec.

This works well as its the public and protected topics that are hand to know if its OK to remove down the line. By forcing their removal we ensure dev/qa/staging environments have been updated and hence users should have confidence no one needs the removed topic. (Plus removing it from the spec should cause builds of downstream domains to fail .... though need to work out how this would work).

Leaving old private topics around isn't so much of an issue, as:

  1. no one else but the domain can access them,
  2. so engineers can easily determine which the app is still using and delete the old.
@big-andy-coates big-andy-coates added the enhancement New feature or request label Feb 2, 2023
@big-andy-coates big-andy-coates moved this to Todo in MVP OSS Feb 2, 2023
@bluemonk3y bluemonk3y pinned this issue Sep 30, 2023
bluemonk3y added a commit that referenced this issue Sep 30, 2023
note on 'Provision' command updated flags:
- use -dry or --dry-run to check action (inc cleaning)
- use -clean or --clean-unspecified to clean resources
bluemonk3y added a commit that referenced this issue Oct 2, 2023
bluemonk3y added a commit that referenced this issue Oct 2, 2023
* Part 2 of #58 - cleanup unspecified schemas

* spotless
bluemonk3y added a commit that referenced this issue Oct 2, 2023
* #58 Topics: remove topics that aren't in the spec.
note on 'Provision' command updated flags:
- use -dry or --dry-run to check action (inc cleaning)
- use -clean or --clean-unspecified to clean resources

* spotless

* Part 1 of #58 - fixing state mgmt

* enhancement - #58 schema cleanup of non-spec - part 2 (#210)

* Part 2 of #58 - cleanup unspecified schemas

* spotless

* fixed compilation
bluemonk3y added a commit that referenced this issue Oct 2, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in MVP OSS Oct 3, 2023
@bluemonk3y bluemonk3y unpinned this issue Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

No branches or pull requests

2 participants