Proposal for a single rucio chart with complete reproducible deployment of rucio

[volodymyrss]

The principle agreed with @bari12 on Rucio-Dirac workshop, but not the technical details of the chart implementation.

In CTAO we made a chart of our Bulk Data Management System (BDMS) including as subcharts rucio charts, optional dependencies charts, and adding jobs for bootstrap and configure. See a talk.

This would require several changes ported to upstream rucio chart:

• put existing rucio-server, rucio-demons, etc charts to be subcharts of rucio chart.
• add bootstrap and configure jobs
• add certificate generation/issuer/manager subchart with options:
  • self-signed CA
  • admin-provider certificates
  • letsencrypt (where possible)
  • IAM x509 certificates (RCAuth-based)
• appropriate documentation

Please let me know if this set of changes makes sense!
The PR is started as a draft, just starting to move.

[volodymyrss]

@bari12, I currently can not make a PR to upstream as we discussed, due to:

[bari12]

Sorry for the late answer @volodymyrss
Can you try again? I am not really sure why this doesn't work. The permissions are certainly set in a way that anyone can open PRs.

[volodymyrss]

Sorry for the late answer @volodymyrss Can you try again? I am not really sure why this doesn't work. The permissions are certainly set in a way that anyone can open PRs.

Thanks for getting back!

No, still not allowed as non-collaborator.

[bari12]

Honestly, I have no idea what is going on here. The repo is setup to allow "non-collaborators" to submit PRs. Many have done in the past, and none of the settings changed.
I find a few other discussions referencing the same issue
https://github.com/orgs/community/discussions/61472
But there is no real fix for it. I'll open a ticket with github.

[volodymyrss]

@maxnoe , is it possible it's because I am lacking some permission in cta-observatory?

[maxnoe]

Yes, I think the error is that you lack permission here so you are not allowed to open PRs using this repo as source.

Will add you.

[maxnoe]

Note that we originally set this repository up as a one-way-mirror from the ctao gitlab to GitHub.

So it might not be the best choice to contribute directly from GitHub to it...

[bari12]

Ok, this is more or less also the response I got from github support, that, since this is not your private fork, it might be related to permissions in the source repository, rather than the destination.

[volodymyrss]

Ok, this is more or less also the response I got from github support, that, since this is not your private fork, it might be related to permissions in the source repository, rather than the destination.

Thanks for following up and sorry for the trouble from our side! The error message was no indeed not too clear to me.

[volodymyrss]

Note that we originally set this repository up as a one-way-mirror from the ctao gitlab to GitHub.

So it might not be the best choice to contribute directly from GitHub to it...

Is it a problem that the one-way mirror will have some branches of its own?
It's not a simple copy, it's a push, so it may keep existing branches with no issues.

We could of course push to gitlab and let it sync, but for a case when we will target contribution upstream, it might be peculiar, unsure.

Like, it will have github actions, but will be stored in gitlab. And it might be easier to use github CLI.

[maxnoe]

I am fine with experimenting here what works best. So if you directly want to contribute to upstream from here with branches that don't exist in the gitlab, that's fine I think.

The idea was that we might have changes to the chart already in production, which would live in the gitlab, and then try to move them upstream.

The only question would be: why not use a personal fork then for your direct contributions?

[volodymyrss]

I am fine with experimenting here what works best. So if you directly want to contribute to upstream from here with branches that don't exist in the gitlab, that's fine I think.

Ok, let's do that then.

The idea was that we might have changes to the chart already in production, which would live in the gitlab, and then try to move them upstream.

Yes. But right now we really aim to contribute commonly useful chart to upstream, without customization for us.

The only question would be: why not use a personal fork then for your direct contributions?

This was my first idea, I think we mentioned it briefly on the workshop, but just that this is potentially a common contribution from CTAO, other people may take part? I am totally fine to do it from my fork though if you prefer?

[volodymyrss]

@maxnoe I just realized that this branch does in fact come from https://gitlab.cta-observatory.org/cta-computing/dpps/bdms/rucio-helm-charts/-/tree/reproducible-deployment?ref_type=heads sync.

Anyway, I close this internal PR and move ahead to rucio#217 .