-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for signing RPM Packages on upload (previously: Native obs-signd support for signing PULP hosted content) #2986
Comments
We should probably try to make this functionality generic enough that it could potentially be used with https://github.com/TommyLike/signatrust and not specifically obs-signd |
Thanks, the project has been moved here: https://gitee.com/openeuler/signatrust |
@praiskup With a quick code search and document reading, I think pulp rpm only signs the repo metadata itself, within this background it would be quite easy to support obs-sign or signatrust in pulp rpm, we can use custom |
Correct, that's the current state of things |
So, a quick summary of some of the more tricky requirements / problems here:
|
Let's moving forward |
Hello @praiskup, I started working on this and I need some more info. I plan to work on a more general rpm signing solution based on Edit: does that approach seem reasonable to you? |
Hello folks, thank you for the update here.
I'm afraid there will be a need for after-upload (re)signing, at least We also have a script for re-signing
/me nods. If we had to keep signing done on Copr (early stages), download
Detached RPM signatures would be nice RFE.. But I agree that
Well, on-upload-only seems like a good start, considering that re-upload
Seems fine. Note that maintaining the keyring is not a trivial thing. |
I've discussed that feature with the RPM team today and for now, I'll proceed with the on-upload-only support, as there is much to discuss about properly handling the signing of existing content. Here's a sample workflow that the implementation will enable:
|
Sounds good, just ad 1., note we have ~122000 keys ATM, so the DB needs to be fairly optimal. Is there some queue of pending tasks of this kind? So the particular load of tasks is going to be handled by some background workers in a precise order so the rest of the service (API) is not DOSed? |
We will take care of setting this up so it's efficient at access, thanks for pointing that out.
The signing-service registration is not called via the API, it's done via django commands (see here), so it it's not handled by workers but it shouldn't interfere with the API capacity. |
The signing service creation in its current state can never be made publicly accessible as it involves handing Pulp a shell script to execute. That's why it's only available as an admin command and isn't part of the API - allowing users to pass arbitrary shell scripts would be a horrible idea. Key management might be a concern, I agree. |
Regarding the process of calling the SigningService on uploaded packages, my earlier implementation did that on the API level. There are some pros and cons of doing it. I'll summarize the discussion we had with pulp team (thanks Daniel, Ina, Grant and Matthias). Early conclusionsWe should do signing on workers, if we have to choose one. Even though it's not ideal for Storage Usage to have the extra round trips to the backend storage, API Service availability and Security best practices criteria are better satisfied using workers. Trade-offsThe tradeoffs are complex and may vary on different scenarios. Here are the key takes I got from our discussions:
[1] - Might not be too bad for both big/small rpms, assuming >V3 signing specs is used, which does not require full payload reading. AlternativeI though about supporting both, as one or the other could be best for different scenarios (package size, deployment setup, etc), but it might not be a good idea to expose such low-level implementation details and have to support them both. Reference
|
Thank you for the analysis! I'm not sure what would be the penalty for [1]... while calling
|
Thanks for the cost benefit analyses. I lean towards signing in workers even through the storage overhead. A good rule of thumb is to not withhold api with anything that is few seconds longer and offload that to the tasks. |
Contact with the API workers is impacted by things like idle-connection-timeouts as well - if the call takes "too long", nginx/apache can decide to drop the connection. This leans me also twoards "do it in a task worker" instead of "inside" the API call directly. |
Great writeup @pedro-psb, everything you've written sounds accurate One additional note is that if we're talking about S3, additional traffic isn't just an efficiency concern but does come with a monetary cost. |
To me, using signing services only in workers is a must. And this is even more important given the fact that Pulp will never get a grip on any private key material. It must be able to operate using hardware crypto devices. At that point, i think we can safely assume that only the worker processes have access to such a device. We may even need to delegate singing tasks to accordingly equipped worker nodes. |
I agree, architecturally speaking I can't really support doing signing outside of a worker. Side note: we should do some at-scale upload performance testing at the rate of 100-or-so packages a second, roundabouts the maximum we would expect COPR to experience. I don't think we did any testing with upload during the first round of scale testing for COPR. |
Moving to another topic: Recently a draft PR was opened by @mdellweg to offer basic keyring management capabilities to Pulp. This should provide some well-crafted models for storing Pubkeys, Keyrings, and Signatures, among others. I prefer waiting to use it because it would help to overcome some of the limitations (of the current feature draft) that I've listed below.
SigningService validation overhead
Signature Management
Key Registration
|
Related discussion about the "Temporary File Roundtrip" problem, thats it, the problem of the extra traffic required for sending uploaded files to workers: pulp/pulpcore#3827 |
Hi, We would like to resign package from remote as well as after upload. We may also want to resign because we updated the signing key. I suggest an /addsign API that accepts an RPM unit href and a signing_service as input. It returns a task id. The tasks return a new unit_href. Upload could rely transparently on this task if a signing_service is given along the file. What do you think of this ? Regards |
Hey, thanks for sharing the idea. It mostly matches with my plans and with our discussions about this. Also, there are some decision to be made about how we'll handle the old unsigned rpm. Just to give a brief context of the problem here: content is immutable, so when we sign something inside Pulp, we are really creating another signed rpm in storage. Also, because of this immutability, we can't directly remove the old unsigned one. So we'll either unlink the unsigned object automatically (so it can be orphan cleaned-up) or we'll let that to the user. |
I agree. I'm aware that signing creates a new artifacts. I intend to upload an rpm or get it from a remote, sign it and then add the signed artifact to the target repository. |
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs Closes pulp#2986
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs Closes pulp#2986
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs Closes pulp#2986
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs Closes pulp#2986
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs Closes pulp#2986
What this does: - Create RpmPackageSigningService - Create RpmTool utility - Add fields to Repository model - Add branch on Package upload to sign the package with the associated SigningService and fingerprint. - Add docs - Use presence of SigningService at Repo as signing trigger Closes pulp#2986
I've renamed that title and I'll create other two issue for requirements that were not met by #3425 , namely, general package signing support and an optimized workflow for on-upload signing. |
It would be nice to have a native support for obs-signd in PULP. The obs-sign
client/server signing method is useful specifically because it is very networking economical.
Calculating the crypto checksums is separated from the signing logic. While the
checksum (I/O intensive task for large RPMs) is done client-side the signatures are
calculated server-side.
It would be nice if this was well integrated into PULP signature mechanisms,
i.e. when RPM is uploaded to PULP (but not yet e.g. to s3 storage) - the RPM was
in-place signed via obs-signd.
HINT: Not entirely sure if strictly related to pulp_rpm, or rather pulpcore.
The text was updated successfully, but these errors were encountered: