Add in-toto format as hash of shard hashes #265
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 27, 2024
mihaimaruseac
force-pushed
the
in-toto-digests-of-shard-digests
branch
from
51463cd
to
2cc7279
Compare
mihaimaruseac
force-pushed
the
in-toto-digests-of-shard-digests
branch
from
2cc7279
to
305e97e
Compare
laurentsimon
previously approved these changes
Jul 31, 2024
| } | ||
| } | ||
| ], | ||
| "predicateType": "https://model_signing/DigestOfShardDigests/v0.1", |
There was a problem hiding this comment.
I'm curious if the non-sharded version can be represented as a special case of this one, with the shard=0.
Not sure it's worth it to try to optimize it. Thinking more about it, I like that it's separate to avoid implementation problems :)
There was a problem hiding this comment.
Hmm, we can try that in the future.
mihaimaruseac
force-pushed
the
in-toto-digests-of-shard-digests
branch
2 times, most recently
from
16feae4
to
7646d4f
Compare
This converts model serialization manifests that record every model file shard hash into an in-toto payload that can then be passed to Sigstore's `sign_intoto` for signing to generate a Sigstore `Bundle` (if using Sigstore). To identify the models, we compute a hash of all hashes of the file shards and use that as the subject. The individual file hashes are used as the payload and we would have the verifier check them as part of the verification process. Signed-off-by: Mihai Maruseac <[email protected]>
mihaimaruseac
force-pushed
the
in-toto-digests-of-shard-digests
branch
from
7646d4f
to
fa08711
Compare
spencerschrock
approved these changes
Aug 1, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Note: This is an experiment serialization, one of the 4 in a series of PRs (#264, #265, #266, #267). Before a stable release of the library, we would standardize on an ergonomic format, with as little corner cases / dangerous corners as possible.
This converts model serialization manifests that record every model file hash into an in-toto payload that can then be passed to Sigstore's
sign_intotofor signing to generate a SigstoreBundle(if using Sigstore).To identify the models, we compute a hash of all hashes of the files and use that as the subject. The individual file hashes are used as the payload and we would have the verifier check them as part of the verification process.
CC @susperius for converting manifest to in-toto. This should cover #111, #224, and #248 (first part of the machinery). CC @laurentsimon and (optionally) @TomHennen to make sure I did not mishandle in-toto.
Note: This is the equivalent of #264, but for file shard hashes instead of file hashes.
Note: This builds on #264. I decided to split every feature into its own PR to make it easier to review what changes (should be only the last commit) and to be able to merge partial work and continue from there.
Release Note
NONE
Documentation
NONE