checksum #17
checksum #17
Conversation
| fn verify_checksum(file: &File, expected_checksum: &str) -> anyhow::Result<bool> { | ||
| let mut reader = std::io::BufReader::new(file); | ||
| let mut hasher = Sha256::new(); | ||
| let mut buffer = Vec::new(); | ||
| reader | ||
| .read_to_end(&mut buffer) | ||
| .context("Failed to read file for checksum verification")?; | ||
| hasher.update(&buffer); | ||
| let actual_checksum = format!("{:x}", hasher.finalize()); | ||
| if actual_checksum != expected_checksum { | ||
| info!( | ||
| "Checksum mismatch: expected {}, but got {}", | ||
| expected_checksum, actual_checksum | ||
| ); | ||
| } | ||
| Ok(actual_checksum == expected_checksum) |
There was a problem hiding this comment.
Mhhh can we maybe use standard tools for this ? it's gonna be easier to port accross platforms and easier to write scripts as well in bash. Like sha256sum or shasum i think. Also probably might be a bit faster.
There was a problem hiding this comment.
But in this case, we would have to download + save before checking checksum.
Which is fine for me btw.
Another solution would be to use blake3 which supports reading from mmap and even do stuff in parallel with rayon and blake3 is known to be much faster, even more for large files.
| pub fn create_prover( | ||
| url: &str, | ||
| dir: &str, | ||
| circuit_file: &str, | ||
| circuit_file_checksum: &str, |
There was a problem hiding this comment.
Can't we simply do the checksum over the whole directory maybe ? Simpler ? It's best to be atomic on this stuff, we don't want to have different versions of the sub-parameters living the same place.
| @@ -11,6 +11,7 @@ pub fn create_prover( | |||
| url: &str, | |||
| dir: &str, | |||
| file: &str, | |||
| checksum: &str, | |||
There was a problem hiding this comment.
same, can't we do it simply over the whole parameter directory and be done with it ? The individual provers shouldn't have to know about this, they should expect to already get the right version.
https://github.com/Lagrange-Labs/zkmr-tracking/issues/109