add default serialization format (#511)

* add default serialization format

* remove SHORT_ID from Ciphersuite trait

book/src/SUMMARY.md

@@ -8,6 +8,7 @@
   - [Signing](tutorial/signing.md)
   - [Distributed Key Generation](tutorial/dkg.md)
 - [User Documentation](user.md)
+  - [Serialization Format](user/serialization.md)
 - [FROST with Zcash](zcash.md)
   - [Technical Details](zcash/technical-details.md)
   - [Ywallet Demo](zcash/ywallet-demo.md)

book/src/tutorial/importing.md

@@ -20,9 +20,21 @@ generation or signing procedure.
 
 FROST is a distributed protocol and thus it requires sending messages between
 participants. While the ZF FROST library does not handle communication, it can
-help with serialization by activating the `serde` feature. When it is enabled,
-you can use [serde](https://serde.rs/) to serialize any structure that needs
-to be transmitted. Import example:
+help with serialization in the following ways:
+
+### Default byte-oriented serialization
+
+With the `serialization` feature, which is enabled by default, all structs that
+need to communicated will have `serialize()` and `deserialize()` methods. The
+serialization format is described in [Serialization
+Format](../user/serialization.md).
+
+### serde
+
+Alternatively, if you would like to user another format such as JSON, you can
+enable the `serde` feature (which is *not* enabled by default). When it is
+enabled, you can use [serde](https://serde.rs/) to serialize any structure that
+needs to be transmitted. The importing would look like:
 
 ```
 [dependencies]
@@ -33,5 +45,5 @@ Note that serde usage is optional. Applications can use different encodings, and
 to support that, all structures that need to be transmitted have public getters
 and `new()`  methods allowing the application to encode and decode them as it
 wishes. (Note that fields like `Scalar` and `Element` do have standard byte
-string encodings; the application can encode those byte strings as it wishes,
-as well the structure themselves and things like maps and lists.)
+string encodings; the application can encode those byte strings as it wishes, as
+well the structure themselves and things like maps and lists.)

book/src/user/serialization.md

@@ -0,0 +1,60 @@
+# Serialization Format
+
+With the `serialization` feature, which is enabled by default, all structs that
+need to communicated will have `serialize()` and `deserialize()` methods.
+
+The format is basically the `serde` encoding of the structs using the
+[`postcard`](https://docs.rs/postcard/latest/postcard/) crate.
+
+- Integers are encoded in [varint
+  format](https://postcard.jamesmunns.com/wire-format#varint-encoded-integers)
+- Fixed-size byte arrays are encoded as-is (e.g. scalars, elements)
+  - Note that the encoding of scalars and elements are defined by the
+    ciphersuites.
+- Variable-size byte arrays are encoded with a length prefix (varint-encoded)
+  and the array as-is (e.g. the message)
+- Maps are encoded as the varint-encoded item count, followed by concatenated
+  item encodings.
+- Ciphersuite IDs are encoded as the 4-byte CRC-32 of the ID string.
+- Structs are encoded as the concatenation of the encodings of its items.
+
+For example, the following Signing Package:
+
+- Commitments (map):
+    - Identifier (byte array): `2a00000000000000000000000000000000000000000000000000000000000000`
+    - Signing Commitments:
+      - Hiding (byte array): `e2f2ae0a6abc4e71a884a961c500515f58e30b6aa582dd8db6a65945e08d2d76`
+      - Bindng (byte array): `6a493210f7499cd17fecb510ae0cea23a110e8d5b901f8acadd3095c73a3b919`
+      - Ciphersuite ID: `"FROST(ristretto255, SHA-512)"`
+- Message (variable size byte array): `68656c6c6f20776f726c64` (`"hello world"` in UTF-8)
+- Ciphersuite ID (4 bytes): `"FROST(ristretto255, SHA-512)"`
+
+Is encoded as
+
+```
+012a000000000000000000000000000000000000000000000000000000000000
+00e2f2ae0a6abc4e71a884a961c500515f58e30b6aa582dd8db6a65945e08d2d
+766a493210f7499cd17fecb510ae0cea23a110e8d5b901f8acadd3095c73a3b9
+19e6811b690b68656c6c6f20776f726c64e6811b69
+```
+
+- `01`: the length of the map
+- `2a00000000000000000000000000000000000000000000000000000000000000`: the identifier
+- `e2f2ae0a6abc4e71a884a961c500515f58e30b6aa582dd8db6a65945e08d2d76`: the hinding commitment
+- `6a493210f7499cd17fecb510ae0cea23a110e8d5b901f8acadd3095c73a3b919`: the binding commitment
+- `e6811b69`: the ciphersuite ID of the SigningCommitments, CRC-32 of "FROST(ristretto255, SHA-512)"
+- `0b`: the length of the message
+- `68656c6c6f20776f726c64`: the message
+- `e6811b69`: the ciphersuite ID of the SigningPackage, CRC-32 of "FROST(ristretto255, SHA-512)"
+
+```admonish note
+The ciphersuite ID is encoded multiple times in this case because `SigningPackage` includes
+`SigningCommitments`, which also need to be communicated in Round 1 and thus also encodes
+its ciphersuite ID. This is the only instance where this happens.
+```
+
+## Test Vectors
+
+Check the
+[`snapshots`](https://github.com/search?q=repo%3AZcashFoundation%2Ffrost+path%3Asnapshots&type=code)
+files in each ciphersuite crate for test vectors.

frost-core/Cargo.toml

@@ -23,10 +23,12 @@ rustdoc-args = ["--cfg", "docsrs"]
 
 [dependencies]
 byteorder = "1.4"
+const-crc32 = "1.2.0"
 document-features = "0.2.7"
 debugless-unwrap = "0.0.4"
 derive-getters = "0.3.0"
 hex = "0.4.3"
+postcard = { version = "1.0.0", features = ["use-std"], optional = true }
 rand_core = "0.6"
 serde = { version = "1.0.160", features = ["derive"], optional = true }
 serdect = { version = "0.2.0", optional = true }
@@ -48,7 +50,7 @@ rand_chacha = "0.3"
 serde_json = "1.0"
 
 [features]
-default = []
+default = ["serialization"]
 #! ## Features
 ## Expose internal types, which do not have SemVer guarantees. This is an advanced
 ## feature which can be useful if you need to build a modified version of FROST.
@@ -58,6 +60,7 @@ internals = []
 ## can use `serde` to serialize structs with any encoder that supports
 ## `serde` (e.g. JSON with `serde_json`).
 serde = ["dep:serde", "dep:serdect"]
+serialization = ["serde", "dep:postcard"]
 # Exposes ciphersuite-generic tests for other crates to use
 test-impl = ["proptest", "serde_json", "criterion"]
 

frost-core/src/error.rs

@@ -101,6 +101,12 @@ pub enum Error<C: Ciphersuite> {
     /// The ciphersuite does not support deriving identifiers from strings.
     #[error("The ciphersuite does not support deriving identifiers from strings.")]
     IdentifierDerivationNotSupported,
+    /// Error serializing value.
+    #[error("Error serializing value.")]
+    SerializationError,
+    /// Error deserializing value.
+    #[error("Error deserializing value.")]
+    DeserializationError,
 }
 
 impl<C> Error<C>
@@ -147,6 +153,8 @@ where
             | Error::UnknownIdentifier
             | Error::IncorrectNumberOfIdentifiers
             | Error::IncorrectNumberOfCommitments
+            | Error::SerializationError
+            | Error::DeserializationError
             | Error::IdentifierDerivationNotSupported => None,
         }
     }

frost-core/src/frost.rs

@@ -23,8 +23,8 @@ pub mod round1;
 pub mod round2;
 
 use crate::{
-    scalar_mul::VartimeMultiscalarMul, Ciphersuite, Element, Error, Field, Group, Scalar,
-    Signature, VerifyingKey,
+    scalar_mul::VartimeMultiscalarMul, Ciphersuite, Deserialize, Element, Error, Field, Group,
+    Scalar, Serialize, Signature, VerifyingKey,
 };
 
 pub use self::identifier::Identifier;
@@ -306,6 +306,22 @@ where
     }
 }
 
+#[cfg(feature = "serialization")]
+impl<C> SigningPackage<C>
+where
+    C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+{
+    /// Serialize the struct into a Vec.
+    pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+        Serialize::serialize(&self)
+    }
+
+    /// Deserialize the struct from a slice of bytes.
+    pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+        Deserialize::deserialize(bytes)
+    }
+}
+
 /// The product of all signers' individual commitments, published as part of the
 /// final signature.
 #[derive(Clone, PartialEq, Eq)]

frost-core/src/frost/keys.rs

@@ -17,7 +17,8 @@ use rand_core::{CryptoRng, RngCore};
 use zeroize::{DefaultIsZeroes, Zeroize};
 
 use crate::{
-    frost::Identifier, Ciphersuite, Element, Error, Field, Group, Scalar, SigningKey, VerifyingKey,
+    frost::Identifier, Ciphersuite, Deserialize, Element, Error, Field, Group, Scalar, Serialize,
+    SigningKey, VerifyingKey,
 };
 
 #[cfg(feature = "serde")]
@@ -425,6 +426,22 @@ where
     }
 }
 
+#[cfg(feature = "serialization")]
+impl<C> SecretShare<C>
+where
+    C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+{
+    /// Serialize the struct into a Vec.
+    pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+        Serialize::serialize(&self)
+    }
+
+    /// Deserialize the struct from a slice of bytes.
+    pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+        Deserialize::deserialize(bytes)
+    }
+}
+
 /// The identifier list to use when generating key shares.
 pub enum IdentifierList<'a, C: Ciphersuite> {
     /// Use the default values (1 to max_signers, inclusive).
@@ -613,6 +630,22 @@ where
     }
 }
 
+#[cfg(feature = "serialization")]
+impl<C> KeyPackage<C>
+where
+    C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+{
+    /// Serialize the struct into a Vec.
+    pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+        Serialize::serialize(&self)
+    }
+
+    /// Deserialize the struct from a slice of bytes.
+    pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+        Deserialize::deserialize(bytes)
+    }
+}
+
 impl<C> TryFrom<SecretShare<C>> for KeyPackage<C>
 where
     C: Ciphersuite,
@@ -684,6 +717,22 @@ where
     }
 }
 
+#[cfg(feature = "serialization")]
+impl<C> PublicKeyPackage<C>
+where
+    C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+{
+    /// Serialize the struct into a Vec.
+    pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+        Serialize::serialize(&self)
+    }
+
+    /// Deserialize the struct from a slice of bytes.
+    pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+        Deserialize::deserialize(bytes)
+    }
+}
+
 fn validate_num_of_signers<C: Ciphersuite>(
     min_signers: u16,
     max_signers: u16,

frost-core/src/frost/keys/dkg.rs

@@ -50,6 +50,8 @@ pub mod round1 {
     use derive_getters::Getters;
     use zeroize::Zeroize;
 
+    use crate::{Deserialize, Serialize};
+
     use super::*;
 
     /// The package that must be broadcast by each participant to all other participants
@@ -92,6 +94,22 @@ pub mod round1 {
         }
     }
 
+    #[cfg(feature = "serialization")]
+    impl<C> Package<C>
+    where
+        C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+    {
+        /// Serialize the struct into a Vec.
+        pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+            Serialize::serialize(&self)
+        }
+
+        /// Deserialize the struct from a slice of bytes.
+        pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+            Deserialize::deserialize(bytes)
+        }
+    }
+
     /// The secret package that must be kept in memory by the participant
     /// between the first and second parts of the DKG protocol (round 1).
     ///
@@ -145,6 +163,8 @@ pub mod round2 {
     use derive_getters::Getters;
     use zeroize::Zeroize;
 
+    use crate::{Deserialize, Serialize};
+
     use super::*;
 
     /// A package that must be sent by each participant to some other participants
@@ -186,6 +206,22 @@ pub mod round2 {
         }
     }
 
+    #[cfg(feature = "serialization")]
+    impl<C> Package<C>
+    where
+        C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+    {
+        /// Serialize the struct into a Vec.
+        pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+            Serialize::serialize(&self)
+        }
+
+        /// Deserialize the struct from a slice of bytes.
+        pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+            Deserialize::deserialize(bytes)
+        }
+    }
+
     /// The secret package that must be kept in memory by the participant
     /// between the second and third parts of the DKG protocol (round 2).
     ///

frost-core/src/frost/round1.rs

@@ -12,7 +12,7 @@ use hex::FromHex;
 use rand_core::{CryptoRng, RngCore};
 use zeroize::Zeroize;
 
-use crate::{frost, Ciphersuite, Element, Error, Field, Group, Scalar};
+use crate::{frost, Ciphersuite, Deserialize, Element, Error, Field, Group, Scalar, Serialize};
 
 #[cfg(feature = "serde")]
 use crate::ElementSerialization;
@@ -311,6 +311,22 @@ where
     }
 }
 
+#[cfg(feature = "serialization")]
+impl<C> SigningCommitments<C>
+where
+    C: Ciphersuite + serde::Serialize + for<'de> serde::Deserialize<'de>,
+{
+    /// Serialize the struct into a Vec.
+    pub fn serialize(&self) -> Result<Vec<u8>, Error<C>> {
+        Serialize::serialize(&self)
+    }
+
+    /// Deserialize the struct from a slice of bytes.
+    pub fn deserialize(bytes: &[u8]) -> Result<Self, Error<C>> {
+        Deserialize::deserialize(bytes)
+    }
+}
+
 impl<C> From<&SigningNonces<C>> for SigningCommitments<C>
 where
     C: Ciphersuite,