SnowMint is a high-performance, distributed unique ID generator based on X's Snowflake algorithm. It provides unique, roughly sortable IDs that are generated using a client/server model with a custom protocol.
SnowMint leverages X’s Snowflake algorithm to generate signed, non-negative 64-bit unique identifiers called SnowMint IDs. The format of the ID consists of:
- Sign: A 1-bit field. It will always be 0.
- Timestamp: A 43-bit field representing the current time in milliseconds since your organization's epoch. This allows for generating 278 years 11 months 2.049 days worth of unique IDs from the epoch.
- DataCenter ID: A 5-bit field identifying the data center where the ID was generated.
- Node ID: A 5-bit field identifying the machine where the ID was generated.
- Sequence: A 10-bit field for a sequence number that resets every millisecond.
This combination ensures that SnowMint can (theoretically) generate more than 1,000,000 unique IDs per second, even in distributed environments.
The SnowMint ID 817347092935625747 is generated by a SnowMint server with the following (configurable) parameters:
- Epoch: 946684800000 (Saturday, January 01 2000 00:00:00.00 GMT+0000)
- DataCenter ID: 0
- Node ID: 19
The binary representation of this ID is presented below. Note that for this particular ID, the timestamp is 1726167730122 (Thursday, September 12, 2024 19:02:10.122 GMT+0000) and the sequence number is equal to 19.
0 0001011010101111100110011011001101111001010 00000 10011 0000010011
62 19 14 9 0
Since the first significant 43 bits represent the timestamp, SnowMint IDs are naturally sortable by creation time. IDs generated earlier will have a smaller numeric value than those generated later, allowing simple chronological ordering by comparing ID values directly. You can easily retrieve this timestamp by the following formula:
(SnowMintID >> 20) + EPOCH
In distributed systems, it is crucial that all nodes maintain synchronized clocks to ensure the uniqueness of the IDs. Since the SnowMint algorithm heavily relies on the system timestamp (43 bits of the ID represent the time), any drift in a node's clock can lead to the generation of duplicate IDs, which breaks the uniqueness guarantee.
To avoid this issue:
- Synchronize Time Across Nodes: Use tools like NTP (Network Time Protocol) or similar to keep system clocks in sync.
- Monitor Time Drift: Ensure that the time drift between nodes is kept to a minimum (e.g., within a few milliseconds).
- Fallback Mechanism: If a node detects that its clock is out of sync, it should halt ID generation until the clock is corrected to prevent collisions.
Failing to synchronize time across all nodes may result in non-unique IDs being generated, which could lead to issues in systems where uniqueness is critical.
SnowMint uses a lightweight, highly optimized custom protocol over raw TCP connections. This design focuses on speed and simplicity, ensuring ultra-fast ID generation and retrieval.
- Connection: Clients open a TCP connection to the SnowMint server.
- Command: The client sends a single
GETcommand to the server. - Response: The server responds immediately with a 64-bit unique ID.
This minimalist protocol reduces overhead, delivering unparalleled speed compared to traditional HTTP-based services.
- Raw TCP: Eliminates HTTP headers and other overhead, reducing the time between a request and response.
- Low-Latency: Designed for microsecond-scale latencies, making it ideal for high-throughput systems.
The SnowMint server accepts the following optional command line arguments:
--address The address for the server to bind to (default: 0.0.0.0)
--port The port for the server to bind to (default: 8080)
--datacenter The server's data center ID between 0 and 31 (default: 0)
--node The server's node ID between 0 and 31 (default: 0)
--epoch Your organization's epoch in milliseconds (default: 0 Thursday, January 01 1970 00:00:00.00 GMT+0000)
Note that in distributed systems, it is crucial that all instances of SnowMint servers are started with the same epoch.
Currently, SnowMint is available on Linux (amd64 and arm64), macOS (amd64 and arm64), and Windows.
- Download the latest release from the SnowMint releases page (or alternatively, you can clone this repository and build the binary yourself with
go build). - Extract the archive, choose the binary compatible with your operating system, and run it with your desired flags; e.g.:
./snowmint --node=<NODE_ID>
To run SnowMint in a Docker container, simply use the following with your desired flags; e.g.:
docker pull mxmlkzdh/snowmint:latest
docker run -d --name snowmint -p 8080:8080 mxmlkzdh/snowmint --node=<NODE_ID>Once an instance of the SnowMint server is up and running, execute the following command in your favorite terminal emulator and you'll receive a newly minted SnowMint ID!
echo GET | nc localhost 8080SnowMint provides easy-to-use SDKs for popular programming languages to integrate with your system and retrieve unique IDs.
| Language | Description |
|---|---|
| Go SDK | The Go client SDK allows seamless integration into Go applications. A simple GET request over TCP fetches the unique ID. |
| Java SDK | The Java SDK offers a similarly efficient way to connect to the SnowMint server, providing support for applications in JVM environments. |
SnowMint has been benchmarked to handle tens of thousands of requests per second, with latencies in the microsecond range. Thanks to the custom protocol and raw TCP connections, it outperforms traditional HTTP-based systems by a significant margin.
- ID Generation Rate: More than 100,000 IDs per second per node.
- Latency: Sub-millisecond, typically under 10 microseconds.
- In a native deployment, response time for each unique ID is roughly between 5 to 10 microseconds.
- In a Docker container, response time for each unique ID is roughly between 10 to 15 microseconds.
The SnowMint project is licensed under the MIT License.