Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: example for a snapshot node #89

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions snapshots-node/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
env
gnoland-data/*
27 changes: 27 additions & 0 deletions snapshots-node/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Snapshots node

## How it's works

1. Setup the environment variables and setup the variables for the [minio-client](https://min.io) with your S3 credentials

``` sh
cp env.example env
```

2. Start the node

By default, a snapshots will occur every 4 hours, and will upload it to S3

``` sh
docker compose up -d
```

## How to force a snapshot now

``` sh
docker compose exec snapshotter sh /scripts/snapshots.sh
```

## TO-DOs

[ ] Prune the node before uploading the snapshots
24 changes: 24 additions & 0 deletions snapshots-node/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: "gnoland-snapshot-node"

services:
gnoland:
image: "ghcr.io/gnolang/gno/gnoland:chain-test4.3"
entrypoint: ["/scripts/entrypoint.gnoland.sh"]
ports:
- 127.0.0.1:26657:26657
volumes:
- ./scripts:/scripts
- ./gnoland-data:/gnoroot/gnoland-data

snapshotter:
image: "docker:27.2.0-cli-alpine3.20"
entrypoint: ["sh", "/scripts/entrypoint.snapshotter.sh"]
env_file: "env"
environment:
CHAIN_ID: "test4"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./scripts:/scripts"
- "./docker-compose.yml:/docker-compose.yml"
- "./env:/env"
- "./gnoland-data:/gnoroot/gnoland-data:ro"
5 changes: 5 additions & 0 deletions snapshots-node/env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

MC_ENDPOINT=""
MC_ACCESS_KEY=""
MC_SECRET_KEY=""
MC_BUCKET=""
albttx marked this conversation as resolved.
Show resolved Hide resolved
16 changes: 16 additions & 0 deletions snapshots-node/scripts/entrypoint.gnoland.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env sh

LOG_LEVEL=${LOG_LEVEL:-"info"}

MONIKER=${MONIKER:-"gnode"}
P2P_LADDR=${P2P_LADDR:-"tcp://0.0.0.0:26656"}
RPC_LADDR=${RPC_LADDR:-"tcp://0.0.0.0:26657"}

CHAIN_ID=${CHAIN_ID:-"dev"}
albttx marked this conversation as resolved.
Show resolved Hide resolved

gnoland config init

gnoland config set rpc.laddr "${RPC_LADDR}"
gnoland config set p2p.laddr "${P2P_LADDR}"

exec gnoland start --lazy --genesis="./gnoland-data/genesis.json" --log-level=${LOG_LEVEL}
17 changes: 17 additions & 0 deletions snapshots-node/scripts/entrypoint.snapshotter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env sh

apk add --no-cache curl jq lz4 minio-client

export DAEMON_HOME="/gnoroot/gnoland-data"

[ -z "$MC_ENDPOINT" ] && echo "MC_ENDPOINT is not set, skip snapshot" && exit 1
[ -z "$MC_ACCESS_KEY" ] && echo "MC_ACCESS_KEY is not set, skip snapshot" && exit 1
[ -z "$MC_SECRET_KEY" ] && echo "MC_SECRET_KEY is not set, skip snapshot" && exit 1

mcli alias set s3 ${MC_ENDPOINT} ${MC_ACCESS_KEY} ${MC_SECRET_KEY}

# exec snapshots script every 4 hours
echo "0 */4 * * * sh /scripts/snapshots.sh" > /etc/crontabs/root
crontab /etc/crontabs/root

exec crond -f
20 changes: 20 additions & 0 deletions snapshots-node/scripts/snapshots.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env sh

DAEMON_HOME="/gnoroot/gnoland-data"

HEIGHT=$(curl -s gnoland:26657/status | jq -r '.result.sync_info.latest_block_height')
echo "[INFO] Snapshotting at height: ${HEIGHT}"

echo "[INFO] Stopping gnoland..."
docker compose stop gnoland

FILE="gnoland-${HEIGHT}-${CHAIN_ID}.tar.lz4"

tar cvf - -C ${DAEMON_HOME} wal db | lz4 > $FILE

mcli cp --attr x-amz-acl=public-read --tags="type=snapshot" ${FILE} "s3/${MC_BUCKET}/${CHAIN_ID}/${FILE}"; echo

rm -vf ${FILE}

echo "[INFO] Starting gnoland ..."
docker compose start gnoland