diff --git a/README.md b/README.md index be4f69c..1b6b347 100644 --- a/README.md +++ b/README.md @@ -1 +1,91 @@ -# babylon-staking-indexer \ No newline at end of file +# Babylon BTC Staking Indexer + +## Overview + +The **Babylon BTC Staking Indexer** is a core component of the +Babylon blockchain’s staking architecture, responsible for syncing delegation +and finality provider events from both the **Babylon blockchain (BBN)** and +**Bitcoin (BTC)**. It processes, stores, and transforms on-chain events into +API-friendly data structures, enabling the Babylon Staking API to efficiently +serve staking-related data to the frontend. + +### Key Responsibilities + +- **Delegation Sync**: Syncs delegation and Babylon&BTC-related events, storing them in +MongoDB for easy retrieval. +- **Finality Provider Sync**: Tracks and updates the state of finality providers + (FPs), including status changes, creation, and edits. +- **Global Parameters Sync**: Syncs global parameters necessary for the staking +mechanism. +- **Event Replay(pending)**: Allows manual operations like event replays, triggered by +the Admin Portal or CLI, to recover or adjust state after chain forks or re-orgs. + +## Architecture + +The Babylon Indexer interacts with the following components: + +- **BBN (Babylon Blockchain)**: Receives delegation and finality provider events + via Cosmos CometBFT `/block_results` (over gRPC for better performance), as well + as websocket subscription on events. +- **BTC (Bitcoin)**: Syncs withdrawal transactions and other BTC-related events. +- **MongoDB**: Serves as the storage layer where delegation, global parameters +and finality provider data is stored. +- **API Event Queue**: The indexer pushes API-related events into a queue +(RabbitMQ), consumed by the Babylon API for frontend-facing operations. +- **Admin Portal/CLI**: Provides interfaces for triggering event replays and +other manual interactions with the indexer. +- **Data Transformation Service (Optional)**: Transforms delegation data from +the indexer into other formats to backfill or migrate data for API as needed. + +![Architecture Diagram](./doc/diagram.jpg) + +## Synchronization Process + +The workflow involves: + +1. **Bootstrap Process**: The indexer starts by syncing all events from the +last processed Babylon block height to the latest height. +This is a continuous process until it catches up with the most recent block. +2. **Real-time Sync**: After catching up, the indexer subscribes to +real-time WebSocket events for ongoing synchronization. +3. **Raw Data Synchronization**: The indexer primarily handles the +synchronization of: + - **Delegation**: Storing and tracking delegation data. + - **Finality Provider**: Monitoring state changes and updates for + finality providers. + - **Global Parameters**: Syncing parameters relevant to staking, unbonding, + and slashing. + +4. **RabbitMQ Messaging**: When a state change occurs in any delegation, +the indexer emits a message into RabbitMQ. This allows the Babylon API to +perform metadata and statistical calculations, such as total value locked (TVL) +computations. +5. **Bitcoin Node Sync**: The indexer also syncs with the Bitcoin node to +check if delegations are in a withdrawn state, ensuring accurate tracking of +withdrawal transactions. + +## Installation & Setup + +### Requirements + +- **Go**: Version `1.23.2` or higher is required. +- **MongoDB**: A MongoDB instance with replica sets enabled is required + +1. Clone the repository + +```bash +git clone git@github.com:babylonlabs-io/babylon-staking-indexer.git +cd babylon-staking-indexer +``` + +2. Install dependencies + +```bash +go mod tidy +``` + +3. Run the service + +```bash +make run-local +``` \ No newline at end of file diff --git a/bin/local-startup.sh b/bin/local-startup.sh index 5c80f49..6cf1d6d 100755 --- a/bin/local-startup.sh +++ b/bin/local-startup.sh @@ -1,13 +1,13 @@ #!/bin/bash # Check if the MongoDB container is already running -MONGO_CONTAINER_NAME="mongodb" +MONGO_CONTAINER_NAME="indexer-mongodb" if [ $(docker ps -q -f name=^/${MONGO_CONTAINER_NAME}$) ]; then - echo "MongoDB container already running. Skipping MongoDB startup." + echo "Indexer mongoDB container already running. Skipping MongoDB startup." else - echo "Starting MongoDB" - # Start MongoDB - docker compose up -d mongodb + echo "Starting indexer mongoDB" + # Start indexer mongoDB + docker compose up -d indexer-mongodb fi # Check if the RabbitMQ container is already running diff --git a/config/config-docker.yml b/config/config-docker.yml index 88e1c5c..a35860b 100644 --- a/config/config-docker.yml +++ b/config/config-docker.yml @@ -4,7 +4,7 @@ poller: db: username: root password: example - address: "mongodb://localhost:27017" + address: "mongodb://indexer-mongodb:27017/?directConnection=true" db-name: babylon-staking-indexer btc: endpoint: localhost:18332 diff --git a/doc/diagram.jpg b/doc/diagram.jpg new file mode 100644 index 0000000..34ae5e3 Binary files /dev/null and b/doc/diagram.jpg differ diff --git a/docker-compose.yml b/docker-compose.yml index fe16c76..b300dfd 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -12,8 +12,8 @@ services: - ./config/config-docker.yml:/home/babylon-staking-indexer/config.yml:Z mongodb: image: mongo:latest - container_name: mongodb - hostname: mongodb + container_name: indexer-mongodb + hostname: indexer-mongodb ports: - "27017:27017" environment: