Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add readme for indexer #12

Merged
merged 2 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 91 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,91 @@
# babylon-staking-indexer
# Babylon BTC Staking Indexer

## Overview

The **Babylon BTC Staking Indexer** is a core component of the
Babylon blockchain’s staking architecture, responsible for syncing delegation
and finality provider events from both the **Babylon blockchain (BBN)** and
**Bitcoin (BTC)**. It processes, stores, and transforms on-chain events into
API-friendly data structures, enabling the Babylon Staking API to efficiently
serve staking-related data to the frontend.

### Key Responsibilities
jrwbabylonlab marked this conversation as resolved.
Show resolved Hide resolved

- **Delegation Sync**: Syncs delegation and Babylon-related events, storing them in
MongoDB for easy retrieval.
- **Finality Provider Sync**: Tracks and updates the state of finality providers
(FPs), including status changes, creation, and edits.
- **Global Parameters Sync**: Syncs global parameters necessary for the staking
mechanism.
- **Event Replay(pending)**: Allows manual operations like event replays, triggered by
the Admin Portal or CLI, to recover or adjust state after chain forks or re-orgs.

## Architecture

The Babylon Indexer interacts with the following components:

- **BBN (Babylon Blockchain)**: Receives delegation and finality provider events
via Cosmos CometBFT `/block_results` (over gRPC for better performance), as well
as websocket subscription on events.
- **BTC (Bitcoin)**: Syncs withdrawal transactions and other BTC-related events.
- **MongoDB**: Serves as the storage layer where delegation, global parameters
and finality provider data is stored.
- **API Event Queue**: The indexer pushes API-related events into a queue
(RabbitMQ), consumed by the Babylon API for frontend-facing operations.
- **Admin Portal/CLI**: Provides interfaces for triggering event replays and
other manual interactions with the indexer.
- **Data Transformation Service (Optional)**: Transforms delegation data from
the indexer into other formats to backfill or migrate data for API as needed.

![Architecture Diagram](./doc/diagram.jpg)

## Synchronization Process

The workflow involves:

1. **Bootstrap Process**: The indexer starts by syncing all events from the
last processed Babylon block height to the latest height.
This is a continuous process until it catches up with the most recent block.
2. **Real-time Sync**: After catching up, the indexer subscribes to
real-time WebSocket events for ongoing synchronization.
3. **Raw Data Synchronization**: The indexer primarily handles the
synchronization of:
- **Delegation**: Storing and tracking delegation data.
- **Finality Provider**: Monitoring state changes and updates for
finality providers.
- **Global Parameters**: Syncing parameters relevant to staking, unbonding,
and slashing.

4. **RabbitMQ Messaging**: When a state change occurs in any delegation,
the indexer emits a message into RabbitMQ. This allows the Babylon API to
perform metadata and statistical calculations, such as total value locked (TVL)
computations.
5. **Bitcoin Node Sync**: The indexer also syncs with the Bitcoin node to
check if delegations are in a withdrawn state, ensuring accurate tracking of
withdrawal transactions.

## Installation & Setup

### Requirements

- **Go**: Version `1.23.2` or higher is required.
- **MongoDB**: A MongoDB instance with replica sets enabled is required

1. Clone the repository

```bash
git clone [email protected]:babylonlabs-io/babylon-staking-indexer.git
cd babylon-staking-indexer
```

2. Install dependencies

```bash
go mod tidy
```

3. Run the service

```bash
make run-local
```
10 changes: 5 additions & 5 deletions bin/local-startup.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash

# Check if the MongoDB container is already running
MONGO_CONTAINER_NAME="mongodb"
MONGO_CONTAINER_NAME="indexer-mongodb"
if [ $(docker ps -q -f name=^/${MONGO_CONTAINER_NAME}$) ]; then
echo "MongoDB container already running. Skipping MongoDB startup."
echo "Indexer mongoDB container already running. Skipping MongoDB startup."
else
echo "Starting MongoDB"
# Start MongoDB
docker compose up -d mongodb
echo "Starting indexer mongoDB"
# Start indexer mongoDB
docker compose up -d indexer-mongodb
fi

# Check if the RabbitMQ container is already running
Expand Down
2 changes: 1 addition & 1 deletion config/config-docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ poller:
db:
username: root
password: example
address: "mongodb://localhost:27017"
address: "mongodb://indexer-mongodb:27017/?directConnection=true"
db-name: babylon-staking-indexer
btc:
endpoint: localhost:18332
Expand Down
Binary file added doc/diagram.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ services:
- ./config/config-docker.yml:/home/babylon-staking-indexer/config.yml:Z
mongodb:
image: mongo:latest
container_name: mongodb
hostname: mongodb
container_name: indexer-mongodb
hostname: indexer-mongodb
ports:
- "27017:27017"
environment:
Expand Down
Loading