Skip to content

Commit

Permalink
[Docs] Minor documentation updates during walkthrough (#46)
Browse files Browse the repository at this point in the history
Updated some documentation while doing an end-to-end walkthrough.
  • Loading branch information
Olshansk authored Aug 15, 2024
1 parent 5905db3 commit 7587dad
Show file tree
Hide file tree
Showing 18 changed files with 710 additions and 245 deletions.
20 changes: 14 additions & 6 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
# Pocket RPC Configuration
POKT_RPC_FULL_HOST=
HTTP_SERVER_PORT=8080
POKT_RPC_TIMEOUT=5s

# Gateway Deployment Configuration
HTTP_SERVER_PORT=8080
ENVIRONMENT_STAGE=development
POKT_APPLICATIONS_ENCRYPTION_KEY=
EMIT_SERVICE_URL_PROM_METRICS=false

# Pocket Business Logic
CHAIN_NETWORK=morse_mainnet
SESSION_CACHE_TTL=75m
DB_CONNECTION_URL=postgres://myuser:mypassword@postgres:5432/postgres?sslmode=disable
API_KEY=
ALTRUIST_REQUEST_TIMEOUT=10s
EMIT_SERVICE_URL_PROM_METRICS=false
CHAIN_NETWORK=morse_mainnet
API_KEY=

# App Stake Management
DB_CONNECTION_URL=postgres://myuser:mypassword@postgres:5433/postgres?sslmode=disable
POKT_APPLICATIONS_ENCRYPTION_KEY=
POKT_APPLICATION_PRIVATE_KEY=
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,10 @@
docker-compose.yml

## MAC
.DS_Store
.DS_Store

## VSCode
.vscode

## Copmiled binary
main
40 changes: 40 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
include .env

########################
### Makefile Helpers ###
########################

.PHONY: prompt_user
# Internal helper target - prompt the user before continuing
prompt_user:
@echo "Are you sure? [y/N] " && read ans && [ $${ans:-N} = y ]

.PHONY: list
list: ## List all make targets
@${MAKE} -pRrn : -f $(MAKEFILE_LIST) 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | egrep -v -e '^[^[:alnum:]]' -e '^$@$$' | sort

.PHONY: help
.DEFAULT_GOAL := help
help: ## Prints all the targets in all the Makefiles
@grep -h -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-60s\033[0m %s\n", $$1, $$2}'

########################
### Database Helpers ###
########################

.PHONY: db_migrate
db_migrate: ## Run database migrations
@echo "Running database migrations..."
./scripts/db_migrate.sh -u

.PHONY: db_migrate
db_migrate: ## Run database migrations
@echo "Running database migrations..."
./scripts/db_migrate.sh -u\


PG_CMD := INSERT INTO pokt_applications (encrypted_private_key) VALUES (pgp_sym_encrypt('$(POKT_APPLICATION_PRIVATE_KEY)', '$(POKT_APPLICATIONS_ENCRYPTION_KEY)'));
db_insert_app_private_key: ## Insert application private key into database
@echo "Running SQL command..."
@echo "$(PG_CMD)"
@psql "$(DB_CONNECTION_URL)" -c "$(PG_CMD)"
144 changes: 105 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,114 +2,180 @@
<img src="https://github.com/pokt-network/gateway-server/blob/main/docs/resources/gateway-server-logo.jpg" width="500" alt="POKT Gateway Server">
</div>

## What is POKT Gateway Server?
# What is POKT Gateway Server? <!-- omit in toc -->

The POKT Gateway Server is a comprehensive solution designed to simplify the integration of applications with the POKT Network. Our goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators.
_tl;dr Streamline access to POKT Network's decentralized supply network._

Learn more about the vision and overall architecture [overview](docs%2Foverview.md)
The POKT Gateway Server is a comprehensive solution designed to simplify the integration of applications with POKT Network. Its goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators.

## Gateway Operator Quick Getting Started
To onboard the gateway server without having to dig deep, you can follow the [Quick Onboarding Guide](docs%2Fquick-onboarding-guide.md)
Learn more about the vision and overall architecture [overview](./docs/overview.md).

#### Interested in learning more? We have an abdundance of information in [docs](docs)
1. [Gateway Server Overview](docs%2Foverview.md)
2. [Gateway Server API Endpoints](docs%2Fapi-endpoints.md)
3. [Gateway Server System Architecture](docs%2Fsystem-architecture.md)
4. [Gateway Server Node Selection](docs%2Fnode-selection.md)
5. [POKT Primer](docs%2Fpokt-primer.md)
6. [POKT's Relay Specification](docs%2Fpokt-relay-specification.md)
- [Gateway Operator Quickstart Guide](#gateway-operator-quickstart-guide)
- [Interested in learning more?](#interested-in-learning-more)
- [Docker Image Releases](#docker-image-releases)
- [Docker Compose](#docker-compose)
- [Minimum Hardware Requirements](#minimum-hardware-requirements)
- [Database Migrations](#database-migrations)
- [Creating a DB Migration](#creating-a-db-migration)
- [Applying a DB Migration](#applying-a-db-migration)
- [DB Migration helpers](#db-migration-helpers)
- [Applying Migrations](#applying-migrations)
- [Migrations Rollbacks](#migrations-rollbacks)
- [Unit Testing](#unit-testing)
- [Generating Mocks](#generating-mocks)
- [Running Tests](#running-tests)
- [Generating DB Queries](#generating-db-queries)
- [Contributing Guidelines](#contributing-guidelines)
- [Project Structure](#project-structure)

## Gateway Operator Quickstart Guide

To onboard the gateway server without having to dig deep, you can follow the [Quick Onboarding Guide](docs/quick-onboarding-guide.md).

### Interested in learning more?

We have an abundance of information in the [docs](docs) section:

1. [Gateway Server Overview](docs/overview.md)
2. [Gateway Server API Endpoints](docs/api-endpoints.md)
3. [Gateway Server System Architecture](docs/system-architecture.md)
4. [Gateway Server Node Selection](docs/node-selection.md)
5. [POKT Primer](docs/pokt-primer.md)
6. [POKT's Relay Specification](docs/pokt-relay-specification.md)

## Docker Image Releases
Every release candidate is published to https://github.com/pokt-network/gateway-server/pkgs/container/pocket-gateway-server

Every release candidate is published to [gateway-server/pkgs/container/pocket-gateway-server](https://github.com/pokt-network/gateway-server/pkgs/container/pocket-gateway-server).

## Docker Compose

There is an all-inclusive docker-compose file available for development [docker-compose.yml](docker-compose.yml.sample)

## Minimum Hardware Requirements to run
## Minimum Hardware Requirements

To run a Gateway Server, we recommend the following minimum hardware requirements:

- 1GB of RAM
- 1GB of storage
- 4 vCPUs+

In production, we have observed memory usage increase to 4GB+. The memory footprint will be dependent on the number of app stakes/chains staked and total traffic throughput.

## Creating a DB Migration
## Database Migrations

<!-- TODO_IMPROVE: Need more details on why we need a DB, how it's used, pre-requisites, etc... -->

### Creating a DB Migration

Migrations are like version control for your database, allowing your team to define and share the application's database schema definition.
Before running a migration make sure to install the go lang migration cli on your machine.
https://github.com/golang-migrate/migrate/tree/master/cmd/migrate

Before running a migration make sure to install the go lang migration cli on your machine. See [golang-migrate/migrate/tree/master/cmd/migrate](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate) for reference.

```sh
./scripts/migration.sh -n {migration_name}
```

This command will generate a up and down migration in `db_migrations`

## Applying a DB Migration
### Applying a DB Migration

DB Migrations are applied upon server start, but as well, it can be applied manually through:

```sh
./scripts/migration.sh {--down or --up} {number_of_times}
./scripts/migration.sh {--down or --up} {number_of_times}
```

### Usage
### DB Migration helpers

#### Apply Migrations
#### Applying Migrations

- To apply all migrations:

```sh
./scripts/migration.sh --up
```

- To apply a specific number of migrations:

```sh
./scripts/migration.sh --up 2
```

#### Rollback Migrations
#### Migrations Rollbacks

Make sure to provide either the number of migrations to rollback or the `--all` flag to rollback all migrations.

- To roll back a specific number of migrations:

```sh
./scripts/migration.sh --down 2
```

- To roll back all migrations:

```sh
./scripts/migration.sh --down --all
```

## Running Tests
## Unit Testing

### Generating Mocks

Install Mockery with
```

```bash
go install github.com/vektra/mockery/[email protected]
```

You can generate the mock files through:

```sh
./scripts/mockgen.sh
```

By running this command, it will generate the mock files in `./mocks` folder.
Reference for mocks can be found here https://vektra.github.io/mockery/latest/

Reference for mocks can be found [here](https://vektra.github.io/mockery/latest).

### Running Tests

Run this command to run tests:

```sh
go test ./...
go test -v -count=1 ./...
```

## Generating DB Queries
Gateway server uses [PGGen]('https://github.com/jschaf/pggen) to create autogenerated type-safe queries.
Queries are added inside [queries.sql](internal%2Fdb_query%2Fqueries.sql) and then regenerated via `./scripts/querygen.sh`.

Gateway server uses [PGGen](https://github.com/jschaf/pggen) to create autogenerated type-safe queries.
Queries are added inside [queries.sql](./internal/Fdb_query/queries.sql) and re-generated via `./scripts/querygen.sh`.

## Contributing Guidelines

1. Create a Github Issue on the feature/issue you're working on.
2. Fork the project
3. Create new branch with `git checkout -b "branch_name"` where branch name describes the feature.
- All branches should be based off `main`
3. Write your code
4. Make sure your code lints with `go fmt ./...` (This will Lint and Prettify)
5. Commit code to your branch and issue a pull request and wait for at least one review.
- Always ensure changes are rebased on top of main branch.
- All branches should be based off `main`
4. Write your code
5. Make sure your code lints with `go fmt ./...` (This will Lint and Prettify)
6. Commit code to your branch and issue a pull request and wait for at least one review.
- Always ensure changes are rebased on top of main branch.
---
## Project Structure
- **cmd:** Contains the entry point of the binaries
- **gateway_server:** HTTP Server for serving requests
- **internal:** Shared internal folder for all binaries
- **pkg:** Distributable dependencies
- **docs:** Project documentation and specifications
A partial high-level view of the code structure (generated)
```bash
.
├── cmd # Contains the entry point of the binaries
│   └── gateway_server # HTTP Server for serving requests
├── internal # Shared internal folder for all binaries
├── pkg # Distributable dependencies
└── scripts # Contains scripts for development
```
_Generate via `tree -L 2`_
---
Olshansky's stream of thought here: https://www.notion.so/buildwithgrove/Olshansky-PATH-Journal-dfc92813ca5945ac88035cca97ccbff5?pvs=4
3 changes: 2 additions & 1 deletion cmd/gateway_server/internal/transform/qos_node.go
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
package transform

import (
"math"

"github.com/pokt-network/gateway-server/cmd/gateway_server/internal/models"
internal_model "github.com/pokt-network/gateway-server/internal/node_selector_service/models"
"math"
)

func ToPublicQosNode(node *internal_model.QosNode) *models.PublicQosNode {
Expand Down
25 changes: 18 additions & 7 deletions docs/altruist-chain-configuration.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,40 @@
# Altruist (Failover) Request
# Chain & Altruist Configurations <!-- omit in toc -->

## Altruist (Failover) Request

- [Altruist (Failover) Request](#altruist-failover-request)
- [Chain Configuration](#chain-configuration)
- [Inserting a custom chain configuration](#inserting-a-custom-chain-configuration)

In rare situations, a relay cannot be served from POKT Network. Some sample scenarios for when this can happen:

1. Dispatcher Outages - Gateway server cannot retrieve the necessary information to send a relay
2. Bad overall QoS - Majority of Node operators may have their nodes misconfigured improperly or there is a lack of node operators supporting the chain with respect to load.
3. Chain halts - In extreme conditions, if the chain halts, node operators may stop responding to relay requests

So the Gateway Server will attempt to route the traffic to a backup chain node. This could be any source, for example:

1. Other gateway operators chain urls
2. Centralized Nodes

# Chain Configuration
## Chain Configuration

Given that every chain has differences and have different sources for fail over, the gateway server allows for optional customization for request timeouts, failover relay, and QoS checks.
The data is stored inside the `chain_configuration` table and is accessed via the [chain_configurations_registry_service.go](..%2Finternal%2Fchain_configurations_registry%2Fchain_configurations_registry_service.go).
The data is stored inside the `chain_configuration` table and is accessed via the [chain_configurations_registry_service.go](../internal/chain_configurations_registry/chain_configurations_registry_service.go).

_While it is **recommended** that you provide a chain configuration, the gateway server will assume defaults provided from the specified [config_provider.go](../internal/global_config/config_provider.go) and the provided QoS [checks](../internal/node_selector_service/checks)_ if not provided.

_While it is **recommended** that you provide a chain configuration, the gateway server will assume defaults provided from the specified [config_provider.go](..%2Finternal%2Fglobal_config%2Fconfig_provider.go) and the provided QoS [checks](..%2Finternal%2Fnode_selector_service%2Fchecks)_ if not provided.
## Inserting a custom chain configuration

# Inserting a custom chain configuration
```sql
-- Insert an example configuration for Ethereum --
INSERT INTO chain_configurations (chain_id, pocket_request_timeout_duration, altruist_url, altruist_request_timeout_duration, top_bucket_p90latency_duration, height_check_block_tolerance, data_integrity_check_lookback_height) VALUES ('0000', '15s', 'https://example.com', '30s', '150ms', 100, 25);
```

- `chain_id` - id of the Pocket Network Chain
- `pocket_request_time` - duration of the maximum amount of time for a network relay to respond
- `altruist_url` - source of the relay in the event that a network request fails
- `altruist_url` - source of the relay in the event that a network request fails
- `altruist_request_timeout_duration` - duration of the maximum amount of time for a backup request to respond
- `top_bucket_p90latency_duration` - maximum amount of latency for nodes to be favored 0 <= x <= `top_bucket_p90latency_duration`
- `top_bucket_p90latency_duration` - maximum amount of latency for nodes to be favored 0 <= x <= `top_bucket_p90latency_duration`
- `height_check_block_tolerance` - number of blocks a node is allowed to be behind (some chains may have node operators moving faster than others)
- `data_integrity_check_lookback_height` - number of blocks data integrity will look behind for source of truth block for other node operators to attest too
Loading

0 comments on commit 7587dad

Please sign in to comment.