Update help text documentation

input-output-hk · Dec 7, 2023 · 3c47358 · 3c47358
1 parent 6eefa79
commit 3c47358
Show file tree

Hide file tree

Showing 7 changed files with 270 additions and 48 deletions.
diff --git a/docs/developer-tools/runtime/services/marlowe-chain-indexer.md b/docs/developer-tools/runtime/services/marlowe-chain-indexer.md
@@ -5,7 +5,7 @@ title: Marlowe chain indexer service
 The `marlowe-chain-indexer` executable follows a local blockchain node and writes the blocks and transactions to a database.
 
 ```console
-marlowe-chain-indexer : a chain indexer for the Marlowe Runtime.
+marlowe-chain-indexer: Chain indexer for the Marlowe Runtime.
 
 Usage: marlowe-chain-indexer [--version] (-s|--socket-path SOCKET_FILE)
                              [-m|--testnet-magic INTEGER]
@@ -16,7 +16,20 @@ Usage: marlowe-chain-indexer [--version] (-s|--socket-path SOCKET_FILE)
                              [--block-cost COST_UNITS] [--tx-cost COST_UNITS]
                              [--max-cost COST_UNITS] [--http-port PORT_NUMBER]
 
-  Chain indexer for Marlowe Runtime.
+  The chain indexer for the Marlowe Runtime. This component connects to a local
+  Cardano Node and follows the chain. It copies a subset of the information
+  contained in every block to a postgresql database. This database can be queried
+  by downstream components, such as marlowe-chain-sync.
+
+  There should only be one instance of marlowe-chain-indexer writing data to a
+  given chain database. There is no need to run multiple indexers. If you would
+  like to scale runtime services, it is recommended to deploy a postgres replica
+  cluster, run one indexer to populate it, and as many marlowe-chain-sync
+  instances as required to read from it.
+
+  Before running the indexer, the database must be created and migrated using
+  sqitch. The migration plan and SQL scripts are included in the source code
+  folder for marlowe-chain-indexer.
 
 Available options:
   -h,--help                Show this help text
@@ -45,7 +58,7 @@ Available options:
                            when persisting blocks. If the cost of the current
                            batch would exceed this value, the chain sync client
                            will wait until the current batch is persisted before
-                           requesting another block. (default: 100000)
+                           requesting another block. (default: 1000000)
   --http-port PORT_NUMBER  Port number to serve the http healthcheck API on
                            (default: 8080)
 ```
diff --git a/docs/developer-tools/runtime/services/marlowe-chain-sync.md b/docs/developer-tools/runtime/services/marlowe-chain-sync.md
@@ -5,15 +5,36 @@ title: Marlowe chain sync service
 The `marlowe-chain-sync` executable provides services for querying the blockchain for information that may relate to Marlowe contracts.
 
 ```console
-marlowe-chain-sync : a chain sync server for the Marlowe Runtime.
+marlowe-chain-sync: Chain query and sync server for the Marlowe Runtime.
 
-Usage: marlowe-chain-sync [--version] (-s|--socket-path SOCKET_FILE)
+Usage: marlowe-chain-sync [--version] (-s|--socket-path SOCKET_FILE) 
                           [-m|--testnet-magic INTEGER]
                           (-d|--database-uri DATABASE_URI) [-h|--host HOST_NAME]
-                          [--port PORT_NUMBER] [--query-port PORT_NUMBER]
+                          [--port PORT_NUMBER] [--query-port PORT_NUMBER] 
                           [--job-port PORT_NUMBER] [--http-port PORT_NUMBER]
 
-  Chain sync server for Marlowe Runtime.
+  The chain query engine for the Marlowe Runtime. This component exposes three
+  protocols through which downstream components can interact with the blockchain.
+  These are: chain seek, chain query, and chain command.
+
+  The chain seek protocol is a synchronization protocol which allows the follower
+  to jump directly ahead to blocks that match a particular query.
+
+  The chain query protocol allows various network parameters and UTxO state to be
+  queried.
+
+  The chain command protocol allows transactions to be submitted to the connected node.
+
+  marlowe-chain-sync relies on the connected database being migrated and populated by
+  a marlowe-chain-indexer instance. While marlowe-chain-sync can operate without
+  marlowe-chain-indexer running, marlowe-chain-indexer must be run first to insert
+  the genesis UTxOs before marlowe-chain-sync can be used, and relies on
+  marlowe-chain-indexer to keep the database up-to-date.
+
+  marlowe-chain-sync is designed to scale horizontally. That is to say, multiple
+  instances can run in parallel to scale with demand. A typical setup for this would
+  involve running multiple marlowe-chain-sync instances in front of a load balancer
+  against a scalable postgres replica cluster being populated by a single chain indexer.
 
 Available options:
   -h,--help                Show this help text

diff --git a/docs/developer-tools/runtime/services/marlowe-contract.md b/docs/developer-tools/runtime/services/marlowe-contract.md
@@ -5,19 +5,49 @@ title: Marlowe contract service
 The `marlowe-contract` executable provides services related to managing the Marlowe Runtime contract store.
 
 ```console
-marlowe-contract : a contract storage service for the Marlowe Runtime.
+marlowe-contract: Contract storage service for the Marlowe Runtime.
 
-Usage: marlowe-contract [-h|--host HOST_NAME] [-p|--port PORT_NUMBER]
-                        [--query-port PORT_NUMBER] [--transfer-port PORT_NUMBER]
-                        [-b|--buffer-size INTEGER] [-s|--store-dir DIR]
-                        [--store-staging-dir DIR]
+Usage: marlowe-contract [--version] [-h|--host HOST_NAME] 
+                        [-p|--port PORT_NUMBER] [--query-port PORT_NUMBER] 
+                        [--transfer-port PORT_NUMBER] 
+                        [--chain-sync-host HOST_NAME] 
+                        [--chain-sync-query-port PORT_NUMBER] 
+                        [--marlowe-sync-host HOST_NAME] 
+                        [--marlowe-bulk-port PORT_NUMBER] 
+                        [-b|--buffer-size INTEGER] [-s|--store-dir DIR] 
+                        [--store-staging-dir DIR] 
                         [--store-lock-microseconds-between-retries MICRO_SECONDS]
-                        [--http-port PORT_NUMBER]
+                        [--http-port PORT_NUMBER] [--min-contract-age MINUTES] 
+                        [--max-store-size BYTES]
 
-  Contract storage service for Marlowe Runtime
+  The contract storage service for the Marlowe Runtime. It manages a crucial component of the
+  Marlowe runtime: the contract store. The contract store is a content-addressable store of
+  contracts indexed by their hashes. Contracts can refer to sub-contracts via their hashes via
+  the MerkleizedCase construct. The contract store is used to store the continuations of a contract
+  after it has been merkleized (a process which recursively replaces Case constructs with MerkleizedCase
+  constructs). It is also used to lookup continuations when applying inputs to a merkleized contract.
+  This component exposes three protocols: marlowe load, marlowe transfer, and contract store query.
+
+  The marlowe load protocol is one way to import a contract incrementally into the store. It presents
+  a stack-based interface for pushing a contract depth-first into the store.
+
+  The marlowe transfer protocol is the other way to import a contract incrementally into the store.
+  It leverages the Marlowe object model to allow bundles of user-defined marlowe objects to be streamed
+  into the store. marlowe-contract will link the contract on-the-fly, merkleize the intermediate contracts,
+  and build the final contract incrementally. This protocol is generally more efficient and flexible than
+  Marlowe load because it allows duplicate sub-contracts to be pre-abstracted by the user. It also supports
+  An export mode which will stream the closure of a contract from the store to the client as a Marlowe object
+  bundle. This can be used to export continuations from the store and share it with other contract stores.
+
+  The contract store query protocol provides queries that allow clients to fetch contracts by their hash,
+  fetch the adjacency or closure sets of a contract, or merkleize an input.
+
+  marlowe-contract depends on a marlowe-sync and marlowe-chain-sync instance to run automatic
+  Garbage collection. These must both be running in order for marlowe-contract to run.
 
 Available options:
   -h,--help                Show this help text
+  --version                Show version.
   -h,--host HOST_NAME      The host name to run the server on.
                            (default: "127.0.0.1")
   -p,--port PORT_NUMBER    The port number to run the marlowe load server on.
@@ -27,14 +57,36 @@ Available options:
   --transfer-port PORT_NUMBER
                            The port number to run the transfer server on.
                            (default: 3729)
+  --chain-sync-host HOST_NAME
+                           The host name of the chain sync server.
+                           (default: "127.0.0.1")
+  --chain-sync-query-port PORT_NUMBER
+                           The chain sync server's query protocol port.
+                           (default: 3716)
+  --marlowe-sync-host HOST_NAME
+                           The hostname of the Marlowe Runtime marlowe-sync
+                           server. Can be set as the environment variable
+                           MARLOWE_RT_SYNC_HOST (default: "127.0.0.1")
+  --marlowe-bulk-port PORT_NUMBER
+                           The port number of the marlowe-sync server's bulk
+                           synchronization API. Can be set as the environment
+                           variable MARLOWE_RT_SYNC_MARLOWE_BULK_PORT
+                           (default: 3730)
   -b,--buffer-size INTEGER The number of contracts to accept from the client
                            before flushing to disk. (default: 512)
-  -s,--store-dir DIR       The root directory of the contract store.
+  -s,--store-dir DIR       The root directory of the contract store
+                           (default: "/home/jamie/.local/share/marlowe/runtime/marlowe-contract/store")
   --store-staging-dir DIR  The root directory of the contract store staging
-                           areas.
+                           areas (default: "/tmp/nix-shell.er7Lmt")
   --store-lock-microseconds-between-retries MICRO_SECONDS
                            The number of microseconds to wait between retries
                            when acquiring the store lock (default: 500000)
   --http-port PORT_NUMBER  Port number to serve the http healthcheck API on
                            (default: 8080)
+  --min-contract-age MINUTES
+                           The minimum age contracts in the store must reach
+                           before they can be garbage collected.
+                           (default: 86400s)
+  --max-store-size BYTES   The maximum allowed size of the contract store, in
+                           bytes. (default: 34359738368)
 ```
diff --git a/docs/developer-tools/runtime/services/marlowe-indexer.md b/docs/developer-tools/runtime/services/marlowe-indexer.md
@@ -5,18 +5,31 @@ title: Marlowe indexer service
 The `marlowe-indexer` executable follows `marlowe-chain-sync` and writes Marlowe contract transactions to a database.
 
 ```console
-marlowe-indexer : a contract indexing service for the Marlowe Runtime.
+marlowe-indexer: Contract indexing service for the Marlowe Runtime.
 
-Usage: marlowe-indexer [--chain-sync-port PORT_NUMBER]
-                       [--chain-sync-query-port PORT_NUMBER]
+Usage: marlowe-indexer [--version] [--chain-sync-port PORT_NUMBER] 
+                       [--chain-sync-query-port PORT_NUMBER] 
                        [--chain-sync-host HOST_NAME]
-                       (-d|--database-uri DATABASE_URI)
+                       (-d|--database-uri DATABASE_URI) 
                        [--http-port PORT_NUMBER]
 
-  Contract indexing service for Marlowe Runtime
+  The contract indexer for the Marlowe Runtime. It connects to a marlowe-chain-sync
+  instance using both the chain seek and chain query protocol. It scans the chain for
+  Marlowe contract transactions and saves them in a postgresql database. This database
+  can be queried by downstream components, such as marlowe-sync.
+
+  There should only be one instance of marlowe-indexer writing data to a given marlowe
+  database. There is no need to run multiple indexers. If you would like to scale runtime
+  services, it is recommended to deploy a postgres replica cluster, run one indexer to
+  populate it, and as many marlowe-sync instances as required to read from it.
+
+  Before running the indexer, the database must be created and migrated using
+  sqitch. The migration plan and SQL scripts are included in the source code
+  folder for marlowe-indexer.
 
 Available options:
   -h,--help                Show this help text
+  --version                Show version.
   --chain-sync-port PORT_NUMBER
                            The port number of the chain sync server.
                            (default: 3715)

diff --git a/docs/developer-tools/runtime/services/marlowe-proxy.md b/docs/developer-tools/runtime/services/marlowe-proxy.md
@@ -5,23 +5,29 @@ title: Marlowe proxy service
 The `marlowe-proxy` executable provides a single unified public API for the whole runtime.
 
 ```console
-marlowe-proxy : an API proxy service for the Marlowe Runtime.
+marlowe-proxy: The API gateway server for the Marlowe Runtime.
 
-Usage: marlowe-proxy [-h|--host HOST_NAME] [-p|--port PORT_NUMBER]
+Usage: marlowe-proxy [--version] [-h|--host HOST_NAME] [-p|--port PORT_NUMBER] 
                      [--port-traced PORT_NUMBER] [--marlowe-sync-host HOST_NAME]
-                     [--marlowe-sync-port PORT_NUMBER]
-                     [--marlowe-header-port PORT_NUMBER]
-                     [--marlowe-query-port PORT_NUMBER]
-                     [--marlowe-contract-host HOST_NAME]
-                     [--marlowe-load-port PORT_NUMBER]
-                     [--marlowe-transfer-port PORT_NUMBER]
-                     [--contract-query-port PORT_NUMBER] [--tx-host HOST_NAME]
+                     [--marlowe-sync-port PORT_NUMBER] 
+                     [--marlowe-header-port PORT_NUMBER] 
+                     [--marlowe-bulk-port PORT_NUMBER] 
+                     [--marlowe-query-port PORT_NUMBER] 
+                     [--marlowe-contract-host HOST_NAME] 
+                     [--marlowe-load-port PORT_NUMBER] 
+                     [--marlowe-transfer-port PORT_NUMBER] 
+                     [--contract-query-port PORT_NUMBER] [--tx-host HOST_NAME] 
                      [--tx-command-port PORT_NUMBER] [--http-port PORT_NUMBER]
 
-  API proxy service for Marlowe Runtime
+  The API gateway server for the Marlowe Runtime. It exposes all the public protocols of the
+  Marlowe runtime as a single multiplexed protocol: marlowe sync, marlowe header sync, marlowe
+  bulk sync, marlowe query, marlowe transaction job, marlowe load, marlowe transfer, and contract
+  store query. Please consult the help text for the individual services for detailed descriptions
+  of these individual protocols.
 
 Available options:
   -h,--help                Show this help text
+  --version                Show version.
   -h,--host HOST_NAME      The host name to run the server on.
                            (default: "127.0.0.1")
   -p,--port PORT_NUMBER    The port number to run the server on. (default: 3700)
@@ -42,6 +48,11 @@ Available options:
                            synchronization API. Can be set as the environment
                            variable MARLOWE_RT_SYNC_MARLOWE_HEADER_PORT
                            (default: 3725)
+  --marlowe-bulk-port PORT_NUMBER
+                           The port number of the marlowe-sync server's bulk
+                           synchronization API. Can be set as the environment
+                           variable MARLOWE_RT_SYNC_MARLOWE_BULK_PORT
+                           (default: 3730)
   --marlowe-query-port PORT_NUMBER
                            The port number of the marlowe-sync server's query
                            API. Can be set as the environment variable

diff --git a/docs/developer-tools/runtime/services/marlowe-sync.md b/docs/developer-tools/runtime/services/marlowe-sync.md
@@ -5,19 +5,50 @@ title: Marlowe sync service
 The `marlowe-sync` executable provides protocols that access Marlowe contract information.
 
 ```console
-marlowe-sync : a contract synchronization and query service for the Marlowe
+marlowe-sync: Contract synchronization and query service for the Marlowe
 Runtime.
 
-Usage: marlowe-sync (-d|--database-uri DATABASE_URI) [--sync-port PORT_NUMBER]
-                    [--header-sync-port PORT_NUMBER] [--query-port PORT_NUMBER]
-                    [-h|--host HOST_NAME] [--chain-sync-host HOST_NAME]
-                    [--chain-sync-query-port PORT_NUMBER]
+Usage: marlowe-sync [--version] (-d|--database-uri DATABASE_URI) 
+                    [--sync-port PORT_NUMBER] [--header-sync-port PORT_NUMBER] 
+                    [--bulk-sync-port PORT_NUMBER] [--query-port PORT_NUMBER] 
+                    [-h|--host HOST_NAME] [--chain-sync-host HOST_NAME] 
+                    [--chain-sync-query-port PORT_NUMBER] 
                     [--http-port PORT_NUMBER]
 
-  Contract synchronization and query service for Marlowe Runtime
+  The contract query engine for the Marlowe Runtime. This component exposes four
+  protocols through which downstream components can interact with the blockchain.
+  These are: marlowe sync, marlowe header sync, marlowe bulk sync, and marlowe query.
+
+  The marlowe sync protocol is a synchronization protocol which follows the history
+  of a specific marlowe contract.
+
+  The marlowe header sync protocol is a synchronization protocol which scans the chain
+  for new Marlowe contracts and presents them as a compact summary called a header.
+
+  The marlowe bulk sync protocol is a synchronization protocol which combines the
+  capabilities of marlowe header sync and marlowe sync. It presents a stream of blocks
+  which contain a combination of all three contract transaction types: creation, input
+  application, and payout withdrawal.
+
+  The marlowe query protocol supports multiple queries that allow clients to fetch
+  data about marlowe contracts as of the current blockchain tip. This means that it
+  cannot guarantee consistent results between different queries, because the chain could
+  update in between queries, changing the result of queries. If consistency is needed,
+  use one of the sync protocols.
+
+  marlowe-sync relies on the connected database being migrated and populated by a
+  marlowe-indexer instance. While marlowe-sync can operate without marlowe-indexer running,
+  The sqitch migrations must first be performed in order to create the expected tables, and
+  marlowe-indexer must be running to keep the database up-to-date.
+
+  marlowe-sync is designed to scale horizontally. That is to say, multiple instances can run
+  in parallel to scale with demand. A typical setup for this would involve running multiple
+  marlowe-sync instances in front of a load balancer against a scalable postgres replica
+  cluster being populated by a single marlowe-indexer.
 
 Available options:
   -h,--help                Show this help text
+  --version                Show version.
   -d,--database-uri DATABASE_URI
                            URI of the database where the contract information is
                            saved.
@@ -26,6 +57,9 @@ Available options:
   --header-sync-port PORT_NUMBER
                            The port number to run the header sync protocol on.
                            (default: 3725)
+  --bulk-sync-port PORT_NUMBER
+                           The port number to run the bulk sync protocol on.
+                           (default: 3730)
   --query-port PORT_NUMBER The port number to run the query protocol on.
                            (default: 3726)
   -h,--host HOST_NAME      The host name to run the server on.