pg debug dashboard (#1570)

PostgreSQL debug tools

book/src/SUMMARY.md

@@ -31,6 +31,7 @@
     - [Metrics](framework/observability/metrics.md)
     - [Logs](framework/observability/logs.md)
     - [Profiling](framework/observability/profiling.md)
+    - [PostgreSQL](framework/observability/postgresql.md)
     - [Traces]()
   - [Blockscout](framework/observability/blockscout.md)
   - [Components](framework/components/overview.md)

book/src/framework/observability/blockscout.md

@@ -2,27 +2,39 @@
 
 You can use local [Blockscout](https://www.blockscout.com/) instance to debug EVM smart contracts.
 
+Some images require `ghcr` auth, login first and pass the token:
+
+```
+gh auth token | docker login ghcr.io -u github-username --password-stdin
+```
+
+Start Blockscout
+
 ```
 ctf bs up
 ```
+
 Your `Blockscout` instance is up on [localhost](http://localhost)
 
 To remove it, we also clean up all Blockscout databases to prevent stale data when restarting your tests.
+
 ```
 ctf bs down
 ```
 
 ## Selecting Blockchain Node
 
 By default, we connect to the first `anvil` node, but you can select the node explicitly
+
 ```
 ctf bs -r http://host.docker.internal:8545 d
 ctf bs -r http://host.docker.internal:8555 d
 ```
 
 <div class="warning">
 
-Blockscout isn’t ideal for local, ephemeral environments, as it won’t re-index blocks and transactions on test reruns. The easiest approach is to set up Blockscout first, initialize the test environment, switch to the [cache](../components/caching.md) config, and run tests without restarting RPC nodes. 
+Blockscout isn’t ideal for local, ephemeral environments, as it won’t re-index blocks and transactions on test reruns. The easiest approach is to set up Blockscout first, initialize the test environment, switch to the [cache](../components/caching.md) config, and run tests without restarting RPC nodes.
 
 Otherwise, use `ctf bs r` each time you restart your test with a fresh docker environment.
-</div>
+
+</div>

book/src/framework/observability/observability_stack.md

@@ -1,12 +1,22 @@
 # Local Observability Stack
 
 You can use a local observability stack, framework is connected to it by default
+
 ```bash
 ctf obs up
 ```
+
 To remove it use
+
 ```bash
 ctf obs down
 ```
 
-Read more about how to check [logs](logs.md) and [profiles](profiling.md)
+Read more about how to check [logs](logs.md) and [profiles](profiling.md)
+
+## Helpful links for your local setup
+
+- [Loki](http://localhost:3000/explore?panes=%7B%22qZw%22:%7B%22datasource%22:%22P8E80F9AEF21F6940%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22P8E80F9AEF21F6940%22%7D%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1)
+- [Prometheus](http://localhost:3000/explore?panes=%7B%22qZw%22:%7B%22datasource%22:%22PBFA97CFB590B2093%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%22,%22range%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22PBFA97CFB590B2093%22%7D%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1)
+- [PostgreSQL](http://localhost:3000/d/000000039/postgresql-database?orgId=1&refresh=10s&var-DS_PROMETHEUS=PBFA97CFB590B2093&var-interval=$__auto_interval_interval&var-namespace=&var-release=&var-instance=postgres_exporter_0:9187&var-datname=All&var-mode=All&from=now-5m&to=now)
+- [Pyroscope](http://localhost:4040)

book/src/framework/observability/postgresql.md

@@ -0,0 +1,18 @@
+# PostgreSQL
+
+## Metrics
+
+You can go to [this](http://localhost:3000/d/000000039/postgresql-database?orgId=1&refresh=10s&var-DS_PROMETHEUS=PBFA97CFB590B2093&var-interval=$__auto_interval_interval&var-namespace=&var-release=&var-instance=postgres_exporter_0:9187&var-datname=All&var-mode=All&from=now-5m&to=now) dashboard and select `Instance`.
+
+We run exporter for the first 5 nodes, that should be enough to debug performance issues.
+
+## Data sources and raw queries
+
+You can use `PostgreSQL X` datasources, first 5 nodes to select things for your [dashboards](http://localhost:3000/explore?panes=%7B%22qrr%22:%7B%22datasource%22:%22P4DD770FAD7295D26%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22datasource%22:%7B%22type%22:%22postgres%22,%22uid%22:%22P4DD770FAD7295D26%22%7D,%22format%22:%22table%22,%22rawSql%22:%22select%20%2A%20from%20sessions;%22,%22editorMode%22:%22code%22,%22sql%22:%7B%22columns%22:%5B%7B%22type%22:%22function%22,%22parameters%22:%5B%5D%7D%5D,%22groupBy%22:%5B%7B%22type%22:%22groupBy%22,%22property%22:%7B%22type%22:%22string%22%7D%7D%5D,%22limit%22:50%7D,%22rawQuery%22:true%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1) or debug.
+
+## Slow queries debug
+
+Use `pg_stat_statements` extension, it is enabled for all databases in `NodeSet`
+```
+select * from pg_stat_statements;
+```

book/src/framework/verify.md

@@ -1,5 +1,7 @@
 # Verifying Contracts
 
+You need to install [Foundry](https://book.getfoundry.sh/getting-started/installation) first, `forge` should be available in your `$PATH`.
+
 Check out our [example](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/framework/examples/myproject/verify_test.go) of programmatically verifying contracts using `Blockscout` and `Foundry`. You'll need to provide:
 
 - The path to your Foundry directory
@@ -11,6 +13,7 @@ Check out our [example](https://github.com/smartcontractkit/chainlink-testing-fr
 			"example_components/onchain",
 			"src/Counter.sol",
 			"Counter",
+            "0.8.20" // check your file compiler version on the first line
 		)
 		require.NoError(t, err)
-```
+```

framework/.changeset/v0.4.6.md

@@ -0,0 +1,2 @@
+- Add PostgreSQL exporters and dashboard
+- Update Blockscout docs, add ghcr auth example

framework/cmd/main.go

@@ -17,8 +17,10 @@ import (
 var embeddedObservabilityFiles embed.FS
 
 const (
-	LocalLogsURL      = "http://localhost:3000/explore"
-	LocalPyroScopeURL = "http://localhost:4040"
+	LocalLogsURL          = "http://localhost:3000/explore?panes=%7B%22qZw%22:%7B%22datasource%22:%22P8E80F9AEF21F6940%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22P8E80F9AEF21F6940%22%7D%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1"
+	LocalPrometheusURL    = "http://localhost:3000/explore?panes=%7B%22qZw%22:%7B%22datasource%22:%22PBFA97CFB590B2093%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%22,%22range%22:true,%22datasource%22:%7B%22type%22:%22prometheus%22,%22uid%22:%22PBFA97CFB590B2093%22%7D%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1"
+	LocalPostgresDebugURL = "http://localhost:3000/d/000000039/postgresql-database?orgId=1&refresh=10s&var-DS_PROMETHEUS=PBFA97CFB590B2093&var-interval=$__auto_interval_interval&var-namespace=&var-release=&var-instance=postgres_exporter_0:9187&var-datname=All&var-mode=All&from=now-5m&to=now"
+	LocalPyroScopeURL     = "http://localhost:4040"
 )
 
 func main() {

framework/cmd/observability.go

@@ -22,6 +22,8 @@ func observabilityUp() error {
 	}
 	fmt.Println()
 	framework.L.Info().Msgf("Loki: %s", LocalLogsURL)
+	framework.L.Info().Msgf("Prometheus: %s", LocalPrometheusURL)
+	framework.L.Info().Msgf("PostgreSQL: %s", LocalPostgresDebugURL)
 	framework.L.Info().Msgf("Pyroscope: %s", LocalPyroScopeURL)
 	return nil
 }

framework/cmd/observability/compose/conf/prometheus.yml

@@ -5,14 +5,29 @@ scrape_configs:
   - job_name: 'ctf'
     metrics_path: /metrics
     docker_sd_configs:
-      - host: "unix:///var/run/docker.sock"
+      - host: 'unix:///var/run/docker.sock'
         refresh_interval: 20s
     relabel_configs:
-      - source_labels: [ __meta_docker_port_private ]
+      - source_labels: [__meta_docker_port_private]
         regex: '6688'
         action: keep
   - job_name: cadvisor
     scrape_interval: 10s
     static_configs:
       - targets:
           - cadvisor:8080
+  - job_name: 'postgres_exporter_0'
+    static_configs:
+      - targets: ['postgres_exporter_0:9187']
+  - job_name: 'postgres_exporter_1'
+    static_configs:
+      - targets: ['postgres_exporter_1:9187']
+  - job_name: 'postgres_exporter_2'
+    static_configs:
+      - targets: ['postgres_exporter_2:9187']
+  - job_name: 'postgres_exporter_3'
+    static_configs:
+      - targets: ['postgres_exporter_3:9187']
+  - job_name: 'postgres_exporter_4'
+    static_configs:
+      - targets: ['postgres_exporter_4:9187']