[RFC] [OpenTelemetry] Export synthetics results

[thibaultleouay]

Summary

We want to implement OpenTelemetry for OpenStatus and its users. In this proposal we will detailed how to export synthetic check results to OpenStatus users OpenTelemetry endpoint.

Motivation

We want to embrace the rest of the o11y stack. Our synthetics checks should be easily exportable for our users.

Guide Implementation

Dashboard

In the dashboard in the setting page we should have an otel tab, where the user can set his otel endpoint

Probes

We need to send the data to the user endpoint

API Server

Prometheus Endpoint : if the user want to scrape our endpoint.

Reference Implementation

The workspace should contain a new config with the users endpoint
we should send it via grpc

When we create the new check for the monitor we should add a new value, exportUrl which is the value where we export the new metrics

Grafana

https://grafana.com/docs/grafana-cloud/send-data/otlp/send-data-otlp/

New Relic

https://docs.newrelic.com/docs/opentelemetry/best-practices/opentelemetry-otlp/

HoneyComb

https://docs.honeycomb.io/send-data/opentelemetry/#about-the-otel_exporter_otlp_endpoint-environment-variable

Drawbacks

Rationale / Alternatives

TBD

Unresolved Questions

** Open Question**: Which metrics should be exported ?