forked from snapp-incubator/thanos-federate-proxy
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
implement OpenMetrics support for real
I discovered that thanos-federate-proxy lies about its capabilities: it's not truly OpenMetrics compliant. The prometheus metrics scrape endpoint representation differs from OpenMetrics in subtle ways. This makes consumers that only support OpenMetrics fail to scrape anything produced by thanos-federate-proxy. I don't think this is a good idea, but wanted to see if it was technically possible. I'm also not sure if this is a stupid contortion and maybe there is a better way to achieve the same. prometheus-federate-proxy retrieves from the upstream using Prometheus Query API. This API returns model.Value, which is a wire representation that does not have all the same information presented in a scrape endpoint (MetricsFamily). If we transformed to MetricsFamily _somehow_, we would be able to use the expfmt package from prometheus/common library that has the official bits for content negotiation and serialization into the scrape endpoint textual representation. So the challenge here was how to transform model.Value to MetricsFamily. I discovered Prometheus offers Metadata API, which gives us the missing bits. Unfortunately Thanos does not expose this API, so this commit requires you to wire up the metadata API through the upstream prometheus. API Queries will hit Thanos, metadata API calls will hit prometheus. The latter is done outside the critical path on an interval, which is currently hardcoded to 1 minute. No guarantees here. I'm not sure if the transformation is done correctly, the logic that transforms Value to MetricsFamiliy os
- Loading branch information
1 parent
f3ae53a
commit bc04d9a
Showing
3 changed files
with
214 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters