Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable prometheus queries #1540

Open
gustavomonarin opened this issue May 3, 2021 · 4 comments · May be fixed by #2127
Open

Configurable prometheus queries #1540

gustavomonarin opened this issue May 3, 2021 · 4 comments · May be fixed by #2127
Labels
functionality A feature request. good first issue A good fit as a first issue in the project.

Comments

@gustavomonarin
Copy link

gustavomonarin commented May 3, 2021

Kafka JMX metrics exported to Prometheus can have a very diverse representation and different calculation approaches (i.e.: promql avg vs jmx pre-calculated mean)

#1366 did a great job introducing the prometheus client however it is not flexible enough to make cruise control adoption less intrusive mainly for already running production clusters, as a common representation / pattern of the jmx metrics follows the example of the jmxexporter project which does not match the provided default queries.

#1366 also adds a nicely extension for providing a query supplier class, however the cruise configuration is not forward what makes complicated to access / add extra properties (i.e: properties file containing all the queries)

I would like to propose a new prometheus query supplier where the queries are extenalised to a properties configuration file.

I have made a simple test, by using the RawMetricType as key and the jmx_exporter example queries as values together with a small change to make the query supplier CruiseControlConfigurable and it works nicely.

Please let me know if you have any concerns/suggestions. If this solution brings value i would be more than happy to polish the code and create a PR in the following days.

@efeg efeg added the functionality A feature request. label May 18, 2021
@rmb938
Copy link

rmb938 commented May 29, 2021

@gustavomonarin This would be really useful for me when using other metrics that aren't node exporter and jmx exporter. For example being able to use container metrics from kubernetes for cpu usage and topic/partition metrics from different exporters like kminion.

@efeg efeg added the good first issue A good fit as a first issue in the project. label Jul 16, 2021
@HimanshuKhatarkar
Copy link

Hi @gustavomonarin is this issue Still Open? Please assign this to me so that I can Help you with this.

@gustavomonarin
Copy link
Author

gustavomonarin commented Nov 20, 2023

Hi @HimanshuKhatarkar ,
I don't have permissions to assign the issue to you, however i believe it is just a question of raising the MR.

Unfortunately, when i was working on this most of the implementation was done as part of my work time as a contractor in a company and the company was not really easy to convince to share the code even though a chunck of it was on my free time, so i put as much details as i could in the description. Hope it is still up to date, but should be straight forward.

gchouet pushed a commit to criteo-forks/cruise-control that referenced this issue Feb 23, 2024
  allow externalized prometheus queries in a properties file

This resolves linkedin#1540
@gchouet gchouet linked a pull request Feb 23, 2024 that will close this issue
@Bismansinghm
Copy link

Can i contribute to this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
functionality A feature request. good first issue A good fit as a first issue in the project.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants