Support for alerting?

[JohnBolander]

Any chance this plugin might be made to supporting alerting through grafana?
https://grafana.com/tutorials/build-a-data-source-backend-plugin/

[complacentsee]

I have some foundational work to support this created, but it is not caught up to the latest commits and needs a bit more work to fully support summary types. aa86f38

This is a foundational change and would need support for the project owners before making discussing any merging.

[coderReview]

@complacentsee do you have this in a fork? Can you grant me access to your fork? Thank you.

[pkoeltz]

Hi everyone,

We would be very much interested into this plugin but we need the Alerting feature as well.

Any chance this is already in your focus for the next releases @elwills ?

If you need help we could participate.

[complacentsee]

My POC branch still needs work to be release ready. The biggest outstanding concerns I can think of are the following. I'm sure there are more that I'm not thinking of as I haven't looked at the code in a week.

1. The existing plugin does not honor the Grafana "Max data points" setting. It directly passes this onto the PI Web API which may respond with up to 5x more data points. This is due to differences of opinion on how OSI views max data vs Grafana. OSI returns up to 5 features for each data point, vs Grafana which is based on the expectation that one horizontal pixel has a single value. If the Web API response with more points than the max data point setting, the frontend data frame buffer loops, and oldest points are thrown away. This causes the trend to not be completely filled from start time to end time. I think there are 2 solutions:
   • Arbitrarily request maxdatapoints/5 (my preferred)
   • Do a recursive request at the expense of longer load times and additional server load.
2. Templating variables is not supported. Basically we need to take the variable injection break the query into multiple queries.
3. Summary data points are not yet supported. This is yet another unique webapi response and needs a special case to be configured to a dataframe.
4. Streaming is supported, however it returns data at realtime or the maximum configured refresh interval on the web api. This can cause the dataframe to be overan quickly if you have a long running query over a fast updating point. I had planned to have each request have the option to update realtime of update interval time. Currently I removed this as I switched to using a single webapi websocket connection for any number of grafana front end users to reduce loading. But it's pretty easy to manage at a "per query" level. I currently generate a UUID for a given query so that we can use persistent websockets. This is also needed as grafana will not update the historical trend data on a query once a streaming channel has been established. A new query is passed to the backend, but if the channel path is identical the front end does not process the update trigger.

[coderReview]

Hello @complacentsee, we are releasing version 4.0.0 by the end of this month or beginning of next month. For the next release (5.0.0) I would like to base it on your work. I will create a new branch once 4.0.0 is completed so we can continue from there. Thank you for your contribution.

[complacentsee]

Is the annotation code base pretty stable for the next release? I forgot that it requires a significant rewrite to be moved to a backend. I can keep looking into that in the meantime.

[coderReview]

@complacentsee we are releasing v4.0.0 this month. Annotations is working properly for PI-AF event frames.

[complacentsee]

Sorry for the delay. I had a couple of work projects keeping me too busy. I'm going to start taking a look at this and will let you know if I see any major obstacles. I think there is still going to be a challenge with Annotations since grafana default is the use standard time based data query for annotations instead of a custom query which means the new backend component may still require coordination depending how the queries are passed through.

Is it okay if I target version grafana v10.0.0 as the minimum for this?

[coderReview]

Hello. Sure we can targe v10.

[coderReview]

Version 4.2.0 is in good shape to be used.

[coderReview]

Hello @complacentsee, were you able to work on this feature?

[complacentsee]

I took some vacation this week and next week to make some measurable progress.

[complacentsee]

I made some significant progress. The majority of the core functionality is is working. I'm working on validating all of the query options are working and have a few more to implement and validate. Current brach for this is here:
https://github.com/complacentsee/osisoftpi-grafana/tree/backend

[coderReview]

Hello, thank you for the work. I will take a look at it next week. Em dom., 23 de jul. de 2023 às 20:00, Adam Traeger ***@***.***> escreveu:

…

I made some significant progress. The majority of the core functionality is is working. I'm working on validating all of the query options are working and have a few more to implement and validate. Current brach for this is here: https://github.com/complacentsee/osisoftpi-grafana/tree/backend — Reply to this email directly, view it on GitHub <#112 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNGUQUNTHELTFT566MIDITXRWUI3ANCNFSM6AAAAAAVN7GAV4> . You are receiving this because you commented.Message ID: ***@***.*** com>

[kristwaa]

@complacentsee , I'm very happy to see you are working on a backend implementation - thank you! I believe this is the way to go for the plugin.
@coderReview , I'm also happy to see that you - as an active contributor to the project - are willing to take a look at the work done by @complacentsee!

Am I correct in assuming this can also be used to support Public Dashboards?
I might be able to help test the implementation - I'll see if I can build it myself.

I agree with the previous statement that this is an important question to consider for the main devs/maintainers of this project. How can the switch to a backend implementation be done? Can both server and browser modes easily be supported? Do both modes need to be supported?
I see reasons why I want the plugin to switch to a backend implementation - are there any technical reasons not to change?

[complacentsee]

This is still a work in development but is moving forward. Most of the work remaining is ensuring consistency with the all of the query options, and supporting summaries. Building it is pretty strait forward, the front end component list still build with yarn build (or whatever option was being used). The backend requires go and mage to be installed. To build the applications run "mage -v" and it will build all of the backend applications for the supported operating systems and architectures.

Yes, public dashboard support and alerting both both have the same dependency on a backend component handling the query. Here is an example of a public dashboard. Alerting configuration is a bit spotty as there may be a bug on the query editor that it runs against. I haven't spent time debugging that issue.

https://grafanaplugintesting.complacentsee.com/public-dashboards/c9b8930a6bf84d8aa3aff4b5a6a76b86

I did most of this work with the intention that the dashboard configurations would be compatible between the front end and the backend. The largest difference is that the existing frontend configuration caches the WebID of an element point and creation time, and mine queries and caches at runtime to allow for easier support of dashboard templates. To my knowledge a plugin cannot change between backend and frontend at runtime.

[coderReview]

Created the branch https://github.com/GridProtectionAlliance/osisoftpi-grafana/tree/backend

[coderReview]

@complacentsee have you changed the react part to use the new go plugin?

[complacentsee]

No, that's a good catch, there are probably a few other methods that need to be cleaned up. I've just made a commit da311b5 which addresses that, and I'll look through the remaining front end code for additional cleanup later this week as I believe there is still more cleanup.

Here is a quick explanation of the process now. The react component calls for asset/attribute/point/etc lookups still hit the datasource.restGet method, which is now adjusted to point to the endpoint of the datasource plugin. These requests hit backend's CallResource (pkg/plugin/datasource.go). I did add a filter to the CallResource requests to ensure that only a small subset of datasource applicable endpoints could be accessed which now works.

[coderReview]

Hello @complacentsee, have you used this https://github.com/grafana/grafana-plugin-examples/tree/main/examples/app-with-backend as the starting point of the backend plugin development?

[complacentsee]

I did use their example as a starting point, but used the datasource-http-backend instead of the app-with-backend (https://github.com/grafana/grafana-plugin-examples/tree/main/examples/datasource-http-backend)
The example was updated between my initial and and version 4 adaptation, so the annotation queries follow the example a bit closer with tracing added.

[coderReview]

@complacentsee are you still working on this? thank you.

[complacentsee]

It should be functionally complete compared to the 4.2 release. I haven't been actively working on it since I wasn't sure if there was interest in moving forward with this. If there is interest and you want any changes please let me know and I can clean up the remaining tasks. In terms of finishing this up for a release there are a few todo items to clean up:

• Clean up TODOs that are gaps between 4.2 release. Today this should only be bad value handling, and some potential optimizations.
• Remove unnecessary commented out debug printing information.
• Determine path forward for issue Calculation creates faulty query #138 Today my backend reflects the 4.2 release. However I believe dynamic intervals would make more sense.

[coderReview]

Initial support in #150