Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Provide Direct Access to EVO KPIs and Plots via Method Calls for Automated Pipelines #696

Open
pavan-opteran opened this issue Oct 16, 2024 · 0 comments

Comments

@pavan-opteran
Copy link

pavan-opteran commented Oct 16, 2024

Is your feature request related to a problem? Please describe.

In our automated verification pipeline, we currently rely on the EVO tool to evaluate pose estimation performance. However, the KPIs (e.g., Absolute Pose Error, Relative Pose Error) and plots are saved in a zip file, which we then have to manually read and extract. This process is not ideal for automated pipelines as it introduces unnecessary storage and file reading steps. We need a way to access the KPIs and plots directly via class method calls in our custom interface.

Describe the solution you'd like

We would like the EVO tool to offer direct access to KPIs and plot data through method calls, rather than storing the results in a zip file and requiring manual extraction. For example, the evo_ape and evo_rpe methods should return KPIs in a structured format (e.g., a dictionary or class object). Additionally, the plot data (e.g., figure objects or raw data for plots) should also be returned as part of the method call so that we can embed them in our verification reports without needing to read from files.

This could be implemented by adding optional parameters to the existing methods, such as return_kpis=True and return_plots=True, which would enable users to access the results programmatically. For eg in their scripts, they can pass in the arguments and retrieve the KPIs

Retrieve KPIs and error arrays via method calls

import evo.main_rpe as main_rpe

self.ape_data, self.ape_error_array = main_ape.run(args)
self.rpe_data, self.rpe_error_array = main_rpe.run(args)

Describe alternatives you've considered

Currently, we store the EVO results in a zip file and manually extract the KPIs and plots from there. While this works, it introduces unnecessary overhead, especially in a fully automated pipeline. Another alternative would be writing custom scripts to parse the output files, but this adds complexity and isn’t efficient for our real-time processing.

Additional context

This feature would be particularly useful in automated testing pipelines where efficiency and direct access to results are critical. By returning KPIs and plots via method calls, users can easily integrate the EVO tool’s results into larger verification workflows and generate reports dynamically without the need for intermediate file storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant