Investigate data in report_test_coverage_by_project_prod

[isabelrios]

The graphs for report_test_coverage_by_project_prod view show that there are untriaged tests for Fenix.
Andi raised the flag since running the report on Test Rail there are no untriaged tests for Full Functional or Smoke test suites.

This is the report in Test Rail:
testrail-report-5779.pdf

This is the data from the Data Base:
https://docs.google.com/spreadsheets/d/15sWdruOkRIo9Anhcbk7dvSXkki24XMaZWXeNEMQ37DU/edit?usp=sharing

In yellow I have highlighted the same data we see on Test Rail, but there is more for the same day Smoke test and untriaged, highlighted in green, and those are being sum up, given the incorrect numbers.
We need to understand why those in green appear and why they are added.

[AndiAJ]

Hi @isabelrios ,

As discussed on Slack there should not be any "Untriaged" test in our Smoke and Sanity test suite.

• Both the test rail report and the numbers (highlighted in 🟠 ) retrieved from the data base are correct. ✅

I've made a quick check and I think I've found root cause because of which there are extra values (highlighted in 🟢 ) retrieved from the data base in regards with our smoke test suite.

• Those numbers look like they are from our "Features" test suite. Test Rail report ❌

Please share your thoughts.

[isabelrios]

yeah, that makes sense to understand why the numbers are wrong in looker.
In order to fix it, not sure if we can remove the smoke from those tests, or if those are actually part of the smoke test if they should be part of the full functional... not sure about the solution yet...
Please if you don't mind share your ideas as you are more familiar with the tests :)

[isabelrios]

As a possible solution to prevent this error from happening in the future, we could add the testrail_test_suites_id for each project, that's going to identify the Full Functional test suite which is the one we want specific data for now. Unless the test suite is removed and recreated, that ID is not going to change. If that happens, we should know as that's a big change.
Wdyt @AndiAJ @rpappalax ?