fix: ensure obsolete tests are not run on 4.0

[wallyworld]

Juju 4.0 is failing ci test runs on jenkins because said functionality is not supported in 4.0. We need a way to mask out such tests. In some cases, the 4.0 branch just leaves stubs behind, so the tests pass, in other cases, the tests have already been deleted so fail. The stubbed out tests in 4.0 were incorrectly not generated any more for 3.6.

This branch changes how the test suite metadata is composed - it now queries githib directly. This is done for the "main" and "3.6" branches, thereby allowing tests which no longer exist in 4.0 to be identified. The conditional regexp applied to each test job is adjusted to allow for tests to be excluded from a certain version onwards if they are marked as removed. The tool config also supports manually specify test removal; this is used where tests have not been deleted from the main branch yet, or to account for individual tests within a suite that have been removed.

There's a bit of churn in the existing conditional yaml to keep the template manageable for the different conditions.

When specifying whole suites in "removed" or "introduced" sections, there's no need for "-*" as it can be implied from the suite name.

Since the github queries can take a minute or so, a progress bar is added.

Note also that some tests from 3.6 were incorrectly removed from jenkins - branches and metrics. This PR allows those to be added back in.

JIRA card: JIRA_7126

[hpidcock]

I think I'm happy with this.