Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
CI/QA: start recording code coverage
This commit makes the necessary changes to start recording and uploading code coverage data to Coveralls.
Includes:
* Updating the GH Actions `test` workflow to run the high/low PHP version builds with code coverage and upload the results.
* Updating the PHPUnit config to record coverage and show an inline code coverage summary.
* Ignoring the potentially generated code coverage files in the `build/*` directory to prevent it being committed.
* Adding scripts to the `composer.json` file to run code coverage.
* Adding a code coverage badge to the README.
Notes:
* Coverage will be recorded using `Xdebug` as the code coverage driver.
* Recording code coverage can be slow and it is not needed to run it on each build to still get a good idea of the code coverage.
With that in mind, code coverage is only recorded for high/low PHP.
* The `php-coveralls/php-coveralls` package is used to convert the coverage information from a format as generated by PHPUnit to a format as can be consumed by Coveralls.
- As this package is only needed for the upload to Coveralls, the package has **not** been added it to the `composer.json` file, but will be (global) installed in the GH Actions workflow only.
- The php-coveralls/php-coveralls package is not 100% compatible with PHP 8.x (yet), so for uploading the coverage generated using PHP 8.x, we normally switch to PHP 7.4 for uploading the code coverage reports, but you can only switch PHP version once per job and as we already do this for building the container, we cannot do it again.
The chance of hitting the PHP 8.x problem is small anyway, so we'll just have to risk it for now.
* Coveralls requires a token to identify the repo and prevent unauthorized uploads.
This token has been added to the repository secrets.
People with admin access to the GH repo automatically also have access to the admin settings in Coveralls.
If ever needed, the Coveralls token can be found (and regenerated) in the Coveralls admin for a repo.
After regeneration, the token as stored in the GH repo Settings -> Secrets and Variables -> Actions -> Repository secrets should be updated.
* As the workflow sends multiple code coverage reports to Coveralls, we need to tell it to process those reports in parallel and give each report a unique name.
That's what the `COVERALLS_PARALLEL` and `COVERALLS_FLAG_NAME` settings are about.
This is also why the Clover files for the unit and the integration tests in the PHPUnit `logging` section have different names.
Additionally, it is why the name for the Clover file for the multi-site run in the integration tests is overruled in the workflow. This allows us to save the coverage results for the single site and the multi site test runs in separate files and to upload both files to Coveralls for a more complete code coverage picture.
* The `coveralls-finish` job will signal to Coveralls that all reports have been uploaded.
This basically gives the "okay" to Coveralls to report on the changes in code coverage via a comment on a GitHub PR as well as via a status check.
The pertinent Coveralls settings used for this repo are:
* "Only send aggregate Coverage updates to SCM": enabled
* "Leave comments": enabled
* "(Comment) Format": detailed
* "Use Status API": enabled
* "Coverage threshold for failure": 36%
* "Coverage decrease threshold for failure": 1.0%- Loading branch information
