Merge pull request #129 from melexis/cq_descr

Add configurability to descriptions of Code Quality report entries

.github/workflows/python-package.yml

@@ -9,7 +9,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11']
+        python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
     steps:
     - uses: actions/checkout@v3
     - name: Set up Python ${{ matrix.python-version }}

README.rst

@@ -30,10 +30,6 @@
     :target: https://codeclimate.com/github/melexis/warnings-plugin
     :alt: Issue Count
 
-.. image:: https://requires.io/github/melexis/warnings-plugin/requirements.svg?branch=master
-    :target: https://requires.io/github/melexis/warnings-plugin/requirements/?branch=master
-    :alt: Requirements Status
-
 .. image:: https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat
     :target: https://github.com/melexis/warnings-plugin/issues
     :alt: Contributions welcome
@@ -109,17 +105,17 @@ Running Command
 ---------------
 
 There are few ways to run warnings plugin. If you are playing with Python on
-your computer you want to use `virtualenv`. This will separate your packages
+your computer you want to use ``virtualenv``. This will separate your packages
 per project and there is less chance of some dependency hell. You can
-initialize virtual environment in current directory by `virtualenv .` .
+initialize virtual environment in current directory by ``virtualenv .`` .
 
 Melexis Warnings plugin can be called directly from shell/console using:
 
 .. code-block:: bash
 
     mlx-warnings -h
 
-It has also possibility to be called as module from `Python3`. In
+It has also possibility to be called as module from ``python3``. In
 that case command will look like:
 
 .. code-block:: bash
@@ -191,15 +187,15 @@ Parse for Coverity Defects
 Coverity is a static analysis tool which has option to run desktop analysis
 on your local changes and report the results back directly in the console.
 You only need to list affected files and below example lists changed files
-between your branch and master, which it then forwards to `cov-run-desktop`:
+between your branch and master, which it then forwards to ``cov-run-desktop``:
 
 .. code-block:: bash
 
     cov-run-desktop --text-output-style=oneline `git diff --name-only --ignore-submodules master`
 
 
 You can pipe the results to logfile, which you pass to warnings-plugin, or you use
-the `--command` argument and execute the `cov-run-desktop` through
+the ``--command`` argument and execute the ``cov-run-desktop`` through
 
 .. code-block:: bash
 
@@ -265,7 +261,7 @@ with command:
 Parse for Robot Framework Test Failures
 ---------------------------------------
 
-When running `Robot Framework`_ tests with `--xunit report.xml`_ as an input
+When running `Robot Framework`_ tests with |--xunit report.xml|_ as an input
 argument, an xUnit compatible result file is generated. The warnings-plugin can
 parse this file and check the amount of failures. By default, the test results
 of all test suites in the file are taken into account. If you only care about
@@ -287,6 +283,7 @@ input file. When this setting is missing, the default value ``true`` is used.
     python3 -m mlx.warnings --robot --name "Suite Name" report.xml
 
 .. _`Robot Framework`: https://robotframework.org/
+.. |--xunit report.xml| replace:: ``--xunit report.xml``
 .. _`--xunit report.xml`: https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#xunit-compatible-result-file
 
 ----------------------------------
@@ -302,6 +299,7 @@ Configuration file is in JSON or YAML_ format with a simple structure.
         "sphinx": {
             "enabled": false,
             "cq_default_path": "doc/source/conf.py",
+            "cq_description_template": "$PRODUCT | $description",
             "min": 0,
             "max": 0
         },
@@ -345,11 +343,11 @@ Configuration file is in JSON or YAML_ format with a simple structure.
     }
 
 
-First key is `checkername`, then it contains a boolean value for key `enabled`,
-value for minimum number of warnings with key `min` and value for maximum
-number of warnings with key `max`. This structure allows simple expansion.
+First key is ``checkername``, then it contains a boolean value for key ``enabled``,
+value for minimum number of warnings with key ``min`` and value for maximum
+number of warnings with key ``max``. This structure allows simple expansion.
 
-To run the plugin with configuration file you simply pass `--config` flag with
+To run the plugin with configuration file you simply pass ``--config`` flag with
 path to configuration file
 
 .. code-block:: bash
@@ -375,7 +373,7 @@ Exclude Matches With Regexes
 In case you want a checker to exclude certain matches, you can configure
 one or more regular expressions in the configuration file on a per-checker basis.
 If a pattern of a regex to exclude is found in a match of the checker's regex, the checker
-won't count that match. Add the regex(es) as a list of string values for the `exclude` key.
+won't count that match. Add the regex(es) as a list of string values for the ``exclude`` key.
 An example configuration for the sphinx checker is given below:
 
 .. code-block:: json
@@ -395,10 +393,10 @@ An example configuration for the sphinx checker is given below:
 Exclude Sphinx Deprecation Warnings
 -----------------------------------
 
-There is a special flag `--exclude-sphinx-deprecation` that lets the sphinx checker exclude
+There is a special flag ``--exclude-sphinx-deprecation`` that lets the sphinx checker exclude
 Sphinx deprecation warnings. These warnings match the following regular expression:
-`RemovedInSphinx\\d+Warning`. Using this flag results in the same behavior as adding this
-regex to the configuration file as value for the `exclude` key for the sphinx checker.
+``RemovedInSphinx\\d+Warning``. Using this flag results in the same behavior as adding this
+regex to the configuration file as value for the ``exclude`` key for the sphinx checker.
 
 Store All Counted Warnings
 --------------------------
@@ -407,23 +405,20 @@ Use `-o, --output <file_path>` to let the plugin write all counted warnings/fail
 This can help you separate the warnings/failures that matter from those that are excluded or from irrelevant text that
 may exist in the input file (or produced by the given command).
 
-Example entries:
-
-    Sphinx:
-      /home/bljah/test/index.rst:5: WARNING: toctree contains reference to nonexisting document u'installation'
-
-    JUnit/RobotFramework:
-      test_warn_plugin_double_fail.myfirstfai1ure: Is our warnings plugin able to trace this random failure msg?
-
 Code Quality Report
 -------------------
 
-Use `-C, --code-quality` to let the plugin generate `a Code Quality report`_ for GitLab CI. All counted
+Use ``-C, --code-quality`` to let the plugin generate `a Code Quality report`_ for GitLab CI. All counted
 Sphinx, Doxygen and XMLRunner will be included. Other checker types are not supported by this feature. The report is
 a JSON file that implements `a subset of the Code Climate spec`_. Define this file `as a codequality report artifact`_
 of the CI job.
-If a warning doesn't contain a path, `"cq_default_path"` from the `configuration file to pass options`_ will be used.
-If not configured, `.gitlab-ci.yml` will be used as a fallback path.
+
+If a warning doesn't contain a path, ``"cq_default_path"`` from the `configuration file to pass options`_ will be used.
+If not configured, ``.gitlab-ci.yml`` will be used as a fallback path.
+
+You can customize the description with ``"cq_description_template"``, see `configuration file to pass options`_.
+Its value should be a template for Python's |string.Template|_. The template should contain ``$description`` and has
+access to all environment variables, e.g. ``$HOME``.
 
 =======================
 Issues and New Features
@@ -441,7 +436,9 @@ Contribute
 There is a Contribution guide available if you would like to get involved in
 development of the plugin. We encourage anyone to contribute to our repository.
 
+.. |string.Template| replace:: ``string.Template``
 .. _YAML: https://yaml.org/spec/1.2.2/
 .. _a Code Quality report: https://docs.gitlab.com/ee/ci/testing/code_quality.html
 .. _a subset of the Code Climate spec: https://docs.gitlab.com/ee/ci/testing/code_quality.html#implement-a-custom-tool
 .. _as a codequality report artifact: https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscodequality
+.. _string.Template: https://docs.python.org/3/library/string.html#string.Template.template

setup.py

@@ -53,6 +53,7 @@
         'Programming Language :: Python :: 3.9',
         'Programming Language :: Python :: 3.10',
         'Programming Language :: Python :: 3.11',
+        'Programming Language :: Python :: 3.12',
         # 'Programming Language :: Python :: Implementation :: CPython',
         # 'Programming Language :: Python :: Implementation :: PyPy',
         # uncomment if you test on these interpreters:

src/mlx/regex_checker.py

@@ -61,7 +61,7 @@ def add_code_quality_finding(self, match):
         groups = {name: result for name, result in match.groupdict().items() if result}
         for name, result in groups.items():
             if name.startswith("description"):
-                finding["description"] = result
+                finding["description"] = self.cq_description_template.substitute(description=result)
                 break
         else:
             return  # no description was found, which is the bare minimum

src/mlx/warnings_checker.py

@@ -1,5 +1,7 @@
 import abc
+import os
 import re
+from string import Template
 
 
 class WarningsChecker:
@@ -20,6 +22,7 @@ def __init__(self, verbose=False):
         self.cq_findings = []
         self.cq_enabled = False
         self.cq_default_path = '.gitlab-ci.yml'
+        self._cq_description_template = Template('$description')
         self.exclude_patterns = []
         self.include_patterns = []
 
@@ -28,6 +31,20 @@ def counted_warnings(self):
         ''' List: list of counted warnings (str) '''
         return self._counted_warnings
 
+    @property
+    def cq_description_template(self):
+        ''' Template: string.Template instance based on the configured template string '''
+        return self._cq_description_template
+
+    @cq_description_template.setter
+    def cq_description_template(self, template_obj):
+        try:
+            template_obj.template = template_obj.substitute(os.environ, description='$description')
+        except KeyError as err:
+            raise ValueError(f"Failed to find environment variable from configuration value "
+                             f"'cq_description_template': {err}") from err
+        self._cq_description_template = template_obj
+
     @abc.abstractmethod
     def check(self, content):
         ''' Function for counting the number of warnings in a specific text
@@ -154,6 +171,8 @@ def parse_config(self, config):
         self.add_patterns(config.get("exclude"), self.exclude_patterns)
         if 'cq_default_path' in config:
             self.cq_default_path = config['cq_default_path']
+        if 'cq_description_template' in config:
+            self.cq_description_template = Template(config['cq_description_template'])
 
     def _is_excluded(self, content):
         ''' Checks if the specific text must be excluded based on the configured regexes for exclusion and inclusion.

tests/test_in/code_quality_format.json

@@ -0,0 +1,79 @@
+[
+    {
+        "severity": "major",
+        "location": {
+            "path": "git/test/index.rst",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "envvar_value | toctree contains reference to nonexisting document u'installation' (12345)",
+        "fingerprint": "678c4f093876aed4280445a61e9556bc"
+    },
+    {
+        "severity": "major",
+        "location": {
+            "path": "doc/source/conf.py",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "envvar_value | List item 'CL-UNDEFINED_CL_ITEM' in merge/pull request 138 is not defined as a checklist-item. (12345)",
+        "fingerprint": "76989a10d5c1d2cd28080ea2aafa3c04"
+    },
+    {
+        "severity": "info",
+        "location": {
+            "path": "doc/doxygen/Doxyfile",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "Output directory `doc/doxygen/framework' does not exist. I have created it for you.",
+        "fingerprint": "256cb9d4b9b3c05ba755e5da30b2afb2"
+    },
+    {
+        "severity": "critical",
+        "location": {
+            "path": "helper/SimpleTimer.h",
+            "lines": {
+                "begin": 19
+            }
+        },
+        "description": "Unexpected character `\"'",
+        "fingerprint": "d71d9f49bd308ace916bce3c54f430c6"
+    },
+    {
+        "severity": "major",
+        "location": {
+            "path": "doc/doxygen/Doxyfile",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "The following parameters of sofa::component::odesolver::EulerKaapiSolver::v_peq(VecId v, VecId a, double f) are not documented:",
+        "fingerprint": "9c91370059e40eb07289790f2b9d6a75"
+    },
+    {
+        "severity": "critical",
+        "location": {
+            "path": "doc/doxygen/Doxyfile",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "Could not read image `/home/user/myproject/html/struct_foo_graph.png' generated by dot!",
+        "fingerprint": "bdd0d0b7cc25b1f4414f630c2351bf3c"
+    },
+    {
+        "severity": "critical",
+        "location": {
+            "path": ".gitlab-ci.yml",
+            "lines": {
+                "begin": 1
+            }
+        },
+        "description": "test_some_error_test (something.anything.somewhere)'",
+        "fingerprint": "cd09ae46ee5361570fd59de78b454e11"
+    }
+]

tests/test_in/config_cq_description_format.json

@@ -0,0 +1,31 @@
+{
+    "sphinx": {
+        "enabled": true,
+        "cq_default_path": "doc/source/conf.py",
+        "cq_description_template": "$FIRST_ENVVAR | $description ($SECOND_ENVVAR)",
+        "min": 0,
+        "max": 0
+    },
+    "doxygen": {
+        "enabled": true,
+        "cq_default_path": "doc/doxygen/Doxyfile",
+        "min": 0,
+        "max": 0
+    },
+    "junit": {
+        "enabled": false
+    },
+    "xmlrunner": {
+        "enabled": true,
+        "min": 0,
+        "max": 0
+    },
+    "coverity": {
+        "enabled": true,
+        "min": 0,
+        "max": 0
+    },
+    "robot": {
+        "enabled": false
+    }
+}

tests/test_integration.py

@@ -1,4 +1,5 @@
 import filecmp
+import os
 from io import StringIO
 from pathlib import Path
 from unittest import TestCase
@@ -16,6 +17,11 @@ def setUp(self):
         if not TEST_OUT_DIR.exists():
             TEST_OUT_DIR.mkdir()
 
+    def tearDown(self):
+        for var in ('FIRST_ENVVAR', 'SECOND_ENVVAR'):
+            if var in os.environ:
+                del os.environ[var]
+
     def test_help(self):
         with self.assertRaises(SystemExit) as ex:
             warnings_wrapper(['--help'])
@@ -309,3 +315,33 @@ def test_code_quality(self, path_cwd_mock):
         ])
         self.assertEqual(2, retval)
         self.assertTrue(filecmp.cmp(out_file, ref_file), '{} differs from {}'.format(out_file, ref_file))
+
+    def test_cq_description_format_missing_envvar(self):
+        os.environ['FIRST_ENVVAR'] = 'envvar_value'
+        filename = 'code_quality_format.json'
+        out_file = str(TEST_OUT_DIR / filename)
+        with self.assertRaises(ValueError) as c_m:
+            warnings_wrapper([
+                '--code-quality', out_file,
+                '--config', 'tests/test_in/config_cq_description_format.json',
+                'tests/test_in/mixed_warnings.txt',
+            ])
+        self.assertEqual(
+            str(c_m.exception),
+            "Failed to find environment variable from configuration value 'cq_description_template': 'SECOND_ENVVAR'")
+
+    @patch('pathlib.Path.cwd')
+    def test_cq_description_format(self, path_cwd_mock):
+        os.environ['FIRST_ENVVAR'] = 'envvar_value'
+        os.environ['SECOND_ENVVAR'] = '12345'
+        path_cwd_mock.return_value = '/home/user/myproject'
+        filename = 'code_quality_format.json'
+        out_file = str(TEST_OUT_DIR / filename)
+        ref_file = str(TEST_IN_DIR / filename)
+        retval = warnings_wrapper([
+            '--code-quality', out_file,
+            '--config', 'tests/test_in/config_cq_description_format.json',
+            'tests/test_in/mixed_warnings.txt',
+        ])
+        self.assertEqual(2, retval)
+        self.assertTrue(filecmp.cmp(out_file, ref_file), '{} differs from {}'.format(out_file, ref_file))