Improvements/python linter harness

[thiagogenez]

Description

Refinement of the script involves encapsulating functionality within functions, refining the logic for determining success or failure based on pylint and mypy results, and enhancing the approach to returning results from these functions. The changes make the codebase more streamlined, modular, and maintainable.

Overview of changes

• Initially, the script directly executed commands without modular separation, making the logic less clear and harder to maintain.
• After, we introduced functions run_pylint and run_mypy to encapsulate respective functionalities, providing a structured, modular approach.

Change 1: Function Encapsulation

• Before: Direct execution of pylint and mypy without function encapsulation.
• After: Encapsulated pylint and mypy executions within run_pylint and run_mypy functions for better modularity and readability.

Change 2: Refined Exit Strategy

• Before: Exit codes were determined without explicitly covering all possible outcomes, potentially leading to ambiguity.
• After: Introduced a clear and explicit branching structure at the script's end to decide the exit code based on the pylint and mypy results.

Change 3: Declaring PYTHON_SOURCE_LOCATIONS Before Use

• Before: The PYTHON_SOURCE_LOCATIONS variable was declared but not strategically placed for optimal use.
• After: Moved the declaration of PYTHON_SOURCE_LOCATIONS above its first use, ensuring the variable is defined ahead of time and adhering to best practices for variable declaration and scope.

Change 4: Enhanced pylint Call with --msg-template

• Before: pylint was called without specifying a message template.
• After: Incorporated --msg-template='{path}:{line}:{column}: {msg_id}: {msg} ({symbol})' in the pylint call to format output messages explicitly, making subsequent grepping for errors more straightforward.

Change 5: Direct Grepping for Errors

• Before: Indirect error filtering using grep -v to exclude non-error messages and lines.
• After: Implemented direct grepping for error messages with grep -E, focusing on capturing lines that match the error message format. This approach is more efficient and reduces complexity by avoiding multiple filtering steps.

---

For code reviewers: code review SOP

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.48%. Comparing base (4ef7cea) to head (af29e9a).

Additional details and impacted files

@@               Coverage Diff               @@
##           release/111     #751      +/-   ##
===============================================
- Coverage        60.50%   60.48%   -0.02%     
===============================================
  Files              195      195              
  Lines            22309    22309              
  Branches          3601     3601              
===============================================
- Hits             13497    13494       -3     
- Misses            7609     7610       +1     
- Partials          1203     1205       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thiagogenez]

Checked files, skipped files were introduced in November 2023 and released 3 weeks ago in the version 3.1.0 at https://github.com/pylint-dev/pylint/releases/tag/v3.1.0

• PR: Print statistics how many files were checked in verbose pylint-dev/pylint#9122
• Commit: pylint-dev/pylint@2b17c39

[twalsh-ebi]

Thanks for this @thiagogenez, I do find it easier to see what's going on now that it's broken up into functions.

I've made a suggestion to tweak the run_mypy function with the hope of simplifying it further.