Uncaught Errors and Performance Inefficiencies When All Tests Are Filtered Out in Cypress

[yonasberhe23]

What would you like?

Description:
When using the @cypress/grep plugin to filter out tests based on tags, if no tests match the filter or all tests are excluded, Cypress still loads all spec files before applying the filter. This leads to unnecessary resource usage and potential uncaught errors during the setup phase (such as issues with base64 decoding or JSON parsing), even though no tests are executed.

Current Behavior:
Even when all tests are filtered out, Cypress still loads all spec files and executes the setup code, which can result in uncaught errors (e.g., JSON parsing or decoding failures). These errors cause the entire test suite to fail. Additionally, this behavior is inefficient, as it wastes system resources by loading and processing spec files that ultimately won’t be run.

Ideal Behavior:
If no tests match the filter criteria or if all tests are excluded, Cypress should skip loading the spec files entirely or, at the very least, avoid executing any setup code. This would prevent errors from impacting the test run and eliminate unnecessary resource consumption.

Please refer to this example code.
To repro the issue run yarn test to execute the script within package.json

[jennifer-shehane]

@yonasberhe23 We have to read the files to see the test definitions themselves. If there's a syntax error in the test file, I'm not sure we could do much with that. We have to read the files to see whether there are tests that need to run or not.

[yonasberhe23]

@jennifer-shehane Thanks for your response! I understand that Cypress needs to read the test files to determine which tests to run. However, the core issue here is that Cypress is still processing (and seemingly compiling) files that should be excluded based on grepFilterSpecs: true and grepOmitFiltered: true.

The syntax error is just a symptom of the underlying issue—Cypress is evaluating files it shouldn't be. Ideally, it should filter out the specs before attempting to process them, preventing unnecessary execution and errors.

Could you clarify if there's a way to ensure that only the relevant spec files are processed before Cypress attempts to compile them?