Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add autofixing of lint warnings for specific packages/files #5187

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
338 changes: 221 additions & 117 deletions scripts/run-eslint.ts
Original file line number Diff line number Diff line change
@@ -11,6 +11,29 @@ const WARNING_THRESHOLDS_FILE = path.join(
'eslint-warning-thresholds.json',
);

/**
* The parsed command-line arguments.
*/
type CommandLineArguments = {
/**
* Whether to cache results to speed up future runs (true) or not (false).
*/
cache: boolean;
/**
* A list of specific files to lint.
*/
files: string[];
/**
* Whether to automatically fix lint errors (true) or not (false).
*/
fix: boolean;
/**
* Whether to only report errors, disabling the warnings quality gate in the
* process (true) or not (false).
*/
quiet: boolean;
};

/**
* A two-level object mapping path to files in which warnings appear to the IDs
* of rules for those warnings, then from rule IDs to the number of warnings for
@@ -49,9 +72,36 @@ type WarningComparison = {
};

/**
* The warning severity of level of an ESLint rule.
* The severity level for an ESLint message.
*/
enum ESLintMessageSeverity {
Warning = 1,
// This isn't a variable.
// eslint-disable-next-line @typescript-eslint/no-shadow
Error = 2,
}

/**
* The result of applying the quality gate.
*/
const WARNING = 1;
enum QualityGateStatus {
/**
* The number of lint warnings increased.
*/
Increase = 'increase',
/**
* The number of lint warnings decreased.
*/
Decrease = 'decrease',
/**
* There was no change to the number of lint warnings.
*/
NoChange = 'no-change',
/**
* The warning thresholds file did not previously exist.
*/
Initialized = 'initialized',
}

// Run the script.
main().catch((error) => {
@@ -63,14 +113,37 @@ main().catch((error) => {
* The entrypoint to this script.
*/
async function main() {
const { cache, fix, quiet } = parseCommandLineArguments();
const { cache, fix, files: givenFiles, quiet } = parseCommandLineArguments();

const eslint = new ESLint({
cache,
errorOnUnmatchedPattern: false,
fix,
ruleFilter: ({ severity }) =>
!quiet || severity === ESLintMessageSeverity.Error,
});

const fileFilteredResults = await eslint.lintFiles(
givenFiles.length > 0 ? givenFiles : ['.'],
);

const filteredResults = quiet
? ESLint.getErrorResults(fileFilteredResults)
: fileFilteredResults;

await printResults(eslint, filteredResults);

const eslint = new ESLint({ cache, fix });
const results = await runESLint(eslint, { fix, quiet });
const hasErrors = results.some((result) => result.errorCount > 0);
if (fix) {
await ESLint.outputFixes(fileFilteredResults);
}
const hasErrors = filteredResults.some((result) => result.errorCount > 0);

const qualityGateStatus = applyWarningThresholdsQualityGate({
results: filteredResults,
});

if (!quiet && !hasErrors) {
evaluateWarnings(results);
if (hasErrors || qualityGateStatus === QualityGateStatus.Increase) {
process.exitCode = 1;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled this out of runESLint and what is now applyWarningThresholdsQualityGate because otherwise it was hard to see.

}
}

@@ -79,61 +152,48 @@ async function main() {
*
* @returns The parsed arguments.
*/
function parseCommandLineArguments() {
return yargs(process.argv.slice(2))
function parseCommandLineArguments(): CommandLineArguments {
const { cache, fix, quiet, ...rest } = yargs(process.argv.slice(2))
.option('cache', {
type: 'boolean',
description: 'Cache results to speed up future runs',
default: false,
})
.option('fix', {
type: 'boolean',
description: 'Automatically fix problems',
description:
'Automatically fix all problems; pair with --quiet to only fix errors',
default: false,
})
.option('quiet', {
type: 'boolean',
description:
'Only report errors, disabling the warnings quality gate in the process',
description: 'Only report or fix errors',
default: false,
})
.help().argv;
.help()
.string('_').argv;

// Type assertion: The types for `yargs`'s `string` method are wrong.
const files = rest._ as string[];

return { cache, fix, quiet, files };
}

/**
* Runs ESLint on the project files.
* Uses the given results to print the output that `eslint` usually generates.
*
* @param eslint - The ESLint instance.
* @param options - The options for running ESLint.
* @param options.quiet - Whether to only report errors (true) or not (false).
* @param options.fix - Whether to automatically fix problems (true) or not
* (false).
* @returns A promise that resolves to the lint results.
* @param results - The results from running `eslint`.
*/
async function runESLint(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff is a bit rough here. Essentially I folded the contents of the runESLint function back into main and then extracted printResults. I felt this slight refactor was necessary because it was easier to see where we are using the output from lintFiles (a list of files and lint rules that were checked, which have been filtered to the given set of files) and where we are using a version of that list that is further filtered to just include errors.

async function printResults(
eslint: ESLint,
options: { quiet: boolean; fix: boolean },
): Promise<ESLint.LintResult[]> {
let results = await eslint.lintFiles(['.']);
const errorResults = ESLint.getErrorResults(results);

if (errorResults.length > 0) {
process.exitCode = 1;
}

if (options.quiet) {
results = errorResults;
}

results: ESLint.LintResult[],
): Promise<void> {
const formatter = await eslint.loadFormatter('stylish');
const resultText = formatter.format(results);
console.log(resultText);

if (options.fix) {
await ESLint.outputFixes(results);
const resultText = await formatter.format(results);
if (resultText.length > 0) {
console.log(resultText);
}

return results;
}

/**
@@ -148,99 +208,133 @@ async function runESLint(
* had increases and decreases. If are were more warnings overall then we fail,
* otherwise we pass.
*
* @param results - The results of running ESLint.
* @param args - The arguments.
* @param args.results - The results from running `eslint`.
* param args.countFileEvenIfNoWarnings - Includes a file in the returned
* object even if there are no recorded warnings for it.
* param args.fileUpdateStrategy - How to update the warning thresholds file.
* @returns True if the number of warnings has increased compared to the
* existing number of warnings, false if they have decreased or stayed the same.
*/
function evaluateWarnings(results: ESLint.LintResult[]) {
function applyWarningThresholdsQualityGate({
results,
}: {
results: ESLint.LintResult[];
}): QualityGateStatus {
const warningThresholds = loadWarningThresholds();
const warningCounts = getWarningCounts(results);
const warningCounts = getWarningCounts({
results,
});

const completeWarningCounts = removeFilesWithoutWarnings({
Copy link
Contributor Author

@mcmire mcmire Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key change to this function. If we are checking a selection of files and not the whole project then the set of lint warning counts we've tabulated will be a subset of what is already inside of the warning thresholds file, so we need to merge in what we have rather than completely overwrite the file. We also clean up the file by removing files that actually have no lint warnings whatsoever (see getWarningCounts below for why we now need to do this).

...warningThresholds,
...warningCounts,
});

if (Object.keys(warningThresholds).length === 0) {
console.log(
chalk.blue(
'The following lint violations were produced and will be captured as thresholds for future runs:\n',
),
);
for (const [filePath, ruleCounts] of Object.entries(warningCounts)) {

for (const [filePath, ruleCounts] of Object.entries(
completeWarningCounts,
)) {
console.log(chalk.underline(filePath));
for (const [ruleId, count] of Object.entries(ruleCounts)) {
console.log(` ${chalk.cyan(ruleId)}: ${count}`);
}
}
saveWarningThresholds(warningCounts);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now return a status from this function so we can simplify the nesting here.

} else {
const comparisonsByFile = compareWarnings(warningThresholds, warningCounts);

const changes = Object.values(comparisonsByFile)
.flat()
.filter((comparison) => comparison.difference !== 0);
const regressions = Object.values(comparisonsByFile)
.flat()
.filter((comparison) => comparison.difference > 0);

if (changes.length > 0) {
if (regressions.length > 0) {
console.log(
chalk.red(
'🛑 New lint violations have been introduced and need to be resolved for linting to pass:\n',
),
);

for (const [filePath, fileChanges] of Object.entries(
comparisonsByFile,
)) {
if (fileChanges.some((fileChange) => fileChange.difference > 0)) {
console.log(chalk.underline(filePath));
for (const {
ruleId,
threshold,
count,
difference,
} of fileChanges) {
if (difference > 0) {
console.log(
` ${chalk.cyan(ruleId)}: ${threshold} -> ${count} (${difference > 0 ? chalk.red(`+${difference}`) : chalk.green(difference)})`,
);
}
}
}
}

process.exitCode = 1;
} else {
console.log(
chalk.green(
'The overall number of ESLint warnings has decreased, good work! ❤️ \n',
),
);

for (const [filePath, fileChanges] of Object.entries(
comparisonsByFile,
)) {
if (fileChanges.some((fileChange) => fileChange.difference !== 0)) {
console.log(chalk.underline(filePath));
for (const {
ruleId,
threshold,
count,
difference,
} of fileChanges) {
if (difference !== 0) {
console.log(
` ${chalk.cyan(ruleId)}: ${threshold} -> ${count} (${difference > 0 ? chalk.red(`+${difference}`) : chalk.green(difference)})`,
);
}
saveWarningThresholds(completeWarningCounts);

return QualityGateStatus.Initialized;
}

const comparisonsByFile = compareWarnings(
warningThresholds,
completeWarningCounts,
);

const changes = Object.values(comparisonsByFile)
.flat()
.filter((comparison) => comparison.difference !== 0);
const regressions = Object.values(comparisonsByFile)
.flat()
.filter((comparison) => comparison.difference > 0);

if (changes.length > 0) {
if (regressions.length > 0) {
console.log(
chalk.red(
'🛑 New lint violations have been introduced and need to be resolved for linting to pass:\n',
),
);

for (const [filePath, fileChanges] of Object.entries(comparisonsByFile)) {
if (fileChanges.some((fileChange) => fileChange.difference > 0)) {
console.log(chalk.underline(filePath));
for (const { ruleId, threshold, count, difference } of fileChanges) {
if (difference > 0) {
console.log(
` ${chalk.cyan(ruleId)}: ${threshold} -> ${count} (${difference > 0 ? chalk.red(`+${difference}`) : chalk.green(difference)})`,
);
}
}
}
}

console.log(
`\n${chalk.yellow.bold(path.basename(WARNING_THRESHOLDS_FILE))}${chalk.yellow(' has been updated with the new counts. Please make sure to commit the changes.')}`,
);
return QualityGateStatus.Increase;
}

console.log(
chalk.green(
'The overall number of lint warnings has decreased, good work! ❤️ \n',
),
);

saveWarningThresholds(warningCounts);
for (const [filePath, fileChanges] of Object.entries(comparisonsByFile)) {
if (fileChanges.some((fileChange) => fileChange.difference !== 0)) {
console.log(chalk.underline(filePath));
for (const { ruleId, threshold, count, difference } of fileChanges) {
if (difference !== 0) {
console.log(
` ${chalk.cyan(ruleId)}: ${threshold} -> ${count} (${difference > 0 ? chalk.red(`+${difference}`) : chalk.green(difference)})`,
);
}
}
}
}

console.log(
`\n${chalk.yellow.bold(path.basename(WARNING_THRESHOLDS_FILE))}${chalk.yellow(' has been updated with the new counts. Please make sure to commit the changes.')}`,
);

saveWarningThresholds(completeWarningCounts);

return QualityGateStatus.Decrease;
}

return QualityGateStatus.NoChange;
}

/**
* Removes properties from the given warning counts object that have no warnings.
*
* @param warningCounts - The warning counts.
* @returns The transformed warning counts.
*/
function removeFilesWithoutWarnings(warningCounts: WarningCounts) {
return Object.entries(warningCounts).reduce(
(newWarningCounts: WarningCounts, [filePath, warnings]) => {
if (Object.keys(warnings).length === 0) {
return newWarningCounts;
}
return { ...newWarningCounts, [filePath]: warnings };
},
{},
);
}

/**
@@ -274,21 +368,31 @@ function saveWarningThresholds(newWarningCounts: WarningCounts): void {
* Given a list of results from an the ESLint run, counts the number of warnings
* produced per file and rule.
*
* @param results - The ESLint results.
* @param args - The arguments.
* @param args.results - The results from running `eslint`.
* param args.countFileEvenIfNoWarnings - Includes a file in the returned
* object even if there are no recorded warnings for it.
* @returns A two-level object mapping path to files in which warnings appear to
* the IDs of rules for those warnings, then from rule IDs to the number of
* warnings for the rule.
*/
function getWarningCounts(results: ESLint.LintResult[]): WarningCounts {
function getWarningCounts({
results,
}: {
results: ESLint.LintResult[];
}): WarningCounts {
const unsortedWarningCounts = results.reduce(
(workingWarningCounts, result) => {
const { filePath } = result;
const relativeFilePath = path.relative(PROJECT_DIRECTORY, filePath);
if (!workingWarningCounts[relativeFilePath]) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is necessary now that we are always merging what this function returns with what already exists in the warning thresholds file (see above).

If all of the lint violations that are warnings are fixed within a file, we want to include that file path in the object that this function returns because otherwise the counts won't get reset for that file in the warning thresholds file and our script will still think that there are warnings.

workingWarningCounts[relativeFilePath] = {};
}
for (const message of result.messages) {
if (message.severity === WARNING && message.ruleId) {
if (!workingWarningCounts[relativeFilePath]) {
workingWarningCounts[relativeFilePath] = {};
}
if (
message.severity === ESLintMessageSeverity.Warning &&
message.ruleId
) {
workingWarningCounts[relativeFilePath][message.ruleId] =
(workingWarningCounts[relativeFilePath][message.ruleId] ?? 0) + 1;
}

Unchanged files with check annotations Beta

import { GasPricesController } from '@metamask/example-controllers';
import type { GasPricesControllerMessenger } from '@metamask/example-controllers';
import type {

Check warning on line 5 in examples/example-controllers/src/gas-prices-controller.test.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

`../../../packages/base-controller/tests/helpers` type import should occur after import of `./network-controller-types`
ExtractAvailableAction,
ExtractAvailableEvent,
} from '../../../packages/base-controller/tests/helpers';
/**
* The service object that is used to obtain gas prices.
*/
#gasPricesService: AbstractGasPricesService;

Check warning on line 196 in examples/example-controllers/src/gas-prices-controller.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

Member '#gasPricesService' is never reassigned; mark it as `readonly`
/**
* Constructs a new {@link GasPricesController}.
*/
async updateGasPrices() {
const { chainId } = this.messagingSystem.call('NetworkController:getState');
const gasPricesResponse = await this.#gasPricesService.fetchGasPrices(

Check warning on line 241 in examples/example-controllers/src/gas-prices-controller.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

Replace `·await·this.#gasPricesService.fetchGasPrices(⏎······chainId,⏎····` with `⏎······await·this.#gasPricesService.fetchGasPrices(chainId`
chainId,
);
this.update((state) => {
* ```
*/
export class GasPricesService {
#fetch: typeof fetch;

Check warning on line 41 in examples/example-controllers/src/gas-prices-service/gas-prices-service.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

Member '#fetch' is never reassigned; mark it as `readonly`
/**
* Constructs a new GasPricesService object.
this.#fetch = fetchFunction;
}
/**

Check warning on line 56 in examples/example-controllers/src/gas-prices-service/gas-prices-service.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

Missing JSDoc @returns declaration
* Makes a request to the API in order to retrieve gas prices for a particular
* chain.
*
ExtractAvailableEvent,
} from '../../../packages/base-controller/tests/helpers';
import { PROTOTYPE_POLLUTION_BLOCKLIST } from '../../../packages/controller-utils/src/util';
import type { PetNamesControllerMessenger } from './pet-names-controller';

Check warning on line 8 in examples/example-controllers/src/pet-names-controller.test.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

`./pet-names-controller` type import should occur before type import of `../../../packages/base-controller/tests/helpers`
import { PetNamesController } from './pet-names-controller';

Check warning on line 9 in examples/example-controllers/src/pet-names-controller.test.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

`./pet-names-controller` import should occur before type import of `../../../packages/base-controller/tests/helpers`
describe('PetNamesController', () => {
describe('constructor', () => {
import type { SnapControllerState } from '@metamask/snaps-controllers';
import { SnapStatus } from '@metamask/snaps-utils';
import type { CaipChainId } from '@metamask/utils';
import * as uuid from 'uuid';

Check warning on line 25 in packages/accounts-controller/src/AccountsController.test.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

No exported names found in module 'uuid'
import type { V4Options } from 'uuid';
import type {
import type { Hex } from '@metamask/utils';
/**
* @type ContactEntry

Check warning on line 17 in packages/address-book-controller/src/AddressBookController.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

'@type' is redundant when using a type system
*
* ContactEntry representation
* @property address - Hex address of a recipient account

Check warning on line 20 in packages/address-book-controller/src/AddressBookController.ts

GitHub Actions / Lint, build, and test / Lint (20.x)

'@Property' is redundant when using a type system
* @property name - Nickname associated with this address
* @property importTime - Data time when an account as created/imported
*/