Skip to content

Commit

Permalink
Add shell script support via shfmt (#1994)
Browse files Browse the repository at this point in the history
  • Loading branch information
nedtwigg authored Jan 15, 2024
2 parents 350cf2e + 886048d commit e2e42e4
Show file tree
Hide file tree
Showing 13 changed files with 372 additions and 6 deletions.
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (
## [Unreleased]
### Added
* New static method to `DiffMessageFormatter` which allows to retrieve diffs with their line numbers ([#1960](https://github.com/diffplug/spotless/issues/1960))
* Format shell via [shfmt](https://github.com/mvdan/sh). ([#1994](https://github.com/diffplug/spotless/pull/1994))
### Fixed
* Fix empty files with biome >= 1.5.0 when formatting files that are in the ignore list of the biome configuration file. ([#1989](https://github.com/diffplug/spotless/pull/1989) fixes [#1987](https://github.com/diffplug/spotless/issues/1987))
* Fix a regression in BufStep where the same arguments were being provided to every `buf` invocation. ([#1976](https://github.com/diffplug/spotless/issues/1976))
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![Maven Plugin](https://img.shields.io/maven-central/v/com.diffplug.spotless/spotless-maven-plugin?color=blue&label=maven%20plugin)](plugin-maven)
[![SBT Plugin](https://img.shields.io/badge/sbt%20plugin-0.1.3-blue)](https://github.com/moznion/sbt-spotless)

Spotless can format <antlr | c | c# | c++ | css | flow | graphql | groovy | html | java | javascript | json | jsx | kotlin | less | license headers | markdown | objective-c | protobuf | python | scala | scss | sql | typeScript | vue | yaml | anything> using <gradle | maven | sbt | anything>.
Spotless can format <antlr | c | c# | c++ | css | flow | graphql | groovy | html | java | javascript | json | jsx | kotlin | less | license headers | markdown | objective-c | protobuf | python | scala | scss | shell | sql | typeScript | vue | yaml | anything> using <gradle | maven | sbt | anything>.

You probably want one of the links below:

Expand Down Expand Up @@ -75,6 +75,7 @@ lib('generic.ReplaceRegexStep') +'{{yes}} | {{yes}}
lib('generic.ReplaceStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('generic.TrimTrailingWhitespaceStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('antlr4.Antlr4FormatterStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('biome.BiomeStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('cpp.ClangFormatStep') +'{{yes}} | {{no}} | {{no}} | {{no}} |',
extra('cpp.EclipseFormatterStep') +'{{yes}} | {{yes}} | {{yes}} | {{no}} |',
lib('gherkin.GherkinUtilsStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
Expand All @@ -101,8 +102,8 @@ lib('npm.TsFmtFormatterStep') +'{{yes}} | {{yes}}
lib('pom.SortPomStepStep') +'{{no}} | {{yes}} | {{no}} | {{no}} |',
lib('protobuf.BufStep') +'{{yes}} | {{no}} | {{no}} | {{no}} |',
lib('python.BlackStep') +'{{yes}} | {{no}} | {{no}} | {{no}} |',
lib('biome.BiomeStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('scala.ScalaFmtStep') +'{{yes}} | {{yes}} | {{yes}} | {{no}} |',
lib('shell.ShfmtStep') +'{{yes}} | {{no}} | {{no}} | {{no}} |',
lib('sql.DBeaverSQLFormatterStep') +'{{yes}} | {{yes}} | {{yes}} | {{no}} |',
extra('wtp.EclipseWtpFormatterStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
lib('yaml.JacksonYamlStep') +'{{yes}} | {{yes}} | {{no}} | {{no}} |',
Expand All @@ -127,6 +128,7 @@ lib('yaml.JacksonYamlStep') +'{{yes}} | {{yes}}
| [`generic.ReplaceStep`](lib/src/main/java/com/diffplug/spotless/generic/ReplaceStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`generic.TrimTrailingWhitespaceStep`](lib/src/main/java/com/diffplug/spotless/generic/TrimTrailingWhitespaceStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`antlr4.Antlr4FormatterStep`](lib/src/main/java/com/diffplug/spotless/antlr4/Antlr4FormatterStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`biome.BiomeStep`](lib/src/main/java/com/diffplug/spotless/biome/BiomeStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`cpp.ClangFormatStep`](lib/src/main/java/com/diffplug/spotless/cpp/ClangFormatStep.java) | :+1: | :white_large_square: | :white_large_square: | :white_large_square: |
| [`cpp.EclipseFormatterStep`](lib-extra/src/main/java/com/diffplug/spotless/extra/cpp/EclipseFormatterStep.java) | :+1: | :+1: | :+1: | :white_large_square: |
| [`gherkin.GherkinUtilsStep`](lib/src/main/java/com/diffplug/spotless/gherkin/GherkinUtilsStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
Expand All @@ -153,8 +155,8 @@ lib('yaml.JacksonYamlStep') +'{{yes}} | {{yes}}
| [`pom.SortPomStepStep`](lib/src/main/java/com/diffplug/spotless/pom/SortPomStepStep.java) | :white_large_square: | :+1: | :white_large_square: | :white_large_square: |
| [`protobuf.BufStep`](lib/src/main/java/com/diffplug/spotless/protobuf/BufStep.java) | :+1: | :white_large_square: | :white_large_square: | :white_large_square: |
| [`python.BlackStep`](lib/src/main/java/com/diffplug/spotless/python/BlackStep.java) | :+1: | :white_large_square: | :white_large_square: | :white_large_square: |
| [`biome.BiomeStep`](lib/src/main/java/com/diffplug/spotless/biome/BiomeStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`scala.ScalaFmtStep`](lib/src/main/java/com/diffplug/spotless/scala/ScalaFmtStep.java) | :+1: | :+1: | :+1: | :white_large_square: |
| [`shell.ShfmtStep`](lib/src/main/java/com/diffplug/spotless/shell/ShfmtStep.java) | :+1: | :white_large_square: | :white_large_square: | :white_large_square: |
| [`sql.DBeaverSQLFormatterStep`](lib/src/main/java/com/diffplug/spotless/sql/DBeaverSQLFormatterStep.java) | :+1: | :+1: | :+1: | :white_large_square: |
| [`wtp.EclipseWtpFormatterStep`](lib-extra/src/main/java/com/diffplug/spotless/extra/wtp/EclipseWtpFormatterStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
| [`yaml.JacksonYamlStep`](lib/src/main/java/com/diffplug/spotless/yaml/JacksonYamlStep.java) | :+1: | :+1: | :white_large_square: | :white_large_square: |
Expand Down
5 changes: 3 additions & 2 deletions gradle/special-tests.gradle
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
apply plugin: 'com.adarshr.test-logger'
def special = [
'Npm',
'Black',
'Buf',
'Clang',
'Buf'
'Npm',
'Shfmt'
]

boolean isCiServer = System.getenv().containsKey("CI")
Expand Down
110 changes: 110 additions & 0 deletions lib/src/main/java/com/diffplug/spotless/shell/ShfmtStep.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
/*
* Copyright 2024 DiffPlug
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.diffplug.spotless.shell;

import java.io.File;
import java.io.IOException;
import java.io.Serializable;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Objects;
import java.util.regex.Pattern;

import javax.annotation.Nullable;

import com.diffplug.spotless.ForeignExe;
import com.diffplug.spotless.FormatterFunc;
import com.diffplug.spotless.FormatterStep;
import com.diffplug.spotless.ProcessRunner;

import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;

public class ShfmtStep {
public static String name() {
return "shfmt";
}

public static String defaultVersion() {
return "3.7.0";
}

private final String version;
private final @Nullable String pathToExe;

private ShfmtStep(String version, @Nullable String pathToExe) {
this.version = version;
this.pathToExe = pathToExe;
}

public static ShfmtStep withVersion(String version) {
return new ShfmtStep(version, null);
}

public ShfmtStep withPathToExe(String pathToExe) {
return new ShfmtStep(version, pathToExe);
}

public FormatterStep create() {
return FormatterStep.createLazy(name(), this::createState, State::toFunc);
}

private State createState() throws IOException, InterruptedException {
String howToInstall = "" +
"You can download shfmt from https://github.com/mvdan/sh and " +
"then point Spotless to it with {@code pathToExe('/path/to/shfmt')} " +
"or you can use your platform's package manager:" +
"\n win: choco install shfmt" +
"\n mac: brew install shfmt" +
"\n linux: apt install shfmt" +
"\n github issue to handle this better: https://github.com/diffplug/spotless/issues/673";
final ForeignExe exe = ForeignExe.nameAndVersion("shfmt", version)
.pathToExe(pathToExe)
.versionRegex(Pattern.compile("(\\S*)"))
.fixCantFind(howToInstall)
.fixWrongVersion(
"You can tell Spotless to use the version you already have with {@code shfmt('{versionFound}')}" +
"or you can download the currently specified version, {version}.\n" + howToInstall);
return new State(this, exe);
}

@SuppressFBWarnings("SE_TRANSIENT_FIELD_NOT_RESTORED")
static class State implements Serializable {
private static final long serialVersionUID = -1825662356883926318L;
// used for up-to-date checks and caching
final String version;
final transient ForeignExe exe;
// used for executing
private transient @Nullable List<String> args;

State(ShfmtStep step, ForeignExe pathToExe) {
this.version = step.version;
this.exe = Objects.requireNonNull(pathToExe);
}

String format(ProcessRunner runner, String input, File file) throws IOException, InterruptedException {
if (args == null) {
args = List.of(exe.confirmVersionAndGetAbsolutePath(), "-i", "2", "-ci");
}

return runner.exec(input.getBytes(StandardCharsets.UTF_8), args).assertExitZero(StandardCharsets.UTF_8);
}

FormatterFunc.Closeable toFunc() {
ProcessRunner runner = new ProcessRunner();
return FormatterFunc.Closeable.of(runner, this::format);
}
}
}
2 changes: 2 additions & 0 deletions plugin-gradle/CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (starting after version `3.27.0`).

## [Unreleased]
### Added
* Support for shell via [shfmt](https://github.com/mvdan/sh).
### Fixed
* Fix empty files with biome >= 1.5.0 when formatting files that are in the ignore list of the biome configuration file. ([#1989](https://github.com/diffplug/spotless/pull/1989) fixes [#1987](https://github.com/diffplug/spotless/issues/1987))=======
* Fix a regression in BufStep where the same arguments were being provided to every `buf` invocation. ([#1976](https://github.com/diffplug/spotless/issues/1976))
Expand Down
33 changes: 33 additions & 0 deletions plugin-gradle/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ Spotless supports all of Gradle's built-in performance features (incremental bui
- [Javascript](#javascript) ([prettier](#prettier), [ESLint](#eslint-javascript), [Biome](#biome))
- [JSON](#json) ([simple](#simple), [gson](#gson), [jackson](#jackson), [Biome](#biome), [jsonPatch](#jsonPatch))
- [YAML](#yaml)
- [Shell](#shell)
- [Gherkin](#gherkin)
- Multiple languages
- [Prettier](#prettier) ([plugins](#prettier-plugins), [npm detection](#npm-detection), [`.npmrc` detection](#npmrc-detection), [caching `npm install` results](#caching-results-of-npm-install))
Expand Down Expand Up @@ -982,6 +983,38 @@ spotless {
}
```
## Shell
`com.diffplug.gradle.spotless.ShellExtension` [javadoc](https://javadoc.io/doc/com.diffplug.spotless/spotless-plugin-gradle/6.23.3/com/diffplug/gradle/spotless/ShellExtension.html), [code](https://github.com/diffplug/spotless/blob/main/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/ShellExtension.java)
```gradle
spotless {
shell {
target 'scripts/**/*.sh' // default: '*.sh'
shfmt() // has its own section below
}
}
```
### shfmt
[homepage](https://github.com/mvdan/sh). [changelog](https://github.com/mvdan/sh/blob/master/CHANGELOG.md).
```gradle
shfmt('3.7.0') // version is optional
// if shfmt is not on your path, you must specify its location manually
shfmt().pathToExe('/opt/homebrew/bin/shfmt')
// Spotless always checks the version of the shfmt it is using
// and will fail with an error if it does not match the expected version
// (whether manually specified or default). If there is a problem, Spotless
// will suggest commands to help install the correct version.
// TODO: handle installation & packaging automatically - https://github.com/diffplug/spotless/issues/674
```
<a name="applying-freshmark-to-markdown-files"></a>
## Gherkin
- `com.diffplug.gradle.spotless.GherkinExtension` [javadoc](https://javadoc.io/doc/com.diffplug.spotless/spotless-plugin-gradle/6.23.3/com/diffplug/gradle/spotless/GherkinExtension.html), [code](https://github.com/diffplug/spotless/blob/main/plugin-gradle/src/main/java/com/diffplug/gradle/spotless/GherkinExtension.java)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
/*
* Copyright 2024 DiffPlug
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.diffplug.gradle.spotless;

import java.util.Objects;

import javax.inject.Inject;

import com.diffplug.spotless.FormatterStep;
import com.diffplug.spotless.shell.ShfmtStep;

public class ShellExtension extends FormatExtension {
private static final String SHELL_FILE_EXTENSION = "*.sh";

static final String NAME = "shell";

@Inject
public ShellExtension(SpotlessExtension spotless) {
super(spotless);
}

/** If the user hasn't specified files, assume all shell files should be checked. */
@Override
protected void setupTask(SpotlessTask task) {
if (target == null) {
target = parseTarget(SHELL_FILE_EXTENSION);
}
super.setupTask(task);
}

/** Adds the specified version of <a href="https://github.com/mvdan/sh">shfmt</a>. */
public ShfmtExtension shfmt(String version) {
Objects.requireNonNull(version);
return new ShfmtExtension(version);
}

/** Adds the specified version of <a href="https://github.com/mvdan/sh">shfmt</a>. */
public ShfmtExtension shfmt() {
return shfmt(ShfmtStep.defaultVersion());
}

public class ShfmtExtension {
ShfmtStep step;

ShfmtExtension(String version) {
this.step = ShfmtStep.withVersion(version);
addStep(createStep());
}

public ShfmtExtension pathToExe(String pathToExe) {
step = step.withPathToExe(pathToExe);
replaceStep(createStep());
return this;
}

private FormatterStep createStep() {
return step.create();
}
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2016-2023 DiffPlug
* Copyright 2016-2024 DiffPlug
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -205,6 +205,12 @@ public void protobuf(Action<ProtobufExtension> closure) {
format(ProtobufExtension.NAME, ProtobufExtension.class, closure);
}

/** Configures the special shell-specific extension. */
public void shell(Action<ShellExtension> closure) {
requireNonNull(closure);
format(ShellExtension.NAME, ShellExtension.class, closure);
}

/** Configures the special YAML-specific extension. */
public void yaml(Action<YamlExtension> closure) {
requireNonNull(closure);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright 2024 DiffPlug
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.diffplug.gradle.spotless;

import java.io.IOException;

import org.junit.jupiter.api.Test;

import com.diffplug.spotless.tag.ShfmtTest;

@ShfmtTest
public class ShfmtIntegrationTest extends GradleIntegrationHarness {
@Test
void shfmt() throws IOException {
setFile("build.gradle").toLines(
"plugins {",
" id 'com.diffplug.spotless'",
"}",
"spotless {",
" shell {",
" shfmt()",
" }",
"}");
setFile("shfmt.sh").toResource("shell/shfmt/shfmt.sh");
gradleRunner().withArguments("spotlessApply").build();
assertFile("shfmt.sh").sameAsResource("shell/shfmt/shfmt.clean");
}
}
30 changes: 30 additions & 0 deletions testlib/src/main/java/com/diffplug/spotless/tag/ShfmtTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Copyright 2024 DiffPlug
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.diffplug.spotless.tag;

import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;

import java.lang.annotation.Retention;
import java.lang.annotation.Target;

import org.junit.jupiter.api.Tag;

@Target({TYPE, METHOD})
@Retention(RUNTIME)
@Tag("Shfmt")
public @interface ShfmtTest {}
Loading

0 comments on commit e2e42e4

Please sign in to comment.