Skip to content

Commit

Permalink
[SPARK-36068][BUILD][TEST] No tests in hadoop-cloud run unless hadoop…
Browse files Browse the repository at this point in the history
…-3.2 profile is activated explicitly

### What changes were proposed in this pull request?

This PR fixes an issue that no tests in `hadoop-cloud` are compiled and run unless `hadoop-3.2` profile is activated explicitly.
The root cause seems similar to SPARK-36067 (apache#33276) so the solution is to activate `hadoop-3.2` profile in `hadoop-cloud/pom.xml` by default.

### Why are the changes needed?

`hadoop-3.2` profile should be activated by default so tests in `hadoop-cloud` also should be compiled and run without activating `hadoop-3.2` profile explicitly.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Confirmed tests in `hadoop-cloud` ran with both SBT and Maven.
```
build/sbt -Phadoop-cloud "hadoop-cloud/test"
...
[info] CommitterBindingSuite:
[info] - BindingParquetOutputCommitter binds to the inner committer (258 milliseconds)
[info] - committer protocol can be serialized and deserialized (11 milliseconds)
[info] - local filesystem instantiation (3 milliseconds)
[info] - reject dynamic partitioning (1 millisecond)
[info] Run completed in 1 second, 234 milliseconds.
[info] Total number of tests run: 4
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

build/mvn -Phadoop-cloud -pl hadoop-cloud test
...
CommitterBindingSuite:
- BindingParquetOutputCommitter binds to the inner committer
- committer protocol can be serialized and deserialized
- local filesystem instantiation
- reject dynamic partitioning
Run completed in 560 milliseconds.
Total number of tests run: 4
Suites: completed 2, aborted 0
Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
```

Closes apache#33277 from sarutak/fix-hadoop-3.2-cloud.

Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
sarutak authored and HyukjinKwon committed Jul 9, 2021
1 parent 698c4ec commit c3c5af8
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions hadoop-cloud/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,9 @@
-->
<profile>
<id>hadoop-3.2</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<extra.source.dir>src/hadoop-3/main/scala</extra.source.dir>
<extra.testsource.dir>src/hadoop-3/test/scala</extra.testsource.dir>
Expand Down

0 comments on commit c3c5af8

Please sign in to comment.