Skip to content
CI

[KYUUBI #5738][FEATURE] Add a spark commit protocol to support compact small file before commit job #19918

Sign in to view logs

[KYUUBI #5738][FEATURE] Add a spark commit protocol to support compact small file before commit job

[KYUUBI #5738][FEATURE] Add a spark commit protocol to support compact small file before commit job #19918

Triggered via pull request March 3, 2024 23:57
@pan3793pan3793
reopened #5758
AngersZhuuuu:KYUUBI-5738
Status Failure
Total duration 1h 13m 52s
Artifacts 1

master.yml

on: pull_request
TPC-H and TPC-DS Tests
30m 48s
TPC-H and TPC-DS Tests
Kyuubi Server On Kubernetes Integration Test
12m 34s
Kyuubi Server On Kubernetes Integration Test
Spark Engine On Kubernetes Integration Test
24m 47s
Spark Engine On Kubernetes Integration Test
Matrix: Kyuubi and Spark Test
Matrix: Flink Test
Matrix: Hive Test
Matrix: JDBC Test
Matrix: Scala Test
Matrix: Trino Test
Matrix: Zookeeper Integration Test
Fit to window
Zoom out
Zoom in

Annotations

48 errors and 42 warnings
Kyuubi Server On Kubernetes Integration Test
Process completed with exit code 1.
Kyuubi Server On Kubernetes Integration Test
Process completed with exit code 1.
Kyuubi Server On Kubernetes Integration Test
Process completed with exit code 1.
Kyuubi Server On Kubernetes Integration Test
buildx failed with: ERROR: failed to solve: process "/bin/sh -c apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y bash python3 && ln -snf /bin/bash /bin/sh && ./build/dist ${MVN_ARG} && mv /workspace/kyuubi/dist /opt/kyuubi && rm -rf ~/.m2 && rm -rf /workspace/kyuubi" did not complete successfully: exit code: 1
Spark Engine On Kubernetes Integration Test
Process completed with exit code 1.
Flink Test (8, 1.18, normal)
Process completed with exit code 1.
Kyuubi and Spark Test (11, 3.2, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.2, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.4, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (11, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.2, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.2, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.3, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Scala Test (2.13, 8, 3.4)
EACCES: permission denied, scandir '/home/runner/work/kyuubi/kyuubi/externals/kyuubi-spark-sql-engine/target/tmp/jetcd_test_etcd0_239287508833753507/member'
Scala Test (2.13, 8, 3.4)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Scala Test (2.13, 8, 3.4)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Scala Test (2.13, 8, 3.4)
Process completed with exit code 1.
Kyuubi and Spark Test (11, 3.1, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.1, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.1, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.1, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (11, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (17, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (17, 3.5, normal)
Cannot invoke "org.apache.kyuubi.engine.KyuubiApplicationManager.killApplication(org.apache.kyuubi.engine.ApplicationManagerInfo, String, scala.Option)" because "$this.engineManager" is null
Kyuubi and Spark Test (8, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.5, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.3, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Kyuubi and Spark Test (8, 3.4, normal)
Timeout to lock on path [/engine_test2_1.9.0-SNAPSHOT_USER_SPARK_SQL_lock/runner/default] after 4500 ms. There would be some problem that other session may create engine timeout.
Zookeeper Integration Test (8, 3.4, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Zookeeper Integration Test (8, 3.5, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Zookeeper Integration Test (8, 3.6, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Zookeeper Integration Test (8, 3.7, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi Server On Kubernetes Integration Test
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi Server On Kubernetes Integration Test
No files were found with the provided path: **/target/unit-tests.log **/target/work/**. No artifacts will be uploaded.
Trino Test (11, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (8, 1.17, -Dflink.archive.mirror=https://archive.apache.org/dist/flink/flink-1.16.3 -D...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (8, 1.17, -Dflink.archive.mirror=https://archive.apache.org/dist/flink/flink-1.18.1 -D...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Hive Test (8, -Dhive.archive.mirror=https://github.com/pan3793/cdh-hive/releases/download/cdh6.3....
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Trino Test (8, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Hive Test (8, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Spark Engine On Kubernetes Integration Test
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Spark Engine On Kubernetes Integration Test
No files were found with the provided path: **/target/unit-tests.log **/kyuubi-spark-sql-engine.log* **/kyuubi-spark-batch-submit.log*. No artifacts will be uploaded.
Hive Test (8, -Dhive.archive.mirror=https://archive.apache.org/dist/hive/hive-2.3.9 -Dhive.archiv...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
TPC-H and TPC-DS Tests
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
JDBC Test (11, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
JDBC Test (8, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (11, 1.16, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (8, 1.18, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (11, 1.17, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (8, 1.16, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (8, 1.17, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Flink Test (11, 1.18, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (11, 3.2, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (11, 3.3, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (17, 3.4, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (11, 3.4, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.2, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (17, 3.3, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.4, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Scala Test (2.13, 8, 3.4)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3, actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (11, 3.1, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.1, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (11, 3.5, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.4, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (17, 3.5, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.4, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.4, -Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark...
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.5, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.3, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Kyuubi and Spark Test (8, 3.4, normal)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache@v3, codecov/codecov-action@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.

Artifacts

Produced during runtime
Name Size
unit-tests-log-java-8-flink-1.18-normal Expired
3.86 MB