From 083f3c1e2971de3b120e15c449b69ec8043e5cd0 Mon Sep 17 00:00:00 2001 From: huangxiaoping <1754789345@qq.com> Date: Wed, 24 Jan 2024 11:47:32 +0800 Subject: [PATCH] [KYUUBI #6010] Improve Gluten usage documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # :mag: Description ## Issue References ๐Ÿ”— This pull request fixes #6010 ## Describe Your Solution ๐Ÿ”ง Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ## Types of changes :bookmark: - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Test Plan ๐Ÿงช #### Behavior Without This Pull Request :coffin: #### Behavior With This Pull Request :tada: #### Related Unit Tests --- # Checklist ๐Ÿ“ - [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html) **Be nice. Be informative.** Closes #6011 from huangxiaopingRD/gluten-doc. Closes #6010 88bce5865 [huangxiaoping] [KYUUBI #6010] Improve Gluten usage documentation Authored-by: huangxiaoping <1754789345@qq.com> Signed-off-by: Cheng Pan --- docs/deployment/spark/gluten.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/deployment/spark/gluten.md b/docs/deployment/spark/gluten.md index 8f6bcdef7af..371b74edfd6 100644 --- a/docs/deployment/spark/gluten.md +++ b/docs/deployment/spark/gluten.md @@ -18,7 +18,7 @@ # Gluten -Gluten is a Spark plugin developed by Intel, designed to accelerate Apache Spark with native libraries. Currently, only CentOS 7/8 and Ubuntu 20.04/22.04, along with Spark 3.2/3.3/3.4, are supported. Users can employ the following methods to utilize the Gluten with Velox native libraries. +[Gluten](https://oap-project.github.io/gluten/) is a Spark plugin developed by Intel, designed to accelerate Apache Spark with native libraries. Currently, only CentOS 7/8 and Ubuntu 20.04/22.04, along with Spark 3.2/3.3/3.4, are supported. Users can employ the following methods to utilize the Gluten with Velox native libraries. ## Building(with velox Backend) @@ -40,11 +40,11 @@ You can use Gluten to accelerate Spark by following steps. ### Installing -add gluten jar: `copy /path/to/gluten/package/target/gluten-velox-bundle-spark3.x_2.12-*.jar $SPARK_HOME/jars/` or specified to `spark.jars` configuration +Add gluten jar: `copy /path/to/gluten/package/target/gluten-velox-bundle-spark3.x_2.12-*.jar $SPARK_HOME/jars/` or specified to `spark.jars` configuration ### Configure -add config into `spark-defaults.conf`: +Add the following minimal configuration into `spark-defaults.conf`: ```properties spark.plugins=io.glutenproject.GlutenPlugin @@ -53,3 +53,4 @@ spark.memory.offHeap.enabled=true spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager ``` +For more configuration can be found in the doc of [Configuration](https://oap-project.github.io/gluten/Configuration.html).