Skip to content

Commit

Permalink
refactor: Rename from com.crealytics to dev.mauch
Browse files Browse the repository at this point in the history
  • Loading branch information
nightscape committed Dec 4, 2024
1 parent 6a41cc0 commit 312edfe
Show file tree
Hide file tree
Showing 83 changed files with 262 additions and 262 deletions.
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/generic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ body:
Steps to Reproduce (for bugs)
Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. Include code to reproduce, if relevant. Example:
Download the example file uploaded here
Start Spark from command line as spark-shell --packages com.crealytics:spark-excel_2.12:x.y.z --foo=bar
Start Spark from command line as spark-shell --packages dev.mauch:spark-excel_2.12:x.y.z --foo=bar
Read the downloaded example file
val df = spark.read
.format("com.crealytics.spark.excel")
.format("dev.mauch.spark.excel")
.option("dataAddress", "'My Sheet'!B3:C35")
.load("example_file_exhibiting_bug.xlsx")
validations:
Expand Down
242 changes: 121 additions & 121 deletions CHANGELOG.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Moreover, please read the [`CHANGELOG.md`](../../blob/master/CHANGELOG.md) file
> reproduce this bug. Include code to reproduce, if relevant.
> Example:
1. Download the example file uploaded [here](http://example.com/)
2. Start Spark from command line as `spark-shell --packages com.crealytics:spark-excel_2.12:x.y.z --foo=bar`
2. Start Spark from command line as `spark-shell --packages dev.mauch:spark-excel_2.12:x.y.z --foo=bar`
3. Read the downloaded example file
```
val df = spark.read
.format("com.crealytics.spark.excel")
.format("dev.mauch.spark.excel")
.option("dataAddress", "'My Sheet'!B3:C35")
.load("example_file_exhibiting_bug.xlsx")
```
Expand Down
36 changes: 18 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

A library for querying Excel files with Apache Spark, for Spark SQL and DataFrames.

[![Build Status](https://github.com/crealytics/spark-excel/workflows/CI/badge.svg)](https://github.com/crealytics/spark-excel/actions)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.crealytics/spark-excel_2.12/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.crealytics/spark-excel_2.12)
[![Build Status](https://github.dev/mauch/spark-excel/workflows/CI/badge.svg)](https://github.dev/mauch/spark-excel/actions)
[![Maven Central](https://maven-badges.herokuapp.com/maven-central/dev.mauch/spark-excel_2.12/badge.svg)](https://maven-badges.herokuapp.com/maven-central/dev.mauch/spark-excel_2.12)


## Co-maintainers wanted
Due to personal and professional constraints, the development of this library has been rather slow.
If you find value in this library, please consider stepping up as a co-maintainer by leaving a comment [here](https://github.com/crealytics/spark-excel/issues/191).
If you find value in this library, please consider stepping up as a co-maintainer by leaving a comment [here](https://github.dev/mauch/spark-excel/issues/191).
Help is very welcome e.g. in the following areas:

* Additional features
Expand All @@ -25,21 +25,21 @@ List of spark versions, those are automatically tested:
```
spark: ["2.4.1", "2.4.7", "2.4.8", "3.0.1", "3.0.3", "3.1.1", "3.1.2", "3.2.4", "3.3.2", "3.4.1"]
```
For more detail, please refer to project CI: [ci.yml](https://github.com/crealytics/spark-excel/blob/main/.github/workflows/ci.yml#L10)
For more detail, please refer to project CI: [ci.yml](https://github.dev/mauch/spark-excel/blob/main/.github/workflows/ci.yml#L10)

## Linking
You can link against this library in your program at the following coordinates:

### Scala 2.12
```
groupId: com.crealytics
groupId: dev.mauch
artifactId: spark-excel_2.12
version: <spark-version>_0.18.0
```

### Scala 2.11
```
groupId: com.crealytics
groupId: dev.mauch
artifactId: spark-excel_2.11
version: <spark-version>_0.13.7
```
Expand All @@ -49,22 +49,22 @@ This package can be added to Spark using the `--packages` command line option.

### Spark compiled with Scala 2.12
```
$SPARK_HOME/bin/spark-shell --packages com.crealytics:spark-excel_2.12:<spark-version>_0.18.0
$SPARK_HOME/bin/spark-shell --packages dev.mauch:spark-excel_2.12:<spark-version>_0.18.0
```

### Spark compiled with Scala 2.11
```
$SPARK_HOME/bin/spark-shell --packages com.crealytics:spark-excel_2.11:<spark-version>_0.13.7
$SPARK_HOME/bin/spark-shell --packages dev.mauch:spark-excel_2.11:<spark-version>_0.13.7
```

## Features
* This package allows querying Excel spreadsheets as [Spark DataFrames](https://spark.apache.org/docs/latest/sql-programming-guide.html).
* From spark-excel [0.14.0](https://github.com/crealytics/spark-excel/releases/tag/v0.14.0) (August 24, 2021), there are two implementation of spark-excel
* From spark-excel [0.14.0](https://github.dev/mauch/spark-excel/releases/tag/v0.14.0) (August 24, 2021), there are two implementation of spark-excel
* Original Spark-Excel with Spark data source API 1.0
* Spark-Excel V2 with data source API V2.0+, which supports loading from multiple files, corrupted record handling and some improvement on handling data types.
See below for further details

To use V2 implementation, just change your .format from `.format("com.crealytics.spark.excel")` to `.format("excel")`.
To use V2 implementation, just change your .format from `.format("dev.mauch.spark.excel")` to `.format("excel")`.
See [below](#excel-api-based-on-datasourcev2) for some details

See the [changelog](CHANGELOG.md) for latest features, fixes etc.
Expand All @@ -80,7 +80,7 @@ import org.apache.spark.sql._

val spark: SparkSession = ???
val df = spark.read
.format("com.crealytics.spark.excel") // Or .format("excel") for V2 implementation
.format("dev.mauch.spark.excel") // Or .format("excel") for V2 implementation
.option("dataAddress", "'My Sheet'!B3:C35") // Optional, default: "A1"
.option("header", "true") // Required
.option("treatEmptyValuesAsNulls", "false") // Optional, default: true
Expand All @@ -104,7 +104,7 @@ and provides a `.excel` method which accepts all possible options and provides d

```scala
import org.apache.spark.sql._
import com.crealytics.spark.excel._
import dev.mauch.spark.excel._

val spark: SparkSession = ???
val df = spark.read.excel(
Expand Down Expand Up @@ -137,7 +137,7 @@ val df = spark.read.excel(
or to read in the names dynamically:

```scala
import com.crealytics.spark.excel.WorkbookReader
import dev.mauch.spark.excel.WorkbookReader
val sheetNames = WorkbookReader( Map("path" -> "Worktime.xlsx")
, spark.sparkContext.hadoopConfiguration
).sheetNames
Expand All @@ -160,7 +160,7 @@ val peopleSchema = StructType(Array(

val spark: SparkSession = ???
val df = spark.read
.format("com.crealytics.spark.excel") // Or .format("excel") for V2 implementation
.format("dev.mauch.spark.excel") // Or .format("excel") for V2 implementation
.option("dataAddress", "'Info'!A1")
.option("header", "true")
.schema(peopleSchema)
Expand All @@ -173,7 +173,7 @@ import org.apache.spark.sql._

val df: DataFrame = ???
df.write
.format("com.crealytics.spark.excel") // Or .format("excel") for V2 implementation
.format("dev.mauch.spark.excel") // Or .format("excel") for V2 implementation
.option("dataAddress", "'My Sheet'!B3:C35")
.option("header", "true")
.option("dateFormat", "yy-mmm-d") // Optional, default: yy-m-d h:mm
Expand Down Expand Up @@ -205,7 +205,7 @@ Currently the following address styles are supported:
The V2 API offers you several improvements when it comes to file and folder handling.
and works in a very similar way than data sources like csv and parquet.

To use V2 implementation, just change your .format from `.format("com.crealytics.spark.excel")` to `.format("excel")`
To use V2 implementation, just change your .format from `.format("dev.mauch.spark.excel")` to `.format("excel")`

The big difference is the fact that you provide a path to read / write data from/to and not
an individual single file only:
Expand Down Expand Up @@ -235,7 +235,7 @@ dataFrame.write
.save("some/path")
````

Need some more examples? Check out the [test cases](src/test/scala/com/crealytics/spark/excel/v2/DataFrameWriterApiComplianceSuite.scala)
Need some more examples? Check out the [test cases](src/test/scala/dev/mauch/spark/excel/v2/DataFrameWriterApiComplianceSuite.scala)
or have a look at our wiki

## Building From Source
Expand All @@ -250,5 +250,5 @@ We are grateful for their support in the initial development and open-sourcing o

## Star History

[![Star History Chart](https://api.star-history.com/svg?repos=crealytics/spark-excel&type=Date)](https://star-history.com/#crealytics/spark-excel&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=nightscape/spark-excel&type=Date)](https://star-history.com/#nightscape/spark-excel&Date)

6 changes: 3 additions & 3 deletions build.sc
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ trait SparkModule extends Cross.Module2[String, String] with SbtModule with CiRe

def pomSettings = PomSettings(
description = "A Spark plugin for reading and writing Excel files",
organization = "com.crealytics",
url = "https://github.com/crealytics/spark-excel",
organization = "dev.mauch",
url = "https://github.com/nightscape/spark-excel",
licenses = Seq(License.`Apache-2.0`),
versionControl = VersionControl.github("crealytics", "spark-excel"),
versionControl = VersionControl.github("nightscape", "spark-excel"),
developers = Seq(Developer("nightscape", "Martin Mauch", "https://github.com/nightscape"))
)

Expand Down
2 changes: 1 addition & 1 deletion src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ Spark-excel V2 introduces spark-version specific code folder, like:
`3.x/.../spark/v2/excel` for all Spark 3.* Data Source API V2
`3.1_3.2/.../spark/v2/excel` for shared code between Spark 3.1 and Spark 3.2 Data Source API V2

These structures are also configured into [build.sc](https://github.com/crealytics/spark-excel/blob/main/build.sc#L13), so it can compile for each Spark version.
These structures are also configured into [build.sc](https://github.dev/mauch/spark-excel/blob/main/build.sc#L13), so it can compile for each Spark version.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package org.apache.spark.crealytics
package org.apache.spark.nightscape

import java.nio.file.{Files, Paths}
import org.apache.spark.rdd.InputFileBlockHolder
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.FileStatus
import org.apache.hadoop.fs.Path
import org.apache.spark.crealytics.ExcelSparkInternal
import org.apache.spark.nightscape.ExcelSparkInternal
import org.apache.spark.deploy.SparkHadoopUtil
import org.apache.spark.sql.SaveMode
import org.apache.spark.sql.SparkSession
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.unsafe.types.UTF8String
import org.apache.spark.sql.catalyst.util._
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.sources
import org.apache.spark.sql.types.StructType
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.catalyst.util.CaseInsensitiveMap
import org.apache.spark.sql.internal.SQLConf
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import _root_.org.apache.spark.sql.catalyst.util.BadRecordException
import org.apache.spark.unsafe.types.UTF8String
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.SparkException
import org.apache.spark.sql.catalyst.InternalRow
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.catalyst.analysis._
import org.apache.spark.sql.types.StructType
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.unsafe.types.UTF8String
import org.apache.spark.sql.catalyst.util._
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.catalyst.csv.CSVFilters
import org.apache.spark.sql.sources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.connector.catalog.Table
import org.apache.spark.sql.execution.datasources.FileFormat
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.fs.{FileStatus, Path}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.mapreduce.TaskAttemptContext
import org.apache.spark.internal.Logging
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.fs.FileStatus
import org.apache.spark.sql.SparkSession
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.mapreduce.Job
import org.apache.hadoop.mapreduce.TaskAttemptContext
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.fs.Path
import org.apache.spark.sql.SparkSession
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.connector.read.{Scan, SupportsPushDownFilters}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.catalyst.util.CaseInsensitiveMap
import org.apache.spark.sql.internal.SQLConf
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import _root_.org.apache.spark.sql.catalyst.util.BadRecordException
import org.apache.spark.unsafe.types.UTF8String
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.hadoop.conf.Configuration
import org.apache.spark.broadcast.Broadcast
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.unsafe.types.UTF8String
import org.apache.spark.sql.catalyst.util._
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.sql.catalyst.OrderedFilters
import org.apache.spark.sql.catalyst.StructFilters
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

package com.crealytics.spark.excel.v2
package dev.mauch.spark.excel.v2

import org.apache.spark.unsafe.types.UTF8String
import org.apache.spark.sql.catalyst.util._
Expand Down
Loading

0 comments on commit 312edfe

Please sign in to comment.