Skip to content

Commit

Permalink
[SPARK-48573][SQL] Upgrade ICU version
Browse files Browse the repository at this point in the history 
### What changes were proposed in this pull request?
Upgrade of ICU version from 72.1 -> 75.1

### Why are the changes needed?
We need to keep the version up-to-date.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Existing tests were not broken.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#47011 from mihailom-db/upgrade-icu.

Authored-by: Mihailo Milosevic <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
  • Loading branch information
@mihailom-db @yaooqinn
mihailom-db authored and yaooqinn committed Jun 26, 2024
1 parent ebacb91 commit c459afb
Show file tree
Hide file tree
Showing 7 changed files with 169 additions and 168 deletions.
2 changes: 1 addition & 1 deletion dev/deps/spark-deps-hadoop-3-hive-2.3
View file
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ hk2-locator/3.0.3//hk2-locator-3.0.3.jar
hk2-utils/3.0.3//hk2-utils-3.0.3.jar
httpclient/4.5.14//httpclient-4.5.14.jar
httpcore/4.4.16//httpcore-4.4.16.jar
icu4j/72.1//icu4j-72.1.jar
icu4j/75.1//icu4j-75.1.jar
ini4j/0.5.4//ini4j-0.5.4.jar
istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar
ivy/2.5.2//ivy-2.5.2.jar
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@
<datasketches.version>6.0.0</datasketches.version>
<netty.version>4.1.110.Final</netty.version>
<netty-tcnative.version>2.0.65.Final</netty-tcnative.version>
<icu4j.version>72.1</icu4j.version>
<icu4j.version>75.1</icu4j.version>
<junit-jupiter.version>5.9.3</junit-jupiter.version>
<junit-platform.version>1.9.3</junit-platform.version>
<sbt-jupiter-interface.version>0.11.1</sbt-jupiter-interface.version>
Expand Down
48 changes: 24 additions & 24 deletions sql/core/benchmarks/CollationBenchmark-jdk21-results.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,53 +2,53 @@ OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - equalsFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 1355 1358 4 0.1 13551.1 1.0X
UTF8_LCASE 4983 4984 3 0.0 49826.4 0.3X
UNICODE 18212 18220 12 0.0 182120.9 0.1X
UNICODE_CI 17568 17577 14 0.0 175677.2 0.1X
UTF8_BINARY 1352 1352 1 0.1 13516.4 1.0X
UTF8_LCASE 4678 4693 21 0.0 46778.6 0.3X
UNICODE 17213 17223 13 0.0 172131.7 0.1X
UNICODE_CI 17101 17133 46 0.0 171009.6 0.1X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - compareFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 1772 1774 3 0.1 17722.3 1.0X
UTF8_LCASE 4365 4365 0 0.0 43649.6 0.4X
UNICODE 16538 16544 9 0.0 165375.5 0.1X
UNICODE_CI 16296 16305 12 0.0 162961.9 0.1X
UTF8_BINARY 1775 1775 0 0.1 17749.8 1.0X
UTF8_LCASE 5342 5367 35 0.0 53423.0 0.3X
UNICODE 17011 17020 12 0.0 170110.1 0.1X
UNICODE_CI 16734 16760 37 0.0 167338.2 0.1X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - hashFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 7279 7280 1 0.0 72791.2 1.0X
UTF8_LCASE 18538 18543 6 0.0 185381.0 0.4X
UNICODE 71514 71520 8 0.0 715144.6 0.1X
UNICODE_CI 60488 60488 0 0.0 604880.9 0.1X
UTF8_BINARY 7253 7256 4 0.0 72529.6 1.0X
UTF8_LCASE 16634 16676 59 0.0 166342.5 0.4X
UNICODE 66146 66163 23 0.0 661461.1 0.1X
UNICODE_CI 54563 54606 62 0.0 545625.5 0.1X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - contains: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 7516 7519 4 0.0 75162.9 1.0X
UTF8_LCASE 120330 120338 12 0.0 1203299.2 0.1X
UNICODE 371784 371946 228 0.0 3717840.7 0.0X
UNICODE_CI 427401 427547 207 0.0 4274009.0 0.0X
UTF8_BINARY 6570 6577 10 0.0 65696.6 1.0X
UTF8_LCASE 120073 120137 91 0.0 1200726.4 0.1X
UNICODE 364027 364291 374 0.0 3640267.9 0.0X
UNICODE_CI 421444 422138 981 0.0 4214438.7 0.0X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - startsWith: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 6504 6507 3 0.0 65044.6 1.0X
UTF8_LCASE 60331 60359 40 0.0 603313.9 0.1X
UNICODE 369394 369404 13 0.0 3693943.0 0.0X
UNICODE_CI 427382 427421 55 0.0 4273819.7 0.0X
UTF8_BINARY 6573 6596 32 0.0 65733.4 1.0X
UTF8_LCASE 60284 60293 12 0.0 602844.4 0.1X
UNICODE 363685 364220 757 0.0 3636848.4 0.0X
UNICODE_CI 422761 423000 337 0.0 4227611.0 0.0X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - endsWith: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 6600 6601 1 0.0 66002.7 1.0X
UTF8_LCASE 58723 58751 39 0.0 587230.1 0.1X
UNICODE 379668 379789 172 0.0 3796677.7 0.0X
UNICODE_CI 437119 437194 106 0.0 4371189.5 0.0X
UTF8_BINARY 6507 6508 1 0.0 65068.3 1.0X
UTF8_LCASE 59098 59118 28 0.0 590983.6 0.1X
UNICODE 378437 378790 499 0.0 3784367.4 0.0X
UNICODE_CI 433987 434294 435 0.0 4339869.2 0.0X

48 changes: 24 additions & 24 deletions sql/core/benchmarks/CollationBenchmark-results.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,53 +2,53 @@ OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - equalsFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 1370 1370 1 0.1 13698.4 1.0X
UTF8_LCASE 4836 4836 0 0.0 48359.5 0.3X
UNICODE 19239 19271 45 0.0 192391.8 0.1X
UNICODE_CI 18936 18954 25 0.0 189362.4 0.1X
UTF8_BINARY 1372 1374 3 0.1 13718.1 1.0X
UTF8_LCASE 6311 6311 0 0.0 63106.7 0.2X
UNICODE 19273 19300 37 0.0 192731.3 0.1X
UNICODE_CI 18991 18998 10 0.0 189906.3 0.1X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - compareFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 1726 1727 1 0.1 17260.4 1.0X
UTF8_LCASE 6293 6304 16 0.0 62927.1 0.3X
UNICODE 18677 18679 4 0.0 186768.3 0.1X
UNICODE_CI 18488 18504 23 0.0 184879.6 0.1X
UTF8_BINARY 1725 1726 2 0.1 17249.0 1.0X
UTF8_LCASE 5806 5828 31 0.0 58061.5 0.3X
UNICODE 19105 19111 8 0.0 191051.5 0.1X
UNICODE_CI 18991 18996 7 0.0 189913.3 0.1X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - hashFunction: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 3028 3029 1 0.0 30283.4 1.0X
UTF8_LCASE 19773 19830 81 0.0 197726.4 0.2X
UNICODE 68565 68594 41 0.0 685646.9 0.0X
UNICODE_CI 53100 53101 2 0.0 530996.0 0.1X
UTF8_BINARY 3019 3021 3 0.0 30194.7 1.0X
UTF8_LCASE 19437 19439 3 0.0 194372.6 0.2X
UNICODE 63550 63568 25 0.0 635504.3 0.0X
UNICODE_CI 57839 57866 39 0.0 578385.0 0.1X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - contains: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 7024 7026 3 0.0 70244.6 1.0X
UTF8_LCASE 118693 118703 15 0.0 1186926.5 0.1X
UNICODE 385409 386299 1257 0.0 3854093.7 0.0X
UNICODE_CI 434618 435527 1285 0.0 4346181.0 0.0X
UTF8_BINARY 7022 7023 2 0.0 70216.8 1.0X
UTF8_LCASE 118836 118887 72 0.0 1188364.9 0.1X
UNICODE 376381 376546 234 0.0 3763807.3 0.0X
UNICODE_CI 427858 427981 174 0.0 4278584.6 0.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - startsWith: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 6069 6090 29 0.0 60691.9 1.0X
UTF8_LCASE 61809 61828 27 0.0 618094.5 0.1X
UNICODE 370523 371729 1705 0.0 3705229.7 0.0X
UNICODE_CI 435805 436945 1612 0.0 4358051.5 0.0X
UTF8_BINARY 6720 6723 5 0.0 67197.9 1.0X
UTF8_LCASE 67132 67177 63 0.0 671324.6 0.1X
UNICODE 368690 369292 852 0.0 3686899.6 0.0X
UNICODE_CI 431481 431583 144 0.0 4314814.9 0.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1022-azure
AMD EPYC 7763 64-Core Processor
collation unit benchmarks - endsWith: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
UTF8_BINARY 6725 6732 10 0.0 67247.9 1.0X
UTF8_LCASE 54990 55010 28 0.0 549896.0 0.1X
UNICODE 380872 383258 3375 0.0 3808722.0 0.0X
UNICODE_CI 443911 444111 283 0.0 4439112.3 0.0X
UTF8_BINARY 7097 7099 3 0.0 70970.8 1.0X
UTF8_LCASE 57327 57351 35 0.0 573265.4 0.1X
UNICODE 375819 376473 924 0.0 3758191.8 0.0X
UNICODE_CI 445167 445212 64 0.0 4451666.7 0.0X

Loading

0 comments on commit c459afb

Please sign in to comment.