[SPARK-44781][SQL] Runtime filter should supports reuse exchange if i… #1500
build_main.yml
on: push
Run
/
Check changes
39s
Run
/
Breaking change detection with Buf (branch-3.5)
1m 9s
Run
/
Scala 2.13 build with SBT
19m 10s
Run
/
Run TPC-DS queries with SF=1
1h 7m
Run
/
Run Docker integration tests
51m 17s
Run
/
Run Spark on Kubernetes Integration test
1h 7m
Matrix: Run / build
Matrix: Run / java-11-17
Run
/
Build modules: sparkr
36m 0s
Run
/
Linters, licenses, dependencies and documentation generation
9m 28s
Matrix: Run / pyspark
Annotations
34 errors and 2 warnings
Run / Linters, licenses, dependencies and documentation generation
Process completed with exit code 1.
|
Run / Run Spark on Kubernetes Integration test
Set() did not contain "decomtest-35ea9189e9256ba9-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Set() did not contain "decomtest-cf78ba89e92674be-exec-1".
|
Run / Run Spark on Kubernetes Integration test
sleep interrupted
|
Run / Run Spark on Kubernetes Integration test
Task io.fabric8.kubernetes.client.utils.internal.SerialExecutor$$Lambda$577/1926024969@b38e86e rejected from java.util.concurrent.ThreadPoolExecutor@4e5fe79[Shutting down, pool size = 3, active threads = 2, queued tasks = 0, completed tasks = 314]
|
Run / Run Spark on Kubernetes Integration test
Task io.fabric8.kubernetes.client.utils.internal.SerialExecutor$$Lambda$577/1926024969@26380f69 rejected from java.util.concurrent.ThreadPoolExecutor@4e5fe79[Shutting down, pool size = 2, active threads = 1, queued tasks = 0, completed tasks = 315]
|
Run / Run Spark on Kubernetes Integration test
Set() did not contain "decomtest-49448a89e93b8adb-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Set() did not contain "decomtest-e2d7d989e93c9b4c-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Set() did not contain "decomtest-d587d389e94083ee-exec-1".
|
Run / Run Spark on Kubernetes Integration test
Status(apiVersion=v1, code=404, details=StatusDetails(causes=[], group=null, kind=pods, name=spark-test-app-5e69a45f81734ca48cb9277a8a74f3cc-driver, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "spark-test-app-5e69a45f81734ca48cb9277a8a74f3cc-driver" not found, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=NotFound, status=Failure, additionalProperties={})..
|
Run / Build modules: sql - extended tests
Process completed with exit code 18.
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-94e942a7-4fa9-43fa-982f-e2c811d830a8/state/0/2 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-174d2ccd-a784-4f9c-a35a-d740241e3be7/state/0/4 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-cf08899b-2762-4fc6-95e2-00221569049e/state/0/1 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-174d2ccd-a784-4f9c-a35a-d740241e3be7/state/0/0 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-9c3a3122-d3c1-48c5-91fe-a3aef4644eae/state/0/2 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-9c3a3122-d3c1-48c5-91fe-a3aef4644eae/state/0/1 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-cf08899b-2762-4fc6-95e2-00221569049e/state/0/2 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-174d2ccd-a784-4f9c-a35a-d740241e3be7/state/0/1 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-9c3a3122-d3c1-48c5-91fe-a3aef4644eae/state/0/3 does not exist
|
Run / Build modules: sql - slow tests
File file:/home/runner/work/spark/spark/target/tmp/spark-cf08899b-2762-4fc6-95e2-00221569049e/state/0/3 does not exist
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q14a):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #18
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #19
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #20
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #21
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #22
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q14b):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q17):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q23b):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #14
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #15
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #16
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk,c_first_name,c_last_name] #13
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 2516749, 20795378, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #15
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_sold_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #16
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #17
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #18
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #19
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #20
WholeStageCodegen (45)
Filter [c_customer_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_bill_customer_sk, 42), 1252627, 12169292, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [ws_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [ws_bill_customer_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #22
WholeStageCodegen (9)
Project [ws_bill_customer_sk,ws_sold_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #23
WholeStageCodegen (1)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q25):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q29):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #11
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #11
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #7
BroadcastExchange #14
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #14
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #16
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q50):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #7
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #9
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q54):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #3
ReusedSubquery [(d_month_seq + 3)] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #3
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_customer_sk, 42), 262672542, 67108864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [ss_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_customer_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #4
ReusedSubquery [(d_month_seq + 3)] #5
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #5
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #12
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #9
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q70):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #6
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #7
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
|
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q93):
TPCDSV1_4_PlanStabilityWithStatsSuite#L290
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
Filter [ss_item_sk,ss_ticket_number]
Subquery #1
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_reason_sk,sr_ticket_number]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]
|
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q14):
TPCDSV2_7_PlanStabilityWithStatsSuite#L316
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
|
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q14a):
TPCDSV2_7_PlanStabilityWithStatsSuite#L316
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #17
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #17
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #18
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #20
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #21
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (467)
HashAggregate [channel,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel] #22
WholeStageCodegen (466)
HashAggregate [channel,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (584)
HashAggregate [sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),channel,i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange #23
WholeStageCodegen (583)
HashAggregate [sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #17
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #18
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #18
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #16
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #20
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #22
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #23
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #24
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #25
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #26
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_clas
|
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q70a):
TPCDSV2_7_PlanStabilityWithStatsSuite#L316
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #9
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #10
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #7
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #8
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #9
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #10
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #11
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
|
Run / Build modules: pyspark-core, pyspark-streaming
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
|
Run / Build modules: pyspark-errors
No files were found with the provided path: **/target/test-reports/*.xml. No artifacts will be uploaded.
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
test-results-catalyst, hive-thriftserver--8-hadoop3-hive2.3
Expired
|
3.01 MB |
|
test-results-core, unsafe, kvstore, avro, network-common, network-shuffle, repl, launcher, examples, sketch, graphx--8-hadoop3-hive2.3
Expired
|
154 KB |
|
test-results-docker-integration--8-hadoop3-hive2.3
Expired
|
133 KB |
|
test-results-hive-- other tests-8-hadoop3-hive2.3
Expired
|
1.1 MB |
|
test-results-hive-- slow tests-8-hadoop3-hive2.3
Expired
|
943 KB |
|
test-results-pyspark-connect--8-hadoop3-hive2.3
Expired
|
319 KB |
|
test-results-pyspark-mllib, pyspark-ml, pyspark-ml-connect--8-hadoop3-hive2.3
Expired
|
424 KB |
|
test-results-pyspark-pandas--8-hadoop3-hive2.3
Expired
|
1.06 MB |
|
test-results-pyspark-pandas-connect--8-hadoop3-hive2.3
Expired
|
1.44 MB |
|
test-results-pyspark-pandas-slow--8-hadoop3-hive2.3
Expired
|
1.27 MB |
|
test-results-pyspark-pandas-slow-connect--8-hadoop3-hive2.3
Expired
|
745 KB |
|
test-results-pyspark-sql, pyspark-resource, pyspark-testing--8-hadoop3-hive2.3
Expired
|
360 KB |
|
test-results-sparkr--8-hadoop3-hive2.3
Expired
|
280 KB |
|
test-results-sql-- extended tests-8-hadoop3-hive2.3
Expired
|
4.3 MB |
|
test-results-sql-- other tests-8-hadoop3-hive2.3
Expired
|
4.66 MB |
|
test-results-sql-- slow tests-8-hadoop3-hive2.3
Expired
|
3.17 MB |
|
test-results-streaming, sql-kafka-0-10, streaming-kafka-0-10, mllib-local, mllib, yarn, mesos, kubernetes, hadoop-cloud, spark-ganglia-lgpl, connect, protobuf--8-hadoop3-hive2.3
Expired
|
2.33 MB |
|
test-results-tpcds--8-hadoop3-hive2.3
Expired
|
22.6 KB |
|
unit-tests-log-sql-- extended tests-8-hadoop3-hive2.3
Expired
|
512 MB |
|