[SPARK-44781][SQL] Runtime filter should supports reuse exchange if i… #1500
GitHub Actions / Report test results
failed
Aug 12, 2023 in 0s
40005 tests run, 1047 skipped, 13 failed.
Annotations
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q14a)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #18
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #19
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #20
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #21
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #22
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #14
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum(sales),sum(number_sales)]
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum(sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id] #1
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,spark_grouping_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
Expand [sales,number_sales,channel,i_brand_id,i_class_id,i_category_id]
InputAdapter
Union
WholeStageCodegen (38)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #8
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #9
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
BroadcastExchange #12
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #13
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #8
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #12
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #14
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (76)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #18
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #19
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
WholeStageCodegen (114)
Project [sales,number_sales,i_brand_id,i_class_id,i_category_id]
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #20
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #21
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #22
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #4
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q14b)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q14b.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14b.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q17)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q17.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q17.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_state,store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_state,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2] [count(ss_quantity),avg(ss_quantity),stddev_samp(cast(ss_quantity as double)),count(sr_return_quantity),avg(sr_return_quantity),stddev_samp(cast(sr_return_quantity as double)),count(cs_quantity),avg(cs_quantity),stddev_samp(cast(cs_quantity as double)),store_sales_quantitycount,store_sales_quantityave,store_sales_quantitystdev,store_sales_quantitycov,as_store_returns_quantitycount,as_store_returns_quantityave,as_store_returns_quantitystdev,store_returns_quantitycov,catalog_sales_quantitycount,catalog_sales_quantityave,catalog_sales_quantitystdev,catalog_sales_quantitycov,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
InputAdapter
Exchange [i_item_id,i_item_desc,s_state] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_state,ss_quantity,sr_return_quantity,cs_quantity] [count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2,count,sum,count,n,avg,m2]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_state,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 4065836, 29674263, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_quarter_name,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_quarter_name]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 23056661, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q23b)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #14
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #15
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #16
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk,c_first_name,c_last_name] #13
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 2516749, 20795378, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #15
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_sold_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #16
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #17
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #18
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #19
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #20
WholeStageCodegen (45)
Filter [c_customer_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_bill_customer_sk, 42), 1252627, 12169292, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [ws_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [ws_bill_customer_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #22
WholeStageCodegen (9)
Project [ws_bill_customer_sk,ws_sold_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #23
WholeStageCodegen (1)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q23b.sf100/explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #14
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #15
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #16
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk,c_first_name,c_last_name] #13
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q23b.sf100.actual.explain.txt
TakeOrderedAndProject [c_last_name,c_first_name,sales]
Union
WholeStageCodegen (27)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #1
WholeStageCodegen (26)
HashAggregate [c_last_name,c_first_name,cs_quantity,cs_list_price] [sum,isEmpty,sum,isEmpty]
Project [cs_quantity,cs_list_price,c_first_name,c_last_name]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (17)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #2
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #3
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #5
WholeStageCodegen (4)
Project [ss_item_sk,d_date]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #6
WholeStageCodegen (1)
Project [d_date_sk,d_date]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_date,d_year]
InputAdapter
ReusedExchange [d_date_sk,d_date] #6
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (6)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_desc]
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
Subquery #3
WholeStageCodegen (7)
HashAggregate [max] [max(csales),tpcds_cmax,max]
InputAdapter
Exchange #10
WholeStageCodegen (6)
HashAggregate [csales] [max,max]
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),csales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #11
WholeStageCodegen (2)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #12
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #12
InputAdapter
WholeStageCodegen (5)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (11)
Project [ss_customer_sk,ss_quantity,ss_sales_price]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_quantity,ss_sales_price,ss_sold_date_sk]
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #9
WholeStageCodegen (13)
Filter [c_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (25)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (19)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #13
WholeStageCodegen (18)
Filter [c_customer_sk]
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 2516749, 20795378, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [cs_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [cs_bill_customer_sk]
InputAdapter
Exchange [cs_bill_customer_sk] #15
WholeStageCodegen (9)
Project [cs_bill_customer_sk,cs_sold_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
SortMergeJoin [cs_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [cs_item_sk]
InputAdapter
Exchange [cs_item_sk] #16
WholeStageCodegen (1)
Filter [cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (24)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (21)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (23)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
WholeStageCodegen (54)
HashAggregate [c_last_name,c_first_name,sum,isEmpty] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),sales,sum,isEmpty]
InputAdapter
Exchange [c_last_name,c_first_name] #17
WholeStageCodegen (53)
HashAggregate [c_last_name,c_first_name,ws_quantity,ws_list_price] [sum,isEmpty,sum,isEmpty]
Project [ws_quantity,ws_list_price,c_first_name,c_last_name]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (44)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (37)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #18
WholeStageCodegen (36)
Project [ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (29)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #19
WholeStageCodegen (28)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
WholeStageCodegen (35)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (32)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (34)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (42)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (39)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (41)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
WholeStageCodegen (52)
SortMergeJoin [c_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (46)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #20
WholeStageCodegen (45)
Filter [c_customer_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_bill_customer_sk, 42), 1252627, 12169292, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [ws_bill_customer_sk] [buf,buf]
WholeStageCodegen (17)
Project [ws_bill_customer_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
SortMergeJoin [ws_bill_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (10)
Sort [ws_bill_customer_sk]
InputAdapter
Exchange [ws_bill_customer_sk] #22
WholeStageCodegen (9)
Project [ws_bill_customer_sk,ws_sold_date_sk]
Project [ws_bill_customer_sk,ws_sold_date_sk]
SortMergeJoin [ws_item_sk,item_sk]
InputAdapter
WholeStageCodegen (2)
Sort [ws_item_sk]
InputAdapter
Exchange [ws_item_sk] #23
WholeStageCodegen (1)
Filter [ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
InputAdapter
WholeStageCodegen (8)
Sort [item_sk]
Project [item_sk]
Filter [cnt]
HashAggregate [_groupingexpression,i_item_sk,d_date,count] [count(1),item_sk,cnt,count]
HashAggregate [_groupingexpression,i_item_sk,d_date] [count,count]
Project [d_date,i_item_sk,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk]
InputAdapter
ReusedExchange [ss_item_sk,d_date] #5
InputAdapter
WholeStageCodegen (7)
Sort [i_item_sk]
InputAdapter
ReusedExchange [i_item_sk,i_item_desc] #7
InputAdapter
WholeStageCodegen (15)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (12)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (14)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_first_name,c_last_name]
InputAdapter
WholeStageCodegen (51)
Sort [c_customer_sk]
Project [c_customer_sk]
Filter [ssales]
ReusedSubquery [tpcds_cmax] #3
HashAggregate [c_customer_sk,sum,isEmpty] [sum((cast(ss_quantity as decimal(10,0)) * ss_sales_price)),ssales,sum,isEmpty]
HashAggregate [c_customer_sk,ss_quantity,ss_sales_price] [sum,isEmpty,sum,isEmpty]
Project [ss_quantity,ss_sales_price,c_customer_sk]
SortMergeJoin [ss_customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (48)
Sort [ss_customer_sk]
InputAdapter
ReusedExchange [ss_customer_sk,ss_quantity,ss_sales_price] #8
InputAdapter
WholeStageCodegen (50)
Sort [c_customer_sk]
InputAdapter
ReusedExchange [c_customer_sk] #9
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q25)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q25.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q25.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_profit,store_returns_loss,catalog_sales_profit]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(UnscaledValue(ss_net_profit)),sum(UnscaledValue(sr_net_loss)),sum(UnscaledValue(cs_net_profit)),store_sales_profit,store_returns_loss,catalog_sales_profit,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_net_profit,sr_net_loss,cs_net_profit] [sum,sum,sum,sum,sum,sum]
Project [ss_net_profit,sr_net_loss,cs_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_net_loss]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_net_profit,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 2834632, 22720219, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 16074715, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_net_loss,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #15
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_net_profit]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_net_profit,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q29)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #11
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #11
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #7
BroadcastExchange #14
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #14
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #16
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q29.sf100/explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #7
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #8
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #10
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #11
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #11
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q29.sf100.actual.explain.txt
TakeOrderedAndProject [i_item_id,i_item_desc,s_store_id,s_store_name,store_sales_quantity,store_returns_quantity,catalog_sales_quantity]
WholeStageCodegen (18)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,sum,sum,sum] [sum(ss_quantity),sum(sr_return_quantity),sum(cs_quantity),store_sales_quantity,store_returns_quantity,catalog_sales_quantity,sum,sum,sum]
InputAdapter
Exchange [i_item_id,i_item_desc,s_store_id,s_store_name] #1
WholeStageCodegen (17)
HashAggregate [i_item_id,i_item_desc,s_store_id,s_store_name,ss_quantity,sr_return_quantity,cs_quantity] [sum,sum,sum,sum,sum,sum]
Project [ss_quantity,sr_return_quantity,cs_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [sr_customer_sk,sr_item_sk,cs_bill_customer_sk,cs_item_sk]
InputAdapter
WholeStageCodegen (13)
Sort [sr_customer_sk,sr_item_sk]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk] #2
WholeStageCodegen (12)
Project [ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc,sr_item_sk,sr_customer_sk,sr_return_quantity]
SortMergeJoin [ss_customer_sk,ss_item_sk,ss_ticket_number,sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
WholeStageCodegen (8)
Sort [ss_customer_sk,ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_customer_sk,ss_item_sk,ss_ticket_number] #3
WholeStageCodegen (7)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name,i_item_id,i_item_desc]
SortMergeJoin [ss_item_sk,i_item_sk]
InputAdapter
WholeStageCodegen (4)
Sort [ss_item_sk]
InputAdapter
Exchange [ss_item_sk] #4
WholeStageCodegen (3)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,s_store_id,s_store_name]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk,ss_item_sk,ss_ticket_number,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
Subquery #5
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 1417316, 13404864, 0, 0),bloomFilter,buf]
Exchange #9
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_quantity,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #10
WholeStageCodegen (2)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_id,s_store_name]
InputAdapter
WholeStageCodegen (6)
Sort [i_item_sk]
InputAdapter
Exchange [i_item_sk] #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_item_id,i_item_desc]
InputAdapter
WholeStageCodegen (11)
Sort [sr_customer_sk,sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_customer_sk,sr_item_sk,sr_ticket_number] #12
WholeStageCodegen (10)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_customer_sk,sr_item_sk,sr_ticket_number]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_bill_customer_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #13
ObjectHashAggregate [cs_bill_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_bill_customer_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_bill_customer_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #7
BroadcastExchange #14
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #14
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 88086186, 67108864, 0, 0),bloomFilter,buf]
Exchange #15
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
WholeStageCodegen (16)
Sort [cs_bill_customer_sk,cs_item_sk]
InputAdapter
Exchange [cs_bill_customer_sk,cs_item_sk] #16
WholeStageCodegen (15)
Project [cs_bill_customer_sk,cs_item_sk,cs_quantity]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_bill_customer_sk,cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_quantity,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #7
InputAdapter
ReusedExchange [d_date_sk] #14
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q50)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #7
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #9
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q50.sf100/explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q50.sf100.actual.explain.txt
TakeOrderedAndProject [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ]
WholeStageCodegen (9)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sum,sum,sum,sum,sum] [sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) <= 30) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 30) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 60)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 60) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 90)) THEN 1 ELSE 0 END),sum(CASE WHEN (((sr_returned_date_sk - ss_sold_date_sk) > 90) AND ((sr_returned_date_sk - ss_sold_date_sk) <= 120)) THEN 1 ELSE 0 END),sum(CASE WHEN ((sr_returned_date_sk - ss_sold_date_sk) > 120) THEN 1 ELSE 0 END),30 days ,31 - 60 days ,61 - 90 days ,91 - 120 days ,>120 days ,sum,sum,sum,sum,sum]
InputAdapter
Exchange [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip] #1
WholeStageCodegen (8)
HashAggregate [s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip,sr_returned_date_sk,ss_sold_date_sk] [sum,sum,sum,sum,sum,sum,sum,sum,sum,sum]
Project [ss_sold_date_sk,sr_returned_date_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [sr_returned_date_sk,ss_store_sk,ss_sold_date_sk]
SortMergeJoin [sr_ticket_number,sr_item_sk,sr_customer_sk,ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (3)
Sort [sr_ticket_number,sr_item_sk,sr_customer_sk]
InputAdapter
Exchange [sr_ticket_number,sr_item_sk,sr_customer_sk] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
WholeStageCodegen (5)
Sort [ss_ticket_number,ss_item_sk,ss_customer_sk]
InputAdapter
Exchange [ss_ticket_number,ss_item_sk,ss_customer_sk] #4
WholeStageCodegen (4)
Filter [ss_ticket_number,ss_item_sk,ss_customer_sk,ss_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_ticket_number,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_item_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_customer_sk, 42), 443806, 5270048, 0, 0),bloomFilter,buf]
Exchange #7
ObjectHashAggregate [sr_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_customer_sk]
BroadcastHashJoin [sr_returned_date_sk,d_date_sk]
Project [sr_customer_sk,sr_returned_date_sk]
Filter [sr_ticket_number,sr_item_sk,sr_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_customer_sk,sr_ticket_number,sr_returned_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_store_sk,ss_ticket_number,ss_sold_date_sk]
InputAdapter
BroadcastExchange #8
WholeStageCodegen (6)
Filter [d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk]
InputAdapter
BroadcastExchange #9
WholeStageCodegen (7)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_store_name,s_company_id,s_street_number,s_street_name,s_street_type,s_suite_number,s_city,s_county,s_state,s_zip]
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q54)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #3
ReusedSubquery [(d_month_seq + 3)] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #3
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_customer_sk, 42), 262672542, 67108864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [ss_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_customer_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #4
ReusedSubquery [(d_month_seq + 3)] #5
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #5
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #12
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #9Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q54.sf100/explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #8
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #3
ReusedSubquery [(d_month_seq + 3)] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #3
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q54.sf100.actual.explain.txt
TakeOrderedAndProject [segment,num_customers,segment_base]
WholeStageCodegen (16)
HashAggregate [segment,count] [count(1),num_customers,segment_base,count]
InputAdapter
Exchange [segment] #1
WholeStageCodegen (15)
HashAggregate [segment] [count,count]
HashAggregate [c_customer_sk,sum] [sum(UnscaledValue(ss_ext_sales_price)),segment,sum]
HashAggregate [c_customer_sk,ss_ext_sales_price] [sum,sum]
Project [c_customer_sk,ss_ext_sales_price]
SortMergeJoin [c_customer_sk,ss_customer_sk]
InputAdapter
WholeStageCodegen (11)
Sort [c_customer_sk]
Project [c_customer_sk]
BroadcastHashJoin [ca_address_sk,c_current_addr_sk]
InputAdapter
BroadcastExchange #2
WholeStageCodegen (2)
Project [ca_address_sk]
BroadcastHashJoin [ca_county,ca_state,s_county,s_state]
Filter [ca_address_sk,ca_county,ca_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer_address [ca_address_sk,ca_county,ca_state]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Filter [s_county,s_state]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_county,s_state]
HashAggregate [c_customer_sk,c_current_addr_sk]
HashAggregate [c_customer_sk,c_current_addr_sk]
Project [c_customer_sk,c_current_addr_sk]
SortMergeJoin [customer_sk,c_customer_sk]
InputAdapter
WholeStageCodegen (8)
Sort [customer_sk]
InputAdapter
Exchange [customer_sk] #4
WholeStageCodegen (7)
Project [customer_sk]
BroadcastHashJoin [item_sk,i_item_sk]
Project [customer_sk,item_sk]
BroadcastHashJoin [sold_date_sk,d_date_sk]
InputAdapter
Union
WholeStageCodegen (3)
Project [cs_sold_date_sk,cs_bill_customer_sk,cs_item_sk]
Filter [cs_item_sk,cs_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_bill_customer_sk,cs_item_sk,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #5
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_moy,d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
WholeStageCodegen (4)
Project [ws_sold_date_sk,ws_bill_customer_sk,ws_item_sk]
Filter [ws_item_sk,ws_bill_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_bill_customer_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #5
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [i_item_sk]
Filter [i_category,i_class,i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_class,i_category]
InputAdapter
WholeStageCodegen (10)
Sort [c_customer_sk]
InputAdapter
Exchange [c_customer_sk] #7
WholeStageCodegen (9)
Filter [c_customer_sk,c_current_addr_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_customer_sk, 42), 262672542, 67108864, 0, 0),bloomFilter,buf]
Exchange #8
ObjectHashAggregate [ss_customer_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_customer_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ReusedSubquery [(d_month_seq + 1)] #4
ReusedSubquery [(d_month_seq + 3)] #5
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
Subquery #4
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 1)]
InputAdapter
Exchange [(d_month_seq + 1)] #10
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 1)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
Subquery #5
WholeStageCodegen (2)
HashAggregate [(d_month_seq + 3)]
InputAdapter
Exchange [(d_month_seq + 3)] #11
WholeStageCodegen (1)
HashAggregate [(d_month_seq + 3)]
Project [d_month_seq]
Filter [d_year,d_moy]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_month_seq,d_year,d_moy]
InputAdapter
ReusedExchange [d_date_sk] #9
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.customer [c_customer_sk,c_current_addr_sk]
InputAdapter
WholeStageCodegen (14)
Sort [ss_customer_sk]
InputAdapter
Exchange [ss_customer_sk] #12
WholeStageCodegen (13)
Project [ss_customer_sk,ss_ext_sales_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_customer_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_customer_sk,ss_ext_sales_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #9
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q70)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #6
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #7
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q70.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #6
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #7
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (11)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [_w0,_w1,_w2]
WholeStageCodegen (10)
Sort [_w1,_w2,_w0]
InputAdapter
Exchange [_w1,_w2] #1
WholeStageCodegen (9)
HashAggregate [s_state,s_county,spark_grouping_id,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,lochierarchy,_w0,_w1,_w2,sum]
InputAdapter
Exchange [s_state,s_county,spark_grouping_id] #2
WholeStageCodegen (8)
HashAggregate [s_state,s_county,spark_grouping_id,ss_net_profit] [sum,sum]
Expand [ss_net_profit,s_state,s_county]
Project [ss_net_profit,s_state,s_county]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #3
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #4
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #3
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 290 in TPCDSV1_4_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV1_4_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v1.4/q93)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
Filter [ss_item_sk,ss_ticket_number]
Subquery #1
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_reason_sk,sr_ticket_number]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q93.sf100/explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q93.sf100.actual.explain.txt
TakeOrderedAndProject [sumsales,ss_customer_sk]
WholeStageCodegen (7)
HashAggregate [ss_customer_sk,sum,isEmpty] [sum(act_sales),sumsales,sum,isEmpty]
InputAdapter
Exchange [ss_customer_sk] #1
WholeStageCodegen (6)
HashAggregate [ss_customer_sk,act_sales] [sum,isEmpty,sum,isEmpty]
Project [ss_customer_sk,sr_return_quantity,ss_quantity,ss_sales_price]
SortMergeJoin [sr_item_sk,sr_ticket_number,ss_item_sk,ss_ticket_number]
InputAdapter
WholeStageCodegen (3)
Sort [sr_item_sk,sr_ticket_number]
InputAdapter
Exchange [sr_item_sk,sr_ticket_number] #2
WholeStageCodegen (2)
Project [sr_item_sk,sr_ticket_number,sr_return_quantity]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_return_quantity,sr_returned_date_sk]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (1)
Project [r_reason_sk]
Filter [r_reason_desc,r_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.reason [r_reason_sk,r_reason_desc]
InputAdapter
WholeStageCodegen (5)
Sort [ss_item_sk,ss_ticket_number]
InputAdapter
Exchange [ss_item_sk,ss_ticket_number] #4
WholeStageCodegen (4)
Project [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price]
Filter [ss_item_sk,ss_ticket_number]
Subquery #1
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_item_sk, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #5
ObjectHashAggregate [sr_item_sk] [buf,buf]
WholeStageCodegen (2)
Project [sr_item_sk]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_item_sk,sr_reason_sk]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(sr_ticket_number, 42), 496193, 5776894, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [sr_ticket_number] [buf,buf]
WholeStageCodegen (2)
Project [sr_ticket_number]
BroadcastHashJoin [sr_reason_sk,r_reason_sk]
Project [sr_reason_sk,sr_ticket_number]
Filter [sr_item_sk,sr_ticket_number,sr_reason_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_returns [sr_item_sk,sr_reason_sk,sr_ticket_number,sr_returned_date_sk]
InputAdapter
ReusedExchange [r_reason_sk] #3
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_customer_sk,ss_ticket_number,ss_quantity,ss_sales_price,ss_sold_date_sk]
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.$anonfun$new$4(PlanStabilitySuite.scala:298)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV1_4_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:290)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 316 in TPCDSV2_7_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q14)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14.sf100/explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #14
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #15
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #16
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #6
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #6
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #17
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14.sf100.actual.explain.txt
TakeOrderedAndProject [i_brand_id,i_class_id,i_category_id,channel,sales,number_sales,channel,i_brand_id,i_class_id,i_category_id,sales,number_sales]
WholeStageCodegen (76)
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
Filter [sales]
Subquery #5
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #15
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #1
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #2
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #2
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
BroadcastExchange #3
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #4
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #5
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #6
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #3
BroadcastExchange #7
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #8
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #9
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #10
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
BroadcastExchange #11
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #7
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #11
InputAdapter
ReusedExchange [d_date_sk] #2
InputAdapter
BroadcastExchange #13
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #4
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #14
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #2
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
BroadcastExchange #16
WholeStageCodegen (75)
Filter [sales]
ReusedSubquery [average_sales] #5
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #17
WholeStageCodegen (74)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #6
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_week_seq,d_date_sk]
ReusedSubquery [d_week_seq] #7
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_week_seq]
Subquery #7
WholeStageCodegen (1)
Project [d_week_seq]
Filter [d_year,d_moy,d_dom]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_week_seq,d_year,d_moy,d_dom]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
InputAdapter
BroadcastExchange #19
WholeStageCodegen (73)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
Subquery #8
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 275041999, 67108864, 0, 0),bloomFilter,buf]
Exchange #20
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #6
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #3
InputAdapter
ReusedExchange [d_date_sk] #18
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #3
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.$anonfun$new$8(PlanStabilitySuite.scala:324)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Check failure on line 316 in TPCDSV2_7_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q14a)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #17
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #17
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #18
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #20
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #21
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (467)
HashAggregate [channel,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel] #22
WholeStageCodegen (466)
HashAggregate [channel,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (584)
HashAggregate [sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),channel,i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange #23
WholeStageCodegen (583)
HashAggregate [sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #17
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #18
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #18
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #16
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #20
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #22
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #23
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #24
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #25
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #26
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_clasRaw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q14a.sf100/explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #3
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #16
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #4
BroadcastExchange #17
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #17
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #4
InputAdapter
ReusedExchange [d_date_sk] #17
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #18
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #3
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #15
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #20
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #21
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (467)
HashAggregate [channel,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel] #22
WholeStageCodegen (466)
HashAggregate [channel,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (584)
HashAggregate [sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),channel,i_brand_id,i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange #23
WholeStageCodegen (583)
HashAggregate [sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q14a.sf100.actual.explain.txt
TakeOrderedAndProject [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
WholeStageCodegen (586)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales] #1
WholeStageCodegen (585)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum_sales,number_sales]
InputAdapter
Union
WholeStageCodegen (116)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id,i_category_id] #2
WholeStageCodegen (115)
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
InputAdapter
Union
WholeStageCodegen (38)
Filter [sales]
Subquery #4
WholeStageCodegen (8)
HashAggregate [sum,count] [avg((cast(quantity as decimal(10,0)) * list_price)),average_sales,sum,count]
InputAdapter
Exchange #17
WholeStageCodegen (7)
HashAggregate [quantity,list_price] [sum,count,sum,count]
InputAdapter
Union
WholeStageCodegen (2)
Project [ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_quantity,ss_list_price,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
WholeStageCodegen (4)
Project [cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_quantity,cs_list_price,cs_sold_date_sk]
SubqueryBroadcast [d_date_sk] #5
BroadcastExchange #18
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #18
WholeStageCodegen (6)
Project [ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #18
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ss_quantity as decimal(10,0)) * ss_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #3
WholeStageCodegen (37)
HashAggregate [i_brand_id,i_class_id,i_category_id,ss_quantity,ss_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ss_quantity,ss_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk,ss_quantity,ss_list_price]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_quantity,ss_list_price,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_moy,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year,d_moy]
InputAdapter
BroadcastExchange #5
WholeStageCodegen (17)
Project [i_item_sk]
BroadcastHashJoin [i_brand_id,i_class_id,i_category_id,brand_id,class_id,category_id]
Filter [i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (16)
SortMergeJoin [brand_id,class_id,category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (11)
Sort [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #7
WholeStageCodegen (10)
HashAggregate [brand_id,class_id,category_id]
InputAdapter
Exchange [brand_id,class_id,category_id] #8
WholeStageCodegen (9)
HashAggregate [brand_id,class_id,category_id]
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ss_item_sk,i_item_sk]
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #2
BroadcastExchange #9
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_year,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_year]
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #10
WholeStageCodegen (8)
SortMergeJoin [i_brand_id,i_class_id,i_category_id,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (3)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #11
WholeStageCodegen (2)
Filter [i_item_sk,i_brand_id,i_class_id,i_category_id]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (7)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #12
WholeStageCodegen (6)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
BroadcastExchange #13
WholeStageCodegen (5)
Filter [i_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
WholeStageCodegen (15)
Sort [i_brand_id,i_class_id,i_category_id]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #14
WholeStageCodegen (14)
Project [i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #2
InputAdapter
ReusedExchange [d_date_sk] #9
InputAdapter
ReusedExchange [i_item_sk,i_brand_id,i_class_id,i_category_id] #13
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #15
WholeStageCodegen (36)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #3
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_item_sk, 42), 5012524, 36583609, 0, 0),bloomFilter,buf]
Exchange #16
ObjectHashAggregate [ss_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ss_item_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Project [ss_item_sk,ss_sold_date_sk]
Filter [ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ss_item_sk,ss_item_sk]
Filter [ss_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_item_sk,ss_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (76)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(cs_quantity as decimal(10,0)) * cs_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #19
WholeStageCodegen (75)
HashAggregate [i_brand_id,i_class_id,i_category_id,cs_quantity,cs_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [cs_quantity,cs_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [cs_item_sk,i_item_sk]
Project [cs_item_sk,cs_quantity,cs_list_price]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_quantity,cs_list_price,cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #20
WholeStageCodegen (74)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #6
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(cs_item_sk, 42), 2517915, 20802585, 0, 0),bloomFilter,buf]
Exchange #21
ObjectHashAggregate [cs_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [cs_item_sk]
BroadcastHashJoin [cs_sold_date_sk,d_date_sk]
Project [cs_item_sk,cs_sold_date_sk]
Filter [cs_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [cs_item_sk,ss_item_sk]
Filter [cs_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.catalog_sales [cs_item_sk,cs_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (114)
Filter [sales]
ReusedSubquery [average_sales] #4
HashAggregate [i_brand_id,i_class_id,i_category_id,sum,isEmpty,count] [sum((cast(ws_quantity as decimal(10,0)) * ws_list_price)),count(1),channel,sales,number_sales,sum,isEmpty,count]
InputAdapter
Exchange [i_brand_id,i_class_id,i_category_id] #22
WholeStageCodegen (113)
HashAggregate [i_brand_id,i_class_id,i_category_id,ws_quantity,ws_list_price] [sum,isEmpty,count,sum,isEmpty,count]
Project [ws_quantity,ws_list_price,i_brand_id,i_class_id,i_category_id]
BroadcastHashJoin [ws_item_sk,i_item_sk]
Project [ws_item_sk,ws_quantity,ws_list_price]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_quantity,ws_list_price,ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #23
WholeStageCodegen (112)
BroadcastHashJoin [i_item_sk,ss_item_sk]
Filter [i_item_sk]
Subquery #7
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ws_item_sk, 42), 1252939, 12171674, 0, 0),bloomFilter,buf]
Exchange #24
ObjectHashAggregate [ws_item_sk] [buf,buf]
WholeStageCodegen (19)
Project [ws_item_sk]
BroadcastHashJoin [ws_sold_date_sk,d_date_sk]
Project [ws_item_sk,ws_sold_date_sk]
Filter [ws_sold_date_sk]
ReusedSubquery [d_date_sk] #1
BroadcastHashJoin [ws_item_sk,ss_item_sk]
Filter [ws_item_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.web_sales [ws_item_sk,ws_sold_date_sk]
InputAdapter
ReusedExchange [ss_item_sk] #5
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.item [i_item_sk,i_brand_id,i_class_id,i_category_id]
InputAdapter
ReusedExchange [ss_item_sk] #5
WholeStageCodegen (233)
HashAggregate [channel,i_brand_id,i_class_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id,i_class_id] #25
WholeStageCodegen (232)
HashAggregate [channel,i_brand_id,i_class_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] [sum(sales),sum(number_salesL),sum_sales,number_sales,sum,isEmpty,sum]
InputAdapter
ReusedExchange [channel,i_brand_id,i_class_id,i_category_id,sum,isEmpty,sum] #2
WholeStageCodegen (350)
HashAggregate [channel,i_brand_id,sum,isEmpty,sum] [sum(sum_sales),sum(number_salesL),i_class_id,i_category_id,sum(sum_sales),sum(number_sales),sum,isEmpty,sum]
InputAdapter
Exchange [channel,i_brand_id] #26
WholeStageCodegen (349)
HashAggregate [channel,i_brand_id,sum_sales,number_sales] [sum,isEmpty,sum,sum,isEmpty,sum]
HashAggregate [c
Check failure on line 316 in TPCDSV2_7_PlanStabilityWithStatsSuite
github-actions / Report test results
TPCDSV2_7_PlanStabilityWithStatsSuite.check simplified sf100 (tpcds-v2.7.0/q70a)
org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #9
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #10
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #7
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #8
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #9
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #10
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #11
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3Raw output
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException:
Plans did not match:
last approved simplified plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/simplified.txt
last approved explain plan: /home/runner/work/spark/spark/sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v2_7/q70a.sf100/explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #6
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #7
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #8
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #9
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #10
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
actual simplified plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.simplified.txt
actual explain plan: /home/runner/work/spark/spark/target/tmp/q70a.sf100.actual.explain.txt
TakeOrderedAndProject [lochierarchy,s_state,rank_within_parent,total_sum,s_county]
WholeStageCodegen (33)
Project [total_sum,s_state,s_county,lochierarchy,rank_within_parent]
InputAdapter
Window [total_sum,lochierarchy,_w0]
WholeStageCodegen (32)
Sort [lochierarchy,_w0,total_sum]
InputAdapter
Exchange [lochierarchy,_w0] #1
WholeStageCodegen (31)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy] [_w0]
InputAdapter
Exchange [total_sum,s_state,s_county,g_state,g_county,lochierarchy] #2
WholeStageCodegen (30)
HashAggregate [total_sum,s_state,s_county,g_state,g_county,lochierarchy]
InputAdapter
Union
WholeStageCodegen (9)
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,g_state,g_county,lochierarchy,sum]
InputAdapter
Exchange [s_state,s_county] #3
WholeStageCodegen (8)
HashAggregate [s_state,s_county,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_county,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
SubqueryBroadcast [d_date_sk] #1
BroadcastExchange #4
WholeStageCodegen (1)
Project [d_date_sk]
Filter [d_month_seq,d_date_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.date_dim [d_date_sk,d_month_seq]
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #5
WholeStageCodegen (7)
BroadcastHashJoin [s_state,s_state]
Filter [s_store_sk]
Subquery #2
ObjectHashAggregate [buf] [bloom_filter_agg(xxhash64(ss_store_sk, 42), 54151237, 67108864, 0, 0),bloomFilter,buf]
Exchange #6
ObjectHashAggregate [ss_store_sk] [buf,buf]
WholeStageCodegen (2)
Project [ss_store_sk]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_county,s_state]
InputAdapter
BroadcastExchange #7
WholeStageCodegen (6)
Project [s_state]
Filter [ranking]
InputAdapter
Window [_w0,s_state]
WindowGroupLimit [s_state,_w0]
WholeStageCodegen (5)
Sort [s_state,_w0]
HashAggregate [sum] [sum(UnscaledValue(ss_net_profit)),_w0,s_state,sum]
InputAdapter
Exchange [s_state] #8
WholeStageCodegen (4)
HashAggregate [s_state,ss_net_profit] [sum,sum]
Project [ss_net_profit,s_state]
BroadcastHashJoin [ss_store_sk,s_store_sk]
Project [ss_store_sk,ss_net_profit]
BroadcastHashJoin [ss_sold_date_sk,d_date_sk]
Filter [ss_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store_sales [ss_store_sk,ss_net_profit,ss_sold_date_sk]
ReusedSubquery [d_date_sk] #1
InputAdapter
ReusedExchange [d_date_sk] #4
InputAdapter
BroadcastExchange #9
WholeStageCodegen (3)
Filter [s_store_sk]
ColumnarToRow
InputAdapter
Scan parquet spark_catalog.default.store [s_store_sk,s_state]
WholeStageCodegen (19)
HashAggregate [s_state,sum,isEmpty] [sum(total_sum),total_sum,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange [s_state] #10
WholeStageCodegen (18)
HashAggregate [s_state,total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
WholeStageCodegen (29)
HashAggregate [sum,isEmpty] [sum(total_sum),total_sum,s_state,s_county,g_state,g_county,lochierarchy,sum,isEmpty]
InputAdapter
Exchange #11
WholeStageCodegen (28)
HashAggregate [total_sum] [sum,isEmpty,sum,isEmpty]
HashAggregate [s_state,s_county,sum] [sum(UnscaledValue(ss_net_profit)),total_sum,sum]
InputAdapter
ReusedExchange [s_state,s_county,sum] #3
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1564)
at org.scalatest.Assertions.fail(Assertions.scala:933)
at org.scalatest.Assertions.fail$(Assertions.scala:929)
at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1564)
at org.apache.spark.sql.PlanStabilitySuite.checkWithApproved(PlanStabilitySuite.scala:156)
at org.apache.spark.sql.PlanStabilitySuite.$anonfun$testQuery$1(PlanStabilitySuite.scala:271)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.PlanStabilitySuite.testQuery(PlanStabilitySuite.scala:261)
at org.apache.spark.sql.PlanStabilitySuite.testQuery$(PlanStabilitySuite.scala:256)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.testQuery(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.$anonfun$new$8(PlanStabilitySuite.scala:324)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$5(AdaptiveTestUtils.scala:65)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)
at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)
at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)
at org.apache.spark.sql.TPCDSV2_7_PlanStabilityWithStatsSuite.withSQLConf(PlanStabilitySuite.scala:316)
at org.apache.spark.sql.execution.adaptive.DisableAdaptiveExecutionSuite.$anonfun$test$4(AdaptiveTestUtils.scala:65)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:227)
at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:69)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
at scala.collection.immutable.List.foreach(List.scala:431)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
at org.scalatest.Suite.run(Suite.scala:1114)
at org.scalatest.Suite.run$(Suite.scala:1096)
at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:69)
at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Loading