-
-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Handle dynamic steps in the evaluator (#200)
Signed-off-by: Juan Cruz Viotti <[email protected]>
- Loading branch information
Showing
3 changed files
with
34 additions
and
37 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6ba7628
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark (macos/llvm)
Compiler_Draft6_AdaptiveCard
32874253374.999966
ns/iter35087999957.999954
ns/iter0.94
Compiler_2019_09_OMC_JSON_V2
7445065750.000026
ns/iter7364888458.999986
ns/iter1.01
Evaluator_Draft4_Meta_1_No_Callback
316.1394086427567
ns/iter296.1244358352388
ns/iter1.07
Evaluator_Draft4_Required_Properties
450.6620471897142
ns/iter451.16744421819027
ns/iter1.00
Evaluator_Draft4_Many_Optional_Properties_Minimal_Match
29.346204515617565
ns/iter47.80612294599352
ns/iter0.61
Evaluator_Draft4_Few_Optional_Properties_Minimal_Match
16.286284268984875
ns/iter16.61379295712934
ns/iter0.98
Evaluator_Draft4_Items_Schema
846.5621298025235
ns/iter846.7285776901231
ns/iter1.00
Evaluator_Draft4_Nested_Object
4.398489997672311
ns/iter4.378634229480897
ns/iter1.00
Evaluator_Draft4_Properties_Triad_Optional
1104.7126506431478
ns/iter1226.2785436713689
ns/iter0.90
Evaluator_Draft4_Properties_Triad_Closed
792.4494203241684
ns/iter866.149028629775
ns/iter0.91
Evaluator_Draft4_Properties_Triad_Required
1091.4448500181416
ns/iter1094.2018319027748
ns/iter1.00
Evaluator_Draft4_Properties_Closed
121.82840435849756
ns/iter203.66677475000924
ns/iter0.60
Evaluator_Draft4_Non_Recursive_Ref
16.29041964869751
ns/iter16.292672645441897
ns/iter1.00
Evaluator_Draft4_Pattern_Properties_True
1227.6176474215551
ns/iter1262.2574522553184
ns/iter0.97
Evaluator_Draft4_Ref_To_Single_Property
17.926868357641332
ns/iter17.487818255449014
ns/iter1.03
Evaluator_Draft4_Additional_Properties_Type
27.402445994060905
ns/iter28.295128054264598
ns/iter0.97
Evaluator_Draft4_Nested_Oneof
105.15139021138607
ns/iter101.58861325151202
ns/iter1.04
Evaluator_Draft4_Long_Enum
24.683145991176023
ns/iter25.372809010334787
ns/iter0.97
Evaluator_Draft4_Type_Object
10.400092870201158
ns/iter10.432657396444796
ns/iter1.00
Evaluator_Draft6_Property_Names
331.0307148358441
ns/iter344.18004495004277
ns/iter0.96
Evaluator_Draft7_If_Then_Else
44.84361965934845
ns/iter45.24852014257997
ns/iter0.99
Evaluator_Draft7_Vercel_1
98866.23491837735
ns/iter99551.52203976469
ns/iter0.99
Evaluator_2019_09_Unevaluated_Properties
234.43542275415837
ns/iter239.77455704136364
ns/iter0.98
Evaluator_2019_09_OMC_JSON_V2_1
6773.275414380326
ns/iter7138.695379147711
ns/iter0.95
Evaluator_2020_12_Dynamic_Ref
712.5673270192636
ns/iter744.1682319213471
ns/iter0.96
This comment was automatically generated by workflow using github-action-benchmark.
6ba7628
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark (linux/llvm)
Compiler_Draft6_AdaptiveCard
57933851975.99997
ns/iter57902491956.00001
ns/iter1.00
Compiler_2019_09_OMC_JSON_V2
10993956199.000025
ns/iter11007720694
ns/iter1.00
Evaluator_Draft4_Meta_1_No_Callback
478.72336305802963
ns/iter485.8900804417328
ns/iter0.99
Evaluator_Draft4_Required_Properties
957.7069340217801
ns/iter956.9830314447219
ns/iter1.00
Evaluator_Draft4_Many_Optional_Properties_Minimal_Match
58.33495333838399
ns/iter58.69705252758853
ns/iter0.99
Evaluator_Draft4_Few_Optional_Properties_Minimal_Match
39.28950076407092
ns/iter38.85853045506177
ns/iter1.01
Evaluator_Draft4_Items_Schema
1501.3066449057028
ns/iter1519.4547887699023
ns/iter0.99
Evaluator_Draft4_Nested_Object
12.09205275682082
ns/iter12.305833660015493
ns/iter0.98
Evaluator_Draft4_Properties_Triad_Optional
1309.1830674469916
ns/iter1325.0447365987034
ns/iter0.99
Evaluator_Draft4_Properties_Triad_Closed
1024.0597257346692
ns/iter1030.230764006626
ns/iter0.99
Evaluator_Draft4_Properties_Triad_Required
1357.2495267278734
ns/iter1369.847654020241
ns/iter0.99
Evaluator_Draft4_Properties_Closed
172.84447143631616
ns/iter170.37340233390216
ns/iter1.01
Evaluator_Draft4_Non_Recursive_Ref
47.3755330993062
ns/iter47.38491713397765
ns/iter1.00
Evaluator_Draft4_Pattern_Properties_True
1977.3750106161124
ns/iter2061.9241182706046
ns/iter0.96
Evaluator_Draft4_Ref_To_Single_Property
39.725934085974785
ns/iter38.416839763054966
ns/iter1.03
Evaluator_Draft4_Additional_Properties_Type
76.47396712204572
ns/iter76.65638077313281
ns/iter1.00
Evaluator_Draft4_Nested_Oneof
191.61618376326635
ns/iter190.2045175072331
ns/iter1.01
Evaluator_Draft4_Long_Enum
34.7875848922764
ns/iter35.11093926277793
ns/iter0.99
Evaluator_Draft4_Type_Object
24.13156642440499
ns/iter24.543594672723792
ns/iter0.98
Evaluator_Draft6_Property_Names
649.8082558529774
ns/iter624.6762008583129
ns/iter1.04
Evaluator_Draft7_If_Then_Else
89.44994357483324
ns/iter89.56655036345732
ns/iter1.00
Evaluator_Draft7_Vercel_1
141986.1800239179
ns/iter150160.56724067335
ns/iter0.95
Evaluator_2019_09_Unevaluated_Properties
279.5282933130534
ns/iter285.70069478104836
ns/iter0.98
Evaluator_2019_09_OMC_JSON_V2_1
10436.90279111681
ns/iter10472.246625676395
ns/iter1.00
Evaluator_2020_12_Dynamic_Ref
1131.8964518490318
ns/iter1075.8061865317113
ns/iter1.05
This comment was automatically generated by workflow using github-action-benchmark.
6ba7628
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark (linux/gcc)
Evaluator_2020_12_Dynamic_Ref
1077.5724619727575
ns/iter1084.148469028361
ns/iter0.99
Evaluator_2019_09_Unevaluated_Properties
337.7914690348356
ns/iter349.9776965208393
ns/iter0.97
Evaluator_2019_09_OMC_JSON_V2_1
12824.18571905082
ns/iter13016.488932253244
ns/iter0.99
Evaluator_Draft7_If_Then_Else
79.80617312882747
ns/iter80.38128407644824
ns/iter0.99
Evaluator_Draft7_Vercel_1
149051.41849662238
ns/iter161636.7065217256
ns/iter0.92
Evaluator_Draft6_Property_Names
1256.1841736682943
ns/iter1320.169730344837
ns/iter0.95
Evaluator_Draft4_Meta_1_No_Callback
544.99518359693
ns/iter586.6146567888844
ns/iter0.93
Evaluator_Draft4_Required_Properties
1864.9251306247927
ns/iter1974.6285506898405
ns/iter0.94
Evaluator_Draft4_Many_Optional_Properties_Minimal_Match
52.71227745717979
ns/iter52.907227049779245
ns/iter1.00
Evaluator_Draft4_Few_Optional_Properties_Minimal_Match
41.54889875985314
ns/iter43.49640800730207
ns/iter0.96
Evaluator_Draft4_Items_Schema
1336.364236164436
ns/iter1338.85534250799
ns/iter1.00
Evaluator_Draft4_Nested_Object
8.177937381646633
ns/iter6.820188739599651
ns/iter1.20
Evaluator_Draft4_Properties_Triad_Optional
1361.7414123384053
ns/iter1398.3039726191378
ns/iter0.97
Evaluator_Draft4_Properties_Triad_Closed
1066.9785102862131
ns/iter1111.522762015356
ns/iter0.96
Evaluator_Draft4_Properties_Triad_Required
1434.7913154158143
ns/iter1468.6204527497794
ns/iter0.98
Evaluator_Draft4_Properties_Closed
212.71472332369993
ns/iter219.805344188962
ns/iter0.97
Evaluator_Draft4_Non_Recursive_Ref
59.011878098750415
ns/iter59.00494842698748
ns/iter1.00
Evaluator_Draft4_Pattern_Properties_True
2095.041575388007
ns/iter2222.7752493391945
ns/iter0.94
Evaluator_Draft4_Ref_To_Single_Property
45.97740634419776
ns/iter47.93170450237031
ns/iter0.96
Evaluator_Draft4_Additional_Properties_Type
113.62693423086438
ns/iter113.5416877605631
ns/iter1.00
Evaluator_Draft4_Nested_Oneof
156.7140709518004
ns/iter168.89045493634853
ns/iter0.93
Evaluator_Draft4_Long_Enum
26.990644426976097
ns/iter26.982814005936074
ns/iter1.00
Evaluator_Draft4_Type_Object
18.68449182122194
ns/iter19.591935664594114
ns/iter0.95
Compiler_2019_09_OMC_JSON_V2
11400943141.000084
ns/iter11373929497.999939
ns/iter1.00
Compiler_Draft6_AdaptiveCard
66186420902.00001
ns/iter65197155854.000015
ns/iter1.02
This comment was automatically generated by workflow using github-action-benchmark.
6ba7628
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark (macos/gcc)
Compiler_Draft6_AdaptiveCard
45999326944.3512
ns/iter48723938941.95557
ns/iter0.94
Compiler_2019_09_OMC_JSON_V2
8437051057.815552
ns/iter9046262025.83313
ns/iter0.93
Evaluator_Draft4_Meta_1_No_Callback
310.39710450552474
ns/iter321.3179259295507
ns/iter0.97
Evaluator_Draft4_Required_Properties
529.1378941506025
ns/iter542.882813282086
ns/iter0.97
Evaluator_Draft4_Many_Optional_Properties_Minimal_Match
32.575315646709804
ns/iter34.275650355771525
ns/iter0.95
Evaluator_Draft4_Few_Optional_Properties_Minimal_Match
19.420506930437327
ns/iter19.662920103242392
ns/iter0.99
Evaluator_Draft4_Items_Schema
932.7842495410666
ns/iter975.4220197948879
ns/iter0.96
Evaluator_Draft4_Nested_Object
4.1703415731794795
ns/iter4.172624492064155
ns/iter1.00
Evaluator_Draft4_Properties_Triad_Optional
1114.4984831819413
ns/iter1122.5498494975775
ns/iter0.99
Evaluator_Draft4_Properties_Triad_Closed
891.3909887756215
ns/iter902.5222248831755
ns/iter0.99
Evaluator_Draft4_Properties_Triad_Required
1105.2265452650315
ns/iter1167.6079714575817
ns/iter0.95
Evaluator_Draft4_Properties_Closed
124.20954752349449
ns/iter140.20024409479078
ns/iter0.89
Evaluator_Draft4_Non_Recursive_Ref
25.530977471349644
ns/iter29.132127684307324
ns/iter0.88
Evaluator_Draft4_Pattern_Properties_True
1699.562062296388
ns/iter1703.8387722850462
ns/iter1.00
Evaluator_Draft4_Ref_To_Single_Property
18.565421623550186
ns/iter18.49553273750065
ns/iter1.00
Evaluator_Draft4_Additional_Properties_Type
40.92814758197576
ns/iter42.73189509381196
ns/iter0.96
Evaluator_Draft4_Nested_Oneof
112.84177435943627
ns/iter125.58628558162319
ns/iter0.90
Evaluator_Draft4_Long_Enum
17.695104771587694
ns/iter18.261653299949412
ns/iter0.97
Evaluator_Draft4_Type_Object
12.404142579231737
ns/iter12.924768504848778
ns/iter0.96
Evaluator_Draft6_Property_Names
443.48401770068085
ns/iter447.4183978610088
ns/iter0.99
Evaluator_Draft7_If_Then_Else
50.136399269104004
ns/iter51.77199023425334
ns/iter0.97
Evaluator_Draft7_Vercel_1
103391.80970252064
ns/iter114176.82197090218
ns/iter0.91
Evaluator_2019_09_Unevaluated_Properties
253.68247344459735
ns/iter283.0707227343961
ns/iter0.90
Evaluator_2019_09_OMC_JSON_V2_1
6842.760967584698
ns/iter7403.441109473292
ns/iter0.92
Evaluator_2020_12_Dynamic_Ref
857.3820668328074
ns/iter885.7528493808336
ns/iter0.97
This comment was automatically generated by workflow using github-action-benchmark.
6ba7628
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Benchmark (windows/msvc)
Compiler_Draft6_AdaptiveCard
120826007699.99998
ns/iter118165276700.00005
ns/iter1.02
Compiler_2019_09_OMC_JSON_V2
25532280399.999992
ns/iter25309185000.00007
ns/iter1.01
Evaluator_Draft4_Meta_1_No_Callback
899.0429468558253
ns/iter873.7793526784848
ns/iter1.03
Evaluator_Draft4_Required_Properties
1346.0219214189892
ns/iter1359.15890216138
ns/iter0.99
Evaluator_Draft4_Many_Optional_Properties_Minimal_Match
146.90506696426147
ns/iter148.87412946425877
ns/iter0.99
Evaluator_Draft4_Few_Optional_Properties_Minimal_Match
96.79843764290874
ns/iter94.18291990254993
ns/iter1.03
Evaluator_Draft4_Items_Schema
2721.4215975792386
ns/iter2700.582858052826
ns/iter1.01
Evaluator_Draft4_Nested_Object
54.442383928566024
ns/iter55.8033928571433
ns/iter0.98
Evaluator_Draft4_Properties_Triad_Optional
4816.434166458041
ns/iter4817.072158100167
ns/iter1.00
Evaluator_Draft4_Properties_Triad_Closed
3878.147321428723
ns/iter3867.4034510651904
ns/iter1.00
Evaluator_Draft4_Properties_Triad_Required
4783.09322151148
ns/iter4875.053000000662
ns/iter0.98
Evaluator_Draft4_Properties_Closed
360.85856744905055
ns/iter359.72104284465627
ns/iter1.00
Evaluator_Draft4_Non_Recursive_Ref
150.13650669034217
ns/iter145.74418545784638
ns/iter1.03
Evaluator_Draft4_Pattern_Properties_True
7767.006696427968
ns/iter7808.330357142218
ns/iter0.99
Evaluator_Draft4_Ref_To_Single_Property
101.06735937501553
ns/iter94.72218750001814
ns/iter1.07
Evaluator_Draft4_Additional_Properties_Type
235.2376411565392
ns/iter223.379656249989
ns/iter1.05
Evaluator_Draft4_Nested_Oneof
379.6849679133975
ns/iter359.2358352337847
ns/iter1.06
Evaluator_Draft4_Long_Enum
122.84446428570358
ns/iter117.33951562501233
ns/iter1.05
Evaluator_Draft4_Type_Object
87.48571484438837
ns/iter86.43973274821839
ns/iter1.01
Evaluator_Draft6_Property_Names
1157.56906250013
ns/iter1150.6292857144933
ns/iter1.01
Evaluator_Draft7_If_Then_Else
187.57951674815314
ns/iter190.1719991225202
ns/iter0.99
Evaluator_Draft7_Vercel_1
244447.40542355165
ns/iter234126.91663872314
ns/iter1.04
Evaluator_2019_09_Unevaluated_Properties
710.754196428606
ns/iter702.5023437500064
ns/iter1.01
Evaluator_2019_09_OMC_JSON_V2_1
15533.627232139834
ns/iter15735.850446430633
ns/iter0.99
Evaluator_2020_12_Dynamic_Ref
2205.183125000332
ns/iter2123.984687499814
ns/iter1.04
This comment was automatically generated by workflow using github-action-benchmark.