Test criterion reports

[sergejparity]

Increase measurement time for translate tests to check whether it will help to fix report.

Use mean metric when slope is not produced by Criterion

[paritytech-cicd-pr]

BENCHMARKS

	NATIVE			WASMTIME
BENCHMARK	MASTER	PR	DIFF	MASTER	PR	DIFF	WASMTIME OVERHEAD
execute/ br_table	1.49ms	1.51ms	⚪ 1.43%	1.41ms	1.33ms	🟢 -5.91%	🟢 -12%
execute/ call/host/1	44.65µs	45.14µs	⚪ 1.09%	65.31µs	74.00µs	🔴 13.31%	🟡 64%
execute/ call/rec	169.40µs	168.06µs	⚪ -0.79%	365.11µs	349.00µs	🟢 -4.41%	🔴 108%
execute/ count_until	7.56ms	7.45ms	⚪ -1.46%	7.49ms	7.48ms	⚪ -0.10%	🟢 0%
execute/ divrem	6.26ms	6.28ms	⚪ 0.20%	6.97ms	6.99ms	⚪ 0.29%	🟢 11%
execute/ factorial/iter	238.20µs	243.56µs	🔴 2.25%	321.33µs	313.55µs	🟢 -2.42%	🟢 29%
execute/ factorial/rec	736.72µs	685.57µs	🟢 -6.94%	1.38ms	1.28ms	🟢 -6.95%	🟡 87%
execute/ fibonacci/iter	1.36ms	1.40ms	🔴 2.98%	1.28ms	1.28ms	⚪ 0.16%	🟢 -9%
execute/ fibonacci/rec	6.20ms	6.08ms	🟢 -2.03%	14.00ms	13.43ms	🟢 -4.07%	🔴 121%
execute/ fibonacci/tail	1.35ms	1.36ms	⚪ 0.33%	4.14ms	3.59ms	🟢 -13.39%	🔴 164%
execute/ fuse	7.41ms	7.35ms	⚪ -0.80%	11.60ms	11.63ms	⚪ 0.20%	🟡 58%
execute/ global/bump	1.32ms	1.32ms	⚪ -0.26%	1.53ms	1.53ms	⚪ -0.04%	🟢 16%
execute/ global/get_const	727.51µs	730.46µs	⚪ 0.41%	747.42µs	750.49µs	⚪ 0.41%	🟢 3%
execute/ is_even/rec	1.13ms	1.10ms	⚪ -2.30%	2.28ms	2.22ms	🟢 -2.71%	🔴 102%
execute/ memory/fill_bytes	1.14ms	1.14ms	⚪ -0.10%	1.32ms	1.32ms	⚪ 0.02%	🟢 16%
execute/ memory/sum_bytes	1.10ms	1.12ms	⚪ 1.58%	1.23ms	1.23ms	⚪ 0.23%	🟢 10%
execute/ memory/vec_add	2.95ms	2.95ms	⚪ 0.10%	3.59ms	3.58ms	⚪ -0.10%	🟢 21%
execute/ recursive_scan	187.31µs	189.08µs	⚪ 0.94%	395.81µs	382.36µs	🟢 -3.40%	🔴 102%
execute/ recursive_trap	15.63µs	15.48µs	⚪ -0.98%	34.95µs	34.68µs	⚪ -0.77%	🔴 124%
execute/ regex_redux	591.66µs	592.21µs	⚪ 0.09%	1.10ms	1.03ms	🟢 -6.24%	🟡 74%
execute/ rev_complement	444.40µs	445.74µs	⚪ 0.30%	652.93µs	629.68µs	🟢 -3.56%	🟢 41%
execute/ tiny_keccak	350.62µs	350.62µs	⚪ 0.00%	379.20µs	380.60µs	⚪ 0.37%	🟢 9%
execute/ trunc_f2i	612.20µs	612.46µs	⚪ 0.04%	952.01µs	952.67µs	⚪ 0.07%	🟡 56%
instantiate/ wasm_kernel	51.70µs	55.01µs	🔴 6.40%	55.42µs	53.40µs	🟢 -3.65%	🟢 -3%
overhead/ call/typed/0	1.23ms	1.20ms	🟢 -2.83%	766.29µs	770.66µs	⚪ 0.57%	🟢 -36%
overhead/ call/typed/16	1.61ms	1.61ms	⚪ 0.53%	1.86ms	1.87ms	⚪ 0.59%	🟢 16%
overhead/ call/untyped/0	1.62ms	1.62ms	⚪ 0.19%	1.23ms	1.23ms	⚪ 0.31%	🟢 -24%
overhead/ call/untyped/16	2.65ms	2.49ms	🟢 -6.00%	3.92ms	3.92ms	⚪ 0.02%	🟡 57%
translate/ bz2/checked/default	1.39ms	1.39ms	⚪ -0.17%	2.47ms	2.47ms	⚪ 0.07%	🟡 78%
translate/ bz2/checked/fuel	1.44ms	1.43ms	⚪ -0.77%	2.62ms	2.62ms	⚪ 0.16%	🟡 83%
translate/ bz2/unchecked/default	1.14ms	1.14ms	⚪ -0.55%	1.96ms	1.97ms	⚪ 0.23%	🟡 73%
translate/ bz2/unchecked/fuel	1.18ms	1.17ms	⚪ -0.45%	2.09ms	2.08ms	⚪ -0.42%	🟡 77%
translate/ erc1155/checked/default	284.17µs	283.75µs	⚪ -0.15%	481.74µs	480.85µs	⚪ -0.18%	🟡 69%
translate/ erc1155/checked/fuel	304.28µs	301.70µs	⚪ -0.85%	509.38µs	531.02µs	🔴 4.25%	🟡 76%
translate/ erc1155/unchecked/default	234.98µs	235.80µs	⚪ 0.35%	383.55µs	387.94µs	⚪ 1.14%	🟡 65%
translate/ erc1155/unchecked/fuel	251.07µs	251.99µs	⚪ 0.36%	407.38µs	419.38µs	🔴 2.95%	🟡 66%
translate/ erc20/checked/default	136.99µs	136.76µs	⚪ -0.17%	230.12µs	231.98µs	⚪ 0.81%	🟡 70%
translate/ erc20/checked/fuel	144.11µs	144.47µs	⚪ 0.25%	240.61µs	243.95µs	🔴 1.39%	🟡 69%
translate/ erc20/unchecked/default	114.25µs	114.06µs	⚪ -0.16%	185.24µs	186.52µs	⚪ 0.69%	🟡 64%
translate/ erc20/unchecked/fuel	120.79µs	121.24µs	⚪ 0.38%	192.71µs	193.11µs	⚪ 0.21%	🟡 59%
translate/ erc721/checked/default	195.05µs	194.77µs	⚪ -0.15%	334.34µs	337.24µs	⚪ 0.87%	🟡 73%
translate/ erc721/checked/fuel	239.82µs	204.82µs	⚪ -14.59%	349.07µs	350.09µs	⚪ 0.29%	🟡 71%
translate/ erc721/unchecked/default	160.37µs	159.69µs	⚪ -0.42%	267.34µs	266.18µs	⚪ -0.43%	🟡 67%
translate/ erc721/unchecked/fuel	170.75µs	168.75µs	⚪ -1.17%	276.17µs	280.55µs	⚪ 1.59%	🟡 66%
translate/ pulldown_cmark/checked/default	3.82ms	3.77ms	⚪ -1.31%	6.43ms	6.48ms	⚪ 0.71%	🟡 72%
translate/ pulldown_cmark/checked/fuel	3.89ms	3.87ms	⚪ -0.46%	6.84ms	6.79ms	⚪ -0.74%	🟡 75%
translate/ pulldown_cmark/unchecked/default	3.13ms	3.12ms	⚪ -0.43%	5.08ms	5.12ms	⚪ 0.67%	🟡 64%
translate/ pulldown_cmark/unchecked/fuel	3.22ms	3.23ms	⚪ 0.49%	5.33ms	5.42ms	🔴 1.63%	🟡 68%
translate/ spidermonkey/checked/default	89.33ms	88.65ms	⚪ -0.76%	154.07ms	155.55ms	⚪ 0.96%	🟡 75%
translate/ spidermonkey/checked/fuel	91.57ms	91.22ms	⚪ -0.38%	161.50ms	163.10ms	⚪ 0.99%	🟡 79%
translate/ spidermonkey/unchecked/default	72.11ms	71.89ms	⚪ -0.30%	121.39ms	122.00ms	⚪ 0.51%	🟡 70%
translate/ spidermonkey/unchecked/fuel	74.16ms	74.33ms	⚪ 0.22%	127.53ms	127.70ms	⚪ 0.13%	🟡 72%
translate/ wasm_kernel/checked/default	5.08ms	5.07ms	⚪ -0.18%	8.69ms	8.80ms	🔴 1.27%	🟡 74%
translate/ wasm_kernel/checked/fuel	5.21ms	5.18ms	⚪ -0.64%	9.10ms	9.23ms	🔴 1.44%	🟡 78%
translate/ wasm_kernel/unchecked/default	4.13ms	4.13ms	⚪ 0.03%	6.84ms	6.92ms	⚪ 1.11%	🟡 68%
translate/ wasm_kernel/unchecked/fuel	4.25ms	4.26ms	⚪ 0.08%	7.13ms	7.24ms	🔴 1.43%	🟡 70%

Link to pipeline

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (38c968e) 81.03% compared to head (1c4b595) 81.03%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #836   +/-   ##
=======================================
  Coverage   81.03%   81.03%           
=======================================
  Files         257      257           
  Lines       22446    22446           
=======================================
+ Hits        18188    18189    +1     
+ Misses       4258     4257    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Robbepop]

@sergejparity Great work! Thanks a lot for the benchmark improvements!