Add interpreter microbenchmarks

[smarr]

The main purpose of the PR is to add interpreter microbenchmarks.
Though, it also updates the core-lib to the latest version.

[som-rs-benchmarker]

Here are the benchmark results for interpreter-microbenchmarks (commit: d00b505):

AST interpreter

+-----------------+----------------------------------------+------------------------------------+
| Benchmark       | master (base)                          | interpreter-microbenchmarks (head) |
+-----------------+----------------------------------------+------------------------------------+
| Bounce          | 187.80 ms ± 14.27 (180.30..226.50)     | 0.97x ± 0.12 (0.77..1.05)          |
| BubbleSort      | 295.56 ms ± 40.40 (247.85..359.85)     | 1.09x ± 0.17 (0.97..1.22)          |
| DeltaBlue       | 158.00 ms ± 10.64 (145.77..179.94)     | 1.02x ± 0.11 (0.87..1.09)          |
| Dispatch        | 190.50 ms ± 17.87 (175.62..235.94)     | 1.04x ± 0.11 (0.94..1.14)          |
| Fannkuch        | 135.53 ms ± 16.51 (117.74..161.73)     | 0.96x ± 0.21 (0.70..1.18)          |
| Fibonacci       | 362.35 ms ± 20.48 (344.20..411.01)     | 1.00x ± 0.09 (0.86..1.07)          |
| FieldLoop       | 332.06 ms ± 20.93 (302.28..363.86)     | 1.02x ± 0.12 (0.86..1.12)          |
| GraphSearch     | 82.23 ms ± 2.66 (77.64..86.12)         | 0.95x ± 0.11 (0.73..1.04)          |
| IntegerLoop     | 312.76 ms ± 13.12 (299.91..338.04)     | 0.96x ± 0.06 (0.89..1.02)          |
| JsonSmall       | 210.96 ms ± 10.65 (188.72..224.00)     | 1.13x ± 0.06 (1.08..1.16)          |
| List            | 243.55 ms ± 25.21 (215.51..289.30)     | 1.02x ± 0.15 (0.80..1.10)          |
| Loop            | 400.72 ms ± 16.73 (384.24..439.95)     | 0.94x ± 0.05 (0.90..1.02)          |
| Mandelbrot      | 263.85 ms ± 18.85 (243.05..306.27)     | 0.99x ± 0.09 (0.89..1.07)          |
| NBody           | 208.68 ms ± 17.07 (191.05..243.21)     | 0.97x ± 0.10 (0.85..1.04)          |
| PageRank        | 290.91 ms ± 15.24 (276.71..330.15)     | 0.99x ± 0.07 (0.92..1.09)          |
| Permute         | 320.40 ms ± 26.45 (285.17..374.44)     | 1.05x ± 0.11 (0.93..1.14)          |
| Queens          | 250.46 ms ± 23.18 (222.92..289.92)     | 1.11x ± 0.11 (1.06..1.15)          |
| QuickSort       | 73.19 ms ± 2.59 (68.60..77.45)         | 0.99x ± 0.07 (0.87..1.07)          |
| Recurse         | 281.76 ms ± 25.58 (253.35..324.30)     | 1.00x ± 0.12 (0.84..1.12)          |
| Richards        | 3946.50 ms ± 156.53 (3730.69..4206.02) | 1.01x ± 0.04 (0.97..1.04)          |
| Sieve           | 429.54 ms ± 36.24 (376.03..493.01)     | 1.07x ± 0.11 (0.91..1.15)          |
| Storage         | 89.85 ms ± 5.43 (83.78..102.04)        | 1.00x ± 0.15 (0.78..1.15)          |
| Sum             | 165.59 ms ± 11.40 (153.57..190.27)     | 0.99x ± 0.08 (0.93..1.10)          |
| Towers          | 325.67 ms ± 22.64 (293.89..376.75)     | 1.04x ± 0.11 (0.91..1.12)          |
| TreeSort        | 158.27 ms ± 13.36 (147.74..183.84)     | 0.97x ± 0.12 (0.78..1.08)          |
| WhileLoop       | 349.99 ms ± 4.22 (344.04..356.88)      | 0.96x ± 0.04 (0.90..1.02)          |
|                 |                                        |                                    |
| Average Speedup |               (baseline)               | 1.01x ± 0.02 (0.94..1.13)          |
+-----------------+----------------------------------------+------------------------------------+

The raw ReBench data files are available for download here: baseline and head

Bytecode interpreter

+-----------------+---------------------------------------+------------------------------------+
| Benchmark       | master (base)                         | interpreter-microbenchmarks (head) |
+-----------------+---------------------------------------+------------------------------------+
| Bounce          | 83.91 ms ± 12.13 (71.82..105.74)      | 1.09x ± 0.26 (0.71..1.21)          |
| BubbleSort      | 110.89 ms ± 12.03 (99.67..134.81)     | 1.13x ± 0.12 (1.11..1.16)          |
| DeltaBlue       | 57.97 ms ± 2.94 (54.86..64.09)        | 0.95x ± 0.11 (0.78..1.05)          |
| Dispatch        | 83.34 ms ± 4.03 (77.81..90.94)        | 1.06x ± 0.10 (0.90..1.17)          |
| Fannkuch        | 51.75 ms ± 10.90 (43.13..71.02)       | 1.07x ± 0.24 (0.92..1.16)          |
| Fibonacci       | 131.66 ms ± 5.85 (126.71..144.46)     | 0.97x ± 0.07 (0.87..1.03)          |
| FieldLoop       | 161.02 ms ± 9.98 (147.89..179.82)     | 0.96x ± 0.09 (0.87..1.05)          |
| GraphSearch     | 31.03 ms ± 0.79 (29.66..32.24)        | 0.94x ± 0.08 (0.79..1.02)          |
| IntegerLoop     | 138.05 ms ± 6.37 (129.12..148.73)     | 1.02x ± 0.13 (0.80..1.12)          |
| JsonSmall       | 89.54 ms ± 15.90 (79.05..133.54)      | 1.08x ± 0.20 (0.98..1.13)          |
| List            | 105.37 ms ± 9.95 (98.15..132.32)      | 0.98x ± 0.11 (0.86..1.10)          |
| Loop            | 181.76 ms ± 12.09 (166.66..211.90)    | 1.03x ± 0.10 (0.90..1.11)          |
| Mandelbrot      | 108.89 ms ± 3.54 (105.35..115.39)     | 0.98x ± 0.09 (0.81..1.06)          |
| NBody           | 80.39 ms ± 5.94 (75.17..95.85)        | 0.99x ± 0.11 (0.83..1.06)          |
| PageRank        | 118.81 ms ± 10.32 (108.51..144.57)    | 0.96x ± 0.10 (0.89..1.04)          |
| Permute         | 107.77 ms ± 3.54 (103.59..113.51)     | 1.00x ± 0.05 (0.91..1.04)          |
| Queens          | 90.28 ms ± 10.56 (83.52..119.30)      | 1.06x ± 0.13 (0.96..1.12)          |
| QuickSort       | 28.20 ms ± 0.96 (27.16..30.31)        | 1.00x ± 0.06 (0.91..1.06)          |
| Recurse         | 107.27 ms ± 5.19 (101.23..117.58)     | 0.97x ± 0.09 (0.82..1.02)          |
| Richards        | 1421.74 ms ± 68.97 (1335.70..1601.63) | 1.01x ± 0.06 (0.94..1.04)          |
| Sieve           | 158.78 ms ± 3.79 (152.23..167.59)     | 1.00x ± 0.05 (0.92..1.05)          |
| Storage         | 38.97 ms ± 7.64 (30.77..46.92)        | 1.14x ± 0.26 (0.90..1.30)          |
| Sum             | 72.24 ms ± 3.91 (68.11..78.31)        | 1.04x ± 0.08 (0.94..1.12)          |
| Towers          | 122.86 ms ± 7.45 (116.52..141.74)     | 1.04x ± 0.11 (0.84..1.09)          |
| TreeSort        | 53.79 ms ± 5.96 (48.24..65.69)        | 1.12x ± 0.13 (1.08..1.17)          |
| WhileLoop       | 173.82 ms ± 14.35 (160.18..202.11)    | 1.10x ± 0.09 (1.05..1.13)          |
|                 |                                       |                                    |
| Average Speedup |              (baseline)               | 1.03x ± 0.03 (0.94..1.14)          |
+-----------------+---------------------------------------+------------------------------------+

The raw ReBench data files are available for download here: baseline and head

The benchmarks were run using ReBench v1.2.0
The statistical analysis was done using rebench-tabler v0.1.0

The source code of this benchmark runner is available as a GitHub Gist for more details about the setup

[som-rs-benchmarker]

Here are the benchmark results for interpreter-microbenchmarks (commit: 4888a61):

AST interpreter

+-----------------+----------------------------------------+------------------------------------+
| Benchmark       | master (base)                          | interpreter-microbenchmarks (head) |
+-----------------+----------------------------------------+------------------------------------+
| Bounce          | 185.55 ms ± 15.66 (172.50..217.23)     | 0.98x ± 0.13 (0.77..1.07)          |
| BubbleSort      | 265.01 ms ± 25.78 (242.15..327.00)     | 1.02x ± 0.12 (0.91..1.08)          |
| DeltaBlue       | 159.03 ms ± 13.63 (144.30..184.06)     | 1.02x ± 0.11 (0.92..1.11)          |
| Dispatch        | 189.63 ms ± 12.76 (174.27..209.39)     | 0.98x ± 0.10 (0.88..1.06)          |
| Fannkuch        | 131.63 ms ± 4.25 (121.86..137.90)      | 1.04x ± 0.12 (0.86..1.16)          |
| Fibonacci       | 364.84 ms ± 17.45 (342.87..391.00)     | 0.99x ± 0.07 (0.90..1.04)          |
| FieldLoop       | 334.00 ms ± 28.35 (308.95..405.30)     | 0.99x ± 0.11 (0.90..1.10)          |
| GraphSearch     | 83.62 ms ± 6.42 (74.79..91.65)         | 0.91x ± 0.16 (0.72..1.08)          |
| IntegerLoop     | 320.60 ms ± 25.33 (293.96..373.81)     | 1.06x ± 0.09 (1.00..1.11)          |
| JsonSmall       | 199.51 ms ± 31.18 (181.72..281.29)     | 1.05x ± 0.18 (0.88..1.13)          |
| List            | 253.13 ms ± 30.27 (215.84..314.28)     | 1.12x ± 0.15 (1.00..1.18)          |
| Loop            | 410.86 ms ± 21.76 (382.39..452.29)     | 0.97x ± 0.07 (0.91..1.08)          |
| Mandelbrot      | 269.44 ms ± 25.83 (241.69..337.06)     | 1.10x ± 0.11 (1.04..1.17)          |
| NBody           | 205.49 ms ± 7.80 (192.62..221.36)      | 0.98x ± 0.09 (0.83..1.06)          |
| PageRank        | 300.22 ms ± 26.13 (270.56..363.11)     | 1.03x ± 0.10 (0.96..1.08)          |
| Permute         | 298.30 ms ± 10.09 (282.84..311.26)     | 0.98x ± 0.07 (0.89..1.05)          |
| Queens          | 242.95 ms ± 13.61 (219.42..268.06)     | 1.04x ± 0.07 (1.00..1.11)          |
| QuickSort       | 73.48 ms ± 9.30 (65.04..97.73)         | 1.03x ± 0.14 (0.91..1.07)          |
| Recurse         | 296.36 ms ± 36.22 (256.70..369.04)     | 1.05x ± 0.14 (0.96..1.14)          |
| Richards        | 3895.98 ms ± 108.33 (3690.10..4039.40) | 0.98x ± 0.05 (0.93..1.05)          |
| Sieve           | 410.67 ms ± 16.98 (388.92..436.20)     | 0.98x ± 0.08 (0.84..1.03)          |
| Storage         | 89.70 ms ± 14.17 (76.81..119.68)       | 1.09x ± 0.19 (0.96..1.18)          |
| Sum             | 172.43 ms ± 20.60 (154.01..216.22)     | 1.10x ± 0.13 (1.06..1.14)          |
| Towers          | 322.17 ms ± 22.35 (295.09..365.32)     | 1.07x ± 0.10 (0.96..1.17)          |
| TreeSort        | 157.30 ms ± 18.78 (140.86..198.62)     | 0.95x ± 0.14 (0.80..1.04)          |
| WhileLoop       | 366.73 ms ± 28.79 (333.83..429.77)     | 1.05x ± 0.11 (0.88..1.12)          |
|                 |                                        |                                    |
| Average Speedup |               (baseline)               | 1.02x ± 0.02 (0.91..1.12)          |
+-----------------+----------------------------------------+------------------------------------+

The raw ReBench data files are available for download here: baseline and head

Bytecode interpreter

+-----------------+---------------------------------------+------------------------------------+
| Benchmark       | master (base)                         | interpreter-microbenchmarks (head) |
+-----------------+---------------------------------------+------------------------------------+
| Bounce          | 71.08 ms ± 3.17 (67.30..78.40)        | 1.00x ± 0.06 (0.95..1.07)          |
| BubbleSort      | 101.45 ms ± 5.67 (97.16..116.70)      | 0.97x ± 0.08 (0.85..1.01)          |
| DeltaBlue       | 56.29 ms ± 1.97 (53.77..59.75)        | 0.88x ± 0.10 (0.70..0.97)          |
| Dispatch        | 76.56 ms ± 5.82 (71.92..91.56)        | 0.99x ± 0.09 (0.89..1.07)          |
| Fannkuch        | 45.45 ms ± 2.50 (42.62..50.20)        | 0.91x ± 0.08 (0.78..0.97)          |
| Fibonacci       | 141.02 ms ± 12.63 (132.99..165.62)    | 0.96x ± 0.12 (0.82..1.09)          |
| FieldLoop       | 158.30 ms ± 6.76 (150.13..170.11)     | 1.00x ± 0.05 (0.97..1.04)          |
| GraphSearch     | 32.35 ms ± 0.94 (30.60..33.93)        | 0.98x ± 0.05 (0.91..1.06)          |
| IntegerLoop     | 133.07 ms ± 3.61 (127.07..138.39)     | 0.94x ± 0.04 (0.87..0.98)          |
| JsonSmall       | 81.23 ms ± 6.31 (75.08..92.54)        | 1.02x ± 0.09 (0.95..1.07)          |
| List            | 104.05 ms ± 15.78 (93.69..144.56)     | 0.96x ± 0.18 (0.77..1.06)          |
| Loop            | 182.29 ms ± 8.78 (170.91..196.61)     | 0.94x ± 0.11 (0.77..1.05)          |
| Mandelbrot      | 111.86 ms ± 5.73 (105.58..122.94)     | 1.00x ± 0.07 (0.89..1.05)          |
| NBody           | 80.79 ms ± 6.00 (74.47..92.73)        | 0.96x ± 0.12 (0.78..1.07)          |
| PageRank        | 119.97 ms ± 8.80 (111.66..138.99)     | 1.03x ± 0.10 (0.87..1.11)          |
| Permute         | 112.89 ms ± 10.59 (103.62..138.33)    | 1.02x ± 0.10 (0.95..1.07)          |
| Queens          | 82.66 ms ± 3.47 (78.44..91.79)        | 0.90x ± 0.11 (0.70..0.98)          |
| QuickSort       | 27.30 ms ± 0.82 (26.48..29.23)        | 0.90x ± 0.15 (0.64..1.01)          |
| Recurse         | 117.34 ms ± 12.01 (103.60..143.31)    | 1.07x ± 0.14 (0.88..1.17)          |
| Richards        | 1406.64 ms ± 48.47 (1334.10..1486.42) | 0.95x ± 0.08 (0.84..1.03)          |
| Sieve           | 166.72 ms ± 18.71 (151.60..212.76)    | 1.03x ± 0.15 (0.82..1.11)          |
| Storage         | 31.87 ms ± 2.92 (29.64..39.53)        | 0.88x ± 0.22 (0.62..1.06)          |
| Sum             | 67.10 ms ± 2.08 (63.20..70.17)        | 0.97x ± 0.08 (0.81..1.04)          |
| Towers          | 110.78 ms ± 2.42 (108.69..116.03)     | 0.92x ± 0.04 (0.88..0.97)          |
| TreeSort        | 47.34 ms ± 4.46 (44.55..59.53)        | 0.97x ± 0.11 (0.87..1.10)          |
| WhileLoop       | 160.80 ms ± 5.91 (152.04..171.03)     | 0.89x ± 0.18 (0.64..1.04)          |
|                 |                                       |                                    |
| Average Speedup |              (baseline)               | 0.96x ± 0.02 (0.88..1.07)          |
+-----------------+---------------------------------------+------------------------------------+

The raw ReBench data files are available for download here: baseline and head

The benchmarks were run using ReBench v1.2.0
The statistical analysis was done using rebench-tabler v0.1.0

The source code of this benchmark runner is available as a GitHub Gist for more details about the setup