Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add benchmarks against musl libm #314

Open
wants to merge 3 commits into
base: tgross35/test-refactoring
Choose a base branch
from
Open

Add benchmarks against musl libm #314

wants to merge 3 commits into from
+136 −134

Conversation

tgross35
Copy link
Contributor

@tgross35 tgross35 commented Oct 21, 2024 •
edited
Loading

The current benchmarks only test a single number for a fixed amount of iterations, and only test the functions in our crate. Add benchmark comparisons between our crate and the build musl libraries so we have a better basis.

@tgross35 tgross35 force-pushed the tgross35/test-refactoring branch 2 times, most recently from 09d82a7 to 2722bde Compare October 22, 2024 00:06
@tgross35 tgross35 force-pushed the musl-benchmarks branch 3 times, most recently from 6ee7a75 to b9198d4 Compare October 22, 2024 00:18
@tgross35 tgross35 force-pushed the tgross35/test-refactoring branch 3 times, most recently from 6bf4102 to f778935 Compare October 22, 2024 01:02
@tgross35 tgross35 force-pushed the musl-benchmarks branch 8 times, most recently from 087c0b8 to eff53ce Compare October 22, 2024 01:57
@tgross35 tgross35 force-pushed the musl-benchmarks branch 3 times, most recently from f13d1c2 to e542cd9 Compare October 22, 2024 02:35
@tgross35
Copy link
Contributor Author

This PR should be ready. Also depends on #300 so it can't merge directly.

@tgross35
Copy link
Contributor Author

Finished running on my machine, it looks like our implementations are within 0.5x ~ 2.0x of musl which is pretty good.

Full results

Ryzen 9 5900X

acos/crate              time:   [2.3531 µs 2.3587 µs 2.3661 µs]
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  8 (8.00%) high mild
  1 (1.00%) high severe
acos/musl               time:   [2.4770 µs 2.4817 µs 2.4870 µs]
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

acosf/crate             time:   [1.4621 µs 1.4684 µs 1.4785 µs]
Found 20 outliers among 100 measurements (20.00%)
  15 (15.00%) low mild
  4 (4.00%) high mild
  1 (1.00%) high severe
acosf/musl              time:   [1.4645 µs 1.4687 µs 1.4750 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

acosh/crate             time:   [1.5441 µs 1.5465 µs 1.5490 µs]
acosh/musl              time:   [1.7567 µs 1.7626 µs 1.7709 µs]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

acoshf/crate            time:   [1.0569 µs 1.0593 µs 1.0614 µs]
Found 19 outliers among 100 measurements (19.00%)
  19 (19.00%) high severe
acoshf/musl             time:   [1.4883 µs 1.4946 µs 1.5023 µs]
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

asin/crate              time:   [2.3785 µs 2.3868 µs 2.3984 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
asin/musl               time:   [2.6089 µs 2.6255 µs 2.6519 µs]
Found 14 outliers among 100 measurements (14.00%)
  8 (8.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  4 (4.00%) high severe

asinf/crate             time:   [1.1507 µs 1.1615 µs 1.1753 µs]
Found 14 outliers among 100 measurements (14.00%)
  10 (10.00%) low mild
  4 (4.00%) high severe
asinf/musl              time:   [1.5579 µs 1.5625 µs 1.5716 µs]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

asinh/crate             time:   [6.4153 µs 6.4303 µs 6.4486 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
asinh/musl              time:   [5.9226 µs 5.9319 µs 5.9399 µs]

asinhf/crate            time:   [4.2619 µs 4.2799 µs 4.3086 µs]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
asinhf/musl             time:   [4.4332 µs 4.4543 µs 4.4838 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe

atan/crate              time:   [2.7891 µs 2.8420 µs 2.9446 µs]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
atan/musl               time:   [3.0645 µs 3.0775 µs 3.0963 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

atan2/crate             time:   [4.9788 µs 5.0458 µs 5.1685 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
atan2/musl              time:   [4.5453 µs 4.5557 µs 4.5682 µs]
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

atan2f/crate            time:   [3.1751 µs 3.1938 µs 3.2144 µs]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high severe
atan2f/musl             time:   [3.6322 µs 3.6617 µs 3.6978 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

atanf/crate             time:   [1.3173 µs 1.3257 µs 1.3375 µs]
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe
atanf/musl              time:   [1.9278 µs 1.9346 µs 1.9429 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

atanh/crate             time:   [4.3200 µs 4.4151 µs 4.6090 µs]
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
atanh/musl              time:   [4.7550 µs 4.7808 µs 4.8119 µs]
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

atanhf/crate            time:   [4.2374 µs 4.2502 µs 4.2659 µs]
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe
atanhf/musl             time:   [3.8825 µs 3.9038 µs 3.9287 µs]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

cbrt/crate              time:   [2.8136 µs 2.8292 µs 2.8484 µs]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
cbrt/musl               time:   [3.1112 µs 3.1512 µs 3.1981 µs]
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe

cbrtf/crate             time:   [1.7807 µs 1.7966 µs 1.8157 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
cbrtf/musl              time:   [2.4143 µs 2.4732 µs 2.5595 µs]
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe

ceil/crate              time:   [616.62 ns 628.08 ns 643.69 ns]
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
  7 (7.00%) high severe
ceil/musl               time:   [928.30 ns 931.93 ns 936.65 ns]
Found 13 outliers among 100 measurements (13.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  6 (6.00%) high severe

ceilf/crate             time:   [486.59 ns 489.59 ns 493.65 ns]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
ceilf/musl              time:   [1.0283 µs 1.0419 µs 1.0614 µs]
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe

copysign/crate          time:   [313.41 ns 314.68 ns 316.23 ns]
Found 17 outliers among 100 measurements (17.00%)
  8 (8.00%) low mild
  2 (2.00%) high mild
  7 (7.00%) high severe
copysign/musl           time:   [691.21 ns 698.83 ns 709.11 ns]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

copysignf/crate         time:   [303.47 ns 303.83 ns 304.11 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
copysignf/musl          time:   [686.63 ns 687.43 ns 688.20 ns]
Found 30 outliers among 100 measurements (30.00%)
  18 (18.00%) low mild
  5 (5.00%) high mild
  7 (7.00%) high severe

cos/crate               time:   [2.2145 µs 2.2194 µs 2.2291 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe
cos/musl                time:   [1.8884 µs 1.8947 µs 1.9067 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe

cosf/crate              time:   [2.8150 µs 2.8166 µs 2.8185 µs]
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
cosf/musl               time:   [1.1962 µs 1.1984 µs 1.2018 µs]
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low mild
  8 (8.00%) high mild
  3 (3.00%) high severe

cosh/crate              time:   [3.2763 µs 3.2813 µs 3.2860 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
cosh/musl               time:   [4.7954 µs 4.8021 µs 4.8091 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

coshf/crate             time:   [2.4978 µs 2.5057 µs 2.5186 µs]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe
coshf/musl              time:   [3.0555 µs 3.0599 µs 3.0643 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

erf/crate               time:   [2.0711 µs 2.0743 µs 2.0778 µs]
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
erf/musl                time:   [1.8230 µs 1.8249 µs 1.8273 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

erff/crate              time:   [2.0717 µs 2.0747 µs 2.0777 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
erff/musl               time:   [1.8276 µs 1.8297 µs 1.8319 µs]

exp/crate               time:   [2.9187 µs 2.9308 µs 2.9490 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe
exp/musl                time:   [1.5294 µs 1.6070 µs 1.6993 µs]
Found 18 outliers among 100 measurements (18.00%)
  18 (18.00%) high severe

exp10/crate             time:   [3.1688 µs 3.1743 µs 3.1804 µs]
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild
  2 (2.00%) high severe
exp10/musl              time:   [2.5878 µs 2.6060 µs 2.6253 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

exp10f/crate            time:   [2.3156 µs 2.3286 µs 2.3453 µs]
Found 27 outliers among 100 measurements (27.00%)
  6 (6.00%) low severe
  8 (8.00%) low mild
  4 (4.00%) high mild
  9 (9.00%) high severe
exp10f/musl             time:   [2.3279 µs 2.3410 µs 2.3556 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

exp2/crate              time:   [2.1565 µs 2.1699 µs 2.1883 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
exp2/musl               time:   [1.1964 µs 1.2001 µs 1.2062 µs]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  6 (6.00%) high severe

exp2f/crate             time:   [1.5756 µs 1.5799 µs 1.5844 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
exp2f/musl              time:   [1.0238 µs 1.0248 µs 1.0263 µs]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

expf/crate              time:   [1.7271 µs 1.7362 µs 1.7492 µs]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
expf/musl               time:   [1.0911 µs 1.0945 µs 1.1007 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

expm1/crate             time:   [3.3158 µs 3.3183 µs 3.3208 µs]
Found 16 outliers among 100 measurements (16.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  7 (7.00%) high mild
  6 (6.00%) high severe
expm1/musl              time:   [3.4947 µs 3.5069 µs 3.5263 µs]
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  4 (4.00%) high severe

expm1f/crate            time:   [2.0032 µs 2.0055 µs 2.0081 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
expm1f/musl             time:   [2.3627 µs 2.3650 µs 2.3676 µs]
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  7 (7.00%) high mild
  3 (3.00%) high severe

fabs/crate              time:   [231.32 ns 231.47 ns 231.63 ns]
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) low mild
  4 (4.00%) high mild
  4 (4.00%) high severe
fabs/musl               time:   [676.13 ns 677.67 ns 680.45 ns]
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

fabsf/crate             time:   [226.28 ns 226.44 ns 226.64 ns]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
fabsf/musl              time:   [679.83 ns 684.02 ns 689.65 ns]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

fdim/crate              time:   [473.36 ns 474.57 ns 476.95 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
fdim/musl               time:   [865.75 ns 868.09 ns 871.87 ns]
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) high mild
  9 (9.00%) high severe

fdimf/crate             time:   [471.95 ns 472.90 ns 474.49 ns]
Found 15 outliers among 100 measurements (15.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  8 (8.00%) high severe
fdimf/musl              time:   [924.68 ns 937.64 ns 951.96 ns]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) high mild
  7 (7.00%) high severe

floor/crate             time:   [647.53 ns 649.94 ns 653.54 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
floor/musl              time:   [916.12 ns 921.20 ns 927.88 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

floorf/crate            time:   [465.43 ns 465.82 ns 466.38 ns]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
floorf/musl             time:   [685.00 ns 688.67 ns 693.60 ns]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

fma/crate               time:   [3.4150 µs 3.4214 µs 3.4339 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
fma/musl                time:   [3.8859 µs 3.9167 µs 3.9592 µs]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  7 (7.00%) high severe

fmaf/crate              time:   [608.16 ns 608.87 ns 609.57 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
fmaf/musl               time:   [932.09 ns 936.25 ns 941.94 ns]
Found 11 outliers among 100 measurements (11.00%)
  2 (2.00%) high mild
  9 (9.00%) high severe

fmax/crate              time:   [324.64 ns 325.19 ns 325.82 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
fmax/musl               time:   [800.39 ns 805.79 ns 814.71 ns]
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

fmaxf/crate             time:   [315.43 ns 316.98 ns 319.96 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
fmaxf/musl              time:   [948.80 ns 965.11 ns 980.32 ns]
Found 22 outliers among 100 measurements (22.00%)
  20 (20.00%) high mild
  2 (2.00%) high severe

fmin/crate              time:   [325.80 ns 327.85 ns 330.56 ns]
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
fmin/musl               time:   [909.20 ns 913.80 ns 919.87 ns]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe

fminf/crate             time:   [319.36 ns 319.84 ns 320.32 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
fminf/musl              time:   [910.93 ns 913.59 ns 918.00 ns]
Found 18 outliers among 100 measurements (18.00%)
  10 (10.00%) low mild
  4 (4.00%) high mild
  4 (4.00%) high severe

fmod/crate              time:   [1.3215 µs 1.3254 µs 1.3309 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
fmod/musl               time:   [2.1858 µs 2.1947 µs 2.2103 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

fmodf/crate             time:   [1.4756 µs 1.4908 µs 1.5066 µs]
Found 13 outliers among 100 measurements (13.00%)
  1 (1.00%) high mild
  12 (12.00%) high severe
fmodf/musl              time:   [2.2351 µs 2.2430 µs 2.2550 µs]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

frexp/crate             time:   [794.86 ns 796.12 ns 797.55 ns]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
frexp/musl              time:   [812.89 ns 815.51 ns 819.17 ns]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

frexpf/crate            time:   [913.28 ns 920.43 ns 928.96 ns]
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high severe
frexpf/musl             time:   [811.92 ns 813.21 ns 814.51 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

hypot/crate             time:   [2.0946 µs 2.0971 µs 2.0997 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
hypot/musl              time:   [2.9626 µs 2.9745 µs 2.9977 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe

hypotf/crate            time:   [1.1311 µs 1.1350 µs 1.1419 µs]
Found 13 outliers among 100 measurements (13.00%)
  3 (3.00%) high mild
  10 (10.00%) high severe
hypotf/musl             time:   [1.7255 µs 1.7409 µs 1.7574 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

ilogb/crate             time:   [459.66 ns 461.87 ns 465.12 ns]
Found 14 outliers among 100 measurements (14.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  8 (8.00%) high severe
ilogb/musl              time:   [903.91 ns 907.62 ns 913.79 ns]
Found 15 outliers among 100 measurements (15.00%)
  11 (11.00%) high mild
  4 (4.00%) high severe

ilogbf/crate            time:   [448.67 ns 451.23 ns 454.81 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
ilogbf/musl             time:   [1.0140 µs 1.0203 µs 1.0306 µs]
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe

j0/crate                time:   [2.1964 µs 2.2044 µs 2.2136 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
j0/musl                 time:   [2.3310 µs 2.3428 µs 2.3638 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe

j0f/crate               time:   [2.0676 µs 2.0692 µs 2.0709 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
j0f/musl                time:   [2.2431 µs 2.2551 µs 2.2765 µs]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high severe

j1/crate                time:   [2.5925 µs 2.6082 µs 2.6276 µs]
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe
j1/musl                 time:   [2.1753 µs 2.1805 µs 2.1871 µs]
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

j1f/crate               time:   [1.9152 µs 1.9263 µs 1.9468 µs]
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
j1f/musl                time:   [1.8168 µs 1.8260 µs 1.8393 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

Benchmarking jn/crate: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 16.4s, or reduce sample count to 30.
jn/crate                time:   [163.69 ms 164.19 ms 164.80 ms]
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe
Benchmarking jn/musl: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 14.2s, or reduce sample count to 30.
jn/musl                 time:   [141.42 ms 141.68 ms 142.02 ms]
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe

Benchmarking jnf/crate: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 15.2s, or reduce sample count to 30.
jnf/crate               time:   [151.54 ms 152.16 ms 153.00 ms]
Found 18 outliers among 100 measurements (18.00%)
  6 (6.00%) high mild
  12 (12.00%) high severe
Benchmarking jnf/musl: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 11.3s, or reduce sample count to 40.
jnf/musl                time:   [112.77 ms 113.10 ms 113.62 ms]
Found 17 outliers among 100 measurements (17.00%)
  8 (8.00%) high mild
  9 (9.00%) high severe

ldexp/crate             time:   [1.2392 µs 1.2436 µs 1.2478 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
ldexp/musl              time:   [1.5638 µs 1.5665 µs 1.5700 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

ldexpf/crate            time:   [1.2819 µs 1.2836 µs 1.2852 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
ldexpf/musl             time:   [1.6569 µs 1.6727 µs 1.6952 µs]
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

lgamma/crate            time:   [6.2914 µs 6.3157 µs 6.3532 µs]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  4 (4.00%) high severe
lgamma/musl             time:   [4.1301 µs 4.1437 µs 4.1663 µs]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

lgamma_r/crate          time:   [5.4631 µs 5.4910 µs 5.5316 µs]
Found 21 outliers among 100 measurements (21.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  5 (5.00%) high mild
  14 (14.00%) high severe
lgamma_r/musl           time:   [4.2352 µs 4.2542 µs 4.2938 µs]
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

lgammaf/crate           time:   [4.5018 µs 4.5116 µs 4.5234 µs]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
lgammaf/musl            time:   [3.9311 µs 3.9436 µs 3.9592 µs]
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe

lgammaf_r/crate         time:   [4.4094 µs 4.4218 µs 4.4390 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
lgammaf_r/musl          time:   [3.7310 µs 3.7458 µs 3.7684 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

log/crate               time:   [2.8446 µs 2.8523 µs 2.8616 µs]
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe
log/musl                time:   [1.9970 µs 2.0008 µs 2.0048 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

log10/crate             time:   [3.6193 µs 3.6458 µs 3.6875 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
log10/musl              time:   [3.6553 µs 3.6707 µs 3.6932 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

log10f/crate            time:   [2.7740 µs 2.7900 µs 2.8126 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
log10f/musl             time:   [2.6469 µs 2.6567 µs 2.6726 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

log1p/crate             time:   [2.7373 µs 2.7440 µs 2.7507 µs]
log1p/musl              time:   [3.6641 µs 3.6833 µs 3.7106 µs]
Found 17 outliers among 100 measurements (17.00%)
  4 (4.00%) low severe
  8 (8.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe

log1pf/crate            time:   [2.7763 µs 2.7852 µs 2.8001 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
log1pf/musl             time:   [2.0328 µs 2.0398 µs 2.0498 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

log2/crate              time:   [4.4259 µs 4.4458 µs 4.4767 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
log2/musl               time:   [2.4240 µs 2.4390 µs 2.4582 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe

log2f/crate             time:   [2.5556 µs 2.5596 µs 2.5640 µs]
Found 15 outliers among 100 measurements (15.00%)
  5 (5.00%) high mild
  10 (10.00%) high severe
log2f/musl              time:   [1.5984 µs 1.6004 µs 1.6027 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

logf/crate              time:   [2.1180 µs 2.1327 µs 2.1509 µs]
Found 10 outliers among 100 measurements (10.00%)
  9 (9.00%) high mild
  1 (1.00%) high severe
logf/musl               time:   [1.5969 µs 1.5993 µs 1.6017 µs]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high severe

modf/crate              time:   [520.37 ns 521.22 ns 522.17 ns]
modf/musl               time:   [916.49 ns 923.35 ns 934.14 ns]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe

modff/crate             time:   [516.56 ns 517.39 ns 518.15 ns]
modff/musl              time:   [906.66 ns 908.97 ns 912.30 ns]
Found 13 outliers among 100 measurements (13.00%)
  11 (11.00%) high mild
  2 (2.00%) high severe

nextafter/crate         time:   [843.54 ns 845.01 ns 846.35 ns]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
nextafter/musl          time:   [1.2995 µs 1.3157 µs 1.3346 µs]
Found 19 outliers among 100 measurements (19.00%)
  2 (2.00%) high mild
  17 (17.00%) high severe

nextafterf/crate        time:   [844.90 ns 847.85 ns 850.60 ns]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) low mild
  3 (3.00%) high mild
nextafterf/musl         time:   [1.3372 µs 1.3393 µs 1.3417 µs]
Found 20 outliers among 100 measurements (20.00%)
  1 (1.00%) low severe
  14 (14.00%) low mild
  5 (5.00%) high mild

pow/crate               time:   [16.101 µs 16.203 µs 16.343 µs]
Found 18 outliers among 100 measurements (18.00%)
  4 (4.00%) low mild
  7 (7.00%) high mild
  7 (7.00%) high severe
pow/musl                time:   [5.8881 µs 5.8954 µs 5.9034 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

powf/crate              time:   [20.033 µs 20.047 µs 20.064 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
powf/musl               time:   [2.9276 µs 2.9372 µs 2.9547 µs]
Found 11 outliers among 100 measurements (11.00%)
  9 (9.00%) high mild
  2 (2.00%) high severe

remainder/crate         time:   [1.8469 µs 1.8626 µs 1.8816 µs]
remainder/musl          time:   [2.8224 µs 2.8280 µs 2.8351 µs]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

remainderf/crate        time:   [1.7836 µs 1.7877 µs 1.7914 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
remainderf/musl         time:   [3.2819 µs 3.2866 µs 3.2907 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

remquo/crate            time:   [2.1180 µs 2.1276 µs 2.1415 µs]
Found 15 outliers among 100 measurements (15.00%)
  11 (11.00%) low mild
  4 (4.00%) high severe
remquo/musl             time:   [2.5211 µs 2.5323 µs 2.5445 µs]

remquof/crate           time:   [1.9163 µs 1.9225 µs 1.9291 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
remquof/musl            time:   [2.8397 µs 2.8540 µs 2.8735 µs]
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

rint/crate              time:   [413.34 ns 414.83 ns 416.95 ns]
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) low mild
  11 (11.00%) high severe
rint/musl               time:   [909.86 ns 911.42 ns 912.92 ns]

rintf/crate             time:   [409.73 ns 410.42 ns 411.10 ns]
rintf/musl              time:   [914.16 ns 915.25 ns 916.69 ns]
Found 14 outliers among 100 measurements (14.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  5 (5.00%) high mild
  7 (7.00%) high severe

round/crate             time:   [863.42 ns 866.77 ns 871.49 ns]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
round/musl              time:   [1.0436 µs 1.0465 µs 1.0502 µs]
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

roundf/crate            time:   [832.80 ns 837.91 ns 845.08 ns]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high severe
roundf/musl             time:   [1.1009 µs 1.1085 µs 1.1180 µs]
Found 19 outliers among 100 measurements (19.00%)
  2 (2.00%) high mild
  17 (17.00%) high severe

scalbn/crate            time:   [1.2509 µs 1.2541 µs 1.2569 µs]
Found 28 outliers among 100 measurements (28.00%)
  19 (19.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
scalbn/musl             time:   [1.5487 µs 1.5501 µs 1.5519 µs]
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

scalbnf/crate           time:   [1.2920 µs 1.3107 µs 1.3461 µs]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
scalbnf/musl            time:   [1.5575 µs 1.5657 µs 1.5771 µs]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

sin/crate               time:   [1.7109 µs 1.7154 µs 1.7213 µs]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
sin/musl                time:   [1.6492 µs 1.6595 µs 1.6736 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

sincos/crate            time:   [2.6231 µs 2.6358 µs 2.6534 µs]
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe
sincos/musl             time:   [5.1396 µs 5.1467 µs 5.1555 µs]
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe

sincosf/crate           time:   [1.5599 µs 1.5618 µs 1.5648 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
sincosf/musl            time:   [2.8286 µs 2.8325 µs 2.8365 µs]
Found 14 outliers among 100 measurements (14.00%)
  12 (12.00%) high mild
  2 (2.00%) high severe

sinf/crate              time:   [1.1077 µs 1.1089 µs 1.1101 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
sinf/musl               time:   [1.2200 µs 1.2261 µs 1.2343 µs]
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe

sinh/crate              time:   [4.2182 µs 4.2327 µs 4.2534 µs]
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe
sinh/musl               time:   [4.9623 µs 4.9953 µs 5.0548 µs]
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe

sinhf/crate             time:   [2.8610 µs 2.8732 µs 2.8927 µs]
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low severe
  3 (3.00%) high mild
  5 (5.00%) high severe
sinhf/musl              time:   [3.8295 µs 3.8312 µs 3.8331 µs]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

sqrt/crate              time:   [1.0005 µs 1.0039 µs 1.0089 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
sqrt/musl               time:   [996.49 ns 999.93 ns 1.0053 µs]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

sqrtf/crate             time:   [558.82 ns 559.29 ns 559.82 ns]
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild
sqrtf/musl              time:   [567.56 ns 570.53 ns 575.03 ns]
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

tan/crate               time:   [3.7414 µs 3.7501 µs 3.7592 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
tan/musl                time:   [3.5511 µs 3.5675 µs 3.5913 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe

tanf/crate              time:   [3.7394 µs 3.7472 µs 3.7551 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
tanf/musl               time:   [1.4986 µs 1.4996 µs 1.5008 µs]
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  6 (6.00%) high mild
  2 (2.00%) high severe

tanh/crate              time:   [4.9657 µs 4.9761 µs 4.9871 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
tanh/musl               time:   [6.3968 µs 6.4266 µs 6.4671 µs]
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe

tanhf/crate             time:   [3.6020 µs 3.6072 µs 3.6136 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
tanhf/musl              time:   [4.5482 µs 4.5756 µs 4.6119 µs]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

tgamma/crate            time:   [29.273 µs 29.419 µs 29.632 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
tgamma/musl             time:   [14.404 µs 14.420 µs 14.438 µs]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

tgammaf/crate           time:   [29.919 µs 29.982 µs 30.044 µs]
tgammaf/musl            time:   [15.546 µs 15.607 µs 15.710 µs]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

trunc/crate             time:   [601.53 ns 604.76 ns 609.13 ns]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
trunc/musl              time:   [695.68 ns 696.49 ns 697.34 ns]
Found 17 outliers among 100 measurements (17.00%)
  10 (10.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe

truncf/crate            time:   [600.21 ns 600.90 ns 601.64 ns]
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) low mild
  5 (5.00%) high mild
truncf/musl             time:   [693.61 ns 695.72 ns 698.88 ns]
Found 16 outliers among 100 measurements (16.00%)
  5 (5.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high mild
  5 (5.00%) high severe

libm-benches-e542cd93.tar.gz

@tgross35 tgross35 force-pushed the tgross35/test-refactoring branch 2 times, most recently from 12bf64a to 8ef0575 Compare October 26, 2024 04:05
@tgross35 tgross35 force-pushed the musl-benchmarks branch 2 times, most recently from bcb5fbf to bbd31c9 Compare October 27, 2024 02:07
@tgross35
Add benchmarks against musl libm
c6aadd4 
Add a benchmark for each function that checks against `musl_math_sys`.
@tgross35
Remove libm-bench
0644e9f 
This has been superseded by the benchmarks in `libm-test`.
@tgross35
Check benchmarks in CI
629243e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quaternic quaternic quaternic left review comments

Assignees

@Amanieu Amanieu

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tgross35 @quaternic @Amanieu