From 117488c269db1d90642519eba56e03fce42b2326 Mon Sep 17 00:00:00 2001 From: Ievgen Pyrogov Date: Fri, 3 May 2024 09:31:44 +0200 Subject: [PATCH] Refresh all benchmark statistics --- bench/results/all.md | 119 ++++++++++ bench/results/insert.md | 109 ++++----- bench/results/load.md | 337 +++++++++++++-------------- bench/results/to_sql.md | 505 +++++++++++++++++++--------------------- 4 files changed, 570 insertions(+), 500 deletions(-) create mode 100644 bench/results/all.md diff --git a/bench/results/all.md b/bench/results/all.md new file mode 100644 index 0000000..4d55936 --- /dev/null +++ b/bench/results/all.md @@ -0,0 +1,119 @@ +Benchmark + +Benchmark run from 2024-05-04 10:50:11.956493Z UTC + +## System + +Benchmark suite executing on the following system: + + + + + + + + + + + + + + + + + + + + + +
Operating SystemmacOS
CPU InformationApple M3 Max
Number of Available Cores16
Available Memory128 GB
Elixir Version1.16.2
Erlang Version26.2.4
+ +## Configuration + +Benchmark suite executing with the following configuration: + + + + + + + + + + + + +
:time10 s
:parallel1
:warmup2 s
+ +## Statistics + + + +Run Time + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameIPSAverageDevitationMedian99th %
MyXQL Repo.all/2639.681.56 ms±12.01%1.54 ms2.20 ms
SQLite3 Repo.all/2465.992.15 ms±4.82%2.13 ms2.43 ms
Pg Repo.all/2214.164.67 ms±12.05%4.66 ms5.86 ms
+ + +Run Time Comparison + + + + + + + + + + + + + + + + + + + + + + + + +
NameIPSSlower
MyXQL Repo.all/2639.68 
SQLite3 Repo.all/2465.991.37x
Pg Repo.all/2214.162.99x
\ No newline at end of file diff --git a/bench/results/insert.md b/bench/results/insert.md index 3e8b360..9858f83 100644 --- a/bench/results/insert.md +++ b/bench/results/insert.md @@ -1,7 +1,6 @@ +Benchmark -# Benchmark - -Benchmark run from 2021-03-24 02:05:58.706995Z UTC +Benchmark run from 2024-05-04 10:49:35.563150Z UTC ## System @@ -10,22 +9,22 @@ Benchmark suite executing on the following system: - + - + - + - + - +
Operating SystemLinuxmacOS
CPU InformationAMD Ryzen 7 PRO 4750U with Radeon GraphicsApple M3 Max
Number of Available Cores 16
Available Memory14.92 GB128 GB
Elixir Version1.11.31.16.2
Erlang Version23.2.626.2.4
@@ -50,7 +49,6 @@ Benchmark suite executing with the following configuration: - __Input: Changeset__ Run Time @@ -67,35 +65,35 @@ Run Time SQLite3 Insert - 7218.07 - 0.139 ms - ±43.60% - 0.123 ms - 0.37 ms + 26.72 K + 37.42 µs + ±89.66% + 32.88 µs + 74.21 µs Pg Insert - 421.57 - 2.37 ms - ±12.13% - 2.37 ms - 2.90 ms + 9.65 K + 103.63 µs + ±68.38% + 102.75 µs + 177.39 µs MyXQL Insert - 284.25 - 3.52 ms - ±13.34% - 3.53 ms - 5.05 ms + 5.49 K + 182.25 µs + ±49.23% + 182.33 µs + 233.08 µs -Comparison +Run Time Comparison @@ -104,28 +102,26 @@ Comparison - + - - + + - - + +
Slower
SQLite3 Insert7218.0726.72 K  
Pg Insert421.5717.12x9.65 K2.77x
MyXQL Insert284.2525.39x5.49 K4.87x
-
- __Input: Struct__ @@ -143,35 +139,35 @@ Run Time SQLite3 Insert - 7765.76 - 0.129 ms - ±32.88% - 0.122 ms - 0.28 ms + 26.71 K + 37.44 µs + ±87.15% + 32.92 µs + 70.50 µs Pg Insert - 422.86 - 2.36 ms - ±10.49% - 2.36 ms - 3.02 ms + 9.34 K + 107.08 µs + ±13.80% + 106.87 µs + 132.46 µs MyXQL Insert - 274.00 - 3.65 ms - ±38.43% - 3.59 ms - 4.75 ms + 5.67 K + 176.45 µs + ±70.69% + 176.79 µs + 234.70 µs -Comparison +Run Time Comparison @@ -180,25 +176,20 @@ Comparison - + - - + + - - + + -
Slower
SQLite3 Insert7765.7626.71 K  
Pg Insert422.8618.37x9.34 K2.86x
MyXQL Insert274.0028.34x5.67 K4.71x
- - - -
- + \ No newline at end of file diff --git a/bench/results/load.md b/bench/results/load.md index ec27e51..c36ec4d 100644 --- a/bench/results/load.md +++ b/bench/results/load.md @@ -1,7 +1,6 @@ +Benchmark -# Benchmark - -Benchmark run from 2021-03-24 01:58:24.995583Z UTC +Benchmark run from 2024-05-04 10:39:52.658755Z UTC ## System @@ -10,22 +9,22 @@ Benchmark suite executing on the following system: - + - + - + - + - +
Operating SystemLinuxmacOS
CPU InformationAMD Ryzen 7 PRO 4750U with Radeon GraphicsApple M3 Max
Number of Available Cores 16
Available Memory14.92 GB128 GB
Elixir Version1.11.31.16.2
Erlang Version23.2.626.2.4
@@ -50,7 +49,6 @@ Benchmark suite executing with the following configuration: - __Input: Big 1 Million__ Run Time @@ -66,36 +64,36 @@ Run Time - SQLite3 Loader - 0.35 - 2.82 s - ±15.78% - 2.82 s - 3.13 s + MyXQL Loader + 1.90 + 526.95 ms + ±7.52% + 520.79 ms + 590.85 ms Pg Loader - 0.34 - 2.92 s - ±17.97% - 2.92 s - 3.29 s + 1.89 + 527.72 ms + ±7.17% + 523.29 ms + 589.90 ms - MyXQL Loader - 0.33 - 2.99 s - ±13.08% - 2.99 s - 3.26 s + SQLite3 Loader + 1.89 + 529.12 ms + ±7.67% + 522.49 ms + 594.22 ms -Comparison +Run Time Comparison @@ -103,29 +101,27 @@ Comparison - - + + - - + + - - - + + +
IPS Slower
SQLite3 Loader0.35MyXQL Loader1.90  
Pg Loader0.341.04x1.891.0x
MyXQL Loader0.331.06xSQLite3 Loader1.891.0x
-
- __Input: Date attr__ @@ -142,36 +138,36 @@ Run Time - Pg Loader - 4.61 - 217.08 ms - ±13.97% - 214.42 ms - 271.56 ms + MyXQL Loader + 26.37 + 37.93 ms + ±7.64% + 37.53 ms + 48.13 ms - MyXQL Loader - 4.56 - 219.25 ms - ±18.19% - 210.43 ms - 300.02 ms + Pg Loader + 25.98 + 38.49 ms + ±7.63% + 38.31 ms + 48.20 ms SQLite3 Loader - 4.49 - 222.95 ms - ±15.03% - 232.42 ms - 266.39 ms + 25.91 + 38.59 ms + ±7.51% + 38.46 ms + 48.29 ms -Comparison +Run Time Comparison @@ -179,29 +175,27 @@ Comparison - - + + - - + + - - + +
IPS Slower
Pg Loader4.61MyXQL Loader26.37  
MyXQL Loader4.56Pg Loader25.98 1.01x
SQLite3 Loader4.491.03x25.911.02x
-
- __Input: Medium 100 Thousand__ @@ -219,35 +213,35 @@ Run Time MyXQL Loader - 4.53 - 220.75 ms - ±15.57% - 209.17 ms - 287.22 ms + 26.27 + 38.07 ms + ±5.01% + 38.36 ms + 42.73 ms - Pg Loader - 4.40 - 227.35 ms - ±16.30% - 222.39 ms - 335.90 ms + SQLite3 Loader + 25.64 + 39.01 ms + ±5.25% + 39.42 ms + 43.28 ms - SQLite3 Loader - 4.35 - 230.09 ms - ±19.61% - 236.94 ms - 309.96 ms + Pg Loader + 25.44 + 39.31 ms + ±6.97% + 39.50 ms + 49.12 ms -Comparison +Run Time Comparison @@ -256,28 +250,26 @@ Comparison - + - - - + + + - - - + + +
Slower
MyXQL Loader4.5326.27  
Pg Loader4.401.03xSQLite3 Loader25.641.02x
SQLite3 Loader4.351.04xPg Loader25.441.03x
-
- __Input: Small 1 Thousand__ @@ -294,36 +286,36 @@ Run Time - Pg Loader - 883.24 - 1.13 ms - ±62.88% - 0.93 ms - 2.71 ms + MyXQL Loader + 2.95 K + 339.32 µs + ±13.05% + 330.33 µs + 493.99 µs - SQLite3 Loader - 873.53 - 1.14 ms - ±60.06% - 0.92 ms - 2.90 ms + Pg Loader + 2.92 K + 342.75 µs + ±13.24% + 332.17 µs + 499.14 µs - MyXQL Loader - 862.33 - 1.16 ms - ±62.27% - 0.92 ms - 2.77 ms + SQLite3 Loader + 2.91 K + 343.63 µs + ±12.64% + 330.21 µs + 495.04 µs -Comparison +Run Time Comparison @@ -331,29 +323,27 @@ Comparison - - + + - - + + - - - + + +
IPS Slower
Pg Loader883.24MyXQL Loader2.95 K  
SQLite3 Loader873.53Pg Loader2.92 K 1.01x
MyXQL Loader862.331.02xSQLite3 Loader2.91 K1.01x
-
- __Input: Time attr__ @@ -370,36 +360,36 @@ Run Time - SQLite3 Loader - 3.40 - 294.30 ms - ±15.60% - 281.23 ms - 367.65 ms + MyXQL Loader + 21.75 + 45.98 ms + ±11.17% + 45.25 ms + 61.90 ms - MyXQL Loader - 3.18 - 314.55 ms - ±17.11% - 313.35 ms - 415.89 ms + Pg Loader + 21.43 + 46.67 ms + ±10.90% + 45.76 ms + 61.66 ms - Pg Loader - 3.03 - 329.99 ms - ±16.46% - 321.68 ms - 457.42 ms + SQLite3 Loader + 20.72 + 48.26 ms + ±9.03% + 48.63 ms + 62.19 ms -Comparison +Run Time Comparison @@ -407,29 +397,27 @@ Comparison - - + + - - - + + + - - - + + +
IPS Slower
SQLite3 Loader3.40MyXQL Loader21.75  
MyXQL Loader3.181.07xPg Loader21.431.01x
Pg Loader3.031.12xSQLite3 Loader20.721.05x
-
- __Input: UUID attr__ @@ -447,35 +435,35 @@ Run Time SQLite3 Loader - 3.61 - 277.20 ms - ±16.59% - 266.05 ms - 372.08 ms + 23.23 + 43.06 ms + ±8.19% + 41.64 ms + 51.14 ms - Pg Loader - 2.75 - 363.06 ms - ±14.14% - 382.70 ms - 437.37 ms + MyXQL Loader + 15.19 + 65.81 ms + ±7.56% + 63.25 ms + 76.27 ms - MyXQL Loader - 2.73 - 365.91 ms - ±19.81% - 367.23 ms - 515.29 ms + Pg Loader + 14.89 + 67.14 ms + ±7.74% + 64.12 ms + 80.65 ms -Comparison +Run Time Comparison @@ -484,25 +472,20 @@ Comparison - + - - - + + + - - - + + + -
Slower
SQLite3 Loader3.6123.23  
Pg Loader2.751.31xMyXQL Loader15.191.53x
MyXQL Loader2.731.32xPg Loader14.891.56x
- - - -
- + \ No newline at end of file diff --git a/bench/results/to_sql.md b/bench/results/to_sql.md index c20b658..2e3bd44 100644 --- a/bench/results/to_sql.md +++ b/bench/results/to_sql.md @@ -1,7 +1,6 @@ +Benchmark -# Benchmark - -Benchmark run from 2021-03-24 02:02:16.278354Z UTC +Benchmark run from 2024-05-04 10:48:53.108179Z UTC ## System @@ -10,22 +9,22 @@ Benchmark suite executing on the following system: - + - + - + - + - +
Operating SystemLinuxmacOS
CPU InformationAMD Ryzen 7 PRO 4750U with Radeon GraphicsApple M3 Max
Number of Available Cores 16
Available Memory14.92 GB128 GB
Elixir Version1.11.31.16.2
Erlang Version23.2.626.2.4
@@ -50,7 +49,6 @@ Benchmark suite executing with the following configuration: - __Input: Complex Query 2 Joins__ Run Time @@ -66,36 +64,36 @@ Run Time - MyXQL Query Builder - 105.28 K - 9.50 μs - ±116.92% - 8.66 μs - 24.79 μs + Pg Query Builder + 382.74 K + 2.61 µs + ±374.25% + 2.38 µs + 3.42 µs - Pg Query Builder - 96.97 K - 10.31 μs - ±220.90% - 8.66 μs - 26.75 μs + SQLite3 Query Builder + 374.17 K + 2.67 µs + ±382.43% + 2.46 µs + 3.42 µs - SQLite3 Query Builder - 90.46 K - 11.05 μs - ±204.34% - 8.66 μs - 39.53 μs + MyXQL Query Builder + 360.95 K + 2.77 µs + ±360.52% + 2.54 µs + 3.58 µs -Comparison +Run Time Comparison @@ -103,29 +101,27 @@ Comparison - - + + - - - + + + - - - + + +
IPS Slower
MyXQL Query Builder105.28 KPg Query Builder382.74 K  
Pg Query Builder96.97 K1.09xSQLite3 Query Builder374.17 K1.02x
SQLite3 Query Builder90.46 K1.16xMyXQL Query Builder360.95 K1.06x
-
- __Input: Complex Query 4 Joins__ @@ -142,36 +138,36 @@ Run Time - Pg Query Builder - 100.25 K - 9.97 μs - ±92.25% - 9.15 μs - 23.68 μs + MyXQL Query Builder + 342.45 K + 2.92 µs + ±313.75% + 2.67 µs + 4.08 µs - MyXQL Query Builder - 98.47 K - 10.16 μs - ±88.64% - 9.22 μs - 28.70 μs + Pg Query Builder + 340.76 K + 2.93 µs + ±350.68% + 2.71 µs + 3.88 µs SQLite3 Query Builder - 90.40 K - 11.06 μs - ±117.48% - 9.22 μs - 38.69 μs + 336.08 K + 2.98 µs + ±328.89% + 2.79 µs + 3.88 µs -Comparison +Run Time Comparison @@ -179,29 +175,27 @@ Comparison - - + + - - - + + + - - + +
IPS Slower
Pg Query Builder100.25 KMyXQL Query Builder342.45 K  
MyXQL Query Builder98.47 K1.02xPg Query Builder340.76 K1.0x
SQLite3 Query Builder90.40 K1.11x336.08 K1.02x
-
- __Input: Fetch First Registry__ @@ -218,36 +212,36 @@ Run Time - Pg Query Builder - 173.31 K - 5.77 μs - ±389.85% - 4.47 μs - 20.18 μs + MyXQL Query Builder + 752.12 K + 1.33 µs + ±868.39% + 1.17 µs + 1.63 µs - MyXQL Query Builder - 173.06 K - 5.78 μs - ±411.02% - 4.82 μs - 16.20 μs + Pg Query Builder + 721.56 K + 1.39 µs + ±820.83% + 1.25 µs + 1.67 µs SQLite3 Query Builder - 167.17 K - 5.98 μs - ±420.59% - 4.82 μs - 19.42 μs + 703.84 K + 1.42 µs + ±817.39% + 1.29 µs + 1.71 µs -Comparison +Run Time Comparison @@ -255,29 +249,27 @@ Comparison - - + + - - - + + + - - + +
IPS Slower
Pg Query Builder173.31 KMyXQL Query Builder752.12 K  
MyXQL Query Builder173.06 K1.0xPg Query Builder721.56 K1.04x
SQLite3 Query Builder167.17 K1.04x703.84 K1.07x
-
- __Input: Fetch Last Registry__ @@ -294,36 +286,36 @@ Run Time - Pg Query Builder - 177.36 K - 5.64 μs - ±446.37% - 4.47 μs - 17.67 μs + MyXQL Query Builder + 731.59 K + 1.37 µs + ±795.20% + 1.21 µs + 1.71 µs - SQLite3 Query Builder - 169.74 K - 5.89 μs - ±394.73% - 4.82 μs - 18.16 μs + Pg Query Builder + 716.25 K + 1.40 µs + ±814.43% + 1.25 µs + 1.67 µs - MyXQL Query Builder - 162.83 K - 6.14 μs - ±383.32% - 4.82 μs - 22.14 μs + SQLite3 Query Builder + 707.98 K + 1.41 µs + ±839.02% + 1.25 µs + 1.67 µs -Comparison +Run Time Comparison @@ -331,29 +323,27 @@ Comparison - - + + - - - + + + - - - + + +
IPS Slower
Pg Query Builder177.36 KMyXQL Query Builder731.59 K  
SQLite3 Query Builder169.74 K1.04xPg Query Builder716.25 K1.02x
MyXQL Query Builder162.83 K1.09xSQLite3 Query Builder707.98 K1.03x
-
- __Input: Ordinary Delete All__ @@ -370,36 +360,36 @@ Run Time - Pg Query Builder - 335.81 K - 2.98 μs - ±775.40% - 2.03 μs - 9.43 μs + MyXQL Query Builder + 1.34 M + 747.69 ns + ±2442.90% + 666 ns + 917 ns - SQLite3 Query Builder - 325.92 K - 3.07 μs - ±948.36% - 2.02 μs - 8.31 μs + Pg Query Builder + 1.32 M + 758.40 ns + ±2490.71% + 667 ns + 875 ns - MyXQL Query Builder - 301.67 K - 3.31 μs - ±832.06% - 2.03 μs - 11.59 μs + SQLite3 Query Builder + 1.31 M + 762.71 ns + ±2451.66% + 667 ns + 875 ns -Comparison +Run Time Comparison @@ -407,29 +397,27 @@ Comparison - - + + - - - + + + - - - + + +
IPS Slower
Pg Query Builder335.81 KMyXQL Query Builder1.34 M  
SQLite3 Query Builder325.92 K1.03xPg Query Builder1.32 M1.01x
MyXQL Query Builder301.67 K1.11xSQLite3 Query Builder1.31 M1.02x
-
- __Input: Ordinary Order By__ @@ -446,36 +434,36 @@ Run Time - Pg Query Builder - 186.48 K - 5.36 μs - ±400.39% - 4.40 μs - 17.81 μs + MyXQL Query Builder + 742.88 K + 1.35 µs + ±918.40% + 1.17 µs + 1.67 µs - MyXQL Query Builder - 182.02 K - 5.49 μs - ±402.64% - 4.40 μs - 17.39 μs + Pg Query Builder + 729.59 K + 1.37 µs + ±913.47% + 1.21 µs + 1.58 µs SQLite3 Query Builder - 172.19 K - 5.81 μs - ±427.38% - 4.47 μs - 21.44 μs + 721.11 K + 1.39 µs + ±918.31% + 1.21 µs + 1.58 µs -Comparison +Run Time Comparison @@ -483,29 +471,27 @@ Comparison - - + + - - + + - - + +
IPS Slower
Pg Query Builder186.48 KMyXQL Query Builder742.88 K  
MyXQL Query Builder182.02 KPg Query Builder729.59 K 1.02x
SQLite3 Query Builder172.19 K1.08x721.11 K1.03x
-
- __Input: Ordinary Select All__ @@ -522,36 +508,36 @@ Run Time - Pg Query Builder - 200.19 K - 5.00 μs - ±657.77% - 3.49 μs - 14.88 μs + MyXQL Query Builder + 838.25 K + 1.19 µs + ±1122.54% + 1.04 µs + 1.46 µs - SQLite3 Query Builder - 199.77 K - 5.01 μs - ±698.35% - 3.49 μs - 13.90 μs + Pg Query Builder + 822.53 K + 1.22 µs + ±1179.00% + 1.08 µs + 1.38 µs - MyXQL Query Builder - 191.70 K - 5.22 μs - ±601.95% - 3.56 μs - 16.06 μs + SQLite3 Query Builder + 809.74 K + 1.23 µs + ±1198.22% + 1.08 µs + 1.42 µs -Comparison +Run Time Comparison @@ -559,20 +545,20 @@ Comparison - - + + - - - + + + - - + + @@ -580,8 +566,6 @@ Comparison -
- __Input: Ordinary Update All__ @@ -598,36 +582,36 @@ Run Time - - - - - - + + + + + + - - - - - - + + + + + + - - - - - - + + + + + +
IPS Slower
Pg Query Builder200.19 KMyXQL Query Builder838.25 K  
SQLite3 Query Builder199.77 K1.0xPg Query Builder822.53 K1.02x
MyXQL Query Builder191.70 KSQLite3 Query Builder809.74 K 1.04x
SQLite3 Query Builder263.35 K3.80 μs±1304.70%2.86 μs10.13 μsMyXQL Query Builder1.03 M971.15 ns±1801.37%834 ns1167 ns
MyXQL Query Builder260.96 K3.83 μs±760.59%2.86 μs10.20 μsPg Query Builder1.01 M990.87 ns±1533.42%875 ns1125 ns
Pg Query Builder249.05 K4.02 μs±827.04%2.86 μs12.99 μsSQLite3 Query Builder1.00 M999.05 ns±1416.75%875 ns1166 ns
-Comparison +Run Time Comparison @@ -635,29 +619,27 @@ Comparison - - + + - - - + + + - - - + + +
IPS Slower
SQLite3 Query Builder263.35 KMyXQL Query Builder1.03 M  
MyXQL Query Builder260.96 K1.01xPg Query Builder1.01 M1.02x
Pg Query Builder249.05 K1.06xSQLite3 Query Builder1.00 M1.03x
-
- __Input: Ordinary Where__ @@ -675,35 +657,35 @@ Run Time MyXQL Query Builder - 162.26 K - 6.16 μs - ±343.86% - 5.31 μs - 17.74 μs + 579.57 K + 1.73 µs + ±668.48% + 1.54 µs + 2.17 µs - SQLite3 Query Builder - 159.31 K - 6.28 μs - ±305.50% - 5.38 μs - 19.35 μs + Pg Query Builder + 574.41 K + 1.74 µs + ±631.02% + 1.58 µs + 2.13 µs - Pg Query Builder - 154.28 K - 6.48 μs - ±355.38% - 5.45 μs - 20.04 μs + SQLite3 Query Builder + 568.60 K + 1.76 µs + ±635.93% + 1.62 µs + 2.13 µs -Comparison +Run Time Comparison @@ -712,25 +694,20 @@ Comparison - + - - - + + + - - - + + + -
Slower
MyXQL Query Builder162.26 K579.57 K  
SQLite3 Query Builder159.31 K1.02xPg Query Builder574.41 K1.01x
Pg Query Builder154.28 K1.05xSQLite3 Query Builder568.60 K1.02x
- - - -
- + \ No newline at end of file