Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildkite fixes #229

Merged
merged 11 commits into from
Jan 27, 2024
Merged

buildkite fixes #229

merged 11 commits into from
Jan 27, 2024

Conversation

Krastanov
Copy link
Member

No description provided.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 26, 2024 •
edited
Loading

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 26 Jan 2024 - 20:46
  • Baseline: 26 Jan 2024 - 20:52
  • Package commits:
  • Target: 945a13
  • Baseline: 3c796e
  • Julia commits:
  • Target: c42df6
  • Baseline: c42df6
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.92 (5%) ✅ 1.00 (1%)
["circuitsim", "mctrajectories_union", "q1001_r1"] 0.95 (5%) ✅ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_destab"] 0.87 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 0.79 (5%) ✅ 0.99 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.00 (5%) ✅ 0.01 (1%) ✅
["stabilizer", "tensor", "diag_pow5_20"] 1.46 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2551 MHz 2161 s 0 s 171 s 6208 s 0 s
 #2 3057 MHz 2846 s 0 s 174 s 5529 s 0 s
 #3 2445 MHz 2640 s 0 s 207 s 5699 s 0 s
 #4 3243 MHz 2625 s 0 s 192 s 5724 s 0 s
 Memory: 15.60690689086914 GB (13626.49609375 MB free)
 Uptime: 858.44 sec
 Load Avg: 1.03 1.05 0.77
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3192 MHz 3504 s 0 s 220 s 8780 s 0 s
 #2 2445 MHz 3377 s 0 s 204 s 8930 s 0 s
 #3 3275 MHz 3647 s 0 s 251 s 8611 s 0 s
 #4 3241 MHz 3657 s 0 s 232 s 8615 s 0 s
 Memory: 15.60690689086914 GB (13518.60546875 MB free)
 Uptime: 1255.49 sec
 Load Avg: 1.15 1.09 0.9
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 26 Jan 2024 - 20:46
  • Package commit: 945a13
  • Julia commit: c42df6
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.272 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 174.615 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.130 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.390 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.239 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.408 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.501 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 169.475 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.093 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.991 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.332 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 154.077 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.083 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.843 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.488 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.932 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.927 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.055 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.364 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.306 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.666 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.604 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.575 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.256 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.389 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.275 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.285 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.664 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.114 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.623 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.441 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 685.146 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 567.857 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.749 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.648 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 2.871 ms (5%) 3.19 MiB (1%) 82083
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.503 ms (5%) 4.52 MiB (1%) 102380
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 5.063 ms (5%) 4.59 MiB (1%) 102959
["ecc", "evaluate_decoder", "shor_pybp_comm"] 16.967 ms (5%) 4.92 MiB (1%) 125524
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 32.365 ms (5%) 8.37 MiB (1%) 192598
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 33.141 ms (5%) 8.45 MiB (1%) 193548
["ecc", "evaluate_decoder", "shor_table_comm"] 737.262 μs (5%) 1.21 MiB (1%) 35095
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.183 ms (5%) 972.00 KiB (1%) 20493
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.725 ms (5%) 1.03 MiB (1%) 21450
["ecc", "evaluate_decoder", "toric8_bp_comm"] 92.610 ms (5%) 28.03 MiB (1%) 211961
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 142.740 ms (5%) 3.364 ms 41.94 MiB (1%) 312600
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 197.391 ms (5%) 44.16 MiB (1%) 341323
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 743.879 ms (5%) 10.49 MiB (1%) 144976
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 939.376 ms (5%) 20.40 MiB (1%) 265495
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.006 s (5%) 22.72 MiB (1%) 295011
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 30.359 ms (5%) 6.98 MiB (1%) 143976
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 76.835 ms (5%) 13.41 MiB (1%) 263495
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 131.215 ms (5%) 15.73 MiB (1%) 293011
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.429 ms (5%) 2.31 MiB (1%) 43713
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 31.006 ms (5%) 5.08 MiB (1%) 86022
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 84.870 ms (5%) 7.41 MiB (1%) 115635
["pauli", "mul", "100"] 14.440 ns (5%)
["pauli", "mul", "1000"] 17.905 ns (5%)
["pauli", "mul", "100000"] 722.142 ns (5%)
["pauli", "mul", "20000000"] 194.191 μs (5%)
["stabilizer", "canon", "cano500"] 3.023 ms (5%)
["stabilizer", "canon", "diag_cano500"] 709.060 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.222 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 626.877 μs (5%)
["stabilizer", "canon", "gott500"] 6.599 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.530 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.564 ms (5%)
["stabilizer", "canon", "rref500"] 3.079 ms (5%)
["stabilizer", "project", "destabilizer"] 18.444 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.556 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.433 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.960 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.559 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 23.423 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2551 MHz 2161 s 0 s 171 s 6208 s 0 s
 #2 3057 MHz 2846 s 0 s 174 s 5529 s 0 s
 #3 2445 MHz 2640 s 0 s 207 s 5699 s 0 s
 #4 3243 MHz 2625 s 0 s 192 s 5724 s 0 s
 Memory: 15.60690689086914 GB (13626.49609375 MB free)
 Uptime: 858.44 sec
 Load Avg: 1.03 1.05 0.77
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 26 Jan 2024 - 20:52
  • Package commit: 3c796e
  • Julia commit: c42df6
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.091 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 170.006 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.335 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 117.278 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.001 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 116.256 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 60.222 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 170.767 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.098 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.051 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.153 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 156.321 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.082 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.831 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.448 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.612 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 133.207 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.055 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.354 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.662 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.614 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 49.613 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.174 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.479 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.254 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.302 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.658 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.594 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.450 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 685.175 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.158 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 568.328 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.759 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.679 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.284 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 3.616 ms (5%) 3.20 MiB (1%) 82364
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.736 ms (5%) 4.52 MiB (1%) 102292
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 5.086 s (5%) 58.923 ms 420.30 MiB (1%) 8652549
["ecc", "evaluate_decoder", "shor_pybp_comm"] 16.942 ms (5%) 4.92 MiB (1%) 125524
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 32.461 ms (5%) 8.37 MiB (1%) 192598
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 32.797 ms (5%) 8.45 MiB (1%) 193548
["ecc", "evaluate_decoder", "shor_table_comm"] 756.177 μs (5%) 1.21 MiB (1%) 35117
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.162 ms (5%) 972.28 KiB (1%) 20500
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.724 ms (5%) 1.03 MiB (1%) 21443
["ecc", "evaluate_decoder", "toric8_bp_comm"] 90.950 ms (5%) 28.15 MiB (1%) 212767
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 140.571 ms (5%) 42.21 MiB (1%) 314320
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 195.958 ms (5%) 44.48 MiB (1%) 343423
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 744.182 ms (5%) 10.49 MiB (1%) 144976
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 951.454 ms (5%) 20.40 MiB (1%) 265495
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.007 s (5%) 22.72 MiB (1%) 295011
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 30.235 ms (5%) 6.98 MiB (1%) 143976
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 77.385 ms (5%) 13.41 MiB (1%) 263495
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 132.026 ms (5%) 15.73 MiB (1%) 293011
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.460 ms (5%) 2.31 MiB (1%) 43735
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 31.030 ms (5%) 5.08 MiB (1%) 86018
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 85.066 ms (5%) 7.41 MiB (1%) 115530
["pauli", "mul", "100"] 14.903 ns (5%)
["pauli", "mul", "1000"] 17.764 ns (5%)
["pauli", "mul", "100000"] 724.540 ns (5%)
["pauli", "mul", "20000000"] 196.375 μs (5%)
["stabilizer", "canon", "cano500"] 3.019 ms (5%)
["stabilizer", "canon", "diag_cano500"] 709.188 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.227 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 627.007 μs (5%)
["stabilizer", "canon", "gott500"] 6.598 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.524 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.561 ms (5%)
["stabilizer", "canon", "rref500"] 3.082 ms (5%)
["stabilizer", "project", "destabilizer"] 18.303 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.605 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 980.866 μs (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.851 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 30.958 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 23.593 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3192 MHz 3504 s 0 s 220 s 8780 s 0 s
 #2 2445 MHz 3377 s 0 s 204 s 8930 s 0 s
 #3 3275 MHz 3647 s 0 s 251 s 8611 s 0 s
 #4 3241 MHz 3657 s 0 s 232 s 8615 s 0 s
 Memory: 15.60690689086914 GB (13518.60546875 MB free)
 Uptime: 1255.49 sec
 Load Avg: 1.15 1.09 0.9
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.86
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 26 Jan 2024 - 20:50
  • Baseline: 26 Jan 2024 - 20:57
  • Package commits:
  • Target: 37621c
  • Baseline: 3c796e
  • Julia commits:
  • Target: c42df6
  • Baseline: c42df6
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.95 (5%) ✅ 1.00 (1%)
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 1.08 (5%) ❌ 1.00 (1%)
["circuitsim", "mctrajectories_union", "q101_r1"] 1.08 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r1"] 1.14 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r100"] 1.13 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r10000"] 1.13 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q101_r1"] 1.06 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.08 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.11 (5%) ❌ 1.00 (1%)
["clifford", "dense", "cnot_on_dense500_destab"] 0.94 (5%) ✅ 1.00 (1%)
["clifford", "dense", "cnot_on_diag500_stab"] 0.93 (5%) ✅ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_destab"] 0.87 (5%) ✅ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_stab"] 1.08 (5%) ❌ 1.00 (1%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 0.93 (5%) ✅ 1.00 (1%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 0.92 (5%) ✅ 1.00 (1%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 0.92 (5%) ✅ 1.00 (1%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 0.92 (5%) ✅ 1.00 (1%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 0.92 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.00 (5%) ✅ 0.01 (1%) ✅
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.05 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_table_comm"] 1.07 (5%) ❌ 1.00 (1%)
["pauli", "mul", "100"] 1.08 (5%) ❌ 1.00 (1%)
["pauli", "mul", "1000"] 0.89 (5%) ✅ 1.00 (1%)
["pauli", "mul", "100000"] 1.05 (5%) ❌ 1.00 (1%)
["stabilizer", "canon", "diag_cano500"] 0.92 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "diag_gott500"] 0.94 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "gott500"] 1.06 (5%) ❌ 1.00 (1%)
["stabilizer", "canon", "md_cano500"] 0.92 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "md_rref500"] 0.90 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "rref500"] 1.08 (5%) ❌ 1.00 (1%)
["stabilizer", "project", "destabilizer"] 1.08 (5%) ❌ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 1.10 (5%) ❌ 1.00 (1%)
["stabilizer", "tensor", "pow5_20"] 1.08 (5%) ❌ 1.00 (1%)
["stabilizer", "trace", "destabilizer"] 1.08 (5%) ❌ 1.00 (1%)
["stabilizer", "trace", "stabilizer"] 1.09 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2675 MHz 2367 s 0 s 177 s 8031 s 0 s
 #2 2445 MHz 2757 s 0 s 229 s 7588 s 0 s
 #3 3243 MHz 2833 s 0 s 199 s 7543 s 0 s
 #4 2975 MHz 2236 s 0 s 188 s 8152 s 0 s
 Memory: 15.60690689086914 GB (13631.90625 MB free)
 Uptime: 1061.7 sec
 Load Avg: 1.0 1.04 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3242 MHz 3220 s 0 s 214 s 11090 s 0 s
 #2 2445 MHz 3527 s 0 s 276 s 10720 s 0 s
 #3 2521 MHz 4218 s 0 s 241 s 10065 s 0 s
 #4 2445 MHz 3133 s 0 s 240 s 11152 s 0 s
 Memory: 15.60690689086914 GB (13487.546875 MB free)
 Uptime: 1457.31 sec
 Load Avg: 1.04 1.03 0.88
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 26 Jan 2024 - 20:50
  • Package commit: 37621c
  • Julia commit: c42df6
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.594 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 183.492 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.070 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.992 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.355 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 118.040 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 62.857 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 178.644 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.145 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.331 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 43.901 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 158.306 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.102 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.861 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.448 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.213 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 137.096 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.086 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.394 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 10.807 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.707 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.034 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 517.786 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 46.897 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 25.177 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.129 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 12.404 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.351 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.708 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 904.879 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 561.568 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.447 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 633.483 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.070 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 525.681 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.388 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.295 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.318 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.113 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 2.836 ms (5%) 3.19 MiB (1%) 82023
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.491 ms (5%) 4.53 MiB (1%) 102654
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 5.016 ms (5%) 4.59 MiB (1%) 103020
["ecc", "evaluate_decoder", "shor_pybp_comm"] 16.882 ms (5%) 4.92 MiB (1%) 125524
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 32.929 ms (5%) 8.37 MiB (1%) 192598
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 32.633 ms (5%) 8.45 MiB (1%) 193548
["ecc", "evaluate_decoder", "shor_table_comm"] 692.181 μs (5%) 1.21 MiB (1%) 35139
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.070 ms (5%) 972.00 KiB (1%) 20493
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.689 ms (5%) 1.03 MiB (1%) 21457
["ecc", "evaluate_decoder", "toric8_bp_comm"] 90.064 ms (5%) 28.00 MiB (1%) 211795
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 141.464 ms (5%) 42.05 MiB (1%) 313281
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 192.443 ms (5%) 44.63 MiB (1%) 344308
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 733.138 ms (5%) 10.49 MiB (1%) 144976
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 912.049 ms (5%) 20.40 MiB (1%) 265495
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.000 s (5%) 22.72 MiB (1%) 295011
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 29.598 ms (5%) 6.98 MiB (1%) 143976
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 74.694 ms (5%) 13.41 MiB (1%) 263495
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 128.062 ms (5%) 15.73 MiB (1%) 293011
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.451 ms (5%) 2.31 MiB (1%) 43713
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 30.904 ms (5%) 5.08 MiB (1%) 86026
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 85.065 ms (5%) 7.41 MiB (1%) 115523
["pauli", "mul", "100"] 14.506 ns (5%)
["pauli", "mul", "1000"] 16.831 ns (5%)
["pauli", "mul", "100000"] 726.394 ns (5%)
["pauli", "mul", "20000000"] 190.165 μs (5%)
["stabilizer", "canon", "cano500"] 2.764 ms (5%)
["stabilizer", "canon", "diag_cano500"] 654.512 μs (5%)
["stabilizer", "canon", "diag_gott500"] 3.801 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 650.494 μs (5%)
["stabilizer", "canon", "gott500"] 6.454 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.448 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.485 ms (5%)
["stabilizer", "canon", "rref500"] 3.063 ms (5%)
["stabilizer", "project", "destabilizer"] 18.224 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.446 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.487 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.858 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 33.553 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 23.754 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2675 MHz 2367 s 0 s 177 s 8031 s 0 s
 #2 2445 MHz 2757 s 0 s 229 s 7588 s 0 s
 #3 3243 MHz 2833 s 0 s 199 s 7543 s 0 s
 #4 2975 MHz 2236 s 0 s 188 s 8152 s 0 s
 Memory: 15.60690689086914 GB (13631.90625 MB free)
 Uptime: 1061.7 sec
 Load Avg: 1.0 1.04 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 26 Jan 2024 - 20:57
  • Package commit: 3c796e
  • Julia commit: c42df6
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.422 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 182.931 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.802 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 110.026 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 14.041 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 109.314 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 55.073 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 157.545 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.010 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.991 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.103 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 153.877 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.022 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.875 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.518 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.602 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.097 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 978.687 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.347 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.709 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.054 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 517.997 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 49.673 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.095 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.139 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.305 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.346 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.510 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 1.035 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 517.856 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.433 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 678.597 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.624 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.428 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.678 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 2.721 ms (5%) 3.21 MiB (1%) 82448
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.577 ms (5%) 4.52 MiB (1%) 102268
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.979 s (5%) 40.795 ms 420.33 MiB (1%) 8653021
["ecc", "evaluate_decoder", "shor_pybp_comm"] 17.095 ms (5%) 4.92 MiB (1%) 125524
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 32.875 ms (5%) 8.37 MiB (1%) 192598
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 32.573 ms (5%) 8.45 MiB (1%) 193548
["ecc", "evaluate_decoder", "shor_table_comm"] 698.634 μs (5%) 1.21 MiB (1%) 35117
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.096 ms (5%) 972.56 KiB (1%) 20507
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.604 ms (5%) 1.03 MiB (1%) 21457
["ecc", "evaluate_decoder", "toric8_bp_comm"] 91.468 ms (5%) 28.01 MiB (1%) 211826
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 144.507 ms (5%) 1.625 ms 41.77 MiB (1%) 311199
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 199.298 ms (5%) 44.52 MiB (1%) 343988
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 723.472 ms (5%) 10.49 MiB (1%) 144976
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 938.495 ms (5%) 20.40 MiB (1%) 265495
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 987.750 ms (5%) 22.72 MiB (1%) 295011
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 29.713 ms (5%) 6.98 MiB (1%) 143976
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 74.310 ms (5%) 13.41 MiB (1%) 263495
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 129.227 ms (5%) 15.73 MiB (1%) 293011
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.294 ms (5%) 2.31 MiB (1%) 43702
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 29.562 ms (5%) 5.08 MiB (1%) 86066
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 84.635 ms (5%) 7.41 MiB (1%) 115584
["pauli", "mul", "100"] 13.472 ns (5%)
["pauli", "mul", "1000"] 18.820 ns (5%)
["pauli", "mul", "100000"] 689.315 ns (5%)
["pauli", "mul", "20000000"] 182.110 μs (5%)
["stabilizer", "canon", "cano500"] 2.768 ms (5%)
["stabilizer", "canon", "diag_cano500"] 708.693 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.041 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 649.603 μs (5%)
["stabilizer", "canon", "gott500"] 6.078 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.575 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.647 ms (5%)
["stabilizer", "canon", "rref500"] 2.839 ms (5%)
["stabilizer", "project", "destabilizer"] 16.822 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.426 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.353 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.644 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 30.927 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 21.700 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3242 MHz 3220 s 0 s 214 s 11090 s 0 s
 #2 2445 MHz 3527 s 0 s 276 s 10720 s 0 s
 #3 2521 MHz 4218 s 0 s 241 s 10065 s 0 s
 #4 2445 MHz 3133 s 0 s 240 s 11152 s 0 s
 Memory: 15.60690689086914 GB (13487.546875 MB free)
 Uptime: 1457.31 sec
 Load Avg: 1.04 1.03 0.88
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.86
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 26 Jan 2024 - 21:55
  • Baseline: 26 Jan 2024 - 22:02
  • Package commits:
  • Target: 24a3ba
  • Baseline: 3c796e
  • Julia commits:
  • Target: c42df6
  • Baseline: c42df6
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.90 (5%) ✅ 1.00 (1%)
["circuitsim", "pftrajectories", "q1001_r1"] 1.06 (5%) ❌ 1.00 (1%)
["circuitsim", "pftrajectories", "q101_r1"] 1.05 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.00 (5%) ✅ 0.01 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_comm"] 0.93 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 0.94 (5%) ✅ 0.99 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 1.09 (5%) ❌ 1.00 (1%)
["stabilizer", "tensor", "pow5_20"] 1.06 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 2445 MHz 2137 s 0 s 198 s 6638 s 0 s
 #2 3137 MHz 2906 s 0 s 217 s 5856 s 0 s
 #3 3243 MHz 2683 s 0 s 216 s 6067 s 0 s
 #4 3229 MHz 3166 s 0 s 218 s 5604 s 0 s
 Memory: 15.60690689086914 GB (13547.74609375 MB free)
 Uptime: 903.27 sec
 Load Avg: 1.03 1.09 0.84
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.0-DEV.1384
Commit c42df604d99 (2024-01-26 16:41 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.3 LTS
 uname: Linux 6.2.0-1018-azure #18~22.04.1-Ubuntu SMP Tue Nov 21 19:25:02 UTC 2023 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 3243 MHz 2887 s 0 s 250 s 9904 s 0 s
 #2 2445 MHz 3336 s 0 s 267 s 9441 s 0 s
 #3 3242 MHz 3968 s 0 s 261 s 8806 s 0 s
 #4 3197 MHz 4713 s 0 s 275 s 8070 s 0 s
 Memory: 15.60690689086914 GB (13504.41796875 MB free)
 Uptime: 1310.89 sec
 Load Avg: 1.0 1.02 0.92
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 26 Jan 2024 - 21:55
  • Package commit: 24a3ba
  • Julia commit: c42df6
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.628 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 182.941 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.192 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 119.192 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.393 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.078 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 61.505 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 176.789 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.146 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.181 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 43.821 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 159.287 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.134 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.837 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.478 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.612 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 136.765 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.102 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.344 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.660 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.244 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.384 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 22.812 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.289 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 12.984 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.353 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.659 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 979.956 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.213 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.437 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.219 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.370 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.759 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.669 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.304 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 2.833 ms (5%) 3.18 MiB (1%) 81891
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.559 ms (5%) 4.51 MiB (1%) 102071
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 5.121 ms (5%) 4.62 MiB (1%) 103628
["ecc", "evaluate_decoder", "shor_pybp_comm"] 17.530 ms (5%) 4.92 MiB (1%) 125524
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 33.242 ms (5%) 8.37 MiB (1%) 192598
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 33.692 ms (5%) 8.45 MiB (1%) 193548
["ecc", "evaluate_decoder", "shor_table_comm"] 744.978 μs (5%) 1.21 MiB (1%) 35117
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 1.163 ms (5%) 972.00 KiB (1%) 20493
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.695 ms (5%) 1.03 MiB (1%) 21457
["ecc", "evaluate_decoder", "toric8_bp_comm"] 89.735 ms (5%) 27.92 MiB (1%) 211004
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 141.920 ms (5%) 41.86 MiB (1%) 312053
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 198.617 ms (5%) 44.48 MiB (1%) 343394
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 756.694 ms (5%) 10.49 MiB (1%) 144976
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 931.909 ms (5%) 20.40 MiB (1%) 265495
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.012 s (5%) 22.72 MiB (1%) 295011
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 30.920 ms (5%) 6.98 MiB (1%) 143976
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 77.669 ms (5%) 13.41 MiB (1%) 263495
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 133.455 ms (5%) 15.73 MiB (1%) 293011
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.430 ms (5%) 2.31 MiB (1%) 43735
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 30.947 ms (5%) 5.08 MiB (1%) 86019
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 85.494 ms (5%) 7.41 MiB (1%) 115566
["pauli", "mul", "100"] 14.596 ns (5%)
["pauli", "mul", "1000"] 17.909 ns (5%)
["pauli", "mul", "100000"] 727.627 ns (5%)
["pauli", "mul", "20000000"] 193.410 μs (5%)
["stabilizer", "canon", "cano500"] 3.007 ms (5%)
["stabilizer", "canon", "diag_cano500"] 676.461 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.086 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 627.068 μs (5%)
["stabilizer", "canon", "gott500"] 6.475 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.583 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.620 ms (5%)
["stabilizer", "canon", "rref500"] 3.094 ms (5%)
["stabilizer", "project", "destabilizer"] 18.554 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.446 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.100 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.879 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 34.054 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 23.384 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • `["circuitsim", "mctrajectories_union"]...[Comment body truncated]

@codecov Codecov
Copy link

codecov bot commented Jan 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (3c796e7) 82.90% compared to head (222feef) 82.90%.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master     #229   +/-   ##
=======================================
  Coverage   82.90%   82.90%           
=======================================
  Files          55       55           
  Lines        3673     3673           
=======================================
  Hits         3045     3045           
  Misses        628      628

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Krastanov Krastanov merged commit 1b9db49 into master Jan 27, 2024
12 of 13 checks passed
@Krastanov Krastanov deleted the buildkite_fixes branch January 27, 2024 01:20
This was referenced Jan 27, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Krastanov