This repository has been archived by the owner on Mar 22, 2023. It is now read-only.
Benchmark scenarios #45
Labels
Comments
|
Basic scope: Key/value sizes: Benchmarks: Number of elements: Number of threads (for concurrent engines): Number of dimms: Medium:
Engines: |
karczex
pushed a commit
to karczex/pmemkv-bench
that referenced
this issue
Feb 19, 2021
Instead of passing JSON, user may pass python script with implemented method generate(), which returns dictionary. Such dictionary is validated against json schema file. To do so, build and benchmark configurations was split into two files. generate_obj_based_scope.py may be run as standalone application, which prints configuration json (also validated against schema) to stdout. Such once generated json may be saved and passed to run_benchmark.py. ref pmem#45
karczex
pushed a commit
to karczex/pmemkv-bench
that referenced
this issue
Mar 17, 2021
Instead of passing JSON, user may pass python script with implemented method generate(), which returns dictionary. Such dictionary is validated against json schema file. To do so, build and benchmark configurations was split into two files. generate_obj_based_scope.py may be run as standalone application, which prints configuration json (also validated against schema) to stdout. Such once generated json may be saved and passed to run_benchmark.py. ref pmem#45
karczex
pushed a commit
to karczex/pmemkv-bench
that referenced
this issue
Apr 8, 2021
Instead of passing JSON, user may pass python script with implemented method generate(), which returns dictionary. Such dictionary is validated against json schema file. To do so, build and benchmark configurations was split into two files. generate_obj_based_scope.py may be run as standalone application, which prints configuration json (also validated against schema) to stdout. Such once generated json may be saved and passed to run_benchmark.py. ref pmem#45
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Ideally we would test all the combinations. After analysis we can reduce the numbers to the most representative ones.
Metrics we should collect:
bandwidth, ops/sec, latency
Key/value sizes:
8/8, 24/24, 8/128, 128/128, 256/256,
Benchmarks:
fillseq,
fillrandom
readseq, readseq (after fillseq)
readseq, readseq (after fillrandom)
readrandom, readrandom (after fillseq)
readrandom, readrandom (after fillrandom)
overwrite, overwrite (after fillseq)
overwrite, overwrite (after fillrandom)
readmissing, readmissing (after filrandom or fillseq)
deleteseq (after fillseq)
deleteseq (after fillrandom)
deleterandom (after fillseq)
deleterandom (after fillsrandom)
readwhilewriting
readwhilewriting (after fillrandom)
readwhilewriting (after fillseq)
readrandomwriterandom
readrandomwriterandom (after fillrandom)
readrandomwriterandom (after fillseq)
Number of elements:
1 000 000, 10 000 000, 100 000 000
Number of threads (for concurrent engines):
1, 2, 4, 6, 8, 12, 18, 24
Flags:
disjoint=1
disjoint=0
Number of dimms:
1, 6 interleaved
Numa node:
OPTIONAL (second step):
Pool utilization (TODO: think how to approximate the pool size):
10%, 50%, 90%
fsdax/devdax
compilers:
clang++
g++
The text was updated successfully, but these errors were encountered: