Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: export option for full fidelity Histogram? #82

Closed
heckj opened this issue Mar 4, 2023 · 3 comments
Closed

Question: export option for full fidelity Histogram? #82

heckj opened this issue Mar 4, 2023 · 3 comments

Comments

@heckj
Copy link
Contributor

heckj commented Mar 4, 2023

I was looking through the export options, spotted the JMH and .tsv exports, but one of the things I wanted to explore was taking the full-fidelity Histogram from package-histogram, loading the file back in using Histogram's Codable conformance, and exploring some local visualization with it (SwiftUI Charts, etc).

I'm not familiar with .tsv (other than an assumption is stands for "time series values") and wanted to ask - is that a more sane path for reloading into my own Histogram instance, or would it be reasonable to extend the exports to drop out a JSON from Histogram's codable? I'm reading and learning on package-histogram, but not 100% on all the pieces and parts, and what's legit for reproducing it by reading in a file.

I'd presumed, but not yet traced, that dumping a ton of values into even a HdrHistogram was fundamentally lossy - and that it didn't store the entirety of all the values submitted to it. I'm happy to do the PR to enable this thing I want, figured I'd best ask first - maybe there's a path that's easily there I'm overlooking.

@hassila
Copy link
Contributor

hassila commented Mar 4, 2023

https://github.com/ordo-one/package-histogram/issues/2

Would be the proper solution I think - the upstream hdrhistogram supports a binary format for this that should be supported in that context.

The histogram basically stores a count of all values with the specified precision so it's possible to get out all values with an iterator (as is done by eg. Jmh export).

The codable support for histogram works ("all" samples are encoded logically), but it's not very efficient as the vector of counts (not individual samples) can be reasonably large depending on the magnitude and precision of captured values and json is... json.

@hassila
Copy link
Contributor

hassila commented Mar 4, 2023

(Adding codable to histogram was really a short term fix for the use of benchmark)

That being said, I don't see any issue with adding an export option for the histograms codable representation- "--format=histogram-json" or "histogram-codable"? Happily take a PR for it.

Later we can add the binary representation "hgrm".

Side note is that the tsv export should be improved to support multiple baselines.
It's primarily intended to be used as input to youplot or gnuplot.

Also need to standardize file naming a bit I think - let me think a bit, perhaps BenchmarkResult should get a function that returns a standardized file name for the multi export case.

hassila pushed a commit that referenced this issue Mar 6, 2023
A work-around solution to #82, adding a feature to export the underlying
histograms using their codable conformance. The preferred longer term
solution requires additional features/capabilities on package-histogram
to support cross-platform encoding and decoding of the raw histogram
data.

This encoded histogram *does not* include the units of measurement
selected for the various benchmarks but is in raw form (which is `nanoseconds` for time measurements and just `count` for countable measurements).
@hassila
Copy link
Contributor

hassila commented Mar 6, 2023

Fixed by #84

@hassila hassila closed this as completed Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants