chore: Updated banchmark README.md #410
Conversation
|
Also added a man page for #325 |
There was a problem hiding this comment.
Hey @Walker-00,
in contrast to #357, this PR only updates the table of benchmark results, while leaving the benchmark itself untouched. However, I would not want to put performance numbers into our README unless they are backed by a reproducible benchmark setup.
In addition to the comment regarding the man page, I would also prefer if you could work on these two concerns in two separate PRs, so that the discussions don't overlap.
| | Client (50 runs, 31.01.2025) | Programming Language | Mean in ms | Deviation in ms | Comments | | ||
| | :---: | :---: | :---: | :---: | :---: | | ||
| | [`outfieldr`][outfieldr-gh] | Zig | ??? | ??? | lacks maintenance and features | | ||
| | `tealdeer` | Rust | 2.0 | 0.4 | | | ||
| | [`fast-tldr`][fast-tldr-gh] | Haskell | 2.9 | 0.4 | no example highlighting | | ||
| | [`tldr-c`][c-gh] | C | 4.9 | 0.6 | | | ||
| | [`tldr-hs`][hs-gh] | Haskell | 12.4 | 0.2 | no example highlighting | | ||
| | [`tldr-python-client`][python-gh] | Python | 59.3 | 2.5 | | | ||
| | [`tldr-bash`][bash-gh] | Bash | 100.5 | 7.1 | | | ||
| | [`tldr-node-client`][node-gh] | JavaScript / NodeJS | 282.8 | 10.7 | | |
There was a problem hiding this comment.
How did you generate these numbers? The main reason we haven't updated the benchmarks already is that no one felt like updating the Dockerfile mentioned in the paragraph above the table. As described in the README, the benchmark should be easily reproducible, otherwise the numbers have little meaning
There was a problem hiding this comment.
I didn't run Dockerfile, I manually downloaded them and ran them on my machine. I will fix the Docker and rewrite the benchmarks.
| @@ -0,0 +1,132 @@ | |||
| TEALDEER(1) User Commands TEALDEER(1) | |||
There was a problem hiding this comment.
In #325 we concluded that we want to have a man page that is automatically generated from the documentation. This looks like it is created by hand, which I would prefer to not add, as it adds more places that need to be updated when, for example, command line parameters change
There was a problem hiding this comment.
You mean like auto generated from the cargo-doc?
| @@ -116,7 +117,7 @@ Thanks to @severen for coming up with the name "tealdeer"! | |||
| [c-gh]: https://github.com/tldr-pages/tldr-c-client | |||
| [hs-gh]: https://github.com/psibi/tldr-hs | |||
| [fast-tldr-gh]: https://github.com/gutjuri/fast-tldr | |||
| [bash-gh]: https://4e4.win/tldr | |||
| [bash-gh]: https://github.com/raylee/tldr-sh-client | |||
There was a problem hiding this comment.
For context, it seems like the URL we used previously was changed some time ago: pepa65/tldr-bash-client@6321f7c
However, the client there seems to be unmaintained anyways, so using the one from raylee seems sensible. Still, it would also have to be changed in the Dockerfile
Hey there!, I'm back.
Here’s what I’ve updated in this PR:
Fixed the
tldr-bashlink:The old link (
https://4e4.win/tldr) wasn’t working for me, so I updated it to point to the GitHub repo:https://github.com/raylee/tldr-sh-client. This should make things clearer and easier for anyone trying to find the right repo.Updated the benchmark section in the README:
The current benchmarks were last updated in 2021, and since it’s now 2025, I thought it was time for a refresh! I ran new benchmarks (50 runs on 31.01.2025) and replaced the old data with the latest results.
Added a note about
outfieldr:I couldn’t get
outfieldrto compile on my Arch Linux machine unless I downgradedclang,llvm,zig, etc., which feels like a lot of hassle. I added a note in the README about its lack of maintenance so users know what to expect.