←back to thread

Hyperfine: A command-line benchmarking tool

(github.com)
238 points hundredwatt | 1 comments | 18 Nov 24 21:47 UTC | HN request time: 0s | source
Show context
forrestthewoods ◴[19 Nov 24 06:37 UTC] No.42180588[source]▶
Hyperfine is hyper frustrating because it only works with really really fine microsecond level benchmarks. Once you get into the millisecond range it’s worthless.
replies(2): >>42180660 #>>42182084 #
anotherhue ◴[19 Nov 24 06:50 UTC] No.42180660[source]▶
It spawns a new process each time right? I would think that would but a cap on how accurate it can get.

For my purposes I use it all the time though, quick and easy sanity-check.

replies(2): >>42180722 #>>42180749 #
forrestthewoods ◴[19 Nov 24 07:03 UTC] No.42180722[source]▶
The issue is it runs a kajillion tests to try and be “statistical”. But there’s no good way to say “just run it for 5 seconds and give me the best answer you can”. It’s very much designed for nanosecond to low microsecond benchmarks. Trying to fight this is trying to smash a square peg through a round hole.
replies(3): >>42180876 #>>42180891 #>>42182129 #
1. gforce_de ◴[19 Nov 24 07:39 UTC] No.42180876[source]▶
At least it gives some numbers and point in a direction:

  $ hyperfine --warmup 3 './hello-world-bin-sh.sh' './hello-world-env-python3.py'
  Benchmark 1: ./hello-world-bin-sh.sh
    Time (mean ± σ):       1.3 ms ±   0.4 ms    [User: 1.0 ms, System: 0.5 ms]
  ...
  Benchmark 2: ./hello-world-env-python3.py
    Time (mean ± σ):      43.1 ms ±   1.4 ms    [User: 33.6 ms, System: 8.4 ms]
  ...