2 Benchmark API
(require (planet schematics/benchmark:2)) |
2.1 Overview
2.2 Recording running times
The benchmark-time form records the running (CPU) time of some expressions.
Example:
(benchmark-time (expt 20 20))
The same functionality is also available as a function
| |||||||
thunk : (-> any) |
2.3 Analysing running times
There are a number of macros and functions that are useful for analysing run times.
The strings name1 and name2 are used in reporting the results.
Here’s an example of faster? comparing computing a big factorial to a constant:
(faster? |
["factorial" (factorial 2048)] |
["constant" 5]) |
The output should be similar to this:
factorial was NOT faster than constant |
------------------------------------------------------------ |
Statistics for factorial |
Mean: 3.6999999999999997 |
Std dev.: 0.48304589153964794 |
|
Statistics for constant |
Mean: 0.0 |
Std dev.: 0.0 |
|
factorial took +inf.0 the time of constant |
------------------------------------------------------------ |
#f |
If you want more control the faster*? and as-fast*? functions may be useful.
(faster*? name1 times1 name2 times2 [verbose?]) → (or/c #t #f) |
name1 : string? |
times1 : (vectorof number?) |
name2 : string? |
times2 : (vectorof number?) |
verbose? : (or/c #t #f) = #t |
The verbose? flag determines if output is displayed to current-output-port detailing the mean and standard deviation of the runtimes, and the ratio of the mean runtimes.
The name arguments name1 and name2 are used in the reported generated by a verbose call.
(as-fast*? name1 times1 name2 times2 [verbose?]) → (or/c #t #f) |
name1 : string? |
times1 : (vectorof number?) |
name2 : string? |
times2 : (vectorof number?) |
verbose? : (or/c #t #f) = #t |
Here’s the above example using faster*?:
(faster*? |
"factorial" (benchmark-time (factorial 2048)) |
"constant" (benchmark-time 5)) |
The output looks the same:
factorial was NOT faster than constant |
------------------------------------------------------------ |
Statistics for factorial |
Mean: 3.9 |
Std dev.: 0.31622776601683794 |
|
Statistics for constant |
Mean: 0.10000000000000002 |
Std dev.: 0.316227766016838 |
|
factorial took 38.99999999999999 the time of constant |
------------------------------------------------------------ |
#f |
2.4 SchemeUnit Integration
If you’re optimising code it is useful to record your assumptions about optimisations with your other tests. This way if something changes to invalidate an optimisation you will be made aware of it. For this reason the Benchmark library includes features that integrate with SchemeUnit.
(check-faster name1 thunk1 name2 thunk2) → #t |
name1 : string? |
thunk1 : (-> any) |
name2 : string? |
thunk2 : (-> any) |
(check-as-fast name1 thunk1 name2 thunk2) → #t |
name1 : string? |
thunk1 : (-> any) |
name2 : string? |
thunk2 : (-> any) |