About the Benchmarks

Latest run: August 29, 2006

The pages linked below show some benchmark results on a collection of fairly standard (mostly Gabriel) Scheme benchmarks.

Tables show relative performance, with the actual time for the fastest run shown on the left. So, by default, 1 is the fastest, but select any implementation to normalize the table with respect to that implementation's speed. A -- appears when a benchmark didn't run in an implementation (and you should assume a benchmark problem, rather than an implementation problem).

Small gray numbers are (relative) compile times.

Run times are averaged over three runs. All reported times are CPU time (system plus user). Where available, the times are based on the output of the implementation's time syntactic form, otherwise /usr/bin/time is used.

Compiler configuration:

These configurations are all "safe mode", but they allow the compiler to assume that built-in Scheme functions are not redefined and (except in the case of Larceny) that no top-level defintion is ever changed. Such assumptions correspond to putting the benchmark in an R6RS library (we expect).

In general, we attempt to use the various implementations in a compentent way, but not in a sophisticated way. For example, we do not tweak inlining parameters or specify fixnum arithmetic (where appropriate), which could produce significant improvements from some compilers.

For a larger set of benchmarks and a more sophisticated use of the compilers, see Marc Feeley's page: http://www.iro.umontreal.ca/~gambit/bench.html.

For further details on the benchmarks here, see the benchmark source and infrastructure, which is available form the PLT SVN repository:

http://svn.plt-scheme.org/plt/trunk/collects/tests/mzscheme/benchmarks/common/

Results