SchemeUnit: Unit Testing in Scheme

_SchemeUnit: Unit Testing in Scheme_

By  Noel Welsh (noelwelsh at yahoo dot com)
and Ryan Culpepper (ryan_sml at yahoo dot com)

This manual documents SchemeUnit version 3.3
Time-stamp: <2006-11-28 16:40:13 noel>

Keywords: _schemeunit_, _test_, _testing_, _unit testing_,
          _unit test_

Table of Contents

Quick Start
Converting Code from Previous Versions
SchemeUnit API
  Core SchemeUnit Types
  Constructing Test Cases and Test Suites
  Predefined Checks
  Providing Additional Information With Checks
  User Defined Checks
  User Defined Shortcuts
  User Interfaces
  Running Tests and Inspecting Results
Tips For Using SchemeUnit
Extended Example
Further Reading


Unit testing is the testing in isolation of individual
elements of a program.  SchemeUnit is a framework for
defining, organizing, and executing unit tests written in
PLT Scheme (  SchemeUnit draws
inspiration from two strands of work: existing practice in
interactive environments and the development of unit testing
frameworks following the growth of Extreme Programming

In an interactive environment it is natural to write in a
``code a little, test a little'' cycle: evaluating
definitions and then immediately testing them in the
read-eval-print loop (REPL).  We take the simplicity and
immediacy of this cycle as our model.  By codifying these
practices we preserve the test cases beyond the running time
of the interpreter allowing the tests to be run again when
code changes.

Unit testing is one of the core practices of the Extreme
Programming software development methodology.  Unit testing
is not new to Extreme Programming but Extreme Programming's
emphasis on unit testing has spurred the development of
software frameworks for unit tests.  SchemeUnit draws
inspiration from, and significantly extends the
expressiveness of, these frameworks.

Quick Start

Suppose you want to test the code contained in "file.scm".
Create a file called "file-test.scm".  This file will
contain the test cases.

At the top of file-test.scm import file.scm and the
SchemeUnit library:

  (require "file.scm"
  	   (planet "" ("schematics" "schemeunit.plt" 2)))

Now create a test suite to hold all the tests:

  (define file-tests
     "Tests for file.scm"

Now we define test cases within the test suite.  Let's
assume that file.scm implements versions of + and - called
my-+ and my--.  We are going to test my-+ and my-- for
integer arthimetic.

  (define file-tests
     "Tests for file.scm"
     (test-equal? "Simple addition" (my-+ 1 1) 2)
     (test-equal? "Simple subtraction" (my-- 1 1) 0))))

Above we implemented two simple test cases.  They have the

  (test-equal? name actual expected)

For example

  (test-equal? "Simple addition" (my-+ 1 1) 2)

creates a test case called "Simple addition" that tests that 

  (equal? (my-+ 1 1) 2)

is true.  There are test case constructors that test for
properties other than equality.  For example, we could

  (test-true "Simple addition" (= (my-+ 1 1) 2))

However this will give a less informative error message if
the test fails.

Now lets run our tests, using SchemeUnit's simple textual
user interface (there are fancier interfaces available but
this will do for our example).  In file-test.scm we import
the textual user interface, and then run our tests using it.

  (require (planet "" ("schematics" "schemeunit.plt" 2)))

  (test/text-ui file-tests)

Now we can execute this file in DrScheme and the textual
user interface will report the success or failure of tests.

Let's say you want to test that a number of properties hold.
How do you do this in a single test case?  So far we've only
seen test cases that hold a single expression.  It turns out
that the test case constructors we've seen so far are
shortcuts for common patterns that involve a test case and a
single check.  In general a test case may consist of any
number of expressions, some of which may be checks.  For
example, here's a test case that checks a list has both
length 4 and all its members are even:

   "List has length 4 and all elements even"
   (let ((lst (list 2 4 6 8)))
     (check = (length lst) 4)
       (lambda (elt)
         (check-pred even? elt))

These are the basics of SchemeUnit.  Refer to the
documentation below for more advanced topics, such as
defining your own checks.  Have fun!

Converting Code from Previous Versions

There is a simple script provided to convert code that uses
previous versions of SchemeUnit.  The script converts:

  - require statements
  - make-test-case => test-case
  - make-test-suite => test-suite
  - assertion => check
  - assert => check

To use the script:

  (require (planet "" ("schematics" "schemeunit.plt" 2)))
  (convert base-dir)

Base-dir is a path or string specifying the root directory
under which the script searches for and converts files that
appear to contain SchemeUnit tests.  Note that backups (the
original file with extension .backup) are created in case of

SchemeUnit API

PLaneT is the main distribution mechanism for SchemeUnit.
To use SchemeUnit insert the require statement below, unless
the documentation for a specific section states otherwise.

  (require (planet "" ("schematics" "schemeunit.plt" 2)))

Core SchemeUnit Types

The _test_ is the core type in SchemeUnit.  A test is either
a _test case_, which is a single action to test, or a _test
suite_, which is a collection of tests.  Test suites may
have optional setup and teardown actions.

> test       :=  test-case | test-suite
> test-case  := name action
> test-suite := name [before] [after] (list-of test)

An _check_ is a predicate that checks a condition.  The
check fails if it's condition is false.  Otherwise it
succeeds.  In SchemeUnit checks fail by raising an
exception of type _exn:test:check_.  Otherwise
they return #t.

A test-case _action_ is a function of no arguments.  A
test-case succeeds if the action evaluates without raising
an exception; otherwise it fails.  Test-case failures are
divided into two cases: those thrown as the result of an
check failure, which we call _failures_ and those thrown
due to other reasons, which we call _errors_.  We define a
type _test-result_ to encapsulate these concepts:

> test-result   := test-failure | test-error | test-success
> test-failure  := test-case failure-exn
> test-error    := test-case error-exn
> test-success  := test-case result

Constructing Test Cases and Test Suites

Test cases are constructed using the _test-case_ macro.

> (test-case name expr ...)

The name is a string that is reported as the name of the
test case.

The exprs are one or more expressions that are run as the
test case action.


   (check = (+ 1 1) 2)
   (check string=? (symbol->string 'foo) "foo"))

Often a test case will include actions that must be executed
before or after the tested expressions.  For example, a test
might require a database connection is opened before it can
run, or require that a temporary file is deleted regardless
of test success or failure.  The _before_, _after_, and
_around_ macros can be used to specify these actions.

> (before before-expr expr1 expr2 ...)

Whenever control enters the scope execute the before-expr
before executing expr-1, and expr-2 ...

> (after expr-1 expr-2 ... after-expr)

Whenever control exits the scope execute the after-expr
after executing expr-1, and expr-2 ...  After-expr are
executed even if expr raise an exception.

> (around before-expr expr-1 expr-2 after-expr)

Whenever control entres the scope execute the before-expr
before expr, and the after-expr after expr.  After-expr is
executed even if before-expr or expr raise an exception.


The test below checks that the file "test.dat" contains the
string "foo".  The before action writes to this file.  The
after action deletes it.

   "The name"
     (with-output-to-file "test.dat"
        (lambda ()
          (write "foo")))
     (with-input-from-file "test.dat"
       (lambda ()
         (check-equal? "foo" (read))))
     (delete-file "test.dat")))

Test suites are constructed using the _test-suite_

> (test-suite name [#:before thunk] [#:after thunk] test ... )

The name is a string that is reported as the name of the
test suite.

The tests are one or more tests (test cases or test suites).

The optional before thunk is an function of no arguments
that is executed before the tests are run.  The before thunk
is signalled by placing the keyword #:before before the

The optional after thunk is a function of no arguments that
is executed after the tests are run.  The after thunk is
signalled by placing the keyword #:after before the thunk.

Before and after thunks, if specified, must be given before
any tests.


   "Suite name"
   #:before (lambda () (display "before"))
   #:after (lambda () (display "after"))
     (check = 1 1)))

Predefined Checks

SchemeUnit provides a rich library of predefined checks.
Every check takes an optional _message_, which is a
string that is displayed if the check fails.  Each
check comes in two variants: a macro with the given
name, and a function with a * appended to the name.  The
macro version grabs source location information, which aids
debugging, and so should be used wherever possible.

Binary checks expect to receive the actual computed
value first, and the expected value second (think
alphabetical order).

The predefined checks are:

> (check binary-predicate actual expected [message])
> (check-equal? actual expected [message])
> (check-eqv? actual expected [message])
> (check-eq? actual expected [message])
> (check-= actual expected epsilon [message])
> (check-true actual [message])
> (check-false actual [message])
> (check-not-false actual [message])
> (check-pred unary-predicate actual [message])
> (check-exn exn-predicate thunk [message])
> (check-not-exn thunk [message])
> (fail [message])


check-= tests that (< (abs (- actual expected)) epsilon)

check-exn takes a thunk as checks are, conceptually,
functions and hence it is necessary to delay the evaluation
of the expressions within the think.


For the common case where a test case consists of a single
check there are shortcuts that create simultaneously a
test case and check.  The predefined shortcuts are:

> (test-check name operator expr1 expr2)
> (test-pred name predicate expr)
> (test-equal? name actual expected)
> (test-eq? name actual expected)
> (test-eqv? name actual expected)
> (test-= name actual expected epsilon)
> (test-true name expr)
> (test-false name expr)
> (test-not-false name expr)
> (test-exn name pred thunk)
> (test-not-exn name thunk)


  (test-equal? "Example" 1 1)

is equivalent to

   (check-equal? 1 1))

  (test-check "Example 2" < 1 2)

is equivalent to

   "Example 2"
   (check < 1 2))

Providing Additional Information With Checks

When an check fails it stores information including the
name of the check, the location and message (if
available), the expression the check is called with, and
the parameters to the check.  Additional information can
be stored by using the _with-check-info*_ function, and
the _with-check-info_ macro.

> (make-check-info name value)

An check-info structure stores information associated
with the context of execution of an check.  Name is a
symbol.  Value is any value.

The are several predefined functions that create check
information structures with predefined names.  This avoids
misspelling errors:

> (make-check-name name)
> (make-check-params params)
> (make-check-location loc)
> (make-check-expression msg)
> (make-check-message msg)

> (with-check-info* info thunk)

The info is a list of check-info structures.

The thunk is a function of no arguments

The with-check-info* function stores the given info in
the check information stack for the duration of the
execution of thunk.

     (list (make-check-info 'time (current-seconds)))
     (lambda () (check = 1 2))))

When this check fails the message

  time: <current-seconds-at-time-of-running-check>

will be printed along with the usual information on an
check failure.

> (with-check-info ((name val) ...) body ...)

The with-check-info macro stores the given information
in the check information stack for the duration of the
execution of the body expressions.  Name is a quoted symbol
and val is any value.


    (lambda (elt)
       (('current-element elt))
       (check-pred odd? elt)))
    (list 1 3 5 7 8)))

When this test fails the message

  current-element: 8

will be displayed along with the usual information on an
check failure.

User Defined Checks

SchemeUnit provides a way for user's to extend its builtin
collection of checks using the
_define-simple-check_, _define-binary-check_, and
_define-check_ macros.  To understand these macros it is
useful to understand a few details about an checks
evaluation model.

Firstly, an check should be considered a function, even
though most uses are actually macros.  In particular,
checks always evaluate their arguments exactly once
before executing any expressions in the body of the
checks.  Hence if you wish to write checks that
evalute user defined code that code must be wrapped in a
thunk (a function of no arguments) by the user.  The
predefined check-exn is an example of this type of

It is also useful to understand how the check
information stack operates.  The stack is stored in a
parameter and the with-check-info forms evaluate to
calls to parameterize.  Hence check information has
lexical scope.  For this reason simple checks (see
below) cannot usefully contain calls to with-check-info
to report additional information.  All checks created
using define-simple-check or define-check grab some
information by default: the name of the checks and the
values of the parameters.  Additionally the macro forms of
checks grab location information and the expressions
passed as parameters.

> (define-simple-check (name param ...) expr ...)

The define-simple-check macro constructs an check
called name that takes the params as arguments and an
optional message and evaluates the exprs.  The check
fails if the result of the exprs is #f.  Otherwise the
check succeeds.  Note that simple checks cannot
report extra information using with-check-info.

The define-simple-check macro actually constructs two
checks: a macro with the given name that collects source
location information, and a function with name name* that
does not collection source location information but can be
used in a higher-order fashion.  In my experience
higher-order checks are useful for testing SchemeUnit
but not much else.


To define checks check-odd? and check-odd?* we could
do this:

  (define-simple-check (check-odd? number)
    (odd? number))

We could use these checks in the usual way:

  (check-odd? 3)  ;; Success
  (check-odd? 2) ;; Failure

> (define-binary-check (name pred actual expected))
> (define-binary-check (name actual expected) expr ...)

The define-binary-check macro constructs a macro that
tests a binary predicate.  This allows actual and expected
values to be reported.  The first form of the macro accepts
a binary predicate and tests if the predicate holds for the
given values.  The second form tests if expr ... return


Here's the first form, where we use a predefined predicate
to construct a binary check:

  (define-binary-check (check-char=? char=? actual expected))

In use:

  (check-char=? (read-char a-port) #\a)

If the expression is more complicated the second form should
be used.  For example, below we define a binary check that tests a number if within 0.01 of the expected value:

  (define-binary-check (check-in-tolerance actual expected)
    (< (abs (- actual expected)) 0.01))

> (define-check (name param ...) expr ...)

The define-check acts in exactly the same way as the
define-simple-check macro, except the check only
fails if the macro _fail-check_ is called in the body of
the check.  This allows more flexible checks, and in
particular more flexible reporting options.

> (fail-check)

The fail-check macro raises an exn:test:check with
the contents of the check information stack.


Below is an check that tests if two lists are the same,
and if they aren't it reports the first elements that differs.

  (define-check (check-list= lst1 lst2)
     (lambda (elt1 elt2)
        (('element1 elt1)
         ('element2 elt2))
        (if (not (equal? elt1 elt2))

We could even get fancy, and allow the equality predicate to
specified, and reuse check-pred to test for equality:

  (define-check (check-list= pred lst1 lst2)
     (lambda (elt1 elt2)
        (('element1 elt1)
         ('element2 elt2))
        (check-pred pred elt1 elt2)))

You are encouraged to submit libraries of checks to

User Defined Shortcuts

To define your own shortcuts use the _define-shortcut_ macro

> (define-shortcut (name param ...) expr)

Name is the name of the shortcut.

Param ... are the parameters the shortcut takes, not
including the test case name.

Expr is a single expression that performs the test

The define-shortcut macro will create a macro of the form

  (name test-case-name param ...)


The predefined test-equal? shortcut is defined as follows:

  (define-shortcut (test-equal? expr1 expr2)
    (check-equal? expr1 expr2))

User Interfaces

As of version 3.1, SchemeUnit provides both text and graphical user

To use the text user interface:

  (require (planet "" ("schematics" "schemeunit.plt" 2)))

The text UI is run via the function

> (test/text-ui test [verbosity])
   test/text-ui : test (U 'quiet 'normal 'verbose) -> integer

The given test is run and the result of running it output to
the current-output-port.  The output is compatable with the
(X)Emacs next-error command (as used, for example, by
(X)Emac's compile function)

The optional verbosity is one of 'quiet, 'normal, or
'verbose.  Quiet output displays only the number of
successes, failures, and errors.  Normal reporting
suppresses some extraneous check information (such as
the expression).  Verbose reports all information.

test/text-ui returns the number of unsuccessful tests.

To use the graphical user interface:

  (require (planet "" ("schematics" "schemeunit.plt" 2 1)))

The graphical UI is run via the function

> (test/graphical-ui test)
   test/graphical-ui : test -> void

The graphical UI displays a split-pane view, with a tree view of the
test suite hierarchy on the left and the details of the selected test
on the right.


To use the following definitions:

  (require (planet "" ("schematics" "schemeunit.plt" 2)))

Sometimes it is useful to test definitions that are not
exported by a module.  SchemeUnit supports this via the
_require/expose_ macro.

> (require/expose module (id ...))

The require/expose macro is like a normal require form,
except it can require definitions that are defined but not
provided (i.e. exported) by a module.  The ids are the
identifiers that are to be exposed in the current module or
namespace.  Module is a module specifier (as given to
require) where the ids are found.

The _test-suite*_ macro provided a shortcut for the common
case of a test suite that simply defines a number of test

> (test-suite* name (test-case-name test-case-expr ...) ...)

Makes a test suite with the given name that contains test
cases with the given names and actions.

> (check-regexp-match regex-or-string string-or-port [message])

An check that tests if a regular expression (string or
regexp) matches a string or port.

Running Tests and Inspecting Results

SchemeUnit provides an API for running tests, from which
custom UIs can be created.


> (exn:test:check stack)
  struct exn:test:check : (list-of check-info)

An exn:test:check is raised when an check fails, and
contains the contents of the check-info stack at the
time of failure.

> (test-result test-case-name)
  struct test-result : string

A test-result is the result of running the test case with
the given name.

> (test-failure result)
  struct (test-failure test-result) : exn:test:check

Subtype of test-result representing a test failure.

> (test-error result)
  struct (test-error test-result) : exn

Subtype of test-result representing a test error

> (test-success result)
  struct (test-success test-result) : any

Subtype of test-result representing a test success


> (run-test-case name action)
   run-test-case : string thunk -> test-result

> (run-test test)
   run-test : test -> (list-of test-result)

Runs the given test (test case or test suite) returning a
tree (list of lists) of results


  > (run-test
      (test-case "Dummy" (check-equal? 1 2))))

> (fold-test-results result-fn seed test #:run run #:fdown fdown #:fup fup)
   fold-test-results : ('b 'c ... 'a -> 'a)
                        #:run   (string (() -> any) -> 'b 'c ...)
                        #:fdown (string 'a -> 'a)
                        #:fup   (string 'a -> 'a)

Fold collector pre-order left-to-right depth-first over the
results of run.  By default run is run-test-case and fdown
and fup just return the seed, so result-fn is folded over
the test results.

This function is useful for writing custom folds (and hence
UIs) over test results without you having to take care of
all the expected setup and teardown.  For example,
fold-test-results will run test suite before and after
actions for you.  However it is still flexible enough, via
its keyword arguments, to do almost anything that foldts
can.  Hence it should be used in preference to foldts.

Result-fn is a function from the results of run (default
default a test-result) and the seed to a new seed

Seed is any value

Test is a test-case or test-suite

Run is a function from a test case name (string) and action
(thunk) to any values.

FDown is a function from a test suite name (string) and the
seed, to a new seed

FUp is a function from a test suite name (string) and the
seed, to a new seed.

Note that fold-test-results has changed in recent versions
of SchemeUnit.  If this causes you problems, and you are not
me, I would love to hear it.  As far as I know there is only
person who actually uses these functions.


The following code counts the number of successes

  (define (count-successes test)
     (lambda (result seed)
       (if (test-success? result)
           (add1 seed)

When run over the SchemeUnit test suite:

  > (count-successes all-schemeunit-tests)

The following code return the symbol burp instead of running
test cases.  Note how the result-fn receives the value of

  (define (burp test)
     (lambda (result seed) (cons result seed))
     #:run (lambda (name action) 'burp)))

> (foldts fdown fup fhere seed test)
   foldts : (test-suite string thunk thunk 'a -> 'a) 
            (test-suite string thunk thunk 'a 'a -> 'a)
            (test-case string thunk 'a -> 'a)
         -> 'a

Foldts is a nifty tree fold (created by Oleg Kiselyov) that
folds over a test in a useful way (fold-test-results isn't
that useful as you can't specify actions around test cases)

Fdown is a function of test suite, test suite name, before
action, after action, and the seed.  It is run when a test
suite is encountered on the way down the tree (pre-order).

Fup is a function of test suite, test suite name, before
action, after action, the seed at the current level, and the
seed returned by the children.  It is run on the way up the
tree (post-order).

Fhere is a function of the test case, test case name, the
test case action, and the seed. (Note that this might change
in the near future to just the test case.  This change would
be to allow fhere to discriminate subtypes of test-case,
which in turn would allow test cases that are, for example,


Here's the implementation of fold-test-results in terms of

  (define (fold-test-results suite-fn case-fn seed test)
     (lambda (suite name before after seed)
       (suite-fn name seed))
     (lambda (suite name before after seed kid-seed)
     (lambda (case name action seed)
        (run-test-case name action)

If you're used to folds you'll probably be a bit surprised
that the functions you pass to foldts receive both the
structure they operate on, and the contents of that
structure.  This is indeed unusual.  It is done to allow
subtypes of test-case and test-suite to be run in customised
ways.  For example, you might define subtypes of test case
that are ignored (not run), or have their execution time
recorded, and so on.  To do so the functions that run the
test cases need to know what type the test case has, and
hence is is necessary to provide this information.

If you've made it this far you truly are a master SchemeUnit
hacker.  As a bonus prize we'll just mention that the code
in and might be of interest for
constructing user interfaces.  The API is still in flux, so
isn't documented here.  However, do look at the
implementation of test/text-ui for examples of use.

Tips For Using SchemeUnit

I create one module of tests for each module of code.  If
the module is called "foo" the test module is called
"foo-test" and exports a single test suite, called
"foo-tests".  For each project (or collection) called, say,
"bar" I have a single module "all-bar-tests" that exports a
single test suite (called "all-bar-tests") which collects
together all the test suites for that project.  I often
create another file, called "" which runs
"all-bar-tests' using the text user interface.  To run all
the project's tests I can then simply execute ""
in Dr/MzScheme or (X)Emacs.  

To run tests from (X)Emacs the following command is can be
given as the "Compile command" to compile:

   mzscheme -qe '(begin (load "") (exit))'

This command causes mzscheme to execute the code in
"" and then exit.  If some preventing loading of
"", for example a syntax error, mzscheme will
pause waiting for input and you will have to kill it next
time you run compile ((X)Emacs asks if you want to do this).

Defining your own checks is one of the most powerful
features of SchemeUnit.  Whenever you find yourself writing
out the same series of checks define your own check
and use that instead.  Your checks will act just like
the pre-defined checks; they take optional message
strings and will display locations and parameters.  They are
really easy to create as well.  For instance, to define an
check called "check-is-2" that checks a value is equal
to 2, you'd just evaluate

  (define-simple-check (check-is-2 x)
    (= x 2))

Checks compose as well, so you could write:

  (define-check (check-is-2 x)
    (check = x 2))

If you find you're creating a library of checks please
submit them to us so we can share them with all SchemeUnit

Extended Example

This example test suite is included in the SchemeUnit
distribution as "":

  ;; The tests below are intended as examples of how to use
  ;; the test package.  They test PLT Scheme's arithmetic
  ;; operations and some simple file reading

  (require (file ""))
  (require (file ""))
    "Example tests"
     "Arithmetic tests"
     (test-check "Multiply by zero" = (* 1234 0) 0)
     (test-check "Add zero" = (+ 1234 0) 1234)
     (test-check "Multiply by one" = (* 123.0 1) 123)
     (test-check "Add one" = (+ 123.0 1) 124)
     (test-check "Expt 0 0" = 1 (expt 0 0))
     (test-check "Expt 0 1" = 0 (expt 0 1))
     (test-check "Expt 0.0 0.0" = 1.0 (expt 0.0 0.0))
     (test-check "Expt 0.0 1.0" = 0.0 (expt 0.0 1.0))
     "File tests"
     ;; An example with an after action
      "String port read"
      (let ((port (open-input-string "this is a test string")))
         (check-equal? "this is a test string" (read-line port))
         ((close-input-port port)))))
     ;; An example with an around action
      "File port read"
       ((with-output-to-file "test.dat"
          (lambda ()
            (write "foo"))))
       (with-input-from-file "test.dat"
         (lambda ()
           (check-equal? "foo" (read))))
       ((delete-file "test.dat"))))

Further Reading

The paper "Two Little Languages: SchemeUnit and SchemeQL"
describes the design rationale and implementation of the 1.x
series of SchemeUnit.

There are comments in the source code, and tests of the same
are good examples of usage.  The file "", provided
as part of SchemeUnit, is a good example of a reasonably
complex test suite.

Good references for the Extreme Programming methodology




  -; JUnit}, the Java unit test

Most of the code at Schematics
( has extensive test
suites written using SchemeUnit.


The following people have contributed to SchemeUnit:

  - Jacob Matthews and Guillaume Marceau for bug reports and

  - Eric Hanchow suggested test/text-ui return a useful

  - Ray Racine and Richard Cobbe provided require/expose

  - John Clements suggested several new checks

  - Jose A. Ortega Ruiz alerted me a problem in the
    packaging system and helped fix it.

  - Sebastian H. Seidel provided help packaging SchemeUnit
    into a .plt

  - Don Blaheta provided the method for grabbing line number
    and file name in checks

  - Patrick Logan ported to version 1.3

  - The PLT team made PLT Scheme

  - The Extreme Programming community started the whole
    testing framework thing