Ticket #156 (accepted task)

Opened 5 years ago

Last modified 5 years ago

test-suite semantics change makes my code infinite loop

Reported by: pnkfelix@… Owned by: schematics
Priority: major Milestone:
Component: schematics/schemeunit.plt Keywords:
Cc: Version: (3 4)
Racket Version: 4.1.4

Description

I was driving myself crazy trying to figure out why my Intel laptop was running my tests while my old Power Mac was taking forever to run them.

Eventually I realized it was because my Power Mac had downloaded a more recent version of Scheme Unit.

Consider the following:

#lang scheme

#;
(require (planet schematics/schemeunit:3) 
         (planet schematics/schemeunit:3/text-ui))
(require (planet "main.ss" ("schematics" "schemeunit.plt" 3 (= 3)))
         (planet "text-ui.ss" ("schematics" "schemeunit.plt" 3 (= 3))))

(define felixs-suite (test-suite ""))

(set! felixs-suite 
      (test-suite "snoc" 
                  felixs-suite
                  (test-case "test" (check-equal? 1 1))))

(run-tests felixs-suite)

I use the pattern above (in a more complicated manner but abstracted under a macro) to incrementally construct a set of test cases whose evaluation will be delayed until I invoke run-tests.

I can see in the Release Notes for version 3.4 that you say that you have changed things to allow "arbitrary expressions" in test suites. But I would claim that you have merely changed the set of allowable expressions, since my example worked better under the old semantics. The documentation should be improved to make this clearer.


In the meantime, can you tell me how you would achieve the effect I desire? (One way to think of the desired effect is like the Student Language's check-expect form, which delays invocation of the tests until the entire module body has been evaluated.)

Change History

Changed 5 years ago by pnkfelix

Just in case its not clear, there are actually two criteria that the above hack was trying to satisfy:

  1. Adding test cases incrementally
  2. All added tests are accumulated in a single test suite, where the one invocation of run-tests summarizes the results for all additions.

That second criteria is why the alternative design of building up a list of test suites and then doing (for-each run-tests suite-list would not suffice. If there were a procedure for turning a list of test suites into one single test suite, then the latter approach could be made to work, but I was not able to find such a procedure in the documentation.

Changed 5 years ago by pnkfelix

Ah. I figured out another work-around: force felixs-suite to be evaluated outside of the test-suite form.

#lang scheme
(require (planet schematics/schemeunit:3) 
         (planet schematics/schemeunit:3/text-ui))

(define felixs-suite (test-suite ""))

(set! felixs-suite 
      (let ((last-suite felixs-suite))
        (test-suite "snoc" 
                    last-suite
                    (test-case "test" (check-equal? 1 1)))))

(run-tests felixs-suite)

I still would like to know how you Scheme Unit experts would accomplish this effect, since I do not think my approach is particularly elegant. But now that my problem is solved, I regard this ticket as a documentation issue rather than one of functionality.

Changed 5 years ago by schematics

  • status changed from new to accepted

Thanks for reporting this. It's only by people telling me how they use SchemeUnit? that I can make it better. I'll add a programmatic API to construct test suites and test cases (i.e. make-test-suite, make-test-case etc.)

Changed 5 years ago by schematics

In the SchemeUnit? collection in the PLT trunk there are now make-test-case and make-test-suite functions. These will arrive in the Planet release as I get time to document and merge the changes across.

Note: See TracTickets for help on using tickets.