While avoiding family this Christmas-day I started trying to figure out way for me to do TDD in Common Lisp (as one does…). That led to SLIME¹ and QuickLisp² and lisp-unit³ and asdf⁴ (&c. &c).

…it’s yaks the whole way down.

The hardest part for me was to find a simple way to load my code and run the tests. ASDF ships with asdf:test-system defined but it does nothing by default⁵. So I did some digging around and saw some other’s solutions and synthesized my version.

Here is an example ASDF system definition for a Game of Life⁶ implementation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

(asdf:defsystem #:gol
  :description "Conways Game of Life in Lisp"
  :author "Mark Simpson <verdammelt@gmail.com>"
  :version "0.0.0"
  :depends-on ("lisp-unit")
  :components ((:module "src"
			:serial t
			:components ((:file "package")
				     (:file "gol"))
                (:module "test"
			:serial t
            :depends-on ("src")
			:components ((:file "package")
				     (:file "gol-test"))))
  :in-order-to ((test-op (load-op :gol)))
  :perform (test-op (o c)
		    (progv
			(list (intern "*PRINT-ERRORS*" :lisp-unit)
			      (intern "*PRINT-FAILURES*" :lisp-unit))
			'(t t)
		      (funcall (find-symbol "RUN-TESTS" :lisp-unit)
			       :all :gol-test))))

My God… it’s full of yaks

I’m just going to let that sink in for a bit while I get some coffee.

Ok, now that I’m back I will explain each part. The first interesting bit is on line 6:

1

  :depends-on ("lisp-unit")

This line tells ASDF that it should find and load a system called lisp-unit because our system depends upon it. This is where QuickLisp makes things simple since that is its job and it does a fine one at that.

Next is the definition of our components:

1
2
3
4
5
6
7
8
9

  :components ((:module "src"
			:serial t
			:components ((:file "package")
				     (:file "gol"))
                (:module "test"
			:serial t
            :depends-on ("src")
			:components ((:file "package")
				     (:file "gol-test"))))

I’ve decided that my project should have two main directories, one to contain all the production source code and one for all the test source code. Each one will be in their own packages, with the :gol-test package :use-ing the :gol package. :serial t by the way tells ASDF that each item in the component list depends upon the one before it.

Now the hairier parts which tell ASDF to run our tests:

1
2
3
4
5
6
7
8

  :in-order-to ((test-op (load-op :gol)))
  :perform (test-op (o c)
		    (progv
			(list (intern "*PRINT-ERRORS*" :lisp-unit)
			      (intern "*PRINT-FAILURES*" :lisp-unit))
			'(t t)
		      (funcall (find-symbol "RUN-TESTS" :lisp-unit)
			       :all :gol-test))))

The first line is simply stating that in order to do test-op ASDF needs to perform load-op on the :gol system. This way every time we run the tests we’ll reload the system if needed.

Now… this next bit probably shows how rusty my Lisp is. Maybe there is an easier way. Basically we want to run (lisp-unit:run-tests :all :gol-test) with lisp-unit:*print-errors* and lisp-unit:*print-failures* both bound to t (they default to nil and that doesn’t give enough info IMNSHO). However when Lisp is reading and evaluating this code lisp-unit has probably not yet been loaded - so we have to be crafty and find the symbol by name in the package then funcall it. Binding the variables was trickier but I stumbled upon progv⁷ and that seems to be just what I needed.

progv creates new dynamic bindings for the symbols in the list which is its first argument. These symbols are determined at runtime. That means we can use intern to find or create these symbols in the right package⁸. progv binds these symbols to the values in the list which is its second parameter and then finally executes the sexp which is its third argument in an environment which contains these new bindings.

With this work done I can now type in (asdf:test-system 'gol) into my REPL and it loads and runs my tests and gives me useful output of those tests. I might go so far as to bind that to a very short function name, or even a key in Emacs.

There are still more yaks to shave.

What is left to be determined is if this is the best way to do things.

Some documentation/blogs I read lead me to believe that putting the tests in the same system with the production code is verboten (or at least a no-no). I tried having the tests be in their own system definition (depending upon the production code) but then ASDF & QuickLisp both strongly pushed me to have that definition it its own .asd file⁹ which seemed awkward to me. I personally have no problem shipping tests with the production code, and I believe that if needed all the symbols of the test package could be unintern-ed before an image was saved if desired.

I’ll play around with this setup and see how things go.