Modularizing Code in Common Lisp
Not that many years ago, using other people's Common Lisp libraries was a pretty tedious affair. You had to download their code files, see what other code files were required, hope there weren't any incompatibilities, etc. Fortunately, these days we have Quicklisp to automate most of the process. Quicklisp in turn uses ASDF, a non-standard but popular tool for defining systems, i.e., collections of files and how they depend on each other and other systems.
Quicklisp
Quicklisp is a library manager, similar in some ways to Ruby's gem system. Once installed, Quicklisp makes it easy to
- Download, compile and load any library it knows about -- and there are about 1000
- Automatically and recursively download, compile and install any libraries the library you chose needs
- Compile and load your own libraries
- Update your local copies of libraries when the remote versions are updated
To do this, Quicklisp has a server full of Common Lisp code, but it also needs to know what files and libraries each library needs. To do that, it uses ASDF.
ASDF
Common Lisp has no standard for libraries. There have been many implementations of ideas for defining libraries. Mark Kantrowitz's DEFSYSTEM was one of the more popular early ones. It used the term "system" for "library." It was succeeded by ASDF (Another System Definition Facility). The current version is ASDF 3. Many Lisps have ASDF installed, though it might be version 2.
ASDF definitions are (very) roughly like Unix makefiles. They describe what libraries and files needs to be compiled and loaded to make a system, and what the file dependencies are. If A depends on B and B changes, A should be recompiled as well.
ASDF definitions can get quite complicated. For our purposes, we're going to use the simplest form. Here's the ASDF definition for the CS325 code library:
(asdf:defsystem #:cs325 :serial t :depends-on (#:lisp-unit) :components ((:file "package") (:file "tables") (:file "extend-match") (:file "lisp-critic") (:file "ddr") (:file "exercise-tests") (:file "lisp-rules") (:file "ddr-tests") (:file "ddr-exs-tests") ))
This says that the system is named cs325
. That means you can load this
with (ql:quickload "cs325")
.
The cs325 system consists of 12 files. The :serial t
says that there's a
linear (serial) dependency: later files depend on earlier files.
So, if lisp-critic.lisp
changes, it and every file after it in the list will be recompiled and reloaded
the next time (ql:quickload "cs325")
is called.
This isn't really necessary. Most of these files only depend ondependencies.lisp
andpackage.lisp
. A few depend ontables.lisp
.lisp-rules.lisp
depends onlisp-critic.lisp
, etc. As a result, if, say,tables.lisp
is modified, more files will be compiled and reloaded than necessary. It's not hard to change the ASDF to do the right thing, but we won't bother since there's not that many files and they all compile quite quickly.
The cs325 system doesn't depend on any libraries, but the JSON Demo system does.
(asdf:defsystem #:json-demo :serial t :depends-on ("simple-server" "cl-json") :components ((:file "json-demo")))
The :depends-on
clause lists other systems that are needed, before the files are loaded.
In this case, there's one local system simple-server
and a
Quicklisp-managed JSON library
cl-json. CL-JSON in turn
depends on many other libraries, that Quicklisp will automatically download, compile and load.
Local Systems
You add a local project to Quicklisp by putting a directory of code inside
~/quicklisp/local-projects/
containing a file with the extension
asd
that has the ASDF definition of your system.
(ql:register-local-projects)
will recursively
scan all the directories in local-projects
for .asd
files and put them in a list in ~/quicklisp/local-projects/system-index.txt
.
ql:quickload
checks this file first before looking
for modules online.
SBCL Windows note: SBCL requires that text files
has Unix line endings, even on Windows. Quicklisp will break reading
the local projects file if the line endings are Windows-style. This can
happen if you edit the file by hand, or call
(ql:register-local-projects)
in a different Lisp
on your machine.
Readings
Phil Eaton's Starting a minimal Common Lisp project is a good introduction to modern Common Lisp library packaging with ASDF and Quicklisp.