Introduction

When a large system is broken down into modules, inter-module dependencies will arise. In particular, one module may call functions and macros defined in one or more other modules. This raises several problems:

• how to make sure those other modules are loaded when testing the module in question
• how to make sure all modules are loaded in the correct order when building the whole system
• how to make sure all modules are packaged up when moving the system to another machine

Ordering is particularly important with modules that define packages and macros. These must be loaded before any modules that use those packages or macros.

There are several methods in general use in Common Lisp for specifying how various modules of a large system go together:

• a master "build" file with package definition and file loading commands
• a master "build" file with a defsystem form
• require and provide forms in each module

The first two methods are centralized approaches to system definition. The third method is decentralized. There are arguments to be made in favor of both approaches, and, in my opinion, no clear winner on those grounds.

The first method is simple, but not very flexible. It also doesn't help developers working with individual modules.

The second method is a generalization of the first. defsystem is a special form for declaring how a system depends on its modules and how they depend on each other. Unfortunately, there is no standard defsystem for Common Lisp. There are two major prototypes in use. Both are complex and they disagree significantly.

We will use the third method, because it's simple, flexible, handles individual modules as well as whole systems. After a brief disappearance, it has returned as part of the Common Lisp standard.

Unfortunately, there are several important aspects of how require and provide work that were unspecified in the original Common Lisp manual. That's why they were deprecated in Common Lisp 2. We'll have to do a little work to get cross-platform compatibility.

Using require and provide

The basic idea of require and provide is simple.

• (require module-name) loads the file associated with module-name unless that module is already loaded.
• (provide module-name) registers a module as having been loaded.

A module name can be a string, e.g., "tables", or a keyword, e.g., :tables. It is never a file name, such as "tables.lsp".

We simply put the necessary require forms at the front of a file, and, optionally, one provide form at the end of the file. provide is only needed if the filename is different than the module name.

When the file is loaded, only those modules requested that are not already loaded will be loaded into the Lisp system.

For example, suppose the rules module needs the tables module. The following would go at the front of the file rules.lisp, before any code definitions:

(in-package ...) ;; whatever package the code should be in

(eval-when (:compile-toplevel :load-toplevel :execute)
  (require :tables)
  (use-package :tables))

The following could go at the end of the same file:

(provide "rules")
 
 

but it's not necessary because the module name and file name are the same. Some authors recommend putting provide at the front of the file, in order to

• advertise up front what module the file provides, and
• avoid endless loops if there be any circularities (file A requires file B which requires file A)

I recommend putting the provide at the end, because

• if the file fails to load completely, a provide at the front will fool Lisp into thinking that the module has been loaded
• circular dependencies should be fixed, not supported

What require and provide do

(provide module-name) simply adds the module name to *modules*, a list of loaded modules.

(require module-name) checks to see if the module is already in *modules*, using string=. If it is, require does nothing. If not, require gets the file name associated with the module name and loads that file.

It's that last step that is both the strength and weakness of require:

• By separating module names from file names, the require forms in each file can still work when you move the files to other machines.
• Unfortunately, how module names are mapped to file names is not standardized.

Customizing require

In most Lisps, (require module-name) will look for a file of the name module-name.ext where ext is lisp or lsp or whatever is appropriate for your Lisp.

This works fine if all your files are sitting in the same directory as your Lisp image. Unfortunately, of course, eventually you're going to want to stuff those files away in subdirectories. Then the problem becomes telling require how to find them.

You have two options:

• read the documentation on require that comes with your Lisp. Most describe one or more variables that can be set to tell require where to look for files.
• use add-search-path defined in cs325.lisp

(add-search-path directory-path) adds the given directory path to require's search list. It's currently defined to work for both Allegro and Macintosh Common Lisp. The file cs325.lisp is set up to automatically add the directory containing cs325.lisp, so if you keep all your code there, you're done.

If you have another directory that you want searched, add a call to add-search-path in your Lisp's initialization file, e.g., startup.cl in Allegro, or init.lisp in Macintosh Common Lisp. For example,

(add-search-path "d:/lisp/utils/")

would add that "/lisp/utils" directory to require's search list.

Pathnames in Common Lisp

No matter which version of require you use, in order to tell it where to look, you'll need to specify file pathnames. A pathname is so called because it specifies not only the name of the file, but the path of directories and subdirectories that lead to that file.

There are two ways to write pathnames in Common Lisp. One is simple, the other is portable.

Pathname strings

The simplest and most common method is to simply write the file's pathname as a string. For example, on my Unix machine, I can write "~/lisp/require.lisp" to refer to the file require.lisp in the directory /home/riesbeck/lisp/.

String pathnames are simple. Unfortunately, they're not portable across machines. To refer to the same file on my Macintosh, I have to say "Riesbeck's HD:MCL 2.0.1:require.lisp". Not only is the path different, but the punctuation and available abbreviations are different.

Furthermore, an important thing to know about a directory path is whether is starts from "the top," or "where you are right now." The punctuation for specifying this is quite variable.

This is really not a big deal, as long as you stick to one machine, but there is an alternative.

make-pathname

The function make-pathname in Common Lisp can be used to build pathname structures (which usually print out in the form #P"...") from the basic pieces. Those pieces are:

• the device, which is the disk drive name on Windows machines, e.g., C or D
• the directory path, which is a list of directory names, each of which is a string. The list starts with either :absolute or :relative, depending on whether you want the path to start from the top or not.
• the file name, which is a string
• the file type, which is a string

Within these strings, punctuation does not mean anything special, e.g., a period does not indicate the start of the file type.

For example, here's how I would specify the pathname for require on my Macintosh:

(make-pathname
  :directory '(:absolute "Riesbeck's HD" "MCL 2.0.1")
  :name "require"
  :type "lisp")
 
 

When make-pathname is important

make-pathname is particularly important if you are writing functions that generate pathnames.

For example, suppose you want your program to do the following:

• when the program starts, it asks for the user's name
• if a "log file" exists for that user, it loads it. The log file enables the system to restart the user where they last left off.
• Otherwise, the program creates the log file.
• Later, when the user quits, where they were in the program is saved in the log file for future use.

We could create the log file name like this:

(defun make-log-file (user-name)
  (concatenate 'string
      "/home/demo/log-files/"
      user-name
      ".log"))
 
 

For example,

> (make-log-file "riesbeck")
"/home/demo/log-files/riesbeck.log"
 
 

Unfortunately, this code would only work on a Unix machine. We'd have to rewrite it for Macintosh's and DOS/Windows.

Furthermore, this code would have to be edited and recompiled every time we moved where we wanted the log files stored.

Here's the better way to do it:

(defvar *log-file-path*
  '(:absolute "home" "demo" "log-files"))
 
(defun make-log-file (user-name)
  (make-pathname
    :directory *log-file-path*
    :name user-name
    :type "log"))
 
 

Not only does this work on most machines, but we can redirect where log files are stored by simply resetting *log-file-path*.

Comments? Send mail to Chris Riesbeck.