Introduction

When a large system is broken down into modules, inter-module dependencies will arise. In particular, one module may call functions and macros defined in one or more other modules. This raises several problems:

Ordering is particularly important with modules that define packages and macros. These must be loaded before any modules that use those packages or macros.

There are several methods in general use in Common Lisp for specifying how various modules of a large system go together:

The first two methods are centralized approaches to system definition. The third method is decentralized. There are arguments to be made in favor of both approaches, and, in my opinion, no clear winner on those grounds.

The first method is simple, but not very flexible. It also doesn't help developers working with individual modules.

The second method is a generalization of the first. defsystem is a special form for declaring how a system depends on its modules and how they depend on each other. Unfortunately, there is no standard defsystem for Common Lisp. ASDF is a popular open-source centralized module loading system.

In one of the few places where I go against common practice, I use the third method for this course, because it's the simplest for end-users. When they want a file, they just load it, and any other files are loaded as needed.

Using require and provide

The basic idea of require and provide is simple.

A module name can be a string, e.g., "tables", or a keyword, e.g., :tables. It is never a file name, such as "tables.lsp".

We simply put the necessary require forms at the front of a file, and, optionally, one provide form at the end of the file. provide is only needed if the filename is different than the module name.

When the file is loaded, only those modules requested that are not already loaded will be loaded into the Lisp system.

For example, suppose the rules module needs the tables module. The following would go at the front of the file rules.lisp, before any code definitions:

(in-package ...) ;; whatever package the code should be in
    
    (eval-when (:compile-toplevel :load-toplevel :execute)
      (require :tables)
      (use-package :tables))

The following could go at the end of the same file:

(provide "rules")
     
     

but it's not necessary because the module name and file name are the same. Some authors recommend putting provide at the front of the file, in order to

I recommend putting the provide at the end, because

What require and provide do

(provide module-name) simply adds the module name to *modules*, a list of loaded modules.

(require module-name) checks to see if the module is already in *modules*, using string=. If it is, require does nothing. If not, require gets the file name associated with the module name and loads that file.

It's that last step that is both the strength and weakness of require:

Require and relative pathnames

An alternative, for sets of files that go together, such as all the EECS 325 course files, is to specify in each REQUIRE where to get the file(s) you need, using a pathname format that is relative to the file being loaded.

require takes an optional second argument. That argument should be either NIL, a pathname, or a list of pathnames. You might therefore write

(require :tables "tables.lisp")

Unfortunately, since the pathname is not a complete one, where Lisp will look for tables.lisp is not defined. It will depend on the setting of some "current working directory" internal variable, that every Lisp has, but for which there's no Common Lisp standard. Furthermore, what if there's a compiled version of tables.lisp? We'd rather load that unless the source code is more recent.

Therefore, we use the following function, defined in CS325-user, to get a code file for a module:

(defun get-code-file (name)
  (let* ((source-path 
          (merge-pathnames name 
                           (or *compile-file-truename* *load-truename*)))
         (object-path (compile-file-pathname source-path))
         (source-file (probe-file source-path))
         (object-file (probe-file object-path)))
    (cond ((null object-file) source-file)
          ((null source-file) object-file)
          ((> (file-write-date source-file)
              (file-write-date object-file))
           source-file)
          (t object-file))))

This does the following:

To use this function in a CS325 code file:

(in-package :cs325-user)

(eval-when (:compile-toplevel :load-toplevel :execute)
  (require :tables #.(get-code-file "tables"))
  )

The EVAL-WHEN form is used to tell the compiler and loader to execute this code before processing the rest of the file. The #. reader macro is to make sure that the pathname is generated before the compiler compiles the REQUIRE form.

Pathnames in Common Lisp

No matter which version of require you use, in order to tell it where to look, you'll need to specify file pathnames. A pathname is so called because it specifies not only the name of the file, but the path of directories and subdirectories that lead to that file.

There are two ways to write pathnames in Common Lisp. One is simple, the other is portable.

Pathname strings

The simplest and most common method is to simply write the file's pathname as a string. For example, on my Unix machine, I can write "~/lisp/require.lisp" to refer to the file require.lisp in the directory /home/riesbeck/lisp/.

String pathnames are simple. Unfortunately, they're not portable across machines. To refer to the same file on my Macintosh, I have to say "Riesbeck's HD:MCL 2.0.1:require.lisp". Not only is the path different, but the punctuation and available abbreviations are different.

Furthermore, an important thing to know about a directory path is whether is starts from "the top," or "where you are right now." The punctuation for specifying this is quite variable.

This is really not a big deal, as long as you stick to one machine, but there is an alternative.

make-pathname

The function make-pathname in Common Lisp can be used to build pathname structures (which usually print out in the form #P"...") from the basic pieces. Those pieces are:

Within these strings, punctuation does not mean anything special, e.g., a period does not indicate the start of the file type.

For example, here's how I would specify the pathname for require on my Macintosh:

(make-pathname
      :directory '(:absolute "Riesbeck's HD" "MCL 2.0.1")
      :name "require"
      :type "lisp")
     
     

When make-pathname is important

make-pathname is particularly important if you are writing functions that generate pathnames.

For example, suppose you want your program to do the following:

We could create the log file name like this:

(defun make-log-file (user-name)
      (concatenate 'string
          "/home/demo/log-files/"
          user-name
          ".log"))
     
     

For example,

> (make-log-file "riesbeck")
    "/home/demo/log-files/riesbeck.log"
     
     

Unfortunately, this code would only work on a Unix machine. We'd have to rewrite it for Macintosh's and DOS/Windows.

Furthermore, this code would have to be edited and recompiled every time we moved where we wanted the log files stored.

Here's the better way to do it:

(defvar *log-file-path*
      '(:absolute "home" "demo" "log-files"))
     
    (defun make-log-file (user-name)
      (make-pathname
        :directory *log-file-path*
        :name user-name
        :type "log"))
     
     

Not only does this work on most machines, but we can redirect where log files are stored by simply resetting *log-file-path*.

Faculty: Chris Riesbeck
Time: Monday, Wednesday, Friday: 1pm - 2pm
Location:Annenberg G15

Contents

Important Links