On the one hand, packages solve an important problem: reducing name conflicts between independent code modules. On the other hand, using them incorrectly can cause maddening file loading problems and, paradoxically, increased name conflicts. Most modern languages have something similar, e.g., namespaces in C++ and packages in Java, and all of them confuse novices.
The Name Conflict Problem
The name conflict problem comes up in large system development, no matter what programming language is being used. Large systems are broken into modules, that are developed by independent teams of programmers, and then loaded together to form the final system. This last step is usually called system integration.
Unfortunately, sometimes modules that worked perfectly when tested individually fail to work when integrated. The cause is usually one or more name conflicts, that is, the same name is used for a function or global variable in two different modules in two different ways. The hardest bugs to find are when the functions were intended to be the same but differ slightly, for example, the function in one module is an older version of the one in the other module.
Avoiding this problem involves two steps:
- When the system is broken down into separate modules, the functions and global variables each one "exports" to the whole system are carefully defined, and any potential name conflicts are resolved up front.
- When a module is implemented, any functions and variables defined that are not exported are hidden in some manner.
In Common Lisp, hiding is done with packages.
What are Packages?
Lisp, like other programming languages, distinguishes symbols, like
from strings, like
"car". Symbols are used to name variables
and functions. When your code is read, the Lisp reader converts characters into
symbols, numbers, strings, and so on, according to the punctuation rules of
Lisp. In order for your code to work, the reader has to return the same symbol,
car, every time it reads the same sequence of characters,
e.g., c, a and r. To do this, the reader keeps
a table of all the symbols it has seen. If a character sequence has not been
seen before, a new symbol for it is created, stored in the table, and returned.
If it has been seen before, the previously created symbol is returned. This
table is called a package.
The current package is determined by the value of the special variable
You can change the current package by either setting this variable, or by executing
When you create a library of code, you make sure that the names you use don't
conflict with any other code library by defining the library in a new package.
You do this with
in-package, as described
below. Then, anyone can use your library by simply loading the code files and
makes a link between the current package and the package named. It does not
The reader looks for symbols in the current package, as determined by
then in the packages linked to that package by
it finds a matching symbol anywhere, it uses that one. Otherwise, it creates
a new symbol in the current package.
Symbols in packages are marked as external or internal. When you call
you only get external symbols.
Lisp comes with two packages pre-defined:
common-lispcontains the symbols for all the variables and functions defined. Any code that uses just the symbols in this package should run in any Common Lisp implementation. Neither you nor any vendor should add any external symbols to this package.
common-lisp-useris the most common default package. It uses the
common-lisppackage plus other packages vendors provide with editing and debugging tools. It's also called
There are two steps to "packaging" a module:
- Creating the new package, using defpackage.
- Starting a file with module code with an in-package form.
You can create a package with the following template:
(defpackage #:package-name (:use #:common-lisp ...) (:export #:symbol1 #:symbol2 ...))
Here's what the above does:
- It defines a package called package-name.
- It lists the packages that this package needs, i.e., packages that define functions used by code in this package. Almost always, you need the common-lisp package.
- It lists the symbols that are external in this package.
- It uses the special reader syntax
#:symbolto refer to all symbols.
The last point is easy to do but somewhat obscure to understand. Here's the problem: how do you refer to symbols in a package that don't exist yet? Suppose you wrote
(defpackage foo (:use common-lisp) (:export fun1 fun2))
The Lisp reader will read this form, and create the symbols
fun2 in the current package. Then the Lisp
evaluator would execute this form, create the package
the external symbols
fun2. But look what
fun2 have been created in the current
package. If you try to use the
foo package, you'll get a name conflict
fun2 exist in both the current package
foo. But of course you didn't want to put
fun2 into the current package.
One way to avoid this problem is this:
(defpackage "FOO" (:use "COMMON-LISP") (:export "FUN1" "FUN2"))
This creates no symbols at read-time. It only uses name strings for symbols, not symbols. You have to use uppercase because standard Lisp stores all symbols with uppercase names. The standard Lisp reader is case insensitive. It converts all symbol characters to uppercase internally.
Unfortunately, the above won't work in what Franz calls the "modern"
version of Allegro. This version uses a case sensitive reader to support linking
with Java, which uses case-sensitive names. The version stores standard Lisp
names in lowercase. To make the above code work, you'd have to write
"fun2" above, but then you'd have code that wouldn't
work in standard Common Lisp.
A common alternative is to write this:
(defpackage :foo (:use :common-lisp) (:export :fun1 :fun2))
This uses keyword symbols. defpackage uses the internal symbol name strings
so in standard Common Lisp, it will get
and in "modern" Allegro it will get
"fun2". This works but adds every symbol in two places:
the package being defined and the keyword package.
#: approach solves this problem.
#: tells the
Lisp reader to create a symbol but not put it in any package. As soon as the
symbol's not used anywhere, it will be garbage-collected.
common-lisp is an example of a code package, i.e., a package intended
to hold a library of useful functions.
common-lisp-user is an example
of a user package. Allegro for Windows defines many code packages, and a user
cg-user. For this course, there are many code packages,
mops, and a user package
A user package is intended for interactive use in a Lisp listener window. You create a user package, link it to code packages, and make it the current package. A user package lets you control which code packages are present, hides the internal symbols in the code packages, and protects the code packages from accidental changes.
User packages are treated differently than code packages in two ways:
- You don't export anything from a user package. You use
in-packageto switch into the package to do work.
- You don't
use-packagea user package. That's a recipe for accidental and unnecessary name conflicts.
Where to create packages
The Hyperspec recommends
putting all package definitions in a single file.
This works well with
ASDF systems. This file is typically
package.lisp. By loading that package file
before anything else, you don't run into any timing issues.
For one-off utility files that don't merit an ASDF system, I put
defpackage at the front of the file.
A programmer can simply load that file and not have to worry about package definition.
Note that if the package definition uses packages other than
you need to make sure those packages have already been loaded. If that's
the case, the first approach is better.
The very first form in a file (or second, if there is a defpackage) should be:
This tells Lisp, the Lisp editor, if any, and all Lisp programmers, what package the code in this file is in. Some Lisp implementations will warn you if a file doesn't start with in-package. Others will assume you meant to say (in-package #:common-lisp-user).
There are two simple good programming rules to follow:
- Always define a package because calling in-package.
- Use only one in-package per file.
Classic Mistakes with Packages
Has something like this ever happened to you?
> (load "tables") > (deftable find-rule) Error: Undefined function: find-rule... > (use-package #:tables) Error: Name conflict: deftable already exists in common-lisp-user
Here's what happened:
- Lisp loaded tables, created the tables package, but didn't automatically connect that package to the current package. (Nor should it, since this might not be what you want.)
- Lisp read (deftable find-rule). Since there was no deftable in the current package, the reader made one and added it to the current package. But of course this new symbol had no function attached, so an error occurred.
- Calling use-package caused Lisp to try and connect the current package to tables. It properly noticed that you already have a deftable, namely the one it just created in the previous expression.
First, you need to get into the habit of calling use-package immediately after load or require.
Second, you can recover by using unintern to remove the offending symbols and calling use-package again. Many Lisp environments offer an automatic unintern as an error response option.
Using in-package instead of use-package
Do you do this?
> (load "mops") > (in-package #:mops) > (load "my-code") > ...test stuff...
If you have, then you've missed the whole point of packages.
Packages are supposed to avoid name conflicts by hiding internal function and variable names. But if you do an in-package, you put your code into the package in question, thereby guaranteeing that if you use any of the same names, you will clobber the package's code.
in-package should only appear at the front of files that define or extend an existing package.
Use use-package to make the exported symbols of packages you need available to your code. Use in-package at the Listener window only to switch into a user package.
Linking to User Packages
Have you done something like this?
(defpackage #:rules (:use #:common-lisp-user ...))
If you have, then the odds are good that your code will not be portable. In particular, when you load this code into another Common Lisp system, you will get a name conflict, or unexpected "undefined function" errors.
common-lisp-user is a working environment. It gathers together a number of packages, provides a home for temporary test variables, and so on. What's in common-lisp-user varies from implementation to implementation, and even from one release to the next. The only guarantee is that common-lisp-user will include the common-lisp package.
If your package uses common-lisp-user, two problems can arise:
- Your packages defines a function, say show-window with no conflict in your Lisp system. In another system, it turns out that common-lisp-user uses a graphics package that also defines show-window. Your code will accidentally redefine the built-in function.
- Your code uses a function, say show-window, which you mistakenly believe is standard Common Lisp. In fact, it's there because your common-lisp-user uses a graphics package. When your code is loaded into another system, it breaks with undefined function show-window.
Any package you create should use common-lisp (lisp in Common Lisp 1), plus any other non-user packages it needs. That way, you know exactly which functions you're getting and which ones your code depends on.