MCL In Fab
Volume Number: 13
Issue Number: 3
Column Tag: Tools Of The Trade
MCL in the Motorola Chip Factory
By Stephen L. Hain, Digitool, Inc.
Creating a "fab" solution in a web browser using Macintosh
Common Lisp and MIT's CL-HTTP server
MCL Conducts an Orchestra
Macintosh Common Lisp (MCL) is in active use in projects at several universities and
corporations worldwide. One of these is Orchestra, a Motorola project to develop
automated tools for scheduling wafer fabs (factories in which semiconductor wafers
are fabricated) in their Semiconductor Products Sector.
When the Orchestra implementors decided to improve the user interface for the "Fab
Browser" part of the system by putting it on the web, they found a simple and elegant
solution using only a hundred additional lines of MCL code and CL-HTTP [Mallery 94],
a Lisp program from MIT which I describe below. A description of that solution, based
on a presentation by Byron Davies at the 1996 Dynamic Objects Workshop in Boston,
appears later in the article.
An Overview of Lisp
Lisp differs from traditional languages such as C and Pascal in several ways. Lisp
variables can hold values of any type, and there are quite a few available: strings,
symbols, lists, vectors, structures, hash tables, integers, floats, ratios, and packages
to name but a few. Lisp functions can be created and used without requiring an
edit-compile-link cycle; they can be generated by Lisp code and passed as arguments to
other functions; and they can "close over" data values in sophisticated ways. A macro
facility allows for compile-time manipulation and generation of Lisp expressions, and
a package system segregates program symbols (variable and function names) into
separate namespaces to ease the integration of programs and development of large
systems. There are powerful output formatting functions, a stream mechanism, and
comprehensive mathematics functions and data types, including bignums
(unlimited-size integers), complex numbers, and ratios. There are high-level control
structures for dealing with non-local exits, and an object-oriented exception
mechanism. And of course, there is CLOS (Common Lisp Object System), an extensible
object-oriented framework that's particularly useful for implementing other
object-oriented frameworks, if you're into that sort of thing. CLOS objects are
instances of classes, and they are operated on by functions called methods.
What's "common" about Common Lisp is that it's a widely-used standard for the
language, and code or portions of code that are compliant to the standard are able to run
on platforms implementing a compliant Common Lisp. CL-HTTP is an example of a
large Common Lisp program which runs on many platforms.
What MCL Brings to the Party
MCL is the Macintosh implementation of Common Lisp. MCL was originally developed at
Coral Software, then became an Apple product when that company was acquired by
Apple, and is now a product of Digitool, Inc. Over the last decade or so it has kept
evolving - the current version, MCL 4.0, generates native PowerPC machine code.
MCL was used at Apple as the host environment for the Apple Dylan and SK8 projects.
Besides being true to the standard embodied in [Steele 90] (commonly referred to as
CLtL2), MCL provides a rich development environment based on the Macintosh user
interface, and several useful extensions to the standard. Typically, the user interacts
with MCL through one or more listener windows, specialized text editors that evaluate
Lisp forms that are typed or pasted in, display their results, and maintain a history of
the interactions. Files are edited and displayed in Fred windows (Fred Resembles
Emacs Deliberately!) and, like other Emacs text editors, Fred is extensible and
programmable.
MCL supports both low-level and high-level Toolbox interfaces. You can access the
Toolbox most directly using the supplied interfaces, which define the constants,
records, and routine ("trap") calls we've all come to know and love. Object files and
shared libraries can be dynamically linked to an MCL program, and the interfaces to
them are defined in a similar manner. In general, it's better to use the high-level
interfaces to access the Toolbox. These interfaces include the menu, window, and dialog
classes and methods, the file system functions and file streams, as well as a host of
others (QuickDraw, MacTCP, serial streams, etc.) With these, a bug in your code will
likely lead to a Lisp error instead of a crash into the debugger.
Built-in development tools include a window-based graphical data inspector; a
stack-backtrace window which displays program execution frames, allows frame
values to be changed, and let's you restart execution from any frame; facilities for
quickly finding the definition(s) or users of a selected variable, function, or method; a
programmable trace facility for tracing function calls, also accessible via dialog; an
advise facility for modifying the behavior of functions; and a step facility for
single-stepping through Lisp expressions.
The MCL development environment makes use of multiple processes (threads), as can
MCL programs. Processes allow different pieces of Lisp code to run concurrently.
Details of MCL's processes can be found in the MCL Reference Manual. Some of the
many additional library and example modules that come with MCL are a graphical class
grapher, WOOD (an object-oriented database), and even a turtle-graphics package.
Figure 1. MCL Listener, Get Info, and Inspector windows.
Figures 1 and 2 show several different MCL tools and windows in action. In figure 1
there is a listener, currently in a debugging break loop; a Get Info window displaying
all of the callers of the window-close function; and several Inspector windows
displaying information about the installed menus and menu items.
In Figure 2 there is a stack-backtrace window (associated with the listener process
that is in the break loop); a Processes window displaying the status of all processes;
and a Fred window displaying one of the example files (in two viewing panes).
Figure 2. MCL Stack Backtrace, Processes, and multi-paned Fred windows.
There is not enough room here to go into all of the features and tools in
the MCL development environment; more information on them can be
found in the "Getting Started with Macintosh Common Lisp" manual
(available on-line at
95].
Under the Hood
The MCL program environment consists of a standard Macintosh heap, containing
pointers, handles, resources, and the like, and a Lisp heap, containing Lisp data,
including the functions of the development system itself and any user code. In addition,
MCL 4.0 on the PowerPC makes use of shared libraries to separate various parts of the
program. Once code is loaded into MCL, a new image (MCL application) can be created
with a simple Lisp function call (see listing 1).
Listing 1: Loading your stuff and saving an MCL
image
; load all source files in a folder:
(loop for file in (directory "HD:sources:*.lisp") do (load file))
; save out a new image:
(save-application "MCL + My Stuff")
Forms and functions "evaluated" from a Listener or Fred window are usually first
compiled automatically by an incremental compiler, and then run as native code. Lisp
source files can be compiled into object files which then load very quickly into MCL.
There are two garbage collectors to pick from in MCL: the "mark/sweep" collector,
which runs when memory has filled up, displaying a GC cursor as it pauses the
program to do its job; and the ephemeral garbage collector, which operates in the
background, cleaning up unused data as a program runs, rarely (if ever) causing a
pause. Every listener window you create has its own process, so you can start some
Lisp code running and continue to work with the MCL environment. There is a separate
process running that handles events (mouse clicks, menu selections, key presses,
Apple Events, etc.); should an error occur in that process - for example, in a function
you've defined that is called from a menu item - all is not lost! A new, stand-in event
process is created so that life can go on while you debug the problem. The latest
versions of MCL can even grow the stacks of processes that use them up.
What (Good) is This Thing Called Lisp?
There are many different reasons why Lisp or MCL is ideal for some tasks; here are a
few of my favorites:
• The dynamic environment. You can add and change functions quickly, and have
access to the program you're working on while you're in the development environment.
Or you can think of it as having access to the entire development environment while
your program is running. Might you wonder if changing a program while it's running
isn't asking for a trip to MacsBug? Well...
• Safety. If you stick to the high-level interfaces and supplied mechanisms, you're
less likely to crash than to cause Lisp errors, which are handled more gracefully.
Because Lisp is weakly typed (variables can hold values of any type), it spends some of
its time checking for valid data before passing it down to the inflexible operating
system routines below. (Declarations can be added to a Lisp program to speed it up by
bypassing some of the type-checking, trading some safety for increased speed.)
• Elegance. Whatever you're programming style, you can't beat Lisp (well, I can't)
for giving you so many facilities for writing clear and well-organized code. Macros,
packages, and CLOS come into play here, as well as the exception mechanism and a
wealth of control structures. (See listing 2)
Listing 2: Defining and using a macro that uses
unwind-protect
; Define a macro named with-locked-handle.
(defmacro with-locked-handle (handle lisp-form)
`(progn ; just groups the following forms together
(lock-handle ,handle) ; lock the handle
(unwind-protect ; ensure that unlock-handle is called,
even if
; lisp-form "throws out" to a higher frame
,lisp-form ; this "expands" to the code in the macro
call
(unlock-handle ,handle))))
; Define a function that uses the with-locked-handle macro.
; (Assume that dereference dereferences a handle to obtain a pointer,
and
; that deposit-bytes deposits bytes from a list into memory.)
(defun fill-handle (a-handle list-of-bytes)
(with-locked-handle a-handle
(deposit-bytes list-of-bytes (dereference a-handle))))
• Ease of debugging. The combination of the Inspector, function tracing and
advising, Lisp breakpoints and break loops (Listeners created to allow debugging
"paused" code), and having errors caught by Lisp instead of a low-level debugger can
speed the debugging process considerably. Say you want to advise a function to write its
arguments and result to a log file every time it's called. A lot of work? Not really - see
listing 3. When you're happy with the operation of the function, you unadvise it.
Listing 3: Advising a function to log to a file
; Make hairy-function log its arguments and result to a file.
; with-open-file is a macro - it takes some "keyword arguments" to
guide it.
; loop is also a macro, that parses an English-like form and generates
tight Lisp code.
; :if-does-not-exist, :create and other colon-prefixed symbols are
"keywords" - symbols that
; evaluate to themselves (unlike variables, which evaluate to the
values they are bound to)