Gazelle: Modules and Macros and Hygiene, Oh My!

In my last post I gave a guided tour of using Gazelle's module system, and while I mentioned that one can use define-macro+ to define an external macro, I did not provide an example. Turns out that macros and module systems interact in ways that require some thought. This post describes, via an example, how Gazelle attempts to resolve this tension.

The Surface Problem

Suppose we want to add a type safe delay operation to Javascript. A not typesafe delay operation, the lambda with no arguments containing the expression we want to delay, presents itself immediately, but such delays are not easily distinguishable from regular functions, and we may wish to delay functions in certain contexts where we wish to accept functions, delayed functions, or other kinds of delayed things. Which is to say that we want to have a new class with its own prototype we can use for dispatching which implements delay.

(module ()
  (define (Delay fun-value)
    (set! this.function fun-value)
    this)

  (set! Delay.prototype.force 
   (lambda () (this.function))))

So far so good, now Delay objects are instanceof Delay. We can dispatch with pattern matching, using the instanceof pattern, for instance.

So what about some syntactic sugar? As it stands, we construct a new Delay like so:

(new Delay (lambda () some-expression))

But this is a bit redundant - delays should really be syntatically like a lambda without an argument list, eg:

(delay some-expression)

Luckily, we have a Lisp, so we are tempted to write the following macro:

(define-macro+ delay (expr)
 `(new Delay (lambda () ,expr)))

This expression defines an external macro called delay which appears to expand in a straightforward manner to an appropriate invocation of new. But this is unfortunately not the case ...

What's in a Name?

So what is wrong with our macro, above? Well, if we use that macro like this:

(require (("delay/delay" :all))
  (var d (delay 10))
  (d.force))

We are in the clear. But if we are a bit more careful/clever:

(require (("delay/delay" (:as ($ delay))))
  (var d ($ 10))
  (d.force))

Where the require form now imports only the delay macro, and renames it $, we will get an error that there is no constructor corresponding to Delay, because we haven't imported any such binding from the delay module. An error immediately on invocation of new is the best outcome, in fact: if some other value is floating around bound to Delay, we might have to wait until a very confusing moment to find out there is a problem.

The problem is that macro expansion takes place in the module the macro is expanded in, and symbols in a macro expansion refer to symbols in that module, not the symbols in the module where the macro was defined.

Of course if your macros never want to "capture", then you don't encounter this problem. For instance, we could have written delay like this instead:

(define-macro+ delay (constructor expr)
  `(new ,constructor (lambda () ,expr)))

Where we require the user to pass the constructor, presumably Delay, to the macro to ensure that it expands correctly. I felt that this was particularly ridiculous in this instance, because it reduces delay to an alias for new with the restriction of a single argument.

The Better Solution

The other alternative is to somehow allow the user to qualify which value or macro their macro expansions refer to. In Gazelle, you do this in the following way:

(define-macro+ delay (expr)
  `(new (from "delay/delay" Delay) (lambda () ,expr)))

Where we use the special form from to refer to values from a specific module, regardless of where the macro is expanded. Indeed, from can be used in ordinary code to refer to module level values at any time. Only public module objects and macros can be referred to.

So how does this solution stack up?

1. The Good: Obviously the ability to refer to specific values from specific modules despite the expansion environment is almost required for meaningful macros.

2. The Bad: However, resolving the issue of macro hygiene at only module-level granularity falls pretty far short of the nicer properties of hygienic macros or the implicit properties of Common Lisp Style macros.

3. The Ugly: You've got to remember to qualify your macro expansions when you want the module behavior, AND the from syntax incurs a slight run time penalty for run-time values, because they are looked up via require.js at each use. Presumably, this lookup is cached in require.js and so should not be a big penalty. Still!

Conclusions

The solution outlined above is a reasonable compromise. The implementation of Gazelle, which uses the Emacs Lisp Reader, would have to be significantly more complex to support full macro hygiene (also, I don't yet know how to do that).

The important thing is that this solution enables most of what you want for certain kinds of macros. Probably in the next iteration of Gazelle, I'll think of something a little nicer.