Yes. In most Lisps (Arc included), macro calls are expanded when function definitions are evaluated. So you have to define macros before any code that depends on them.
I am new to Lisp as well, so please excuse my ignorance as I am probably missing something thats obvious to others here.
Does this mean then, that whenever we make a change to a macro definition we must also redefine every function that already depends on the original macro definition? If this is the case then I would imagine there's a lot of tricky maintenance required for a program that has lots of 'layers' when a 'base' macro is changed.
Yes. The way most Lisp hackers deal with this is to (a) define macros before functions that use them and (b) use the function load, which evaluates all the expressions in a file in order.
This breaks down on bigger projects spread across multiple source files, at which point one needs some "make" tool such as ASDF for CL where one can pile macros into earlier files and then make other files with code using those macros dependent on the macro-bearing file(s). This can make for a lot of recompilation after minor changes, so I just take my chances and (yep) recompile all when in doubt (which with today's systems goes pretty fast).
Right. Usually I'm happy to trade speed for increased flexibility, but expanding macros at runtime doesn't seem to let you do all that much more. If anything it seems conceptually cleaner to think of macros as being expanded away at the time a function is evaluated.
One idea I'm mulling for some time is that, when you compile a function you mark it with the macros you've used, and when one of the macros is redefined it triggers recompilation of the original function. You can delay the recompilation until the function is called next time, of course.
(Actually it's not only for macros, but it will work well for inlining built-in functions.)
Possible drawback is that the effect of macro redefinition isn't very obvious in such a system. You change a macro in running server, and it can trigger recompilation of large potion of the code...
Interesting idea. However, for that to work, one would need to recompile (or mark as dirty) _anything_ that referenced the macro, which could be lists or even a single variable who's value is simply a symbol.
i.e.
(mac foo () ''a)
(= bar (foo))
And if a macro expansion used the value of bar, i.e.
(mac baz () `(prn ',bar))
then anying using baz also would need to be recompiled.
I don't think so. Since Arc (and most Lisp dialects) are applicative order, (foo) is evaluated once when (= bar (foo)) is evaluated. Redefining foo won't affect the value of bar (and it doesn't matter whether foo is a macro or a function).
Of course, if (= bar (foo)) is a part of a function, then the function should be recompiled.
Why?
No matter whether foo is a macro or a function, a closed value of 'x' is fixed in the closure returnd by bar, so redefining foo shouldn't affect the closure bound to x. (But the next call of bar returns a closure that closes 2 as 'x'). Am I missing something?
Like I said -- the usual solution, which you think about, is to lower your expectations. And I'm not joking when I say that: you really just decide to expect that `x' is not going to changed since it's "closed", so you're happy with the result. Some people set their expectations lower, and they expect a macro redefinition to not change anything about existing bindings, so they won't think about your hack as necessary.
And BTW, this is not just some cooked up academic non-problem. It's a practical issue. Consider this example:
If that macro change will (with your suggestion) lead to recompiling `bar', but that will not affect the current active call. This is a very common problem with hacking live servers (as pg often talk about):
* You cannot change the main server handler loop unless you make it call itself via its name, so the recursive call will use the new definition.
* Even if you do, you should still be extremely careful since you might change the code when there's a live thread around which was compiled with the old macros, and might rely on the old macros. For example, `f1' and `f2' use `foo', `f1' is invoked in a thread, now `foo' gets redefined, after that's done `f1' continues to call `f2': old code is calling new one, things break.
I still believe you are talking different things. My "expectation" is derived from these axioms:
* Macro is a local source-code transformation that replaces macro call with the macro definition (with some hygienic magic, if you prefer).
* The language adopts applicative order.
If you want redefining foo to affect the value of the closed variable 'x', then either you have a non-trivial definition of macros (macro transformation requires transformation of code surrounding the macro call) or you are adopting different evaluation semantics (like x is bound by call-by-name).
Note that your problem occurs even foo is a procedure. It's not a macro problem. It's a problem of semantics of closed variables.
