Arc Forumnew | comments | leaders | submitlogin
The real problem with unhygienic macros
17 points by andreuri2000 6485 days ago | 20 comments
The more important problem solved by Scheme's macros is /not/ the one solved by gensym/uniq. Consider for example

  (let oaf 1
    (when 2 3))
With unhygienic macros, this may or may not work depending on the definition of WHEN. There is no way of fixing this with UNIQ. Here is a definition of WHEN for which the above code breaks

  (mac when (test . body)
    `(x ,test (do ,@body))) 
  (mac oaf (test conseq)
     `(if ,test ,conseq))
In other words, to use any Arc non-primitive syntax with any confidence, the programmer has to know the full definition of all the macros involved. In other words, non-hygienic macros do not define closed abstractions.

While this may be less of a problem in a multi-namespace Lisp, it certainly becomes more of an issue in a single-namespace language like Arc.



6 points by vsingh 6485 days ago | link

This just made me notice something interesting.

This works:

    (eval (list 'oaf 1 2))
    2
But this does not:

    (eval (list oaf 1 2))
    Error: "Bad object in expression #3(tagged mac #<procedure>)"
In other words, you can't evaluate a form in which the car is the actual macro object itself, rather than the name of a macro. If we could make that work, we could solve your problem by inserting the macro objects we want where we want, in effect telling Arc which version of 'oaf' we want in advance.

It would make macro expansions pretty ugly to look at, though.

-----

9 points by pg 6485 days ago | link

In other words, you can't evaluate a form in which the car is the actual macro object itself

The single biggest compromise I had to make because of MzScheme was not being able to put objects like functions and hashtables into macroexpansions. I don't know if I would have wanted to do what you tried to, but in other ways I'm constantly inconvenienced by this restriction.

You can do it in most CL implementations, though the spec is ambiguous on the matter.

-----

1 point by vincenz 6482 days ago | link

This was brought up before, namely here:

http://arclanguage.com/item?id=804

And there was a partial solution as well.

-----

2 points by akkartik 6485 days ago | link

This came up before: http://arclanguage.org/item?id=654

Why do you think it's ugly?

-----

4 points by kennytilton 6485 days ago | link

With power comes responsibility. Java sees programmers as irresponsible, so it gives them no power. Arc/Lisp gives us power, including unhygienic macros, and leaves it to us to use them responsibly. /Can/ you contrive an example that breaks things? Yes, but that is no surprise, PG said Arc would do that. What would be more compelling is an example of real-world coding that ran afoul of capture. In CL, btw, I do not think it is the extra namespaces that help us, I think it is that ghastly warning we get when we try to redefine a bit of the language.

-----

7 points by andreuri2000 6484 days ago | link

Hygiene does not diminish power. Various hygienic macro systems exist that are at least as powerful as defmacro. An extremely simple one is explicit renaming, which allows simple controlled capture. Syntax-case (see R6RS) is strictly more powerful than MAC (in the sense that MAC can be expressed in terms of syntax-case but not vice versa). Explicit renmaing and syntax-case have the power to respect lexical scoping, while no amount of tricks can stop MAC from breaking lexical scoping. Syntax-case also allows controlled capture, indeed much finer control over capture than MAC does. So an argument based on power would in fact favor hygienic macro systems over MAC.

-----

1 point by kennytilton 6481 days ago | link

But variable capture is just a hobgoblin to frighten the kiddies, nothing that ever Actually Happens(tm) to those using unhygienic macros in anger. Meanwhile, syntax-case may theoretically be able to express MAC, but the reality is that it is a PITA to program with. The power of syntax-case is theoretical, the power of MAC is in practice. When I have to write some code, I'll take the latter.

-----

2 points by andreuri2000 6479 days ago | link

It is not "downward" variable capture that bothers me. I can insert gensyms as well as anybody else to avoid that. It is lexical scoping, as in

  (def sort (sequence <)
    ----
    (some-macro-use)
    ---
    )
I can never be sure that this is correct unless I go and read the definitions of (some-macro-use) and all the macros that it may in turn depend on, to make sure that they do not depend on, say, the standard binding of < and perhaps some toplevel binding of "sequence". And I have to do this for /every/ local variable binding surrounding every macro use in the whole program. Also, I had better be sure that no-one will go and change the definition of some-macro-use, or any of the macros on which it depends, at any time to depend on the toplevel <, a toplevel "sequence", or /any/ other lcoal variable that may surround /any/ use of some-macro-call anywhere in the code.

The fact that you have not run into this problem shows amazing discipline, and may have something to do with CL not being a Lisp-1. I have no such discipline, and prefer having the language take care of lexical scoping for me. But then, I don't plan on using Arc, so feel free to ignore this.

-----

2 points by kennytilton 6479 days ago | link

It is not clear that the nightmare outlined above applies to actual programming. I am a little weak on my Lisp-1, but IIUC the above changes < to be whatever comparison operator got passed to sort, so the author of the above sort should be fired for breaking <. As Arc matures it might not hurt for the compiler to issue a warning over such sabotage, which is really what it is so I am not sure why we are discussing it as a Real World Problem.

As for "sequence" and someone depending on some /other/ higher level binding, ie, it is capturing the variable sequence by design, (a) intentional variable capture is as rare as it is powerful and the premise is that one has a suite of cooperating macros that work together to bind and then use the rare variable designated for capture, so one is not invoking one of these macros in isolation unaware of how it works or what it captures. A good example from my code at one time was the binding and capture of SELF, being used in some OO wizardry to stand for the "instance at hand" a la smalltalk. If I was rebinding self outside the cooperating suite of macros (which I did, rarely) it was because I had coded myself into a corner, but I knew damn well what I was doing and why I was doing it and I still crossed my fingers.

btw, a trick the CL crowd uses for lexical vs dynamic binding (the latter being an opportunity for a similar kind of confusion as the one you fear) is simply to name dynamic variables +like-this+. (not really, we use asterisks, but this forum would just show like-this if I bracket it with asterisks.) That simple convention solves the whole problem, as long as we fire anyone naming a scratch variable that way. :)

-----

2 points by hakk 6485 days ago | link

Going further, many CL implementations also come with a package locking mechanism.

-----

4 points by kennytilton 6484 days ago | link

There cannot be a problem with unhygienic macros. I have 527 of them across 103 Common Lisp source files (I like small source files) and have not once ever had a problem with unintended capture, Not even when I was new to macros. QED.

Hygiene is a solution without a problem. Bad solution, bad!

btw, 2800 defuns and 800 defmethods (OO defuns, in brief) in the same source.

-----

2 points by andreuri2000 6485 days ago | link

Sorry, I meant:

  (mac when (test . body)
    `(oaf ,test (do ,@body))) 
  (mac oaf (test conseq)
     `(if ,test ,conseq))

-----

3 points by greatness 6485 days ago | link

The odd thing about this is that I KNOW it should break, but I can't make it do so.

  arc> (let oaf 1 (when 2 3))
  3
I'm pretty sure it is because it evaluates the macro expansion ahead of time.

  arc> (macex '(when 2 3))
  (if 2 (do 3))
Additionally, does anyone know why if isn't considered an object:

  arc> if
  Error: "reference to undefined identifier: _if"

-----

3 points by randallsquared 6485 days ago | link

The symbol 'if is a special operator, not a macro or function. The special operators in Arc's compiler are quote, quasiquote, if, fn, and set.

Probably it would be convenient to have an object tagged as a special operator for those, but it would be documentation, not functional code.

-----

4 points by pg 6485 days ago | link

Special operator is what they're called in CL. I don't know offhand what they're called in Scheme. But what it amounts to is that if only exists as a clause in the compiler.

-----

2 points by randallsquared 6485 days ago | link

"Special operator is what they're called in CL."

Yeah, I know much less about Scheme than CL, so I used the CL term.

-----

1 point by jimbokun 6482 days ago | link

What are they called in Arc? :)

-----

2 points by pg 6481 days ago | link

Special operator sounds good. Or axiom, if we want to be fancy.

-----

1 point by vsingh 6485 days ago | link

Hmm.

I guess the macros in the form are expanded with reference to the context of the entire form (the global context).

Does this mean we don't have macrolet?

-----

1 point by simonb 6485 days ago | link

It probably has more to do with if being one of the primitives (or axioms if you will).

-----