I myself have implemented large amounts of Arc in PLT Scheme, hitting some roadblocks, passing some of them and giving up on others. I've done the same thing, more recently and extensively, in Common Lisp. I can show you what I've done if you are interested.
Note that one could write an 'fn macro that, depending on what's in its parameter list (which is certainly available at macroexpansion time), might expand directly to a 'lambda form, or might expand to (lambda args (destructuring-bind <parameter list> args <body ...>)). And Scheme does have something called 'case-lambda, which looks like it can be used to make functions with optional parameters. I have implemented a pretty good 'fn in Common Lisp (which supports optional parameters):
(Note some things I've done: I wrote 'mac so it supports the . notation for rest parameters--which is what 'restify deals with, by the way; I wrote 'with so that you can either go (with (x 1 y 2) ...) or (with x 1 ...), since 'let is already defined; and I wrote 'dsb, for "destructuring bind".)
Also... with PLT's "Pretty Big" language, in the Languages menu (click Show Details), you can uncheck a box called "Disallow redefinition of initial bindings", and then you are free to redefine 'let and 'if and so on.
And it would, in fact, be possible to make = reach inside structures. You would just have to tell it how to do so--give it a bunch of patterns, like (= (car x) y) --> (set-car! x y). Hmm, also, it turns out there's a procedure named 'make-set!-transformer, which sounds like something we could use.
But I didn't figure out how to make = work well for assigning to variables that don't exist yet. The general variable assignment procedure in Scheme is set!, as in (set! x 3). But this is an error when x is undefined. To assign to an undefined variable, it seems you must use either namespace-set-variable-value! or define; but 'define can't be used in the middle of an expression (to allow for "block structure", so you can use 'define to make local recursive functions and stuff), and if we simply make (= a b) expand to (namespace-set-variable-value! 'a b), then that won't work when we want to modify a local variable. Here's the best I can do, but I imagine the overhead of the 'with-handlers thing would make it take obscenely long for, say, incrementing the variable i 10000 times... having said that, I tried it out. It takes 33 msec to go (set! x (+ x 1)) 100000 times, and it takes 100 msec to go (= x (+ x 1)) 100000 times. Which is not nearly as bad as I feared--likely any other code your loop executes will take much longer than the difference this makes. I guess this actually isn't a problem. Anyway, my code:
Here, the body of 'thing contains a call to 'expand-thing, which is a function that you just defined. In Scheme, if you define a function and then try to use it in the body of a macro, it won't work--when you use the macro, it will complain of an undefined function. Now, it is possible to get around this: Scheme has a special form, 'begin-for-syntax, which means "execute this code at compile-time". We could, therefore, go
(begin-for-syntax
(define (expand-thing x y) ...))
(define-macro (thing x y)
(expand-thing x y))
That would work. But, now, suppose we had a macro of our own, such as our version of 'if, that we wanted to use in the body of thing or of expand-thing. This does not work:
It will try to expand the call to 'meh and find that 'my-if is an undefined variable. Macros, defined at compile-time, do not get expanded in the bodies of things that are defined at compile-time. (Actually the PLT docs talk about "phases", rather than "compile-time" and "run-time"; it seems you can have arbitrarily many phases of syntax expansion before the final evaluation.) To make this happen, you would sorta have to go (begin-for-syntax (define-macro ...)), but then it doesn't recognize the word 'define-macro. I think it's possible to load mzscheme at earlier syntax phases so it recognizes 'define-macro 2 phases up, but then if you wanted to use a macro in the body of 'my-if, you'd need to go 3 phases up, and so on: it's just too terrible.
My best idea seems to be this: At compile-time, create a table (an assoc-list?) of macro names and their body-functions. Then make your 'mac and 'def and all the other things you want evaluated at compile-time, make them macroexpand their bodies. Do this by checking if it's a list: if so, check if the car is the name of a macro: if so, apply the macro's body-function to the cdr, and repeat; otherwise, leave the list itself as is, but look at its elements (recursively) and macroexpand them. But what if this macro call is inside a quote, or a quasiquote? Oh god. You have to write a whole goddamn code-walker to make this work. I stopped at that point.
Common Lisp works fine on the macros issue. It also agrees with Arc about nil. And SBCL is pretty damn fast. But it has a separate namespace for functions, which is pretty non-foolable with, and it seems rather unwilling to allow redefinition of 'if and 'let and 'map (they call it 'mapcar) and so on. I tend to switch between Common Lisp and Arc these days.
The general variable assignment procedure in Scheme is set!, as in (set! x 3). But this is an error when x is undefined.
I just noticed compile-allow-set!-undefined the other day:
Welcome to MzScheme v4.2.1 [3m], Copyright (c) 2004-2009 PLT Scheme Inc.
> (set! a 3)
set!: cannot set undefined identifier: a
> (compile-allow-set!-undefined #t)
> (set! a 3)
>
fallintothis: I'm aware of the destructuring use of the Arc 'let. But I find that when I want destructuring, I usually want to go (mapcar (fn ((a b)) ...) (tuples 2 xs)), and I rarely use an explicit 'dsb; if I want it, though, it's there. My version of 'with is all I need most of the time.
I am trying out your "arc.lisp" now. Seems great so far.
There is much exploring to do, but my initial confusion with =. Seems it's not working and maybe not even defined. I read your discussion above of the problems with =, but I thought that was about your Arc in PLT, not CL. I haven't looked closely at the guts of "arc.lisp" yet, so maybe that will answer my question.
Oh, I forgot to include that in the list of idiosyncrasies. Basically, use setf wherever you would use = in Arc. The reason, in short, is that = is already the numerical-equality function, and CL doesn't like people to redefine things.
In PLT Scheme, you just have to uncheck a box that says "Disallow redefinition of initial bindings" and then you can redefine almost everything, but in CL, it will complain about breaking a package lock and go into the debugger whenever you try to redefine a built-in function, of which = is one. It's possible to tell it "Yes, ignore package lock", but I don't want to do that for every single function every time I start up SBCL. I think it is possible to tell it to automatically ignore the lock... But this is the way it is right now. Also, when you try to redefine 'if, you just run into a brick wall:
The special operator IF can't be redefined as a macro.
I stumbled on one other workaround, too... you can create a new package and choose not to import (all of) the stuff from CL-USER. So far, though, this doesn't even import the symbol nil, and when I do import it, it gets printed with a "COMMON-LISP:" prefix.
I guess one might be able to get used to that. Might try it out if I feel adventurous. For now, put up with using setf, mapcar, iff, and my version of with.
Gahd, that was a fun read. I'd love to hear about your experiences with SBCL[1]. I've been porting my app over to it for the last 24 hours, lazily taking arc functions/macros as I go. My approach so far is to put up with &rest, extra parens in let, explicit destructuring-bind[2], etc.
[1] Perhaps offline; email in my profile; did you know the email field itself is not public?
[2] I wrote a bind macro variant a few years ago (http://akkartik.name/lisp.html), I may start using that. Still a little more verbose. Hmm, if I eliminate multiple-value-bind I can get rid of :db..
