This seems really useful. I got the motivation off the bat, but I still don't understand the spec. Can you give an example session showing how functions are defined, and how they are called?
"The next one refers to s and str as well. Is that intended?"
Yes. Only <s> appears in the pattern itself, but the meaning of <str> is explained by "...where <str> is the string name of the symbol <s>."
---
"Also, I don't follow the distinction between the string case and the symbol case."
Did you see this part? -- "(That may look simple enough, but a completely different symbol pattern applies when matching a list element. See below.)"
A symbol which occurs in a list acts as a hybrid keyword/positional argument. A symbol by itself acts just like a string.
---
Sorry I'm being terse with this stuff. I found it hard to express my points in informal English, which is why I went with a spec document style in the first place.
Let's see...
My motive is to show you a way your desired features can be accomplished in an extensible ravioli style, rather than one big chunk of spaghetti. What you want is achievable with a simple enough collection of parser combinators, although sometimes their interfaces may be unusual due to expression evaluation being part of the parsing process.
For instance, in the system I gave, the parser combinators use a few unusual design patterns:
- The combinators must take in an evaluation environment.
- The combinators frequently quote a value so they can pass it off to another combinator that may eval it again. (I think I've seen you use this style before, actually.)
- The combinators must sometimes return the evaluated expression, just so that the (<a> | <b>) pattern can avoid double evaluation.
Another important point I'm making is that you probably don't need to reorder the argument list for the purposes of keyword arguments. My system instead (effectively) reorders the parameter pattern so the keywords are processed first.
I got this system a bit wrong because I don't support multiple keyword aliases for the same parameter. My (<a> | <b>) pattern instead binds multiple parameter variables to the same argument value, so I got that backwards.
In case it helps (or just drives a discussion), I've ironed out my own design spec for the features you're going for.
It's not quite the same as what you're describing. For one thing, it doesn't include the new restrictions you're talking about.
For another thing, it includes a new pattern type. Symbols have a lot of duties in your patterns because they deal with evaluation and keywords. I found it easiest to separate symbol patterns into two different types of patterns, only one of which deals with keywords. The non-keyword patterns I represent as strings (just to distinguish them from symbols). It would be easy to replace this with a special form or with some other literal type.
...Oh, I guess you don't have an explicit keyword parameter syntax either. I assumed you did, so it's in here too. (It could actually help clarify what's going on here. The symbol pattern deals with keywords and does just a bit more on top of that, so I like the side-by-side comparison.)
---
The pattern language has several kinds of patterns. A pattern operates on a single unevaluated s-expression (henceforth "expr") and an evaluation environment, and it outputs a map of variable bindings. It also outputs an evaluation result, if it evaluates the expr as part of its processing. (The evaluation result is only used for (<a> | <b>) patterns, to prevent duplicate evaluations.)
These are the available patterns:
<str> ; where <str> is a string
Evals the current expr and binds the variable named <s> to the result value.
<s> ; where <s> is a symbol
Runs the pattern <str> on the current expr, where <str> is the string name of the symbol <s>.
Uses those variable bindings and that evaluation result.
(That may look simple enough, but a completely different symbol pattern applies when matching a list element. See below.)
'<a>
Runs the pattern <a> on a quoted version of the current expr.
Uses all variable bindings produced by <a>.
()
Evals the current expr. If the result isn't nil, raises an error.
; any cons cell
Does several special things depending on the contents of the pattern's cons cell:
(<a> | <b>)
Runs pattern <a> on the current expr.
If that pattern returned an evaluation result, runs pattern <b> on a quoted version of that result value. Otherwise, runs pattern <b> on the current expr.
Uses all variable bindings produced by <a> and <b>, preferring the bindings in <b>.
Uses the evaluation result of <a> or <b> if available, preferring the result of <b>.
(<before...> <k> <v> ... <rest>)
; where <before...> is any number of non-keyword, non-symbol
; elements; <k> is a keyword; and <rest> could be anything,
; especially an improper list
This pattern treats the current expr as an unevaluated improper list, looking for the first occurrence of <k> with another element after it (which we will refer to as the value expr).
If the keyword binding is found: Matches <v> against the value expr. Matches <rest> against an alteration of the current expr, changed to remove the keyword and the value expr. Uses all variable bindings produced by <v> and <rest>, preferring the bindings in <rest>.
If the keyword binding is not found: Matches the pattern <rest> against the entire expr. Uses those variable bindings.
(<before...> <s> ... <rest>)
; where <before...> is any number of non-keyword, non-symbol
; elements; <s> is a symbol; and <rest> could be anything,
; especially an improper list
Let <k> be the keyword with the same name as the symbol <s>, and let <str> be this name as a string.
This pattern treats the current expr as an unevaluated improper list, looking for the first occurrence of <k> with another element after it (which we will refer to as the value expr).
If the keyword binding is found: Matches <str> against the value expr. Matches <rest> against an alteration of the current expr, changed to remove the keyword and the value expr. Uses all variable bindings produced by <str> and <rest>, preferring the bindings in <rest>.
If the keyword binding is not found: Matches the pattern (<str> ... <rest>) against the entire expr. Uses those variable bindings.
; for any other kind of cons cell...
(<a> ... <b>)
Evals the current expr. If the result isn't a cons cell, raises an error. Otherwise, runs the patterns <a> and <b> on quoted versions of the result's car and cdr.
Uses all variable bindings produced by <a> and <b>, preferring the bindings in <b>.
By the way, I discovered it with my tool, which I'm writing in Arc to learn this language. This tool crawls GitHub starting from Anarki repository, to find other interesting projects, written in Arc.
Since you are not running arc from the arc folder, you need to use either the full path, or the path relative from where you are running arc. If you are running arc in dir/foo/ and the file is in dir/bar/lib/http.arc, you could use:
Well, probably I should ask another question, addressed to the different implementations' authors: "Why do you started to write your own implementation instead of making efforts to improve Anarki?" :)
I don't know how to answer it :) There's a lot of code in the world. I can help answer questions on some of it, but I have no idea what's most promising.
I upvoted you because I was happy to see the list, and I'd be interested in other people's answers.
If pg ever contributes a line of code to anarki I'll eat my hat :) Not that that's a bad thing; it works perfectly well for each side to port changes from the other.
I wonder what will happen to Arc now that pg has passed YC on to Sam? How is he going to pick up work again? Will he just keep working on arc 3.1, or will he just contribute to anarki?
There might be more, but these should get you started.
I should probably mention that most of that documentation is probably targeted at the the vanilla arc 3.1, which is the lowest common denominator at this point for all of the arc implementations. Every other arc implementation has its own set of additional libraries, and usually a way to access libraries from its host language. I.e., anarki also includes multiple ways to call racket code directly.
I think it might help to point out how the macro expands there:
(inl 1 '(1 2 3))
is transformed by the reader to
(inl 1 (quote (1 2 3)))
which is then macro expanded by
`(in ,elt ,@lst)
where elt is bound to '1, and lst is bound to '(quote (1 2 3)). This means that when lst is spliced in, you get:
'(in 1 quote (1 2 3))
As stated, the problem is largely the choice of macro instead of function. Macros are primarily useful to control evaluation; they do not evaluate their arguments by default. A macro will only let you work with the literal input, not their values. You can't (with arc) have your cake and eat it too; you can only work with either the syntax, or the values. You wouldn't be able to do something like:
(let x '(1 2 3)
(inl 1 x))
for instance, because it would attempt to treat the symbol 'x as if it were a list to be spliced, not the value of x. That's why you need a function, like pos.
So the problem is that '(1 2 3) really is a list with two things in it: the symbol 'quote and the list (1 2 3). I'm a little bit uncomfortable with this explanation, but this seems to mean that you don't want to pass a quoted list into a macro, but unquoted. This is the way most macros are used, but I hesitate to make it a global rule. Some day I'll understand macros better.
That probably explains the error message. (Quotes expand to the symbol quote.)
So all you have to do is not quote your argument:
arc> (inl 1 (1 2 3))
t
As a general rule, however, try not to use a macro unless you have to. In this case there's a pure function that does what you want:
arc> (pos 1 '(1 2 3))
0
arc> (pos 4 '(1 2 3))
nil
(You can use pos in a conditional just like inl because it is guaranteed to only return nil if and only if the element doesn't exist.)
More info: http://arclanguage.github.io/ref/list.html#pos. As an exercise try implementing pos for yourself :) And feel free to ask us more questions! I hadn't realized there were questions at stack overflow. I'll try to hang out there, but there's really nothing off-topic here.
In Python (at least in Python 2.x, where map and filter return lists instead of generators), the equivalence looks like this:
[expr_involving_x for x in xs] == map(lambda x: expr_involving_x, xs)
[x for x in xs if expr_involving_x] == filter(lambda x: expr_involving_x, xs)
For example,
>>> [x + 1 for x in [1,2,3]] == map(lambda x: x + 1, [1,2,3])
True
>>> [x for x in [1,2,3] if x == 2] == filter(lambda x: x == 2, [1,2,3])
True
You can apply these transformations recursively until you remove the list comprehension notation altogether. E.g.,
[x + 1 for x in [1,2,3] if x == 2] -> map(lambda x: x + 1, [x for x in [1,2,3] if x == 2])
-> map(lambda x: x + 1, filter(lambda x: x == 2, [1,2,3]))
Nested comprehensions have a transformation rule, too, but it's somewhat more complicated:
[expr_involving_y for x in xs for y in expr_involving_x] == sum([[expr_involving_y for y in expr_involving_x] for x in xs], [])
For example,
>>> [y+1 for x in [1,2,3] for y in [x+1,x+2,x+3]] \
... == sum([[y+1 for y in [x+1,x+2,x+3]] for x in [1,2,3]], []) \
... == sum([map(lambda y: y+1, [x+1,x+2,x+3]) for x in [1,2,3]], []) \
... == sum(map(lambda x: map(lambda y: y+1, [x+1,x+2,x+3]), [1,2,3]), [])
True
So, list comprehensions are just a way of saying the same things as higher-order functions. Arc generally favors the higher-order function flavor of this idiom. map is still called map, filter is called keep, and Python's sum(..., []) is spelled (apply join ...):
You might think Arc's preference for these comes from Lisp dialects naturally eschewing specialized syntax. But, Arc does like syntactic shortcuts; just ones that apply more generally to functions, like bracket notation (e.g., [+ _ 1]) and ssyntax (e.g., ~f and f:g come up a lot in higher-order calls).
It's also interesting to note that due to the previous rules, you can easily write a macro that turns list comprehensions into the equivalent chain of higher-order function calls.
There are many Arc builtins that can help build lists, too. They're all defined in Arc itself, so just take a look through arc.arc to figure out how they work. Functions I spy that may help for list building (list of lists or otherwise): rem, mappend, trues, firstn, nthcdr, cut, tuples, pair, n-of, bestn, split, retrieve, dedup, reduce, rreduce, range, ... Really just depends on what you're looking to do!
In these examples, the first error happens during macroexpansion, when the compiler is processing the 'fn special form. The second error happens at run time. They both recover to the REPL just fine.
The second stack trace doesn't include 'foo or 'map, but I bet that's because they made a call in tail position, leaving no trace of their stack frame. Let's change 'foo so it doesn't do that:
Arc seems to throw away source location information, but I think we could get that back too if we tried. It'll involve using Racket syntax objects somehow.
Thanks for speaking up about this. This seems like something that could help a lot of people. XD
Thank you for writing up such a clear explanation.
There are definitely safety benefits to using the simplest form. But I would prefer to use the most general form (withs) because I won't need to worry about using the most idiomatic form or to switch from one form to another as the program changes. It frees me to worry about other things.
Being able to tell simpler parts from more complex parts at a single glance isn't as much of a concern for me. I noticed I don't go back to even read code of programs that didn't solve a big enough problem. The programs reach a dead end and don't develop further. There are more important things to worry about when writing a program.
Also note what let, with, and withs offer could be provided in other languages too. Them being implemented as macros in Arc is incidental. You're not giving up the benefits of lisp by not using them.
