Given that Anarki exists and appears to be updated reasonably frequently, it seems likely that, unless pg does something, sheer genetic drift will cause Anarki and ArcN to diverge enough to become separate and incompatible dialects. If someone makes a killer app in Arc, and uses Anarki-specific extensions, it may very well be that merging ArcN into Anarki may be too difficult and possibly even counterproductive (e.g. if too many names in an Arc3 conflict with names on Arc2-based Anarki).
If it goes on for longer, both may even evolve into entirely different languages (like Scheme and CL). Might make a good study of memetics and the origin of programming languages ^^.
It's because 'quote doesn't necessarily create a new list every time (at least not in function definitions). If you do
(def foo ()
'(a b c))
(foo) will return the same list every time, so if you modify the list in or before your next call, you'll get the modified instead of the original value.
Strange, downloading lmr.tgz gives a demo.lmr which seems to be a corrupted copy or something of lmr.arc.
What is the file format, anyway? It looks like Arc readable format but I can't figure out the structure.
Regarding the intersect code: generally most systems work with shapes that are just the basic triangle plane. This also generally means that the intersect code would really just return one point, not many - currently, your code seems to return a list of points for each shape. Even if your "shape" becomes an entire tree of shapes (say a BSP tree), I would assume that only one intersection - the one nearest to the ray source - would "win", so intersections on shape trees should (I think) return just one intersection.
Returning just one intersection saves memory - you don't need to store several lists. In particular, all you have to do is to get the intersection with the least distance:
(def gen-intersection (ray)
(= intersection nil)
(each sh shape
(aif (intersect ray sh)
(when (or (no intersection) (< (car it) (car intersection)))
(= intersection it))))
intersection)
Can you give your reasons why you chose to retain multiple intersections?
If you really think retaining multiple intersections is better, I would suggest you also study lib/tconc.arc, which includes a somewhat-fast list concatenate function (but you need a transparent wrapper around the list you are building).
Depending on how motivated you are at studying the list structure and advanced stuff, you might prefer to use explicit head+tail catenations, although head+tail tends to be better used on continuation passing style, especially in languages (e.g. Arc) which don't naturally support multiple-value-returns (lists don't count, you have to destructure them, which takes time).
Also: avoid using nested maps - each map reallocates memory:
(of course, cross3 doesn't seem to be used anyway)
Finally: avoid using 't as a variable name.
I think the greatest consumer of time would probably be 'map. 'map is nice but it creates a structure, and the implementation is not written in tail-recursive-modulo-cons ^^. If you're just going to work with individual lists, you can use map1, which is like map but only works on one list, but is slightly faster.
The other thing I think that the time is being consumed in might be with the use of 'alref. It might be better to use map 'listtab on your shapes, so that you don't have to use alref, which always does linear search.
First off, thanks for the pointers! I'll certainly look into those when I have a chance.
I have no idea why the file was corrupted. I'll re-archive and update the file on the server.
You're right about the multiple intersections I indeed sort them by distance. But I forgot to return only the first two elements (needed for entering and exiting a shape for refraction). I'll fix that.
Ugh - I used t as a var? I should have known better than that. Must be remnants of my terse C coding style :)
> But I forgot to return only the first two elements (needed for entering and exiting a shape for refraction).
Strange; I would have thought you needed only one intersection, because my mental model for refraction would be:
|
-----+-------
\
\ <-----new ray
--------+----
|
|
viewer
So you really need only one intersection - the nearest, because the second intersection wouldn't really be aligned to the refraction ray. But then I haven't read the book you are reading.
Basically at each intersection point I'd expect to split into three rays: a reflection ray (cross product to the normal), a refraction ray (if at least partially transparent) and a shadow ray (towards any source(s) of light).
That's pretty much what my idea was in the "has-a vs. is-a" foolery ^^. Basically interface==type class; I just tended to avoid "type class" because the classic "class type" sounds too near it.
1) Hmm. Currently I've been experimenting around with Arc's type system. The type system is "not bad" but there's a definite problem: making a user-defined type quack like a built-in type is very difficult. My "Create your own collection" series is probably of interest in this research.
If however you are using only user types (not masquerading existing types), then nex3's defm macro is your friend: http://arclanguage.com/item?id=4644
2) LOL. However, if you need just the "then" branch, or just the "else" branch, you can use the 'when and 'unless macros:
(from "arc.arc")
[mac] (when test . body)
When `test' is true, do `body'.
See also [[unless]] [[if]] [[awhen]]
nil
arc> (help unless)
(from "arc.arc")
[mac] (unless test . body)
When `test' is not true, do `body'.
See also [[when]] [[if]] [[no]]
nil
4) Definite problem. In fact, pg's own code doesn't seem very modularized: bits and pieces of the various parts of the webserver are in srv.arc, html.arc, and app.arc. Fortunately Arkani's 'help command also includes which file the function is defined in, so at least you can track files a little (although in practice I often find myself looking for definitions using grep).
5) Arkani has a "CONVENTIONS" file for some conventions. However it's incomplete IMO, so probably we need to fill this in somewhat.
7) True, true. Sure, Arc code is fun to write, until it encounters a problem. Then it's hard to debug/optimize/whatever
To discover the line of the error the interpreter should provide a way to associate every form or symbol passed to the macro with its line number. As an example there could be a function such as (line-num form-with-an-error) to get the line number.
My "attachments" idea was conceived with this potential use too - jus attach the line number to the object, which acts just like the original object (i.e. matches with 'is, etc.).
I don't think it's 'delay-parser, per se, just the interrecursion: 'formatting calls 'bolded-text calls 'formatting. I think this is what's making the profiler give weird numbers.
I am having trouble to replicating your problem with mutual recursion. I get the expected result with this mutually recursive test. Can you give me an example that reports incorrectly?
(def foo (li)
(if li (do (sleep 0.25) (bar (cdr li)) (sleep 0.25))
(sleep 2)))
(def bar (li)
(if li (do (sleep 0.25) (foo (cdr li)) (sleep 0.25))
(sleep 1)))
(profile foo bar)
(foo '(a a a a a a a))
(profiler-report)
Hmm. I'll check again later. In the meantime, what I did in the great-grandparent post was just this, on a fresh start:
1. turn on profiling in wiki-arc/wiki-arc.arc, by setting the variable whose name escapes me at the moment to t
2. Load wiki-arc/wiki-arc.arc and (thread (asv)) the server.
3. Load a single test page, with various formatting, including '''a bolded text's attempt to ''confuse the parser''''', [[link]]s, [[sample link|links with different text]]s etc. etc.
4. (profiler-report)
I'll check later on it, and possibly also post the sample wikitext.
nice ^^. I'll be trying to hack this into Arki's parser and see how things go.
Another parsing approach I've seen floating around is to use continuations - have a "pass" continuation and a "fail" continuation. It would still approximately be using combinators, but basic parsers would look more like this:
Not sure; might actually be faster, since supposedly scheme is much more continuation-friendly than lazy-evaluation-friendly. Or not, I'm not sure
Anyway I haven't managed to wrap my head around how to implement 'seq in a continuation style; also, there's the problem of how 'cut on a parser would work in a nonlazy language - supposedly as soon as the parser given to 'cut consumes anything, 'cut will somehow dispose the 'fail continuation. I'd assume 'cut will be nearer to cut-seq, which would approximately be a cut:seq -
(def cut-seq (parser . rest)
(let next (seq rest)
(fn (remaining pass fail)
(parser
; since the name "remaining" is reused here,
; a smart compiler should be able to
; dispose of the outer function's remaining
(fn (parsed remaining actions)
(next remaining
(fn (parsed2 remaining actions2)
(pass (+ parsed parsed2)
remaining
(+ actions actions2)))
[err "parse error!"]]))
fail))))
Since fail is disposed upon continuing to next (which is the rest of the sequence) everything which fail holds could then be reclaimed. 'alt's pass is its own pass, but its 'fail contains a closure.
If we assume that the base compiler isn't very optimizing (i.e. captures closures when it doesn't need to) we could use helper functions for new 'pass functions:
I've just built a continuation-approach parser combinator library, pushed on the git. It's currently faster on 'many and 'seq, but slower on 'alt. It's considerably faster on Arki's use case, however (about 33% less time).
That said I think the main issues slowing down the monadic treeparse are:
1. Use of 'return. I'm not sure if mzscheme can optimize this away (it's just being used as a multiple-value return, and is immediately destructured by the calling function). Heck, I'm not even sure Arc destructuring is in a form optimizable by Scheme. Using continuations, this tuple becomes the parameters of the call to the continuation, and that I'm somewhat convinced that Scheme (all Schemes) are capable of optimizing. Also, by avoiding explicit destructuring, we can be reasonably sure that Arc gives the parameter list plain to Scheme (not that I've actually studied ac.scm much).
2. Copying of 'parsed. Unfortunately, we have to copy 'parsed, because of 'actions! This is because 'sem attaches the 'parsed part to the 'actions field (via closure), and so we can't reuse this 'parsed part (alternatively we could just copy 'parsed inside 'sem, instead of copying on 'many and 'seq). IMO: drop 'actions ^^. Copying 'parsed, I suspect, may be the single most time-consuming part of the monadic treeparse.
3. Using a 'tconc cell, while neater, is less efficient (about 20% IIRC) than explicit and separate head+tail variables. My goal in optimizing treeparse was to maintain legibility while adding speed; my goal in cps_treeparse was to optimize, legibility be damned^H^H^H^H^H^H^H consigned to hell.
4. There's a few more bits and pieces that may be optimizable - for example, most of my 'filt usage is (filt [map something _] parser). If we are sure that 'parsed will not need to be reused (assured by dropping 'actions, or giving the responsibility of copying 'parsed to 'sem) then we can define a (filt-map something parser) which returns the same list as 'parser, but with their car's transformed by 'something:
Using filt-map would reduce consing, and especially in the cps case would remove the scan through 'parsed to find the 'parsedtl.
Further optimizations abound: another common idiom I found in my wiki parser was (filt nilfn (seq-str "something")), which basically ignores the 'parsed return value of 'seq, It may be possible to create a variant of 'seq, 'seq-nil, which wouldn't bother catenating the 'parsed return values of subparsers.
So no, I don't think continuation-passing per se causes most of the speed difference, except possibly for eliminating the 'return structure.
Another possible issue is the use of a list-like structure. 'scanner-string also conses on each 'cdr, and the total memory usage actually exceeds (coerce "string" 'cons) - it's just that scanners are sexier ^^.
For this case it may be possible to create a variant of the CPS style which takes a tuple of (string, index, end) - as explicit and separate variables of course, so that underlying scheme has a shot at optimizing it away - and use indexing, which I suspect is faster on strings (and which scanner-string ends up doing anyway). This would remove the need for consing too.
