P.S. I'm really, really, really happy that you've narrowed the problem down to a garbage collection issue. Implementing Arc lists with mpair's seemed to fix the queue bug, but I had no proof. I was still possible that I had merely moved data structures around in memory or whatever enough so that the bug simply wasn't manifesting by running the queue test. Now that we know that it's a GC problem, I have much more confidence that the mpair implementation is in fact one way to fix the bug.
Just wondering, I haven't looked into ar that deeply yet:
How invasive is the change to mpairs really? Is it drastic enough you needed to create most of a new runtime, or is that just because you're like the rest of us and want to experiment ^^
http://awwx.ws/mpair0 was my attempt to simply modify Arc 3.1's ac.scm to use mpair's. I got enough working to see that using mpair's appeared to fix the queue bug, but my implementation was otherwise quite buggy. (Lots of places where Racket lists appeared where Arc lists should have been, and vice versa; and weird mixes of the two, etc.) I have no doubt that someone cleverer than me could get it to work, but I just kept getting lost trying to figure out my bugs.
So I decided to implement the compiler incrementally, where I could test each step as I went. It would take a lot more work (a bit like getting my jeep stuck in the swamp, and so hauling myself out ten feet at a time with a winch), but it was an approach that I knew I would be successful at.
Then, since I was rolling through a compiler rewrite anyway, I decided to also go ahead and reflect the compiler into Arc to make it more hackable ^_^.
One of the 3 goals listed in the readme is "to make Arc more hackable," so I would guess that mpairs isn't most of it. Reorganizing ac.scm and adding unit tests so that the internals are easier to modify is probably a significant motivation.
