It would be a mistake to take the atomic out of expansions of =.
Operators that modify things have to be atomic in the stretch between reading the value and writing the modified value.
Suppose x is initially 0 and you have two threads both evaluating (++ x). If the ++s
aren't atomic, you could end up with this sequence:
thread 1 reads the current value of x, 0
thread 2 runs in its entirety, leaving x = 1
thread 1 resumes, setting x to 1 + the 0 read earlier, or 1
You could similarly have two threads evaluating pushes onto the same list that
ended up losing one of the pushed values.
I don't think you should expect to be able to throw control out of an atomic expression-- at least not short of some abort-as-disaster operator. That's the definition of atomicity: it all has to complete.
It would be a mistake to take the atomic out of expansions of =
Right, but it's also a mistake to leave them in. Too much locking is as bad as too little; e.g. (obj a (readfile "foo")) will hang my web server if reading foo happens to take a long time.
Here's my current chain of reasoning around locking...
Arc's approach to programming is exploratory, building larger programs out of small, composable parts.
Locking is a problem with exploratory programming because buggy locking code usually works most of the time, unlike bugs in functional code which are usually visible. With exploratory programming, you try things and see if they work. Sure, with functional code there are the corner cases that you miss and the occasional incorrect algorithm that happens to return the right value for the input you give it, but most of the time with exploratory programming you try things and you get to see that they don't work. But with locking, you throw together some locking code and try it out, and hey, your program runs and doesn't hang and gives the right answer. The bug, if there is one, only bites once in a blue moon when the different threads happen the hit the code in exactly the wrong way.
There's an interesting social aspect to this as well. I've noticed that if I tell someone about a bug in their code, it's less likely to get fixed if it's a threading bug. If their code returns the wrong value for an input, they say "oh my gosh!" and fix it right away :-). But if it's a threading bug, well, yes, it looks like a bug, but the program appears to run ok anyway, so there's little urgency, and how do we know if we've really fixed it or haven't added another threading problem?
The social aspect goes both ways. One of the things I find so delightful about Arc is that because of your work to write concisely, I can look at a function and say "oh, there's a bug". Or, at least that the function isn't doing what I want it to do. Which I can't do with most code, not as easily, because it is surrounded with so much cruft. But I don't have the same feeling of clarity when I look at Arc's locking code. I can read through the code and perhaps pick up on a locking bug or two (e.g. atomic-invoke), but overall, is everything locked that needs to be? Anything locked that shouldn't be? I can't tell. This part of Arc feels like regular software to me... complex enough so that I imagine there are probably bugs, and I don't expect to be able to get them all.
Composibility with locking is a problem too. You have a couple of perfectly good expressions (obj a 1) and (readfile "foo") and you put them together and they break.
My next thought in the chain is, so why use threading anyway? MzScheme only runs on one CPU, so what threading gives us is a) not having to call yield in a long CPU intensive calculation and b) having our program execution randomized for us so that our program doesn't return the same output for the same inputs... unless we very carefully add the right locking in the right places.
So my current inclination is to rewrite the web server to a single threaded event driven model.
Or maybe something like Erlang, where you're not stuck with a single thread, but you're also not trying to deal with sharing modifiable data between threads either.
Hmm, a different way of looking at the issue just occurred to me: pushing atomic inside of expansions of = may make Arc plus News shorter but Arc plus my program longer.
If ++ didn't do locking, then in places where you were using x from multiple threads you'd need to say (atomic (++ x)). Pushing atomic inside ++ makes this shorter because now you can just say (++ x). But pushing atomic into expansions of = also means that locking occurs at other times, and so I can't use it in places where I'm doing things like throw and readfile that break with locking, which makes my program longer.
Which leads to a fascinating idea, if we get a large enough body of open source Arc code that we can start optimizing for code size globally... :)
That would increase the conceptual load of programming in Arc a lot. It would make people have to think about the expansions of operators like ++ to know when to wrap things in atomic and when not to. You need to be able to treat built-in operators like that as black boxes. Once you start thinking about macroexpansions, it's as if you had to write them.
Hmm, well, I can only speak with any knowledge about my own conceptual load... I expect with your background (professor, Lisp book author, tutorial writer, mentor, etc.) you have a much better idea of what other programmers would find easy or difficult.
I know that some things should be atomic, such as accessing shared mutable data structures, and some things that I need to avoid being atomic, such as doing I/O.
I find Arc's making some operations atomic for me doesn't help me all that much, because without knowing the details of the macroexpansion, I don't know if everything that needs to be atomic has been made so. And I find it unhelpful in other cases, when I need an operation to not be atomic, and so I need to look at the macroexpansion of = to find out if that particular expansion is doing something that I need it to not do, or if it's ok.
On the other hand, I have no alternative to offer yet ^_^. I surmise that if I factored Arc + News + my code, perhaps I might come up with a useful suggestion to offer, and if I do, I'll certainly post about it!
