I'm probably at the limit of my ideas :) I don't want to overstate how confident I feel about them. So far the best way I can think of to prevent module overuse from making codebases harder to understand is: don't support namespaces.
But I hadn't considered inexact alternatives, and I still don't understand what that entails. But it might well work better than dropping namespaces entirely. So my previous comment was in the vein of vague encouragement. Just throwing out ideas as they occur to me.
---
"What if the only way to share a program were by example?"
There might well be something here! We'd need a system that can integrate multiple scenarios into a single function. But I have no idea how to build such a system, and it seems far harder than what I'm trying to build. Then again, maybe that will change as we talk about it.
---
"What would distinguish program notation from scenario notation?"
I don't know yet. I suspect every program would need to invent its own notations mimicking its target domain. A browser may need notations like "when click(...) ...", while a server might need a notation like "when received(...) ...". Maybe those two can be integrated.
The general template for a scenario seems to be when X then Y. X would include events and fakes to insert into the system at various points, either carefully orchestrated or randomized like in Quickcheck. Y would include assertions of constraints of various kinds, either on the output, or on the trace generated by the system, or on invariants at various static points in the code. That's what I have so far. Can you think of anything else?
"But I hadn't considered inexact alternatives, and I still don't understand what that entails. But it might well work better than dropping namespaces entirely."
What's your goal with dropping namespaces? You keep bringing up namespaces as though they're what I'm talking about, but I don't even know what kind of namespaces you have in mind. :)
I am indeed interested in dropping certain approaches almost entirely in favor of inexact alternatives, but until I understand what those alternatives are, I can't claim it'll work very well. :-p
One example of an inexact encapsulation technique is physical separation: Two people build Lego structures on separate corners of a table, and sometimes they may construct bridges linking the two, but they can return to working on separate parts at any time. (Unfortunately, this is still an example of making things.)
There are probably lessons in architecture, improvisational theatre, sociology, and a lot of other fields where the boundaries are soft. Just because the boundaries are soft doesn't mean they can't go fast enough to compete with the automation power of logical precision... or does it?
One thing that might help in the meantime is to think not about making things but about augmenting ourselves. This way we may still be dealing with discrete this-person/that-person divisions, but at least we aren't dealing with a discrete creator/creation divide on top of that.
(David Barbour has been talking about encouraging the self-augmenting task of programming the programmer-UI, but I'm not sure I remember his motivation. Maybe I'm getting this from him.)
---
"The general template for a scenario seems to be when X then Y . [...] That's what I have so far. Can you think of anything else?"
That actually sounds a lot like the Era module system. Not in the idiosyncratic details, but in the fact that we're both managing top-level sets of things that do Y when hedged under some modality X. I'm thinking of them as programs with imports, and you're thinking of them as testing scenarios with initial conditions. Perhaps scenarios are a special case of programs?
I'd say "perhaps they're complements," except that it's hard to be specific about any concept without it turning into a program at some point.
This might be why I'm interested in understanding programs as a special case of scenarios, if that's possible.
Sorry, no offense intended. I think I'm ascribing to you something closer to the mainstream view than my own, so you're a stand in for the world :) Not entirely without reason; you haven't said anything in this thread but at other times you've wanted to define your modules in such a way that they'd continue to work regardless of what surrounding modules contain. I'm not sure how far you've moved away from such ideas, and I don't mean to attack you, so where I imply 'you', please assume 'everyone but me'. :)
My radical approach is: "let the caller hack my library to get it to work." Have you been converted quite that far? :)
Er, sorry, I think I must talk in a decisive way that makes me look offended. I'm so indecisive in general, I have to take what I can get. :)
I don't know what would have offended me, so don't worry about it.
---
"I think I'm ascribing to you something closer to the mainstream view than my own, so you're a stand in for the world :)"
As someone who has spent this thread denouncing the evils of truth and making things, when I hear you calling me mainstream, it's heartwarming. :)
---
"Not entirely without reason; you haven't said anything in this thread but at other times you've wanted to define your modules in such a way that they'd continue to work regardless of what surrounding modules contain."
Yes, that's something I still consider an essential feature of a module system. And I think module systems themselves are a matter of course if we have a certain culture around collecting discrete snapshots (research).
But I'm interested in how much we can avoid discrete snapshots in the first place, as well as what different cultural attitudes we could adopt.
---
"My radical approach is: "let the caller hack my library to get it to work." Have you been converted quite that far? :) "
That depends on what context we're discussing this in.
- In the here and now, I'm taking this day to day. I don't have a strong opinion on how much I should indulge my fondness of making things, what I should make or procure for whom, and how much I should work closely with whom. Nor do I have a strong opinion on what you should do. :-p Or maybe it's more accurate to say that I might have strong opinions if presented with more specific situations.
- For thousands of years down the line, I hope we get to "let the caller communicate with me if they want to, under the ethical and logistical supervision of our local peers." There's rarely a "library" involved, and if there is, it's the one who's saying "me" here.
- In the projects I've been working on and pondering, I already have a (very incomplete) plan to allow Era modules to break each other's encapsulation. The idea is that one module should have special privileges with another module as long as it can prove it already knows that module's implementation code or has that module author's permission. A user must have the implementation code of a module in order to install it in the first place, so they can of course hack on it and break compatibility and find themselves rewriting a whole chain of depenencies to use their customization, but this way they can sometimes just add a new module that takes advantage of having extra insider knowledge about the original.
In fact, now that I think about it, this "extra insider knowledge" access is similar to the extra information you're trying to access from your scenarios concept. Hey, we might be able to design a formal system together. :) Did you ever get around to looking at linear type systems?
EDIT to add: I was encouraged to think about this "extra insider knowledge" plan thanks to a recent LtU thread about "private" and "public" access controls: http://lambda-the-ultimate.org/node/4965. The Era module system has only one top-level definition per module, so a private definition doesn't make sense unless there's some way to break into it. Before that thread, I was already worried about what it would take to allow an author to prove extra things about the other definitions they had already published, but that's where I started thinking about a solution.
"Did you ever get around to looking at linear type systems?"
I was (and still am) waiting for you to teach them to me :p
My recent explorations are hardening my bias for operational/ugly/imperative approaches over declarative/elegant/functional ones. The trouble with being elegant is that the implementation is harder to tinker with without causing subtle issues. Later readers have to understand global properties at a deep level.
Don't get me wrong, there's room for high-level languages in my world-view. They're just harder to communicate the big picture of, and I want to start with simpler problems. High-level languages and type systems are hard crystals, and I'm currently more interested in building squishy cells.
"I'm currently more interested in building squishy cells."
I respect that, but just to get it written down, here's what I've come up with so far:
All lambda expressions must be lambda-lifted so they're at the top level of some module. (I guess this might count as a lambda-free logical framework.)
Every function type (X -> Y) is annotated as (X -> [C] Y) with the implementation code C of that function's source module. A simple export/import of an X-to-Y function will actually bind an import variable of the type (existscode C. (SatisfiesSignature S C * (X -> [C] Y))), where S is the import signature.
If a module has access to installed implementation code C and it knows SatisfiesSignature S C and SatisfiesSignature S C', then it can conclude C = C'. It can take advantage of this knowledge to get access to fuller knowledge about how an (X -> [C'] Y) function behaves.
---
"I was (and still am) waiting for you to teach them to me :p"
Really? I don't remember where we left off, but let's get in touch on this.
