I'm assuming the challenge was something like this:

  # library1
  (import library3-version1)
  
  # library2
  (import library3-version2)
  
  # application
  (import library1)
  (import library2)

---

rocketnia: "Regarding "they'd like the new library to be able to occupy both orignal spaces," your code is again incomplete. If the two originals are library1 and library2, an intermediate library library3 depends on library1, and the new one is library4, then how do you get library3 to start using library4 instead of library1? Keep in mind you can't modify any of the libraries."

The challenge was the following:

  # library1
  (def foo ...)
  
  # library2
  (def bar ...)
  
  # library3 = library1 + library2
  (def foo ...)
  (def bar ...)


  # application1
  (import library1)
  
  # application2
  (import library2)
  
  # application3
  (import library3)

There's no need to change application1 and application2: they can just keep using library1 and library2. You might say that's inefficient and you'd rather them use library3, but that's up to them whether they want to switch or not. I'm also imagining the situation that library3 actually does cause some backwards compat issues, in which case it would be desirable to keep using library1/library2. The application chooses which libraries to import.

You seem to be suggesting some mechanism in which a module can actually modify what a different module imports and exports. I fail to see the benefit in that: that module's code is written under the assumption that its imports won't be changing. Could you provide an example where that would be useful?

---

rocketnia: "Regarding the licensing example with "they'd like it to be a drop-in replacement [...] An application I'm working on uses two libraries, one of which requires the replacement library [...] The other [...] uses the original basically by default": I think they're asking for a mechanism that lets the application choose whether the "other" library uses the original or the replacement. I don't see how your code makes this possible, and your "this could instead import replacement" comment suggests that you'd actually modify library2."

No I do not think that's what they're asking. I think what they're asking is: can the original library coexist at the same time as the replacement, even though they have name collisions. Notice they used the word "reconcile". This suggests reconciling the name collisions. There's no need to "reconcile" the other library to use the replacement: it can just keep using the original.

But if I'm wrong and they were in fact not talking about name collisions at all but about the orthogonal issue of replacing another module's imports... well, no, Nulan doesn't let you do that, because I don't see any useful benefit from doing so.

In any case, that's a completely different issue from avoiding name collisions, and seems to me like it'd be a part of some sort of build system or dependency resolution system, like RubyGems. That might be useful, but I don't see it as being a part of the namespace system per se.

---

rocketnia: "Regarding "conrol the names you import [...] such that [...] a new export to an existing library will not affect the meaning," it looks like you do have a good response to this. Personally, I wouldn't want there to be any imports that could possibly break that way, but I guess I'm more picky than they are. :)"

Yup, I try to be pretty laid back (at least about my code). If you're anal about it, you can use "include", but I think it's nice to have the option of just doing unqualified imports. It's up to the library/application author, in my opinion.

---

rocketnia: "You missed my point. If the two libraries really use the same namespace, your code for importing both of them will turn into something like this:"

Nope, I don't think namespace == module. You are importing the same module twice, but it's in the same namespace. And if you did this:

  (import library1)
  (import library2)

You might say this is undesirable. I agree it can be undesirable in certain situations, and I think the solution is to import each library in a separate namespace. But hyper-static scope can be implemented with a single namespace, so whether it's single-namespace or multi-namespace is an orthogonal issue.

---

rocketnia: "I often consider what would happen if two versions of a single library were both useful in a single application. If the library author didn't anticipate and avoid self-conflict, the two versions would host a train wreck of collisons on every single name, even the library name. I think that's the kind of challenge Casey McMann has in mind here."

In that case whichever one is defined last "wins", and you can use "include" and "exclude" to control which ones you want. I could also provide a way to import a library into a separate namespace to solve that case:

  (import-as library1 lib1)
  (import-as library2 lib2)

  (lib1 'bar)
  (lib2 'bar)

I'm assuming the example is supposed to look like this instead:

  ; library1 (coded with library3's version 1 in mind)
  (import library3)
  
  ; library2 (coded with library3's version 2 in mind)
  (import library3)

---

"You seem to be suggesting some mechanism in which a module can actually modify what a different module imports and exports. I fail to see the benefit in that: that module's code is written under the assumption that its imports won't be changing. Could you provide an example where that would be useful?"

Here's Casey McCann's full description of the challenge again, just for clarity's sake:

Casey McCann: "At some point, two popular libraries with related functionality get tired of duplicating effort, so they collaborate to produce a third library providing the best of the shared features. To avoid unnecessarily breaking client code with the new versions, they'd like the new library to be able to occupy both original namespaces, or otherwise be easily substituted in. How would the namespace system handle this?"

I had it wrong before, when I made the sweeping statement that only the application's code should have to change in these challenges. This challenge is a case where the library developers are trying to change their code without causing disruption for anyone else.

Ideally, the only thing an application developer should have to do to upgrade is to replace the existing two libraries with the new one. This is even something that an end user might handle using a package manager.

---

"I think what they're asking is: can the original library coexist at the same time as the replacement, even though they have name collisions. Notice they used the word "reconcile". This suggests reconciling the name collisions. There's no need to "reconcile" the other library to use the replacement: it can just keep using the original."

I think the point is that the library has been upgraded, and the library developer would like the upgrade to propagate to all code that uses the library, even though a few library users might actually prefer otherwise. This is a conflict of preferences, and the language might give tools to certain participants to "reconcile" this conflict.

I prefer to view this as a last resort. The module system should give a complete(-feeling) range of freedom to every participant, and the ranges should not overlap. I would allow upstream developers to propagate their changes only as far as downstream developers have opted in. This turns into a design for package versioning.

---

"Nope, I don't think namespace == module."

I agree. The challenges are supposed to be realistic cases where name collisions would actually occur and cause havoc, so naturally they mostly involve programs made up of parts that have been written by multiple people independently. So we should expect to consider modules here, but yeah, the modules don't have to correspond one-to-one with namespaces.

However, I think Casey McCann is assuming a namespace system built on implicit name prefixes, which may or may not incorporate a way to take a name prefix and find a module that addresses it. While this may be a fine way to think of namespaces Java, Racket, and Haskell, the challenges which arise from this assumption may not apply to Nulan. No worries. :)

That said, if you change Nulan's design in order to accommodate some of these other challenges, you may come back to this one and find you've broken it. That is to say, this is still a meaningful test case even if it passes at the moment.

---

"My point is that even if a language doesn't provide include/exclude/import-as, it'd still be better off using hyper-static rather than nothing at all."

Well, that either won't work (it would load the same "library3" in both cases), or it would work with a sufficiently smart lookup algorithm.

Basically, when importing a file, I think it should look for it in this order: the cwd, the folder (and subfolders) where the importee is defined, then lastly the folder where Nulan itself is defined.

In that case, assuming library1 and library2 are in different folders and have different versions of library3 in their path, it would work.

This implies bundling the libraries you use with your app, which you pretty much have to do anyways.

In that case, this has nothing to do with name resolution, it has to do with file resolution: finding which file to import when given a name.

You could argue that's a good case for a package manager like RubyGem, and that may be true, but I consider such a system to be in a layer above the namespace system.

---

"This challenge is a case where the library developers are trying to change their code without causing disruption for anyone else."

And Nulan's namespace system handles that just fine.

---

"I think the point is that the library has been upgraded, and the library developer would like the upgrade to propagate to all code that uses the library, even though a few library users might actually prefer otherwise. This is a conflict of preferences, and the language might give tools to certain participants to "reconcile" this conflict."

I still don't get the impression they were talking about some sort of package management or version dependency resolution system. It's certainly okay to discuss such things, but I think they're outside the scope of a namespace system. That thread (and post) seemed to me to be talking specifically about resolving naming conflicts.

I think the talk about "multiple libraries" and "multiple versions" was just to create a situation where naming conflicts occur. I don't think it was meant to imply that the goal is to be able to easily change library versions on the fly, like you were discussing.

If I wanted some sort of system like that in Nulan, it wouldn't be particularly hard to do... you'd create a new file, like say... "dependencies.nu" and you'd then put a bunch of imports into it:

  (import foo)
  (import bar)
  (import qux)

---

"However, I think Casey McCann is assuming a namespace system built on implicit name prefixes, which may or may not incorporate a way to take a name prefix and find a module that addresses it."

That is quite possible.

---

"By "nothing at all," do you mean a late-bound global scope like Arc's?"

You: "And Nulan's namespace system handles that just fine."

What I meant was that the library developer wants our applications to use their new library version without any involvement of the application developers. I seriously doubt that's what you're claiming to support.

---

"You could argue that's a good case for a package manager like RubyGem, and that may be true, but I consider such a system to be in a layer above the namespace system."

I think the only place we really disagree now is that you're content to neglect the "RubyGem" parts of these challenges for the purposes this discussion, while I consider them the hardest parts. I think if you do try to design a package manager, you may very well get most of a namespace system out of the deal, or even a whole programming language.

Unfortunately, package managers and filesystems aren't quite a full layer above the languages they're used with. Program units tend to link together using hardcoded filenames or package names, a practice that ties some of the filesystem or package manager complexity into the complexity of the language proper.

The module system I've been working on recently (ever since [1]) is meant to help resolve that, by making the non-language-specific complexity as simple and universal as possible. It's based on two notions: A program is a multiset of independently maintained extensions, and it exists in the context of an expansive historical record containing all knowledge ever discovered (or at least all knowledge we have on hand).

The extensions and units of background knowledge may ultimately be encoded as files and managed at a "layer above" using traditional package managers and filesystems. However, they may also be manipulated as first-class values in a host program, or even as visual elements in a UI.

As an argument in favor of this viewpoint, I'll quote the original challenge:

"Now I'm writing a piece of code that uses two libraries, each of which use different, incompatible versions of some third library, causing namespace collisions between the two versions. How does the namespace system handle this?"

The challenge seems to imply that the two incompatible libraries were loaded successfully: it's NOT talking about file resolution. It's assuming the file resolution proceeded successfully, and thus the only issue remaining is resolving the name collisions.

From what you quoted, technically that could be true.

---

"You keep talking about things like dependency resolution systems, swapping in different libraries, etc. That's fine, but that does not seem to be what the challenges are talking about."

Objection! :)

I don't get the impression they're saying, "we release this new library and all the users of the existing libraries automatically switch to it". I think what they're saying is, "this library can be used by existing applications, but they still need to choose to use it". That's how I interpreted the "drop-in replacement" phrase.

Technically speaking Nulan can support that... you would change library1 and library2 to just import library3:

  # library1
  (import library3)
  
  # library2
  (import library3)
  
  # library3
  (def foo ...)
  (def bar ...)

---

"I think the only place we really disagree now is that you're content to neglect the "RubyGem" parts of these challenges for the purposes this discussion, while I consider them the hardest parts. I think if you do try to design a package manager, you may very well get most of a namespace system out of the deal, or even a whole programming language."

They are the hardest parts, I just don't think it has to do with namespace support, which that entire topic was devoted to. Nor am I particularly interested in designing a package manager.

So my suggestion still stands: if you feel this strongly about package managers, make up your own semi-contrived challenges that try to demonstrate what a package manager should be capable of doing. I'm sure it will include all kinds of stuff that was left out of the namespace challenges, because package managers are much more complex and involved than namespaces.

Then you can point to those challenges and I can say, "Nulan doesn't support that stuff" rather than trying to insist that Nulan is failing the namespace challenges because you're interpreting them far larger than what they actually said. I think it's useful to have "package manager challenges", but I think it's also useful to have "namespace challenges". I don't see why we need to throw out the namespace challenges just because you view it as "not as hard" as package managers.

---

"The challenge we're talking about puts the words "different, incompatible versions" in italics. If direct substitution were unimportant, the challenge would not even need to use the word "version"; it could just say there were two separate libraries with name collisions. And yet that's precisely the scenario of the previous challenge."

Like I said, I don't get the impression they're talking about package managers. I think they used the phrase "different incompatible versions" because that's a situation that can actually occur in real life. They're using it as an example, that's all.

Oh, I can understand that point of view. Just because there were multiple examples doesn't mean each one introduced new requirements.

---

"It's obvious that you want to think in terms of package managers, and that's fine, but I really do not get the impression that's what they're talking about."

I just wanted to help you elaborate on the claims you're making about Nulan's namespace system. You originally quoted Casey McCann's examples, but I perceived them differently than you did, and I had trouble understanding your perception from your terse example code.

Now this seems to be resolved. We may not agree in our interpretations, but now I understand your claims relative to both of them. Thanks for your patience with me. ^_^

---

"Ultimately, it seems the only way to resolve this is to actually ask them what they meant."