One of the things I've liked about Arc so far is the tiny number of types: characters, strings, symbols, numbers, lists, hashes, functions and macros. Having a very small number of types means that every function can be written to do something useful on each type. For example, I just wrote a library for converting any Arc value into its JSON equivalent so I could send them to a web client. It was really easy, because I had so few types to deal with. No need for polymorphic whatnot and abstract do-daas. Just use a simple case statement and you're there.

Elaborate type systems can solve a lot of problems. For example, I had a problem with alists, because I wanted them to be converted into objects, but general lists needed to be converted to arrays. How do you tell the difference between an alist and a normal list? If you had some kind of type system that allowed you to declare that your list is an alist then that would solve it. This is how complex type systems get started, but it's all downhill from there. Once your language starts relying on its type system for its power, you begin taking huge risks.

The problem is that when complex type systems fail, they fail hard. Despite endless academic papers on the subject, and a plethora of practical programming languages, no one has yet developed a type system that is powerful enough to allow people to express everything they want in a reasonably complex program. That wouldn't be a problem if the type system got you 99% of the way there and you had to do a small workaround for the last 1%. Unfortunately, what usually happens is that the 1% left over breaks the whole program. Suddenly you can't use that really useful parsing library, because it's defined over an abstract type that your type can't quite be mapped into. Suddenly the polymorphism features of your language become useless because your type isn't quite polymorphic in the way the language designers anticipated (perhaps you want function polymorphism but the language supports object polymorphism, or vice versa). All type systems fail eventually, and if your language relies on its type system for its power, then you might as well be writing machine code for all the use it will be.

Let's take just two examples. OOP started with multiple inheritance, but then people decided this was too error prone, so people moved so single inheritance. But that wasn't expressive enough, so we got interface inheritance, then generics, then mixins, then "composition over inheritance". Despite all this, you still can't express the relationship between circles and ellipses in any OO language.

Example 2: Haskell was inspired by the very powerful Hindley-Milner type system, which has type variables, polymorphic types and abstract types, but this still wasn't good enough, so they added classes. The ML people still didn't think this was good enough, so they added functors. Some people still didn't think this was good enough, so they proposed higher-level functors. There are endless proposals to add generics to the language too, and then there's dependent typing and all that malarkey. No matter how powerful your type system is, it's never enough.

Arc doesn't get its power from having a complex type system. It gets it from being able to express computations on a small number of types in a very concise manner. This kind of power can fail too, but it doesn't fail hard because Arc has the tools to make any problem simple: functions, macros and reprogramming the programming language. Type systems can save you writing lots of similar functions for slightly different types, but so can macros because they can generate the code for you. Type systems can distinguish between different implementations of cdr and car for different types, but you can also do that by redefining cdr and car to cope with those types, or by passing your own versions of cdr and car along with the object (perhaps even stored in the object itself: see the appendix on object systems in ANSI Common Lisp).

Complex type systems give you power, but they are a seduction that we should resist. They allow you to solve problems by constructing interesting types, but the set of problems you can solve is always strictly limited by the type system (and just because you can write your own type system doesn't mean the problem goes away, unless you can get your system to work seamlessly with everyone else's system).

A powerful set of functions over a small number of types is a much better way of doing this. Types are the sockets that allow functions to be connected together. You can either work with a small number of types that allow you to connect all your functions to each other, or you can create all kinds of different types and then use polymorphism/inheritance/type variables/has-a relationships to try and patch up the fragmentation problem. If you really have lots of types that are similar enough for your functions to work across them, then you should probably use fewer types.

You state that there should only be the few basic types currently defined in Arc. Paul's idea was to eventually get rid of strings (they are a special kind of list) and even numbers (they are a special kind of list too...). But he finally didn't, and won't, at least for numbers. He also said that this view (as few basic types as possible) finally forced him to develop a basic type system (with annotate, type, rep and isa) to distinguish between the raw list '(a a a a a) as the number 5, as the string "aaaaa" or as an actual list of 5 symbols.

In a way or another, you need explicit types if you want some kind of dynamic typing. Assembly language work the opposite way : e.g. you state (explictly or not) the arg of your function is a number. If the user gave you what he considers a string, too bad for him, because you can't distinguish between them. That means your function can't be polymorphic and you are stuck in an even more contrived space than with user types.

You need an isa function (call it isa or hasa, never mind as for now). For example, car should have a list, and nothing else. To do so, you have to check its type. If you want to redefine car so as to take scanners, generators, ... into consideration, that's easy too : just define your own version of car : if arg is a list, call the original car, else arg is a generator à la Python, so funcall it :

  (let _ car car
    (def car (x)
      (if (isa x 'list)
        (_car x)
        ((x)))))

  (let _car car
    (def car (x)
      (if (isa x 'list)
          (_car x)
        (isa x 'generator)
          ((rep x))
        (isa x 'array)
          (a-get x 1)
          (err "Not a valid type for car : " type.x))))

That means you can do something this way, for example :

  (annotate (listtab `((car . ,(fn (self) (self 'car))) (cdr . ,(fn (self) (self 'cdr))) x)

  (let _car car
    (def car (x)
      (if (isa x 'list)
        (_car x)
        (let tx type.x
          (tx!car rep.x)))))

  (= _car car _cdr cdr)

  (def hasa (obj typ)
    (if (no typ)
      t
      (and (type.obj (car typ)) (hasa obj (cdr typ)))))

  (def car (x)
    (if (acons x)
        (_car x)
      (hasa x scanner)
        (let tx type.x
          (annotate tx (tx!car rep.x)))
        (err "bad object for car")))

  ; define scanner type-class
  (= scanner '(car cdr))

  (= mycar [_ 0] mycdr [cut _ 1])
  ; Say s can answer to car and cdr
  (= fns (listtab `((car ,mycar) (cdr ,mycdr))))
  (= s (annotate fns "abcde"))
  (car s)
  -> #3(tagged #hash((cdr . #<procedure: mycdr>) (car . #<procedure: mycar>)) #\a)
  (car rep.s)
  -> #\a

Well someone has to load-test this thing!

> And it is even a very clever one, actually.

Thanks :)

> having to use rep on annotated data is, I think, the biggest mistake of that type system.

This is exactly this problem that got me thinking about types. I've been tempted to create a few types with annotate already, but each time I stopped because I didn't want to have to reimplement every function to work on my new type. Each time, I found a different solution to the problem that didn't require new types, which started me thinking "hey, perhaps we don't need new types after all!"

But you're right that you'll always need new types eventually. The solution you suggested is, I think, the right one. It's a bit like the object system in ANSI Common Lisp (pg even used hash tables to store the object's methods!) but it uses annotate to associate methods with objects.

  (redef hasa (obj typ)
    (if (and (isa obj 'cons) (is typ scanner))
      t))

In such a case a has-a semantics means that circle "has-a" function to compute its area, a function to compute its circumference, etc. An ellipse "has-a" function to compute its area, a function to compute its circumference, etc. It doesn't matter whether the user thinks of circles as special cases of ellipses or not.

Perhaps the way to go would be to support interfaces without requiring type checking. Basically you simply say "all I care about is that this object can be passed to that function".

That said a semiformal way of expressing this - probably by giving a name to an interface (which is just a set of function symbols that an object supports) - might be useful. This way wouldn't necessarily be checked by the program - it might be useful to have it be read by the programmers as part of the code/documentation.

  (typeclass 'scanner 'car 'cdr)
  ; programmer reads: a scanner is anything that somehow supports 'car and 'cdr

  (def foo (a)
    " A ridiculously complex library function which does
      a lot of useful things and which the library user
      probably doesn't want to read in full, because he or
      she is using the library so that he or she doesn't
      have to think about it.
      See also [[bar]] "
    (must-have a 'scanner)
    ; programmer reads:
    ; "Anything I define which supports 'car and
    ; 'cdr can work on this function"
    (ridiculously-complex-expression-involving a))

The kind of thing you're suggesting here does look powerful (and most importantly, optional!) If you combine it with sacado's suggestion for putting the methods in the tag, then you have a very powerful type system indeed.

I just have one contentious thing left to say: when you move away from is-a typing to has-a typing, does it really make sense to use the word 'type' at all? Aren't we really talking about what your functions can do, rather that what your objects are? For example, if you define car and cdr to work on strings, have you added strings to a new sequence type or have you expanded the power of your functions? I prefer to think that you've done the latter, and save the word 'type' for the basic is-a types that every language has to have.

a) Lots of operations operate on "sequences" (tables, lists, and string). Lots of other operations operate only on lists, or only on string and tables. There doesn't seem to be any real pattern. There also doesn't seem to be any fundamental "sequence" abstraction that supports implementation of more complex operations. It's more like "if list, do this. If string, do that. If table, do that."

b) Lots of string operations treat a list of characters and a string interchangeably. Lots of operations don't. Again, it seems pretty random.

c) Non-fundamental types, such as queues (http://www.arcfn.com/doc/queue.html) don't interoperate at all. That is, a sequence operation such as map will do entirely the wrong thing to a queue.

As for is-a vs. has-a, I was wondering, isn't the biggest mistake considering that a piece of data only has one type ? That is, isa & type are broken ?

Considering values have a list of types (instead of a single type) would fix a few problems. For example, what is the type of nil ? Is it a symbol or a list ? Yes, it is both, but to do so, pg had to define the alist function ! (type nil) returns sym and cannot let you know nil is a list. What is '(1 2 3) ? Is it a cons or a list ? It is a list, but its type is only cons. type should return a list (it is already possible through annotate) and the definition of isa should be :

  (def isa (x typ)
    (mem typ (type x))

  (type nil) -> '(list sym)
  (type '(1 2 3)) -> '(list cons)
  (type '(1 . 2)) -> '(cons)

  (type s) -> '(cons scanner)
  (isa cons s) -> t

  (defm something ((t f scanner))
    (do-something-on-scanner f))
  (defm something ((t f cons))
    (do-something-on-real-cons-cell f))

This is the main reason I'm advocating is-a and has-a separation. We can say that an object is-a 'cons cell and has-a scanner. If something requires that an object is-a real, element-pointer-and-next-pointer 'cons cell, as opposed to somethingthing that requires that an object has-a 'car and 'cdr, we can make the distinction.

So we can say that an object is-a 'cons cell - it's what it really is, what it's implemented with. However, a 'cons cell has-a scanner interface, and if it's proper it has-a list interface, etc.

Separating is-a and has-a could also be useful for optimization of basic parts.

For example, a basic non-optimized diff algorithm might operate on has-a 'scanner, and use 'car and 'cdr operations. However a string-scanner is really just a wrapper around a string and an index into the string, as well as the string's length. Each 'string-scanner object contains three slots: one for the string, one for the index, and one for the length. This applies to each 'cdr on a string-scanner.

Now suppose we have a version of the diff algo which specifically detects if an object is-a string-scanner. It destructures the string-scanner into the string, index, and end, and instead of carrying around a triple of (string, index, length) it only carries the index, leaving string and length into local variables. This reduces memory consumption to only one-third.

- type declaration of real-implementation (what you call "is-a"),

- type declaration in the sense of "capabilities" an object has (what you call has-a).

I think they should really be distinguished. The former is about optimized compilation, the latter about which functions can be applied to a given object.

But optimization is linked to variables (e.g. "in this block n always holds an integer, s always holds a string and l is always a cons) and does not need to be declared until you want to compile something.

On the opposite, capabilities are linked to values (e.g., "n, s and l are all scanners, they all have scanner capabilities, you can apply car and cdr to all of them. This is currently true, but could change if values referenced by n, s or l change). These are mandatory, and have to be known dynamically (this is not a declaration in the static meaning, they can even change later). When you apply car to a variable, you must know if its attached value can answer it (and eventually how).

  (= str (string-scanner "foo bar baz"))
  (type str)
  -> (scanner string)

  (def scan (s)
    (if (no s)
      ""
      (cons (foo (car s)) (cdr s))))

Now suppose we want more. All we have to do is :

  (def scan (s)
    (istype string-scanner s
      (if (no s)
        (cons (foo (car s) (cdr s)))))

  (def my-isa (obj typ)
    (let a type.obj
      (if atom.a
          (is a typ)
          (some [is _ typ] a)))

I tried this, it does work :

  (redef isa (x typ)
    (isa-rec type.x typ))

  (def isa-rec (types typ)
    (if
      (no types) nil
      (is types typ) t
      (isnt type.types 'cons) nil
      (is (car types) typ) t
      (isa-rec (cdr types) typ)))

  (isa nil 'sym) -> t
  (isa (annotate '(sym list) nil) 'sym) -> t
  (isa (annotate '(sym list) nil) 'list) -> t
  (isa (annotate '(sym list) nil) 'cons) -> nil

  (redef car (x)
    (if (isa x 'int)
      (if (> rep.x 0)
        'a
        nil)
      (old x)))

  (redef cdr (x)
    (if (isa x 'int)
      (if (> rep.x 0)
        (annotate type.x (- rep.x 1))
        nil)
      (old x)))

  (car 1)
  -> a
  (cdr 1)
  -> 0
  (cddr 2)
  -> 0
  (car (annotate '(int cons)) 3)
  -> a
  (len (annotate '(int cons)) 3)
  -> 3

  (each c (annotate '(int cons) 3) (prn c))
  -> Error: "Function call on inappropriate object #3(tagged (int cons . nil) 3) (0)"

  ; In arc.arc, or maybe something equivalent in
  ; ac.scm.
  (typeclass 'cons 'car 'cdr 'scar 'scdr 'apply)

  ; In our library, where we can't (or at least,
  ; "shouldn't") redefine the built-in Arc 'cons
  ; typeclass.
  ; A perfect subset of 'cons.  
  (typeclass 'scanner 'car 'cdr)

  ; Dumps out a character scanner or list
  ; to a port
  (def dump-out (port l)
    ; Note: we expect has-a to return t
    ; even if l is a 'cons - i.e. the type
    ; system should be smart enough to
    ; realize that 'cons provides all the
    ; interfaces of 'scanner.  Prolly something
    ; like:
    ; (= all-functions (mappend [instance-functions-of _] (type-classes-of ob)))
    ; (all
    ;   [some _ all-functions]
    ;   (instance-functions-of requested-type-class))
    ; above can be optimized by using tables - put all
    ; of the requested type class's instance functions
    ; in the table, then remove everything that exists in
    ; all-functions; if we finish all-functions while
    ; table entries exist, we know the object doesn't
    ; have that type class
    (unless (has-a l 'scanner)
      (err "Not a scanner"))
    (map [write _ port] l))

  (def dump-out (port l)
    (map [wrie _ port] l))

  (unless (has-a a 'scanner)
    (err "type error"))

  (mac type-declaration (a typ)
    `(unless (has-a ,a ,typ)
       (err "type error")))

  (def dump-out (port l)
    ; a completely optional declaration
    ; for:
    ; (1) an non-optimizing interpreter, so that
    ; you have some chance of checking errors, and
    ; (2) the optimizing compiler, so that it can
    ; do some shortcuts.
    (type-declaration l 'scanner)
    (map [write _ port] l))

You can then trivially redefine type-declaration as:

  (= type-declaration
     (annotate 'mac nilfn))

Oh and yeah: type checking is plenty in-place, if you're building a library. If the library's interface functions simply passed the object to library-internal code without checking, the error will pop up as coming from some code deep in your library, where your user might be less inclined to trace (because it might be your bug, not theirs). At least if you do the check on the interface functions themselves you have proof that the bug is in the user's code.

  This function accepts a list of ordinal types

  (type-declaration a 'Scanner)
  (type-declaration (car a) 'Ordinal)

  { 2 7 1 } 3 add { 6 5 } append

  [ foo bar ] \ baz add [ foobaz barbaz ] append

  4 7 add { "bar" "baz" } append 3 append

I think any object system we put in the language should include the ability to create types that define their own versions of functions like join, and the ability to inherit from other types.

This seems to fit well with the following design philosophy:

Can I? Yes.

Do I have to? No.

  {- merge used for mergesort -}
  merge :: Ord a => [a] -> [a] -> [a]
  {- The "Ord a" above asserts that the type "a"
     should be an "Ord"inal, i.e. it should support
     < <= > >= == methods
  -}
  merge [] [] = []
  merge a []  = a
  merge [] b  = b
  merge a:as b:bs
        | a < b     = a:(merge as b:bs)
        | Otherwise = b:(merge a:as bs)
  {- Of course, I haven't programmed Haskell in a while,
     so the above code might be wrong. -}

  (def merge (a b)
    (unless (and (has-a (car a) 'Ord)
                 (has-a (car b) 'Ord)
                 (is (type a) (type b)) )
      (err "Type error, needs Ord"))
    (if
      (no a)
        b
      (no b)
        a
      ; now we're assured < works
      (< (car a) (car b))
        (cons (car a) (merge (cdr a) b))
        (cons (car b) (merge a (cdr b))) ))

  (type-check (list a b)
     (Ord a) ((a . as) (a . as)))

This is where I assume Arc will end up building its type system, if it ever does embrace a single type philosophy[1]. Even if it doesn't have a message-passing model, its dynamism makes the implicit has-a model a good fit.

The only language I've seen that embraces explicit checks for has-a relationships, as opposed to expressing them as is-a or making them implicit, is Javascript. This is mostly because you're dealing with objects that have an inherently variable and ill-defined interface that you don't know a lot about at compile-time. I think most JS developers view it as an annoying necessity, though, so I'm inclined to believe it's the least pleasant way of dealing with has-a (despite being the most explicit).

  module MyMixin # Like an interface
    ...
  end
  
  class MyClass
    includes MyMixin # Like "implements MyMixin"
    ...
  end
  
  foo = MyClass.new
  if foo.is_a? MyMixin
    puts "is-a"
  else
    puts "has-a"
  end
  # Output: "is-a"

Also, as I mentioned elsewhere (http://www.arclanguage.org/item?id=4864), a properly-designed type system can do amazing things for increasing the conciseness of a language. For example, take Ruby's Enumerable mixin. This relies on an implicit has-a relationship (that the mixed-in class will define an #each method) to automatically make practically every list operation you could ever want - map, fold, zip - work on any type that it would make sense for.

In any case, I agree wholeheartedly: classes, if used properly, are an incredibly good way to streamline code. (Don't use them for everything, though. Then you get Java.) Ruby's mixins are actually very, very convenient, and seem to me to mesh well with Arc's approach: anything that can be cared and cdred can be mapped, too, and Arc doesn't care. But I feel like referring to this style as has-a is slightly weird, and it threw me off for quite a while. An object doesn't have-an Enumerable, it is-an Enumerable. On the other hand, an object does have-an #each method, so it does make some sort of sense.

  arc> (= a (my-listtype 'foo nil t 'bar))
  (foo nil t bar)
  arc> (car a)  ; has-a car
  foo
  arc> (cdr a)  ; has-a cdr
  (nil t bar)
  arc> (map no a)  ; has-a map (?!)
  (nil t nil nil)
  arc> (rev a)  ; has-a rev (?!)
  (bar t nil foo)
  arc> (some no a)  ; has-a some (?!)
  t
  arc> (all no a)  ; has-a all (?!)
  nil

Now suppose we have another type, which has-a 'collide function, which handles the events where it is collided with in the game space. A ship has-a collide function, and the basic collission detection code is something like this:

  (thread
    (while t
      ((afn (game-elements)
         (iflet (first . rest) game-elements
           (each e rest
             (when (overlapping first e)
               (collide first) (collide e))))
       game-elements)))