Arc Forumnew | comments | leaders | submit | lojic's commentslogin
5 points by lojic 5996 days ago | link | parent | on: Arc Challenge

Norvig's original article: http://norvig.com/spell-correct.html

Norvig's article has a large file for training the corrector that you can download, or use your own.

My Ruby translation: http://lojic.com/blog/2008/09/04/how-to-write-a-spelling-cor...

Reference for the Clojure version above: http://en.wikibooks.org/wiki/Clojure_Programming#Norvig.27s_...

Previous post here: http://arclanguage.com/item?id=7906

-----


You can really make apps as fast as what Rails or Django offer

I find this quite hard to believe. I've been developing Ruby/Rails web applications for over 2 years, and while I like the idea of Arc and would love to be as productive in a Lisp as I am in Ruby/Rails, that is not reality yet because of the libraries.

The toy example pg put up way back when did show off Arc, but once you add real world requirements, Arc comes up a little lacking.

  * scalability of not requiring session affinity
  * database libraries
  * regular expressions
  * Rails *huge* library
It's trivial (takes one line) to have a class with a name different than the table:

  class MyPet < ActiveRecord::Base
    set_table_name 'pets'
    ...
  end
You can turn off pluralization in one line.

I hardly consider those minor changes to be reconfiguring the whole thing. You may want to actually develop some non-trivial applications with Ruby/Rails before comparing to Arc.

-----

3 points by sacado 6017 days ago | link

> "I find this quite hard to believe. I've been developing Ruby/Rails web applications for over 2 years, and while I like the idea of Arc and would love to be as productive in a Lisp as I am in Ruby/Rails, that is not reality yet because of the libraries.

> The toy example pg put up way back when did show off Arc, but once you add real world requirements, Arc comes up a little lacking."

Oh, you are totally right about that. That's why I reimplemented the Arc web server in Lua. Now, I have libraries too (even if Lua isn't the richest language in this regard, at least communicating with the outside world is damn easy -- that's the language's goal). I think I'll try to write a small CRUD framework to see what can be done.

> "It's trivial (takes one line) to have a class with a name different than the table [...] You can turn off pluralization in one line.

> I hardly consider those minor changes to be reconfiguring the whole thing."

Ok, that was a little exageration. But to me it feels like bad design : why didn't they default to "the table has the same name as the class" ? That's simpler, easier to implement (no exceptions to take care of) and foreign-language friendly.

> "You may want to actually develop some non-trivial applications with Ruby/Rails before comparing to Arc."

I did it. It's especially when the application starts to be non-trivial that I hate having to fight with Rails designers' points of view to make my stuff work. On the opposite, with Arc/Luarc, the more my application grow, the better its design feels, because I have to make explicit things I considered implicit / subject to change in the previous versions.

That is probably only my own vision, but with the Arc model, I can see my applications becoming more mature and stable, while with the Rails model, I see them getting more and more patched, with more and more bureaucracy to make everything work.

Of course, it is still better than with, say, PHP, where it is bureaucracy and patches all the way down (that could be a motto : "PHP : patches all the way down").

-----

6 points by lojic 6017 days ago | link | parent | on: Where are we going?

I have to admit that Clojure looks very interesting. Functional programming, concurrency with software transactional memory, access to a huge Java library, conciseness, etc. It's also a Lisp-1 like Arc w/ Common Lisp style macros, but it has namespaces. It doesn't have TCO due to JVM limitations and I don't think it has continuations, so an Arc-like web server might be more challenging.

The concurrency story is particularly interesting given the proliferation of cores and slowing of GHz.

The Clojure version of Norvig's spelling corrector appears to be one line shorter than Norvig's Python version:

http://arclanguage.com/item?id=8554

I would really like to see an Arc version of that. I translated Norvig's Python version to Ruby easily, so I expect that someone fluent in Arc could do it easily.

The other Lisp versions I've seen (Scheme/Common Lisp) were quite a bit longer and less elegant IMO.

I'm not super excited about Clojure's dependence on the JVM, but on the other hand, that gives it a huge head start with libraries and state of the art VM/JIT technology.

Rich Hickey doesn't seem to have the star power of pg, but he is very involved in the language and community. He seems like more of a Matz or Guido which doesn't seem that bad - he seems quite sharp in his web cast presentations.

I could be way off, but it seems like it would be easier to add Arc innovations to Clojure than to add Clojure-like concurrency support to Arc.

-----

4 points by almkglor 6017 days ago | link

> continuations, so an Arc-like web server might be more challenging.

Arc's server doesn't actually use continuations - it uses functions that serve as continuatinos. There's no call to 'ccc or any of the wrapping forms in srv.arc (at least none that I remember)

> The concurrency story is particularly interesting given the proliferation of cores and slowing of GHz.

This is true and I think the "extra-cycles-to-waste" property expected by PG isn't going to put out in a hundred years.

> state of the art VM/JIT technology

Forget the libraries. This is what's scary about any JVM language.

-----

1 point by sacado 6017 days ago | link

"Arc's server doesn't actually use continuations - it uses functions that serve as continuatinos. There's no call to 'ccc or any of the wrapping forms in srv.arc (at least none that I remember)"

That's right, no 'ccc. That's why I could implement it so easily in Lua (no real continuation in Lua either). As for TCO, I'm not sure it's a real problem in this case. Sure, a lot of simple functions are written in a recursive style, but they can be trivially rewritten in an iterative style.

-----

2 points by fjl 5998 days ago | link

arc is a movement, not a language.

the primary goal was not to build a language but to get an avalanche of new lisp implementations started.

that lisp is quite easy to implement makes the idea even more compelling.

-----

2 points by Zak 5930 days ago | link

>It doesn't have TCO due to JVM limitations

This is only half-true. Clojure cannot automagically convert a self-call in the tail position in to iteration, but it does have the recur form to provide essentially the same semantics. I've also found it useful that recur throws an exception if used outside of the tail position, making it obvious which recursive forms consume stack space and which do not.

-----

1 point by lojic 6017 days ago | link | parent | on: Norvig's spelling corrector in Arc?

I found the following implementation in Clojure which compares much more favorably to Norvig's Python version than the other Scheme/Lisp versions I've seen. Still curious about how Arc might compare.

Source: http://en.wikibooks.org/wiki/Clojure_Programming#Norvig.27s_...

  (defn words [text] (re-seq #"[a-z]+" (. text (toLowerCase))))

  (defn train [features]
    (reduce (fn [model f] (assoc model f (inc (get model f 1)))) 
            {} features))

  (def *nwords* (train (words (slurp "big.txt"))))

  (defn edits1 [word]
    (let [alphabet "abcdefghijklmnopqrstuvwxyz", n (count word)]
      (distinct (concat
        (for [i (range n)] (str (subs word 0 i) (subs word (inc i))))
        (for [i (range (dec n))]
          (str (subs word 0 i) (nth word (inc i)) (nth word i) (subs word (+ 2 i))))
        (for [i (range n) c alphabet] (str (subs word 0 i) c (subs word (inc i))))
        (for [i (range (inc n)) c alphabet] (str (subs word 0 i) c (subs word i)))))))

  (defn known [words nwords] (for [w words :when (nwords w)]  w))

  (defn known-edits2 [word nwords] 
    (for [e1 (edits1 word) e2 (edits1 e1) :when (nwords e2)]  e2))

  (defn correct [word nwords]
    (let [candidates (or (known [word] nwords) (known (edits1 word) nwords) 
                         (known-edits2 word nwords) [word])]
      (apply max-key #(get nwords % 1) candidates)))

-----


How do you think either a new Lisp dialect such as Arc, or a new Lisp web development framework, can improve on Ruby/Rails for typical web development tasks?

-----

2 points by cchooper 6019 days ago | link

Arc already does some things better than Rails. Consider the difference in effort between creating the Arc "hello world" application and the Rails one.

-----

13 points by lojic 6019 days ago | link | parent | on: Where are we going?

What do you think about the history of Arc so far?

From my perspective, it appears to be a history of different expectations.

On the one hand, it seems that pg:

  1. is taking a long term view of the language
  2. is focusing on core language issues
  3. doesn't want to be held to a schedule
  4. prefers less communication with the community
whereas, it seems at least part of the community wants:

  1. to see *signs of life*
  2. useful libraries
  3. to know the plan (or whether there is one)
  4. more interaction from pg (communication, acceptance of bug fixes, etc.)
As with many misunderstandings, better communication might go along way in helping to set proper expectations.

For example, I have some questions for pg that might be shared by others in the community:

1) What is a reasonable upper bound for producing new releases of Arc - 6 months, 3 years, ... ?

2) Will you accept bug fixes for the language in the future? If so, approximately when?

3) Besides the profiler you mentioned, what contributions would you like from the community? Do you want any contributions from the community?

I think some of the disappointment that is felt by some in the community is due in part to the potential we see in the language. Yes, we know you're taking a long term view, but when we saw the potential to use Arc in place of current tools, we hoped we could use it soon. Obviously it's usable now, but that's a relative term.

-----

15 points by pg 6019 days ago | link

The first three are right. As for 4, it's not so much that I prefer less communication as that I sometimes may be too busy to visit the site.

I think the thing people may not understand is that I can only work on Arc intermittently. In the spring as well as a YC cycle we had startup school and I got married. The next day the summer YC cycle started. During the summer I was busy with a larger than usual number of startups. As soon as it was over I went on honeymoon, during which I barely touched a computer. I got back to the US just before applications for the winter cycle were due. I've been reading them all the past week, and I still have over 100 left. (I probably wouldn't even be writing this if I weren't desperate for ways to procrastinate.) Then we have to move YC and ourselves to the west coast, then do interviews. I'm hoping that I'll be free to work on Arc in mid or late November, but I don't want to promise anything.

So it's not so much that I don't want to be held to a schedule as that I couldn't do things to a schedule even if I wanted to.

I'm hoping that ultimately doing so many things will make them all turn out better. But it does mean I can't do any of them more than part time.

In answer to your specific questions: I don't think I can commit to an upper bound for new releases, because the factors that determine whether or not I can work on Arc aren't cyclic. Yes, I'll incorporate bug fixes, the next time I get to work on the language, which I hope will be late this year. At the moment the most valuable things people interested in Arc could do are (a) find bugs in the current implementation, (b) think about the core language and specifically what new operators, if they existed, would make existing Arc programs like news.arc significantly smaller in tokens, and (c) try using Arc to write different kinds of applications and report what happens.

-----

[deleted]
5 points by almkglor 6019 days ago | link

Did you have to be so crude?

Well I'm also kinda pissed that PG doesn't seem to be noticing anything Arc-related anymore, but his loss IMO.

-----

4 points by silence 6018 days ago | link

Thanks for all your hard work almkglor. I have learned a lot.

-----

4 points by lojic 6061 days ago | link | parent | on: Arc's web server in Lua

Agreed re: libraries. I expect that libraries might start showing up once/if folks feel that pg hasn't simply left entirely as it appears currently. Typical catch-22 - who wants to write libraries for a language that might stagnate or die?

-----

1 point by shader 6061 days ago | link

What killer libraries do y'all think would make arc interesting/useful enough to move beyond novelty?

-----

3 points by almkglor 6061 days ago | link

The boring stuff, like building nice parameterized SQL queries and getting back the data from SQL. Launching a system process in parallel and keeping track of its status (and potentially aborting it if e.g. it takes too long)

-----

1 point by shader 6061 days ago | link

If we do all of the boring stuff in a clean, concise way, that makes everything easy, with the option of adding macros on top to boot, the boring stuff might well become fun, or at the very least, painless.

Which boring thing would you start with?

-----

2 points by tokipin 6061 days ago | link

GUI imo. wxWidgets binding wouldn't be bad

-----

1 point by stefano 6061 days ago | link

Some times ago I started a GTK+ binding, now "paused". It's more boring than I thought initially. If you wish look at it for a starting point (file gtk.arc in Anarki). I now think a binding towards tcl/tk would look nicer and easier to use, though.

-----

1 point by stefano 6061 days ago | link

These would require a standard FFI system. Or else we would end up writing Anarki specific code. Such a fork would be a real Arc killer (in the bad sense of the term).

-----

3 points by almkglor 6061 days ago | link

sacado built an FFI on Anarki.... well forks are generally bad but with PG asleep until october or so .... (maybe he's getting ready for oktoberfest or something ^^)

-----

1 point by stefano 6061 days ago | link

Maybe he is preparing an Arc community summit at the oktoberfest? :)

-----

2 points by bOR_ 6061 days ago | link

If rails is to be considered the killer library of ruby, it took five years for it to show up.. so we have some time left ;)

-----

2 points by stefano 6061 days ago | link

That's true. Programming languages take years to evolve, not months. Moreover, Arc is in the language design phase. As of today, the syntax and the semantics of the language are more important than libraries. For example, I'd like to see something like Kenny Tilton's cells (http://github.com/stefano/cells-doc/) nicely integrated within the core language.

-----

2 points by lojic 6060 days ago | link

Is it in the language design phase? Does anyone know if this is in fact true, and if so, what is currently being designed? The impression I have is that it just sort of stopped.

-----

2 points by shader 6060 days ago | link

I would suppose that since a) this language is not used in anything "important" yet, and b) it's open source; yes, it can be in the design phase. I should think that the design phase persists until, for some reason, the language becomes "formalized", or until it is impossible to change anything major without ruining things for a lot of people. At that point you can still "design" the language, but since it has a generally understood "style" and so forth, it won't be able to change too much unless you make a whole new language.

What do you want to be designed? One of the major problems about "designing" a new lisp is that, since anyone can "do it" in their spare time, they don't see the point. Maybe they're right. ^^

Sorry for all of the quotes; it looks kind of funny, I'm sure.

-----

1 point by lojic 6070 days ago | link | parent | on: Norvig's spelling corrector in Arc?

Here's a translation of Norvig's code to Ruby for comparison:

http://lojic.com/blog/2008/09/04/how-to-write-a-spelling-cor...

-----

5 points by lojic 6071 days ago | link | parent | on: I know no Lisp at all, should I learn Arc?

I think it's premature to worry about a next release :)

-----

5 points by pg 6068 days ago | link

I'll have a good deal of time to work on Arc in October and November. I'll probably release some new versions then.

-----

5 points by lojic 6074 days ago | link | parent | on: FreeBSD deployment experiences

Wow, what a frustrating experience. I just burned a bunch of hours trying to install FreeBSD w/ X11 and gnome using the ports system. Lots of dependency failures when building gnome20 with ports. I'd fix one and get a little further then something else would blow up. Had to do the following a lot:

  cd /usr/ports/<whatever>
  make deinstall
  make reinstall
  cd /usr/ports/x11/gnome20
  make all install clean
  # wait for next problem and repeat
Older versions of packages were installed instead of the ones expected, etc.

In the end, I never did get it working so reloaded Ubuntu 8.04 on the machine. I'm sure FreeBSD works great has a headless server, but I thought I'd load it onto a desktop or two to get familiar with it, and to possibly convert my other Ubuntu machines to it to avoid having to support OSX, Linux & FreeBSD.

Another problem is that even though I use dedicated hosts, I'd like the option of deploying on a vps such as slicehost.com and they don't support FreeBSD.

I know folks use it as a desktop without problems, so I'm not sure what the difference is. I installed 7.0 from a DVD and followed install instructions from an article verbatim. Accepted defaults, etc., so it should've been a plain vanilla install.

If I thought FreeBSD offered clear advantages over Linux, I'd probably be willing to put in more effort, but from what I can tell, I expect they're more similar than different from an application developer's perspective.

If folks know of compelling advantages, I'd like to hear about them.

-----

3 points by rkts 6073 days ago | link

Some advantages of FreeBSD over Debian / Ubuntu:

1. Ports aren't changed randomly for no reason (cf. the Debian OpenSSL debacle). A port is just that--a port of a program to FreeBSD, not some attempt to incorporate it into a holistic user experience.

2. Ports are easy to configure and tweak.

3. The source code is located in one directory and can be built with a few commands. Compare http://www.linuxfromscratch.org/

4. The BSD license means you can do what you want with the code.

5. The overall system is clean, simple and very well documented. Most of what you need to know is in the handbook or man pages.

The advantages are more evident for a server. For a desktop, I think BSD and Linux are both pretty awful compared with OS X, but FreeBSD is slightly less awful, overall.

-----

1 point by eekee 6064 days ago | link

Points 1 & 2 also apply to Source Mage GNU/Linux, and for point 5, man pages are good for most packages and for the system these days, there is a developing wiki maintained by the distro, and documentation that may be missing is made up for by the helpfulness of the people in the IRC channel.

Point 3 largely applies in that building every package you install is far more automated than in LFS, and most of the "spells" (ports, packages) are kept up to date regularly (Gnome included), and it's very easy to update any that may have been forgotten, or to add missing packages. It took me about 5 minutes to clone the mzscheme spell to "mzscheme352" for Arc use. OTOH you still have to deal with the behemoth that is the Linux kernel configuration. I cheated; I got hold of a Debian kernel and used it as a base for make oldconfig, later tweaking a little.

-----

More