Arc Forumnew | comments | leaders | submitlogin
Web Scaffolding
9 points by drcode 6136 days ago | 14 comments
Hi, I've written a scaffolding system in arc for avoiding repetitive code when implementing similar (but not the identical) entities in an application.

The classic example for this is a website that has products, customers, orders, etc- Each of these needs to be tracked by themselves and there must be similar (but different) mechanisms for creating/updating/deleting each on a site.

My system extends the current arc template system and uses an extremely minimalistic approach, in keeping with arc. The current system has the following overall relationship:

  template -> instance
Scaffolding.arc defines new functions that models the following relationship:

  scaffold template -> scaffold instance -> entity
In concrete terms, you might have a template for web interactions called webscaff created using "deftem". You would create an instance of the template called mybooksite using "inst". Then you can call "inst-entity" (a new command) on mybooksite to generate versions of the source code for the customers, books, orders, suppliers, etc.

For those of you who know Rails, this is analogous to Rails scaffolding, except that the code isn't created by a byzantine code generator. Instead, it is created dynamically when the program is loaded, basically using macro-like operations. The scaffold and app can therefore change independently and are stitched together dynamically when you load the app. This was designed to be in keeping with the arc/lisp philosophy.

To start off, I created a sample scaffold template defined in webscaff.arc. It is pretty much a line-by-line refactoring of pg's blog.arc. Here is an example of the scaffolding code that handles a request to view an individual item through the website:

  (addtemscaff webscaff view
	     `(defop ,ename!view req
		(aif (,(ename) (arg req "id")) 
		     (,ename!-singleton-page (get-user req) it) 
		     (notfound))))
"addtemscaff" is a macro basically identical to addtem tweaked a bit for making scaffolding a little easier. This code creates a new property in the webscaff template. The anaphoric variable "ename" contains the entity name. It will be expanded at the time each entity is defined, creating different pages such as "bookview", "customerview", "orderview", etc. More accurately, "ename" is a function that returns the entity name as a result, but can be given an extra parameter that will be concatenated as a suffix. This means if the entity is "book" then (ename "add") creates the symbol "bookadd". If you use the intrasymbol arc syntax, this can be elegantly written as ename!add

Other anaphoric variables available from the scaffolds are sname (the name of the scaffold, i.e. "mybooksite") scaffold (allows you to access the instance of the current scaffold directly in case you've put extra scaffold-level data points there, like a title for your site) and entity (allows you to access any optional data points sent to the inst-entity command- The most common such data points would be the field names/types for a book or whatever)

As a working example, I created a Blog app with comment support. It is in "blog-fancy.arc". As you will see, the amount code needed for this app is minimal. It just runs inst-entity once for "posts" and once for "comments". Then, it overrides a bit of the scaffolded code so that the program implements the one->many relationship between posts and comments. This means you can create comments linked to a post and can view post-specific comments. Note that most of the derived functions are just slightly modified from the scaffold- With additional refactoring of the scaffolding system, I plan to reduce this code even more drastically. Also note that nothing in scaffolding.arc is web specific (only the specific scaffold declared in webscaff.arc) and could conceivably be used in other contexts. It could also be inherited from (deftem allows inheritance) and specific fragments overridden by the new class by using the property names assigned to the parts of the scaffold.

This is definitely still a very early implementation of this scaffolding system. I hope to push improvements to anarki over time. Please feel free to push your own changes as well if you like- There is definitely much room for improvement at this time.

The files are on anarki: http://git.nex-3.com/arc-wiki.git

-Conrad Barski



1 point by almkglor 6136 days ago | link

Looks interesting. Haven't yet grokked the how-to-use, but then I'm supposed to be working right now, not hacking ^^.

Some nitpicks:

1. scaffold.arc / inst-entity1 uses global variables. Is this deliberate? Why?

2. Personally I would recommend using the following interface to inst-entity1:

  (def inst-entity1 (temname scaffold scaff rinf)
    ...)
  (mac inst-entity (temname scaff . rinf)
    `(inst-entity1 ,temname ,scaff ',scaff ',rinf))
However, I'm not 100% sure why you're using global variables in inst-entity1 in the first place - is there some special reason?

3. docstrings would be nice ^^.

4. The actual code seems to use addtemscaff (not addscaff) and rname (not ename). Or am I looking at the wrong code?

Edit:

5. You seem to be storing the code added by addtemscaff as lists of expressions. Why not store them in a function? That way we can capture variables into an environment instead of forcing the user to always use globals.

6. The (each v tablist.scaffold ...) could probably be transformed to (ontable k v scaffold ...)

-----

1 point by drcode 6136 days ago | link

> 1. scaffold.arc / inst-entity1 uses global variables. Is this deliberate? Why?

The arc eval doesn't seem to support local environments or passing of environments, so this is the only way to pass in anaphoric variables I could find on my first pass. This function could probably be rewritten without eval and with proper scoped anaphoric variables, but given what is happening in the code this requires serious macro-fu. I do sort of know what I'm doing here- The evals and global vars don't make me feel good either... I will clean this up eventually, but won't complain if someone else figures out how to do it first :)

> 2. Personally I would recommend using the following interface to inst-entity1:

  (def inst-entity1 (temname scaffold scaff rinf)
    ...)
  (mac inst-entity (temname scaff . rinf)
    `(inst-entity1 ,temname ,scaff ',scaff ',rinf))
Yeah, you're probably right here... my brain is hurting right now... once it's doing better I will look into it...

-----

1 point by almkglor 6136 days ago | link

> requires serious macro-fu

True, that's what it looks like. It seems nearer to a macro-defining-macrosystem rather than plain macros, actually. Should be a challenge ^^.

-----

1 point by treef 6136 days ago | link

Its not always good to have macros writing macros one has to maintain it :)

-----

1 point by almkglor 6136 days ago | link

Yes, but at least I don't have to write the macro directly, ne? ^^

-----

1 point by drcode 6136 days ago | link

> 5. You seem to be storing the code added by addtemscaff as lists of expressions. Why not store them in a function? That way we can capture variables into an environment instead of forcing the user to always use globals.

I am planning on adding a command for debugging scaffolds, where the instances of the scaffold are instantiated and pretty-printed to make writing scaffolds less uber-meta. Having functions would prevent this- But you are right that using functions might be the Right Thing. I'll give it some thought.

> 6. The (each v tablist.scaffold ...) could probably be transformed to (ontable k v scaffold ...)

Yes I think you're right. This is definetely "release early, release often" type code, so there's lots of room for improvement.

-----

1 point by almkglor 6136 days ago | link

> a command for debugging scaffolds, where the instances of the scaffold are instantiated and pretty-printed ... Having functions would prevent this

Looks like we should really be solving the correct problem: make function code pretty-printable ^^. Yes, I think I'll solve this using attachments again... ^^ Go attachments! http://arclanguage.com/item?id=3698

Edit: Which reminds me, I haven't actully grokked this scaffolding thing at all yet. I'll be home in maybe 3-4 hours, hopefully I can get the weekend for this, unless I do something stupid, like track down where the girl I'm trying to see is hiding ^^.

-----

1 point by drcode 6136 days ago | link

Numbers 1, 2, 4, 5, and 6 are now resolved. (#6 is resolved because my new code makes ontable impossible :-)

I used your advice to store functions- Used properly, this will not affect the debug code I had in mind afterall.

-----

1 point by drcode 6136 days ago | link

#4 is fixed now.

-----

1 point by drcode 6136 days ago | link

I just commited new changes in anarki so that I no longer have to hang my head in shame about 3 evals and global variable declarations in "inst-entity".

I believe it is pretty clean now, with only one appropriately-used eval and no global variables mucking up the environment.

I was pleased to see the radical changes I made to the scaffolding system required no changes to my blog app, which speaks well to my hopes of having scaffolds and apps knitted together that can both be improved independently. (There was one unrelated bug I fixed though in blog-fancy.arc)

-----

1 point by almkglor 6135 days ago | link

Hmm. Somehow I don't quite like the "scaffold" anamorphic variable, since it strikes me that it is accessible (indirectly) via (ename):

  (addtemscaff foo thingy
    `(= (,(ename) 'property) 'value))
edit: using (ename) would also allow us to dynamically change the entity itself:

  (addtemscaff foo thingy2
    `(def ,ename!-foo-function ()
        (prn (,ename 'property))))
end edit.

By eliminating the scaffold anamorphic we can then create a purely macro-based system without having to use eval, which personally makes me nervous.

  (mac addtemscaff (scaff name body)
    `(addtem ,scaff (scaff ,(++ scaff-maxid*) ,name)
             (fn (sname entity ename)
                 ,body)))

  (mac inst-entity (ename sname . entity)
    (inst-entity1 sname entity ename))

  (def inst-entity1 (sname entity ename)
    (let ordered-scaffs (sort (fn (a b) (< (cadr:car a) (cadr:car b))) (keep (fn ((k v)) (and (acons k) (is 'scaff (car k)))) tablist.scaffold))
      (each ((x y fname) v) ordered-scaffs
            (prn "scaffolding " ename ": " fname)
            (v
                     (fn ((o suff nil))
                         (sym:string sname suff))
                     entity
                     (fn ((o suff nil))
                         (sym:string ename suff))))))

-----

1 point by drcode 6135 days ago | link

> Hmm. Somehow I don't quite like the "scaffold" anamorphic variable, since it strikes me that it is accessible (indirectly) via (ename)

It is true that you could access the scaffold template object by using sname (which stands for "scaffold name" as opposed to ename, which stands for "entity name")

This might make sense... I'll give it some thought...

> By eliminating the scaffold anamorphic we can then create a purely macro-based system without having to use eval, which personally makes me nervous.

This doesn't seem to be true... the reason the "eval" is needed is because calling the scaffold function ("v" in this case) just returns an sexp, which then still needs to be evaluated. I don't think the code sample you gave without the eval could be made to work. (That eval is basically the "magic" that lets you write a single scaffold for a "def" and have it become multiple defs, one for each entity (i.e. producteview, customerview, orderview, etc.)

The only way I can see to get rid of this final eval is to not have an inst-entity1 function and elevate all the logic to be part of the inst-entity macro. Then, the entire set of scaffold sexps could be unquote-spliced into the body of the macro- This is probably a good idea, but I haven't gotten to it yet. (I actually tried doing something like this once in scheme, but scheme "define" has limitations when not used at the toplevel- I believe they fixed this in R6RS because of complaints. Arc defs can be used without constraints at any level I think, so that specific problem wouldn't exist)

-----

1 point by almkglor 6135 days ago | link

Err, I am using inst-entity1 as a call within the inst-entity macro ^^, so the return value of inst-entity1 is evaluated.

So I guess it should really be more like:

  (cons 'do (w/collect:each ....
    (collect (v ...))))
Where w/collect is a macro similar to one I built some time ago. http://arclanguage.org/item?id=4390

-----

1 point by drcode 6135 days ago | link

Ah I see now...

It still doesn't look like it would work though, since a scaffold could have like 30 sexps and only the last one would be evaluated by your sample I think...

but if the "each" was changed to a "map" and a "do" was inside the macro I think it could be made to work that way- That's along the lines of what I was thinking

-----