Arc Forumnew | comments | leaders | submitlogin
Unit-test.arc 1.0 incoming
4 points by zck 3167 days ago | 14 comments
I've had a revamp of my unit-test library coming for a while. It's finally about ready. I want to look for some feedback before officially releasing the 1.0.

The branch that's going to become 1.0 is here: https://bitbucket.org/zck/unit-test.arc/src/8b9dbe91702c38666759f3995b09e002b7b57200?at=run-test . The documentation there is up to date for the branch.

I'd appreciate feedback before I officially release it as 1.0. I'm especially looking for changes I'd have to make that are not backwards compatible. I'm happy to answer questions or help write test cases.

Thanks to akkartik for suggestions and assistance at many points along the way.



3 points by akkartik 3162 days ago | link

Hey, one last-minute suggestion: suite-w/setup is a pretty long and ugly name. How about if we just always include a set of setup bindings in suite, even if they're empty? It would look nice and consistent with def and mac:

  ; without setup
  (suite foo ()
    (test must-bar
      (assert-same 2 (+ 1 1)))

  ; with setup
  (suite foo (a 1
              b 2)
    (test must-bar
      (assert-same b (+ a a)))
I wouldn't be too disappointed if you decide against this, partly since it would save me the trouble of fixing my translator all over again :)

-----

2 points by zck 3162 days ago | link

Interesting. I like the consistency. I don't love how it makes the most common case (I think the no-setup case is most common) and adds more code to it.

Maybe I can come up with a simpler, less awful thing.

Perhaps:

    (suite foo (setup a 1
                      b 2)
           (test must-bar
                 (assert-same b (+ a a))))
It does seem relatively simple. Hrm.

-----

2 points by akkartik 3162 days ago | link

I'm kinda growing to like my idea the more I think about it. You're right that it adds 3 characters to the common case, but lisp has a long tradition of empty parens in various places. Saving characters shouldn't be a high priority, IMO. Paul Graham's notion of conciseness counts tokens, not characters.

But yeah, happy to see what other ideas we can come up with. I think the setup keyword above is worse; lisps don't tend to have keywords that aren't functions or macros. Then again, test is already a keyword that's not a function.. Hmm, I like it better if you indent it like this:

  (suite foo
         (setup a 1
                b 2)
         (test must-bar
               (assert-same b (+ a a))))
The benefit of this approach is that it makes the syntax seem extensible. It's obvious how new features should be added.

Ok, I could live with this :)

-----

3 points by akkartik 3167 days ago | link

Looking forward to playing with this!

Edit 33 minutes later: I'm not much of a Mercurial user. How do I switch branches?

-----

2 points by zck 3166 days ago | link

`hg checkout run-test` will do it.

If you want to view all branches, `hg branches`. There are some old ones I need to finish and merge back in. To view the current branch, `hg branch`.

Or if you want to look at it online, the link above (https://bitbucket.org/zck/unit-test.arc/src/8b9dbe91702c3866...) should take you to the right branch. (Edit: you'll know you're on the right branch if the second code block runs tests with (test), not (run-suite))

-----

1 point by akkartik 3166 days ago | link

Ah, thanks. Crazy that https://www.mercurial-scm.org/wiki/GitConcepts doesn't mention hg checkout.

-----

2 points by akkartik 3165 days ago | link

I just tried running the 'run-test' branch with the Anarki tests, but I get an error. Is it expected to be backwards-incompatible?

-----

2 points by zck 3164 days ago | link

It's not backwards compatible. Hence me trying to get feedback before pushing the 1.0.

The specific backwards incompatibility (I assume you got "We can't parse this as a suite body") is moving from this format for the tests:

    (suite suite-name
           test-name (test-body))
to the format you suggested last year:

    (suite suite-name
           (test test-name (test-body)))
I don't think supporting both ways of writing tests is useful. I'm also planning on migrating the anarki tests when I roll out the 1.0. Actually, I could get started on a branch even before. That'd be something to do.

-----

1 point by akkartik 3164 days ago | link

Ah, I'd forgotten :)

Yeah, no reason to be compatible. I was just confused. I'll migrate the tests to the new format, and update Anarki publicly once you release 1.0.

-----

2 points by zck 3163 days ago | link

If you start doing some of the migration work, push the in-progress stuff to a branch and I'll help work on it. I'll do the same if I get to it before you. Thanks.

-----

2 points by akkartik 3163 days ago | link

I spent a few minutes trying to build an automatic script to upgrade our tests -- and found a bug in Arc's serialize/unserialize (my fault) :)

https://github.com/arclanguage/anarki/commit/9efd58992d

What's the equivalent of (run-all-suites) in 1.0? I didn't see this covered in the Readme.

Other than that my script seems to be barfing on nested suites at the moment. I'll continue working on it tomorrow.

-----

2 points by zck 3162 days ago | link

Oh, fancy writing a script to do the upgrade. It'd be cool to see it.

You can run all tests with (test). I should add that to the instructions, thanks.

-----

2 points by akkartik 3162 days ago | link

This must be the most fun thing on my radar right now, because I got to it first thing this morning :)

  ; translate.arc
  (def translate (expr)
    (accum acc
      (translate-2 expr acc)))

  (def translate-2 (expr acc)
    (if (atom expr)
          (acc expr)
        (is car.expr 'suite)
          (do (acc 'suite)
              (let (suite-name . suite-body)  cdr.expr
                (acc suite-name)
                (translate-suite-body suite-body acc)))
        (is car.expr 'suite-w/setup)
          (do (acc 'suite-w/setup)
              (let (suite-name suite-setup . suite-body)  cdr.expr
                (acc suite-name)
                (acc suite-setup)
                (translate-suite-body suite-body acc)))
        'else
          (map acc expr)))

  (def translate-suite-body (suite-body acc)
    (if suite-body
      (if (acons car.suite-body)
        ; nested suite
        (let (nested-suite . rest) suite-body
          (acc (accum acc2
                  (translate-2 nested-suite acc2)))
          (translate-suite-body rest acc))
        ; test name must be atomic
        (let (test-name test-body . rest)  suite-body
          (acc `(test ,test-name ,test-body))
          (translate-suite-body rest acc)))))

  ; bootstrap tests for a test harness :)
  ; suite with tests
  (assert:iso '(suite a (test t1 b1) (test t2 b2))
              (translate '(suite a t1 b1 t2 b2)))

  ; suite with tests and nested suites
  (assert:iso '(suite a (test t1 b1) (suite s2 (test t3 b3)) (test t2 b2))
              (translate '(suite a t1 b1 (suite s2 t3 b3) t2 b2)))

  ; suite with setup and tests
  (assert:iso '(suite-w/setup a (x 1 y 2) (test t1 b1) (test t2 b2))
              (translate '(suite-w/setup a (x 1 y 2) t1 b1 t2 b2)))

  ; suite with setup and tests and nested suites
  (assert:iso '(suite-w/setup a (x 1 y 2) (test t1 b1) (suite s2 (test t3 b3)) (test t2 b2))
              (translate '(suite-w/setup a (x 1 y 2) t1 b1 (suite s2 t3 b3) t2 b2)))

  ; run
  (each f cdr.argv
    (prn f)
    (fromfile string.f
      (tofile (+ string.f ".2")
        (each expr (drain:read)
          (let out translate.expr
            (ppr out))))))
Run it like so:

  $ arc translate.arc *.t lib/*.t lib/tests/*
I haven't committed it anywhere yet because I'm not too happy with the state of Anarki's pretty-printer. Would you mind if I change the indentation style for suites and tests in Anarki? I was thinking something like this:

  (suite atom
    (test includes-int (assert-t (atom 3)))
    (test includes-float
      (assert-t (atom 3.14159)))
    (test includes-exact (assert-t (atom 3/16)))
    (test includes-symbol (assert-t (atom 'a)))
    (test includes-char (assert-t (atom #\a)))
    (test includes-string
      (assert-t (atom "hello")))
    (test includes-nil (assert-t (atom nil)))
    (test excludes-list
      (assert-nil (atom '(1 2 3))))
    (test excludes-table
      (assert-nil (atom (obj a 1 b 2))))
    (test excludes-tagged-types
      (assert-nil (atom (annotate 'foo 34)))))
..rather than this current output:

  (suite atom
         (test includes-int (assert-t (atom 3)))
         (test includes-float
               (assert-t (atom 3.14159)))
         (test includes-exact (assert-t (atom 3/16)))
         (test includes-symbol (assert-t (atom 'a)))
         (test includes-char (assert-t (atom #\a)))
         (test includes-string
               (assert-t (atom "hello")))
         (test includes-nil (assert-t (atom nil)))
         (test excludes-list
               (assert-nil (atom '(1 2 3))))
         (test excludes-table
               (assert-nil (atom (obj a 1 b 2))))
         (test excludes-tagged-types
               (assert-nil (atom (annotate 'foo 34)))))

-----

2 points by akkartik 3162 days ago | link

All done:

  atom:  	10 tests,  	0 nested suites.
  memtable:  	5 tests,  	0 nested suites.
  do:  	2 tests,  	0 nested suites.
  for:  	3 tests,  	0 nested suites.
  ssyntax:  	13 tests,  	0 nested suites.
  ...
  strings:  	0 tests,  	15 nested suites.
      trim:  	4 tests,  	0 nested suites.
      tokens:  	2 tests,  	0 nested suites.
      endmatch:  	4 tests,  	0 nested suites.
      subst:  	3 tests,  	0 nested suites.
  ...
  types:  	0 tests,  	3 nested suites.
      coerce:  	2 tests,  	1 nested suite.
          strings:  	9 tests,  	5 nested suites.
              numbers:  	6 tests,  	3 nested suites.
                  rational-rounds-with-unused-base-arg:  	6 tests,  	0 nested suites.
                  rational-rounds-to-even:  	6 tests,  	0 nested suites.
                  float-rounds-to-even:  	6 tests,  	0 nested suites.
              characters:  	4 tests,  	0 nested suites.
              coercions-to-same-type:  	3 tests,  	0 nested suites.
              string->num:  	18 tests,  	0 nested suites.
              string->int:  	13 tests,  	0 nested suites.
      type:  	15 tests,  	0 nested suites.
      annotation:  	4 tests,  	0 nested suites.
  Suite module: all 3 tests passed!
  Suite special-syntax: all 4 tests passed!
  Suite html: the single test passed!
  There are no tests directly in suite special-forms.
      Suite special-forms.function-definition: all 22 tests passed!
      Suite special-forms.if: all 5 tests passed!
      Suite special-forms.quasiquote: all 15 tests passed!
      Suite special-forms.quote: all 3 tests passed!
      Suite special-forms.assign: all 3 tests passed!
  There are no tests directly in suite types.
      Suite types.coerce: all 2 tests passed!
          Suite types.coerce.strings: all 9 tests passed!
              Suite types.coerce.strings.numbers: all 6 tests passed!
                  Suite types.coerce.strings.numbers.rational-rounds-with-unused-base-arg: all 6 tests passed!
                  Suite types.coerce.strings.numbers.rational-rounds-to-even: all 6 tests passed!
                  Suite types.coerce.strings.numbers.float-rounds-to-even: all 6 tests passed!
              Suite types.coerce.strings.characters: all 4 tests passed!
              Suite types.coerce.strings.coercions-to-same-type: all 3 tests passed!
              Suite types.coerce.strings.string->num: all 18 tests passed!
              Suite types.coerce.strings.string->int: all 13 tests passed!
      Suite types.type: all 15 tests passed!
      Suite types.annotation: all 4 tests passed!
  Suite len: all 3 tests passed!
  Suite bracket-fn: all 3 tests passed!
  ...

  Yay! All 606 tests passed!

-----