Version 3 (modified by moritz, 6 years ago)

Added preamble.

About

This is a mind map for a draft or RFC of the Perl 6 synopsis S24, Testing. So far mostly written by Moritz Lenz, but feel free to enhance.

Please excuse my (ab)use of the parrot wiki for this only slightly related page.

Purpose, goals

  • built-in, very easy testing
  • gain exactness
  • good diagnostics

Definition

  • series of discrete operations with boolean result, marked as tests

Syntax

overall structure

  • plan(2); tests; # done
  • plan(*); tests; tests-done(); # names subject to discussion ;-)

single tests

we have two proposal:

adverb on operators

                $x eq 'foo'     :ok('Test descriptions');
                $x == 23        :ok('explicit numeric test');
                ?$x             :ok<stuff>      # used to be ok($x, 'stuff');
                try { dispatch() } :ok<...>     # used to be lives_Ok { ... }, '...';

(it was also proposed to name it :test instead of :ok)

pros:

  • good diagnose of output
  • force exact comparison semantics
  • no pollution of namespace

cons:

  • have to auto-generate ops with adverbs
  • more complicated grammar needed
  • the fact that a line contains a test is somewhere in the middle, so not too obvious

normal sub calls, like now

                ok $x eq 'foo', 'description';
                dies_ok { ... }, 'description';

pros:

  • easier to implement
  • it's been proven to work out well
  • visually tests are very easy to spot

cons:

  • it's *very* hard to get good diagnostics out of failed tests (advanced macro fiddling needed)
  • clutters up namespace
  • if we don't deprecate is(), we have to find sane comparison semantics, fix up the tests (we have to do that anyway...) and educate our test writes. Currently infix:<eqv> is a candidate (advocated by pmichaud, because it's very dwimmy), infix:<eq> being another (advocated by mority, because it's very simple and easy to explain).

Backends and standard backend

  • all the test logic should be easily overridable (for example for storing

test results in a db, or different test output format)

  • simplest idea:

there's a Test::Backend object in $*TEST_BACKEND, and plan() class the .plan method, :ok('...') adverbs on ops call.

  • the default backend will emit TAP output
  • the default backend will consider a test run failed if any single test failed, or in case of plan mismatch