Version 1 (modified by moritz, 6 years ago)

First mind map of Perl 6 S24, Testing

Purpose, goals

  • built-in, very easy testing
  • gain exactness
  • good diagnostics

Definition

  • series of discrete operations with boolean result, marked as tests

Syntax

overall structure

  • plan(2); tests; # done
  • plan(*); tests; tests-done(); # names subject to discussion ;-)

single tests

we have two proposal:

adverb on operators

$x eq 'foo' :ok('Test descriptions'); $x == 23 :ok('explicit numeric test'); ?$x :ok<stuff> # used to be ok($x, 'stuff'); try { dispatch() } :ok<...> # used to be lives_Ok { ... }, '...';

(it was also proposed to name it :test instead of :ok) pros:

  • good diagnose of output
  • force exact comparison semantics
  • no pollution of namespace

cons:

  • have to auto-generate ops with adverbs
  • more complicated grammar needed
  • the fact that a line contains a test is somewhere in the middle, so not too obvious

normal sub calls, like now

ok $x eq 'foo', 'description'; dies_ok { ... }, 'description';

pros:

  • easier to implement
  • it's been proven to work out well
  • visually tests are very easy to spot

cons:

  • it's *very* hard to get good diagnostics out of failed tests (advanced macro fiddling needed)
  • clutters up namespace
  • if we don't deprecate is(), we have to find sane comparison semantics, fix up the tests (we have to do that anyway...) and educate our test writes. Currently infix:<eqv> is a candidate (advocated by pmichaud, because it's very dwimmy), infix:<eq> being another (advocated by mority, because it's very simple and easy to explain).

Backends and standard backend

  • all the test logic should be easily overridable (for example for storing

test results in a db, or different test output format)

  • simplest idea:

there's a Test::Backend object in $*TEST_BACKEND, and plan() class the .plan method, :ok('...') adverbs on ops call.

  • the default backend will emit TAP output
  • the default backend will consider a test run failed if any single test failed, or in case of plan mismatch