Daily updated test coverage:  http://cv.perl6.cz/

A collection of information and links about Parrot's code coverage statistics.

  • Configuration and build tools test coverage:  http://thenceforward.net/parrot/coverage/configure-build/coverage.html
    • At the present time (October 2007 and until further notice), the pages at and under the link above describe the extent to which tests in t/configure/, t/postconfigure/, t/tools/pmc2cutils/, t/tools/ops2pmutils/ and t/tools/ops2cutils/ provide coverage of the code underlying Configure.pl, tools/build/pmc2cutils.pl, tools/build/ops2pm.pl and tools/build/ops2c.pl. All of these programs are written in Perl 5 and so the focus of these tests is the quality of Perl 5 components of the Parrot project -- not the functionality or quality of Parrot code per se.
  • The approach taken in these tests is to apply the best testing practices developed in the Perl community in the last decade, including:
  • Test suites written with Test::More and run with Test::Harness utility prove.
  • Measurement via Devel::Cover of the extent to which the tests exercise code statements.
  • Refactoring of the code to eliminate unreachable code and achieve high test coverage.
  • While Parrot developers are not required to develop in a "test-driven" manner, they are encouraged to do so and are *strongly* encouraged, when reworking any of the programs or modules covered in these tests, to run the pre- and post-configuration test suites to guarantee that:
    • All tests in those suites continue to pass.
    • There is no diminution in code coverage as a result of ongoing revisions of the programs and packages being tested.
  • Here are some ways to run the tests:
    • perl Configure.pl --test: Runs t/configure/, Configure.pl itself, t/postconfigure/ and tests in the three t/tools/ directories listed above.
    • perl Configure.pl --test=configure: Runs t/configure/ and Configure.pl.
    • perl Configure.pl --test=build: Runs Configure.pl, t/postconfigure/ and t/tools/.
    • make buildtools''tests: Valid once Configure.pl has been run, runs tests in the three t/tools/ directories.
    • Here is a shell script to run the tests with coverage analysis: 

     COVERAGEDIR=/home/user/parrot/coverage
     SANDBOX=/home/user/work/parrot
     # coverconf: configure-build coverage
     coverconf () 
     {
     cd $SANDBOX && \
     cover -delete $COVERAGEDIR && \
     PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR/ \
     perl $SANDBOX/Configure.pl --test --configure_trace "$@" && \
     cover $COVERAGEDIR \
     -ignore_re 'Configure\.pl' -ignore_re '^t\/' -ignore_re '^\/usr\/local\/bin' \
     -ignore_re '^lib\/(?!Parrot)' && \
     chmod 0755 $COVERAGEDIR/
     }
  • See also [Perl 5 Components: Test Coverage of Configuration and Build Tools]
  • Note: Should the coverage statistics not be visible, wait a while, as they may be in the process of being regenerated. 
     make cover
     cd cover_db
     rm -r runs/ structure/
     tar -cvzf c_coverage.tar.gz *
     scp c_coverage.tar.gz thenceforward.net:/home/ftp/parrot/coverage/c-components/
     ssh thenceforward.net
     cd /home/ftp/parrot/coverage/c-components
     rm *.html cover.12 cover.css
     tar -xvzf c_coverage.tar.gz