Create new testing target: 'make test' + more

[jkeenan]

In the  Dec 28 parrotsketch, it was suggested that we develop a new make target that would be more encompassing than make test, but less encompassing (and therefore quicker) than make fulltest.

Several variants were suggested. My own preference would be a target that would include:

make test
make examples_tests
make codetest # (except not t/codingstd/perlcritic.t)

How does make moretests sound?

I will create a branch for this, but discussion is still open as to content and name of the target.

Thank you very much.

kid51

[NotFound]

I think the Makefile is already too big and complex. Tasks that doesn't need complex handling of dependencies may be better driven by parrot programs than from long and complicated make variables and targets. Let's the make target do the minimal dependency handling and run that programs.

[jkeenan]

Replying to NotFound:

I think the Makefile is already too big and complex. Tasks that doesn't need complex handling of dependencies may be better driven by parrot programs than from long and complicated make variables and targets. Let's the make target do the minimal dependency handling and run that programs.

NotFound:

You are proposing a large-scale overhaul of our make system. I think that would be scope-creep for this ticket. Can you either start an RFC ticket or open a discussion on parrot-dev?

Thank you very much.

kid51

[coke]

>  make codetest # (except not t/codingstd/perlcritic.t)

Note that this test is easily skippable (see the POD for details).

-- 
Will "Coke" Coleda

[jkeenan]

Replying to coke:

> >  make codetest # (except not t/codingstd/perlcritic.t)
> 
> Note that this test is easily skippable (see the POD for details).

Coke, from make help I see that you can run perlcritic by itself via make perlcritic -- but I don't see a way to run all the t/codingstd/*.t tests except perlcritic.t.

Am I missing something?

kid51

[coke]

On Tue, Dec 28, 2010 at 8:19 PM, Parrot <parrot-tickets@lists.parrot.org> wrote:
> #1908: Create new testing target: 'make test' + more
> ---------------------+------------------------------------------------------
>  Reporter:  jkeenan  |       Owner:  jkeenan
>     Type:  todo     |      Status:  assigned
>  Priority:  normal   |   Milestone:
> Component:  testing  |     Version:  2.10.0
>  Severity:  medium   |    Keywords:
>     Lang:           |       Patch:
>  Platform:           |
> ---------------------+------------------------------------------------------
>
> Comment(by jkeenan):
>
>  Replying to [comment:4 coke]:
>  {{{
>  > >  make codetest # (except not t/codingstd/perlcritic.t)
>  >
>  > Note that this test is easily skippable (see the POD for details).
>  }}}
>
>  Coke, from `make help` I see that you can run ''perlcritic'' by itself via
>  `make perlcritic` -- but I don't see a way to run all the
>  ''t/codingstd/*.t'' tests '''except''' ''perlcritic.t''.
>
>  Am I missing something?
>
>  kid51

PERLCRITIC.T(1)       User Contributed Perl Documentation      PERLCRITIC.T(1)



NAME
       t/codingstd/perlcritic.t - use perlcritic for perl coding stds.

SYNOPSIS
        # test all files
        % prove t/codingstd/perlcritic.t

        % perl t/codingstd/perlcritic.t [--theme=sometheme]

        # test specific files
        % perl t/codingstd/perlcritic.t src/foo.pl lib/parrot/bar.pm

        # Skip perlcritic when running a testing target.
        % PARROT_TEST_NO_PERLCRITIC=1 make codetest

[jkeenan]

The patch below demonstrates new make target make moretests, which consists of the standard test suite, the examples tests, and all coding standards tests except perlcritic.t. coke++ for the hint about how to avoid running perlcritic.

I conducted this work in the tt1908/moretests_target branch, which has been pushed to github. Here is a diff of that branch against current master.

$ git diff master...tt1908/moretests_target |cat

diff --git a/config/gen/makefiles/root.in b/config/gen/makefiles/root.in
index 927e648..792836f 100644
--- a/config/gen/makefiles/root.in
+++ b/config/gen/makefiles/root.in
@@ -789,6 +789,7 @@ help :
        @echo "  buildtools_tests:  Test build tool pmc2c.pl."
        @echo "  manifest_tests:    Test code underlying mk_manifest_and_skip.pl."
        @echo "  examples_tests:    Test the example scripts."
+       @echo "  moretests:         Run standard suite, examples and most coding standards tests."
        @echo ""
        @echo "Smoke Testing:"
        @echo "  smolder_test:      Run the test suite and send report to the smolder server"
@@ -2108,6 +2109,9 @@ run_tests :
 perl_tests :
        $(PERL) t/harness $(PERL_TEST_FILES)
 
+nocritic_codetest :
+       PARROT_TEST_NO_PERLCRITIC=1 $(PERL) t/harness $(EXTRA_TEST_ARGS) --code-tests
+
 codetest :
        $(PERL) t/harness $(EXTRA_TEST_ARGS) --code-tests
 
@@ -2128,6 +2132,12 @@ manifest_tests :
 examples_tests : test_prep
        $(PERL) t/harness $(EXAMPLES_TEST_FILES)
 
+# moretests: Good to use before branch merges
+moretests: test_prep \
+    test \
+    examples_tests \
+    nocritic_codetest
+
 # benchmark tests
 benchmark_tests : test_prep
        $(PERL) t/harness $(BENCHMARK_TEST_FILES)

Please review. As the use of this target may be of benefit to the GCI students in the remaining 11 days of that project, I would like to merge it in to trunk more rapidly than I normally would.

Thank you very much.

kid51

[jkeenan]

Per discussion with whiteknight on #parrot, merged this into master at commit 76b7908. Will keep ticket open until Tuesday for comments/complaints.

Thank you very much.

kid51

[jkeenan]

Since there have been no complaints, and since it's now Tuesday in some part of the world, I'm closing this ticket.