Define a 'quickcover' make target

[jkeenan]

Our Google Code-In students have had several tasks to improve the coverage of our source code by our tests. However, it has become apparent to them and to me that our current make cover target takes too long to run and has a scope to broad to be useful in improving the coverage of .c and .pmc files.

So I propose a quickcover target which will implement a subset of cover. quickcover, to put it approximately, will jettison the runs of the alternate runcores as well as the tests of the Perl 5 components.

I hope to get this working tonight. Comments?

kid51

[whiteknight]

Strong +1 to me. I would very much love to see something like this implemented. I worry that only running a subset of tests may skew the numbers, but we should be able to mark files which are artificially under-covered during this test so people can ignore them.

[jkeenan]

whiteknight && GCI students working on code coverage tasks: Please review: parrot/tt1893_quickcover: review:  https://github.com/parrot/parrot/commit/69d6ea18ef

Thanks.

kid51

[jkeenan]

With this commit,  https://github.com/parrot/parrot/commit/d3bdc7924d, the tt1893_quickcover branch is at what I consider to be a mergeable point.

Our existing target make cover does coverage analysis on (more or less) the same suite of tests as make fulltest. Needless to say, it takes a long time to run.

make quickcover, in contrast, only runs those files covered by t/harness --runcore-tests. Its running time is therefore more on the order of that of make test.

Both of these targets produce HTML output. For make quickcover, I have created the Configure.pl option --coveragedir. Provide an absolute path to this option to indicate the top-level of the HTML output, i.e., the directory at which coverage.html will reside. If you place this in your web server's path, you can make your coverage analysis viewable over the network.

As to merging this into master: On the one hand, we're close enough to our December release to hold off until after the release. On the other hand, the sooner it gets into master, the sooner the GCI students working on code coverage tasks can use it.

Thank you very much.

kid51

[jkeenan]

diff of tt1893_quickcover branch from its branch point to date

[nwellnhof]

+1

I'd even rename the current 'make cover' to 'make fullcover' and use quickcover when running 'make cover'.

[jkeenan]

Per discussion with whiteknight on IRC, I merged the branch into master. This was the major commit:  https://github.com/parrot/parrot/commit/425088ab04

Replying to nwellnhof:

+1 I'd even rename the current 'make cover' to 'make fullcover' and use quickcover when running 'make cover'.

That's a reasonable suggestion, but I'd like to defer that until a Phase II of work on the coverage targets after this week's release. Reason: I suspect that actual use will reveal bugs. We can discuss renaming one target (which is slightly more disruptive than adding a new target) as we handle those bugs.

I'll keep this ticket open until Tuesday for comments or complaints.

Thank you very much.

kid51

[jkeenan]

Kristaba++ for adding some documentation about this in docs/dev/coverage.pod. Closing ticket.