Ticket #1893 (closed todo: fixed)

Opened 4 years ago

Last modified 4 years ago

Define a 'quickcover' make target

Reported by: jkeenan Owned by: jkeenan
Priority: normal Milestone: 2.11
Component: testing Version: 2.10.0
Severity: medium Keywords:
Cc: rfw, Yuki`N, fbrito, whiteknight Language:
Patch status: applied Platform:

Description

Our Google Code-In students have had several tasks to improve the coverage of our source code by our tests. However, it has become apparent to them and to me that our current make cover target takes too long to run and has a scope to broad to be useful in improving the coverage of .c and .pmc files.

So I propose a quickcover target which will implement a subset of cover. quickcover, to put it approximately, will jettison the runs of the alternate runcores as well as the tests of the Perl 5 components.

I hope to get this working tonight. Comments?

kid51

Attachments

tt1893_quickcover.aaf80082aa.diff Download (12.3 KB) - added by jkeenan 4 years ago.
diff of tt1893_quickcover branch from its branch point to date

Change History

  Changed 4 years ago by whiteknight

Strong +1 to me. I would very much love to see something like this implemented. I worry that only running a subset of tests may skew the numbers, but we should be able to mark files which are artificially under-covered during this test so people can ignore them.

  Changed 4 years ago by jkeenan

  • status changed from new to assigned
  • cc whiteknight added

whiteknight && GCI students working on code coverage tasks: Please review: parrot/tt1893_quickcover: review:  https://github.com/parrot/parrot/commit/69d6ea18ef

Thanks.

kid51

  Changed 4 years ago by jkeenan

With this commit,  https://github.com/parrot/parrot/commit/d3bdc7924d, the tt1893_quickcover branch is at what I consider to be a mergeable point.

Our existing target make cover does coverage analysis on (more or less) the same suite of tests as make fulltest. Needless to say, it takes a long time to run.

make quickcover, in contrast, only runs those files covered by t/harness --runcore-tests. Its running time is therefore more on the order of that of make test.

Both of these targets produce HTML output. For make quickcover, I have created the Configure.pl option --coveragedir. Provide an absolute path to this option to indicate the top-level of the HTML output, i.e., the directory at which coverage.html will reside. If you place this in your web server's path, you can make your coverage analysis viewable over the network.

As to merging this into master: On the one hand, we're close enough to our December release to hold off until after the release. On the other hand, the sooner it gets into master, the sooner the GCI students working on code coverage tasks can use it.

Thank you very much.

kid51

Changed 4 years ago by jkeenan

diff of tt1893_quickcover branch from its branch point to date

follow-up: ↓ 5   Changed 4 years ago by nwellnhof

+1

I'd even rename the current 'make cover' to 'make fullcover' and use quickcover when running 'make cover'.

in reply to: ↑ 4   Changed 4 years ago by jkeenan

  • type changed from RFC to todo
  • patch set to applied

Per discussion with whiteknight on IRC, I merged the branch into master. This was the major commit:  https://github.com/parrot/parrot/commit/425088ab04

Replying to nwellnhof:

+1 I'd even rename the current 'make cover' to 'make fullcover' and use quickcover when running 'make cover'.

That's a reasonable suggestion, but I'd like to defer that until a Phase II of work on the coverage targets after this week's release. Reason: I suspect that actual use will reveal bugs. We can discuss renaming one target (which is slightly more disruptive than adding a new target) as we handle those bugs.

I'll keep this ticket open until Tuesday for comments or complaints.

Thank you very much.

kid51

  Changed 4 years ago by jkeenan

  • status changed from assigned to closed
  • resolution set to fixed

Kristaba++ for adding some documentation about this in docs/dev/coverage.pod. Closing ticket.

Note: See TracTickets for help on using tickets.