make warnings_tests: Correct tests which run with warnings

[jkeenan]

In the course of work on TT #1922 this evening, I had occasion to run make warnings_tests for the first time. This target will FAIL test files which would otherwise PASS if they emit warnings. We should fix all tests such that they run warnings-free.

Here are the files which failed due to warnings:

Test Summary Report
-------------------
t/compilers/imcc/syn/const.t      (Wstat: 1280 Tests: 35 Failed: 5)
  Failed tests:  15-18, 21
  Non-zero exit status: 5
t/compilers/imcc/syn/errors.t     (Wstat: 1280 Tests: 7 Failed: 5)
  Failed tests:  2-5, 7
  Non-zero exit status: 5
t/compilers/imcc/syn/macro.t      (Wstat: 512 Tests: 42 Failed: 2)
  Failed tests:  31-32
  Non-zero exit status: 2
t/compilers/imcc/syn/pasm.t       (Wstat: 256 Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
t/compilers/imcc/syn/regressions.t (Wstat: 256 Tests: 21 Failed: 1)
  Failed test:  15
  Non-zero exit status: 1
t/op/basic.t                      (Wstat: 256 Tests: 23 Failed: 1)
  Failed test:  23
  Non-zero exit status: 1
t/pmc/exception-old.t             (Wstat: 256 Tests: 22 Failed: 1)
  Failed test:  16
  Non-zero exit status: 1
t/pmc/filehandle.t                (Wstat: 512 Tests: 27 Failed: 2)
  Failed tests:  3, 21
  Non-zero exit status: 2
t/pmc/nci.t                       (Wstat: 1280 Tests: 73 Failed: 5)
  Failed tests:  1-3, 60-61
  Non-zero exit status: 5
t/pmc/stringhandle.t              (Wstat: 256 Tests: 25 Failed: 1)
  Failed test:  9
  Non-zero exit status: 1
t/pmc/sub.t                       (Wstat: 768 Tests: 78 Failed: 3)
  Failed tests:  31-32, 44
  Non-zero exit status: 3
Files=205, Tests=7028, 66 wallclock secs ( 0.87 usr  0.39 sys +
 37.84 cusr 10.37 csys = 49.47 CPU)
Result: FAIL
make: *** [warnings_tests] Error 1

[jkeenan]

I spot-checked the individual test files listed above. It appears that in all, or almost all, cases, the tests that are "running with warnings" are ... tests that are testing warnings!

Which suggests that make warnings_tests is not a very useful operation. Or, at least not until we have for PIR tests something that is the equivalent of IO::CaptureOutput::capture() which we use in Perl 5 tests.

Since I doubt that many people have ever used make warnings_tests and since it would take a lot of work to make it do something useful, my recommendation is that we drop it.

Thoughts?

kid51

[cotto]

Can the getstderr and setstderr ops be used for this?

[bacek]

Replying to cotto:

Can the getstderr and setstderr ops be used for this?

Yes, it's possible. I used them in pirate test suite, afair.

[jkeenan]

Have this file pass 'make warnings_tests'

[jkeenan]

output from 'make warnings_tests' for this file

[jkeenan]

output from 'make warnings_tests' for this file

[jkeenan]

1. As of today (commit d76e2cc1) the number of tests failing under make warnings_tests is much smaller.

Test Summary Report
-------------------
t/compilers/imcc/syn/errors.t     (Wstat: 256 Tests: 8 Failed: 1)
  Failed test:  7
  Non-zero exit status: 1
t/pmc/nci.t                       (Wstat: 1280 Tests: 73 Failed: 5)
  Failed tests:  1-3, 60-61
  Non-zero exit status: 5
t/pmc/sub.t                       (Wstat: 768 Tests: 78 Failed: 3)
  Failed tests:  31-32, 44
  Non-zero exit status: 3
Files=208, Tests=7094, 71 wallclock secs ( 0.99 usr  0.42 sys + 
  40.23 cusr  9.89 csys = 51.53 CPU)
Result: FAIL
make: *** [warnings_tests] Error 1

2. I am attaching a patch which eliminates the failure in t/compilers/imcc/syn/errors.t. Please review.

3. I am attaching two other patches which display the sections of the output of 'make warnings_tests' relevant to t/pmc/nci.t and t/pmc/sub.t. I wonder: Are the tests written in such a way as to anticipate the possibility of warnings output or other output to STDERR?

[jkeenan]

Replying to jkeenan:

2. I am attaching a patch which eliminates the failure in t/compilers/imcc/syn/errors.t. Please review.

mikehh applied this particular patch in b058323a on Feb 12. So we're down to the failures in t/pmc/nci.t and t/pmc/sub.t.