Ticket #380 (closed bug: fixed)

Opened 4 years ago

Last modified 4 years ago

t/pmc/exceptionhandler.t - Bad plan. You planned 8 tests but ran 6

Reported by: mikehh Owned by:
Priority: normal Milestone:
Component: core Version: trunk
Severity: medium Keywords: pmc
Cc: Language:
Patch status: Platform:

Description

This test started failing between r36930 and r36946.

It passes at build r36930:

1..8
ok 1 - Instantiated ExceptionHandler
ok 2 - Min and Max severity for exception handlers
ok 3 - Min and Max severity for exception handlers
ok 4 - Exception Handler type checks work
ok 5 - Exception Handler type checks work
ok 6 - Exception Handler subclass popped
ok 7 - Exception Handler subclass with can_handle method catch exception
ok 8 - Exception Handler subclass catch exception

but does not run the last two tests after r36946

1..8
ok 1 - Instantiated ExceptionHandler
...
ok 6 - Exception Handler subclass popped

mhk@mhk-desktop:~/parrot$ prove --verbose t/pmc/exceptionhandler.t
t/pmc/exceptionhandler.t ..
1..8
ok 1 - Instantiated ExceptionHandler
ok 2 - Min and Max severity for exception handlers
ok 3 - Min and Max severity for exception handlers
ok 4 - Exception Handler type checks work
ok 5 - Exception Handler type checks work
ok 6 - Exception Handler subclass popped
Failed 2/8 subtests

Test Summary Report

t/pmc/exceptionhandler.t (Wstat: 11 Tests: 6 Failed: 0)

Non-zero wait status: 11
Parse errors: Bad plan. You planned 8 tests but ran 6.

Files=1, Tests=6, 0 wallclock secs ( 0.03 usr 0.02 sys + 0.01 cusr 0.01 csys = 0.07 CPU)
Result: FAIL

This happens on i386 Linux and AMD64 Linux.

I looked through the posts for r36931 through r36946 but could not see anything obvious.

Will continue checking.

Change History

  Changed 4 years ago by mikehh

I think I should have marked this as trunk rather than 0.9.1

in reply to: ↑ description   Changed 4 years ago by jkeenan

  • keywords pmc added
  • version changed from 0.9.1 to trunk
  • component changed from none to core

I could not reproduce this failure at r37017 (tested on Linux/i386). 

$ prove -v t/pmc/exceptionhandler.t 
t/pmc/exceptionhandler....
1..8
ok 1 - Instantiated ExceptionHandler
ok 2 - Min and Max severity for exception handlers
ok 3 - Min and Max severity for exception handlers
ok 4 - Exception Handler type checks work
ok 5 - Exception Handler type checks work
ok 6 - Exception Handler subclass popped
ok 7 - Exception Handler subclass with can_handle method catch exception
ok 8 - Exception Handler subclass catch exception
ok
All tests successful.
Files=1, Tests=8,  0 wallclock secs ( 0.01 usr  0.00 sys +  0.03 cusr  0.01 csys =  0.05 CPU)
Result: PASS

Can you re-test? Thank you very much.
kid51

  Changed 4 years ago by mikehh

Ok - it's the --optimize build that is the problem here.

I was in error in saying that it failed on my AMD64 build - it passes there.

I did a new checkout and built the system again.

I ran:
perl Configure.pl --optimize --test
make world
prove -v t/pmc/exceptionhandler.t
t/pmc/exceptionhandler.t ..
1..8
ok 1 - Instantiated ExceptionHandler
ok 2 - Min and Max severity for exception handlers
ok 3 - Min and Max severity for exception handlers
ok 4 - Exception Handler type checks work
ok 5 - Exception Handler type checks work
ok 6 - Exception Handler subclass popped
Failed 2/8 subtests

OK, then I did a clean checkout again:

perl Configure.pl --test
make world
prove --verbose t/pmc/exceptionhandler.t
t/pmc/exceptionhandler.t ..
1..8
ok 1 - Instantiated ExceptionHandler
ok 2 - Min and Max severity for exception handlers
ok 3 - Min and Max severity for exception handlers
ok 4 - Exception Handler type checks work
ok 5 - Exception Handler type checks work
ok 6 - Exception Handler subclass popped
ok 7 - Exception Handler subclass with can_handle method catch exception
ok 8 - Exception Handler subclass catch exception
ok
All tests successful.
Files=1, Tests=8, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.04 cusr 0.00 csys = 0.07 CPU)
Result: PASS

So it looks to me that this is due to a failure in the optimizer - i.e. running gcc with the -O2 flag set.

There seem to be no other failures in the test suite.

I will continue investigating.

  Changed 4 years ago by mikehh

I was wrong - there was a failure in the test suite.

Test Summary Report

t/dynpmc/foo.t (Wstat: 256 Tests: 9 Failed: 1)

Failed test: 3
Non-zero exit status: 1

Files=396, Tests=11702, 362 wallclock secs ( 4.11 usr 1.10 sys + 137.80 cusr 45.93 csys = 188.94 CPU)
Result: FAIL

I have had the following results:

Linux i386 With --optimize flag - r37017:

 http://smolder.plusthree.com/app/public_projects/report_details/18353

11,700 ok, 0 failed, 250 todo, 570 skipped and 0 unexpectedly succeeded

Linux AMD64 (no --optimize flag) - r37017:

 http://smolder.plusthree.com/app/public_projects/report_details/18357

11,703 ok, 0 failed, 250 todo, 570 skipped and 0 unexpectedly succeeded

Linux i386 (no --optimize flag) - r37017:

 http://smolder.plusthree.com/app/public_projects/report_details/18361

11,701 ok, 1 failed, 250 todo, 570 skipped and 0 unexpectedly succeeded

I did a make realclean, perl Configure --test, make world, make smoke

 http://smolder.plusthree.com/app/public_projects/report_details/18362

11,701 ok, 1 failed, 250 todo, 570 skipped and 0 unexpectedly succeeded

This worries me a bit.

Lots of tests - something seriously wrong - don't use it.

In one situation we have - on the same platform - different test results with or without the optimize flag in use - BUT - only two test files out of 390 - three tests out of 11702.

The test that fails passes on AMD64 (without the optimize which I haven't tested this week but was still failing last week - the optimize that is, but, a little further down the line).

  Changed 4 years ago by mikehh

This appears to be fixed now - as of r37030 the test passes with the --optimize flag set.

 http://smolder.plusthree.com/app/public_projects/report_details/18380

I still have a problem as reported in the previous comment.

Perhaps the ticket can now be closed, although the reason still seems a mystery to me.

  Changed 4 years ago by jkeenan

  • status changed from new to closed
  • resolution set to wontfix

  Changed 4 years ago by coke

  • status changed from closed to reopened
  • resolution wontfix deleted

"I still have a problem as reported in the previous comment." - This problem needs to be singled out and addressed before we close the ticket.

follow-up: ↓ 9   Changed 4 years ago by mikehh

The original reported problem now seems to have cleared up.

However my build without the optimize flag is failing t/dynpmc/foo.t 03

 http://smolder.plusthree.com/app/public_projects/report_details/18419

as opposed to my build with --optimize which passes that test

 http://smolder.plusthree.com/app/public_projects/report_details/18415

both builds fail t/pmc/packfileconstanttable.t (failed test 3), but then so are the other smoke tests at the moment

I did a make realclean, perl Configure.pl, make, make test and the test failed.

I then did a new svn co, perl Configure.pl, make, make smoke and the test failed. That is smoke test 18419 at r37050.

None of the other smoke tests seem to fail this test.

I am running on Kubuntu Intrepid i386 with all recent updates and the same for CPAN. It does not fail the test on Kubuntu Intrepid AMD64 either.

I can't think of any settings that would cause this, or what I could have done that would cause the test to pass when I run perl Configure --optimise as opposed to perl Configure.pl

I have included the prove results for the tests without the optimize and with. 

mhk@mhk-desktop:~/parrot.nopt$ prove --verbose t/dynpmc/foo.t
t/dynpmc/foo.t ..
1..9
ok 1 - get_integer
ok 2 - loadlib with relative pathname, no ext
not ok 3 - loadlib with absolute pathname, no ext

#   Failed test 'loadlib with absolute pathname, no ext'
#   at t/dynpmc/foo.t line 57.
# Exited with error code: 1
# Received:
# Class 'Foo' not found
# current instr.: 'main' pc 16 (/home/mhk/parrot.nopt/t/dynpmc/foo_3.pir:15)
#
# Expected:
# 42
#
ok 4 - loadlib with relative pathname & ext
ok 5 - loadlib with absolute pathname & ext
ok 6 - inherited add
ok 7 - Foo subclass isa Integer
ok 8 - .HLL 1
ok 9 - .HLL 2
# Looks like you failed 1 test of 9.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests

Test Summary Report
-------------------
t/dynpmc/foo.t (Wstat: 256 Tests: 9 Failed: 1)
  Failed test:  3
  Non-zero exit status: 1
Files=1, Tests=9,  0 wallclock secs ( 0.05 usr  0.00 sys +  0.20 cusr  0.14 csys =  0.39 CPU)
Result: FAIL
mhk@mhk-desktop:~/parrot.nopt$


mhk@mhk-desktop:~/parrot$ prove --verbose t/dynpmc/foo.t
t/dynpmc/foo.t ..
1..9
ok 1 - get_integer
ok 2 - loadlib with relative pathname, no ext
ok 3 - loadlib with absolute pathname, no ext
ok 4 - loadlib with relative pathname & ext
ok 5 - loadlib with absolute pathname & ext
ok 6 - inherited add
ok 7 - Foo subclass isa Integer
ok 8 - .HLL 1
ok 9 - .HLL 2
ok
All tests successful.
Files=1, Tests=9,  1 wallclock secs ( 0.04 usr  0.01 sys +  0.21 cusr  0.12 csys =  0.38 CPU)
Result: PASS
mhk@mhk-desktop:~/parrot$

I have no idea at the moment how to proceed here. I will keep working on it.

As regards this ticket the original problem I reported seems to have gone away.

Should a new ticket be created?

Has anybody else had a problem in this area?

in reply to: ↑ 8   Changed 4 years ago by jkeenan

Replying to mikehh:

The original reported problem now seems to have cleared up. As regards this ticket the original problem I reported seems to have gone away. Should a new ticket be created?

Since the problem with exceptionhandler.t has cleared up, it would be misleading to discuss a failure in a different test in this ticket.

So, yes, it would be good to start a separate ticket for foo.t problems, so that we can close this ticket. Could you repost your prove output in the new ticket?

Thank you very much.
kid51

follow-up: ↓ 11   Changed 4 years ago by mikehh

I have isolated the problem with the t/dynpmc/foo.t test

It has nothing to do with the --optimize flag at all - although the original stated problem, I think, did.

In essence the test in question is related to no extension. I did however, have an extension on the parrot directory as in parrot.noopt which it seems to find and hence fails the test. I will create a new ticket for this - hopefully with a patch, a little later on.

This ticket may now be closed.

in reply to: ↑ 10   Changed 4 years ago by jkeenan

  • status changed from reopened to closed
  • resolution set to fixed

Replying to mikehh:

I have isolated the problem with the t/dynpmc/foo.t test

Thanks for your thorough investigation of these issues. Closing ticket.

Note: See TracTickets for help on using tickets.