t/compilers/tge/grammar.t: FAIL under 'make test', but passes with 'prove': Darwin

[jkeenan]

Running make test on Darwin/PPC, I am getting an overall result of FAIL due to the following result:

t/compilers/tge/grammar.t ...................
Dubious, test returned 1 (wstat 256, 0x100)Failed 1/3 subtests

However, when I run this file by itself with prove -v, I get this:

[parrot] 507 $ prove -v t/compilers/tge/grammar.t
t/compilers/tge/grammar.t .. 
1..3
ok 1 - test compiling anonymous and named grammars
ok 2 - complete example: Branch/Leaf tree grammar
not ok 3 - two rules of the same name can apply to the same node, when called with a different dummy type # TODO unresolved bug
#   Failed (TODO) test 'two rules of the same name can apply to the same node, when called with a different dummy type'
#   at t/compilers/tge/grammar.t line 279.
# Exited with error code: 1
# Received:
# error:imcc:syntax error, unexpected IDENTIFIER, expecting INTV or FLOATV or STRINGV or PMCV ('object')
#       in file '/Users/jimk/work/parrot/t/compilers/tge/grammar_3.pir' line 24
# error:imcc:syntax error, unexpected '=', expecting '(' ('=')
#       in file '/Users/jimk/work/parrot/t/compilers/tge/grammar_3.pir' line 25
# error:imcc:syntax error, unexpected IDENTIFIER, expecting COMMA or ')' ('testing')
#       in file '/Users/jimk/work/parrot/t/compilers/tge/grammar_3.pir' line 34
# 
# Expected:
# in tiddlywinks
# in first twister
# in second twister
# 
ok
All tests successful.
Files=1, Tests=3,  5 wallclock secs ( 0.10 usr  0.05 sys +  0.87 cusr  0.28 csys =  1.30 CPU)
Result: PASS

I first observed this at r41089 on Sept 06. My last previous PASS on make test on this box was at r40966 on Sept 05.

I am not observing this failure on my Linux/i386 box. The test file itself has not been modified in any way since March.

Can anyone suggest an explanation? Thank you very much.

kid51

[jkeenan]

On further inspection of my Smolder results, I see that the 'dubious result' is not referring to some misinterpretation of the TODO by the test harness.

Rather, the previous test in this file is exiting prematurely. See  this Smolder result or  this earlier one.

1..3
ok 1 - test compiling anonymous and named grammars
not ok 2 - complete example: Branch/Leaf tree grammar

#   Failed test 'complete example: Branch/Leaf tree grammar'
#   at t/compilers/tge/grammar.t line 76.
# Exited with error code: [SIGNAL 10]
# Received:
# the global minimum attribute value is: 1 of type: Integer
# before transform, the value of the left-most leaf is: 2
# after transform, the value of the left-most leaf is: 1
# before transform, the value of the right-most leaf is: 9
# after transform, the value of the right-most leaf is: 1
# 
# Expected:
# the global minimum attribute value is: 1 of type: Integer
# before transform, the value of the left-most leaf is: 2
# after transform, the value of the left-most leaf is: 1
# before transform, the value of the right-most leaf is: 9
# after transform, the value of the right-most leaf is: 1
# 

So why would this be exiting with SIGNAL 10? Any why only on Darwin/PPC? (We have a pass at the same revision on Darwin/i386.)

The plot thickens.

[dukeleto]

Do you have an installed parrot that is mucking things up? "make" prefers the current build directory for loading shared libraries, prove might be using an installed version.

[jkeenan]

Replying to dukeleto:

Do you have an installed parrot that is mucking things up?

No, I have never actually installed Parrot.

I can reproduce the problem as follows:

$ perl t/harness --gc-debug t/compilers/tge/grammar.t 
t/compilers/tge/grammar.t .. 2/3 
#   Failed test 'complete example: Branch/Leaf tree grammar'
#   at t/compilers/tge/grammar.t line 76.
# Exited with error code: [SIGNAL 10]
# Received:
# the global minimum attribute value is: 1 of type: Integer
# before transform, the value of the left-most leaf is: 2
# after transform, the value of the left-most leaf is: 1
# before transform, the value of the right-most leaf is: 9
# after transform, the value of the right-most leaf is: 1
# 
# Expected:
# the global minimum attribute value is: 1 of type: Integer
# before transform, the value of the left-most leaf is: 2
# after transform, the value of the left-most leaf is: 1
# before transform, the value of the right-most leaf is: 9
# after transform, the value of the right-most leaf is: 1
# 

not ok 3 - two rules of the same name can apply to the same node, when called with a different dummy type # TODO unresolved bug
# Looks like you failed 1 test of 3.
t/compilers/tge/grammar.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests 

Test Summary Report
-------------------
t/compilers/tge/grammar.t (Wstat: 256 Tests: 3 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
Files=1, Tests=3,  4 wallclock secs ( 0.07 usr  0.02 sys +  1.82 cusr  0.30 csys =  2.21 CPU)
Result: FAIL

[jkeenan]

Replying to jkeenan:

Still more analysis shows that this only occurs when t/harness is called with the --gc-debug option -- which, of course, is what make test always does:

$> perl t/harness  t/compilers/tge/grammar.t
t/compilers/tge/grammar.t .. 2/3 
not ok 3 - two rules of the same name can apply to the same node, when called with a different dummy type # TODO unresolved bug
t/compilers/tge/grammar.t .. ok   
All tests successful.
Files=1, Tests=3,  2 wallclock secs ( 0.08 usr  0.02 sys +  0.86 cusr  0.27 csys =  1.23 CPU)
Result: PASS

Compare with previous post, where --gc-debug was called.

So what would cause a test to halt prematurely with [SIGNAL 10] when called with --gc-debug, but not without it?

Thank you very much.
kid51

[jkeenan]

Replying to jkeenan:

This bug appears to have cleared up. Observed at r41167:

$ perl t/harness --gc-debug t/compilers/tge/grammar.t
t/compilers/tge/grammar.t .. 3/3 
not ok 3 - two rules of the same name can apply to the same node, when called with a different dummy type # TODO unresolved bug
t/compilers/tge/grammar.t .. ok   
All tests successful.
Files=1, Tests=3,  5 wallclock secs ( 0.08 usr  0.04 sys +  2.38 cusr  0.36 csys =  2.86 CPU)
Result: PASS

See also  this Smolder report on Darwin/PPC.

Unfortunately, I cannot yet say when it cleared up or why it cleared up. I can say that my best estimate of the point of failure is that it occurred at r41087, when config/gen/platform/darwin/hires_timer.c was introduced. But since the build was broken from r41081 (Merge pluggable_runcore branch into trunk.) until approx. r41087, I cannot pinpoint the failure as I normally would.

So I am going to first take this ticket, cc-ing contributors who I believe are interested in the hi-res timer issue, and then close it. And then hope that it stays closed.

Thank you very much.
kid51

[jkeenan]

Well, that didn't stay closed very long. When I went to check my own Smolder report (cited above), I noticed a  failure submitted on Darwin/PPC just minutes earlier, with the only difference being that it was configured with Perl 5.8.8 rather than 5.10.1.

Sigh!

[dukeleto]

I am also still seeing this on my darwin/ppc smoke box:  http://smolder.plusthree.com/app/public_projects/report_details/27120#first_failure

[jkeenan]

bacek++ TODO-ed the two failing tests, so they did not constitute an impediment to today's release. But, of course, the underlying problem is unresolved.

[jkeenan]

TT #1160 is tracking the TODO in failure in 'two rules of the same name'. So we don't need this ticket open as well. Closing.

kid51