test failures with t/compilers/imcc/syn/macro.t and regressions.t in fast, cgoto and switch cores

[mikehh]

since r41742 the following test failures have occurred in make fulltest:

in testf, testg and testS:

t/compilers/imcc/syn/macro.t - Failed tests: 2-4

t/compilers/imcc/syn/regressions.t - Failed test: 7

#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 28.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc -1 ((unknown file):-1)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:4\))/
# '
# './parrot --runcore=fast -D40 --gc-debug  "/home/mhh/parrot/t/compilers/imcc/syn/macro_2.pir"' failed with exit code 1

#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 41.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc -1 ((unknown file):-1)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:5\))/
# '
# './parrot --runcore=fast -D40 --gc-debug  "/home/mhh/parrot/t/compilers/imcc/syn/macro_3.pir"' failed with exit code 1

#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 53.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc -1 ((unknown file):-1)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:6\))/
# '
# './parrot --runcore=fast -D40 --gc-debug  "/home/mhh/parrot/t/compilers/imcc/syn/macro_4.pir"' failed with exit code 1


#   Failed test 'off by one error message (TT \#1016)'
#   at t/compilers/imcc/syn/regressions.t line 96.
#                   'Null PMC access in get_bool()
# current instr.: 'foo' pc -1 ((unknown file):-1)
# '
#     doesn't match '/(?s:Null PMC access in get_bool.*current instr.*:4\))/
# '
# './parrot --runcore=fast -D40 --gc-debug  "/home/mhh/parrot/t/compilers/imcc/syn/regressions_7.pir"' failed with exit code 1

the only differences in the failures is that --runcore=cgoto and --runcore=switch replaces --runcore=fast in the above 4 sub-tests.

chromatic proposed a patch which enabled the fast core tests to pass but the cgoto and switch cores still failed.

the tests were still failing at r41771

[jkeenan]

test failures on Darwin/PPC at r41837

[jkeenan]

I am confirming these failures. See attached report run on Darwin/PPC.

kid51

[jkeenan]

Confirm still failing during make fulltest on Linux/i386 at r41883.

Since these were passing in the past, they constitute blockers to our release on Tuesday October 20. I am therefore upping their priority and severity.

kid51

[bacek]

Test fixed in r41884. Resolving ticket.

[jkeenan]

Replying to bacek:

Test fixed in r41884. Resolving ticket.

Not so fast. I'm still getting the failures in make testS.

$ perl t/harness --gc-debug -S t/compilers/imcc/syn/macro.t t/compilers/imcc/syn/regressions.t 
t/compilers/imcc/syn/macro.t ........ 1/41 
#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 28.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc 0 (/home/jimk/work/parrot/t/compilers/imcc/syn/macro_2.pir:2)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:(4|-1)\))/
# '
# './parrot --runcore=switch -D40 --gc-debug  "/home/jimk/work/parrot/t/compilers/imcc/syn/macro_2.pir"' failed with exit code 1
t/compilers/imcc/syn/macro.t ........ 3/41 
#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 41.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc 0 (/home/jimk/work/parrot/t/compilers/imcc/syn/macro_3.pir:3)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:(5|-1)\))/
# '
# './parrot --runcore=switch -D40 --gc-debug  "/home/jimk/work/parrot/t/compilers/imcc/syn/macro_3.pir"' failed with exit code 1

#   Failed test 'macro, zero parameters, line number'
#   at t/compilers/imcc/syn/macro.t line 53.
#                   'Null PMC access in get_string()
# current instr.: 'test' pc 0 (/home/jimk/work/parrot/t/compilers/imcc/syn/macro_4.pir:2)
# '
#     doesn't match '/(?s:Null PMC access .*current instr.*:(6|-1)\))/
# '
# './parrot --runcore=switch -D40 --gc-debug  "/home/jimk/work/parrot/t/compilers/imcc/syn/macro_4.pir"' failed with exit code 1
t/compilers/imcc/syn/macro.t ........ 29/41 # Looks like you failed 3 tests of 41.
t/compilers/imcc/syn/macro.t ........ Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/41 subtests 
t/compilers/imcc/syn/regressions.t .. 1/17 
not ok 4 - comments between .param(TT \#1035) # TODO broken
not ok 6 - whitespace between .param(TT \#1035) # TODO broken

#   Failed test 'off by one error message (TT \#1016)'
#   at t/compilers/imcc/syn/regressions.t line 96.
#                   'Null PMC access in get_bool()
# current instr.: 'foo' pc 0 (/home/jimk/work/parrot/t/compilers/imcc/syn/regressions_7.pir:3)
# '
#     doesn't match '/(?s:Null PMC access in get_bool.*current instr.*:(4|-1)\))/
# '
# './parrot --runcore=switch -D40 --gc-debug  "/home/jimk/work/parrot/t/compilers/imcc/syn/regressions_7.pir"' failed with exit code 1
t/compilers/imcc/syn/regressions.t .. 9/17 not ok 13 - long register numbers in PASM (TT \#1025) # TODO works in PIR, not PASM
ok 16 # skip Hang on Linux i386 and amd64
# Looks like you failed 1 test of 17.
t/compilers/imcc/syn/regressions.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/17 subtests 
        (less 1 skipped subtest: 15 okay)

Test Summary Report
-------------------
t/compilers/imcc/syn/macro.t      (Wstat: 768 Tests: 41 Failed: 3)
  Failed tests:  2-4
  Non-zero exit status: 3
t/compilers/imcc/syn/regressions.t (Wstat: 256 Tests: 17 Failed: 1)
  Failed test:  7
  Non-zero exit status: 1
Files=2, Tests=58,  5 wallclock secs ( 0.05 usr  0.00 sys +  0.57 cusr  0.14 csys =  0.76 CPU)
Result: FAIL

This was on Linux/i386 at r41884. The tests are passing in the other two make targets where they were previously failing.

[mikehh]

I get the same results at r41884 on Ubuntu 9.10 (beta updated) amd64

testf and testg now PASS but testS still fails.

All the other tests PASS (pre/post-config, smoke (#29064) and rest of fulltest)

[mikehh]

ok - I fixed the remaining failing tests in testS at r41886

All tests PASS (pre/post-config, smoke (#29075), fulltest) at r41886 - Ubuntu 9.10 (beta) amd64

if jkeenan/kid51 can check that this is also ok on i386 then we can finally close the ticket

I don't know if we have had the problem on other platforms - if anyone still has any problems can they please report them here.

[jkeenan]

Replying to mikehh:

ok - I fixed the remaining failing tests in testS at r41886 All tests PASS (pre/post-config, smoke (#29075), fulltest) at r41886 - Ubuntu 9.10 (beta) amd64 if jkeenan/kid51 can check that this is also ok on i386 then we can finally close the ticket

Thanks, Mike. At r41886, I am now getting passes for both files on testf, testg and testS. Closing ticket.

kid51

[coke]

Can someone explain to me this line:

/(?s:Null PMC access in get_bool.*current instr.*:(4|-1|3)\))/ 

4 I get. It's on line 4.

-1 I get. I assume this is what we show when don't have real line numbers.

Why is 3 ok? This looks like one core was reporting the wrong value, and we just said that wrong value was OK for testing to quiet the test.

[mikehh]

Coke: mea culpa - I was essentially looking at getting the test to pass.

It actually looks as if the switch core returns the line number -1 - possibly zero based rather than unit based.

Will try and track this down further.

It might help if bacek++ and/or chromatic++ could have a look too.

[mikehh]

It looks that my initial thought on the matter were wrong - the switch core returns different values:

in t/compilers/imcc/syn/regressions.t

Null PMC access in get_bool()
current instr.: 'foo' pc 0 (/home/mhb/parrot/t/compilers/imcc/syn/regressions_7.pir:3)

where the expectation was 4 rather than 3 (off by one maybe)

but in t/compilers/imcc/syn/macro.t:

Null PMC access in get_string()

current instr.: 'test' pc 0 (/home/mhb/parrot/t/compilers/imcc/syn/macro_2.pir:2)
current instr.: 'test' pc 0 (/home/mhb/parrot/t/compilers/imcc/syn/macro_3.pir:3)
current instr.: 'test' pc 0 (/home/mhb/parrot/t/compilers/imcc/syn/macro_4.pir:2)

where the expectation was 4, 5 and 6 respectively so the off by one thought was wrong and last line pir:2 completely blew my theory away.

[mikehh]

I have reverted r41886 and marked the failing tests in testS as TODO for the moment in r41889

[mikehh]

inadvertently changed the summary

I am not sure if the priorities are correct on this at the moment - I still think this issue is very important - especially as the tests indicate that the macro line numbers are not returned by the fast and cgoto cores and are incorrect in the switch core. We need to fix the fundamental issue here - not just the tests that check it.

[mikehh]

The tests covered by this ticket now PASS and I am closing the ticket