Ticket #1189 (closed todo: fixed)
t/steps/*/*.t: Replace reliance on init::defaults
|Reported by:||jkeenan||Owned by:||jkeenan|
This ticket moves to Trac discussion of issues found in several tickets in the older RT system, principally:
* t/configure/1*.t tests incorrectly rely on init::defaults, opened by Andy Dougherty in November 2007.
* Remove config::init::defaults From configure tests, opened by chromatic, also in November 2007.
Discussion in these RTs was, to say the least, extensive. Let me quote from the original posts in the two tickets to give a flavor of the issues. The tests in question have moved to a different directory structure since these RTs were originally filed. I've reflected those file movements in the following citations.
[The tests in t/steps/] rely on init::defaults. That simply won't work if the configuration being used to build parrot is sufficiently different from that which was used to build perl. That's what happened to me in this case.
This probably doesn't show up often because nearly everyone building parrot right now uses the same configuration as was used to build perl. However, that's not necessarily the norm. Vendors such as Sun supply a perl pre-built with their compiler, but many end users have gcc installed instead.
If a test for a configuration step relies on certain Config values, the only correct approach is to use the Config values determined by Configure.pl. There is no guarantee that the init::defaults values will work. They may well have been changed, either by the user (via --ask or command line overrides) or by the Configure.pl system itself.
config::init::defaults pulls the libs setting out of the Perl 5 configuration and various tests of the Parrot configuration process use config::init::defaults to set up the environment for testing.
When these tests attempt to compile and link programs, they may fail because the Perl 5 configuration may not reflect the actual run-time environment of the code.
Parrot::Config::Generated is a much more reliable source of information, and if the tests truly need information about the local system for compiling and linking purposes, they should fetch the information from there, not from the Perl 5 configuration which does not necessarily reflect the state of the local machine.
I can imagine an objection to this suggestion, specifically But these tests should be runnable without having previously configured Parrot! We cannot rely on the configuration process working correctly unless we can test that process'' [Comment: This has indeed been my basic objection -- kid51] Yet if we cannot rely on the accuracy of the tests, they provide little value. Clearly Parrot's configuration process works on my machine, as demonstrated by the fact that Parrot builds successfully and passes all functional tests. Yet the configuration test fails with a false negative.
To get the maximum value out of the tests of the configuration system, the tests must be robust and reliable. Failures should indicate actual problems of behavior. I don't believe that we can rely on these tests until we remove the assumption that the local Perl 5 configuration is correct for Parrot.
Thank you very much.