Configure.pl relies too heavily on perl5 ccflags settings.

[doughera]

See  Original RT for more details.

[jkeenan]

This is part of our long-term technical debt. Excerpts from the discussion in the original RT:

Leopold Toetsch on Mar 31 2006:

I've tried:

$ perl Configure.pl --cc=gcc --link=gcc --ld=gcc --cxx=gcc
--verbose-step=gcc

and got:

Determining if your C compiler is actually gcc...gcc
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xarch=v8 -D_TS_ERRNO
-I./include -c test.c
gcc: language arch=v8 not recognized

From perl -V:
Compiler:
cc='cc', ccflags ='-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-xarch=v8 -D_TS_ERRNO',
optimize='-xO3 -xspace -xildoff',

That is, our config system is still using perl5 config vars blindly w/o
further inspection and ignores commandline option at least partially.

$ uname -a
SunOS z1.t2000.develooper.com 5.10 Generic_118833-03 sun4v sparc
SUNW,Sun-Fire-T200

$ perl -v
This is perl, v5.8.4 built for sun4-solaris-64int

$ cc -V
cc: Sun C 5.8 2005/10/13

$ gcc --version
gcc (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath)

Andy Dougherty subsequently explained that this is not a Solaris-specific problem:

...[B]ut Solaris is probably one of the more
popular platforms where this is appears. (The usual issue is that the
vendor supplies a perl compiled with the vendor's compiler, but an end
user wishes to use gcc.)

Andy also noted:

Anyone can check [for the existence of this problem] by editing 
their perl Config settings and seeing that they get passed through 
unquestioningly to parrot.

So, the problem with ccflags is really a subset of our larger dependency on the Perl t %Config with which we begin our configuration in the second configuration step: init::defaults. The problem will go away whenever we can figure out a way to extract ourselves from that dependency. TT #619 is a long-term roadmap ticket covering this issue and then some.

kid51

[doughera]

>  So, the problem with `ccflags` is really a subset of our larger dependency
>  on the Perl t `%Config` with which we begin our configuration in the
>  second configuration step: `init::defaults`.  The problem will go away
>  whenever we can figure out a way to extract ourselves from that
>  dependency.  TT #619 is a long-term roadmap ticket covering this issue and
>  then some.

I have never understood TT #619, particularly the bit about replacing 
reasonably portable Perl with a Unix shell script (and, presumably, 
parallel sets of hand-maintained config files for any non-unix platforms, 
such as Windows).

-- 
    Andy Dougherty		doughera@lafayette.edu

[nwellnhof]

See branch nwellnhof/compiler_flags which doesn't read ccflags from Perl's $Config hash.

[doughera]

Replying to nwellnhof:

See branch nwellnhof/compiler_flags which doesn't read ccflags from Perl's $Config hash.

Thank you for taking on this challenge. I would, however, recommend being careful not to throw out the baby with the bathwater. The perl %Config hash contains a lot of information. In the absence of any other useful information, it's probably better than nothing.

Phrased differently: What should Configure.pl do when faced with a new system for which you haven't yet put in the correct flags (e.g. there is no hints file, or the hints file has not been updated with the correct flags)? Perhaps in that situation, as long as the same compiler is being used, the perl %Config values should be used.

[nwellnhof]

IMO, the Configure script should never use flags from the Perl config hash by default. But we could add an extra option to Configure.pl to use the compiler and linker flags from Perl as default. Something like --from-perl or --perl-defaults.

[jkeenan]

Replying to nwellnhof:

IMO, the Configure script should never use flags from the Perl config hash by default.

We have to have something as default, if only because we have to have a starting point.

What would be a better choice than %Config?

kid51

[nwellnhof]

The problem with %Config is that it's completely unreliable on platforms where more than one C compiler is commonly used. Especially on Windows. I don't think we will ever have a reliable build system on Windows if we use %Config as default.

The default compiler and linker flags should simply depend on the compiler with some additional tweaks by platform. I don't see a problem with that. I don't know of any other open source project that relies on Perl5's Config to build.

[doughera]

I still don't understand why you want to throw away information even if it's the same compiler and you don't have anything else to go on, but since I'm not the one doing the work, I won't pursue it any further.

[nwellnhof]

First of all, it's a long term goal to reduce the dependency on Perl 5 in Parrot's config, build and test system.

Then, as I described above, using basically random compiler and linker flags from another project breaks if Perl was compiled with a different compiler.

Even if you use the same compiler, there are some compiler and linker flags in Perl's Config hash that Parrot simply doesn't need. Even worse, some of them cause build failures with Parrot. Have a look at all the code in the configuration system that removes certain conflicting flags. It seems that this has caused numerous build failures in that past and it might cause build failures with every new or untested Perl version.

I had a look at many configs from Smolder builds, and sometimes Parrot is linked against completely unneeded libraries just because Perl happens to use them. Do we want to special case every one of them?

If we are building with a supported compiler on a supported platform, it isn't very hard to determine the correct compiler and linker flags. So I don't see what information we are throwing away. IMO, the whole thing has been a hack from the beginning that was probably nice to get the config system started but should have been fixed a long time ago.

And as I said above, I can offer to add an option to use the Perl flags as default.

[jkeenan]

Replying to nwellnhof:

See branch nwellnhof/compiler_flags which doesn't read ccflags from Perl's $Config hash.

I'm searching for a way to see what has happened in the branch from the point at which it was created. I knew how to do this in Subversion, but that method won't work in Git. My hunch is that this was the first commit in the branch:

commit d2c2cc40f16a4aa5f7dd979808b282fb78664f02
Author: Nick Wellnhofer <wellnhofer@aevum.de>
Date:   Sat Oct 2 06:36:55 2010 +0200

    [configure] Don't use compiler and linker flags from Perl

... but I'm having trouble figuring out what the state of 'master' was at the point where this branch was created.

Suggestions?

Thank you very much.

kid51

[nwellnhof]

To see the list of commits that are new in my branch, run:

git log master..nwellnhof/compiler_flags

To see a diff of those commits, run:

git diff master...nwellnhof/compiler_flags

Notice the three dots.

Or have a look on github. Click on "Branch List" on the main page, then on "Compare" next to my branch. This should take you to the here:  https://github.com/parrot/parrot/compare/master...nwellnhof%2Fcompiler_flags

Hope this helps,

Nick

[jkeenan]

Looking at this ticket over the past few days, I have come to believe that we'll need empirical data to decide which way to proceed. At some point, it will be a judgment call as to which way of setting configuration data is 'better', but data can inform this judgment.

I wrote a program tonight, based on Parrot::Configure::Trace, which looks at how elements in the data section of the Parrot::Configure object evolve over the course of configuration. By focusing on the elements modified in the compiler_flags branch, we can see the differences in how they evolve over the course of configuration.

Please see attached patches. One is the program I wrote. The other is an edited version of the output comparing the master and nwellnhof/compiler_flags branches on Linux/i386 through the first 14 configuation steps.

Thank you very much.

kid51

[jkeenan]

'master' vs 'compiler_flags': impact on affected elements

[jkeenan]

Program used to compare branches

[jkeenan]

Replying to nwellnhof:

First of all, it's a long term goal to reduce the dependency on Perl 5 in Parrot's config, build and test system.

nwellnhof,

Can we get an update on the status of the issues discussed in this ticket and in the nwellnhof/compiler_flags branch?

Thank you very much.

kid51

[nwellnhof]

I didn't get any further feedback on the compiler_flags branch. I still think it's the right approach but we will need a lot of testing on all kinds of platforms.

[jkeenan]

Replying to nwellnhof:

I didn't get any further feedback on the compiler_flags branch. I still think it's the right approach but we will need a lot of testing on all kinds of platforms.

Thanks for responding to the cage cleaner's inquiry.

1. In order to continue work on this, I will take the ticket. I have merged master into the branch in order to capture what has been done in master since you originally created the branch.

2. I had merge conflicts in three files during that merge. The ones I'm most worried about are in config/init/hints/mswin32.pm. I fear I may have deleted work that was done after the branch point which should be retained. So more eyeballs would be helpful here.

3. I have tested the branch on linux/i386 with an all-gcc build and an all=g++ build. make test completes successfully in both cases. I am currently testing the branch on darwin/ppc with my customary mixed build, gcc for the compiler and g++ for ld and link. I expect all tests will pass except the one in t/dynpmc/select.t that is currently failing.

4. I agree with you that we will need to get this branch tested on other OSes before we can merge it, particularly MSWin32 and the Cygwin/Msys variants that are now getting some attention.

Thank you very much.

kid51

[jkeenan]

Replying to jkeenan:

3. I am currently testing the branch on darwin/ppc with my customary mixed build, gcc for the compiler and g++ for ld and link. I expect all tests will pass except the one in t/dynpmc/select.t that is currently failing.

I was too optimistic. I'm getting a build failure on Darwin. Will attach the tail of the make log. Motion backward toward Square One.

kid51

[jkeenan]

at 82fc9fd911b