config/init/defaults.pm: Profiling options are too specific to GCC

[jkeenan]

This ticket was created in  RT #41497 by Paul T Cochrane in February 2007. I am moving it into the Trac system to raise its visibility. I believe it could be moved toward resolution by someone -- a newcomer, perhaps -- who was willing to spend some time doing Internet searching on the profiling options of various C compilers.

Original description:
The profiling options used in config/init/defaults.pm are specific to gcc. This should probably be specified in the relevant hints file.

The relevant code in config/init/defaults.pm is this:

    if ( $conf->options->get('profile') ) {
        $conf->data->set(
            cc_debug => " -pg ",
            ld_debug => " -pg ",
        );
    }

In the original ticket, Infinoid subsequently commented: This flag convinces gcc to generate profiling code specifically for the GNU profiler, gprof. I find it *extremely* unlikely that this flag would be available and/or mean the same thing for other compilers ...

In addition, Andy Dougherty commented: [These options] are gcc-specific. A plain -p is common on Unix systems, but -pg is gcc-specific. For example, with Sun's C compiler, -xpg would be the equivalent flag. (Though Sun's C compiler also has other profile options selected by -xprofile ...)

Could someone research profiling options on a variety of C compilers, including the principal C compilers for Windows?

Thank you very much.
kid51

[jkeenan]

Note: There is a  closed RT ticket, #41496, which called for putting determination of compiler-specific profiling values in its own configuration step. I closed that ticket because we seemed to be getting along fine without it. But if we do get a better idea of what other C compilers need in that regard, we can consider creating a specific configuration step. Or perhaps it can go into config/inter/progs.pm.

kid51

[jkeenan]

Researching this older ticket, I found the following information for the Intel C compiler:

Windows:       /Qprof-gen
Mac and Linux: -prof_gen

Sources:  Wikipedia and  software.intel.com

Can someone verify that these would be appropriate?

Thank you very much.

kid51

[jkeenan]

Researching this older ticket, I found the following information for Solaris and Sun Studio: -g

Sources:  Sun Solaris docs and  Sun Studio docs

Can someone verify that these would be appropriate?

Thank you very much.

kid51

[jkeenan]

For the record, here's what the GCC docs have to say about this (excerpts):

-g

Produce debugging information in the operating system's native 
format (stabs, COFF, XCOFF, or DWARF 2). GDB can work with this 
debugging information.

-p

Generate extra code to write profile information suitable for 
the analysis program prof. 

-pg

Generate extra code to write profile information suitable for 
the analysis program gprof.

Sources:  gcc Debugging Options.

[doughera]

Replying to jkeenan:

Researching this older ticket, I found the following information for Solaris and Sun Studio: -g Sources:  Sun Solaris docs and  Sun Studio docs Can someone verify that these would be appropriate?

It all depends on what you want to do. There are different things one might do, all of which might be considered "profiling". For Sun Studio compilers, -g will produce data suitable for analysis with the analyzer(1) program. -xpg will produce data suitable for prof or gprof . -xprofile={collect|use} will collect and use data suitable for the compiler itself to use for optimizations, and -xprofile=tcov will collect data suitable for tcov(1) . All of these should be specified both at the compile phase and at the link phase.

Note too that these flags depend on the compiler, not on the operating system, so they don't really belong in an operating-system-specific hints file. (This would also be the case for Intel's compiler.) If Configure.pl wants to test for them at all, it should be after the compiler is known.

[jkeenan]

Replying to doughera:

How about if, on the KISS principle, we said that for Parrot at this stage of its development, we defined profiling as "... [producing] data suitable for analysis with prof or gprof"?

[doughera]

Replying to jkeenan:

How about if, on the KISS principle, we said that for Parrot at this stage of its development, we defined profiling as "... [producing] data suitable for analysis with prof or gprof"?

Sounds sensible to me. For Sun Studio, all three flags, cc_debug, ld_debug, and link_debug, should be set to -xpg. For gcc, all three should be set to -pg. Since you don't know which compiler you have until the config::inter:progs step, this code should probably all be moved there from its current home of init::defaults.

[jkeenan]

Replying to doughera:

Since you don't know which compiler you have until the 
config::inter:progs step, this code should probably all 
be moved there from its current home of init::defaults.

Andy, I noticed that in inter::progs we already have some code for doing a "debugging build" of Parrot:

    143 sub _get_debug {
    144     my ($conf, $ask) = @_;
    145     my $debug = 'n';
    146     $debug = 'y' if $conf->options->get('debugging');
    147     $debug = prompt( "Do you want a debugging build of Parrot?", $debug         )
    148         if $ask;
    149     return $debug;
    150 }
...
    157 sub _set_debug_and_warn {
    158     my ($conf, $debug) = @_;
    159     if ( $debug =~ /n/i ) {
    160         $conf->data->set( 
    161             cc_debug   => '',
    162             link_debug => '',
    163             ld_debug   => ''
    164         );
    165     }

So it seems that whatever is currently being set in init::defaults, it is being overwritten in inter::progs.

Is there any meaningful distinction between the profile configuration option handled in init::defaults and the debugging option handled in inter::progs? Couldn't we collapse the two into each other? And if we do so, which should be the name of the option: 'profile' or 'debugging'?

(Also will the regex m/sun|solaris/i suffice for recognizing those compilers?)

Thank you very much.

kid51

[doughera]

Replying to jkeenan:

Is there any meaningful distinction between the profile configuration option handled in init::defaults and the debugging option handled in inter::progs? Couldn't we collapse the two into each other? And if we do so, which should be the name of the option: 'profile' or 'debugging'?

I would favor collapsing the two into "debugging", even though they are logically distinct. There are very many different things one might imagine asking a compiler to do; it doesn't make sense to make up separate Configure.pl ones for each. Ultimately, as long as there is some way to pass arbitrary options to the compiler through Configure.pl, people will be able to do whatever it is they are trying to do.

[doughera]

Replying to jkeenan:

(Also will the regex m/sun|solaris/i suffice for recognizing those compilers?)

Running

    $cc -V

on Solaris 10 produces this output:

cc: Sun C 5.11 SunOS_i386 2010/08/13
usage: cc [ options ] files.  Use 'cc -flags' for details

[jkeenan]

The more I look at this ticket, the more of a morass it seems. Assignments to the various *_debug attributes are strewn over several configuration steps.

find config -type f | xargs grep -n debug | grep -vE '(debug\(|/gen)'
config/init/defaults.pm:90:        debugging => $conf->options->get('debugging') ? 1 : 0,
config/init/defaults.pm:140:        cc_debug   => '-g',
config/init/defaults.pm:141:        link_debug => '',
config/init/defaults.pm:161:        # include debug info in executable
config/init/defaults.pm:162:        ld_debug => '',
config/init/defaults.pm:245:        # debugging internals.
config/init/defaults.pm:256:#            cc_debug => " -pg ",
config/init/defaults.pm:257:#            ld_debug => " -pg ",
config/init/hints/mswin32.pm:91:            cc_debug            => '-Zi',
config/init/hints/mswin32.pm:92:            ld_debug            => '-debug',
config/init/hints/mswin32.pm:125:        # We can't use -opt: and -debug together.
config/init/hints/mswin32.pm:126:        if ( $conf->data->get('ld_debug') =~ /-debug/ ) {
config/init/hints/mswin32.pm:143:            cc_debug            => '-Zi',
config/init/hints/mswin32.pm:146:            ld_debug            => '-debug',
config/init/hints/mswin32.pm:164:        # We can't use -opt: and -debug together.
config/init/hints/mswin32.pm:165:        if ( $conf->data->get('ld_debug') =~ /-debug/ ) {
config/init/hints/mswin32.pm:182:            cc_debug   => '-v',
config/init/hints/mswin32.pm:188:            ld_debug       => '-v',
config/inter/progs.pm:10:lexical analyzer generator and parser generator to use. Also whether debugging
config/inter/progs.pm:42:    return unless defined $debug;
config/inter/progs.pm:144:sub _get_debug {
config/inter/progs.pm:146:    my $debug = 'n';
config/inter/progs.pm:147:    $debug = 'y' if $conf->options->get('debugging');
config/inter/progs.pm:148:    $debug = prompt( "Do you want a debugging build of Parrot?", $debug )
config/inter/progs.pm:150:    ( $debug =~ /^[yn]$/i ) ? return $debug : return;
config/inter/progs.pm:153:sub _set_debug {
config/inter/progs.pm:154:    my ($conf, $debug) = @_;
config/inter/progs.pm:155:    if ( $debug =~ /n/i ) {
config/inter/progs.pm:157:            cc_debug   => '',
config/inter/progs.pm:158:            link_debug => '',
config/inter/progs.pm:159:            ld_debug   => ''
config/init/optimize.pm:68:    # disable debug flags.
config/init/optimize.pm:69:    $conf->data->set( cc_debug => '' );

Let's translate this into humanspeak. init::defaults sets a key debugging if the command-line option debugging has been set. Then, regardless of the setting of debugging, it sets -g as the default value for cc_debug, an empty string as the default value for link_debug and ld_debug. Then, if command-line option profile has been set, -pg is set as the value for cc_debug and ld_debug -- but not link_debug! (I'm looking at code in the tt855_debugging_options branch here.)

Next, if and only if you're on Win32, init::hints::mswin32 sets values for cc_debug and ld_debug depending -- to oversimplify -- on which C compiler you're using. It also makes use of -opt and -debug mutually exclusive.

Next, inter::progs allows interactive setting of debugging options. Or, to be more precise, it permits the user to interactively request debugging. If at this point the user explicitly defines a debugging build, all previous settings to the three attributes are wiped out.

Finally, in init::optimize, Configure.pl asks whether the user has requested an optimized build. If so, then cc_debug is set to an empty string, regardless of any earlier settings -- but link_debug and ld_debug are untouched.

Confused? You're not alone.

One thing that has become apparent to me is that we have to distinguish between the user's request for a debugging build and the assignment of correct values to the 3 debugging attributes.

Another thing that has become apparent to me is that we have one determination of which operating system we're on in or prior to init::hints, but we do not set definitive values for osname until a later configuration step, auto::arch.

That's where my understanding of the issues currently stands. Thank you very much.

kid51

[jkeenan]

I've stared at this ticket from time to time over the past two months. Some further observations:

1. It's interesting that the config step which makes the final determination re a debugging build, init::optimize (step 17), occurs long after the next-to-last step in this process, inter::progs (step 5).

2. It's also interesting that the POD to config/init/optimize.pm asks the question:

Should this be part of config/inter/progs.pm? XXX

If the answer to that question were Yes, then we could handle our debugging and optimization concerns much earlier in the process.

3. Unfortunately, init::optimize relies on two config data that are determined between steps 5 and 15:

a. Are we using gcc and, if so, which version?

    my $gccversion = $conf->data->get( 'gccversion' );

(determined in Step 9, auto::gcc)

b. Are we on AMD64?

    if ($conf->data->get('cpuarch') eq 'amd64') {
        $conf->data->set('optimize::src/gc/system.c','');
    }

(cpuarch is determined in Step 15, auto::arch)

4. This raises the possibility that the best way to proceed might be to use step 17 more explicitly for making the final determinations for both debugging and optimization. After all, if we're requesting optimization, we don't want the debugging information. And (I think) vice versa.

That would imply consolidating assignments to cc_debug, ld_debug and link_debug in step 17 -- and not doing them earlier. What we would do earlier is to define appropriate values for appropriate OS/compiler/platform combinations.

How does that sound?

Thank you very much.

kid51

[jkeenan]

Follow-up question:

If a user has requested an optimized build, does it make any sense to have values for link_debug and ld_debug? In the optimize case, shouldn't they be set to empty strings just like cc_debug?

Thank you very much.

kid51

[darbelo]

Replying to jkeenan:

Follow-up question: If a user has requested an optimized build, does it make any sense to have values for link_debug and ld_debug? In the optimize case, shouldn't they be set to empty strings just like cc_debug?

The answer to that depends on wether you consider building an optimized parrot should exclude the possibility of building a debug-able parrot. While it's true that the configure system has no direct provisions for this, it is (or used to be, I haven't tried in som time) relatively simple to build an optimized parrot with debugging symbols. IIRC chromatic used to do this regularly so that he could get line number information out of gdb when debugging optimized builds, it's likely we should take this use case into consideration when evaluating the impact of changes to the debug flags in optimized builds.

[coke]

On Mon, Jan 3, 2011 at 2:12 PM, Parrot <parrot-tickets@lists.parrot.org> wrote:
> #855: config/init/defaults.pm:  Profiling options are too specific to GCC
> -----------------------+----------------------------------------------------
>  Reporter:  jkeenan    |       Owner:  jkeenan
>     Type:  bug        |      Status:  assigned
>  Priority:  minor      |   Milestone:
> Component:  configure  |     Version:  1.3.0
>  Severity:  medium     |    Keywords:  profile gcc debug
>     Lang:             |       Patch:
>  Platform:  all        |
> -----------------------+----------------------------------------------------
>
> Comment(by darbelo):
>
>  Replying to [comment:13 jkeenan]:
>  > Follow-up question:
>  >
>  > If a user has requested an optimized build, does it make any sense to
>  have values for `link_debug` and `ld_debug`?  In the optimize case,
>  shouldn't they be set to empty strings just like `cc_debug`?
>
>  The answer to that depends on wether you consider building an optimized
>  parrot should exclude the possibility of building a debug-able parrot.
>  While it's true that the configure system has no direct provisions for
>  this, it is (or used to be, I haven't tried in som time) relatively simple
>  to build an optimized parrot with debugging symbols. IIRC chromatic used
>  to do this regularly so that he could get line number information out of
>  gdb when debugging optimized builds, it's likely we should take this use
>  case into consideration when evaluating the impact of changes to the debug
>  flags in optimized builds.
>

And others; I regularly build with --optimize and --cflags="-g" ; this
should continue to work.

Regards.
-- 
Will "Coke" Coleda