Measuring compile time performance

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Measuring compile time performance

dan marsden
Hi

We're looking at optimizing some of the compile time performance in Boost.Fusion. I'm looking for any experience in Boost of measuring build times / compile time cost, how to structure tests that cover it etc. Does Boost.Build provide any support for measuring compile time for example?

I'm aware there's a Perl script by Steven Watanabe that measures template instantiations if I understand correctly. Are there any other techniques that are being used that I should be aware of?

Cheers

Dan


Send instant messages to your online friends http://uk.messenger.yahoo.com 
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Gubenko, Boris
To measure compile time, I modify toolset to add 'time' to the compile command, as the following:

actions acc.compile.c++
{
    time $(CONFIG_COMMAND) ...
}

And then, the time reported by the 'time' appear in bjam.log.

HTH,
  Boris

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of dan marsden
> Sent: Friday, October 17, 2008 2:46 AM
> To: Boost
> Subject: [boost] Measuring compile time performance
>
> Hi
>
> We're looking at optimizing some of the compile time
> performance in Boost.Fusion. I'm looking for any experience
> in Boost of measuring build times / compile time cost, how to
> structure tests that cover it etc. Does Boost.Build provide
> any support for measuring compile time for example?
>
> I'm aware there's a Perl script by Steven Watanabe that
> measures template instantiations if I understand correctly.
> Are there any other techniques that are being used that I
> should be aware of?
>
> Cheers
>
> Dan
>
>
> Send instant messages to your online friends
> http://uk.messenger.yahoo.com
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Dave Abrahams
In reply to this post by dan marsden

on Fri Oct 17 2008, dan marsden <danmarsden-AT-yahoo.co.uk> wrote:

> Hi
>
> We're looking at optimizing some of the compile time performance in
> Boost.Fusion. I'm looking for any experience in Boost of measuring
> build times / compile time cost, how to structure tests that cover it
> etc. Does Boost.Build provide any support for measuring compile time
> for example?

Actually it does.  In fact, I added the facility that allows it to bjam
long ago.  I think it might be visible in the XML output, but
I've forgotten more than I ever knew about bjam/BB, so you'd have to ask
someone else.

> I'm aware there's a Perl script by Steven Watanabe that measures
> template instantiations if I understand correctly. Are there any other
> techniques that are being used that I should be aware of?

It's pretty simple just to use

  time g++ ...whatever...

If you need to run windoze commands, you can just install cygwin and use
its implementation of time.

--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Joel de Guzman-2
David Abrahams wrote:

> on Fri Oct 17 2008, dan marsden <danmarsden-AT-yahoo.co.uk> wrote:
>
>> I'm aware there's a Perl script by Steven Watanabe that measures
>> template instantiations if I understand correctly. Are there any other
>> techniques that are being used that I should be aware of?
>
> It's pretty simple just to use
>
>   time g++ ...whatever...
>
> If you need to run windoze commands, you can just install cygwin and use
> its implementation of time.

time is good enough for benchmarking. OTOH, Steven's perl sript
is nice because it can (as he says) actually gather info when
templates are being instantiated. I haven't used it yet, but it
seems to be just the tool we need for profiling templates.
Steven, please tell us more about your script.

Regards,
--
Joel de Guzman
http://www.boostpro.com
http://spirit.sf.net

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
AMDG

Joel de Guzman wrote:
> time is good enough for benchmarking. OTOH, Steven's perl sript
> is nice because it can (as he says) actually gather info when
> templates are being instantiated. I haven't used it yet, but it
> seems to be just the tool we need for profiling templates.
> Steven, please tell us more about your script.

The scripts are actually a combination of bjam + perl + C++.

At the moment, the only thing that it does reliably is to
count the number of times each template is instantiated.

A month or two ago, I tried to make it produce a gprof-like
call graph by parsing the instantiation backtraces, but it
doesn't really work, and I broke gcc support in the process.
Hmmm, I'll try to fix this over fall break.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
AMDG

Steven Watanabe wrote:
> A month or two ago, I tried to make it produce a gprof-like
> call graph by parsing the instantiation backtraces, but it
> doesn't really work, and I broke gcc support in the process.
> Hmmm, I'll try to fix this over fall break.

Ok.  I've fixed this and added support for gcc 4.3.0.

For the record I'm testing with the following compilers:

gcc-3.4.4 (cygwin)
gcc-4.2.3 (cygwin)
gcc-3.4.4 (cygwin)
msvc-8.0express
msvc-9.0express

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Alp Mestan-3
Great ! Thank you Steven, I'll use it, be sure !

On 10/17/08, Steven Watanabe <[hidden email]> wrote:

>
> AMDG
>
> Steven Watanabe wrote:
>
>> A month or two ago, I tried to make it produce a gprof-like
>> call graph by parsing the instantiation backtraces, but it
>> doesn't really work, and I broke gcc support in the process.
>> Hmmm, I'll try to fix this over fall break.
>>
>
> Ok.  I've fixed this and added support for gcc 4.3.0.
>
> For the record I'm testing with the following compilers:
>
> gcc-3.4.4 (cygwin)
> gcc-4.2.3 (cygwin)
> gcc-3.4.4 (cygwin)
> msvc-8.0express
> msvc-9.0express
>
> In Christ,
> Steven Watanabe
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>



--
Alp Mestan
--- http://blog.mestan.fr/
--- http://alp.developpez.com/
--- In charge of the Qt, Algorithms and Artificial Intelligence sections on
Developpez
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Alex Mohr-2
In reply to this post by Steven Watanabe-4
> Ok.  I've fixed this and added support for gcc 4.3.0.
>
> For the record I'm testing with the following compilers:

Sorry -- I'm highly interested in this, but I didn't see it mentioned if
or where this was available.  Is it available somewhere?

Thanks!

Alex
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

dan marsden
In reply to this post by dan marsden
Steven Watanabe wrote:

>Joel de Guzman wrote:
>> time is good enough for benchmarking. OTOH, Steven's perl sript
>> is nice because it can (as he says) actually gather info when
>> templates are being instantiated. I haven't used it yet, but it
>> seems to be just the tool we need for profiling templates.
>> Steven, please tell us more about your script.
>
>The scripts are actually a combination of bjam + perl + C++.
>
>At the moment, the only thing that it does reliably is to
>count the number of times each template is instantiated.
>
>A month or two ago, I tried to make it produce a gprof-like
>call graph by parsing the instantiation backtraces, but it
>doesn't really work, and I broke gcc support in the process.
>Hmmm, I'll try to fix this over fall break.

Thanks Steven, that would be great. We'd certainly be interested
in trying what you've got on Fusions compile time performance
tests.

Thanks to everyone else that provided ideas on this.

Dan

Send instant messages to your online friends http://uk.messenger.yahoo.com 
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Juergen Hunold-2
In reply to this post by Alex Mohr-2
Hi Alex !
On Friday 17 October 2008 22:37:43 Alex Mohr wrote:
> Sorry -- I'm highly interested in this, but I didn't see it mentioned if
> or where this was available.  Is it available somewhere?

Sandbox-Svn:
svn ls http://svn.boost.org/svn/boost/sandbox/tools/profile_templates

or online via trac:
http://svn.boost.org/trac/boost/browser/sandbox/tools/profile_templates

Yours,

Jürgen
--
* Dipl.-Math. Jürgen Hunold       ! Ingenieurgesellschaft für
* voice: ++49 511 262926 57       ! Verkehrs- und Eisenbahnwesen mbH  
* fax  : ++49 511 262926 99       ! Lister Straße 15
* [hidden email]        ! www.ivembh.de
*
* Geschäftsführer:                ! Sitz des Unternehmens: Hannover
* Prof. Dr.-Ing. Thomas Siefer    ! Amtsgericht Hannover, HRB 56965
* PD Dr.-Ing. Alfons Radtke       !






_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
In reply to this post by Alex Mohr-2
AMDG

Alex Mohr wrote:
>> Ok.  I've fixed this and added support for gcc 4.3.0.
>>
>> For the record I'm testing with the following compilers:
>
> Sorry -- I'm highly interested in this, but I didn't see it mentioned
> if or where this was available.  Is it available somewhere?
You can check it out of SVN:
http://svn.boost.org/svn/boost/sandbox/tools/profile_templates

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Dave Abrahams
In reply to this post by dan marsden

on Fri Oct 17 2008, dan marsden <danmarsden-AT-yahoo.co.uk> wrote:

> Steven Watanabe wrote:
>>Joel de Guzman wrote:
>>> time is good enough for benchmarking. OTOH, Steven's perl sript
>>> is nice because it can (as he says) actually gather info when
>>> templates are being instantiated. I haven't used it yet, but it
>>> seems to be just the tool we need for profiling templates.
>>> Steven, please tell us more about your script.
>>
>>The scripts are actually a combination of bjam + perl + C++.
>>
>>At the moment, the only thing that it does reliably is to
>>count the number of times each template is instantiated.
>>
>>A month or two ago, I tried to make it produce a gprof-like
>>call graph by parsing the instantiation backtraces, but it
>>doesn't really work, and I broke gcc support in the process.
>>Hmmm, I'll try to fix this over fall break.
>
> Thanks Steven, that would be great. We'd certainly be interested
> in trying what you've got on Fusions compile time performance
> tests.
>
> Thanks to everyone else that provided ideas on this.

Remember that although the number of template instantiations is
important, it doesn't necessarily have a linear relationship to compile
times.  I think measuring time is still important, and probably more
important.

--
Dave Abrahams
BoostPro Computing
http://www.boostpro.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
AMDG

David Abrahams wrote:

>> Thanks Steven, that would be great. We'd certainly be interested
>> in trying what you've got on Fusions compile time performance
>> tests.
>>
>> Thanks to everyone else that provided ideas on this.
>>    
>
> Remember that although the number of template instantiations is
> important, it doesn't necessarily have a linear relationship to compile
> times.  I think measuring time is still important, and probably more
> important.
>  

Absolutely.  If we had a way to determine how much time was spent
instantiating each template, it would be more better.  Unfortunately,
the time it takes to compile a whole translation unit is not very
useful for isolating specific templates to optimize.  The time does provide
an excellent way to measure how much good any particular optimization
did--once we have an idea of what to try.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
In reply to this post by Alex Mohr-2
AMDG

Alex Mohr wrote:
> Sorry -- I'm highly interested in this, but I didn't see it mentioned
> if or where this was available.  Is it available somewhere?

I've updated the documentation an uploaded a zip to the vault.
http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Tools

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Joel Falcou-2
Steven Watanabe a écrit :
> I've updated the documentation an uploaded a zip to the vault.
> http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Tools 
>
>
Sorry to bring up this old discussion.
How difficult could it be to reuse this into a bjam free environnement
or as a standalone process ?

Regards

--
___________________________________________
Joel Falcou - Assistant Professor
PARALL Team - LRI - Universite Paris Sud XI
Tel : (+33)1 69 15 66 35


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
AMDG

Joel Falcou wrote:
> Steven Watanabe a écrit :
>> I've updated the documentation an uploaded a zip to the vault.
>> http://www.boostpro.com/vault/index.php?&direction=0&order=&directory=Tools 
>>
>>
> Sorry to bring up this old discussion.
> How difficult could it be to reuse this into a bjam free environnement
> or as a standalone process ?

The steps are:

a) compile postprocess.cpp
b) preprocess the source cpp file
c) run perl preprocess.pl on the preprocessor output
d) compile the result, adding the tool's root directory to the include
path and capture the output.
e) run postprocess on the compiler's output.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Joel Falcou-2
Steven Watanabe a écrit :
> a) compile postprocess.cpp
> b) preprocess the source cpp file
> c) run perl preprocess.pl on the preprocessor output
> d) compile the result, adding the tool's root directory to the include
> path and capture the output.
> e) run postprocess on the compiler's output.
OK, is there any special warning option to pass to g++ 4.3 cause I got
no output whatsoever even simple code.

--
___________________________________________
Joel Falcou - Assistant Professor
PARALL Team - LRI - Universite Paris Sud XI
Tel : (+33)1 69 15 66 35


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Measuring compile time performance

Steven Watanabe-4
AMDG

Joel Falcou wrote:
> Steven Watanabe a écrit :
>> a) compile postprocess.cpp
>> b) preprocess the source cpp file
>> c) run perl preprocess.pl on the preprocessor output
>> d) compile the result, adding the tool's root directory to the
>> include path and capture the output.
>> e) run postprocess on the compiler's output.
> OK, is there any special warning option to pass to g++ 4.3 cause I got
> no output whatsoever even simple code.

You're using the current version in the sandbox, right?

Here's the command that bjam generates for me

"g++-4.3.0"   -O0 -fno-inline -Wall -g   -D PROFILE_TEMPLATES -I".."
-I"test_dir" -c "bin\gcc-4.3.0\debug\test.cpp" 2>&1 |
"..\src\bin\gcc-4.3.0\release\link-static\filter.exe" --call-graph
 >"bin\gcc-4.3.0\debug\test.rtp"

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost