Compiling multiple sources

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Compiling multiple sources

Phillip Seaver
In this somewhat-ancient thread, I asked about the possibility of
compiling multiple source files with the same command-line:
http://lists.boost.org/boost-build/2007/07/16817.php

This use case isn't as frequent for me as just changing a few files and
recompiling, but if I change a primary header in a library, it can cause
many files to be recompiled (or if I bring in a new version of a
third-party library, e.g.).

I ran a test with a library with 48 files, running the compiler
directly.  With a .bat file containing one command-line per file, it
took about 8.6 seconds.  With a single command-line containing all the
source files, it took about 5.3 seconds.

I know I'm probably an outlier in how I use boost-build.  A few of my
projects use wxWidgets, and we compile it and all the third-party
libraries we use (and that it uses), so we have some fairly large projects.

Phillip
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: Compiling multiple sources

Steven Watanabe-4
AMDG

On 03/12/2014 09:19 AM, Phillip Seaver wrote:

> In this somewhat-ancient thread, I asked about the possibility of
> compiling multiple source files with the same command-line:
> http://lists.boost.org/boost-build/2007/07/16817.php
>
> This use case isn't as frequent for me as just changing a few files and
> recompiling, but if I change a primary header in a library, it can cause
> many files to be recompiled (or if I bring in a new version of a
> third-party library, e.g.).
>
> I ran a test with a library with 48 files, running the compiler
> directly.  With a .bat file containing one command-line per file, it
> took about 8.6 seconds.  With a single command-line containing all the
> source files, it took about 5.3 seconds.
>

I'm not sure that this would be a good thing
even if it were possible.  Have you considered
the impact on parallel builds?  Batching compiles
like this is going to reduce parallelism.

> I know I'm probably an outlier in how I use boost-build.  A few of my
> projects use wxWidgets, and we compile it and all the third-party
> libraries we use (and that it uses), so we have some fairly large projects.
>

I don't think that b2's dependency graph is capable
of representing this.

# First of all, the combining needs to be done
# by b2.  It can't be handled by the msvc
# toolset, because we need to include only
# sources that need to be built.
#
# action modifiers:
#   together  - combine actions (merging $(>)) for equal values of $(<).
#   piecemeal - split into multiple actions (splitting $(>)) to
#               avoid overflowing the command line limits.
#   updated   - only include targets that have been updated in $(>)
#
# Notes:
#  - $(>) is the sources of an action.
#  - $(<) is the targets produced by an action.
actions piecemeal together updated compile.c++.combined
{
}

# For together to work, we need to create a target
# for each set of build options.  This allows b2
# to determine which groups of actions can be merged.
rule get-combined-target ( obj-target )
{
  local options = # get some kind of hash of the variables
  return <$(options)>$(target-directory) ;
}

# Finally, we can add
rule compile.c++ ( target : sources * : properties * )
{
  local group-target = [ get-combined-target $(target) ] ;
  compile.c++.combined $(group-target) : $(sources) ;
}

# Now we're left with a bit of a problem.
# We can't consider $(target) to be updated
# until after $(group-target) has been updated,
# but at the same time, we don't want to force
# add DEPENDS $(target) : $(group-target) ;
# Further, we really want each source to be
# included if the corresponding target is
# updated.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Reply | Threaded
Open this post in threaded view
|

Re: Compiling multiple sources

B.V. Raghav
This is a required feature for unit-tests built up from multiple sources

[Example 17] on http://www.boost.org/doc/libs/1_55_0/libs/test/doc/html/utf/user-guide/test-organization/auto-test-suite.html 

Or, is there another way out?

Thanks,
r