PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
Travis's OS X resources for open source projects seem to be insufficient, so
OS X jobs are very slow and seem to be falling further and further behind.
Therefore, library authors are encouraged to keep the OS X jobs to the
minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting
time per job at a quick estimate).

On a not entirely unrelated note, Rene Rivera has added a new feature
<cxxstd> to Boost.Build that controls the C++ standard in use. So for
instance, instead of the old

    b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11

one can now use

    b2 libs/mylib/test toolset=gcc cxxstd=11

In addition to being more convenient, this also allows several invocations
to be combined into one:

    b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z

which can be leveraged to cut down on the number of jobs.

An example of using cxxstd in .travis.yml can be seen here:

https://github.com/boostorg/system/blob/develop/.travis.yml 


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
On Thu, Oct 26, 2017 at 12:43 PM, Peter Dimov via Boost <
[hidden email]> wrote:

> Travis's OS X resources for open source projects seem to be insufficient,
> so OS X jobs are very slow and seem to be falling further and further
> behind. Therefore, library authors are encouraged to keep the OS X jobs to
> the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes
> waiting time per job at a quick estimate).
>
> On a not entirely unrelated note, Rene Rivera has added a new feature
> <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> instance, instead of the old
>
>    b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
>
> one can now use
>
>    b2 libs/mylib/test toolset=gcc cxxstd=11
>
> In addition to being more convenient, this also allows several invocations
> to be combined into one:
>
>    b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
>
> which can be leveraged to cut down on the number of jobs.
>
> An example of using cxxstd in .travis.yml can be seen here:
>
> https://github.com/boostorg/system/blob/develop/.travis.yml
>

In Travis if one updates gcc and clang as part of the update scripts, you
end up with "clang++" and "clang++-5.0" for example, where the first one is
the version that came with the distribution.
Is there a bjam directive to change the compiler name, for example it would
be nice if I could do:

toolset=gcc,clang compiler={g++,g++-7},{clang++,clang++-5.0}

Right now I muck with the user-config.jam file to force it one way or the
other.
Perhaps that's still the best way to do it?

Thanks,

Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
James E. King, III wrote:
> In Travis if one updates gcc and clang as part of the update scripts, you
> end up with "clang++" and "clang++-5.0" for example, where the first one
> is the version that came with the distribution.
>
> Is there a bjam directive to change the compiler name, for example it
> would be nice if I could do:
>
> toolset=gcc,clang compiler={g++,g++-7},{clang++,clang++-5.0}

I've been informed that

    toolset=gcc-7

already works and automagically chooses g++-7 as the compiler.

Unfortunately, this trick doesn't work for clang (yet?), so one still need
the user-config.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Le 26/10/2017 à 18:43, Peter Dimov via Boost a écrit :
> Travis's OS X resources for open source projects seem to be
> insufficient, so OS X jobs are very slow and seem to be falling
> further and further behind. Therefore, library authors are encouraged
> to keep the OS X jobs to the minimum necessary. Spawning many OS X
> jobs takes hours (~25 minutes waiting time per job at a quick estimate).
>
Yes, I've notice it. Do you mean that the route cause are the new Boost
builds? Or is it a bug in travis?

> On a not entirely unrelated note, Rene Rivera has added a new feature
> <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> instance, instead of the old
>
>    b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
>
> one can now use
>
>    b2 libs/mylib/test toolset=gcc cxxstd=11
>
> In addition to being more convenient, this also allows several
> invocations to be combined into one:
>
>    b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
>
> which can be leveraged to cut down on the number of jobs.
>
> An example of using cxxstd in .travis.yml can be seen here:
>
> https://github.com/boostorg/system/blob/develop/.travis.yml
Thanks Peter for the information.

Best,
vicente

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
It seems like this would be a great instance where the steering committee
could allocate some of the organization's funds to directly improve the
infrastructure being used by our team. I believe that we could upgrade our
Travis account to a paid one for a couple hundred a month to get more build
resources.

They also claim on https://travis-ci.org/ that:

> Testing your open source project is 10000% free
> Seriously. Always. We like to think of it as our way of giving back to a
community that gives us so much as well.

Maybe we just need to have someone on the project contact them and get our
free quota raised?

Tom


On Thu, Oct 26, 2017 at 11:43 AM, Peter Dimov via Boost <
[hidden email]> wrote:

> Travis's OS X resources for open source projects seem to be insufficient,
> so OS X jobs are very slow and seem to be falling further and further
> behind. Therefore, library authors are encouraged to keep the OS X jobs to
> the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes
> waiting time per job at a quick estimate).
>
> On a not entirely unrelated note, Rene Rivera has added a new feature
> <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> instance, instead of the old
>
>    b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
>
> one can now use
>
>    b2 libs/mylib/test toolset=gcc cxxstd=11
>
> In addition to being more convenient, this also allows several invocations
> to be combined into one:
>
>    b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
>
> which can be leveraged to cut down on the number of jobs.
>
> An example of using cxxstd in .travis.yml can be seen here:
>
> https://github.com/boostorg/system/blob/develop/.travis.yml
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman
> /listinfo.cgi/boost
>

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
Tom Kent wrote:

> It seems like this would be a great instance where the steering committee
> could allocate some of the organization's funds to directly improve the
> infrastructure being used by our team. I believe that we could upgrade our
> Travis account to a paid one for a couple hundred a month to get more
> build resources.

One should keep in mind though that we currently get 5 jobs, which is only
exceeded by the most expensive paid plan ($489).

https://travis-ci.com/plans

> Maybe we just need to have someone on the project contact them and get our
> free quota raised?

The problem is not in our quota. Travis has 180 macOS jobs allocated for
open source projects, and those 180 jobs do not appear sufficient.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
> Tom Kent wrote:
>
> > It seems like this would be a great instance where the steering
> > committee could allocate some of the organization's funds to directly
> > improve the infrastructure being used by our team. I believe that we
> > could upgrade our Travis account to a paid one for a couple hundred a
> > month to get more build resources.
>
> One should keep in mind though that we currently get 5 jobs, which is only
> exceeded by the most expensive paid plan ($489).
>
> https://travis-ci.com/plans

... although to clarify, I completely support the idea. If we can afford to
do so, we should get a paid Travis account.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
On Fri, Oct 27, 2017 at 1:22 PM, Peter Dimov via Boost <
[hidden email]> wrote:

> Tom Kent wrote:
>>
>> > It seems like this would be a great instance where the steering >
>> committee could allocate some of the organization's funds to directly >
>> improve the infrastructure being used by our team. I believe that we >
>> could upgrade our Travis account to a paid one for a couple hundred a >
>> month to get more build resources.
>>
>> One should keep in mind though that we currently get 5 jobs, which is
>> only exceeded by the most expensive paid plan ($489).
>>
>> https://travis-ci.com/plans
>>
>
> ... although to clarify, I completely support the idea. If we can afford
> to do so, we should get a paid Travis account.
>

I would recommend attempting to negotiate with them on something that works
for everyone.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [boost-steering] Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Tom,

 

I’m not certain who we’d contact, but I suppose [hidden email] is the place to start.

 

I’m willing to contact them on behalf of the project (although I think it would be better if you did this because you know more of what we need).

 

What can you tell me about our OS X resources/requirements/quota or resources/requirements/quota generally that I should know before contacting them? Do we have an account name/number/identifier of some type?

 

Jon

 

From: Boost Steering Committee <[hidden email]> on behalf of Tom Kent <[hidden email]>
Reply-To: Boost Steering Committee <[hidden email]>
Date: Friday, October 27, 2017 at 10:04 AM
To: Boost Developers <[hidden email]>, Boost Steering Committee <[hidden email]>
Subject: [boost-steering] Re: [boost] PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

 

It seems like this would be a great instance where the steering committee could allocate some of the organization's funds to directly improve the infrastructure being used by our team. I believe that we could upgrade our Travis account to a paid one for a couple hundred a month to get more build resources.

 

They also claim on https://travis-ci.org/ that:

> Testing your open source project is 10000% free

> Seriously. Always. We like to think of it as our way of giving back to a community that gives us so much as well.

 

Maybe we just need to have someone on the project contact them and get our free quota raised?

 

Tom

 

 

On Thu, Oct 26, 2017 at 11:43 AM, Peter Dimov via Boost <[hidden email]> wrote:

Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).

On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old

   b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11

one can now use

   b2 libs/mylib/test toolset=gcc cxxstd=11

In addition to being more convenient, this also allows several invocations to be combined into one:

   b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z

which can be leveraged to cut down on the number of jobs.

An example of using cxxstd in .travis.yml can be seen here:

https://github.com/boostorg/system/blob/develop/.travis.yml 

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

 

--
The Boost Steering Committee webpage: https://sites.google.com/a/boost.org/steering/
---
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/boost-steering.
To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/CAArKS8gEiuYw1OUBkFRf3MD6DVkg7GufZR_jpBCgvfLXS8Q6WQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [boost-steering] Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
On Fri, Oct 27, 2017 at 8:47 PM, Jon Kalb via Boost <[hidden email]>
wrote:

> Tom,
>
>
>
> I’m not certain who we’d contact, but I suppose [hidden email] is
> the place to start.
>
>
I have a contact from some Apache project work that I will ask about this
and see
if we can get contact info for decision makers.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 26/10/2017 18:43, Peter Dimov via Boost wrote:

> Travis's OS X resources for open source projects seem to be
> insufficient, so OS X jobs are very slow and seem to be falling further
> and further behind. Therefore, library authors are encouraged to keep
> the OS X jobs to the minimum necessary. Spawning many OS X jobs takes
> hours (~25 minutes waiting time per job at a quick estimate).
>
> On a not entirely unrelated note, Rene Rivera has added a new feature
> <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> instance, instead of the old
>
>     b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
>
> one can now use
>
>     b2 libs/mylib/test toolset=gcc cxxstd=11
>
> In addition to being more convenient, this also allows several
> invocations to be combined into one:
>
>     b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
>
> which can be leveraged to cut down on the number of jobs.

Thanks for the information Peter, and thanks Rene for the feature. I
test different standard versions locally and I need to set up different
toolsets for that. This simplifies things noticeably.

Best,

Ion

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 10/26/17 19:43, Peter Dimov via Boost wrote:

>
> On a not entirely unrelated note, Rene Rivera has added a new feature
> <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> instance, instead of the old
>
>     b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
>
> one can now use
>
>     b2 libs/mylib/test toolset=gcc cxxstd=11

Sounds great. Does it always expand to c++XX or is there a way to
specify gnu++XX?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
Andrey Semashev wrote:

> On 10/26/17 19:43, Peter Dimov via Boost wrote:
> >
> > On a not entirely unrelated note, Rene Rivera has added a new feature
> > <cxxstd> to Boost.Build that controls the C++ standard in use. So for
> > instance, instead of the old
> >
> >     b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
> >
> > one can now use
> >
> >     b2 libs/mylib/test toolset=gcc cxxstd=11
>
> Sounds great. Does it always expand to c++XX or is there a way to specify
> gnu++XX?

gnu++XX can be requested with `cxxstd=XX cxxstd-dialect=gnu`.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [boost-steering] Re: PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Fri, Oct 27, 2017 at 8:47 PM, Jon Kalb via Boost <[hidden email]>
wrote:

> Tom,
>
> I’m not certain who we’d contact, but I suppose [hidden email] is
> the place to start.
>
> I’m willing to contact them on behalf of the project (although I think it
> would be better if you did this because you know more of what we need).
>
> What can you tell me about our OS X resources/requirements/quota or
> resources/requirements/quota generally that I should know before contacting
> them? Do we have an account name/number/identifier of some type?
>
> Jon
>
>
The info I got was:

The normal, free package gives you five concurrent jobs (which equals five
concurrent workers, so to speak).
Beyond that, the next step up includes 10 extra concurrent jobs,
effectively raising the limit to 15, for $500/month, paid annually and in
advance.
We are able to offer a 30% reduction for certified non-profits.

So it sounds like for $4,200/yr the project can triple the number of build
workers, and that's without any additional negotiation.
If folks who have authority want to pursue this, you can email
[hidden email] to continue discussions.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost