Quantcast

Much longer linux regression run times

Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Much longer linux regression run times

Boost - Testing mailing list
I've noticed that my runners that used to take 1-2 hours have jumped to 4-6 hours. At first, I thought this was due to some configuration changes on my end (some compilers weren't getting the correct <cxxflags> inputs). However, after further investigation, this seems to be going back further than when I started making these changes. 

The regression times jump up starting around Feb 18, and I didn't start my changes until the 22nd. Also, the jump in regression times seems to only be affecting develop, not master, even though I'm using the same configurations on both branches' runs. 

I'm not really sure how to trace this down. Is there any way to log the time it takes the various libraries to complete their test suites? I'm guessing that will show one library that is using 3+ hours. Although, it is possible that a change went in to a higher level library that just adds a few seconds to each call and is used across many of the libraries.

Thoughts?

Tom

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Much longer linux regression run times

Boost - Testing mailing list
On Tue, Feb 28, 2017 at 8:55 PM, Tom Kent via Boost-Testing <[hidden email]> wrote:

I'm not really sure how to trace this down. Is there any way to log the time it takes the various libraries to complete their test suites?

Not currently. The individual tests can run in parallel. To give you that number we would have to save the information as part of the regression data and sum it up as part of the regression reports. 
 
I'm guessing that will show one library that is using 3+ hours. Although, it is possible that a change went in to a higher level library that just adds a few seconds to each call and is used across many of the libraries.

Thoughts?

Binary search manually? You can limit what tests you run on the command line by using the "--include-tests" b2 option <https://github.com/boostorg/boost/blob/develop/status/Jamfile.v2#L18>. So start off by only running tests for [a-m] or [n-z], then [a-g], and so on. Until you find the time hog.

Other than that, maybe it's one of the libraries tested on Travis <https://travis-ci.org/boostorg/boost>. And maybe check the length of those first if they are long on Travis perhaps they are also long on regular tests.



--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Much longer linux regression run times

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list
On Tuesday, February 28, 2017 8:55:18 PM CST Tom Kent via Boost-Testing wrote:

> The regression times jump up starting around Feb 18, and I didn't start my
> changes until the 22nd. Also, the jump in regression times seems to only be
> affecting develop, not master, even though I'm using the same
> configurations on both branches' runs.

Yes, I can see the same here: the develop run jumped from 3-4 hours (prior to
February 18) to 8-9 hours (Feb 18 and forward).

-Steve

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Much longer linux regression run times

Boost - Testing mailing list


On Tue, Feb 28, 2017 at 10:13 PM, Steve Robbins via Boost-Testing <[hidden email]> wrote:
On Tuesday, February 28, 2017 8:55:18 PM CST Tom Kent via Boost-Testing wrote:

> The regression times jump up starting around Feb 18, and I didn't start my
> changes until the 22nd. Also, the jump in regression times seems to only be
> affecting develop, not master, even though I'm using the same
> configurations on both branches' runs.

Yes, I can see the same here: the develop run jumped from 3-4 hours (prior to
February 18) to 8-9 hours (Feb 18 and forward).

Maybe ask on the main list? Perhaps someone can remember making a change to develop that might be the culprit.

--Beman


_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Much longer linux regression run times

Boost - Testing mailing list
In reply to this post by Boost - Testing mailing list


On Tue, Feb 28, 2017 at 9:08 PM, Rene Rivera <[hidden email]> wrote:
On Tue, Feb 28, 2017 at 8:55 PM, Tom Kent via Boost-Testing <[hidden email]> wrote:

I'm not really sure how to trace this down. Is there any way to log the time it takes the various libraries to complete their test suites?

Not currently. The individual tests can run in parallel. To give you that number we would have to save the information as part of the regression data and sum it up as part of the regression reports. 
 
I'm guessing that will show one library that is using 3+ hours. Although, it is possible that a change went in to a higher level library that just adds a few seconds to each call and is used across many of the libraries.

Thoughts?

Binary search manually? You can limit what tests you run on the command line by using the "--include-tests" b2 option <https://github.com/boostorg/boost/blob/develop/status/Jamfile.v2#L18>. So start off by only running tests for [a-m] or [n-z], then [a-g], and so on. Until you find the time hog.

That sounds  tough. As a shortcut, I've looked at the libraries that had commits to develop on the 18th:
https://github.com/boostorg/boost/commits/develop?after=Y3Vyc29yOudbdLMAs4GRkxN6cblbXkNSu2O%2FKzY5

It looks like hana, chrono, ratio, and thread (with wave and core/winapi the previous day). If I get a chance tonight I'll start checking through  those. Meanwhile, if those authors could look at those and see if there is anything that might be causing this, it would be appreciated.
 

Other than that, maybe it's one of the libraries tested on Travis <https://travis-ci.org/boostorg/boost>. And maybe check the length of those first if they are long on Travis perhaps they are also long on regular tests.

 I could find the travis scripts for the superset repo (no change in build time), but I'm not clear on what that is doing. Is it testing individual libraries? 

Tom

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Much longer linux regression run times

Boost - Testing mailing list
On Wed, Mar 1, 2017 at 6:54 AM, Tom Kent via Boost-Testing <[hidden email]> wrote:

It looks like hana, chrono, ratio, and thread (with wave and core/winapi the previous day). If I get a chance tonight I'll start checking through  those. Meanwhile, if those authors could look at those and see if there is anything that might be causing this, it would be appreciated.

There is a thread on the dev list titled "[boost] [thread]Forever sleeping". I'd say with high degree of confidence that thread is the culprit.

--
-- Rene Rivera
-- Grafik - Don't Assume Anything
-- Robot Dreams - http://robot-dreams.net
-- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [boost] Much longer linux regression run times

Boost - Testing mailing list
Le 01/03/2017 à 14:01, Rene Rivera via Boost a écrit :
On Wed, Mar 1, 2017 at 6:54 AM, Tom Kent via Boost-Testing <
[hidden email]> wrote:

It looks like hana, chrono, ratio, and thread (with wave and core/winapi
the previous day). If I get a chance tonight I'll start checking through
 those. Meanwhile, if those authors could look at those and see if there is
anything that might be causing this, it would be appreciated.

There is a thread on the dev list titled "[boost] [thread]Forever
sleeping". I'd say with high degree of confidence that thread is the
culprit.

Hi,


yes, I committed something that I expect to be fixed it now.

Please, elet me know if there is an issue with the develop branch as it is now.


Best,

Vicente


_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Loading...