Boost Test Matrix - no clang release builds?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Boost Test Matrix - no clang release builds?

Boost - Dev mailing list
I was putting together some additional testing for a library and my CI jobs
were failing in release mode.  I was able to reproduce the issue locally.
On examination of the boost test matrix however I was unable to see the
same failures.  When I dug into it a little bit more, it looks like the
clang linux build jobs only build the default variant (debug), and
therefore will not catch release-mode optimization issues when running the
unit tests.

Here's one of the job details:
https://www.boost.org/development/tests/develop/teeks99-02-dc6-17-Docker-64on64.html

Given this matrix is the last line of defense before code is merged from
develop to master, I'd suggest that the build jobs should all be targeting
variant=release instead of variant=debug.  The former directly impacts a
release, and the latter does not.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Boost Test Matrix - no clang release builds?

Boost - Dev mailing list
On 11/11/2018 5:26 PM, James E. King III via Boost wrote:

> I was putting together some additional testing for a library and my CI jobs
> were failing in release mode.  I was able to reproduce the issue locally.
> On examination of the boost test matrix however I was unable to see the
> same failures.  When I dug into it a little bit more, it looks like the
> clang linux build jobs only build the default variant (debug), and
> therefore will not catch release-mode optimization issues when running the
> unit tests.
>
> Here's one of the job details:
> https://www.boost.org/development/tests/develop/teeks99-02-dc6-17-Docker-64on64.html
>
> Given this matrix is the last line of defense before code is merged from
> develop to master, I'd suggest that the build jobs should all be targeting
> variant=release instead of variant=debug.  The former directly impacts a
> release, and the latter does not.

There are regression tests for both 'develop' and 'master' branches.

>
> - Jim


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Boost Test Matrix - no clang release builds?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Sun, Nov 11, 2018 at 4:26 PM James E. King III via Boost <
[hidden email]> wrote:

> I was putting together some additional testing for a library and my CI jobs
> were failing in release mode.  I was able to reproduce the issue locally.
> On examination of the boost test matrix however I was unable to see the
> same failures.  When I dug into it a little bit more, it looks like the
> clang linux build jobs only build the default variant (debug), and
> therefore will not catch release-mode optimization issues when running the
> unit tests.
>
> Here's one of the job details:
>
> https://www.boost.org/development/tests/develop/teeks99-02-dc6-17-Docker-64on64.html
>
> Given this matrix is the last line of defense before code is merged from
> develop to master, I'd suggest that the build jobs should all be targeting
> variant=release instead of variant=debug.  The former directly impacts a
> release, and the latter does not.
>

Interesting catch. There is the 'teeks99-02-mc7-opt-Docker-64on64' [1]
configuration that manually specifies the -O2 flag (and a similar one for
gcc [2]). Does this find the issue that you are looking for?

What does variant=release do on posix platforms? For visual studio builds
there is a specific set of things that are defaulted for "release" builds
vs. "debug" builds, but I don't think that translates exactly to GCC/Clang.
(Please correct me if I'm wrong).

Tom

[1]
https://www.boost.org/development/tests/master/teeks99-02-mc7-opt-Docker-64on64.html
[2]
https://www.boost.org/development/tests/master/teeks99-02-mg8-opt-Docker-64on64.html

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: Boost Test Matrix - no clang release builds?

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On 12/11/2018 11:26, James E. King III wrote:
> I was putting together some additional testing for a library and my CI jobs
> were failing in release mode.  I was able to reproduce the issue locally.
> On examination of the boost test matrix however I was unable to see the
> same failures.  When I dug into it a little bit more, it looks like the
> clang linux build jobs only build the default variant (debug), and
> therefore will not catch release-mode optimization issues when running the
> unit tests.

Excluding compiler bugs and intentional differences (asserts, iterator
debugging, and the like), differences in behaviour between debug and
release builds usually indicates that the code relies on undefined
behaviour somewhere.

Consider running UBsan or a similar tester over the affected library.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost