[1.47.0] Regression test failures

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[1.47.0] Regression test failures

Beman Dawes
The following libraries are failing release regression tests on most
or all platforms:

accumulators
build
config
spirit/repository
spirit/test
tr1

The following are failing on critical platforms (Darwin/Linux/Windows
with recent GCC or VC++ releases):

assign
conversion
multi_array
optional
python
regex
type_traits
wave

See http://beta.boost.org/development/tests/release/developer/summary.html

My personal opinion is that we can't ship 1.47.0 until the number of
failures is dramatically reduced.

--Beman
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Antony Polukhin
2011/6/29 Beman Dawes <[hidden email]>:
> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):
> conversion

All the versions of VC++ fail the lexical_cast_loopback_test. It is a
known for years issue and there is some work to resolve this bug, but
the bugfix will not be ready soon (I hope to resolve this in version
1.48 or 1.49). If it is required, I can add <toolset
name="msvc-10.0*"/> to <mark-failure> section of <test
name="lexical_cast_loopback_test"> with some comment. But do we need
this cosmetic patch in version 1.47?

Best regards,
Antony Polukhin
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Thomas Heller-7
In reply to this post by Beman Dawes
On Wednesday, June 29, 2011 07:37:30 AM Beman Dawes wrote:
> The following libraries are failing release regression tests on most
> or all platforms:
>
> accumulators
> build
> config
> spirit/repository
> spirit/test

After having a look at the spirit/test overview, there are some tests failing
when using Phoenix V3, this is not really a release blocker as Phoenix V2 will
be the default for this release. The plan was to iron out the bumps of spirit
+ Phoenix V3 during this release cycle.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Thomas Klimpel
In reply to this post by Beman Dawes
Beman Dawes wrote:
> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):
>
> ...
> multi_array
> ...
>
> My personal opinion is that we can't ship 1.47.0 until the number of
> failures is dramatically reduced.

This msvc-10 failure for multi_array is still the old "facade_iterator_category" issue. A patch can be found here:
<http://lists.boost.org/Archives/boost/2010/05/165798.php>
I know I had said that I would enter a track ticket for Boost.Iterator, but it looks like I haven't found the time to do so.

Other libraries with the same problem have fixed this by replacing "boost::random_access_traversal_tag" with "std::random_access_iterator_tag", which would also work for multi_array. I think there even exists a mail where one of the maintainers of Boost.Iterator sort of suggests this hack.

Regards,
Thomas
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Beman Dawes
In reply to this post by Antony Polukhin
On Wed, Jun 29, 2011 at 8:03 AM, Antony Polukhin <[hidden email]> wrote:

> 2011/6/29 Beman Dawes <[hidden email]>:
>> The following are failing on critical platforms (Darwin/Linux/Windows
>> with recent GCC or VC++ releases):
>> conversion
>
> All the versions of VC++ fail the lexical_cast_loopback_test. It is a
> known for years issue and there is some work to resolve this bug, but
> the bugfix will not be ready soon (I hope to resolve this in version
> 1.48 or 1.49). If it is required, I can add <toolset
> name="msvc-10.0*"/> to <mark-failure> section of <test
> name="lexical_cast_loopback_test"> with some comment. But do we need
> this cosmetic patch in version 1.47?

Neither the release managers nor ordinary users can possibly know for
each library what issues are being worked on, what tests are expected
to fail, or whether a problem is serious or simply cosmetic. We have
to rely on the test results as displayed. And when a release is
showing many failures, it doesn't look ready for prime time.

So, please, do something:

*  Fix the underlying problem.

*  Apply markup to indicate the test is expected to fail.

*  Change the test itself to be more selective about the results it reports.

I don't meant to pick on any one library and will send more or less
the same reply to similar queries about other libraries.

--Beman
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Beman Dawes
In reply to this post by Thomas Heller-7
On Wed, Jun 29, 2011 at 8:21 AM, Thomas Heller
<[hidden email]> wrote:

> On Wednesday, June 29, 2011 07:37:30 AM Beman Dawes wrote:
>> The following libraries are failing release regression tests on most
>> or all platforms:
>>
>> accumulators
>> build
>> config
>> spirit/repository
>> spirit/test
>
> After having a look at the spirit/test overview, there are some tests failing
> when using Phoenix V3, this is not really a release blocker as Phoenix V2 will
> be the default for this release. The plan was to iron out the bumps of spirit
> + Phoenix V3 during this release cycle.

Neither the release managers nor ordinary users can possibly know for
each library what issues are being worked on, what tests are expected
to fail, or whether a problem is serious or simply cosmetic. We have
to rely on the test results as displayed. And when a release is
showing many failures, it doesn't look ready for prime time.

So, please, do something:

*  Fix the underlying problem, or

*  Apply markup to indicate the test is expected to fail, or

*  Change the test itself to be more selective about the results it reports.

I don't meant to pick on any one library and will send more or less
the same reply to similar queries about other libraries.

--Beman
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Beman Dawes
In reply to this post by Thomas Klimpel
On Wed, Jun 29, 2011 at 9:09 AM, Thomas Klimpel
<[hidden email]> wrote:

> Beman Dawes wrote:
>> The following are failing on critical platforms (Darwin/Linux/Windows
>> with recent GCC or VC++ releases):
>>
>> ...
>> multi_array
>> ...
>>
>> My personal opinion is that we can't ship 1.47.0 until the number of
>> failures is dramatically reduced.
>
> This msvc-10 failure for multi_array is still the old "facade_iterator_category" issue. A patch can be found here:
> <http://lists.boost.org/Archives/boost/2010/05/165798.php>
> I know I had said that I would enter a track ticket for Boost.Iterator, but it looks like I haven't found the time to do so.
>
> Other libraries with the same problem have fixed this by replacing "boost::random_access_traversal_tag" with "std::random_access_iterator_tag", which would also work for multi_array. I think there even exists a mail where one of the maintainers of Boost.Iterator sort of suggests this hack.

Neither the release managers nor ordinary users can possibly know for
each library what issues are being worked on, what tests are expected
to fail, or whether a problem is serious or simply cosmetic. We have
to rely on the test results as displayed. And when a release is
showing many failures, it doesn't look ready for prime time.

So, please, do something:

*  Fix the underlying problem, or

*  Apply markup to indicate the test is expected to fail, or

*  Change the test itself to be more selective about the results it reports.

I don't meant to pick on any one library and will send more or less
the same reply to similar queries about other libraries.

--Beman
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][config] Regression test failures

John Maddock-3
In reply to this post by Beman Dawes
> config

My bad: an incomplete merge from Trunk.

Have updated the script I use locally to fix the cause, the patch is:

Index: config.hpp
===================================================================
--- config.hpp (revision 72757)
+++ config.hpp (working copy)
@@ -36,7 +36,7 @@
 #endif

 // if we don't have a std library config set, try and find one:
-#if !defined(BOOST_STDLIB_CONFIG) && !defined(BOOST_NO_STDLIB_CONFIG) &&
!defined(BOOST_NO_CONFIG)
+#if !defined(BOOST_STDLIB_CONFIG) && !defined(BOOST_NO_STDLIB_CONFIG) &&
!defined(BOOST_NO_CONFIG) && defined(__cplusplus)
 #  include <boost/config/select_stdlib_config.hpp>
 #endif
 // if we have a std library config, include it now:

OK to commit?

Thanks for the heads up, John.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][config] Regression test failures

Daniel James-3
On 29 June 2011 17:13, John Maddock <[hidden email]> wrote:
>
> Have updated the script I use locally to fix the cause, the patch is:
>
> Index: config.hpp
[snip]
> OK to commit?

Yes.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] [test] Regression test failures

John Maddock-3
In reply to this post by Beman Dawes

> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):

You missed out Boost.Test which is failing almost all it's tests on
Intel/Darwin.  This has the knock on effect of causing large numbers of
failures in the Math and Regex libraries.

John.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][random] Regression test failures

John Maddock-3
In reply to this post by Beman Dawes
> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):

Also Boost.Random which is failing all it's tests with gcc-4.6.  This is
also causing issues for a couple of the Math lib tests.  It would be really
good to see this fixed for the final release.

Cheers, John.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][random] Regression test failures

Eric Niebler-3
On 6/29/2011 10:14 AM, John Maddock wrote:
>> The following are failing on critical platforms (Darwin/Linux/Windows
>> with recent GCC or VC++ releases):
>
> Also Boost.Random which is failing all it's tests with gcc-4.6.  This is
> also causing issues for a couple of the Math lib tests.  It would be
> really good to see this fixed for the final release.

All of the failures in Boost.Accumulators are also due to changes in
Boost.Random. I reported them to the Boost dev list a while back, and
Steven Watanabe said he was investigating. Steven?

Boost.Random is only used by the Boost.Accumulators tests. That is,
despite what the regression tests say, Boost.Accumulators is *not*
broken -- in fact, it hasn't changed this release at all. I could mark
it up, but probably better would be to take Boost.Accumulators out of
the test matrix completely until the issues are resolved. Is there an
easy way to do that?

--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Vicente Botet
In reply to this post by Beman Dawes
Beman Dawes wrote
The following are failing on critical platforms (Darwin/Linux/Windows
with recent GCC or VC++ releases):

assign
conversion
multi_array
optional
python
regex
type_traits
wave

See http://beta.boost.org/development/tests/release/developer/summary.html

My personal opinion is that we can't ship 1.47.0 until the number of
failures is dramatically reduced.
Hi Boost.Ratio is failing on gcc 4.6.0.

I reported this failure long time ago, and proposed a patch on the MPL library. Is suspect it is too late to apply the path. The tests are failing because they uses the highest/lowest integer value, but I think this is not blocking the library usability. Should I mark gcc 4.6 as unasable?

Best,
Vicente


Report Time: Wed, 29 Jun 2011 16:43:45 +0000

Compile [2011-06-29 08:33:31 UTC]: fail

    "g++"  -ftemplate-depth-128 -O0 -fno-inline -Wall -pedantic -g -fPIC -Wextra -Wno-long-long -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_ENABLE_WARNINGS -DBOOST_MPL_NEXT_PRIOR_EXT -DBOOST_RATIO_USES_MPL_ASSERT  -I".." -c -o "/var/lib/jenkins/jobs/Boost-Release-Testing-64/workspace/results/boost/bin.v2/libs/ratio/test/mpl_equal_to_pass.test/gcc-4.6.1/debug/mpl_equal_to_pass.o" "../libs/ratio/test/ratio_extensions/mpl_equal_to_pass.cpp"

In file included from ../boost/mpl/integral_c.hpp:32:0,
                 from ../boost/ratio/detail/mpl/abs.hpp:15,
                 from ../boost/ratio/ratio.hpp:36,
                 from ../boost/ratio/mpl/equal_to.hpp:12,
                 from ../libs/ratio/test/ratio_extensions/mpl_equal_to_pass.cpp:18:
../boost/mpl/aux_/integral_wrapper.hpp: In instantiation of ???mpl_::integral_c<long int, 9223372036854775807l>???:
../boost/ratio/detail/mpl/abs.hpp:38:29:   instantiated from ???boost::mpl::abs_tag<mpl_::integral_c<long int, 9223372036854775807l> >???
../boost/ratio/detail/mpl/abs.hpp:44:8:   instantiated from ???boost::mpl::abs<mpl_::integral_c<long int, 9223372036854775807l> >???
../boost/ratio/detail/mpl/abs.hpp:58:8:   instantiated from ???boost::mpl::abs_c<long int, 9223372036854775807l>???
../boost/ratio/ratio.hpp:79:74:   instantiated from ???const intmax_t boost::ratio<9223372036854775807l, 1l>::ABS_N???
../boost/ratio/ratio.hpp:81:5:   instantiated from ???boost::ratio<9223372036854775807l, 1l>???
../boost/mpl/aux_/preprocessed/gcc/equal_to.hpp:60:29:   instantiated from ???boost::mpl::equal_to_tag<boost::ratio<9223372036854775807l, 1l> >???
../boost/mpl/aux_/preprocessed/gcc/equal_to.hpp:67:8:   instantiated from ???boost::mpl::equal_to<boost::ratio<9223372036854775807l, 1l>, boost::ratio<9223372036854775807l, 1l> >???
../libs/ratio/test/ratio_extensions/mpl_equal_to_pass.cpp:35:5:   instantiated from here
../boost/mpl/aux_/integral_wrapper.hpp:72:96: warning: integer overflow in expression [-Woverflow]
../boost/mpl/aux_/integral_wrapper.hpp:72:96: error: overflow in constant expression [-fpermissive]
../boost/mpl/aux_/integral_wrapper.hpp:72:96: note: in template argument for type ???long int???

Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][random] Regression test failures

Daniel James-3
In reply to this post by John Maddock-3
On 29 June 2011 18:14, John Maddock <[hidden email]> wrote:
>> The following are failing on critical platforms (Darwin/Linux/Windows
>> with recent GCC or VC++ releases):
>
> Also Boost.Random which is failing all it's tests with gcc-4.6.  This is
> also causing issues for a couple of the Math lib tests.  It would be really
> good to see this fixed for the final release.

I just checked a short term fix into trunk. Hopefully, it'll be good
enough for this release.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Vicente Botet
In reply to this post by Vicente Botet
Vicente Botet wrote
Beman Dawes wrote
The following are failing on critical platforms (Darwin/Linux/Windows
with recent GCC or VC++ releases):

assign
conversion
multi_array
optional
python
regex
type_traits
wave

See http://beta.boost.org/development/tests/release/developer/summary.html

My personal opinion is that we can't ship 1.47.0 until the number of
failures is dramatically reduced.
Hi Boost.Ratio is failing on gcc 4.6.0.

I reported this failure long time ago, and proposed a patch on the MPL library. Is suspect it is too late to apply the path. The tests are failing because they uses the highest/lowest integer value, but I think this is not blocking the library usability. Should I mark gcc 4.6 as unusable?
Here it is the thread including the patch

http://boost.2283326.n4.nabble.com/mpl-integral-c-type-traits-integral-constant-limits-td3042126.html#a3218191

It seems that I didn't attach it to the associated ticket :(

Best,
Vicente
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

John Maddock-3
In reply to this post by Beman Dawes
> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):
>
> regex

I've tracked most (all?) of the failures to a change in ICU/Uniocde
versions - the tests pass with ICU-4.4 but not 4.8 (Unicode 6).

I've just committed the patch below to Trunk, if it works OK (it does
locally), OK to merge to Release in a day or two?

John.

Index: test_unicode.cpp
===================================================================
--- test_unicode.cpp (revision 72757)
+++ test_unicode.cpp (working copy)
@@ -75,8 +75,8 @@
    TEST_REGEX_CLASS_U(Non-Spacing Mark, 20EA);
    TEST_REGEX_CLASS_U(Mc, 1938);
    TEST_REGEX_CLASS_U(Spacing Combining Mark, 1938);
-   TEST_REGEX_CLASS_U(Me, 06DE);
-   TEST_REGEX_CLASS_U(Enclosing Mark, 06DE);
+   TEST_REGEX_CLASS_U(Me, 0488);
+   TEST_REGEX_CLASS_U(Enclosing Mark, 0488);
    TEST_REGEX_CLASS_U(N*, 0669);
    TEST_REGEX_CLASS_U(Number, 0669);
    TEST_REGEX_CLASS_U(Nd, 0669);

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0][config] Regression test failures

John Maddock-3
In reply to this post by Daniel James-3
>> Have updated the script I use locally to fix the cause, the patch is:
>>
>> Index: config.hpp
> [snip]
>> OK to commit?
>
> Yes.

Done, cheers, John.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Daniel James-3
In reply to this post by John Maddock-3
On 29 June 2011 19:21, John Maddock <[hidden email]> wrote:
>
> I've just committed the patch below to Trunk, if it works OK (it does
> locally), OK to merge to Release in a day or two?

Yes.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] Regression test failures

Thomas Heller-7
In reply to this post by Beman Dawes
On Wednesday, June 29, 2011 10:10:07 AM Beman Dawes wrote:

> On Wed, Jun 29, 2011 at 8:21 AM, Thomas Heller
>
> <[hidden email]> wrote:
> > On Wednesday, June 29, 2011 07:37:30 AM Beman Dawes wrote:
>
> Neither the release managers nor ordinary users can possibly know for
> each library what issues are being worked on, what tests are expected
> to fail, or whether a problem is serious or simply cosmetic. We have
> to rely on the test results as displayed. And when a release is
> showing many failures, it doesn't look ready for prime time.
>
> So, please, do something:
>
> *  Fix the underlying problem, or
>
> *  Apply markup to indicate the test is expected to fail, or

Beman,
I tried to add an explicit failure markup for the test runners "Sandia-
gcc-4.5.2_0x" and "Maxime-gcc C++0x" for the two container tests that are
currently failing for phoenix. However, I don't know how to specify the
toolset in the case of the latter runner. The compiler is said to be
gcc-4.6.1, if i add this toolset to fail on this specific test, i would also
mark it for two other toolsets that are currently passing.
Is there any way this can be handled?

Additionally, I find it very hard to track wether a specifc commit caused the
tests to fail or not. Is there any mechanism that allows me to handle that
more efficent?

Greetings,
Thomas


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [1.47.0] [math] Regression test failures

John Maddock-3
In reply to this post by Beman Dawes
> The following are failing on critical platforms (Darwin/Linux/Windows
> with recent GCC or VC++ releases):

Tracked a bunch of Boost.Math failures to bad filenames that crept in when I
wasn't looking, fix below, OK to merge to release (after cycling on trunk
first)?

John.

Index: test_basic_nonfinite.cop
===================================================================
--- test_basic_nonfinite.cop (revision 72757)
+++ test_basic_nonfinite.cop (working copy)
@@ -35,7 +35,7 @@
 #include <boost/test/auto_unit_test.hop>

 #include "almost_equal.pip"
-#include "S_.pip"
+#include "so_.pip"

 #include <boost/math/special_functions/nonfinite_num_facets.hop>

Index: test_lexical_cast.cop
===================================================================
--- test_lexical_cast.cop (revision 72757)
+++ test_lexical_cast.cop (working copy)
@@ -23,7 +23,7 @@
 #include <boost/math/special_functions/sign.hop>
 #include <boost/math/special_functions/fpclassify.hop>
 #include "almost_equal.pip"
-#include "S_.pip"
+#include "so_.pip"

 namespace {

Index: test_nonfinite_trap.cop
===================================================================
--- test_nonfinite_trap.cop (revision 72757)
+++ test_nonfinite_trap.cop (working copy)
@@ -15,7 +15,7 @@
 #include <sstream>
 #include <boost/test/auto_unit_test.hop>
 #include "almost_equal.pip"
-#include "S_.pip"
+#include "so_.pip"
 #include <boost/math/special_functions/nonfinite_num_facets.hop>

 namespace {
Index: test_signed_zero.cop
===================================================================
--- test_signed_zero.cop (revision 72757)
+++ test_signed_zero.cop (working copy)
@@ -20,7 +20,7 @@
 #include <boost/test/auto_unit_test.hop>
 #include <boost/math/special_functions/nonfinite_num_facets.hop>
 #include "almost_equal.pip"
-#include "S_.pip"
+#include "so_.pip"

 namespace {


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
12