problem with test matrix truncating output

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

problem with test matrix truncating output

Robert Ramey
I have a problem.

I have some sort of bug on the building of the serialization library
with msvc compilers.

Looking at the test matrix doesn't help since since building the library
provokes some warnings.  The actual error is in a portion of the output
which has been truncated.  I had hoped to get around this without
bugging the build/test developers by setting up my old laptop to run the
serialization library tests.  After a certain amount of pain, I got the
test setup working - but things pass on this platform.  Clearly I'm out
of sync somewhere.

This demonstrates that there really is no substitute for making changes
to the regression test system so that it does not truncate the output
from failed tests.

Robert Ramey

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|

Re: problem with test matrix truncating output

Adam Wulkiewicz
Robert Ramey wrote:

> I have a problem.
>
> I have some sort of bug on the building of the serialization library
> with msvc compilers.
>
> Looking at the test matrix doesn't help since since building the
> library provokes some warnings.  The actual error is in a portion of
> the output which has been truncated.  I had hoped to get around this
> without bugging the build/test developers by setting up my old laptop
> to run the serialization library tests.  After a certain amount of
> pain, I got the test setup working - but things pass on this
> platform.  Clearly I'm out of sync somewhere.
>
> This demonstrates that there really is no substitute for making
> changes to the regression test system so that it does not truncate the
> output from failed tests.

Which test do you have in mind exactly?
One with
[The content has been trimmed by the report system because it exceeds
65536 bytes]
at the end of the output?

If that's the case then you can download the xml file from the FTP
server and check the compiler output manually.

I'm not sure if disabling trimming is a good idea. Is it possible to
esstimate the size increase of the reports after that?

Instead, we could trim in a different way, e.g. take half of the trimmed
output from the beginning of the original and the rest from the end or
do something more intelligent like searching for the first error and
showing the part of the output around it.

In case you wanted to play with it, here are the lines responsible for
writing the logs:
https://github.com/boostorg/regression/blob/develop/reports/src/links_page.cpp#L277
https://github.com/boostorg/regression/blob/develop/reports/src/links_page.cpp#L288
https://github.com/boostorg/regression/blob/develop/reports/src/links_page.cpp#L313
which calls operator trimming the result:
https://github.com/boostorg/regression/blob/develop/reports/src/xml.cpp#L257

Regards,
Adam

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|

Re: problem with test matrix truncating output

Robert Ramey
On 3/31/16 2:37 PM, Adam Wulkiewicz wrote:

> I'm not sure if disabling trimming is a good idea. Is it possible to
> esstimate the size increase of the reports after that?

The library_status creates a (large) links.html file with all the output
in it.  Then the test matrix links into an anchor on that one page.  Of
course the amount of output is the same, but it seems easier to deal
with one long file than many of them.

> Regards,
> Adam

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|

Re: problem with test matrix truncating output

Steven Watanabe-4
AMDG

On 03/31/2016 05:13 PM, Robert Ramey wrote:

> On 3/31/16 2:37 PM, Adam Wulkiewicz wrote:
>
>> I'm not sure if disabling trimming is a good idea. Is it possible to
>> estimate the size increase of the reports after that?
>
> The library_status creates a (large) links.html file with all the output
> in it.  Then the test matrix links into an anchor on that one page.  Of
> course the amount of output is the same, but it seems easier to deal
> with one long file than many of them.
>

  I don't see how multiple files vs. a single
file is related to whether the output is should
be trimmed.

In Christ,
Steven Watanabe

_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing
Reply | Threaded
Open this post in threaded view
|

Re: problem with test matrix truncating output

Adam Wulkiewicz
In reply to this post by Robert Ramey
Robert Ramey wrote:
> On 3/31/16 2:37 PM, Adam Wulkiewicz wrote:
>
>> I'm not sure if disabling trimming is a good idea. Is it possible to
>> esstimate the size increase of the reports after that?
>
> The library_status creates a (large) links.html file with all the
> output in it.  Then the test matrix links into an anchor on that one
> page.  Of course the amount of output is the same, but it seems easier
> to deal with one long file than many of them.

I asked because in case if the amount of the additional output was small
in general (which I think is the case) we could simply disable trimming.

Regards,
Adam
_______________________________________________
Boost-Testing mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-testing