UBSan question

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

UBSan question

Boost - Dev mailing list
Hi,

I am trying to correct for the errors reported by UBSAN in boost.test. A
lot of them were false positive, as ... in boost.test, we check the
capacity of the framework to catch various errors in the program.

I compile/run the tests with the following:

../../../b2 \
    --toolset=clang \
    cxxflags=-std=c++11 \
    cxxflags=-fno-omit-frame-pointer \
    cxxflags=-fsanitize=undefined \
    cxxflags=-fno-sanitize-recover=undefined \
    cxxflags=-fsanitize-recover=integer-divide-by-zero \
    cxxflags=-fsanitize-recover=float-divide-by-zero \
    linkflags=-fsanitize=undefined \
    linkflags=-fno-sanitize-recover=undefined \
    linkflags=-fsanitize-recover=integer-divide-by-zero \
    linkflags=-fsanitize-recover=float-divide-by-zero \
    linkflags=-fuse-ld=gold \
    variant=debug

I have an exception that is failing the tests on clang-6/ubuntu18.04
(UBSAN exiting the process), but not on gcc-8/ubuntu18.04 (full log below).

UndefinedBehaviorSanitizer:DEADLYSIGNAL
==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address

This FPE is happening for instance after reading from a null pointer.

I've searched on how to silence/blacklist this, but none of the
approaches worked so far, and I am not sure what is provoking this FPE.

Is there any UBsan specialist? Does anyone have an idea?

Thanks,
Raffi


clang-linux.compile.c++.without-pch
../../../bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/prg_exec_monitor-ts/system-exception-test.o
clang-linux.link
../../../bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test
testing.capture-output
../../../bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test.run
====== BEGIN OUTPUT ======
prg_exec_monitor-ts/system-exception-test.cpp:38:15: runtime error:
division by zero
    #0 0x42d03f
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42d03f)
    #1 0x42f344
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42f344)
    #2 0x42e8b1
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42e8b1)
    #3 0x43dafa
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43dafa)
    #4 0x43a6f2
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43a6f2)
    #5 0x436d13
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x436d13)
    #6 0x43726f
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43726f)
    #7 0x42d148
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42d148)
    #8 0x42e216
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42e216)
    #9 0x7f648e20fb96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #10 0x405859
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x405859)

UndefinedBehaviorSanitizer:DEADLYSIGNAL
==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address
0x00000042d049 (pc 0x00000042d049 bp 0x7ffe501ec660 sp 0x7ffe501ec630 T8303)
    #0 0x42d048
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42d048)
    #1 0x42f344
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42f344)
    #2 0x42e8b1
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42e8b1)
    #3 0x43dafa
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43dafa)
    #4 0x43a6f2
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43a6f2)
    #5 0x436d13
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x436d13)
    #6 0x43726f
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x43726f)
    #7 0x42d148
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42d148)
    #8 0x42e216
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x42e216)
    #9 0x7f648e20fb96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #10 0x405859
(/home/bambooagent/bamboo-agent-home/xml-data/build-dir/RS-BCB187-CLAN/library/bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test+0x405859)

UndefinedBehaviorSanitizer can not provide additional info.
==8303==ABORTING

EXIT STATUS: 1
====== END OUTPUT ======
(failed-as-expected)
../../../bin.v2/libs/test/test/system-exception-test.test/clang-linux-6.0.0/debug/threading-multi/visibility-hidden/system-exception-test.run


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
On 16.01.19 02:00, Raffi Enficiaud via Boost wrote:

> Hi,
>
> I am trying to correct for the errors reported by UBSAN in boost.test. A
> lot of them were false positive, as ... in boost.test, we check the
> capacity of the framework to catch various errors in the program.
>
> I compile/run the tests with the following:
>
> .../../../b2 \
>     --toolset=clang \
>     cxxflags=-std=c++11 \
>     cxxflags=-fno-omit-frame-pointer \
>     cxxflags=-fsanitize=undefined \
>     cxxflags=-fno-sanitize-recover=undefined \
>     cxxflags=-fsanitize-recover=integer-divide-by-zero \
>     cxxflags=-fsanitize-recover=float-divide-by-zero \
>     linkflags=-fsanitize=undefined \
>     linkflags=-fno-sanitize-recover=undefined \
>     linkflags=-fsanitize-recover=integer-divide-by-zero \
>     linkflags=-fsanitize-recover=float-divide-by-zero \
>     linkflags=-fuse-ld=gold \
>     variant=debug
>
> I have an exception that is failing the tests on clang-6/ubuntu18.04
> (UBSAN exiting the process), but not on gcc-8/ubuntu18.04 (full log below).
>
> UndefinedBehaviorSanitizer:DEADLYSIGNAL
> ==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address
>
> This FPE is happening for instance after reading from a null pointer.
>
> I've searched on how to silence/blacklist this, but none of the
> approaches worked so far, and I am not sure what is provoking this FPE.
>
> Is there any UBsan specialist? Does anyone have an idea?
>
> Thanks,
> Raffi
>

I am a bit blocked by this now, any specialist in the room or should I
just discard ubsan with clang 6?

Thanks!
Raffi



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
Raffi Enficiaud wrote:

> > I have an exception that is failing the tests on clang-6/ubuntu18.04
> > (UBSAN exiting the process), but not on gcc-8/ubuntu18.04 (full log
> > below).
> >
> > UndefinedBehaviorSanitizer:DEADLYSIGNAL
> > ==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address
> >
> > This FPE is happening for instance after reading from a null pointer.
> >
> > I've searched on how to silence/blacklist this, but none of the
> > approaches worked so far, and I am not sure what is provoking this FPE.

Are you setting the environment variable UBSAN_OPTIONS=print_stacktrace=1?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
On 25.01.19 20:08, Peter Dimov via Boost wrote:

> Raffi Enficiaud wrote:
>> > I have an exception that is failing the tests on clang-6/ubuntu18.04
>> > (UBSAN exiting the process), but not on gcc-8/ubuntu18.04 (full log
>> > below).
>> >
>> > UndefinedBehaviorSanitizer:DEADLYSIGNAL
>> > ==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address
>> >
>> > This FPE is happening for instance after reading from a null pointer.
>> >
>> > I've searched on how to silence/blacklist this, but none of the >
>> approaches worked so far, and I am not sure what is provoking this FPE.
>
> Are you setting the environment variable UBSAN_OPTIONS=print_stacktrace=1?
>

Yes. Do I need to change that?

Thanks,
Raffi



_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
On Fri, Jan 25, 2019 at 1:56 PM Raffi Enficiaud via Boost <
[hidden email]> wrote:

> On 16.01.19 02:00, Raffi Enficiaud via Boost wrote:
> > Hi,
> >
> > I am trying to correct for the errors reported by UBSAN in boost.test. A
> > lot of them were false positive, as ... in boost.test, we check the
> > capacity of the framework to catch various errors in the program.
> >
> > I compile/run the tests with the following:
> >
> > .../../../b2 \
> >     --toolset=clang \
> >     cxxflags=-std=c++11 \
> >     cxxflags=-fno-omit-frame-pointer \
> >     cxxflags=-fsanitize=undefined \
> >     cxxflags=-fno-sanitize-recover=undefined \
> >     cxxflags=-fsanitize-recover=integer-divide-by-zero \
> >     cxxflags=-fsanitize-recover=float-divide-by-zero \
> >     linkflags=-fsanitize=undefined \
> >     linkflags=-fno-sanitize-recover=undefined \
> >     linkflags=-fsanitize-recover=integer-divide-by-zero \
> >     linkflags=-fsanitize-recover=float-divide-by-zero \
> >     linkflags=-fuse-ld=gold \
> >     variant=debug
> >
> > I have an exception that is failing the tests on clang-6/ubuntu18.04
> > (UBSAN exiting the process), but not on gcc-8/ubuntu18.04 (full log
> below).
> >
> > UndefinedBehaviorSanitizer:DEADLYSIGNAL
> > ==8303==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address
> >
> > This FPE is happening for instance after reading from a null pointer.
> >
> > I've searched on how to silence/blacklist this, but none of the
> > approaches worked so far, and I am not sure what is provoking this FPE.
> >
> > Is there any UBsan specialist? Does anyone have an idea?
> >
> > Thanks,
> > Raffi
> >
>
> I am a bit blocked by this now, any specialist in the room or should I
> just discard ubsan with clang 6?
>
> Thanks!
> Raffi
>
>
>
Something you could do (for now) is leave the CI job in place but mark it
as an allowed failure.
That way the CI job will pass even if this fails, and you will have a
constant reminder
of it whenever you look at build results in CI.

- Jim

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
In reply to this post by Boost - Dev mailing list
Raffi Enficiaud wrote:
> On 25.01.19 20:08, Peter Dimov via Boost wrote:
> > Are you setting the environment variable
> > UBSAN_OPTIONS=print_stacktrace=1?
>
> Yes. Do I need to change that?

No, I was just wondering whether you were getting a stack trace on the
failure, and if not, why not.

What I do in cases like this, is set UBSAN=1 for the UBSan runs, then in the
Jamfile or in the .cpp files disable the known-to-fail tests. Like for
instance here:
https://github.com/boostorg/system/blob/develop/test/Jamfile.v2#L75


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
On 25.01.19 20:23, Peter Dimov via Boost wrote:
> Raffi Enficiaud wrote:
>> On 25.01.19 20:08, Peter Dimov via Boost wrote:
>> > Are you setting the environment variable >
>> UBSAN_OPTIONS=print_stacktrace=1?
>>
>> Yes. Do I need to change that?
>
> No, I was just wondering whether you were getting a stack trace on the
> failure, and if not, why not.

I do have the stacktrace (in my original post), it is not exactly
pointing to the line causing the issue though. I tried trillions of
combinations, and the read from NULL is apparently causing this.

>
> What I do in cases like this, is set UBSAN=1 for the UBSan runs, then in
> the Jamfile or in the .cpp files disable the known-to-fail tests. Like
> for instance here:
> https://github.com/boostorg/system/blob/develop/test/Jamfile.v2#L75

Ok, right. So I can say I tried my best. I will just use something
similar to yours for avoiding this specific test then. I was quite happy
with the diagnostics though, it pointed me to nasty visibility issues.

I am tempted to think that this is a UBSAN/clang issue, I will try to
reach their community (I just cannot let this go :) )

Thanks,
Raffi


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: UBSan question

Boost - Dev mailing list
Raffi Enficiaud wrote:

> I do have the stacktrace (in my original post), ...

Yes, I see it now. You had a stacktrace, but no symbols for some reason.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost