Using Boost 1.63 and 1.66 together with GNU unique global objects

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Using Boost 1.63 and 1.66 together with GNU unique global objects

Boost - Users mailing list
Apologies if this is a newbie question, but I have done my best to search Google, etc., and haven't been able to find a solution.

The project I am working on uses Boost 1.63 statically linked and a third-party shared library that  uses Boost 1.66 statically linked on CentOS 7.  For reasons that are beyond my control, these are the versions that must be used.

I am running into a problem with g++ 4.5.8 and the GNU extension to ELF for unique global symbols.

Take as example the function get_netdb_category() in boost/asio/impl/error.ipp.  The function has a static boost::asio::error::detail::netdb_category object that it returns a const reference to.  When I use nm to inspect the main executable the symbol appears in the data segment and the type for boost::asio::error::detail::netdb_category()::instance is "u", which indicates that the compiler has marked this as a globally-unique symbol.

The problem occurs when the loader loads the third-party shared library.  Because the symbol boost::asio::error::detail::netdb_category()::instance has been marked as unique in the shared library as well (verified by running readelf on the .so), the loader decides that it will simply use the one from the executable.  Because the shared library's version of the object is larger than the executable's version, when it is constructed it overwrites the 16 bytes beyond the end of the executable's object, trashing the contents of the data segment that were there previously.

I tried using the -fno-gnu-unique flag when compiling our executable, but that doesn't fix the problem - it marks the symbol as weak in the main executable, but the shared library, because I cannot re-compile, still has the symbol marked as unique.  I tried using dlopen with RTLD_LOCAL and that fixes the problem of overwriting the main executable's memory, however the corresponding symbols in the shared library have pointers back into the main executable's data segment (where the corresponding netdb_category static lives), which means that when if the shared library accesses these objects it will be following invalid pointers as it will overlay a larger v1.66 object onto a smaller v1.63 object.

Has anyone encountered a problem like this?  And, if so, did you find a good solution?

Much thanks,
Jamie



_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Using Boost 1.63 and 1.66 together with GNU unique global objects

Boost - Users mailing list
Correction.  It should be g++ 4.8.5 (I accidentally fat-fingered the version).

Jamie

> On Apr 29, 2019, at 3:11 PM, Jamie Jason <[hidden email]> wrote:
>
> Apologies if this is a newbie question, but I have done my best to search Google, etc., and haven't been able to find a solution.
>
> The project I am working on uses Boost 1.63 statically linked and a third-party shared library that  uses Boost 1.66 statically linked on CentOS 7.  For reasons that are beyond my control, these are the versions that must be used.
>
> I am running into a problem with g++ 4.5.8 and the GNU extension to ELF for unique global symbols.
>
> Take as example the function get_netdb_category() in boost/asio/impl/error.ipp.  The function has a static boost::asio::error::detail::netdb_category object that it returns a const reference to.  When I use nm to inspect the main executable the symbol appears in the data segment and the type for boost::asio::error::detail::netdb_category()::instance is "u", which indicates that the compiler has marked this as a globally-unique symbol.
>
> The problem occurs when the loader loads the third-party shared library.  Because the symbol boost::asio::error::detail::netdb_category()::instance has been marked as unique in the shared library as well (verified by running readelf on the .so), the loader decides that it will simply use the one from the executable.  Because the shared library's version of the object is larger than the executable's version, when it is constructed it overwrites the 16 bytes beyond the end of the executable's object, trashing the contents of the data segment that were there previously.
>
> I tried using the -fno-gnu-unique flag when compiling our executable, but that doesn't fix the problem - it marks the symbol as weak in the main executable, but the shared library, because I cannot re-compile, still has the symbol marked as unique.  I tried using dlopen with RTLD_LOCAL and that fixes the problem of overwriting the main executable's memory, however the corresponding symbols in the shared library have pointers back into the main executable's data segment (where the corresponding netdb_category static lives), which means that when if the shared library accesses these objects it will be following invalid pointers as it will overlay a larger v1.66 object onto a smaller v1.63 object.
>
> Has anyone encountered a problem like this?  And, if so, did you find a good solution?
>
> Much thanks,
> Jamie
>
>

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Using Boost 1.63 and 1.66 together with GNU unique global objects

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
Hey hey Jamie,
Apr 30 2019, Jamie Jason via Boost-users has written:
...
> The project I am working on uses Boost 1.63 statically linked and a
> third-party shared library that  uses Boost 1.66 statically linked on
> CentOS 7.
...
Are there (many) good reasons why you can't port your own application to
use Boost 1.69? Even if you need static linkage, Boost 1.69 must be
installed on your system and on the system running your software,
because of the third-party requirement. Sorry, maybe this is a naive
approach.

Best wishes,

Jeanette

--
  * Website: http://juliencoder.de - for summer is a state of sound
  * SoundCloud: https://soundcloud.com/jeanette_c
  * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
  * GitHub: https://github.com/jeanette-c
  * Twitter: https://twitter.com/jeanette_c_s

When you need someone, you just turn around and I will be there <3
(Britney Spears)
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Using Boost 1.63 and 1.66 together with GNU unique global objects

Boost - Users mailing list
On Tue, Apr 30, 2019 at 2:49 AM Jeanette C. via Boost-users
<[hidden email]> wrote:
>
> Hey hey Jamie,
> Apr 30 2019, Jamie Jason via Boost-users has written:
> ...
> > The project I am working on uses Boost 1.63 statically linked and a
> > third-party shared library that  uses Boost 1.66 statically linked on
> > CentOS 7.

Is the 3P open source? On github, or other Git (or other) repository?
Fork it and built it yourself on 1.63/1.66 (whichever is appropriate)
for your environment.

> ...
> Are there (many) good reasons why you can't port your own application to
> use Boost 1.69? Even if you need static linkage, Boost 1.69 must be
> installed on your system and on the system running your software,
> because of the third-party requirement. Sorry, maybe this is a naive
> approach.
>
> Best wishes,
>
> Jeanette
>
> --
>   * Website: http://juliencoder.de - for summer is a state of sound
>   * SoundCloud: https://soundcloud.com/jeanette_c
>   * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
>   * GitHub: https://github.com/jeanette-c
>   * Twitter: https://twitter.com/jeanette_c_s
>
> When you need someone, you just turn around and I will be there <3
> (Britney Spears)
> _______________________________________________
> Boost-users mailing list
> [hidden email]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Using Boost 1.63 and 1.66 together with GNU unique global objects

Boost - Users mailing list
Jeanette/Micheal,

thanks.  But as I stated in the original email, the versions of boost are beyond my control.

Jamie

On Tue, Apr 30, 2019 at 7:57 AM Michael Powell via Boost-users <[hidden email]> wrote:
On Tue, Apr 30, 2019 at 2:49 AM Jeanette C. via Boost-users
<[hidden email]> wrote:
>
> Hey hey Jamie,
> Apr 30 2019, Jamie Jason via Boost-users has written:
> ...
> > The project I am working on uses Boost 1.63 statically linked and a
> > third-party shared library that  uses Boost 1.66 statically linked on
> > CentOS 7.

Is the 3P open source? On github, or other Git (or other) repository?
Fork it and built it yourself on 1.63/1.66 (whichever is appropriate)
for your environment.

> ...
> Are there (many) good reasons why you can't port your own application to
> use Boost 1.69? Even if you need static linkage, Boost 1.69 must be
> installed on your system and on the system running your software,
> because of the third-party requirement. Sorry, maybe this is a naive
> approach.
>
> Best wishes,
>
> Jeanette
>
> --
>   * Website: http://juliencoder.de - for summer is a state of sound
>   * SoundCloud: https://soundcloud.com/jeanette_c
>   * Youtube: https://www.youtube.com/channel/UCMS4rfGrTwz8W7jhC1Jnv7g
>   * GitHub: https://github.com/jeanette-c
>   * Twitter: https://twitter.com/jeanette_c_s
>
> When you need someone, you just turn around and I will be there <3
> (Britney Spears)
> _______________________________________________
> Boost-users mailing list
> [hidden email]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users