[Flyweight] Hidden symbol visibility

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Flyweight] Hidden symbol visibility

Boost - Dev mailing list
Hi there,

I recently made some interesting observation regarding Boost.Flyweight
and (hidden) symbol visibility and have now some questions regarding this.


Background story:
-----------------

In our company's code base we have been using Boost.Flyweight for
several years now, compiling it with GCC (and Clang) and it worked just
fine. (Thanks for it!)

However, lately we are trying to apply hidden symbol visibility to our
code base and some of our unit-tests started failing mysteriously. After
some intensive investigation I found out that it seems to be due to how
we used Boost.Flyweight.

We are instantiating it with some custom types, using the default
`boost::flyweights::static_holder` class, and are compiling it into a
shared library A. However, these Boost.Flyweight types are not only used
within that shared library A but also in some other shared library B and
in some unit-test executables which link against shared library A. (And
these unit-tests started failing.)


Observations:
-------------

Reading the documentation of Boost.Flyweight and looking at the header
`boost/flyweight/static_holder.hpp` I realized that we should have used
`boost::flyweights::intermodule_holder` in the first place instead of
`boost::flyweights::static_holder`, because we were using
Boost.Flyweight from different shared libraries.

Still, with non-hidden visibility applied everything always worked
correctly even when using `boost::flyweights::static_holder`.

Looking at the generated symbols in the different shared libraries I
realized that these `static_holder` class types were marked as "unique
global symbol" (an GNU extension to the standard set of ELF symbol
bindings). So, all shared libraries and executables still used the same
single instance of that `static_holder` class (which is the reason why
everything worked).

With hidden symbol visibility applied by default, these symbols are now
marked as local symbols (from the BSS section). So, each shared library
as well as the executables have their own version of the `static_holder`
class types (which would explain why the equality-comparisions between
different Boost.Flyweight variables in our unit-tests now started failing).

If I understand it correctly, using
`boost::flyweights::intermodule_holder` is the right thing to do, in
particular with hidden symbol visibility.


Questions:
----------

1.
However, apart from longer program startup times, what other
disadvantages might it have to use the `intermodule_holder` instead of
the `static_holder` (assuming it still would be marked as "unique global
symbol")?

2.
And if, as it seems, `static_holder` has some advantages in general,
would it be possible to (maybe conditionally) mark the `static_holder`
(and/or `static_holder_class`) class with `BOOST_SYMBOL_VISIBLE` or
something similar to allow using it even with hidden symbol visibility
from different shared libraries?
- Only applying default visibility to the template instantiations is
ignored by GCC (with warning: "type attributes ignored after type is
already defined [-Wattributes]").
- Besides, the C++ standard does not allow applying C++ attributes to
template instantiations. (There was a proposal for allowing it,
wg21.link/p0537, but I do not know what happened to it.)

3.
Would you have any other recommendations or suggestions I did not think
about?


Thanks for any help or opinion,
Deniz


--
BENOCS GmbH
Dipl.-Inform. Deniz Bahadir
Reuchlinstr. 10 D
10553 Berlin
Germany
Phone: +49 - 30 / 577 0004-22
Email: [hidden email]
www.benocs.com

Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese
Commercial Register: Amtsgericht Bonn HRB 19378

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [Flyweight] Hidden symbol visibility

Boost - Dev mailing list
El 07/02/2021 a las 16:36, Deniz Bahadir via Boost escribió:

> [...]
>
> In our company's code base we have been using Boost.Flyweight for
> several years now,
> compiling it with GCC (and Clang) and it worked just fine. (Thanks for
> it!)
>
> However, lately we are trying to apply hidden symbol visibility to our
> code base and
> some of our unit-tests started failing mysteriously. After some
> intensive investigation
> I found out that it seems to be due to how we used Boost.Flyweight.
>
> We are instantiating it with some custom types, using the default
> `boost::flyweights::static_holder`
> class, and are compiling it into a shared library A. However, these
> Boost.Flyweight types
> are not only used within that shared library A but also in some other
> shared library B
> and in some unit-test executables which link against shared library A.
> (And these unit-tests
> started failing.)
>
> [...]
>
> 1.
> However, apart from longer program startup times, what other
> disadvantages might it have
> to use the `intermodule_holder` instead of the `static_holder`
> (assuming it still would be
> marked as "unique global symbol")?

None in principle, other than, as you say, longer startup times.
intermodule_holder relies on
boost::interprocess::ipcdetail::intermodule_singleton, which is notably
complex.

As for the advantages, intermodule_holder is compatible with Windows,
where you don't have
symbol visibility attributes. I guess this is not a concern to you.

> 2.
> And if, as it seems, `static_holder` has some advantages in general,
> would it be possible to
> (maybe conditionally) mark the `static_holder` (and/or
> `static_holder_class`) class with
> `BOOST_SYMBOL_VISIBLE` or something similar to allow using it even
> with hidden symbol
> visibility from different shared libraries?
> - Only applying default visibility to the template instantiations is
> ignored by GCC (with warning:
> "type attributes ignored after type is already defined [-Wattributes]").
> - Besides, the C++ standard does not allow applying C++ attributes to
> template instantiations.
> (There was a proposal for allowing it, wg21.link/p0537, but I do not
> know what happened to it.)
You can't make static_holder visible without changing Boost.Flyweight
source code, but see below.
> 3.
> Would you have any other recommendations or suggestions I did not
> think about?

It's very easy to provide your own visible static holder:

     struct visible_static_holder:boost::flyweights::holder_marker
     {
       template<typename C> struct apply
       {
         struct type
         {
           BOOST_SYMBOL_VISIBLE static C& get()
           {
             static C c;
             return c;
           }
         };
       };
     };

     ...

     boost::flyweight<std::string,visible_static_holder> fw("hello");

Joaquín M López Muñoz


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Reply | Threaded
Open this post in threaded view
|

Re: [Flyweight] Hidden symbol visibility

Boost - Dev mailing list
Am 08.02.21 um 10:05 schrieb Joaquin M López Muñoz via Boost:

> El 07/02/2021 a las 16:36, Deniz Bahadir via Boost escribió:
>> [...]
>>
>> In our company's code base we have been using Boost.Flyweight for
>> several years now,
>> compiling it with GCC (and Clang) and it worked just fine. (Thanks
>> for it!)
>>
>> However, lately we are trying to apply hidden symbol visibility to
>> our code base and
>> some of our unit-tests started failing mysteriously. After some
>> intensive investigation
>> I found out that it seems to be due to how we used Boost.Flyweight.
>>
>> We are instantiating it with some custom types, using the default
>> `boost::flyweights::static_holder`
>> class, and are compiling it into a shared library A. However, these
>> Boost.Flyweight types
>> are not only used within that shared library A but also in some other
>> shared library B
>> and in some unit-test executables which link against shared library
>> A. (And these unit-tests
>> started failing.)
>>
>> [...]
>>
>> 1.
>> However, apart from longer program startup times, what other
>> disadvantages might it have
>> to use the `intermodule_holder` instead of the `static_holder`
>> (assuming it still would be
>> marked as "unique global symbol")?
>
> None in principle, other than, as you say, longer startup times.
> intermodule_holder relies on
> boost::interprocess::ipcdetail::intermodule_singleton, which is
> notably complex.
>
> As for the advantages, intermodule_holder is compatible with Windows,
> where you don't have
> symbol visibility attributes. I guess this is not a concern to you.

You guessed right.
We are developing on and for Linux and do not really care for Windows
compatibility.


>> 2.
>> And if, as it seems, `static_holder` has some advantages in general,
>> would it be possible to
>> (maybe conditionally) mark the `static_holder` (and/or
>> `static_holder_class`) class with
>> `BOOST_SYMBOL_VISIBLE` or something similar to allow using it even
>> with hidden symbol
>> visibility from different shared libraries?
>> - Only applying default visibility to the template instantiations is
>> ignored by GCC (with warning:
>> "type attributes ignored after type is already defined [-Wattributes]").
>> - Besides, the C++ standard does not allow applying C++ attributes to
>> template instantiations.
>> (There was a proposal for allowing it, wg21.link/p0537, but I do not
>> know what happened to it.)
> You can't make static_holder visible without changing Boost.Flyweight
> source code, but see below.
>> 3.
>> Would you have any other recommendations or suggestions I did not
>> think about?
>
> It's very easy to provide your own visible static holder:
>
>     struct visible_static_holder:boost::flyweights::holder_marker
>     {
>       template<typename C> struct apply
>       {
>         struct type
>         {
>           BOOST_SYMBOL_VISIBLE static C& get()
>           {
>             static C c;
>             return c;
>           }
>         };
>       };
>     };
>
>     ...
>
>     boost::flyweight<std::string,visible_static_holder> fw("hello");

Thanks, I thought about doing it like so.
I might give it a try. Although, if `intermodule_holder` really has no
real runtime-disadvantage we might just switch to using it. (Some of my
colleagues would prefer to not rely on "unique global symbols".)

>
> Joaquín M López Muñoz

Thanks a lot, Joaquín.

Deniz

--
BENOCS GmbH
Dipl.-Inform. Deniz Bahadir
Reuchlinstr. 10 D
10553 Berlin
Germany
Phone: +49 - 30 / 577 0004-22
Email: [hidden email]
www.benocs.com

Board of Management: Stephan Schroeder, Dr.-Ing. Ingmar Poese
Commercial Register: Amtsgericht Bonn HRB 19378


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost