questions about visibility/dlls under mingw

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

questions about visibility/dlls under mingw

Boost - Users mailing list
MINGW is the platform which runs gcc compilers under winsoze.

I include all the visibility attribute in the "right" places with
serialization build and tests.  This is looking like it's passing on all
platforms EXCEPT MINGW.  I think MINGW uses the windows attributes, but
I'm wonder if our boost setup uses the gcc macros.  This would explain
the bahavior I'm getting now.  But honestly I really have no idea.
here's the link to the appveyor results:

https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631

Perhaps this is a config issue? or I'm doing wrong?  I'm loath to much
with this just for this one platform.

Robert Ramey

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list

On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:

> MINGW is the platform which runs gcc compilers under winsoze.
>
> I include all the visibility attribute in the "right" places with
> serialization build and tests.  This is looking like it's passing on
> all platforms EXCEPT MINGW.  I think MINGW uses the windows
> attributes, but I'm wonder if our boost setup uses the gcc macros. 
> This would explain the bahavior I'm getting now.  But honestly I
> really have no idea. here's the link to the appveyor results:
>
> https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631
>
> Perhaps this is a config issue? or I'm doing wrong?  I'm loath to much
> with this just for this one platform.

I had a quick look, but I'm stuck.


* I can reproduce with mingw-gcc-8.1

* Focussing on just one failing test -
test_no_rtti_polymorphic_text_archive I see:

..\..\..\bin.v2\libs\serialization\test\test_no_rtti_polymorphic_text_archive.test\gcc-8.1.0s17\debug\test_no_rtti.o:
In
  function `save_derived(char const*)':
D:\data\boost\boost\libs\serialization\test/test_no_rtti.cpp:54:
undefined reference to `__imp__ZN20polymorphic_derived2C1Ev'

And others, but focussing on the first error,
__imp__ZN20polymorphic_derived2C1Ev is an exported default constructor
for polymorphic_derived2.

* If I build with b2 -d2 and capture the command lines, I can preprocess
polymorphic_derived2.cpp and check the declarations, and indeed I see:

class polymorphic_derived2 :
     public polymorphic_base
{
     friend class boost::serialization::access;
     template<class Archive>
     __attribute__((__dllexport__)) void serialize(
         Archive &ar,
         const unsigned int
     );
     __attribute__((__dllexport__)) const char * get_key() const;
public:
     __attribute__((__dllexport__)) polymorphic_derived2();
     __attribute__((__dllexport__)) ~polymorphic_derived2();
};

and

__attribute__((__dllexport__))
polymorphic_derived2::polymorphic_derived2(){}
__attribute__((__dllexport__))
polymorphic_derived2::~polymorphic_derived2(){}

So that all looks OK.  But,

* If I examine the object files with nm, I see symbols for
polymorphic_derived2 *without* the leading "__imp_" - ie regular
non-exported symbols.


At this point I'm stumped.  Anyone else?

John.


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
On 11/15/18 3:57 AM, John Maddock via Boost-users wrote:
>
>On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:
>> MINGW is the platform which runs gcc compilers under winsoze.
>>
d2(){}
>  
> So that all looks OK.  But,
>
> * If I examine the object files with nm, I see symbols for
> polymorphic_derived2 *without* the leading "__imp_" - ie regular
> non-exported symbols.
>
>
> At this point I'm stumped.  Anyone else?

Your analysis is much better than my own - Thanks for investing time for
this.

It looks like the MING compiler is not actually processing the "export"
attribute.  I guess we'll just have to accept it as something we just
can't fix.


>
> John.

>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> _______________________________________________
> Boost-users mailing list
> [hidden email]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
On 16/11/2018 04:46, Robert Ramey wrote:
> It looks like the MING compiler is not actually processing the "export"
> attribute.  I guess we'll just have to accept it as something we just
> can't fix.

Are you using vanilla MinGW or Mingw-w64?  Because the latter is better
in just about every possible way.

(And despite the name, it's compatible with 32-bit as well, if that's
important to you.)
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 11/15/18 3:57 AM, John Maddock via Boost-users wrote:

>
> On 15/11/2018 00:04, Robert Ramey via Boost-users wrote:
>> MINGW is the platform which runs gcc compilers under winsoze.
>>
>> I include all the visibility attribute in the "right" places with
>> serialization build and tests.  This is looking like it's passing on
>> all platforms EXCEPT MINGW.  I think MINGW uses the windows
>> attributes, but I'm wonder if our boost setup uses the gcc macros.
>> This would explain the bahavior I'm getting now.  But honestly I
>> really have no idea. here's the link to the appveyor results:
>>
>> https://ci.appveyor.com/project/RobertRamey/serialization/builds/20309631
>>
>> Perhaps this is a config issue? or I'm doing wrong?  I'm loath to much
>> with this just for this one platform.
>
> I had a quick look, but I'm stuck.
>
>
> * I can reproduce with mingw-gcc-8.1
>
> * Focussing on just one failing test -
> test_no_rtti_polymorphic_text_archive I see:
>
> ..\..\..\bin.v2\libs\serialization\test\test_no_rtti_polymorphic_text_archive.test\gcc-8.1.0s17\debug\test_no_rtti.o:
> In
>   function `save_derived(char const*)':
> D:\data\boost\boost\libs\serialization\test/test_no_rtti.cpp:54:
> undefined reference to `__imp__ZN20polymorphic_derived2C1Ev'
>
> And others, but focussing on the first error,
> __imp__ZN20polymorphic_derived2C1Ev is an exported default constructor
> for polymorphic_derived2.
>
> * If I build with b2 -d2 and capture the command lines, I can preprocess
> polymorphic_derived2.cpp and check the declarations, and indeed I see:
>
> class polymorphic_derived2 :
>      public polymorphic_base
> {
>      friend class boost::serialization::access;
>      template<class Archive>
>      __attribute__((__dllexport__)) void serialize(
>          Archive &ar,
>          const unsigned int
>      );
>      __attribute__((__dllexport__)) const char * get_key() const;
> public:
>      __attribute__((__dllexport__)) polymorphic_derived2();
>      __attribute__((__dllexport__)) ~polymorphic_derived2();
> };
>
> and
>
> __attribute__((__dllexport__))
> polymorphic_derived2::polymorphic_derived2(){}
> __attribute__((__dllexport__))
> polymorphic_derived2::~polymorphic_derived2(){}
>
> So that all looks OK.  But,
>
> * If I examine the object files with nm, I see symbols for
> polymorphic_derived2 *without* the leading "__imp_" - ie regular
> non-exported symbols.
>
>
> At this point I'm stumped.  Anyone else?

OK - due to some sleuthing on the part of one of the auxiliary
maintainers of the boost serialization library, it's seems that the
problem is that

BOOST_SYMBOL_IMPORT is not defined to be __attribute__((__dllimport__))
under MINGW and that is the source of the problem. It's totally
plausible since gcc visibility attributes are only applied to exports
and it seems that in moving to windows this rule is carried over.  So
I'm thinking that BOOST_SYMBOL_EXPORT needs to be updated for a special
case of MINGW.

Of course, this is subject to confirmation. Hope this is helpful.

It's also been mentioned that the rules for application of these
attributes might be different for functions and variables.  The thought
of this makes me ill.  I don't know if it's true, but it's one more
thing to keep things interesting.

Robert Ramey

>
> John.
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> _______________________________________________
> Boost-users mailing list
> [hidden email]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list

> OK - due to some sleuthing on the part of one of the auxiliary
> maintainers of the boost serialization library, it's seems that the
> problem is that
>
> BOOST_SYMBOL_IMPORT is not defined to be
> __attribute__((__dllimport__)) under MINGW and that is the source of
> the problem.


That's not true, gcc.hpp has:

#    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
#    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))

when _Win32 is defined.

And if I modify config_info.cpp to print these out then I see:

     BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__))
     BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__))
     BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default")))

Which looks good to me.


John.


> It's totally plausible since gcc visibility attributes are only
> applied to exports and it seems that in moving to windows this rule is
> carried over.  So I'm thinking that BOOST_SYMBOL_EXPORT needs to be
> updated for a special case of MINGW.
>
> Of course, this is subject to confirmation. Hope this is helpful.
>
> It's also been mentioned that the rules for application of these
> attributes might be different for functions and variables.  The
> thought of this makes me ill.  I don't know if it's true, but it's one
> more thing to keep things interesting.
>
> Robert Ramey
>>
>> John.
>>
>>
>> ---
>> This email has been checked for viruses by Avast antivirus software.
>> https://www.avast.com/antivirus
>>
>> _______________________________________________
>> Boost-users mailing list
>> [hidden email]
>> https://lists.boost.org/mailman/listinfo.cgi/boost-users
>
>
> _______________________________________________
> Boost-users mailing list
> [hidden email]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
On 20/11/2018 08:07, John Maddock wrote:

>> BOOST_SYMBOL_IMPORT is not defined to be
>> __attribute__((__dllimport__)) under MINGW and that is the source of
>> the problem.
>
> That's not true, gcc.hpp has:
>
> #    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
> #    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
>
> when _Win32 is defined.

Only for GCC >= 4.0 though.  Perhaps they have an older gcc?
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
Mere moments ago, quoth I:

> On 20/11/2018 08:07, John Maddock wrote:
>>> BOOST_SYMBOL_IMPORT is not defined to be
>>> __attribute__((__dllimport__)) under MINGW and that is the source of
>>> the problem.
>>
>> That's not true, gcc.hpp has:
>>
>> #    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
>> #    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
>>
>> when _Win32 is defined.
>
> Only for GCC >= 4.0 though.  Perhaps they have an older gcc?

Also, why does that check use __GNUC__ instead of BOOST_GCC_VERSION?
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
In reply to this post by Boost - Users mailing list

On 19/11/2018 19:07, John Maddock wrote:

>
>> OK - due to some sleuthing on the part of one of the auxiliary
>> maintainers of the boost serialization library, it's seems that the
>> problem is that
>>
>> BOOST_SYMBOL_IMPORT is not defined to be
>> __attribute__((__dllimport__)) under MINGW and that is the source of
>> the problem.
>
>
> That's not true, gcc.hpp has:
>
> #    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
> #    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
>
> when _Win32 is defined.
>
> And if I modify config_info.cpp to print these out then I see:
>
>     BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__))
>     BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__))
>     BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default")))
>
> Which looks good to me.

I *think* I may have twigged on to what's happening here.

To recap, msvc when exporting a symbol from a dll mangles the name with
a __imp_ prefix.  Further it performs "early name mangling" - ie the
compiler does it and the mangled name for the export ends up in the
object file.

GCC normally does not mangle names when changing visibility, instead it
stores "something" in the object file to tell the linker to make the
symbol visible.  On windows it's the *linker* that takes this
information and creates an alias with the appropriately mangled name
when building the dll.  That means that the object files do not contain
records with the __imp_ prefix, only the finished dll.

However, when importing, since dllimport suggests import from a dll, gcc
mangles the name immediately so that the linker knows to look for a
__imp_ prefixed name.

Now look at what you're doing in these failing tests: you're mixing
object files *in the same executable* some of which are using dllexport
and some using dllimport and now we have a mismatch with dllimport
implying the __imp_ prefix but there are no matching __imp_ prefixed
records in the object files.

I think this is a bug in your tests, albeit a very obscure one. To fix
you would need to either:

* Use dllexport consistently throughout an executable: ie if a class is
dllexport in one object file in the dll or exe, then it should be
dllexport in *all the object files in that dll/exe*, including in object
files that only use those symbols not define them.

* Don't export these symbols: so far as I can tell, the exported symbols
are never imported from anything (since we never build a dll with them)
so exporting is completely redundant in this case. You may still need
BOOST_SYMBOL_VISIBLE on them on for *nix systems, not sure about that
though.

* Or, put these exported symbols in a dll and link the test program to it.

Hopefully that makes sense, John.


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
In reply to this post by Boost - Users mailing list

On 20/11/2018 03:52, Gavin Lambert via Boost-users wrote:

> On 20/11/2018 08:07, John Maddock wrote:
>>> BOOST_SYMBOL_IMPORT is not defined to be
>>> __attribute__((__dllimport__)) under MINGW and that is the source of
>>> the problem.
>>
>> That's not true, gcc.hpp has:
>>
>> #    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
>> #    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
>>
>> when _Win32 is defined.
>
> Only for GCC >= 4.0 though.  Perhaps they have an older gcc?

No I can reproduce with gcc-8.  See my other post for an explanation.

John.



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
In reply to this post by Boost - Users mailing list

> Also, why does that check use __GNUC__ instead of BOOST_GCC_VERSION?

It's overkill, since we only need to major version number.



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 11/20/18 9:35 AM, John Maddock via Boost-users wrote:

>
> On 19/11/2018 19:07, John Maddock wrote:
>>
>>> OK - due to some sleuthing on the part of one of the auxiliary
>>> maintainers of the boost serialization library, it's seems that the
>>> problem is that
>>>
>>> BOOST_SYMBOL_IMPORT is not defined to be
>>> __attribute__((__dllimport__)) under MINGW and that is the source of
>>> the problem.
>>
>>
>> That's not true, gcc.hpp has:
>>
>> #    define BOOST_SYMBOL_EXPORT __attribute__((__dllexport__))
>> #    define BOOST_SYMBOL_IMPORT __attribute__((__dllimport__))
>>
>> when _Win32 is defined.
>>
>> And if I modify config_info.cpp to print these out then I see:
>>
>>     BOOST_SYMBOL_EXPORT =__attribute__((__dllexport__))
>>     BOOST_SYMBOL_IMPORT =__attribute__((__dllimport__))
>>     BOOST_SYMBOL_VISIBLE =__attribute__((__visibility__("default")))
>>
>> Which looks good to me.
>
> I *think* I may have twigged on to what's happening here.
>
> To recap, msvc when exporting a symbol from a dll mangles the name with
> a __imp_ prefix.  Further it performs "early name mangling" - ie the
> compiler does it and the mangled name for the export ends up in the
> object file.
>
> GCC normally does not mangle names when changing visibility, instead it
> stores "something" in the object file to tell the linker to make the
> symbol visible.  On windows it's the *linker* that takes this
> information and creates an alias with the appropriately mangled name
> when building the dll.  That means that the object files do not contain
> records with the __imp_ prefix, only the finished dll.
>
> However, when importing, since dllimport suggests import from a dll, gcc
> mangles the name immediately so that the linker knows to look for a
> __imp_ prefixed name.
>
> Now look at what you're doing in these failing tests: you're mixing
> object files *in the same executable* some of which are using dllexport
> and some using dllimport and now we have a mismatch with dllimport
> implying the __imp_ prefix but there are no matching __imp_ prefixed
> records in the object files.
>
> I think this is a bug in your tests, albeit a very obscure one. To fix
> you would need to either:
>
> * Use dllexport consistently throughout an executable: ie if a class is
> dllexport in one object file in the dll or exe, then it should be
> dllexport in *all the object files in that dll/exe*, including in object
> files that only use those symbols not define them.

Right.  I don't think I'm doing this - but I'll double check.
>
> * Don't export these symbols: so far as I can tell, the exported symbols
> are never imported from anything (since we never build a dll with them)
> so exporting is completely redundant in this case. You may still need
> BOOST_SYMBOL_VISIBLE on them on for *nix systems, not sure about that
> though.
>
> * Or, put these exported symbols in a dll and link the test program to it.

I believe that that is what I'm doing.  Again, we'll double check.  This
is the test test_dll_exported...

>
> Hopefully that makes sense, John.

I'm sure it does.  But not to me yet.  I'll spend some time
investigating this with the information you've provided.

Note that in the test there is dll_polymorphic_derived2.  This exports a
couple of functions.  But it depends on dll_polymorphic_base and imports
some symbols from that.  In C++, the class derived2 inherits from the
class base.  So within the same module we're both exporting and
importing albeit different symbols.  The normal windows tests seem to be
fine with this.  I'll try to figure out what mingw does differently.

Thanks for your help with this.  I couldn't do it by myself.

Robert Ramey


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: questions about visibility/dlls under mingw

Boost - Users mailing list

> Note that in the test there is dll_polymorphic_derived2.  This exports
> a couple of functions.  But it depends on dll_polymorphic_base and
> imports some symbols from that.  In C++, the class derived2 inherits
> from the class base.  So within the same module we're both exporting
> and importing albeit different symbols.  The normal windows tests seem
> to be fine with this.  I'll try to figure out what mingw does
> differently.
>

To take a concrete example, in the failing
"test_no_rtti_polymorphic_text_archive", both polymorphic_derived2.o and
test_no_rtti.o are linked directly into the executable (no dll's
involved), but in the former you are dllexport'ing the class and the
latter dllimport'ing, and since it's all one big exectutable with no
dll's the import/export is redundant and ideally you would be doing
neither, but if you must mark them up, then it should be dllexport all
the way.

John.



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users