Re: Serialization : unregistered_cast & otherquestions

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel

Thanks for all your help. I have decided to try to reorganize everything into small parts as you have said, in order to better understand.

 

I am in the process of doing this, but I have already some remarks about your comments.

 

I have chosen the organisation principle you have said:

 

in the class.h, nothing about the serialization is present

 

in serialization/class_serialization.hh : I have the serialization declaration code + some traits

 

in serialization/class_serialization.cc : I have the serialization implementation code + BOOST_EXPORT

 

The only thing where I have not followed your advice is the presence of BOOST_EXPORT on the class_serialization.hh rather than class_serialization.cc.

 

If I put it on the header, since I have to include the header on test.cc, one instantation of something is done while encountering BOOST_EXPORT. Later while compiling class_serialization.cc, since it encounters BOOST_EXPORT one more time another instantiation occurs and I had problems with linking (gcc 3.4 reported multiple definitions of something).

 

I have attached a small set of code that illustrates the problem. As the code is, it will fail to compile (at least on gcc 3.4).

If you just remove BOOST_EXPORT from MObject_serialization.hh and put it into MObject_serialization.cc, everything works and the test passes.

 

I’d like to get feedback on this – did I get everything right ??

 

Jean-Noël

 

 

Ps: I now even have a clue about why the code failed in my old design, but I am waiting until I am more sure of myself…

 

 

 


De : [hidden email] [mailto:[hidden email]] De la part de Robert Ramey
Envoyé : jeudi 23 février 2006 17:44
À : [hidden email]
Objet : Re: [Boost-users] Serialization : unregistered_cast & otherquestions

 

Without actually seeing the code, its hard for me to give intelligent advice. Lucky for

you (maybe) that's not going to inhibit me from speculating. Here is what I recommend

for anyone making a very large program:

 

a) extended_type_info_no_rtti presumes that all classes serialized as pointers

have been exported. Should this not be the case, you'll get the error message indicated.

Double check this.

 

b) Serilaization traits should be in the header files so that they are not "missed" anytime

something like "ar << x" is invoked. Given you're situation I would recommend a file

structure like the following:

 

class.hpp - without seriaiization like you have now.

 

serialization/class.hpp - includes class.hpp. also includes BOOST_EXPORT and any

other traits for this "class".

 

serialization/class.cpp c - includes serialization/class.hpp and includes the

actual serialization code. Also this should include code instantiation for the archives that

you use. Look at demo_pimpl to see how this is done.

 

All the above code should be organized into a library. The reason is that this will generate

code for all the archives mentioned but your application may not use all the archives so

putting the above code in a library and linking to it will mean that your application will

always have what it actuallyuses but nothing it doesn't - avoids "code bloat". Note that

in compiling code for the library make sure to specify "function level linking".

 

Take the "least derived class" - that one at the top of the class hierarchy. Make a small

test program similar to the tests in the serialization library. This should be a very small

job. This code can be class_test.cpp.  Run this test on all archives you use.  When

this test passes, do the same for the other classes until something fails.

 

This might seem to be more work - believe me its not. It will

 

a) guarentee that you make progress toward a robust program

b) give you confidence that your program has minimum of hidden bugs

c) permit you to re-run the tests automatically when you make future changes.

d) should a problem arise with the serialization or other library, it will give

a lot more information in order to be helpful.

 

Robert Ramey

 

Now, the problem is where to put the serialization code for these classes. If I just write them on the implementation file (the .cc) for the class, it does not work. At run time I have always errors telling me that either the class is not exported (unregistered_class exception) or an unregistered_cast happens.

 

 

 

 

The only solution I have found is to put *ALL* serialization code into the same big .cc file. If I do just that, it works. However this file is becoming enormous and has to be recompiled every time I change only *one* of the class to be serialized. It is already taking a minute to compile on a fast machine, so I must really break it up into other parts.

 

But everytime I do that everything stops working.

 

Have you any ideas, Robert? I am using extended_type_info_no_rtti, this may be the cause of the problem…

 

Jean-Noël

 

Ps: I still have the std::string bug, which is also uncomprehensible for me.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel

I forgot the code promised in last email sorry…

 


De : [hidden email] [mailto:[hidden email]] De la part de Robert Ramey
Envoyé : jeudi 23 février 2006 17:44
À : [hidden email]
Objet : Re: [Boost-users] Serialization : unregistered_cast & otherquestions

 

Without actually seeing the code, its hard for me to give intelligent advice. Lucky for

you (maybe) that's not going to inhibit me from speculating. Here is what I recommend

for anyone making a very large program:

 

a) extended_type_info_no_rtti presumes that all classes serialized as pointers

have been exported. Should this not be the case, you'll get the error message indicated.

Double check this.

 

b) Serilaization traits should be in the header files so that they are not "missed" anytime

something like "ar << x" is invoked. Given you're situation I would recommend a file

structure like the following:

 

class.hpp - without seriaiization like you have now.

 

serialization/class.hpp - includes class.hpp. also includes BOOST_EXPORT and any

other traits for this "class".

 

serialization/class.cpp c - includes serialization/class.hpp and includes the

actual serialization code. Also this should include code instantiation for the archives that

you use. Look at demo_pimpl to see how this is done.

 

All the above code should be organized into a library. The reason is that this will generate

code for all the archives mentioned but your application may not use all the archives so

putting the above code in a library and linking to it will mean that your application will

always have what it actuallyuses but nothing it doesn't - avoids "code bloat". Note that

in compiling code for the library make sure to specify "function level linking".

 

Take the "least derived class" - that one at the top of the class hierarchy. Make a small

test program similar to the tests in the serialization library. This should be a very small

job. This code can be class_test.cpp.  Run this test on all archives you use.  When

this test passes, do the same for the other classes until something fails.

 

This might seem to be more work - believe me its not. It will

 

a) guarentee that you make progress toward a robust program

b) give you confidence that your program has minimum of hidden bugs

c) permit you to re-run the tests automatically when you make future changes.

d) should a problem arise with the serialization or other library, it will give

a lot more information in order to be helpful.

 

Robert Ramey

 

Now, the problem is where to put the serialization code for these classes. If I just write them on the implementation file (the .cc) for the class, it does not work. At run time I have always errors telling me that either the class is not exported (unregistered_class exception) or an unregistered_cast happens.

 

 

 

 

The only solution I have found is to put *ALL* serialization code into the same big .cc file. If I do just that, it works. However this file is becoming enormous and has to be recompiled every time I change only *one* of the class to be serialized. It is already taking a minute to compile on a fast machine, so I must really break it up into other parts.

 

But everytime I do that everything stops working.

 

Have you any ideas, Robert? I am using extended_type_info_no_rtti, this may be the cause of the problem…

 

Jean-Noël

 

Ps: I still have the std::string bug, which is also uncomprehensible for me.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users

boost_serialization_example.zip (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Robert Ramey
In reply to this post by RIVASSEAU Jean Noel
 
"RIVASSEAU Jean Noel" <[hidden email]> wrote in message <A href="news:87F60F7FA02FF749AFB02BD9FCAA6B04DA737E@naserv31.nanterre.oberthurcs.com">news:87F60F7FA02FF749AFB02BD9FCAA6B04DA737E@......

Thanks for all your help. I have decided to try to reorganize everything into small parts as you have said, in order to better understand.

 

I am in the process of doing this, but I have already some remarks about your comments.

 

I have chosen the organisation principle you have said:

 

in the class.h, nothing about the serialization is present

 

in serialization/class_serialization.hh : I have the serialization declaration code + some traits

 

in serialization/class_serialization.cc : I have the serialization implementation code + BOOST_EXPORT

 

The only thing where I have not followed your advice is the presence of BOOST_EXPORT on the class_serialization.hh rather than class_serialization.cc.

 

If I put it on the header, since I have to include the header on test.cc, one instantation of something is done while encountering BOOST_EXPORT. Later while compiling class_serialization.cc, since it encounters BOOST_EXPORT one more time another instantiation occurs and I had problems with linking (gcc 3.4 reported multiple definitions of something).

 

****

OK - BOOST_EXPORT has some non-obvious behavior which I'll explain.  Maybe this will help.

 

BOOST_EXPORT has two functions:

 

a) To associate a class with an external unique name (GUID)

b) To instanticiate templated code for archives used.

 

The second is not as obvious as it might seem.  Suppose I have:

 

class A {

};

 

class B : public A {

}

 

A * aptr = new B;

 

text_archive ar;

 

ar << a;

 

The problem is that the compiler never sees anything like ar << b so it has no way to know that the

serialization code for B needs to be instantiated for the archive class text_archive.

 

So each archive class defines a special macro.

 

Then BOOST_EXPORT instantiates code for each combination of the class and archive class.  If this were

not done, one would get a link error with undefined symbol for load<text_archive, B> or something lke that.

 

Of course the above presumes that the archive classes are included BEFORE the serialization/boost/export.hpp

header. 

 

So if you make the following changes in your code I believe you will get the desired results.

 

a) Move BOOST_EXPORT to the class header.

b) In test.cc - move the headers boost/archive/... above the headers boost/serialization/..

 

and let me know that happens.

 

 

Robert Ramey

 

 


De : [hidden email] [mailto:[hidden email]] De la part de Robert Ramey
Envoyé : jeudi 23 février 2006 17:44
À : [hidden email]
Objet : Re: [Boost-users] Serialization : unregistered_cast & otherquestions

 

Without actually seeing the code, its hard for me to give intelligent advice. Lucky for

you (maybe) that's not going to inhibit me from speculating. Here is what I recommend

for anyone making a very large program:

 

a) extended_type_info_no_rtti presumes that all classes serialized as pointers

have been exported. Should this not be the case, you'll get the error message indicated.

Double check this.

 

b) Serilaization traits should be in the header files so that they are not "missed" anytime

something like "ar << x" is invoked. Given you're situation I would recommend a file

structure like the following:

 

class.hpp - without seriaiization like you have now.

 

serialization/class.hpp - includes class.hpp. also includes BOOST_EXPORT and any

other traits for this "class".

 

serialization/class.cpp c - includes serialization/class.hpp and includes the

actual serialization code. Also this should include code instantiation for the archives that

you use. Look at demo_pimpl to see how this is done.

 

All the above code should be organized into a library. The reason is that this will generate

code for all the archives mentioned but your application may not use all the archives so

putting the above code in a library and linking to it will mean that your application will

always have what it actuallyuses but nothing it doesn't - avoids "code bloat". Note that

in compiling code for the library make sure to specify "function level linking".

 

Take the "least derived class" - that one at the top of the class hierarchy. Make a small

test program similar to the tests in the serialization library. This should be a very small

job. This code can be class_test.cpp.  Run this test on all archives you use.  When

this test passes, do the same for the other classes until something fails.

 

This might seem to be more work - believe me its not. It will

 

a) guarentee that you make progress toward a robust program

b) give you confidence that your program has minimum of hidden bugs

c) permit you to re-run the tests automatically when you make future changes.

d) should a problem arise with the serialization or other library, it will give

a lot more information in order to be helpful.

 

Robert Ramey

 

Now, the problem is where to put the serialization code for these classes. If I just write them on the implementation file (the .cc) for the class, it does not work. At run time I have always errors telling me that either the class is not exported (unregistered_class exception) or an unregistered_cast happens.

 

 

 

 

The only solution I have found is to put *ALL* serialization code into the same big .cc file. If I do just that, it works. However this file is becoming enormous and has to be recompiled every time I change only *one* of the class to be serialized. It is already taking a minute to compile on a fast machine, so I must really break it up into other parts.

 

But everytime I do that everything stops working.

 

Have you any ideas, Robert? I am using extended_type_info_no_rtti, this may be the cause of the problem…

 

Jean-Noël

 

Ps: I still have the std::string bug, which is also uncomprehensible for me.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel
In reply to this post by RIVASSEAU Jean Noel

 

Hello Robert,

 

Thanks for your reply. However you did not understand the problem. I read the documentation and understood the purpose of BOOST_EXPORT as explained in the documentation, and as reexplained in your last mail. The problem is at linking, but is not one of undefined reference (template not instantiated), but, on the other hand, *a multiple definition problem.*

 

If, as suggested, you move BOOST_EXPORT to the class header, even if you move the archive headers before the serialization/export header, you will have multiple instantiations of the same template. This is because every time you include the serialization class header in your program there is some code instantiation, since the EXPORT is in that file.  Since this header is included several times in the  program, multiple instantiations of the same code occurs (note: apparently  this happens even if there is no definition of the templated  code to be instantiated (since the implementation is  only in the .cc), only a declaration, because BOOST_EXPORT instantiates some kind of *helper code* that gets multiple defined).

 

Anyway, if you can change the code in order to avoid this multiple definition problem, I’ll be glad to test, but I believe it is impossible. However, putting BOOST_EXPORT in the .cc avoids the problem entirely, so this is acceptable.

 

I am sending again code with your suggested changes, in order to show you your  method does not work.

 

If you look at the code you will see that each serialization header includes <CommonSerialize.h> which contains the archive headers. Then after <CommonSerialize.h> inclusion comes serialization/export.hpp inclusion, so the archive headers are BEFORE export.hpp.

 

Then follows BOOST_EXPORT. At linking I get the multiple definition problem that I mentioned. If you just move BOOST_EXPORT into the .cc no problems at all.

 

You can check for yourself if you try to compile the code.

 

Thank you!!

 

Jean-Noël

 

Ps:  I ran into two other problems that I’ll write mail for later…

 

 

 

 

 

OK - BOOST_EXPORT has some non-obvious behavior which I'll explain. Maybe this will help.

 

 

 

 

BOOST_EXPORT has two functions:

 

a) To associate a class with an external unique name (GUID)

b) To instanticiate templated code for archives used.

 

The second is not as obvious as it might seem. Suppose I have:

 

class A {

};

 

class B : public A {

}

 

A * aptr = new B;

 

text_archive ar;

 

ar << a;

 

The problem is that the compiler never sees anything like ar << b so it has no way to know that the

serialization code for B needs to be instantiated for the archive class text_archive.

 

So each archive class defines a special macro.

 

Then BOOST_EXPORT instantiates code for each combination of the class and archive class. If this were

not done, one would get a link error with undefined symbol for load<text_archive, B> or something lke that.

 

Of course the above presumes that the archive classes are included BEFORE the serialization/boost/export.hpp

header.

 

So if you make the following changes in your code I believe you will get the desired results.

 

a) Move BOOST_EXPORT to the class header.

 

b) In test.cc - move the headers boost/archive/... above the headers boost/serialization/..

 

and let me know that happens.

 

 

Robert Ramey

 

 

 

 

 


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel
In reply to this post by RIVASSEAU Jean Noel

Again I forgot to include the code sorry…

 


De : [hidden email] [mailto:[hidden email]] De la part de Robert Ramey
Envoyé : vendredi 24 février 2006 17:53
À : [hidden email]
Objet : Re: [Boost-users] Serialization : unregistered_cast & otherquestions

 

 

"RIVASSEAU Jean Noel" <[hidden email]> wrote in message <a href="news:87F60F7FA02FF749AFB02BD9FCAA6B04DA737E@naserv31.nanterre.oberthurcs.com">news:87F60F7FA02FF749AFB02BD9FCAA6B04DA737E@......

Thanks for all your help. I have decided to try to reorganize everything into small parts as you have said, in order to better understand.

 

I am in the process of doing this, but I have already some remarks about your comments.

 

I have chosen the organisation principle you have said:

 

in the class.h, nothing about the serialization is present

 

in serialization/class_serialization.hh : I have the serialization declaration code + some traits

 

in serialization/class_serialization.cc : I have the serialization implementation code + BOOST_EXPORT

 

The only thing where I have not followed your advice is the presence of BOOST_EXPORT on the class_serialization.hh rather than class_serialization.cc.

 

If I put it on the header, since I have to include the header on test.cc, one instantation of something is done while encountering BOOST_EXPORT. Later while compiling class_serialization.cc, since it encounters BOOST_EXPORT one more time another instantiation occurs and I had problems with linking (gcc 3.4 reported multiple definitions of something).

 

****

OK - BOOST_EXPORT has some non-obvious behavior which I'll explain. Maybe this will help.

 

BOOST_EXPORT has two functions:

 

a) To associate a class with an external unique name (GUID)

b) To instanticiate templated code for archives used.

 

The second is not as obvious as it might seem. Suppose I have:

 

class A {

};

 

class B : public A {

}

 

A * aptr = new B;

 

text_archive ar;

 

ar << a;

 

The problem is that the compiler never sees anything like ar << b so it has no way to know that the

serialization code for B needs to be instantiated for the archive class text_archive.

 

So each archive class defines a special macro.

 

Then BOOST_EXPORT instantiates code for each combination of the class and archive class. If this were

not done, one would get a link error with undefined symbol for load<text_archive, B> or something lke that.

 

Of course the above presumes that the archive classes are included BEFORE the serialization/boost/export.hpp

header.

 

So if you make the following changes in your code I believe you will get the desired results.

 

a) Move BOOST_EXPORT to the class header.

 

b) In test.cc - move the headers boost/archive/... above the headers boost/serialization/..

 

and let me know that happens.

 

 

Robert Ramey

 

 


De : [hidden email] [mailto:[hidden email]] De la part de Robert Ramey
Envoyé : jeudi 23 février 2006 17:44
À : [hidden email]
Objet : Re: [Boost-users] Serialization : unregistered_cast & otherquestions

 

Without actually seeing the code, its hard for me to give intelligent advice. Lucky for

you (maybe) that's not going to inhibit me from speculating. Here is what I recommend

for anyone making a very large program:

 

a) extended_type_info_no_rtti presumes that all classes serialized as pointers

have been exported. Should this not be the case, you'll get the error message indicated.

Double check this.

 

b) Serilaization traits should be in the header files so that they are not "missed" anytime

something like "ar << x" is invoked. Given you're situation I would recommend a file

structure like the following:

 

class.hpp - without seriaiization like you have now.

 

serialization/class.hpp - includes class.hpp. also includes BOOST_EXPORT and any

other traits for this "class".

 

serialization/class.cpp c - includes serialization/class.hpp and includes the

actual serialization code. Also this should include code instantiation for the archives that

you use. Look at demo_pimpl to see how this is done.

 

All the above code should be organized into a library. The reason is that this will generate

code for all the archives mentioned but your application may not use all the archives so

putting the above code in a library and linking to it will mean that your application will

always have what it actuallyuses but nothing it doesn't - avoids "code bloat". Note that

in compiling code for the library make sure to specify "function level linking".

 

Take the "least derived class" - that one at the top of the class hierarchy. Make a small

test program similar to the tests in the serialization library. This should be a very small

job. This code can be class_test.cpp.  Run this test on all archives you use.  When

this test passes, do the same for the other classes until something fails.

 

This might seem to be more work - believe me its not. It will

 

a) guarentee that you make progress toward a robust program

b) give you confidence that your program has minimum of hidden bugs

c) permit you to re-run the tests automatically when you make future changes.

d) should a problem arise with the serialization or other library, it will give

a lot more information in order to be helpful.

 

Robert Ramey

 

Now, the problem is where to put the serialization code for these classes. If I just write them on the implementation file (the .cc) for the class, it does not work. At run time I have always errors telling me that either the class is not exported (unregistered_class exception) or an unregistered_cast happens.

 

 

 

 

The only solution I have found is to put *ALL* serialization code into the same big .cc file. If I do just that, it works. However this file is becoming enormous and has to be recompiled every time I change only *one* of the class to be serialized. It is already taking a minute to compile on a fast machine, so I must really break it up into other parts.

 

But everytime I do that everything stops working.

 

Have you any ideas, Robert? I am using extended_type_info_no_rtti, this may be the cause of the problem…

 

Jean-Noël

 

Ps: I still have the std::string bug, which is also uncomprehensible for me.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users

export_problem.zip (15K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Robert Ramey
In reply to this post by RIVASSEAU Jean Noel

"RIVASSEAU Jean Noel" <[hidden email]> wrote in message
news:[hidden email]...

Hello Robert,

Thanks for your reply. However you did not understand the problem. I read
the documentation and understood the purpose of BOOST_EXPORT as explained in
the documentation, and as reexplained in your last mail. The problem is at
linking, but is not one of undefined reference (template not instantiated),
but, on the other hand, *a multiple definition problem.*

If, as suggested, you move BOOST_EXPORT to the class header, even if you
move the archive headers before the serialization/export header, you will
have multiple instantiations of the same template. This is because every
time you include the serialization class header in your program there is
some code instantiation, since the EXPORT is in that file.

*** Hmmm - I'll look into this.  I would have expected that additional code
would be generated if and only if <boost/archive/...
headers are included.  If these are included only once, I wouldn't expect
any multiple definitions. The top of export.hpp
includes the following:

// if no archive headers have been included this is a no op
// this is to permit BOOST_EXPORT etc to be included in a
// file declaration header
#if ! defined(BOOST_ARCHIVE_BASIC_ARCHIVE_HPP)
#define BOOST_CLASS_EXPORT_GUID_ARCHIVE_LIST(T, K, ASEQ)

which I would expect would make a BOOST_EXPORT an effective no-op if no
archive files are included.
****

Robert Ramey



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Robert Ramey
In reply to this post by RIVASSEAU Jean Noel
I would recommend the following changes:

CommonSerialize.h - remove the following

#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>

and add the following:

#include <boost/serialization/export.hpp>

test_independent.cc - change the top to look like the following:

// INDEPENDENT TEST

#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>

#include <MObject.h>
#include <MObject_serialization.hh>
#include <MemberVariableSpecification.h>
#include <MemberVariableSpecification_serialization.hh>
#include <fstream>

Robert Ramey



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Hugh Hoover
In reply to this post by Robert Ramey

On Feb 27, 2006, at 08:51, Robert Ramey wrote:

> *** Hmmm - I'll look into this.  I would have expected that  
> additional code
> would be generated if and only if <boost/archive/...
> headers are included.  If these are included only once, I wouldn't  
> expect
> any multiple definitions. The top of export.hpp
> includes the following:
>
> // if no archive headers have been included this is a no op
> // this is to permit BOOST_EXPORT etc to be included in a
> // file declaration header
> #if ! defined(BOOST_ARCHIVE_BASIC_ARCHIVE_HPP)
> #define BOOST_CLASS_EXPORT_GUID_ARCHIVE_LIST(T, K, ASEQ)
>
> which I would expect would make a BOOST_EXPORT an effective no-op  
> if no
> archive files are included.

I'll chime in, as I'm having exactly the same issue, but on darwin/
gcc-4 with code in multiple dynamically loaded libraries... (I have  
other issues too, I get to those later)

The problem is that although export.hpp is only included once per  
generated .o file, there are multiple .o files all including the same  
headers - and each therefore generates the same template  
instantiations, thereby causing the multiple definitions during  
linking.  Also - the same headers may be included in multiple shared  
libraries, exacerbating the problem with the same code instantiated  
into multiple libraries - a serious problem if there are singletons.

I may not be understanding this correctly - here's what >I< think  
needs to happen - please correct me.

For any derived class, there is a set of code (templates) that need  
to be instantiated for each archive type (specifically, the list  
boost::archive::detail::known_archive_types::type, for the templates  
guid_initializer<T> and export_instantiate<T, archive_list>)
 From what I see, these are NOT declarations required by the other  
serialization code, but singletons that, during construction,  
register the serializers to the derived classes with the  
extended_type_info class registry.  Is that incorrect?

If the above is correct, then the BOOST_EXPORT_CLASS should only be  
in exactly one implementation file (.cpp, .C, .cc, whatever).  And  
also, THAT implementation file should ONLY be linked into one shared  
library.

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Hugh Hoover
In reply to this post by Robert Ramey

On Feb 27, 2006, at 08:51, Robert Ramey wrote:

> *** Hmmm - I'll look into this.  I would have expected that  
> additional code
> would be generated if and only if <boost/archive/...
> headers are included.  If these are included only once, I wouldn't  
> expect
> any multiple definitions. The top of export.hpp
> includes the following:
>
> // if no archive headers have been included this is a no op
> // this is to permit BOOST_EXPORT etc to be included in a
> // file declaration header
> #if ! defined(BOOST_ARCHIVE_BASIC_ARCHIVE_HPP)
> #define BOOST_CLASS_EXPORT_GUID_ARCHIVE_LIST(T, K, ASEQ)
>
> which I would expect would make a BOOST_EXPORT an effective no-op  
> if no
> archive files are included.
>

I'll chime in, as I'm having exactly the same issue, but on darwin/
gcc-4 with code in multiple dynamically loaded libraries... (I have  
other issues too, I get to those later)

The problem is that although export.hpp is only included once per  
generated .o file, there are multiple .o files all including the same  
headers - and each therefore generates the same template  
instantiations, thereby causing the multiple definitions during  
linking.  Also - the same headers may be included in multiple shared  
libraries, exacerbating the problem with the same code instantiated  
into multiple libraries - a serious problem if there are singletons.

I may not be understanding this correctly - here's what >I< think  
needs to happen - please correct me.

For any derived class, there is a set of code (templates) that need  
to be instantiated for each archive type (specifically, the list  
boost::archive::detail::known_archive_types::type, for the templates  
guid_initializer<T> and export_instantiate<T, archive_list>)
 From what I see, these are NOT declarations required by the other  
serialization code, but singletons that, during construction,  
register the serializers to the derived classes with the  
extended_type_info class registry.  Is that incorrect?

If the above is correct, then the BOOST_EXPORT_CLASS should only be  
in exactly one implementation file (.cpp, .C, .cc, whatever).  And  
also, THAT implementation file should ONLY be linked into one shared  
library.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Robert Ramey
In reply to this post by Hugh Hoover
Hugh Hoover wrote:

> On Feb 27, 2006, at 08:51, Robert Ramey wrote:
>> *** Hmmm - I'll look into this.  I would have expected that
>> additional code
>> would be generated if and only if <boost/archive/...
>> headers are included.  If these are included only once, I wouldn't
>> expect
>> any multiple definitions. The top of export.hpp
>> includes the following:
>>
>> // if no archive headers have been included this is a no op
>> // this is to permit BOOST_EXPORT etc to be included in a
>> // file declaration header
>> #if ! defined(BOOST_ARCHIVE_BASIC_ARCHIVE_HPP)
>> #define BOOST_CLASS_EXPORT_GUID_ARCHIVE_LIST(T, K, ASEQ)
>>
>> which I would expect would make a BOOST_EXPORT an effective no-op
>> if no
>> archive files are included.
>
> I'll chime in, as I'm having exactly the same issue, but on darwin/
> gcc-4 with code in multiple dynamically loaded libraries... (I have
> other issues too, I get to those later)
>
> The problem is that although export.hpp is only included once per
> generated .o file, there are multiple .o files all including the same
> headers - and each therefore generates the same template
> instantiations, thereby causing the multiple definitions during
> linking.  Also - the same headers may be included in multiple shared
> libraries, exacerbating the problem with the same code instantiated
> into multiple libraries - a serious problem if there are singletons.
>
> I may not be understanding this correctly - here's what >I< think
> needs to happen - please correct me.
>
> For any derived class, there is a set of code (templates) that need
> to be instantiated for each archive type (specifically, the list
> boost::archive::detail::known_archive_types::type, for the templates
> guid_initializer<T> and export_instantiate<T, archive_list>)
> From what I see, these are NOT declarations required by the other
> serialization code, but singletons that, during construction,
> register the serializers to the derived classes with the
> extended_type_info class registry.  Is that incorrect?
>
> If the above is correct, then the BOOST_EXPORT_CLASS should only be
> in exactly one implementation file (.cpp, .C, .cc, whatever).  And
> also, THAT implementation file should ONLY be linked into one shared
> library.

Not quite.  There should be only one source module which includes
boost/archive/... declarations.  If there is more than one of these
then there is a risk of multiple definitions.  My practice is to:

a) include all boost/serialization/.. headers in each class file.
b) include NO boost/archive/... headers in any class file
c) make a "template instantiation module" which instanticiates
code for all archives I plan to use.  This can be either in the
app or in a library.
d) lihk and execute without problem.

Robert Ramey



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel
In reply to this post by RIVASSEAU Jean Noel
Hello,

CommonSerialize.h - remove the following

#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>

and add the following:

#include <boost/serialization/export.hpp>

test_independent.cc - change the top to look like the following:

// INDEPENDENT TEST

#include <boost/archive/xml_oarchive.hpp>
#include <boost/archive/xml_iarchive.hpp>

#include <MObject.h>
#include <MObject_serialization.hh>
#include <MemberVariableSpecification.h>
#include <MemberVariableSpecification_serialization.hh>
#include <fstream>




This won't work. Since you now have the archive headers in only one place, there is no multiple definition problem. However serialization code is *not* instantiated for MObject or MemberVariableSpecification. This is normal since when compiling the .cc that contains this code, there is no archive header included and so the export.hpp has no effect. The code is here but is not instantiated.

So in order to follow this advice:

a) include all boost/serialization/.. headers in each class file.
b) include NO boost/archive/... headers in any class file
c) make a "template instantiation module" which instanticiates code for all archives I plan to use.  This can be either in the app or in a library.
d) lihk and execute without problem.

I have to explicitly instantiate the serialization code for the needed archive in the .cc files. That is, do:

template void save<boost::archive::xml_oarchive>(
    boost::archive::text_oarchive & ar, const MObject & m,
    const unsigned int file_version
);

And same for load. This is what you do in demo_pimpl.cpp. Is that what you usually do and recommend to do?

This has the advantage to avoid big compilations (these code instantiations will only occur when the class to serialize has indeed changed).

However this has the inconvenience of manually instantiating every template (and if you want to change your archive format , eg from text to xml for example, everything needs to be changed).

Jean-Noël




_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel
In reply to this post by RIVASSEAU Jean Noel
Hello

I agree with you, and currently have the same comprehension of the
problem than you. I am currently also including BOOST_EXPORT_CLASS in
only one implementation file *per class*. This seems to work.

Robert is saying however, that even that can cause problem, but as
written on a previous message, I cannot manage to follow his advice on
including the archive headers in only one instantiation module (except
putting all serialization code into that module, which I did previously:
this is a maintenance nightmare, to the point that I ended up trying to
break my code into smaller places).

I'll chime in, as I'm having exactly the same issue, but on darwin/
gcc-4 with code in multiple dynamically loaded libraries... (I have  
other issues too, I get to those later)

The problem is that although export.hpp is only included once per  
generated .o file, there are multiple .o files all including the same  
headers - and each therefore generates the same template  
instantiations, thereby causing the multiple definitions during  
linking.  Also - the same headers may be included in multiple shared  
libraries, exacerbating the problem with the same code instantiated  
into multiple libraries - a serious problem if there are singletons.

I may not be understanding this correctly - here's what >I< think  
needs to happen - please correct me.

For any derived class, there is a set of code (templates) that need  
to be instantiated for each archive type (specifically, the list  
boost::archive::detail::known_archive_types::type, for the templates  
guid_initializer<T> and export_instantiate<T, archive_list>)
 From what I see, these are NOT declarations required by the other  
serialization code, but singletons that, during construction,  
register the serializers to the derived classes with the  
extended_type_info class registry.  Is that incorrect?

If the above is correct, then the BOOST_EXPORT_CLASS should only be  
in exactly one implementation file (.cpp, .C, .cc, whatever).  And  
also, THAT implementation file should ONLY be linked into one shared  
library.


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

Robert Ramey
In reply to this post by RIVASSEAU Jean Noel
RIVASSEAU Jean Noel wrote:

> Hello,
>
> CommonSerialize.h - remove the following
>
> #include <boost/archive/xml_oarchive.hpp>
> #include <boost/archive/xml_iarchive.hpp>
>
> and add the following:
>
> #include <boost/serialization/export.hpp>
>
> test_independent.cc - change the top to look like the following:
>
> // INDEPENDENT TEST
>
> #include <boost/archive/xml_oarchive.hpp>
> #include <boost/archive/xml_iarchive.hpp>
>
> #include <MObject.h>
> #include <MObject_serialization.hh>
> #include <MemberVariableSpecification.h>
> #include <MemberVariableSpecification_serialization.hh>
> #include <fstream>

> This won't work. Since you now have the archive headers in only one
> place, there is no multiple definition problem. However serialization
> code is *not* instantiated for MObject or
> MemberVariableSpecification. This is normal since when compiling the
> .cc that contains this code, there is no archive header included and
> so the export.hpp has no effect. The code is here but is not
> instantiated.
>
> So in order to follow this advice:
>
> a) include all boost/serialization/.. headers in each class file.
> b) include NO boost/archive/... headers in any class file
> c) make a "template instantiation module" which instanticiates code
> for all archives I plan to use.  This can be either in the app or in
> a library.
> d) lihk and execute without problem.
>
> I have to explicitly instantiate the serialization code for the
> needed archive in the .cc files. That is, do:
>
> template void save<boost::archive::xml_oarchive>(
>     boost::archive::text_oarchive & ar, const MObject & m,
>     const unsigned int file_version
> );
>
> And same for load. This is what you do in demo_pimpl.cpp. Is that
> what you usually do and recommend to do?

In a smaller program - example the tests.  One can include the serialization
headers which might contain BOOST_EXPORT after any archives used.
This will guarentee that needed code is instantiated.

In larger programs, I use the method illustrated in demo_pimpl to
explicitly instantiate code for archives that are used.

On a larger project, I would make an instantiation module which
will instantiate code for all the archives I might ever use.  Alternatively,
I might use the polymorphic archive.  In this last case I do the following
to avoid "code bloat".

a) I build a separate library of class modules.
b) I instantiate all code that might be used
c) I use compile time switches to make linking occur at the function
level rather than the module level.
d) I link my "real project" against this class library.

> This has the advantage to avoid big compilations (these code
> instantiations will only occur when the class to serialize has indeed
> changed).

Indeed - and this is a big advantage for xml_archives which take quite
a bit of time to compile.

> However this has the inconvenience of manually instantiating every
> template (and if you want to change your archive format , eg from
> text to xml for example, everything needs to be changed).

This is true.  But this is a minor in my opinion. The root cause of this
situation is that linkers complain about multiply defined functions
- even when those functions are identical.  Maybe someday we'll
have smarter linkers - but until then this works quite well.  It does
require that one spend a little time explicitly considering code
instantiation.  But this isn't necessarily bad.  Letting the compile/linker
do all the work will lead to code bloat and compilation time problems.

Robert Ramey




_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization : unregistered_cast & otherquestions

RIVASSEAU Jean Noel
In reply to this post by RIVASSEAU Jean Noel

This is true.  But this is a minor in my opinion. The root cause of this
situation is that linkers complain about multiply defined functions
- even when those functions are identical.  Maybe someday we'll
have smarter linkers - but until then this works quite well.  It does
require that one spend a little time explicitly considering code
instantiation.  But this isn't necessarily bad.  Letting the compile/linker
do all the work will lead to code bloat and compilation time problems.

I definitely agree with you - smarter linkers would be a big gift. Right now dealing with templates is generally a pain because of linker issues, and requires that one takes a lot of precaution to insure everything compiles/links smoothly.

I had the same problems when using the BGL last year, and also had to resort to manual template instantiation in the end.

Jean-Noël


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users