[Boost.Serialization] (De-)serialization with class versioning

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
Hi,
i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:

BOOST_CLASS_VERSION(GAMEINFO, 1)

template <typename Archive>
void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version)
{
ar & gameSettings.NPODS;
// ...
if (version > 0)
ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS;
}

Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive).
I would have expected to get the version of the data in the saved archive in this case.

Say i saved with an old version=0.
Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1).
But when reading the old archive, i expect version==0 in the function so i can react on it.
Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files!
What am i missing?

regards,
Lars R.


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list


On Sunday, March 11, 2018, Lars Ruoff via Boost-users <[hidden email]> wrote:
Hi,
i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:

BOOST_CLASS_VERSION(GAMEINFO, 1)

template <typename Archive>
void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version)
{
ar & gameSettings.NPODS;
// ...
if (version > 0)
ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS;
}

Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive).
I would have expected to get the version of the data in the saved archive in this case.

Say i saved with an old version=0.
Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1).
But when reading the old archive, i expect version==0 in the function so i can react on it.
Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files!
What am i missing?

regards,
Lars R.



You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely.

Maybe someone will point out what we're both doing wrong.

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list


On Sunday, March 11, 2018, james <[hidden email]> wrote:


On Sunday, March 11, 2018, Lars Ruoff via Boost-users <[hidden email]> wrote:
Hi,
i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:

BOOST_CLASS_VERSION(GAMEINFO, 1)

template <typename Archive>
void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version)
{
ar & gameSettings.NPODS;
// ...
if (version > 0)
ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS;
}

Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive).
I would have expected to get the version of the data in the saved archive in this case.

Say i saved with an old version=0.
Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1).
But when reading the old archive, i expect version==0 in the function so i can react on it.
Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files!
What am i missing?

regards,
Lars R.



You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely.

Maybe someone will point out what we're both doing wrong.

I meant to type 'deserialisation exceptions' instead of 'deserialisation changes' above 

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
Ok, i realized my mistake:
I explicitly called the top level serialization functions giving it the hard-coded version numbers, like:

  replayArchive = new boost::archive::binary_iarchive(replayFile);
  CPodballHeader header;
serialize<boost::archive::binary_iarchive>(*replayArchive, header, boost::serialization::version<CPodballHeader>::value);
serialize<boost::archive::binary_iarchive>(*replayArchive, matchSettings, boost::serialization::version<CMatchSettings>::value);
serialize<boost::archive::binary_iarchive>(*replayArchive, gameSettings, boost::serialization::version<GAMEINFO>::value);

i have changed the latter calls to
*replayArchive & header;
*replayArchive & matchSettings;
*replayArchive & gameSettings;

and now i get the correct old version when reading from old files.
This problem solved.


However i got some weird exceptions now.
What i realized is that the above change seemed to have changed the binary layout of my files.
Older version had this dump from head:
16 00 00 00 73 65 72 69 61 6c 69 7a 61 74 69 6f  ....serializatio
6e 3a 3a 61 72 63 68 69 76 65 10 00 04 04 04 08  n::archive......
01 00 00 00 0e 00 00 00 70 6f 64 62 61 6c 6c 2d  ........podball-
72 65 70 6c 61 79 00 00 04 02 02 00 00 00 40 1f  replay........@.

Newer version has this head:
16 00 00 00 73 65 72 69 61 6c 69 7a 61 74 69 6f  ....serializatio
6e 3a 3a 61 72 63 68 69 76 65 10 00 04 04 04 08  n::archive......
01 00 00 00 00 00 00 00 00 0e 00 00 00 70 6f 64  .............pod
62 61 6c 6c 2d 72 65 70 6c 61 79 00 00 04 02 00  ball-replay.....

"podball-replay" is the content of a std::string and the very first item to be serialized from CPodballHeader.
As you can see there are 5 additional bytes before this in the newer version.
This is probably what is causing problems?
I guess the bytes are for some bookkeeping, but why has it changed  with the way i call things?

(PS: I'm using MS Visual Studio Community 2017)


On Sun, Mar 11, 2018 at 5:35 PM, james <[hidden email]> wrote:


On Sunday, March 11, 2018, james <[hidden email]> wrote:


On Sunday, March 11, 2018, Lars Ruoff via Boost-users <[hidden email]> wrote:
Hi,
i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:

BOOST_CLASS_VERSION(GAMEINFO, 1)

template <typename Archive>
void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version)
{
ar & gameSettings.NPODS;
// ...
if (version > 0)
ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS;
}

Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive).
I would have expected to get the version of the data in the saved archive in this case.

Say i saved with an old version=0.
Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1).
But when reading the old archive, i expect version==0 in the function so i can react on it.
Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files!
What am i missing?

regards,
Lars R.



You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely.

Maybe someone will point out what we're both doing wrong.

I meant to type 'deserialisation exceptions' instead of 'deserialisation changes' above 


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
> Ok, i realized my mistake:
> I explicitly called the top level serialization functions giving it the
> hard-coded version numbers, like:

You should never, ever call the serialize function from a user program.
This should be called by the serialization libary and only the
serialization library.  If you find yourself doing this, you are making
an error and misunderstanding in some way how the library is meant to be
used.

Robert Ramey

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
Thanks Robert,
got it.
Any ideas about the binary file issue?


On Sun, Mar 11, 2018 at 7:41 PM, Robert Ramey via Boost-users <[hidden email]> wrote:
On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
Ok, i realized my mistake:
I explicitly called the top level serialization functions giving it the hard-coded version numbers, like:

You should never, ever call the serialize function from a user program. This should be called by the serialization libary and only the serialization library.  If you find yourself doing this, you are making an error and misunderstanding in some way how the library is meant to be used.

Robert Ramey

_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
On 3/11/18 11:56 AM, Lars Ruoff via Boost-users wrote:
> Thanks Robert,
> got it.
> Any ideas about the binary file issue?
>

Hmmm - changing the version can be expected to change the data in the
archives.

Rerun your test from scratch - everything should function smooth as silk.

If it doesn't try using a text or even better an xml archive.  This will
make it much easier to see what is going on.

Robert Ramey



_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [Boost.Serialization] (De-)serialization with class versioning

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 3/11/18 11:41 AM, Robert Ramey via Boost-users wrote:

> On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
>> Ok, i realized my mistake:
>> I explicitly called the top level serialization functions giving it
>> the hard-coded version numbers, like:
>
> You should never, ever call the serialize function from a user program.
> This should be called by the serialization libary and only the
> serialization library.  If you find yourself doing this, you are making
> an error and misunderstanding in some way how the library is meant to be
> used.

FYI - I recommend making the intrusive serialize functions private.
Then using friend to give the serialization library - and only the
serialization library - access to them.  This makes it much harder to
accidentally misuse the library.  Of course this gives no help for
non-intrusive serialize functions.

Robert Ramey
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users