Serialization of optional<std::string>

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Serialization of optional<std::string>

Boost - Users mailing list
As part of upgrading a product to a later version of boost the compilation
has tripped up on serialization of a boost::optional<std::string>.

The /optional.hpp/ header now has the comment:


...and this fires for std::string.

What, if any, is the recommended way forward for this?

An example code snippet to illustrate:


Previous version of boost: 1.50; new: 1.6x
Compiling with g++ 4.6.4 (although same problem with 4.8.5): -W -Wall
-std=c++0x...
(The real code is actually using the portable_binary_archive, but let's
avoid muddying the waters at this stage :)





--
Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list
OK, so it looks like posting from nabble loses the raw-tag stuff...

The optional.hpp comment:
    // It is an inherent limitation to the serialization of optional.hpp
    // that the underlying type must be either a pointer or must have a
    // default constructor.  It's possible that this could change sometime
    // in the future, but for now, one will have to work around it.  This
can
    // be done by serialization the optional<T> as optional<T *>
    BOOST_STATIC_ASSERT(
        boost::serialization::detail::is_default_constructible<T>::value
        || boost::is_pointer<T>::value
    );

The code snippet:
#include <boost/serialization/string.hpp>
#include <boost/serialization/optional.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <fstream>

int main(int argc, char *argv[])
{
    (void)argc; (void)argv;

    typedef boost::archive::text_oarchive oarchive_type;
    typedef boost::archive::text_iarchive iarchive_type;
    std::ios_base::openmode flags = std::ios_base::openmode();

    {
        std::ofstream ofs("/tmp/tmp.bin", std::ios::out | flags);
        boost::optional<std::string> s = std::string("hello");
        oarchive_type oa(ofs);
        oa << s;
    }
    {
        std::ifstream ifs("/tmp/tmp.bin", std::ios::in | flags);
        boost::optional<std::string> s;
        iarchive_type ia(ifs);  // # ERROR #
        ia >> s;
    }
}




--
Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 2/19/18 4:35 AM, Chardrazle via Boost-users wrote:

> As part of upgrading a product to a later version of boost the compilation
> has tripped up on serialization of a boost::optional<std::string>.
>
> The /optional.hpp/ header now has the comment:
>
>
> ...and this fires for std::string.
>
> What, if any, is the recommended way forward for this?
>

Hmmm- is std::string not default constructible?
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list
Indeed, and my initial workaround was to force that through:

It may be related to: https://svn.boost.org/trac10/ticket/13108
(is_default_constructible support.)

Time to start muddying the waters.
(Note: the binary archives were taken from the latest serialization 'test'
code area.)

Some test code:

#include <boost/serialization/string.hpp>
#include <boost/serialization/optional.hpp>
#include <fstream>

#if 1
#include "boost/portable_binary_iarchive.hpp"
#include "boost/portable_binary_oarchive.hpp"
typedef portable_binary_oarchive oarchive_type;
typedef portable_binary_iarchive iarchive_type;
std::ios_base::openmode flags = std::ios::binary;
#else
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
typedef boost::archive::text_oarchive oarchive_type;
typedef boost::archive::text_iarchive iarchive_type;
std::ios_base::openmode flags = std::ios_base::openmode();
#endif

#if (BOOST_VERSION >= 106400) && (__GNUC__*100 + __GNUC_MINOR__ == 406)
namespace boost { namespace serialization { namespace detail {
    template <>
    struct is_default_constructible<std::string> : boost::true_type {};
}}}
#endif

int main(int argc, char *argv[])
{
    (void)argc; (void)argv;

    {
        std::ofstream ofs("/tmp/tmp.bin", std::ios::out | flags);
        boost::optional<std::string> s = std::string("hello");
        oarchive_type oa(ofs);
        oa << s;
    }
    {
        std::ifstream ifs("/tmp/tmp.bin", std::ios::in | flags);
        boost::optional<std::string> s;
        iarchive_type ia(ifs);  // # CRASH for binary #
        ia >> s;
    }
}


To work around the static_assertion, I tried adding the (serialization)
specialization for is_default_constructible<std::string>.

This then gets everything to compile, and the text_archive version doesn't
have any problems.
However, the binary_archive version crashes when trying to de-serialize.
( basic_binary_iprimitive.ipp(107):         s.resize(l); )

I was concerned that the specialization has activated something undesirable,
hence my question about the correct way forward...




--
Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list
On 2/20/18 1:41 AM, Chardrazle via Boost-users wrote:
> Indeed, and my initial workaround was to force that through:
>
> It may be related to: https://svn.boost.org/trac10/ticket/13108
> (is_default_constructible support.)
>
> Time to start muddying the waters.
> (Note: the binary archives were taken from the latest serialization 'test'
> code area.)

Your on the right track here.

Note that the boost regression tests

http://www.boost.org/development/tests/develop/developer/serialization.html

show serialization of optional passing with all archives - including the
binary one.

So it seems that this is an issue related to portable_binary_archive
which has the status and example not being currently being tested on a
regular basis.  In theory, the code in archive implementations should be
totally agnostic to the data type being serialized.  Alas, s*** happens.
I would recommend tracing with the debugger trough serialization of
optional<string> in portable binary archive to see how it's different
from serialization in binary or text archive.

I've been wanting to promote portable_binary_archive to "offically
supported" status, but I can't justify spending the time.

Robert Ramey
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list

>I've been wanting to promote portable_binary_archive to "offically
>supported" status, but I can't justify spending the time.

This is why I thought I'd better post here, rather than going straight to
trac :)
I'm glad I did.  I performed some further investigation by trying the sample
code on my home set-up, with variations of g++ versions.  I found no
problems whatsoever with the portable_binary_archive.

I suspect that whatever configuration has been used at the company has
really messed up something to do with memory management.




--
Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Serialization of optional<std::string>

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
>I would recommend tracing with the debugger trough serialization of
>optional<string> in portable binary archive to see how it's different
>from serialization in binary or text archive.

I've performed a bit more analysis and it looks like serialization of
optional<> with any non-trivial types is broken in 1.64.

The serialization/optional.hpp load function:

template<class Archive, class T>
void load(
    Archive & ar,
    boost::optional< T > & t,
    const unsigned int /*version*/
)[...]

In 1.63, constructed space for the type with:

        detail::stack_construct<Archive, T> aux(ar, item_version);
        ar >> boost::serialization::make_nvp("value", aux.reference());
        t.reset(aux.reference());

which ensures the constructor is called.  However, for 1.64 this changed to:

    detail::stack_allocate<T> tp; // <<<<<<<<<<<<<<
    ar >> boost::serialization::make_nvp("value", tp.reference());
    t.reset(boost::move(tp.reference()));
    ar.reset_object_address(
        t.get_ptr(),
        & tp.reference()
    );

Unfortunately, this only allocates (zero'd) space for the object, and some
objects (in this case, std::string in g++ 4.6) are not valid as such.

I see that in 1.65 this changed again, to the simpler:

    if(! t.is_initialized())
        t = T();
    ar >> boost::serialization::make_nvp("value", *t);

It seems I was just unlucky to choose 1.64.  I've moved to 1.66 and it's all
working.




--
Sent from: http://boost.2283326.n4.nabble.com/Boost-Users-f2553780.html
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users