[serialization] suggestion for the extension of the functionality

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[serialization] suggestion for the extension of the functionality

Seweryn Habdank-Wojewódzki
Hello

I would like to ask for consideration of some new functionality for
boost::serialization.

If I have a stream and I do not want to close the stream at the end of
serialization I have to make easy, but nasty trick using stringstream as a
buffer between my stream and serialized stream.

More concrete example. I'am using easy library for sockets. Connection is
represented as stream and after connection I do not want to close it. So when
I try to use serialization though the socket, serialisation is hanging - it is
waiting for the end of the stream, but there is no end of stream because it is
socket.

It will be useful just to add to library one fuction for manual sending data -
"simulation" of the end of stream. To avoid or closing socket either using
stringstreams buffers.

Below is example code for this problem - this code is using buffers, but IMHO
better will be that inside library will be functionality for such a
situation.

Best regards, Seweryn Habdank-Wojewódzki.

Ps. Full code I can send if will be needed.

void do_server()
{
        string message("Hello Socket!\nHelloWorld!");

        // create class instance
        const gps_position g(35, 59, 24.567f);

        try
        {
                TCPSocketWrapper sockserver;

                // listen on some port
                sockserver.listen(IPportnumber);

                cout << "server is ready" << endl;

                // accept connection from client
                TCPSocketWrapper sock(sockserver.accept());

                cout << "accepted connection from: "
                        << sock.address() << endl;

                // make the stream around the socket wrapper
                TCPStream stream(sock);

                bool oncemore = true;
                int command;
                while (oncemore)
                {
               // read the command
                        stream >> command;
                       
                        switch (command)
                        {
                        case 1:
                    {
                                cout << "command 1" <<endl;

                // ------------------------------------------------
     
                // buffer

                // ------------------------------------------------


                    std::ostringstream s;
                   
                {
                        boost::archive::text_oarchive oa(s);

                        // write class instance to archive
                        oa << g;
                    }
//------------------------------------------------------------------

// buffer

//------------------------------------------------------------------
                    stream << s.str() << endl;
                    }

                                break;
                        default:
                                cout << "END command" << endl;
                                oncemore = false;
                                break;
                        }
                }
        }
        catch (const SocketRunTimeException &e)
        {
                cout << "socket exception: "
                        << e.what() << endl;
        }
}

void do_client()
{
        string message;

        // ... some time later restore the class instance to its orginal state
        gps_position newg;

        try
        {
                TCPClientStream stream(IPserveraddress, IPportnumber);

                bool oncemore = true;
                int command;
                while (oncemore)
                {
                        cout << "1. request for a serialized object" << endl;
                        cout << "other - end" << endl;

                        cin >> command;
                        stream << command << endl;

                        switch (command)
                        {
                        case 1:
                    {
                // ------------------------------------------------
     
                // buffer

                // ------------------------------------------------

                         std::string line;
                         std::getline(stream, line);
                         istringstream s(line);

                // ------------------------------------------------
     
                // buffer will be filled

                // ------------------------------------------------

                         {
                             boost::archive::text_iarchive ia(s);
                           
                             ia >> newg;
                         }

                         cout << "received: " << newg << endl;
                    }
                                break;
                        default:
                                oncemore = false;
                                break;
                        }
                }
        }
        catch (const SocketRunTimeException &e)
        {
                cout << "socket exception: " << e.what() << endl;
        }
}

--

|\/\/|   Seweryn Habdank-Wojewódzki
`\/\/  


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of thefunctionality

Robert Ramey
Seweryn Habdank-Wojewódzki wrote:
> Hello
>
> I would like to ask for consideration of some new functionality for
> boost::serialization.
>
> If I have a stream and I do not want to close the stream at the end of
> serialization

I can't see why this is a problem. Does the following not work?

{
    o?stream os("stream_name");
    os << ... ; // your own stuff here
    {
        text_oarchive oa(os);
        oa << ....
        os.flush(); // actually I think this is invoked by default when
archive is destroyed
    } // archive destructor called here - stream stays open
    // stream is still open here.
    os << ... ; // more of your own stuff
    os.close();
}

Robert Ramey


I have to make easy, but nasty trick using stringstream

> as a buffer between my stream and serialized stream.
>
> More concrete example. I'am using easy library for sockets.
> Connection is represented as stream and after connection I do not
> want to close it. So when I try to use serialization though the
> socket, serialisation is hanging - it is waiting for the end of the
> stream, but there is no end of stream because it is socket.
>
> It will be useful just to add to library one fuction for manual
> sending data - "simulation" of the end of stream. To avoid or closing
> socket either using stringstreams buffers.
>
> Below is example code for this problem - this code is using buffers,
> but IMHO better will be that inside library will be functionality for
> such a situation.
>
> Best regards, Seweryn Habdank-Wojewódzki.
>
> Ps. Full code I can send if will be needed.
>
> void do_server()
> {
>         string message("Hello Socket!\nHelloWorld!");
>
>         // create class instance
>         const gps_position g(35, 59, 24.567f);
>
>         try
>         {
>                 TCPSocketWrapper sockserver;
>
>                 // listen on some port
>                 sockserver.listen(IPportnumber);
>
>                 cout << "server is ready" << endl;
>
>                 // accept connection from client
>                 TCPSocketWrapper sock(sockserver.accept());
>
>                 cout << "accepted connection from: "
>                         << sock.address() << endl;
>
>                 // make the stream around the socket wrapper
>                 TCPStream stream(sock);
>
>                 bool oncemore = true;
>                 int command;
>                 while (oncemore)
>                 {
>                // read the command
>                         stream >> command;
>
>                         switch (command)
>                         {
>                         case 1:
>                     {
>                                 cout << "command 1" <<endl;
>
>                 // ------------------------------------------------
>
>                 // buffer
>
>                 // ------------------------------------------------
>
>
>                     std::ostringstream s;
>
>                 {
>                         boost::archive::text_oarchive oa(s);
>
>                         // write class instance to archive
>                         oa << g;
>                     }
> //------------------------------------------------------------------
>
> // buffer
>
> //------------------------------------------------------------------
>                     stream << s.str() << endl;
>                     }
>
>                                 break;
>                         default:
>                                 cout << "END command" << endl;
>                                 oncemore = false;
>                                 break;
>                         }
>                 }
>         }
>         catch (const SocketRunTimeException &e)
>         {
>                 cout << "socket exception: "
>                         << e.what() << endl;
>         }
> }
>
> void do_client()
> {
>         string message;
>
>         // ... some time later restore the class instance to its
>         orginal state gps_position newg;
>
>         try
>         {
>                 TCPClientStream stream(IPserveraddress, IPportnumber);
>
>                 bool oncemore = true;
>                 int command;
>                 while (oncemore)
>                 {
>                         cout << "1. request for a serialized object"
>                         << endl; cout << "other - end" << endl;
>
>                         cin >> command;
>                         stream << command << endl;
>
>                         switch (command)
>                         {
>                         case 1:
>                     {
>                 // ------------------------------------------------
>
>                 // buffer
>
>                 // ------------------------------------------------
>
>                          std::string line;
>                          std::getline(stream, line);
>                          istringstream s(line);
>
>                 // ------------------------------------------------
>
>                 // buffer will be filled
>
>                 // ------------------------------------------------
>
>                          {
>                              boost::archive::text_iarchive ia(s);
>
>                              ia >> newg;
>                          }
>
>                          cout << "received: " << newg << endl;
>                     }
>                                 break;
>                         default:
>                                 oncemore = false;
>                                 break;
>                         }
>                 }
>         }
>         catch (const SocketRunTimeException &e)
>         {
>                 cout << "socket exception: " << e.what() << endl;
>         }
> }
>
>
>> \/\/|   Seweryn Habdank-Wojewódzki
> `\/\/   =



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of the functionality

Paul Giaccone
In reply to this post by Seweryn Habdank-Wojewódzki
Seweryn Habdank-Wojewódzki wrote:

>Hello
>
>I would like to ask for consideration of some new functionality for
>boost::serialization.
>
>If I have a stream and I do not want to close the stream at the end of
>serialization I have to make easy, but nasty trick using stringstream as a
>buffer between my stream and serialized stream.
>  
>

I raised this issue a few months ago.  It turns out that the only way of
closing a stream at the end of serialisation is to leave the scope in
which the stream was opened.  The reason for this is, I believe, so that
a "footer" can be written to the stream indicating the end of the file.

It would be handy, as Seweryn suggests, if there could be a close()
function available to users that did this, so that users could close the
stream when they needed to.  One application of this is allowing a
stream to be used for writing and then reading within the same scope.  
At the moment, a new scope is required to do this.

Paul

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of the functionality

Robert Ramey
Paul Giaccone wrote:
> Seweryn Habdank-Wojewódzki wrote:
>

> I raised this issue a few months ago.  It turns out that the only way
> of closing a stream at the end of serialisation is to leave the scope
> in which the stream was opened.  The reason for this is, I believe,
> so that a "footer" can be written to the stream indicating the end of
> the file.

This is a requirement of the xml_archive - I suppose a "close" function
could be added to the xml_archive so that one woudn't have to depend
on the destructor being called - but then one would want to add
it to all the archives where it would be a no-op.

It seems to me that it would just not be worth it to avoid the
{
}
alternative.  In fact, I rather prefer the {} alternative - it makes
more sense to me.

> It would be handy, as Seweryn suggests, if there could be a close()
> function available to users that did this, so that users could close
> the stream *** you mean "close the archive" - leaving the stream open?***
>when they needed to.

One application of this is allowing a
> stream to be used for writing and then reading within the same scope.
> At the moment, a new scope is required to do this.


Robert Ramey




_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of thefunctionality

Seweryn Habdank-Wojewódzki
In reply to this post by Robert Ramey
Hello

# pią stycznia 27 2006 18:31, @ Robert Ramey:
> Seweryn Habdank-Wojewódzki wrote:
> > I would like to ask for consideration of some new functionality for
> > boost::serialization.
> >
> > If I have a stream and I do not want to close the stream at the end of
> > serialization
>
> I can't see why this is a problem. Does the following not work?

The following works fine. This is just a small suggestion. IMHO it is very
nice if I can serialize object (as binary, text or xml) through the socket,
isn't it?

Using additional buffers, just for creating "end of stream" sequence could
make code buggy. Just one function as Pauld said eg. close() or send() will
be nice.

Best regards.

--

|\/\/|   Seweryn Habdank-Wojewódzki
`\/\/

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension ofthefunctionality

Robert Ramey
Seweryn Habdank-Wojewódzki wrote:

> The following works fine. This is just a small suggestion. IMHO it is
> very nice if I can serialize object (as binary, text or xml) through
> the socket, isn't it?

This is indeed an important application.  But the serialization library
is not the place to implement it !!!  The serialization library
depends upon the underlying streambuf (member of ?stream)
to implement any actual i/o - it is here that the connection
to sockets or special buffering should be made.  In short

?socket_ostream os("name of socket, pipe or whatever")"

{
    xml_archive oa(os);
    for(;;){
        oa << ....;
        os.flush(); // send object to socket
    }
}


> Using additional buffers, just for creating "end of stream" sequence
> could make code buggy. Just one function as Pauld said eg. close() or
> send() will be nice.

Look in to makeing a custom stream buffer for your purposes.

Investigate the iostreams library for this purpose.

Robert Ramey




_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension ofthefunctionality

Seweryn Habdank-Wojewódzki
Hi

Robert Ramey wrote:

>> The following works fine. This is just a small suggestion. IMHO it is
>> very nice if I can serialize object (as binary, text or xml) through
>> the socket, isn't it?
>
> This is indeed an important application.  But the serialization library
> is not the place to implement it !!!  The serialization library
> depends upon the underlying streambuf (member of ?stream)
> to implement any actual i/o - it is here that the connection
> to sockets or special buffering should be made.  In short
>
> ?socket_ostream os("name of socket, pipe or whatever")"
>
> {
>     xml_archive oa(os);
>     for(;;){
>         oa << ....;
>         os.flush(); // send object to socket
>     }
> }

Ok, I understand it. It is good solution, thanks.

> Look in to makeing a custom stream buffer for your purposes.
> Investigate the iostreams library for this purpose.

Ok.

Best regards.

--

|\/\/|   Seweryn Habdank-Wojewódzki
`\/\/  


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of the functionality

Paul Giaccone
In reply to this post by Robert Ramey
Robert Ramey wrote:

>Paul Giaccone wrote:
>  
>
>>Seweryn Habdank-Wojewódzki wrote:
>>
>>    
>>
>>I raised this issue a few months ago.  It turns out that the only way
>>of closing a stream at the end of serialisation is to leave the scope
>>in which the stream was opened.  The reason for this is, I believe,
>>so that a "footer" can be written to the stream indicating the end of
>>the file.
>>    
>>
>
>This is a requirement of the xml_archive - I suppose a "close" function
>could be added to the xml_archive so that one woudn't have to depend
>on the destructor being called - but then one would want to add
>it to all the archives where it would be a no-op.
>
>It seems to me that it would just not be worth it to avoid the
>{
>}
>alternative.  In fact, I rather prefer the {} alternative - it makes
>more sense to me.
>
>  
>
>>It would be handy, as Seweryn suggests, if there could be a close()
>>function available to users that did this, so that users could close
>>the stream *** you mean "close the archive" - leaving the stream open?***
>>when they needed to.
>>    
>>
>
>One application of this is allowing a
>  
>
>>stream to be used for writing and then reading within the same scope.
>>At the moment, a new scope is required to do this.
>>    
>>
>
>
>Robert Ramey
>  
>
OK, if it is more hassle than it is worth, that is fine, as I saw you
posted a way of writing and reading in the same scope a few days ago.

Is the necessity of closing the scope to complete the write in the
documentation?

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension of the functionality

Robert Ramey
Paul Giaccone wrote:
> OK, if it is more hassle than it is worth, that is fine,

well it IS more hassle than its worth - but that's only
one reason I would be reluctant to add it.  I'm
wary about accidently tying serialization to streams.
That's why I'm inclined to drag my feet regarding adding
"stream like" api names to serialiazation.  Of course
I'm known to drag my feet regarding adding anything
to the serialization library that doesn't absolutely have
to be there.  This is to keep things actually manageable.

Up until now - all the included achives have depended
on a stream whose reference is passed during
archive construction.  In the next version - this
native binary archives will depend only on a streambuf.
This change is being made to enance performance and
will break little if any usercode.

> Is the necessity of closing the scope to complete the write in the
> documentation?

If you don't see it there - its not there.  I look for a good place to add a
note.

Robert Ramey



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extension ofthefunctionality

Seweryn Habdank-Wojewódzki
In reply to this post by Seweryn Habdank-Wojewódzki
Hello

To sum up our work with boost::serialization. Maybe useful would be to add
comment to the serialization documentation that std::endl and std::eof could
finalize serialization procedure.

IHMO it is not clear there (in documentation).

Best regards.

--

|\/\/|   Seweryn Habdank-Wojewódzki
`\/\/  


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extensionofthefunctionality

Robert Ramey
Even better - I've tweaked the destructor on basic_oarchive_primitive to
append an std::endl to the stream.  So this problem will no longer appear.
Note that problem only came up with text archives.  Other oarchives never
read beyonnd the last data serialized.  So on 1.34 the problem is solved
once and for all.  And all archives classes will be consistent in their
behavior in this regard.

Robert Ramey

Seweryn Habdank-Wojewódzki wrote:

> Hello
>
> To sum up our work with boost::serialization. Maybe useful would be
> to add comment to the serialization documentation that std::endl and
> std::eof could finalize serialization procedure.
>
> IHMO it is not clear there (in documentation).
>
> Best regards.
>
>
>> \/\/|   Seweryn Habdank-Wojewódzki
> `\/\/   =



_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: [serialization] suggestion for the extensionofthefunctionality

Seweryn Habdank-Wojewódzki
Hello

At present after clarifing all that thinks with sockets and serialisation it is
possible to easily do something like RMI. In serialized object I could send to
server parameters and name of function. Then on server side I could take it
and do the function (it could be a functor) and than I can send results to the
client. Moreover if it is functor I could remember last operation and If I
heve another new call but with the same parametres I could just send results
without executing function.

Best regards,

--

|\/\/|   Seweryn Habdank-Wojewódzki
`\/\/  


_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users