IOStream: Determine number of bytes written to array sink

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

IOStream: Determine number of bytes written to array sink

Boost - Users mailing list

Hello

 

The program below decompresses a std::string using Boost.IOStream. The target is an array sink of fixed size.

 

The users know the decompressed size in advance but for security reasons I need to check the number of bytes written to the sink.

If the array is too small, a "write area exhausted" exception is thrown, which is fine.

If the array is too big, decompression succeeds. I would like to check the number of bytes written to the sink and throw an exception.

 

How can I determine how many bytes have been written to the sink?

 

Btw: I cannot use the “counter” filter because of the overhead involved in line counting and because the return value of its characters() method is “int”. The buffers involved in my application require a 64 bit data type.

 

Private base class basic_zlib_decompressor of class basic_gzip_decompressor has method total_out() which seems to return the value in question but its return type is also only an “int”.

 

Regards, Peter.

 

 

#include <iostream>

#include <stdexcept>

#include <iomanip>

#include <boost/iostreams/filtering_streambuf.hpp>

#include <boost/iostreams/device/array.hpp>

#include <boost/iostreams/filter/gzip.hpp>

 

void decompress(const std::size_t p_bufferSize)

{

    // decompressed string is "xy"

    const std::string s("\x1f\x8b\x08\x08\x58\xce\xf6\x5b\x02\x03\x78\x79\x00\xab\xa8\x04\x00\x99\x28\xe6\x8f\x02\x00\x00\x00",25);

 

    boost::iostreams::filtering_streambuf<boost::iostreams::output> filteringStreambuf;

    filteringStreambuf.push(boost::iostreams::gzip_decompressor());

   const std::unique_ptr<char[]> buffer=std::make_unique<char[]>(p_bufferSize);

    filteringStreambuf.push(boost::iostreams::array_sink(buffer.get(),p_bufferSize));

 

    boost::iostreams::write(filteringStreambuf,s.c_str(),s.size());

 

    boost::iostreams::close(filteringStreambuf);

}

 

int main()

{

    try

    {

        decompress(3); // works, although only 2 bytes are required, should be made to fail

        //decompress(2); // works, exactly fills the buffer

        //decompress(1); // throws exception "write area exhausted" as expected

        return 0;

    }

    catch (const std::exception& e) {

        std::cout << e.what() << std::endl;

    }

    return 1;

}


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: IOStream: Determine number of bytes written to array sink

Boost - Users mailing list
AMDG

On 11/22/2018 11:40 AM, Peter Klotz via Boost-users wrote:

> The program below decompresses a std::string using Boost.IOStream. The target is an array sink of fixed size.
>
> The users know the decompressed size in advance but for security reasons I need to check the number of bytes written to the sink.
> If the array is too small, a "write area exhausted" exception is thrown, which is fine.
> If the array is too big, decompression succeeds. I would like to check the number of bytes written to the sink and throw an exception.
>
> How can I determine how many bytes have been written to the sink?
>
> Btw: I cannot use the "counter" filter because of the overhead involved in line counting and because the return value of its characters() method is "int". The buffers involved in my application require a 64 bit data type.
>

Use your own counter filter?  It's not difficult:
(copy/paste from iostreams/filter/counter.hpp and edit)

class counter  {
public:
    typedef char char_type;
    struct category
        : input_tag,
          filter_tag,
          multichar_tag,
          optimally_buffered_tag
        { };
    explicit counter()
        : chars_(0)
        { }
    std::streamsize characters() const { return chars_; }
    std::streamsize optimal_buffer_size() const { return 0; }

    template<typename Sink>
    std::streamsize write(Sink& snk, const char* s, std::streamsize n)
    {
        std::streamsize result = iostreams::write(snk, s, n);
        chars_ += result;
        return result;
    }
private:
    std::streamsize chars_;
};

In Christ,
Steven Watanabe
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: IOStream: Determine number of bytes written to array sink

Boost - Users mailing list
Hi Steven

> -----Original Message-----
> From: Boost-users [mailto:[hidden email]] On Behalf
> Of Steven Watanabe via Boost-users
...
>     std::streamsize optimal_buffer_size() const { return 0; }
>
>     template<typename Sink>
>     std::streamsize write(Sink& snk, const char* s, std::streamsize n)
>     {
>         std::streamsize result = iostreams::write(snk, s, n);
>         chars_ += result;
>         return result;
>     }

Thanks for the tip! I had hoped there was some kind of well hidden buffer pointer to do some pointer arithmetic.

One problem I stumbled into with the counter filter:

If optimal_buffer_size() returns 0, write() is called with 1 byte buffers. In my case this increased the program runtime from 8 seconds to 4 minutes.
Returning a larger value solved this issue for my use case.

Regards, Peter.
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users