Concurrency support in iostream mapped_file_source

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Concurrency support in iostream mapped_file_source

Boost - Users mailing list
Hi,

Does boost iostream mapped_file_source support concurrent read from multiple threads? What is the best way to implement concurrent read on a performance sensitive application?

The file is a read only one and its size is in gigabytes/terabytes

Thanks,
   Lloyd


_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Concurrency support in iostream mapped_file_source

Boost - Users mailing list
On 3/07/2020 18:01, Lloyd wrote:
> Does boost iostream mapped_file_source support concurrent read from
> multiple threads? What is the best way to implement concurrent read on a
> performance sensitive application?

I haven't used it myself, but as far as I can tell from skimming the
docs, as long as you open it beforehand and don't close it while the
threads are reading, then the actual read operations are immutable and
should be safe from any number of concurrent threads, since it's just
const memory read access.

If you construct a stream object around it, however, the stream will
definitely not be thread-safe.  So each thread would need its own.  Or
just access the mapped_file_source directly.
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Concurrency support in iostream mapped_file_source

Boost - Users mailing list
In reply to this post by Boost - Users mailing list
On 03/07/2020 07:01, Lloyd via Boost-users wrote:

> Does boost iostream mapped_file_source support concurrent read from
> multiple threads? What is the best way to implement concurrent read on a
> performance sensitive application?
>
> The file is a read only one and its size is in gigabytes/terabytes

For memory mapped files in general, reads served from kernel cache will
parallelise at CPU concurrency level, whereas reads served from the
storage layer will parallelise at device concurrency level. This means
that i/o which is not served from cache will synchronise CPUs during i/o
submission and completion.

Niall
_______________________________________________
Boost-users mailing list
[hidden email]
https://lists.boost.org/mailman/listinfo.cgi/boost-users