Delayed write file handling

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Delayed write file handling

Hughes, James
Hello all,

not sure if this is the right place to ask, but there maybe something in
boost that can help.

I would like to use/write some file handling classes that implement
something like delayed writing. The reason is that on our embedded device,
we need to log information to flash memory. This information may be
generated very frequently (every second or more). However, since our devices
are in the field for long periods of time (8-10 years) this means that to
continually be writing to flash memory, we can reach the limit of the
capability of the flash re-write cycle (10k to 1mill erases before failure
depending on quality), even when using a load levelling fs like Jffs2
(running on Linux).

By delaying to write of the logging information so that the classes 'cache'
data until either told to write by the system, or a time limit is reached,
we can dramatically reduce the amount of flash erasing done. I would like to
do this transparently to the end user, by perhaps writing a iostream class
that handles it all (with some extra bits to handle urgent shutdowns and
user requested flushes).

My question is this - does Boost provide anything that does this already (or
does anyone know of anything that does), or am I looking at this the wrong
way? Should I look at boost filesystem, and provide extra bits to implement
what we need? Alternatives?

James Hughes
_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Delayed write file handling

me22
On 4/10/06, James Hughes <[hidden email]> wrote:
> My question is this - does Boost provide anything that does this already (or
> does anyone know of anything that does), or am I looking at this the wrong
> way? Should I look at boost filesystem, and provide extra bits to implement
> what we need? Alternatives?
>

I'm not certain that you need Boost for this.

streambufs are already designed to buffer information, so simply
providing a large enough buffer for the amount of information
generated on average in the time span you wish between writes ought to
limit writes enough for your application.

The main issue I foresee would be insuring that std::endl isn't used
when you don't want the flush and insuring that the implementations of
your streambufs actually use the buffer as you're expecting.

~ SWMc
_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Delayed write file handling

Hughes, James
In reply to this post by Hughes, James
> -----Original Message-----
> From: me22 [mailto:[hidden email]]
> Sent: 10 April 2006 16:24
> To: [hidden email]
> Subject: Re: [Boost-users] Delayed write file handling
>
>
> On 4/10/06, James Hughes <[hidden email]> wrote:
> > My question is this - does Boost provide anything that does
> this already (or
> > does anyone know of anything that does), or am I looking at
> this the wrong
> > way? Should I look at boost filesystem, and provide extra
> bits to implement
> > what we need? Alternatives?
> >
>
> I'm not certain that you need Boost for this.
>
> streambufs are already designed to buffer information, so simply
> providing a large enough buffer for the amount of information
> generated on average in the time span you wish between writes ought to
> limit writes enough for your application.
>
> The main issue I foresee would be insuring that std::endl isn't used
> when you don't want the flush and insuring that the implementations of
> your streambufs actually use the buffer as you're expecting.
>
> ~ SWMc

I can see what you are getting at, but I am not sure that it would give us
everything we need.

For example, if our machine power fails, we have  very limited amount of
battery backup to store everything cached to flash before we run out of
watts, so whatever scheme I implement needs to be able to be told to flush
immediately - can I do that with streambufs?

Hmm. I will look in to it though, as I like the concept.

James
_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Reply | Threaded
Open this post in threaded view
|

Re: Delayed write file handling

Andrew Holden
In reply to this post by Hughes, James
> -----Original Message-----
> From: James Hughes [mailto:[hidden email]]
> Sent: Monday, April 10, 2006 11:59 AM
> For example, if our machine power fails, we have  very limited amount
of
> battery backup to store everything cached to flash before we run out
of
> watts, so whatever scheme I implement needs to be able to be told to
flush
> immediately - can I do that with streambufs?

I think you can force a stream to flush immediatly with something
similar to the following:

outputStream << std::flush;

_______________________________________________
Boost-users mailing list
[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users