NAND write buffering
Hi all,
I just got the cited mail from Ferenc. I think it is useful for other people too, that's why it is posted to the list.
Is there anything wrong with our assumptions?
Quoted 7 lines
Quoted 2 lines
JFFS2 is claimed to be powerfail-safe. That's true as far it will _always_ mount, nodes are CRC-protected and scanned at Mount/GC.
But what about write buffering on NAND? Doesn't this break lots of the powerfailsafe-efforts? All the data in the writebuffer will be lost. Assuming we are updating a logfile with small data portions. The portions accumulate in wbuf, waiting to reachc->wbuf_pagesizeso that the buffer is written to flash. Powerfail: all these small updates can be lost. Hm...I think you are right. But anyway, if you call "sync" all data will be flushed. Unfortunatelly NAND can be written only by page, so the end of the wbuf will be filled a padding node. It is a little flash wasting, but the data will be written out immediatelly.
Bye,
Ferenc
Thanks,
Bernhard
Quoted 6 lines · Bernhard Priewasser
Hi all,
I just got the cited mail from Ferenc. I think it is useful for other people too, that's why it is posted to the list.
Is there anything wrong with our assumptions?
The assumptions seem to be fine. The usage case you described is probably the worst case for JFFS2 though. Lots of small writes generates lots of data nodes, which in turn increases mount time and wastes space. EBS helps with the mount time issue, but the flash space consumption is larger than it probably needs to be. But as you correctly pointed out, there is nothing you can do about that if you _must_ have it be on flash after every write.
josh
Quoted 7 lines
The assumptions seem to be fine. The usage case you described is probably the worst case for JFFS2 though. Lots of small writes generates lots of data nodes, which in turn increases mount time and wastes space. EBS helps with the mount time issue, but the flash space consumption is larger than it probably needs to be. But as you correctly pointed out, there is nothing you can do about that if you _must_ have it be on flash after every write.
Hehe... I'd say: rather use the flash unefficiently (sizeof(header) >> sizueof(data)) than loose data!
Bernhard
Quoted 3 lines · Bernhard Priewasser
Hehe... I'd say: rather use the flash unefficiently (sizeof(header) >> sizueof(data)) than loose data!
But remember to do your own math. Once your flash usage is inefficient enough, you'll simply burn through the vendor-guaranteed erase cycles and permanently lose data. ;)
Jörn
Signature · 3 lines
If you're willing to restrict the flexibility of your approach, you can almost always do something better. -- John Carmack
Hi,
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
JFFS2 is claimed to be powerfail-safe. That's true as far it will _always_ mount, nodes are CRC-protected and scanned at Mount/GC.
But what about write buffering on NAND? Doesn't this break lots of the powerfailsafe-efforts? All the data in the writebuffer will be lost. Assuming we are updating a logfile with small data portions. The portions accumulate in wbuf, waiting to reachc->wbuf_pagesizeso that the buffer is written to flash. Powerfail: all these small updates can be lost. Hm...I think you are right. But anyway, if you call "sync" all data will be flushed. Unfortunatelly NAND can be written only by page, so the end of the wbuf will be filled a padding node. It is a little flash wasting, but the data will be written out immediatelly.
When is NAND wbuf flushed?sync()fsync()fflush()?fclose()?
Thanks,
Bernhard
Quoted 3 lines · Bernhard Priewasser
When is NAND wbuf flushed?
sync()
Yes.
Quoted 1 line
fsync()
Yes.
Quoted 1 line
fflush()?
Not necessarily. From the fflush man page:
"Note that fflush() only flushes the user space buffers provided by the C library. To ensure that the data is physically stored on disk the kernel buffers must be flushed too, e.g. with sync(2) or fsync(2)."
Quoted 1 line
fclose()?
Not necessarily. See the man page in the NOTES section. It says the same as fflush.
josh