read-only filesystem support for NAND flash devices
Hi folks,
there had recently been some talks around $Subject... and I'm personally interested in implementing that, both professionally and as an open-source guy :)
The problem, as most of you know well, is that currently existing compressed read-only filesystems don't support bad block handling. OTOH, it's quite an attractive idea to use those for root filesystem of an embedded device.
As a filesystem is read-only, wear-out can't occur during the normal use, only on root filesystem upgrade. Therefore writing it using a tool like nandwrite that bypasses bad blocks seems to be enough, and the only thing needed is either an in-MTD layer that makes a partition look as if there's no babd blocks, or similar means within a filesystem.
The only known implementation goes the former way (though I recall some attempts for squashfs to implement the latter one). It's at http://lists.infradead.org/pipermail/linux-mtd/2004-May/009695.html.
The main drawbacks I see in the implementation are:
- adds more functions/vars to struct mtd_info which is apparently a bad idea (it's already overcomplicated)
- modifies a lot of generic files (i. e. CFI command set implementations, etc.)
- possibly exposed to FTL/NFTL patents
What I'd like to suggest is to rework this patch to get rid of these (and probably some more) drawbacks. Ideas:
1. Implement it on top of NAND layer.
- use NAND bad block table, but make its construction lazy in order
to decrease boot time (might be a useful idea anyway)
- use tree-like structure to calculate the offset between the block
number supplied and the actual block to read (alternatively, just add the how-many-to-add info to BBT entries)
2. Implement it on top of MTD layer.
- might be reasonable to use mtdblock_ro for that
- might be reasonable to make bad block table global for all MTD
devices, not for NAND only and, once again, add an option to construct it lazily.
Comments, corrections and suggestions are welcome.
Thx,
Vitaly
Quoted 5 lines · Vitaly Wool
As a filesystem is read-only, wear-out can't occur during the normal use, only on root filesystem upgrade. Therefore writing it using a tool like nandwrite that bypasses bad blocks seems to be enough, and the only thing needed is either an in-MTD layer that makes a partition look as if there's no babd blocks, or similar means within a filesystem.
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
josh
Quoted 9 lines · Josh Boyer
Quoted 5 lines
As a filesystem is read-only, wear-out can't occur during the normal use, only on root filesystem upgrade. Therefore writing it using a tool like nandwrite that bypasses bad blocks seems to be enough, and the only thing needed is either an in-MTD layer that makes a partition look as if there's no babd blocks, or similar means within a filesystem.
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
Yep, I know about the issue. The recommended way to go here AFAIK is to mark the block as bad and copy its contents to a free one.
However, this will make mapping a lot more complicated so I'd like to go another way, i. e. erasing and rewriting this very block, and if it wears out, cache the data read from it and schedule moving the data forward within the partition in a background. Does that make sense?
Vitaly
Quoted 6 lines · Vitaly Wool
Quoted 4 lines · Josh Boyer
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
Quoted 6 lines
Yep, I know about the issue. The recommended way to go here AFAIK is to mark the block as bad and copy its contents to a free one. However, this will make mapping a lot more complicated so I'd like to go another way, i. e. erasing and rewriting this very block, and if it wears out, cache the data read from it and schedule moving the data forward within the partition in a background. Does that make sense?
Or you could just ignore the issue. The way I read it, your RFC is basically "Don't do the real thing, just a quick hack to make it work at all." Which is perfectly fine to me. It is not perfect, but that is well documented and if people actually use it and care a lot about some particular deficiency, you can solve it then.
Jörn
Signature · 2 lines
Anything that can go wrong, will. -- Finagle's Law
Quoted 14 lines · Jörn Engel
Quoted 6 lines · Vitaly Wool
Quoted 4 lines · Josh Boyer
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
Quoted 6 lines
Yep, I know about the issue. The recommended way to go here AFAIK is to mark the block as bad and copy its contents to a free one. However, this will make mapping a lot more complicated so I'd like to go another way, i. e. erasing and rewriting this very block, and if it wears out, cache the data read from it and schedule moving the data forward within the partition in a background. Does that make sense?
Quoted 5 lines
Or you could just ignore the issue. The way I read it, your RFC is basically "Don't do the real thing, just a quick hack to make it work at all." Which is perfectly fine to me. It is not perfect, but that is well documented and if people actually use it and care a lot about some particular deficiency, you can solve it then.
That's a perfectly acceptable approach to me. I was just pointing something out in case someone already cared enough about it to solve it now.
josh
Quoted 10 lines
Quoted 4 lines
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
Yep, I know about the issue. The recommended way to go here AFAIK is to mark the block as bad and copy its contents to a free one. However, this will make mapping a lot more complicated so I'd like to go another way, i. e. erasing and rewriting this very block, and if it wears out, cache the data read from it and schedule moving the data forward within the partition in a background. Does that make sense?
Then you'll run out of blocks. There are two seperate issues with NAND:
1) Bad blocks, NAND chips are shipped with a certain number of bad blocks, during the chips life time, some of the other blocks will go bad.
2) Reading from a sector degrades the data, but just a little tiny bit. After several thousand reads, you may get a bit flip in your data (which can be corrected by ECC). At this point, you do not have a bad block, you just need to rewrite the data. You can't rewrite it to the current block, because then a power loss would mean disaster. Also, you can't mark the currently block as bad, because if you did, you would decrease the usable life of your system (tons of good blocks marked as bad)
So really, for issue 2, you need some sort of free block list, so you can reuse them. Which also means, you can expect the order of the blocks on your filesystem to start to look more and more random as time goes on. In other words, you also need some algorithm to find which block is where.
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
Quoted 2 lines · Russ Dill
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
A full read-write FTL needs to handle 512-byte writes if it's going to pretend to be a normal block device. A simple FTL for read-only stuff like cramfs/squashfs doesn't need that.
We should probably do this kind of thing as an FTL -- we already have the 'mtd_blkdevs' infrastructure to help you by doing most of the interfacing to MTD and block layers for you. And then you can use any read-only file system on it.
Signature · 1 line
dwmw2
Quoted 11 lines
Quoted 2 lines
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
A full read-write FTL needs to handle 512-byte writes if it's going to pretend to be a normal block device. A simple FTL for read-only stuff like cramfs/squashfs doesn't need that.
We should probably do this kind of thing as an FTL -- we already have the '
mtd_blkdevs' infrastructure to help you by doing most of the interfacing to MTD and block layers for you. And then you can use any read-only file system on it.
True, I also realized another possible shortcut. If a blocks gets an ECC correction on a read, rewrite it to a free block, erase the block, rewrite it from the data written to the free block, and then erase the free block. You'd need to put some info in the OOB so you could know what the state is on power recovery.
Quoted 12 lines · Russ Dill
Quoted 11 lines
Quoted 2 lines
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
A full read-write FTL needs to handle 512-byte writes if it's going to pretend to be a normal block device. A simple FTL for read-only stuff like cramfs/squashfs doesn't need that.
We should probably do this kind of thing as an FTL -- we already have the '
mtd_blkdevs' infrastructure to help you by doing most of the interfacing to MTD and block layers for you. And then you can use any read-only file system on it.
Quoted 5 lines
True, I also realized another possible shortcut. If a blocks gets an ECC correction on a read, rewrite it to a free block, erase the block, rewrite it from the data written to the free block, and then erase the free block. You'd need to put some info in the OOB so you could know what the state is on power recovery.
Um, or simply leave the copied block alone, erase the original block and use it as the spare now. You only need a single block for scrubbing.
josh
Quoted 19 lines · Josh Boyer
Quoted 12 lines · Russ Dill
Quoted 11 lines
Quoted 2 lines
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
A full read-write FTL needs to handle 512-byte writes if it's going to pretend to be a normal block device. A simple FTL for read-only stuff like cramfs/squashfs doesn't need that.
We should probably do this kind of thing as an FTL -- we already have the '
mtd_blkdevs' infrastructure to help you by doing most of the interfacing to MTD and block layers for you. And then you can use any read-only file system on it.Quoted 5 lines
True, I also realized another possible shortcut. If a blocks gets an ECC correction on a read, rewrite it to a free block, erase the block, rewrite it from the data written to the free block, and then erase the free block. You'd need to put some info in the OOB so you could know what the state is on power recovery.
Quoted 3 lines
Um, or simply leave the copied block alone, erase the original block and use it as the spare now. You only need a single block for scrubbing.
At some point, the original block needs to be reused though, I suppose it could go onto your freelist at scantime. I'm just looking for ways to decrease mount time. If any block could be anywhere, you will have long mount times.
If, on the other hand, you have a free block list past the end of the image, you only need to scan this at mount, or even lazy, if a block isn't available when you read it, then at that point, scan the free list until it is found.
Quoted 6 lines · Russ Dill
True, I also realized another possible shortcut. If a blocks gets an ECC correction on a read, rewrite it to a free block, erase the block, rewrite it from the data written to the free block, and then erase the free block. You'd need to put some info in the OOB so you could know what the state is on power recovery.
Oh thanks, that's the idea I'm going to use. It's a pretty reasonable assumption that 1 byte out of OOB is available as there's not going to be anything in OOB but ECC code in case of RO filesystem.
Vitaly
Quoted 2 lines · Russ Dill
Free block list? Bad block list? Tracking of block numbers? Sounds like a full on FTL to me.
I'd rather go modifying mtdblock_ro. I don't want to implement FTL because 1) as David pointed out, I don't need to handle writes; 2) I'm afraid we may end up dealing with patent issues here again.
Vitaly
Quoted 3 lines · Vitaly Wool
I'd rather go modifying
mtdblock_ro. I don't want to implement FTL because 1) as David pointed out, I don't need to handle writes; 2) I'm afraid we may end up dealing with patent issues here again.
mtdblock_ro _is_ an FTL. Just a very very primitive one :)
By all means extend it until it does what we need -- but don't call it mtdblock_ro.
Signature · 1 line
dwmw2
Quoted 4 lines · Josh Boyer
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
If I mounted a JFFS2 filesystem on NAND and _never_ write anything (or mounted read-only), does current MTD/NAND/JFFS2 layer handle those bit flips? I've thought JFFS2 only reassign blocks on writing, so JFFS2 also suffer those "bit flips on read" problem, is it right?
---
Atsushi Nemoto
Quoted 10 lines · Atsushi Nemoto
Quoted 4 lines · Josh Boyer
Not quite the case. You need bad block skipping, yes. But NAND can get bit flips in good blocks still. How do you deal with that? You can't leave the block in that state forever because it will continue to get bit flips and then your data will be unusable.
If I mounted a JFFS2 filesystem on NAND and _never_ write anything (or mounted read-only), does current MTD/NAND/JFFS2 layer handle those bit flips? I've thought JFFS2 only reassign blocks on writing, so JFFS2 also suffer those "bit flips on read" problem, is it right?
If you truly never write anything, then yes it probably suffers from
the same problem. In a typical JFFS2 setup where writes do occur, the
theory is that garbage collection will eventually reclaim that block. Whether that actually happens or not is largely determined on a per use case basis.
josh
Quoted 7 lines · Josh Boyer
Quoted 5 lines · Atsushi Nemoto
If I mounted a JFFS2 filesystem on NAND and _never_ write anything (or mounted read-only), does current MTD/NAND/JFFS2 layer handle those bit flips? I've thought JFFS2 only reassign blocks on writing, so JFFS2 also suffer those "bit flips on read" problem, is it right?
Quoted 5 lines
If you truly never write anything, then yes it probably suffers from the same problem. In a typical JFFS2 setup where writes do occur, the theory is that garbage collection will eventually reclaim that block. Whether that actually happens or not is largely determined on a per use case basis.
The fact remains that jffs2 only scrubs bad blocks by accident on a random basis. And it couldn't do more anyway. Nothing in mtd indicates a correctable error that would require scrubbing.
Jörn
Signature · 2 lines
This above all: to thine own self be true. -- Shakespeare
Quoted 10 lines · Josh Boyer
Quoted 4 lines
If I mounted a JFFS2 filesystem on NAND and _never_ write anything (or mounted read-only), does current MTD/NAND/JFFS2 layer handle those bit flips? I've thought JFFS2 only reassign blocks on writing, so JFFS2 also suffer those "bit flips on read" problem, is it right?
If you truly never write anything, then yes it probably suffers from the same problem. In a typical JFFS2 setup where writes do occur, the theory is that garbage collection will eventually reclaim that block. Whether that actually happens or not is largely determined on a per use case basis.
Thank you. Then something like periodical garbage collection will save the bit flips, but of course unnecessary writes should shorten the life of the device ... so ambivalent.
---
Atsushi Nemoto