mtd, mtdblock and nand ecc.
Is there some reason that the mtd and mtdblock drivers do not use ECC when the backing device is nand with ECC enabled?
I guess that kind of sums up my question.
If the answer is no, I am thinking of patching them so that they do use ECC when available.
I suppose you could run into problems if the backing device were a concatenation of ECC and non-ECC devices, but that could probably be detected either on open or at each page read/write.
Thanks in advance,
David Daney.
Quoted 2 lines · David Daney
Is there some reason that the mtd and mtdblock drivers do not use ECC when the backing device is nand with ECC enabled?
Yes.
If the oobsel structure == NULL, then the nand driver uses the structure, which is associated to this mtd device. This can be done from userspace using MEMSETOOBSEL ioctl.
NAND aware filesystem drivers provide their own oobsel structure and use the xxx_ecc functions.
Quoted 3 lines
I suppose you could run into problems if the backing device were a concatenation of ECC and non-ECC devices, but that could probably be detected either on open or at each page read/write.
Concatenation of NAND and NOR will not work anyway, as the fs must be aware of the FLASH type.
Signature · 8 lines
Thomas ________________________________________________________________________ "Free software" is a matter of liberty, not price. To understand the concept, you should think of "free" as in "free speech,'' not as in "free beer". ________________________________________________________________________ linutronix - competence in embedded & realtime linux http://www.linutronix.de mail: [email protected]
Quoted 12 lines · Thomas Gleixner
Quoted 4 lines · David Daney
Is there some reason that the mtd and mtdblock drivers do not use ECC when the backing device is nand with ECC enabled?
Yes. If the oobsel structure == NULL, then the nand driver uses the structure, which is associated to this mtd device. This can be done from userspace using MEMSETOOBSEL ioctl. NAND aware filesystem drivers provide their own oobsel structure and use the
xxx_eccfunctions.
I am using the cramfs on a NAND partition as my root file system. cramfs is not NAND aware, and I cannot be running userspace programs before mounting as it is the root file system.
I have not completely educated myself on the mtdblock driver. Since the mtdblock driver can be used by non-mtd-aware filesystems, I am proposing making mtdblock NAND aware so that it uses the xxx_ecc functions iff ECC is available. Perhaps there would be a kernel/module command line switch to help manage the behavior.
Thoughts?
David Daney
Quoted 6 lines · David Daney
Quoted 2 lines
NAND aware filesystem drivers provide their own oobsel structure and use the
xxx_eccfunctions.I am using the cramfs on a NAND partition as my root file system. cramfs is not NAND aware, and I cannot be running userspace programs before mounting as it is the root file system.
I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your root fs. Mount it r/o, so you have no hassle at all.
Quoted 7 lines
I have not completely educated myself on the mtdblock driver. Since the mtdblock driver can be used by non-mtd-aware filesystems, I am proposing making mtdblock NAND aware so that it uses the
xxx_eccfunctions iff ECC is available. Perhaps there would be a kernel/module command line switch to help manage the behavior.Thoughts?
mtdblock is a block device driver and only provides an interface. It must not be aware of anything.
Using NAND unaware filesystems on NAND is nothing we want to support. ECC is only one part of NAND support. What about bad blocks? NAND chips can have bad blocks, even when they are new. Only block 0 is guaranteed to be not bad at delivery time. How want you deal with a board, where a bad block is in the partition which is reserved for your cramfs ?
We have two reliable working NAND aware filesystems around. I don't see any reason to provide support for predictable trouble.
Signature · 8 lines
Thomas ________________________________________________________________________ "Free software" is a matter of liberty, not price. To understand the concept, you should think of "free" as in "free speech,'' not as in "free beer". ________________________________________________________________________ linutronix - competence in embedded & realtime linux http://www.linutronix.de mail: [email protected]
Let me start by saying that I am not trying to cause problems, but just to understand the best options...
Quoted 8 lines · Thomas Gleixner
Quoted 4 lines · David Daney
Quoted 4 lines
NAND aware filesystem drivers provide their own oobsel structure and use the
xxx_eccfunctions.
Quoted 9 lines
Quoted 5 lines
I am using the cramfs on a NAND partition as my root file system. cramfs is not NAND aware, and I cannot be running userspace programs before mounting as it is the root file system.
I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your root fs. Mount it r/o, so you have no hassle at all.
With my NAND drivers, booting the linux kernel and mounting a minimal root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for jffs2. Using cramfs it boots in under 0:10.
That is why I am thinking about using a non NAND aware file system for things that can be read-only.
Quoted 8 lines
Quoted 8 lines
I have not completely educated myself on the mtdblock driver. Since the mtdblock driver can be used by non-mtd-aware filesystems, I am proposing making mtdblock NAND aware so that it uses the
xxx_eccfunctions iff ECC is available. Perhaps there would be a kernel/module command line switch to help manage the behavior.
Quoted 6 lines
Quoted 3 lines
Thoughts?
mtdblock is a block device driver and only provides an interface. It must not be aware of anything.
That is not quite correct. mtdblock is well aware of the mtd backend. It does this:
ret = MTD_WRITE (mtd, pos, len, &retlen, buf);
All I am suggesting is to have it do MTD_WRITE_ECC when possible.
Quoted 8 lines
Using NAND unaware filesystems on NAND is nothing we want to support.
ECC is only one part of NAND support. What about bad blocks? NAND chips can have bad blocks, even when they are new. Only block 0 is guaranteed to be not bad at delivery time. How want you deal with a board, where a bad block is in the partition which is reserved for your cramfs ?We have two reliable working NAND aware filesystems around. I don't see any reason to provide support for predictable trouble.
You already support it. /dev/mtd and /dev/mtdblock work off-the-shelf with NAND devices and allow arbitrary programs/filesystems to overwrite bad blocks if they choose.
All I was thinking about was the ECC issue.
David Daney.
Quoted 2 lines · David Daney
Let me start by saying that I am not trying to cause problems, but just to understand the best options...
I was not saying, that you are causing problems :)
Quoted 9 lines
Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · David Daney
Quoted 2 lines
NAND aware filesystem drivers provide their own oobsel structure and use the
xxx_eccfunctions.I am using the cramfs on a NAND partition as my root file system. cramfs is not NAND aware, and I cannot be running userspace programs before mounting as it is the root file system.
Quoted 6 lines
Quoted 2 lines
I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your root fs. Mount it r/o, so you have no hassle at all.
With my NAND drivers, booting the linux kernel and mounting a minimal root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for jffs2. Using cramfs it boots in under 0:10.
He ? 1 minute ? Where is the time spent ?
Thats totaly out of the usual time. My boot time with JFFS2 is well below 30s on an ARM7. I have no YAFFS root fs handy, but it is much faster.
The solution is checking where the time is lost and fixing it.
Quoted 2 lines
That is why I am thinking about using a non NAND aware file system for things that can be read-only.
But this does not answer my question how you want to deal with bad blocks ?
Quoted 7 lines
Quoted 3 lines
mtdblock is a block device driver and only provides an interface. It must not be aware of anything.
That is not quite correct. mtdblock is well aware of the mtd backend. It does this:
ret = MTD_WRITE (mtd, pos, len, &retlen, buf);
Yes, thats a wrapper nothing else. The wrapper is not aware of anything else than the MTD API itself. It does not care of FLASH types. And it will not care of FLASH types. There are a bunch of new FLASH types in the pipeline. Each of them has seperate problems. They have to be handled in the driver, nowhere else.
Quoted 1 line
All I am suggesting is to have it do
MTD_WRITE_ECCwhen possible.
The nand driver provides a mechanism for using ecc without calling the _ecc functions already.
Provide a valid oobsel structure in your hardware driver. Place it into mtd->oobinfo. Then the nand driver uses this when reading from the chip even when it is called by the non ecc functions.
Thats the way we provide ecc to userspace too. As the user must be able to select an ecc / oob scheme for the partiton he wants to access. see utils/nandwrite.c
I'm reworking the driver at the moment. It will have a builtin default ecc scheme then.
Quoted 6 lines
Quoted 6 lines
Using NAND unaware filesystems on NAND is nothing we want to support. ECC is only one part of NAND support. What about bad blocks? NAND chips can have bad blocks, even when they are new. Only block 0 is guaranteed to be not bad at delivery time. How want you deal with a board, where a bad block is in the partition which is reserved for your cramfs ?
Quoted 6 lines
Quoted 2 lines
We have two reliable working NAND aware filesystems around. I don't see any reason to provide support for predictable trouble.
You already support it.
/dev/mtdand/dev/mtdblockwork off-the-shelf with NAND devices and allow arbitrary programs/filesystems to overwrite bad blocks if they choose.
No, bad blocks are protected in the nand driver itself. Try to erase a bad block.
We _cannot_ hold anybody off to use it with any driver, userspace app which is able to access /dev/mtdxxxx. Then we would lock out the NAND aware fs too :)
You can also write nonsense to your harddisk. There is no real limitation.
Signature · 8 lines
Thomas ________________________________________________________________________ "Free software" is a matter of liberty, not price. To understand the concept, you should think of "free" as in "free speech,'' not as in "free beer". ________________________________________________________________________ linutronix - competence in embedded & realtime linux http://www.linutronix.de mail: [email protected]
Quoted 31 lines · Thomas Gleixner
Quoted 4 lines · David Daney
Let me start by saying that I am not trying to cause problems, but just to understand the best options...
I was not saying, that you are causing problems :)
Quoted 19 lines
Quoted 16 lines · Thomas Gleixner
Quoted 9 lines · David Daney
Quoted 4 lines
NAND aware filesystem drivers provide their own oobsel structure and use the
xxx_eccfunctions.I am using the cramfs on a NAND partition as my root file system. cramfs is not NAND aware, and I cannot be running userspace programs before mounting as it is the root file system.
I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your root fs. Mount it r/o, so you have no hassle at all.
Quoted 8 lines
Quoted 5 lines
With my NAND drivers, booting the linux kernel and mounting a minimal root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for jffs2. Using cramfs it boots in under 0:10.
He ? 1 minute ? Where is the time spent ?
How should I know?
Quoted 5 lines
Thats totaly out of the usual time. My boot time with JFFS2 is well below 30s on an ARM7. I have no YAFFS root fs handy, but it is much faster.
The solution is checking where the time is lost and fixing it.
I have a custom nand driver (based on nand.c, but adapted to a non standard nand physical interface) that uses software ECC. On a 300 MHz mips32 cpu.
When ever jffs2 or yaffs are mounted, they both seem to read many pages. Perhaps the ECC overhead of reading all that data.
I suppose I could turn off ECC and see how fast it is...
Quoted 10 lines
Quoted 6 lines
That is why I am thinking about using a non NAND aware file system for things that can be read-only.
But this does not answer my question how you want to deal with bad blocks ?
I want a file system very much like cramfs, but that can have holes in it so that it works on NAND.
When mounted it would start scanning blocks starting from the beginning of the NAND partition until it found a valid superblock (or what ever you would call it). Then it would be done because all of the indexes would be built to work around the bad blocks. Since this filesystem would be read-only with a static structure, you would never have to read more than necessary.
David Daney
Quoted 3 lines · David Daney
Quoted 2 lines
He ? 1 minute ? Where is the time spent ?
How should I know?
Profiling :)
Quoted 8 lines
Quoted 4 lines
Thats totaly out of the usual time. My boot time with JFFS2 is well below 30s on an ARM7. I have no YAFFS root fs handy, but it is much faster.
The solution is checking where the time is lost and fixing it.
I have a custom nand driver (based on
nand.c, but adapted to a non standard nand physical interface) that uses software ECC. On a 300 MHz mips32 cpu.
Why did you modify nand.c ?
Almost everything in nand.c can be overridden by the board driver. Therefor we call all the functions through chip->xxx().
Quoted 3 lines
When ever jffs2 or yaffs are mounted, they both seem to read many pages. Perhaps the ECC overhead of reading all that data. I suppose I could turn off ECC and see how fast it is...
Sure, that would give an estimation.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
That is why I am thinking about using a non NAND aware file system for things that can be read-only.
But this does not answer my question how you want to deal with bad blocks ?
Quoted 8 lines
I want a file system very much like cramfs, but that can have holes in it so that it works on NAND.
When mounted it would start scanning blocks starting from the beginning of the NAND partition until it found a valid superblock (or what ever you would call it). Then it would be done because all of the indexes would be built to work around the bad blocks. Since this filesystem would be read-only with a static structure, you would never have to read more than necessary.
OK, so you are going to write a fs driver, which is NAND aware and behaves similar to cramfs.
Whatfor then the discussion about making mtdblock nand aware ? If you write a nand aware fs then you just call the appropriate functions. This fs will be specific for nand or selectable for nand, so what ? No need to touch anything in mtdblock.c
Signature · 8 lines
Thomas ________________________________________________________________________ "Free software" is a matter of liberty, not price. To understand the concept, you should think of "free" as in "free speech,'' not as in "free beer". ________________________________________________________________________ linutronix - competence in embedded & realtime linux http://www.linutronix.de mail: [email protected]
Quoted 3 lines · Thomas Gleixner
Why did you modify
nand.c? Almost everything innand.ccan be overridden by the board driver. Therefor we call all the functions throughchip->xxx().
We need almost everything in nand.c, but the physical access to the NAND is through a hardware state machine that hides the raw chip registers that nand.c uses. So we made a copy of nand.c and hacked it to work.
Quoted 29 lines
Quoted 5 lines
When ever jffs2 or yaffs are mounted, they both seem to read many pages. Perhaps the ECC overhead of reading all that data. I suppose I could turn off ECC and see how fast it is...
Sure, that would give an estimation.
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
That is why I am thinking about using a non NAND aware file system for things that can be read-only.
But this does not answer my question how you want to deal with bad blocks ?
Quoted 10 lines
I want a file system very much like cramfs, but that can have holes in it so that it works on NAND.
When mounted it would start scanning blocks starting from the beginning of the NAND partition until it found a valid superblock (or what ever you would call it). Then it would be done because all of the indexes would be built to work around the bad blocks. Since this filesystem would be read-only with a static structure, you would never have to read more than necessary.
Quoted 2 lines
OK, so you are going to write a fs driver, which is NAND aware and behaves similar to cramfs.
I MAY do it, but would rather that someone else did all the hard work :)
Quoted 5 lines
Whatfor then the discussion about making mtdblock nand aware ?
If you write a nand aware fs then you just call the appropriate functions. This fs will be specific for nand or selectable for nand, so what ? No need to touch anything inmtdblock.c
I guess I am abandoning that for now...
David Daney.