mtd: add writebufsize field to mtd_info struct

PATCH New Holger Brunck 3 participants 10 messages Jan 20, 2011 – Feb 1, 2011

Subsystems (3)

Memory technology devices (MTD) NAND flash Unsorted block images (UBI)

lore

Share & tools

lore

Holger Brunck

author

2011-01-20 16:17 UTC #0 lore raw

PATCH 1/6 mtd: add writebufsize field to mtd_info struct

This field will be used to indicate the write buffer size of the MTD device. UBI will set it's minimal I/O unit size (min_io_size) to the indicated write buffer size. By this change we intend to fix failed recovery of UBIFS partitions we currently observe on NOR flash when mounting the partition after unclean unmount.

Currently the min_io_size is set to mtd->writesize (which is 1 byte for NOR flash). But flash programming is often done from prepared write buffer containing multiple bytes and is performed in one programming operation which could be interrupted by a power cut or a system reset causing corrupted (partially written) areas in a flash sector. Knowing the size of potentially corrupted areas UBIFS scanning and recovery algorithms are able to perform successful recovery.

Signed-off-by Holger Brunck <[email protected]>

---
 include/linux/mtd/mtd.h |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

include/linux/mtd/mtd.h +11 -0

index 3b18d7d..14d6f70 100644
@@ -129,6 +129,17 @@ struct mtd_info {
 	 */
 	u_int32_t writesize;
 
+	/*
+	 * Size of the write buffer used by the MTD. MTD devices having a write
+	 * buffer can write multiple writesize chunks at a time. E.g. while
+	 * writing 4 * writesize bytes to a device with 2 * writesize bytes
+	 * buffer the MTD driver can (but doesn't have to) do 2 writesize
+	 * operations, but not 4. Currently, all NANDs have writebufsize
+	 * equivalent to writesize (NAND page size). Some NOR flashes do have
+	 * writebufsize greater than writesize.
+	 */
+	u_int32_t writebufsize;
+
 	u_int32_t oobsize;   /* Amount of OOB data per block (e.g. 16) */
 	u_int32_t oobavail;  /* Available OOB bytes per block */

Signature · 1 line

1.7.0.5

Holger Brunck

author

2011-01-20 16:17 UTC #1 lore raw

PATCH 2/6 cfi_mtd: add writebufsize initialization

Initialize mtd->writebufsize to the value obtained by by the CFI informations.

Signed-off-by Holger Brunck <[email protected]>

---
 drivers/mtd/cfi_mtd.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

drivers/mtd/cfi_mtd.c +1 -0

index cbcc165..29e32b2 100644
@@ -241,6 +241,7 @@ int cfi_mtd_init(void)
 		mtd->flags		= MTD_CAP_NORFLASH;
 		mtd->size		= fi->size;
 		mtd->writesize		= 1;
+		mtd->writebufsize	= fi->buffer_size;
 
 		mtd->erase		= cfi_mtd_erase;
 		mtd->read		= cfi_mtd_read;

Signature · 1 line

1.7.0.5

Holger Brunck

author

2011-01-20 16:17 UTC #2 lore raw

PATCH 3/6 mtd, nand: add mtd->writebufsize initialization

In reply to Holger Brunck

Initialize mtd->writebufsize to be equal to mtd->writesize.

Signed-off-by Holger Brunck <[email protected]>

---
 drivers/mtd/nand/nand_base.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

drivers/mtd/nand/nand_base.c +1 -0

index 3b96b0e..26ae720 100644
@@ -2843,6 +2843,7 @@ int nand_scan_tail(struct mtd_info *mtd)
 	mtd->unlock = NULL;
 	mtd->block_isbad = nand_block_isbad;
 	mtd->block_markbad = nand_block_markbad;
+	mtd->writebufsize = mtd->writesize;
 
 	/* propagate ecc.layout to mtd_info */
 	mtd->ecclayout = chip->ecc.layout;

Signature · 1 line

1.7.0.5

Holger Brunck

author

2011-01-20 16:17 UTC #3 lore raw

PATCH 4/6 mtd, onenand: add mtd writebufsize initialization

In reply to Holger Brunck

Initialize mtd->writebufsize to be equal to mtd->writesize.

Signed-off-by Holger Brunck <[email protected]>

---
 drivers/mtd/onenand/onenand_base.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

drivers/mtd/onenand/onenand_base.c +1 -0

index 24e02c2..8881672 100644
@@ -2623,6 +2623,7 @@ static int onenand_probe(struct mtd_info *mtd)
 	mtd->sync = onenand_sync;
 	mtd->block_isbad = onenand_block_isbad;
 	mtd->block_markbad = onenand_block_markbad;
+	mtd->writebufsize = mtd->writesize;
 
 	return 0;
 }

Signature · 1 line

1.7.0.5

Holger Brunck

author

2011-01-20 16:17 UTC #4 lore raw

PATCH 5/6 mtd: initialize writebufsize in the MTD object of a partition

In reply to Holger Brunck

Propagate the writebufsize to the partition's MTD object so that UBI can set correct value for it's minimal I/O size using the writebufsize field of MTD object of the partition.

Signed-off-by Holger Brunck <[email protected]>

---
 drivers/mtd/mtdconcat.c |    1 +
 drivers/mtd/mtdpart.c   |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

drivers/mtd/mtdconcat.c +1 -0

index fc22701..a33dcfc 100644
@@ -606,6 +606,7 @@ struct mtd_info *mtd_concat_create(struct mtd_info *subdev[],	/* subdevices to c
 	concat->mtd.size = subdev[0]->size;
 	concat->mtd.erasesize = subdev[0]->erasesize;
 	concat->mtd.writesize = subdev[0]->writesize;
+	concat->mtd.writebufsize = subdev[0]->writebufsize;
 	concat->mtd.subpage_sft = subdev[0]->subpage_sft;
 	concat->mtd.oobsize = subdev[0]->oobsize;
 	concat->mtd.oobavail = subdev[0]->oobavail;

drivers/mtd/mtdpart.c +1 -0

index f647e43..150f2ad 100644
@@ -296,6 +296,7 @@ static struct mtd_part *add_one_partition(struct mtd_info *master,
 	slave->mtd.flags = master->flags & ~part->mask_flags;
 	slave->mtd.size = part->size;
 	slave->mtd.writesize = master->writesize;
+	slave->mtd.writebufsize = master->writebufsize;
 	slave->mtd.oobsize = master->oobsize;
 	slave->mtd.oobavail = master->oobavail;
 	slave->mtd.subpage_sft = master->subpage_sft;

Signature · 1 line

1.7.0.5

Holger Brunck

author

2011-01-20 16:17 UTC #5 lore raw

PATCH 6/6 UBI: use mtd->writebufsize to set minimal I/O unit size

In reply to Holger Brunck

Previously we used mtd->writesize field to set UBI's minimal I/O unit size. This sometimes caused UBIFS recovery issues when mounting an uncleanly unmounted UBIFS partition on NOR flash since mtd->writesize is 1 byte for NOR flash. The MTD CFI driver however often performs writing multiple bytes in one programming operation using the chip's write buffer. We have to use the size of this write buffer as a minimal I/O unit size for UBI on NOR flash to fix the observed UBIFS recovery issues.

Signed-off-by Holger Brunck <[email protected]>

---
 drivers/mtd/ubi/build.c |   28 +++++++++++++++++++++++++++-
 1 files changed, 27 insertions(+), 1 deletions(-)

drivers/mtd/ubi/build.c +27 -1

index 3ea0e6c..7c2e1dc 100644
@@ -542,7 +542,33 @@ static int io_init(struct ubi_device *ubi)
 	if (ubi->mtd->block_isbad && ubi->mtd->block_markbad)
 		ubi->bad_allowed = 1;
 
-	ubi->min_io_size = ubi->mtd->writesize;
+	/*
+	 * Set UBI min. I/O size (@ubi->min_io_size). We use @mtd->writebufsize
+	 * for these purposes, not @mtd->writesize. At the moment this does not
+	 * matter for NAND, because currently @mtd->writebufsize is equivalent to
+	 * @mtd->writesize for all NANDs. However, some CFI NOR flashes may
+	 * have @mtd->writebufsize which is multiple of @mtd->writesize.
+	 *
+	 * The reason we use @mtd->writebufsize for @ubi->min_io_size is that
+	 * UBI and UBIFS recovery algorithms rely on the fact that if there was
+	 * an unclean power cut, then we can find offset of the last corrupted
+	 * node, align the offset to @ubi->min_io_size, read the rest of the
+	 * eraseblock starting from this offset, and check whether there are
+	 * only 0xFF bytes. If yes, then we are probably dealing with a
+	 * corruption caused by a power cut, if not, then this is probably some
+	 * severe corruption.
+	 *
+	 * Thus, we have to use the maximum write unit size of the flash, which
+	 * is @mtd->writebufsize, because @mtd->writesize is the minimum write
+	 * size, not the maximum.
+	 */
+	if (ubi->mtd->type == MTD_NANDFLASH)
+		ubi_assert(ubi->mtd->writebufsize == ubi->mtd->writesize);
+	else if (ubi->mtd->type == MTD_NORFLASH)
+		ubi_assert(ubi->mtd->writebufsize % ubi->mtd->writesize == 0);
+
+	ubi->min_io_size = ubi->mtd->writebufsize;
+
 	ubi->hdrs_min_io_size = ubi->mtd->writesize >> ubi->mtd->subpage_sft;
 
 	/*

Signature · 1 line

1.7.0.5

Detlev Zundel

2011-01-27 16:18 UTC #6 lore raw

PATCH 1/6 mtd: add writebufsize field to mtd_info struct

In reply to Holger Brunck

Hi Stefan,

do you have any comment on this series? I delegated the patches to you in patchwork already :) No seriously, the patches look sane and are in accordance to what is done in Linux....

Quoted 16 lines

This field will be used to indicate the write buffer size of the MTD device. UBI will set it's minimal I/O unit size (min_io_size) to the indicated write buffer size. By this change we intend to fix failed recovery of UBIFS partitions we currently observe on NOR flash when mounting the partition after unclean unmount.
Currently the min_io_size is set to mtd->writesize (which is 1 byte for NOR flash). But flash programming is often done from prepared write buffer containing multiple bytes and is performed in one programming operation which could be interrupted by a power cut or a system reset causing corrupted (partially written) areas in a flash sector. Knowing the size of potentially corrupted areas UBIFS scanning and recovery algorithms are able to perform successful recovery.

Quoted 1 line

Signed-off-by Holger Brunck <[email protected]>

Thanks
Detlev

Signature · 5 lines

vi vi vi - the roman numeral of the beast.
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

Holger Brunck

author

2011-02-01 08:26 UTC #7 lore raw

PATCH 1/6 mtd: add writebufsize field to mtd_info struct

Hi Detlev,

Quoted 24 lines · Detlev Zundel

Hi Stefan,
do you have any comment on this series? I delegated the patches to you in patchwork already :) No seriously, the patches look sane and are in accordance to what is done in Linux....
Quoted 17 lines
This field will be used to indicate the write buffer size of the MTD device. UBI will set it's minimal I/O unit size (min_io_size) to the indicated write buffer size. By this change we intend to fix failed recovery of UBIFS partitions we currently observe on NOR flash when mounting the partition after unclean unmount.
Currently the min_io_size is set to mtd->writesize (which is 1 byte for NOR flash). But flash programming is often done from prepared write buffer containing multiple bytes and is performed in one programming operation which could be interrupted by a power cut or a system reset causing corrupted (partially written) areas in a flash sector. Knowing the size of potentially corrupted areas UBIFS scanning and recovery algorithms are able to perform successful recovery.
Signed-off-by Holger Brunck <[email protected]>

as I told in another mail, the min I/O size adaption patch leads to incompatibilites for the UBIFS and therefore the similar patch in linux kernel was reverted. But anyway the first five patches in the patch serie are already part of the mtd layer of the linux kernel.

So the patches 1-5 of this series can also be committed to u-boot. Whats your opinion?

Regards
Holger Brunck

Stefan Roese

2011-02-01 09:28 UTC #8 lore raw

PATCH 1/6 mtd: add writebufsize field to mtd_info struct

Hi Holger & Detlev,

Quoted 3 lines · On Tuesday 01 February 2011 09:26:56 Holger Brunck wrote:

Quoted 3 lines
do you have any comment on this series? I delegated the patches to you in patchwork already :) No seriously, the patches look sane and are in accordance to what is done in Linux....

I'm back from vacation now and slowly picking up...

Quoted 18 lines

Quoted 16 lines
Quoted 16 lines
This field will be used to indicate the write buffer size of the MTD device. UBI will set it's minimal I/O unit size (min_io_size) to the indicated write buffer size. By this change we intend to fix failed recovery of UBIFS partitions we currently observe on NOR flash when mounting the partition after unclean unmount.
Currently the min_io_size is set to mtd->writesize (which is 1 byte for NOR flash). But flash programming is often done from prepared write buffer containing multiple bytes and is performed in one programming operation which could be interrupted by a power cut or a system reset causing corrupted (partially written) areas in a flash sector. Knowing the size of potentially corrupted areas UBIFS scanning and recovery algorithms are able to perform successful recovery.
Quoted 1 line
Quoted 1 line
Signed-off-by Holger Brunck <[email protected]>

Quoted 7 lines

as I told in another mail, the min I/O size adaption patch leads to incompatibilites for the UBIFS and therefore the similar patch in linux kernel was reverted. But anyway the first five patches in the patch serie are already part of the mtd layer of the linux kernel.
So the patches 1-5 of this series can also be committed to u-boot. Whats your opinion?

I have no problems with applying patches 1-5. But they have been submitted after the merge window was closed, so they will not make it into the next release.

I'm volunteering to push those patches into "next" (once its available) via one of my git repositories (UBI/UBIFS or CFI) if nobody else objects.

Cheers,
Stefan

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-0 Fax: (+49)-8142-66989-80 Email: office at denx.de

Holger Brunck

author

2011-02-01 12:18 UTC #9 lore raw

PATCH 1/6 mtd: add writebufsize field to mtd_info struct

Hi Stefan,

Quoted 9 lines · Stefan Roese

Quoted 8 lines
as I told in another mail, the min I/O size adaption patch leads to incompatibilites for the UBIFS and therefore the similar patch in linux kernel was reverted. But anyway the first five patches in the patch serie are already part of the mtd layer of the linux kernel.
So the patches 1-5 of this series can also be committed to u-boot. Whats your opinion?

Quoted 7 lines

I have no problems with applying patches 1-5. But they have been submitted after the merge window was closed, so they will not make it into the next release.
I'm volunteering to push those patches into "next" (once its available) via one of my git repositories (UBI/UBIFS or CFI) if nobody else objects.

Ok, thanks.

Regards
Holger