Skip to contentSkip to conversation

NAND bad environment block handling

U-Boot Frans Meulenbroeks 6 participants 13 messages Jan 8, 2009 – Jan 11, 2009

lore
Share & tools
lore
#0 lore raw

Hi,

I'm wondering what the best way is to handle bad environment blocks in NAND.

According to the spec of our supplier a delivered component is considered to be OK if less than 2% of the blocks are not bad.
This means that for our products we need to take into account that worst case 2% of the blocks are bad. But even with 1% bad blocks we have an issue:
If the bad blocks are distributed randomly we have a chance of 1/100 * 1/100 so 1 out of 10.000 that both U-Boot environment blocks are bad.
And actually things could even be worse if the bad blocks are caused by some manufacturing defect at our supplier and both environment blocks happen to be bad in a whole batch of ICs.

Is there a solution for this?
(obviously I am not considering the situation for a single system, where I just would relocate the environment block; this concerns a production situation).

E.g. it would be nice if U-Boot could read the env from the next non-bad block at/after the env address.

Has someone experience in this area? Ideas? Suggestions?

Thanks alot!
Frans Meulenbroeks

#1 lore raw

My understanding is that u-boot already support bad block for the environment config. When reading or writing the config from/to the nand, u-boot skip bad block. However, you need to reserve some spare blocks in nand for environment config. This is done in u-boot-nand.lds:

/* Align to next NAND block. The NAND has block of 128K ==> 0x20000*/

    . = ALIGN(0x20000);
    common/environment.o  (.ppcenv)
    /* Keep some space here for redundant env and potential bad env blocks */
    . = ALIGN(0x80000);

Regards

Sylvain C?t?

-----Original Message-----
From: [email protected] [mailto:u-boot-bounces at lists.denx.de] On Behalf Of Frans Meulenbroeks
Sent: Thursday, January 08, 2009 10:19 AM
To: u-boot at lists.denx.de
Subject: [U-Boot] NAND bad environment block handling

Hi,

I'm wondering what the best way is to handle bad environment blocks in NAND.

According to the spec of our supplier a delivered component is considered to be OK if less than 2% of the blocks are not bad.
This means that for our products we need to take into account that worst case 2% of the blocks are bad. But even with 1% bad blocks we have an issue:
If the bad blocks are distributed randomly we have a chance of 1/100 * 1/100 so 1 out of 10.000 that both U-Boot environment blocks are bad.
And actually things could even be worse if the bad blocks are caused by some manufacturing defect at our supplier and both environment blocks happen to be bad in a whole batch of ICs.

Is there a solution for this?
(obviously I am not considering the situation for a single system, where I just would relocate the environment block; this concerns a production situation).

E.g. it would be nice if U-Boot could read the env from the next non-bad block at/after the env address.

Has someone experience in this area? Ideas? Suggestions?

Thanks alot!
Frans Meulenbroeks

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

#2 lore raw

Sylvain, Scott,

Thanks for the info.
If the bad env blocks are skipped that at least would solve my problem.
I hope that it does not break fw_setenv.
Storing the pointer in oob in block 0 is an interesting idea. I need to think about the impact on that one a little bit more.

With large blocks definitely the waste is big. Ideal solution would be that the hw does bad block mapping. Somewhat less ideal would be to position partitions dynamically and specify the mtd partition start points at the kernel cmd line. Issue is that this is complicates the software upgrade. Even less ideal but common practice today seems to be to reserve slack room for every partition (or even worse, ignore the problem and assume perfect NAND)

Best regards, Frans

Quoted 69 lines · Sylvain

From: Cote, Sylvain <[email protected]>
Subject: RE: [U-Boot] NAND bad environment block handling
To: fransmeulenbroeks at yahoo.com, u-boot at lists.denx.de
Date: Thursday, January 8, 2009, 5:06 PM
My understanding is that u-boot already support bad block
for the environment config. When reading or writing the
config from/to the nand, u-boot skip bad block. However, you need to reserve some spare blocks in nand for environment config. This is done in u-boot-nand.lds:

/* Align to next NAND block. The NAND has block of 128K ==> 0x20000*/

    . = ALIGN(0x20000);
    common/environment.o  (.ppcenv)
    /* Keep some space here for redundant env and potential
bad env blocks */
    . = ALIGN(0x80000);

Regards

Sylvain C?t?

-----Original Message-----
From: u-boot-bounces at lists.denx.de [mailto:u-boot-bounces at lists.denx.de] On Behalf Of Frans Meulenbroeks
Sent: Thursday, January 08, 2009 10:19 AM
To: u-boot at lists.denx.de
Subject: [U-Boot] NAND bad environment block handling

Hi,

I'm wondering what the best way is to handle bad environment blocks in NAND.

According to the spec of our supplier a delivered component is considered to be OK if less than 2% of the blocks are not bad.
This means that for our products we need to take into account that worst case 2% of the blocks are bad. But even with 1% bad blocks we have an issue:
If the bad blocks are distributed randomly we have a chance of 1/100 * 1/100 so 1 out of 10.000 that both U-Boot environment blocks are bad.
And actually things could even be worse if the bad blocks are caused by some manufacturing defect at our supplier and both environment blocks happen to be bad in a whole batch of ICs.

Is there a solution for this?
(obviously I am not considering the situation for a single system, where I just would relocate the environment block; this concerns a production situation).

E.g. it would be nice if U-Boot could read the env from the next non-bad block at/after the env address.

Has someone experience in this area? Ideas? Suggestions?

Thanks alot!
Frans Meulenbroeks

_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

#3 lore raw
In reply to Frans Meulenbroeks
Quoted 7 lines · Frans Meulenbroeks

Is there a solution for this? (obviously I am not considering the situation for a single system, where I just would relocate the environment block; this concerns a production situation).

E.g. it would be nice if U-Boot could read the env from the next non-bad block at/after the env address.

Has someone experience in this area? Ideas? Suggestions?

As others pointed out, this is supported; just make the environment range larger than the environment size.

However, this can be wasteful, especially with large block sizes; someone a while ago proposed putting a pointer to the environment in the OOB of the first block.

-Scott

#4 lore raw
Quoted 6 lines · Scott Wood <scottwood <at> freescale.com> writes:
Quoted 5 lines · Frans Meulenbroeks

Is there a solution for this? (obviously I am not considering the situation for a single system, where I just would relocate the environment block; this concerns a production situation).

E.g. it would be nice if U-Boot could read the env from the next non-bad

block at/after the env address.

Quoted 9 lines
Quoted 2 lines

Has someone experience in this area? Ideas? Suggestions?

As others pointed out, this is supported; just make the environment range larger than the environment size.

However, this can be wasteful, especially with large block sizes; someone a while ago proposed putting a pointer to the environment in the OOB of the first block.

The openmoko guys do have an alternative implementation which dynamically sizes all partitions in NAND according to what bad blocks are found therefore not wasting any space. And they are using it in full-scale production environment for the GTA02. Unfortunately I don't think this made it into mainline just yet.

#5 lore raw

Dear Marcel Ziswiler,

Quoted 5 lines · In message <[email protected]> you wrote:

The openmoko guys do have an alternative implementation which dynamically sizes all partitions in NAND according to what bad blocks are found therefore not wasting any space. And they are using it in full-scale production environment for the GTA02. Unfortunately I don't think this made it into mainline just yet.

This has other problems - bad blocks are not a static thing: new ones develop over time, and shifting partitions around is not exactly nice.

You probably want to look into UBI support...

Best regards,

Wolfgang Denk

Signature · 5 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Advice is seldom welcome; and those who want it the most always like
it the least.                         -- Philip Earl of Chesterfield
#6 lore raw
In reply to Frans Meulenbroeks

Hi!
Am Donnerstag, den 08.01.2009, 07:19 -0800 schrieb Frans Meulenbroeks:

Quoted 2 lines

Has someone experience in this area? Ideas? Suggestions?

1. For our systems we assume, that the env-block never gets bad in productive use, because the env-block is never written again. So we reduced the problem to number of initial/factory bad blocks.
2. the partitioning and environment-saving is done at production-time.
3. we put the environment in the last good block of the device. The search starts at the end of device and ends at the last block of the last data-partition with an error.

Example-Partition-Table on an 32MB-NAND, with blocksize 16kb and max. 45 factory bad-blocks:
* the part-size means the possible data size (without bad-blocks).
* the real partition-size on device depends on the bad-blocks in this
the partition and is calculated automatically at production-time (partition generation).

part-size partition

 200kb:         uboot
 1000kb:        kernel
 15000kb:       rootfs
 15832kb:       datafs

rest of device: rest (maximal 45*16kb (max bad blocks) + 16bk (envblock) = 736kb

with an increasing number of initial/factory bad-blocks the rest-partition shrinks. So the env-block can be written on any nand-device without less than 45 (initial/factory) bad-blocks.

Quoted 4 lines

Thanks alot!
Frans Meulenbroeks

- Manfred

#7 lore raw

Manfred,

Thanks for your suggestions.
Did you modify u-boot to search from the last block of the device backwards? I feel putting the env at the end is quite a good idea.

Unfortunately in our system we need to modify the env at runtime.
For software upgrade we aim to go for a ping pong strategy (so having two kernels and two rfs-es in flash, and upgrade the non active kernel, then modify u-boot env so the next time we boot from the upgraded kernel). Unless of course there is a better strategy for this.

Best regards, Frans.

Quoted 60 lines · Schlaegl Manfred jun.

From: Schlaegl Manfred jun. <[email protected]>
Subject: Re: [U-Boot] NAND bad environment block handling
To: fransmeulenbroeks at yahoo.com
Cc: u-boot at lists.denx.de
Date: Sunday, January 11, 2009, 12:10 PM
Hi!
Am Donnerstag, den 08.01.2009, 07:19 -0800 schrieb Frans Meulenbroeks:

Quoted 2 lines

Has someone experience in this area? Ideas?

Suggestions?

1. For our systems we assume, that the env-block never gets bad in
productive use, because the env-block is never written again. So we
reduced the problem to number of initial/factory bad blocks.
2. the partitioning and environment-saving is done at production-time.
3. we put the environment in the last good block of the device. The
search starts at the end of device and ends at the last block of the
last data-partition with an error.

Example-Partition-Table on an 32MB-NAND, with blocksize 16kb and max. 45
factory bad-blocks:
* the part-size means the possible data size (without
bad-blocks).
* the real partition-size on device depends on the
bad-blocks in this
the partition and is calculated automatically at production-time
(partition generation).

part-size partition

 200kb:         uboot
 1000kb:        kernel
 15000kb:       rootfs
 15832kb:       datafs

rest of device: rest (maximal 45*16kb (max bad blocks) + 16bk (envblock)
= 736kb

with an increasing number of initial/factory bad-blocks the rest-partition shrinks. So the env-block can be written on any
nand-device without less than 45 (initial/factory) bad-blocks.

Quoted 4 lines

Thanks alot!
Frans Meulenbroeks

- Manfred

#8 lore raw

Hi!
Am Sonntag, den 11.01.2009, 05:26 -0800 schrieb Frans Meulenbroeks:

Quoted 6 lines

Manfred,

Thanks for your suggestions.
Did you modify u-boot to search from the last block of the device backwards? I feel putting the env at the end is quite a good idea.

Yes. First I wanted to implement the openmoku-solution: putting the address of env-block in first block oob.
But then I thought it would be much simpler to put the env at the second distinctive position on a nand device - the end.

Yes, I had to do some modification on uboot. But I had to do this on uboot-1.1.3(delivered by our hardware-supplier) which is quite old. I'm currenty on work to port our architecture to 2009q1.

Quoted 2 lines

Unfortunately in our system we need to modify the env at runtime.

The solution to put the environment at end of device may also be combined with redundant environment.

Quoted 2 lines

For software upgrade we aim to go for a ping pong strategy (so having two kernels and two rfs-es in flash, and upgrade the non active kernel, then modify u-boot env so the next time we boot from the upgraded kernel). Unless of course there is a better strategy for this.

Yes. Its a very common strategie. We also use two kernel-partitions.

- Manfred

#9 lore raw
In reply to Schlaegl Manfred jun.

Dear Manfred,

Quoted 4 lines · In message <[email protected]> you wrote:

1. For our systems we assume, that the env-block never gets bad in productive use, because the env-block is never written again. So we reduced the problem to number of initial/factory bad blocks.

That's a pretty risky setup because it is based on an incorrect assumption - on NAND, block become not only bad when being erased or written to, but also after a certain number of read operations is exceeded. Yes, there is a maximum number of reads per block on NAND! People like to ignore that... See for example the Micron Technical
Note "Design and Use Considerations for NAND Flash Memory",
http://download.micron.com/pdf/technotes/nand/tn2917.pdf

Time to look into UBI, me thinks.

Best regards,

Wolfgang Denk

Signature · 5 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
No question is too silly to ask. Of course, some  questions  are  too
silly to to answer...  - L. Wall & R. L. Schwartz, _Programming Perl_
#10 lore raw
Quoted 15 lines

That's a pretty risky setup because it is based on an incorrect
assumption - on NAND, block become not only bad when being erased or
written to, but also after a certain number of read operations is
exceeded. Yes, there is a maximum number of reads per block on NAND!
People like to ignore that... See for example the Micron Technical
Note "Design and Use Considerations for NAND
Flash Memory",
http://download.micron.com/pdf/technotes/nand/tn2917.pdf

Thanks for the link. I was not aware of read degradation. That is what most manufacturers do not tell you.

Quoted 2 lines

Time to look into UBI, me thinks.

For u-boot env settings??

My problem that the designated env blocks can be bad or become bad during the lifetime of the device. Not desirable especially not for expensive devices.

Frans.

#11 lore raw

Dear Frans Meulenbroeks,

Quoted 4 lines · In message <[email protected]> you wrote:
Quoted 2 lines

Time to look into UBI, me thinks.

For u-boot env settings??

Why not? Once the flash is UBI managed, there are virtually no bad blocks any more.

Quoted 1 line

My problem that the designated env blocks can be bad or become bad during the lifetime of the device. Not desirable especially not for expensive devices.

As mentioned before, U-Boot supports bad block handling on NAND, so just allocate a few spare blocks to the environment area.

Best regards,

Wolfgang Denk

Signature · 5 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Keep your eyes wide open before marriage, half shut afterwards.
                                                 -- Benjamin Franklin
#12 lore raw
In reply to Wolfgang Denk

Am Sonntag, den 11.01.2009, 14:28 +0100 schrieb Wolfgang Denk:

Quoted 8 lines

Dear Manfred,

Quoted 4 lines · In message <[email protected]> you wrote:

1. For our systems we assume, that the env-block never gets bad in productive use, because the env-block is never written again. So we reduced the problem to number of initial/factory bad blocks.

Quoted 7 lines

That's a pretty risky setup because it is based on an incorrect assumption - on NAND, block become not only bad when being erased or written to, but also after a certain number of read operations is exceeded. Yes, there is a maximum number of reads per block on NAND! People like to ignore that... See for example the Micron Technical
Note "Design and Use Considerations for NAND Flash Memory",
http://download.micron.com/pdf/technotes/nand/tn2917.pdf

Yes I know. I think my description was a little bit misunderstanding:
Our solution is only a solution for positioning env-block(s) and handling initial/factory bad blocks. This solution can(should) combined with redundant env-blocks too, if write/erase-wearout is an issue.

The problem you discribe is a general nand-flash-problem. To keep the system aware of this you theoretically have to implement a redundant uboot partition too. It makes no sense to have two env-blocks(except for erase/write-wearout-issues), but only one u-boot binary in nand-flash.

Quoted 2 lines

Time to look into UBI, me thinks.

Yes I think too. It would make things much easier .. :-)
Perhaps: Uboot + Environment (ev. Kernel) on NOR-Flash and filesystem-partitions on NAND/UBI.

Best regards,
Manfred