Skip to contentSkip to conversation

Lifetime of flash memory

DISCUSSION John Richard Moser 6 participants 8 messages Mar 21, 2006 – Mar 23, 2006

lore
Share & tools
lore
#0 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have a kind of dumb question. I keep hearing that "USB Flash Memory" or "Compact Flash Cards" and family have "a limited number of writes" and will eventually wear out. Recommendations like "DO NOT PUT A SWAP FILE ON USB MEMORY" have come out of this. In fact, quoting Documentation/laptop-mode.txt:

* If you're worried about your data, you might want to consider using a USB memory stick or something like that as a "working area". (Be aware though that flash memory can only handle a limited number of writes, and overuse may wear out your memory stick pretty quickly. Do _not_ use journalling filesystems on flash memory sticks.)

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes; and that compact flash cards, USB drives, SD cards, and family typically have integrated control chipsets that include wear-leveling algorithms (built-in flash like in an iPaq does not; hence jffs2). Should we really care that in about 95 billion years the thing will wear out (assuming we write its entire capacity once a second)?

I call FUD.

- --
All content of all messages exchanged herein are left in the Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be wasted on re-inventing the wheel when there are so many fascinating new problems waiting out there.
-- Eric Steven Raymond

We will enslave their women, eat their children and rape their cattle!
-- Evil alien overlord from Blasto
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRCAxdws1xW0HCTEFAQJd6w//auD2v3RJYxTbUePJwXFriCTO2d35+uo1 xU80/Brd7Hkdn82hfk/Rozoj6zsZFYYpYqqDhvo0aOKUW/cxZhTymXlEUgNXx0k+ s2hkVM4+nXoJhQhFuLk3/bPXBQlu20xA1tt6pHMscIfavijPSn7aV7gPx+L+SpDD VqGdsmynt68IRk09b9su0gsfuM0OxYrjVAXPN5l+cjzlEk+fyHGIALu26UwiL+31 Gs86zviWaX1MwK5G0IZQ0ITySG/wNGoMNcbSdbm/45r0JnLhHPQjX2WGwIh7t5Y2 UeoYLRZJ5gRF9PT0yP5tMy0XXhKpj0aEtl8ccB/aeOCPsUKAC+2K2SFCfZLZCj8x GOGeJKsutim+H+Qec/lnOng1LYoA9fJaisGzAUEOHYhFuYOioPVvGBKiRQlX6mMf ofCAIOwtzWgxTa4kJrhU3oF0DYhLtP7Je/LCQW0RqmnMrXcR23/AwBa5fHTzhW1C Mb6eL1TtYPYoyoBcKKYgKMmXLXu4d2klgxM4RRpcCrVfrupsHXr5VSzt+XYf7twX TnY6DhmVVqp1YIVbWPXbNHplQuOU7ywdu+Y7q75jywqFBxGqeo+mPoL8ItW3IthZ /zaoJVUH+n0FyydC+FYJ3SWx7AkPx46hZmO2UQmVlOAq2Fuc8I3haaOIQmADt0Ar pwGzS3E92J0=
=48mD
-----END PGP SIGNATURE-----

#1 lore raw
Quoted 3 lines · John Richard Moser

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes;

That's like, uh, 13 orders of magnitudes out...

David Vrabel

#2 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 6 lines · David Vrabel
Quoted 2 lines · John Richard Moser

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes;

That's like, uh, 13 orders of magnitudes out...

Yeah I did more searching, it looks like that was a mass overstatement.
There was one company that did claim to have developed flash memory
with that size (I think it was 3.8x10^18) but it looks like typical drives are 1.0x10^6 with an on-chip wear-leveling algorithm. Assuming the drive is like 256 megs with 64k blocks, that's still 129 years at one write per second.

Bigger drives of course level over larger area and lifetime increases linearly. My 512M drive should last 260 years in that scheme; a 4 gig iPod Nano would last 2080 years; and a 30GiB flash-based hard disk in a Samsung laptop on a single control chip doing the wear leveling over multiple NAND chips would last 15600 years.

In theory anyway. And assuming one write on one block per one second on average for the duration. (obviously the iPod nano sustains many times less than that and will last hundreds of thousands of years in normal usage).

Quoted 2 lines

David Vrabel

- --
All content of all messages exchanged herein are left in the Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be wasted on re-inventing the wheel when there are so many fascinating new problems waiting out there.
-- Eric Steven Raymond

We will enslave their women, eat their children and rape their cattle!
-- Evil alien overlord from Blasto
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRCA30As1xW0HCTEFAQIELQ//TjtHD2EIh+DphWoe10MyAwt/MsWjdJEL UWir35umv6gta6tnv4cI92PquSMAzyqeGcEe5l1Yhi9nGTe5jWqiiwxg8tVfaecq Lf04pt53N9nVYR+lGd7DxDEq3ZCYeQKcE1hY3pnP3IHEnayfEHGl6zb8rTWEeKxm o6miFUQoxVOXqcTHD8bLLJAJcTBsLn1IO6gAS9/WA4tvTYo4471E0m+ORY7WgFYK /3fpq5a+PgbKkcTjRJdODaxhAROIjElwkTCPtjr/3wpjelOl1BpuTRzl8HxpAmEN 9Ybnophs6SnLeccE2WIW6PNC/cjgkyiZigOLE0EWBflJaM5ij9ZeW7Ju/FSxjhYK e2YB6SrREFJ4Gs4eXOvzPy658JE+kbr1OtO3TIfJFykGY2tTsBvtKPvWGdFx2IJt Znp+4vNcOtCO8Wd7uoMv+Sewk7AmqSpB5VPt64UZqGudM94Z3YDkdnUM7FjLkeng ank4DFmzjKln2etmDo+25orQbSbPxR8UNRuWJCjOS0NTNN57fiMyIsqsSFFcuqsO Ud8fwqvsrSLhXs1xzSxsWMvcZm/RsAgvOPAp+oajCjLVEvP2alRoLtu/yGmRgwEk e2+Sa1zrAh2qZv2az0JLVr3gWfjRoKcn39QkF9rmiNpmr3Rf6Jx8PYjF1jYN2UDt 3j34unpH2gM=
=sTVH
-----END PGP SIGNATURE-----

#3 lore raw
Quoted 7 lines · John Richard Moser
Quoted 5 lines · David Vrabel
Quoted 2 lines · John Richard Moser

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes;

Quoted 6 lines
Quoted 1 line

That's like, uh, 13 orders of magnitudes out...

Yeah I did more searching, it looks like that was a mass overstatement.
There was one company that did claim to have developed flash memory
with that size (I think it was 3.8x10^18) but it looks like typical drives are 1.0x10^6 with an on-chip wear-leveling algorithm.

That is still high. Modern flash drives will do 100.000 writes for SLC (single-level cells) or 10.000 writes for MLC (multi-level cells) [1].

Quoted 3 lines

Assuming
the drive is like 256 megs with 64k blocks, that's still 129 years at one write per second.

This is also assuming _perfect_ wear leveling. There are real world drives with crappy (or even buggy) wear levelling. I've seen CF cards die with much less writting than this.

Even then, with just 10.000 writes, this is already reduced to 1.29 years, assuming 64kb/sec average writting.

If you take into consideration that you can actually write 6Mbytes/sec on a modern CF card, you can fry a 256Mb card in just 5 days, if you write continuously.

Signature · 6 lines
Paulo Marques - www.grupopie.com

Pointy-Haired Boss: I don't see anything that could stand in our way.
           Dilbert: Sanity? Reality? The laws of physics?

[1] check out: http://www.kingston.com/products/DMTechGuide.pdf
#4 lore raw
In reply to David Vrabel
Quoted 5 lines · David Vrabel
Quoted 2 lines · John Richard Moser

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes;

That's like, uh, 13 orders of magnitudes out...

Flash drives are cheap anyway. I saw a 256mB one for $10 at Circuit City. A PNY one. After rebates... but still, you can't beat that.

Quoted 7 lines

David Vrabel
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 23 lines
--hackmiester
Walk a mile in my shoes and you will be a mile away in a new pair of shoes.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD/yYl3ApzN91C7BcRAoVVAJ97uhjh30nQ4hd9bQ90gJqiwsLEfgCeKSrg
bVfqEeJ09WhO6Y51WHEHb6o=
=VTUd
-----END PGP SIGNATURE-----

-----BEGIN GEEK CODE BLOCK-----
Version: Geek Code v3.1 (PHP)
GCS/CM/E/IT d-@ s: a- C++$ UBLS*++++$ P+ L+++$ E- W++$ !N-- !o+ K-- !w-- !O-
M++$ V-- PS@ PE@ Y--? PGP++ !t--- 5--? !X-- !R-- tv-- b+ DI++ D++ G+ e++++
h---- r+++ z++++
------END GEEK CODE BLOCK------

Quick contact info:
Work: [email protected]
Personal: [email protected]
Large files/spam: [email protected]
GTalk:hackmiester/AIM:hackmiester1337/Y!:hackm1ester/IRC:irc.7sinz.net/7sinz
#5 lore raw
In reply to John Richard Moser
Quoted 20 lines · John Richard Moser

I have a kind of dumb question. I keep hearing that "USB Flash Memory" or "Compact Flash Cards" and family have "a limited number of writes" and will eventually wear out. Recommendations like "DO NOT PUT A SWAP FILE ON USB MEMORY" have come out of this. In fact, quoting Documentation/laptop-mode.txt:

* If you're worried about your data, you might want to consider using a USB memory stick or something like that as a "working area". (Be aware though that flash memory can only handle a limited number of writes, and overuse may wear out your memory stick pretty quickly. Do _not_ use journalling filesystems on flash memory sticks.)

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes; and that compact flash cards, USB drives, SD cards, and family typically have integrated control chipsets that include wear-leveling algorithms (built-in flash like in an iPaq does not; hence jffs2). Should we really care that in about 95 billion years the thing will wear out (assuming we write its entire capacity once a second)?

Quoted 1 line

I call FUD.

Search for a thread on LKML having to do with enabling "sync" on removable media, especially VFAT media. If you are copying a large file, and the FAT on the device is being updated with every block, you can literally fry your device in a matter of minutes, because the FAT is always in the same spot, thus it is always overwriting the same spot.

j----- k-----

Signature · 3 lines
Joshua Kugler                 PGP Key: http://pgp.mit.edu/
CDE System Administrator             ID 0xDB26D7CE
http://distance.uaf.edu/
#6 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 21 lines · Joshua Kugler
Quoted 20 lines · John Richard Moser

I have a kind of dumb question. I keep hearing that "USB Flash Memory" or "Compact Flash Cards" and family have "a limited number of writes" and will eventually wear out. Recommendations like "DO NOT PUT A SWAP FILE ON USB MEMORY" have come out of this. In fact, quoting Documentation/laptop-mode.txt:

* If you're worried about your data, you might want to consider using a USB memory stick or something like that as a "working area". (Be aware though that flash memory can only handle a limited number of writes, and overuse may wear out your memory stick pretty quickly. Do _not_ use journalling filesystems on flash memory sticks.)

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes; and that compact flash cards, USB drives, SD cards, and family typically have integrated control chipsets that include wear-leveling algorithms (built-in flash like in an iPaq does not; hence jffs2). Should we really care that in about 95 billion years the thing will wear out (assuming we write its entire capacity once a second)?

Quoted 8 lines
Quoted 1 line

I call FUD.

Search for a thread on LKML having to do with enabling "sync" on removable media, especially VFAT media. If you are copying a large file, and the FAT on the device is being updated with every block, you can literally fry your device in a matter of minutes, because the FAT is always in the same spot, thus it is always overwriting the same spot.

I've run with 'sync', it makes the removable device operate at a blazing 1.2k/s transfer rate instead of 13M/s. I actually tried to dd if=/dev/zero of=/media/usbdisk bs=64k to zero out 150 megs of free space, but gave up after about half an hour. This was a test to see the speed of the drive under sync/nosync modes.

I thought these things had wear leveling on the control chips, seriously.

"USB mass storage controller - implements the USB host controller and provides a seamless linear interface to block-oriented serial flash devices while hiding the complexities of block-orientation, block erasure, and wear balancing. The controller contains a small RISC microprocessor and a small amount of on-chip ROM and RAM. (item 2 in the diagram)"

^^^ From a diagram of a USB flash drive on Wikipedia.
http://en.wikipedia.org/wiki/USB_flash_drive

These drives seem to be rated for millions of writes:

"In normal use, mid-range flash drives currently on the market will support several million cycles, although write operations will gradually slow as the device ages."

Although there have been cases...

"A few cheaper USB flash drives have been found to use unsuitable flash memory chips labelled as 'ROM USE ONLY' - these are intended for tasks such as Flash BIOS for Routers rather than for continual rewrite use, and fail after a very small number of cycles. [6]"

At which point we obviously know we shouldn't be doing what we're doing with these things in the first place!

With proper wear balancing, a million writes across a drive should last quite a while. About:

(C*10.0^6bytes * ((64.0 * 1024bytes) / Wblocks)) / (60s/min * 60min/hr * 24.0hr/day * 365.25day/yr)

C=Capacity, W=Written blocks. This gives you how many years the drive lasts writing W blocks per second (the unit is years/blockseconds).

For a 256M flash drive this should be 130 years; a 4GiB iPod nano should last 2080 years under this abuse (1 write to 1 block per 1 second); and a 32GiB "flash hard disk" should last like 16600 years. That does, of course, assume wear-leveling takes the entire storage area into account instead of localizing to a single small area.

Quoted 2 lines

j----- k-----

- --
All content of all messages exchanged herein are left in the Public Domain, unless otherwise explicitly stated.

Creative brains are a valuable, limited resource. They shouldn't be wasted on re-inventing the wheel when there are so many fascinating new problems waiting out there.
-- Eric Steven Raymond

We will enslave their women, eat their children and rape their cattle!
-- Evil alien overlord from Blasto
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQIVAwUBRCBIsQs1xW0HCTEFAQK9EQ/9FYw9OzNYWrJFqLvtQgKegAvCFps2piR0 ZGqiLTHCA+BApZ3k+1vdxdFHKpT9t3HDsvjsS1GoIRqbMyD1u+ZbbjKUBLf5DQSO SyBnpQRfnDOXoXnNM92PRObuRbvsVHzwpMMd79IljFKmneUnUrw0wem2/rv1FEbb Q8c6IzLmzK/18oG0OowxWl8DjhwY8QmAPA45SJ0VE9RL3nM0uigY7ISVNEMK6qEb sNuQ9tYkE8qkzTakbk09pVG7G9GljVbUMbBbkwb2v7aihnSEcFMCl717+KrDxuRL ogGSZHZcF/bNs8FNlHAPBm057xHYpBiGaVwvM9Nm6aXPBxzxVed4iyTQ3NUyiy0K m7YnpLrSJ3vDxge3tB/sOoIvXTg1LlfU5tjqE37YGBGxmuSNXyeP9yWuuw00Fxzd osYKtu7FX6h8REo5NUtOB+1t7HGe3dMa7PAuBjz6f+MbBd0psr/4id/ecm6m7yuU UJDHYVnN5kN/kDahN9VrEWLW7i04tAiCEPccQ/42pBM7EzjUkeBmNSWvn18XVBr+ 2A+EIDY/+BtmJHJyC0WTvFN5mlnJ/tSdNP0zQx/5o5XvqdQEw1SnbmsToYl3w+MD i2k0dt+8VUgLGv3YJYfTGYasllND//qI8kqRFzkcxifSVDI4eWm8p18fw8XSOluK 0RzELRz6VK4=
=9NS3
-----END PGP SIGNATURE-----

#7 lore raw
In reply to John Richard Moser
Quoted 11 lines · John Richard Moser

I have a kind of dumb question. I keep hearing that "USB Flash Memory" or "Compact Flash Cards" and family have "a limited number of writes" and will eventually wear out. Recommendations like "DO NOT PUT A SWAP FILE ON USB MEMORY" have come out of this. In fact, quoting Documentation/laptop-mode.txt:

* If you're worried about your data, you might want to consider using a USB memory stick or something like that as a "working area". (Be aware though that flash memory can only handle a limited number of writes, and overuse may wear out your memory stick pretty quickly. Do _not_ use journalling filesystems on flash memory sticks.)

I thought that journaling filesystems happen to overwrite exactly the same place (where the journal is) many times... Am I mistaken?

So the effect is what we had for floppies (some many years ago) where sector 0 and others where FAT structure was kept were overused and start giving errors - so the only solution was to throw away that floppy.

Hard disks had the same problem, but they have algorithms to relocate bad clusters.

So do these "leveling algorithms" refer to the same? Relocating bad cells? If not, you can see how a journaling system can fry a CF card quickly.

Quoted 10 lines

The question I have is, is this really significant? I have heard quoted that flash memory typically handles something like 3x10^18 writes; and that compact flash cards, USB drives, SD cards, and family typically have integrated control chipsets that include wear-leveling algorithms (built-in flash like in an iPaq does not; hence jffs2). Should we really care that in about 95 billion years the thing will wear out (assuming we write its entire capacity once a second)?

I call FUD.

3x10^18 is a bit overstating, IMHO. Don't have a reference now.

Kalin.

Signature · 3 lines
|[ ~~~~~~~~~~~~~~~~~~~~~~ ]|
+-> http://ThinRope.net/ <-+
|[ ______________________ ]|