Skip to contentSkip to conversation

Compactflash cards dying?

DISCUSSION Pavel Machek 16 participants 28 messages Feb 2, 2003 – Feb 5, 2003

lore
Share & tools
lore
#0 lore raw

Hi!

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

Anyone seen something similar? Are there some known-good compactflash-es?

Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#1 lore raw

Hi Pavel !

I had this same problem with my very first CF (16 MB) connected to a home-made IDE adapter. I quickly discovered that the power wire (+5V) had been cut and that the power was driven through the logic signals, which were strong enough to let the card work correctly... nearly correctly. Because it got uncorrectable defects, detected as bad sectors at IDE level. So may be your adapter is too weak. Or may be you also use it in a battery-powered device which has frequent power outages ?

Cheers,
Willy

Quoted 26 lines · Pavel Machek

Hi!

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

Anyone seen something similar? Are there some known-good compactflash-es?

Pavel

Signature · 7 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
#2 lore raw

Hi!

Quoted 7 lines

I had this same problem with my very first CF (16 MB) connected to a home-made IDE adapter. I quickly discovered that the power wire (+5V) had been cut and that the power was driven through the logic signals, which were strong enough to let the card work correctly... nearly correctly. Because it got uncorrectable defects, detected as bad sectors at IDE level. So may be your adapter is too weak. Or may be you also use it in a battery-powered device which has frequent power outages ?

I've seen it on sharp zaurus, and same flash had bad problems in toshiba 4030cdt notebook. Zaurus is PDA but it *does* monitor voltage; I can't imagine toshiba having problems with power.

Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#3 lore raw
In reply to Pavel Machek

Am Son, 2003-02-02 um 23.30 schrieb Pavel Machek:

Quoted 7 lines

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

Anyone seen something similar? Are there some known-good compactflash-es?

CF has limited write cycles. A few hundred if you're lucky. And depending on the type of flash it's quite likely that every changed byte will result in a whole block being written back.

I'm running dotzends of CF cards and due to some care not a single one has developped bad blocks as of yet.

Signature · 2 lines
Servus,
       Daniel
#4 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Daniel" == Daniel Egger <[email protected]> writes:

Daniel> CF has limited write cycles. A few hundred if you're lucky.

Aah, that's a bit pessimistic. Even for a regular flash.

It is true that the number of write cycles varies between cards. Some vendors have both consumer and industrial grade cards, and the industrial grade ones use better parts inside.

However, I have yet to see a CF card which didn't survive beyond a million writes.

Note that CF cards do transparent wear averaging inside. So it's obviously not a million writes to the same physical spot. Also, most vendors claim they have spare blocks for relocating areas that are completely worn out.

So while a Compact Flash isn't a hard disk, it is at least a couple of orders of magnitude better than "hundreds of writes".

Signature · 2 lines
Martin K. Petersen	Wild Open Source, Inc.
[email protected]	http://www.wildopensource.com/
#5 lore raw
Quoted 11 lines · Martin K. Petersen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Daniel" == Daniel Egger <[email protected]> writes:

Daniel> CF has limited write cycles. A few hundred if you're lucky.

However, I have yet to see a CF card which didn't survive beyond a million writes.

Note that CF cards do transparent wear averaging inside. So it's obviously not a million writes to the same physical spot. Also, most vendors claim they have spare blocks for relocating areas that are completely worn out.

This apears to be the same statement, except that Daniel forgot to add "per block".

Jörn

Signature · 5 lines
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS d- s+: a- C++ UL++++ P+(++) L++++$ !E W++ N+ o? K? w- O- M- V?
PS+(++) PE++ Y+ PGP>+ t+@ 5? X+ R@ !tv b+ DI+ !D G- e h-- r y+
------END GEEK CODE BLOCK------
#6 lore raw
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jörn" == Jörn Engel <[email protected]> writes:

Note that CF cards do transparent wear averaging inside. So it's obviously not a million writes to the same physical spot. Also, most vendors claim they have spare blocks for relocating areas that are completely worn out.

Jörn> This apears to be the same statement, except that Daniel forgot Jörn> to add "per block".

They'd have to use really shitty flash parts to go as low as 100 writes per physical block. Off by a couple of orders of magnitude.

Signature · 2 lines
Martin K. Petersen	Wild Open Source, Inc.
[email protected]	http://www.wildopensource.com/
#7 lore raw
In reply to Daniel Egger

In article <1044232591.545.8.camel@sonja>,

Quoted 3 lines · Daniel Egger

CF has limited write cycles. A few hundred if you're lucky. And depending on the type of flash it's quite likely that every changed byte will result in a whole block being written back.

Then why if I google for 'compact flash write cycles' all specs of flash cards I found say 100000 write cycles at least ?

Mike.

Signature · 2 lines
Anyone who is capable of getting themselves made President should
on no account be allowed to do the job -- Douglas Adams.
#8 lore raw
In reply to Pavel Machek

Use a file system that is designed for use on FLASH devices. In general FLASH devices are not very useable for systems that need to modify data on the disk often.

You may wish to look at the virtual memory file system available in the kernel if you have enough RAM. You would mount your CompactFlash device read only and have all updates go to the virtual memory filesystem. When you want to commit the changes, remount the CompactFlash read/write and save the changes then remount it read only.

You would be surprised how fast a million writes can happen on a disk.

- Bryan

Quoted 18 lines · Pavel Machek

Hi!

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

Anyone seen something similar? Are there some known-good compactflash-es?

Pavel

#9 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Bryan" == Bryan Andersen <[email protected]> writes:

Bryan> Use a file system that is designed for use on FLASH devices. Bryan> In general FLASH devices are not very useable for systems that Bryan> need to modify data on the disk often.

Just a heads up that flash (as in MTD) != Compact Flash.

For all practical purposes a Compact Flash is an IDE disk (Or a an IDE controller with a disk behind it. Depends how you wire it).

Internally, the Compact Flash does all the wear averaging that a flash filesystem like jffs2 provides. Transparently to the application.

All Linux sees is a linear stream of bytes. Just like a disk. And it should be treated as such.

Bryan> You may wish to look at the virtual memory file system Bryan> available in the kernel if you have enough RAM. You would Bryan> mount your CompactFlash device read only and have all updates Bryan> go to the virtual memory filesystem. When you want to commit Bryan> the changes, remount the CompactFlash read/write and save the Bryan> changes then remount it read only.

tmpfs to the rescue!

Bryan> You would be surprised how fast a million writes can happen on Bryan> a disk.

Yup.

You would also be surprised how long it takes to actually *perform* a million writes on a slow piece of media like Compact Flash ;)

Signature · 2 lines
Martin K. Petersen	Wild Open Source, Inc.
[email protected]	http://www.wildopensource.com/
#10 lore raw
Quoted 2 lines

Bryan> You would be surprised how fast a million writes can happen on Bryan> a disk.

especially if you don't mount with noatime.
I mounted my filesystems ro in conjunction with tmpfs and never had problems with CF.

Pádraig.

#11 lore raw
Quoted 4 lines
Quoted 2 lines

Bryan> You would be surprised how fast a million writes can happen on Bryan> a disk.

especially if you don't mount with noatime.

noatime only suppresses access time updating on reads, writes still update the access time.

John

#12 lore raw
In reply to Martin K. Petersen

[snip discussion about Compact Flash cards]

On a somewhat-related subject, is there currently an easy way to use a PROM as a read-only filesystem?

I.E. I'd like to write a raw filesystem image to an PROM using a PROM burner then connect that, probably to the parallel port, and use it as a block device.

It should be fairly simple to build the parallel port -> PROM adaptor, as it would essentially just be a ZIF socket, and a whole load of latches to multiplex the limited number of I/O lines to the 32 or so needed for address and data, and the driver should be straightforward to write as well.

Or is there a reason why this hasn't been done? PROMs are much cheaper than Compact Flash...

John.

#13 lore raw

Take a look at all the embedded Linux distributions. There are solutions for this out there. I haven't looked into them as I just stuck a hole PC into my robot.

The issue with ROM style data storage is once written, any changes require replacement of the devices. That adds up after a few revisions.
All motherboards once had PROMS (usually windowless EPROMS) for
storing the BIOS. It is just the cost of FLASH type devices has come down to the point where they are cheep when you look at the costs of going about it some other way.

- Bryan

Quoted 15 lines

On a somewhat-related subject, is there currently an easy way to use a PROM as a read-only filesystem?

I.E. I'd like to write a raw filesystem image to an PROM using a PROM burner then connect that, probably to the parallel port, and use it as a block device.

It should be fairly simple to build the parallel port -> PROM adaptor, as it would essentially just be a ZIF socket, and a whole load of latches to multiplex the limited number of I/O lines to the 32 or so needed for address and data, and the driver should be straightforward to write as well.

Or is there a reason why this hasn't been done? PROMs are much cheaper than Compact Flash...

#14 lore raw
In reply to John Bradford
Quoted 4 lines · John Bradford

[snip discussion about Compact Flash cards]

Or is there a reason why this hasn't been done? PROMs are much cheaper than Compact Flash...

And zero cameras, printers, PCMCIA cards and USB adaptors for them.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#15 lore raw
Quoted 6 lines
Quoted 4 lines

[snip discussion about Compact Flash cards]

Or is there a reason why this hasn't been done? PROMs are much cheaper than Compact Flash...

And zero cameras, printers, PCMCIA cards and USB adaptors for them.

So? Maybe I just have fond memories of my Sinclair Z88 which was exclusively solid state, using cheap, (well, they weren't so cheap 15 years ago), EPROMs, that I never lost a single bit to, and that are still readable today.

John.

#16 lore raw
In reply to Pavel Machek
Quoted 4 lines

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

last year we had some problems with compactflash. It was only 7bit clean, not 8bit. However the cards worked fine via usb devices, but not when used as IDE device.

test if writing 7bit data is reliable :-)

Andreas

#17 lore raw

Hi!

Quoted 10 lines
Quoted 4 lines

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

last year we had some problems with compactflash. It was only 7bit clean, not 8bit. However the cards worked fine via usb devices, but not when used as IDE device.

test if writing 7bit data is reliable :-)

Ouch... I can't imageine how even VFAT would work on something not 8-bit-clean. I doubt FAT only uses 7bits...

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#18 lore raw
Quoted 2 lines · Pavel Machek

Ouch... I can't imageine how even VFAT would work on something not 8-bit-clean. I doubt FAT only uses 7bits...

So use a pseudo-file-system which lets you store 8-bit data on such a device storing only 7-bit data.

It's no less sensible than taking a real flash device and hacking up a pseudo-file-system which makes it pretend to be a block device, then using a 'normal' file system on top of that.

It's just a shame that CF doesn't generally give you real access to the underlying flash and let you use a real file system designed for the purpose rather than its silly 'translation layer' :)

Signature · 1 line
dwmw2
#19 lore raw
Followup to:  <[email protected]>
By author:    David Woodhouse <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 4 lines

So use a pseudo-file-system which lets you store 8-bit data on such a device storing only 7-bit data.

Quoted 8 lines

It's no less sensible than taking a real flash device and hacking up a pseudo-file-system which makes it pretend to be a block device, then using a 'normal' file system on top of that.

It's just a shame that CF doesn't generally give you real access to the underlying flash and let you use a real file system designed for the purpose rather than its silly 'translation layer' :)

Well, it also let them implement the translation layer using a different storage device, which may be more appropriate for the task, or by playing other hardware-specific games.

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: cris ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
#20 lore raw
In reply to David Woodhouse

Hi!

Quoted 6 lines
Quoted 2 lines

Ouch... I can't imageine how even VFAT would work on something not 8-bit-clean. I doubt FAT only uses 7bits...

So use a pseudo-file-system which lets you store 8-bit data on such a device storing only 7-bit data.

Quoted 7 lines

It's no less sensible than taking a real flash device and hacking up a pseudo-file-system which makes it pretend to be a block device, then using a 'normal' file system on top of that.

It's just a shame that CF doesn't generally give you real access to the underlying flash and let you use a real file system designed for the purpose rather than its silly 'translation layer' :)

Well, if their translation layer at least *worked*, I'd be happy with it.

Unfortunately one error per 50MB is not acceptable...

Pavel

Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.
#21 lore raw
Quoted 2 lines · Pavel Machek

Well, if their translation layer at least *worked*, I'd be happy with it.

Would you? You fill up your FAT or EXT2 file system, then delete all your files. There are lots and lots of sectors with now-unused data.

Then start filling it up again.

To accommodate your writes, the underlying translation layer is busily garbage-collecting all those blocks which are _unused_, copying them from one part of the flash to another to collect 'fresh' copies of data together while reclaiming space from 'obsoleted' copies of changed sectors.

Or you manage to find a vendor who sells reliable cards, hence decide it's actually usable for real medium-term storage and start using EXT3 on it. You write out all your metadata (or even all your data) twice to the device, and it hits the flash twice, because you can't tie in to the _underlying_ journalling which it's already doing.

I wouldn't be happy.

Signature · 1 line
dwmw2
#22 lore raw

Hi!

Quoted 14 lines
Quoted 2 lines

Well, if their translation layer at least *worked*, I'd be happy with it.

Would you? You fill up your FAT or EXT2 file system, then delete all your files. There are lots and lots of sectors with now-unused data.

Then start filling it up again.

To accommodate your writes, the underlying translation layer is busily garbage-collecting all those blocks which are _unused_, copying them from one part of the flash to another to collect 'fresh' copies of data together while reclaiming space from 'obsoleted' copies of changed sectors.

Quoted 3 lines

Or you manage to find a vendor who sells reliable cards, hence decide it's actually usable for real medium-term storage and start using EXT3

Well, I've got old 20-mb PCMCIA, and that worked for me for >2 years. Now I've apacer 256MB CF, and it died within a *month*. I returned it and the "new" one died within *week*. Ouch.

Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#23 lore raw
In reply to Pavel Machek
Quoted 7 lines · Pavel Machek

Hi!

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

Quoted 6 lines

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

That is very bad... I wonder if you do something that the CF does not like - like power off while writing (can actually destroy the disk - read in some newsgroup)

Quoted 4 lines

Anyone seen something similar? Are there some known-good compactflash-es?

I would recomend SanDisk

/RogerL

Signature · 3 lines
Roger Larsson
Skellefteå
Sweden
#24 lore raw
Quoted 9 lines · Roger Larsson
Quoted 4 lines · Pavel Machek

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

That is very bad... I wonder if you do something that the CF does not like - like power off while writing (can actually destroy the disk - read in some newsgroup)

I am led to believe that this is normal for CompactFlash. I seem to recall that the phrase 'bogroll technology' was used by the last person to attempt any serious testing.

It's fine for short-term storage of stuff like digital photos, but don't try using it for long-term storage. Think of it like a floppy disc -- if you have any _important_ data, make sure it's backed up or copied onto at least three of them to avoid loss.

Not that there's any _fundamental_ reason why PCMCIA-ATA should be completely unreliable, that just seems to be the way it's done. Reliability costs, you see.

Signature · 1 line
dwmw2
#25 lore raw

Hi!

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

That is very bad... I wonder if you do something that the CF does not like - like power off while writing (can actually destroy the disk - read in some newsgroup)

Quoted 8 lines

I am led to believe that this is normal for CompactFlash. I seem to recall that the phrase 'bogroll technology' was used by the last person to attempt any serious testing.

It's fine for short-term storage of stuff like digital photos, but don't try using it for long-term storage. Think of it like a floppy disc -- if you have any _important_ data, make sure it's backed up or copied onto at least three of them to avoid loss.

Well, this seems *way* more crappier than floppies. Floppies will not fail within five minutes if you only start using them :-(.
Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?
#26 lore raw
In reply to Roger Larsson
Quoted 11 lines · --On Tuesday, February 04, 2003 00:56:00 +0100 Roger Larsson…
Quoted 10 lines · Pavel Machek

Hi!

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

Quoted 7 lines
Quoted 3 lines

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

That is very bad... I wonder if you do something that the CF does not like - like power off while writing (can actually destroy the disk - read in some newsgroup)

Maybe your system is feeding it the wrong supply voltage?

Quoted 7 lines
Quoted 5 lines

Anyone seen something similar? Are there some known-good compactflash-es?

I would recomend SanDisk

Seconded, they're generally really good.

Andrew

#27 lore raw
In reply to Roger Larsson

Hi!

Quoted 8 lines
Quoted 8 lines

I had compactflash from Apacer (256MB), and it started corrupting data in few months, eventually becoming useless and being given back for repair. They gave me another one and it is just starting to corrupt data.

First time I repartitioned it; now I only did mke2fs, and data corruption can be seen by something as simple as

Quoted 7 lines
Quoted 3 lines

cat /mnt/cf/mp3/* > /mnt/cf/delme; md5sum /mnt/cf/delme.

[Fails 1 in 5 tries].

That is very bad... I wonder if you do something that the CF does not like - like power off while writing (can actually destroy the disk - read in some newsgroup)

I don't think I did anything bad :-(. That "1 in 5 tries bad" was on the running system, with no reboots, powerdowns, etc.

It is possible that zaurus went into powersave mode, but I've certainly seen corruption without powersave, too.
Pavel

Signature · 2 lines
Worst form of spam? Adding advertisment signatures ala sourceforge.net.
What goes next? Inserting advertisment *into* email?