Skip to contentSkip to conversation

raw aio write guarantee

DISCUSSION [email protected] 3 participants 6 messages Sep 28, 2005

lore
Share & tools
lore
#0 lore raw

Hi all,

Just a quick question: How can a user-mode application ensure that an AIO write on a raw block device (i.e. open()ed with O_DIRECT) has really -really- been written to the disk and not residing in an on-disk write cache where it could be lost in case of a power failure? Is physical write guaranteed to have taken place before the AIO completion? Or is there a system call that sends a cache flush command to the drive (and waits for the drive to commit)?

#1 lore raw
Quoted 4 lines · On Wed, 28 Sep 2005 00:12:33 PDT, [email protected] said:

Just a quick question: How can a user-mode application ensure that an AIO write on a raw block device (i.e. open()ed with O_DIRECT) has really -really- been written to the disk and not residing in an on-disk write cache where it could be lost in case of a power failure?

Step 1: Make sure you buy disk drives that don't lie through their teeth regarding details such as "is the write cache enabled?" or "did the flush cache work?". Such hardware can generate vast amounts of bad karma....

Step 2: Buy a UPS. A good one. You can't lose the cache during a power failure that doesn't actually hit.

Step 3: http://catb.org/~esr/jargon/html/M/molly-guard.html - You want these. Lots of them.

#2 lore raw
Quoted 11 lines · On 9/28/05, [email protected] <[email protected]> wrote:
Quoted 4 lines · On Wed, 28 Sep 2005 00:12:33 PDT, [email protected] said:

Just a quick question: How can a user-mode application ensure that an AIO write on a raw block device (i.e. open()ed with O_DIRECT) has really -really- been written to the disk and not residing in an on-disk write cache where it could be lost in case of a power failure?

Step 1: Make sure you buy disk drives that don't lie through their teeth regarding details such as "is the write cache enabled?" or "did the flush cache work?". Such hardware can generate vast amounts of bad karma....

Quoted 6 lines

Step 2: Buy a UPS. A good one. You can't lose the cache during a power failure that doesn't actually hit.

Step 3: http://catb.org/~esr/jargon/html/M/molly-guard.html - You want these. Lots of them.

I was asking what a user-application can do to prevent data loss, not an application-user.

#3 lore raw
Quoted 2 lines · On Wed, 28 Sep 2005 01:56:05 PDT, [email protected] said:

I was asking what a user-application can do to prevent data loss, not an application-user.

Right. However, if you actually care about the distinction between "made it to the disk cache" and "made it to the platter", those are things you'll want to address - in particular, if you have one of the evil disk drives I mentioned, there's very little a user application can do to work around it.

#4 lore raw
Quoted 10 lines · On 9/28/05, [email protected] <[email protected]> wrote:
Quoted 2 lines · On Wed, 28 Sep 2005 01:56:05 PDT, [email protected] said:

I was asking what a user-application can do to prevent data loss, not an application-user.

Right. However, if you actually care about the distinction between "made it to the disk cache" and "made it to the platter", those are things you'll want to address - in particular, if you have one of the evil disk drives I mentioned, there's very little a user application can do to work around it.

Erm, is the hardware problem really as great as you're implying? Have you personally encountered any bad drives made by reputable brands? Mostly I've only heard only of people crying wolf and then realizing it was a problem with their reasoning or with the assumption that fsync() actually works properly on kernel X, where X doesn't even have to be that ancient a version of linux ;-)

#5 lore raw
Quoted 1 line · On Sep 28, 2005, at 12:42:50, [email protected] wrote:

Erm, is the hardware problem really as great as you're implying? Have you personally encountered any bad drives made by reputable brands? Mostly I've only heard only of people crying wolf and then realizing it was a problem with their reasoning or with the assumption that fsync() actually works properly on kernel X, where X doesn't even have to be that ancient a version of linux ;-)

No, I've seen several sample cases on this list of drives where the IDE or SCSI cache flush commands did not trigger any disk activity and the only way to force it out of the cache was to write several meg of garbage to some file. In some cases (Like RAID cards with good battery backup, for example), they may ignore the flush cache command as it isn't really useful (although on such good cards, they usually haw a way to turn it off, too).

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r !y?(-) ------END GEEK CODE BLOCK------