Skip to contentSkip to conversation

scsi vs ide performance on fsync's

DISCUSSION David Balazic 4 participants 4 messages Mar 6, 2001

lore
Share & tools
lore
#0 lore raw

(( please CC me , not subscribed , [email protected] )

Quoted 6 lines · Jonathan Morton

The OS needs to know the physical act of writing data has finished before it tells the m/board to cut the power - period. Pathological data sets included - they are the worst case which every engineer must take into account. Out of interest, does Linux guarantee this, in the light of what we've uncovered? If so, perhaps it could use the same technique to fix fdatasync() and family...

Linux currently ignores write-cache, AFAICT.
Recently I asked a similar question , about flushing drive caches at shutdown :
Subject : "Flusing caches on shutdown"
message archived at :
http://boudicca.tux.org/hypermail/linux-kernel/2001week08/0157.html
Body attached at end of this message.

The answer ( and only reply ) was :
[ archived at : http://boudicca.tux.org/hypermail/linux-kernel/2001week08/0211.html ] --- begin quote ---
From: Ingo Oeser ([email protected])

Quoted 2 lines · David Balazic

It is a good idea IMO to flush the write cache of storage devices at shutdown and other critical moments.

Not needed. All device drivers should disable write caches of their devices, that need another signal than switching it off by the power button to flush themselves.

Quoted 4 lines

Loosing data at powerdown due to write caches have been reported, so this is no a theoretical problems. Also the journaled filesystems are safe only in theory if the journal is not stored on non-volatile memory, which is not guarantied in the current kernel.

Fine. If users/admins have write caching enabled, they either know what they do, or should disable it (which is the default for all mass storage drivers AFAIK).

Hardware Level caching is only good for OSes which have broken drivers and broken caching (like plain old DOS).

Linux does a good job in caching and cache control at software level.

Regards

Ingo Oeser
--- end quote ---

My original mail :
--- begin quote ---
(( CC me the replies, as I'm not subscribed to LKML ))

Hi!

It is a good idea IMO to flush the write cache of storage devices at shutdown and other critical moments.
I browsed through linux-2.4.1 and see no use of the SYNCHRONIZE CACHE SCSI command ( curiously it is defined in several other files besides include/scsi/scsi.h , grep returns :
drivers/scsi/pci2000.h:#define SCSIOP_SYNCHRONIZE_CACHE 0x35 drivers/scsi/psi_dale.h:#define SCSIOP_SYNCHRONIZE_CACHE 0x35 drivers/scsi/psi240i.h:#define SCSIOP_SYNCHRONIZE_CACHE 0x35 )

I couldn't find evidence to the use of the equivalent ATA command either ( FLUSH CACHE , command code E7h ).
Also add ATAPI to the list. ( and all other interfaces. I checked just SCSI and ATA )

Loosing data at powerdown due to write caches have been reported, so this is no a theoretical problems. Also the journaled filesystems are safe only in theory if the journal is not stored on non-volatile memory, which is not guarantied in the current kernel.

What is the official word on this issue ?
I think this is important to the "enterprise" guys, at the least.

Sincerely,
david

PS: CC me , as I'm not subscribed to LKML
--- end quote ---

Signature · 10 lines
David Balazic
--------------
"Be excellent to each other." - Bill & Ted
- - - - - - - - - - - - - - - - - - - - - -

-- 
David Balazic
--------------
"Be excellent to each other." - Bill & Ted
- - - - - - - - - - - - - - - - - - - - - -
#1 lore raw

On Tue, Mar 06, 2001 at 06:14:15PM +0100, David Balazic wrote:
[snip]

Quoted 5 lines

Hardware Level caching is only good for OSes which have broken drivers and broken caching (like plain old DOS).

Linux does a good job in caching and cache control at software level.

Read caching, yes. But for writes, the drive can often do a lot more optimization because of it's synchronous operation with the platter and greater knowledge of internal disk geometry.

What would be useful, as Alan said, is a barrier operation.

#2 lore raw
In reply to David Balazic
Quoted 14 lines
Quoted 7 lines · Jonathan Morton

The OS needs to know the physical act of writing data has finished before
it tells the m/board to cut the power - period. Pathological data sets included - they are the worst case which every engineer must take into account. Out of interest, does Linux guarantee this, in the light of what we've uncovered? If so, perhaps it could use the same technique to fix fdatasync() and family...

Linux currently ignores write-cache, AFAICT.
Recently I asked a similar question , about flushing drive caches at shutdown :

Quoted 8 lines
Quoted 2 lines · David Balazic

It is a good idea IMO to flush the write cache of storage devices at shutdown and other critical moments.

Not needed. All device drivers should disable write caches of their devices, that need another signal than switching it off by the power button to flush themselves.

Sounds like a sensible place to implement it - in the device driver. I also note the existence of an ATA flush-buffer command, which should probably be used in sync() and family. The call(s) to the sync() family on shutdown should probably be performed by the filesystem itself on unmount (or remount read-only), and if journalled filesystems need synchronisation they should use sync() (or a more fine-grained version) themselves as necessary.

Doesn't sound like too much of a headache to implement, to me - unless some drives ignore the ATA FLUSH command, in which case said drives can be considered seriously broken. :P I don't agree that write-caching in itself is a bad thing, particularly given the amount of CPU overhead that IDE drives demand while attached to the controller (orders of magnitude higher than a good SCSI controller) - the more overhead we can hand off to dedicated hardware, the better. What does matter is that drives implementing write-caching are handled in a safe and efficient manner, especially in cases where they refuse to turn such caching off (eg. my Seagate Barracuda *glares at drive*).

Recalling my recent comments on worst-case drive-shutdown timings, I also remember seeing drives with 18ms *average* seek times quite recently - this was a Quantum Bigfoot (yes, a 5.25" HD), found in a low-end Compaq desktop
- if anyone still believes Compaq makes high-quality machines for their low-end market, they're totally mistaken. The machine sped up quite a lot when a new 3.5" IBM DeskStar was installed, with an 8.5ms average seek and an almost doubling in rotational speed. :)

--------------------------------------------------------------

from:     Jonathan "Chromatix" Morton
mail:     [email protected]  (not for attachments)
big-mail: [email protected]
uni-mail: [email protected]

The key to knowledge is not to rely on people to teach you it.

Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/

-----BEGIN GEEK CODE BLOCK-----
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r- y+
-----END GEEK CODE BLOCK-----

#3 lore raw
Quoted 3 lines

itself is a bad thing, particularly given the amount of CPU overhead that IDE drives demand while attached to the controller (orders of magnitude higher than a good SCSI controller) - the more overhead we can hand off to

I know this is just a troll by a scsi-believer, but I'm biting anyway.

on current machines and disks, ide costs a few % CPU, depending on which CPU, disk, kernel, the sustained bandwidth, etc. I've measured this using the now-trendy method of noticing how much the IO costs a separate, CPU-bound benchmark: load = 1 - (unloadedPerf / loadedPerf). my cheesy duron/600 desktop typically shows ~2% actual cost when running bonnie's block IO tests.