Question about spun-down USB disk

Hello all!

I am using an Asus EEE Box with Fedora 9 as a NAS.
I have found a way to spin-down my USB harddisks with the sg3 utils,
but the system occasionally times out on write actions to the disk.
I usually access the disk through SMB of NFS, which forces the disk to spin up first.
Sometimes the system opens the disk by itself, throwing an error.
lost page write due to I/O error on sdf1

Is there any way to increase the system timeout when accessing/writing-to the drive?
Or should I go ask the ext3 people?
Or does sg_start not notify the kernel the device has been stopped?

When the system fails, I get the following error:

Nov  4 23:50:42 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:42 EEE-Box kernel: end_request: I/O error, dev sdf, sector
976767873
Nov  4 23:50:42 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 122095984
Nov  4 23:50:42 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:42 EEE-Box kernel: end_request: I/O error, dev sdf, sector
976767873
Nov  4 23:50:42 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 122095984
Nov  4 23:50:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:52 EEE-Box kernel: end_request: I/O error, dev sdf, sector 4361
Nov  4 23:50:52 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 545
Nov  4 23:50:52 EEE-Box kernel: EXT3-fs error (device sdf1): ext3_readdir:
directory #2 contains a hole at offset 0
Nov  4 23:50:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:52 EEE-Box kernel: end_request: I/O error, dev sdf, sector 1
Nov  4 23:50:52 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 0
Nov  4 23:50:52 EEE-Box kernel: lost page write due to I/O error on sdf1
Nov  4 23:50:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:52 EEE-Box kernel: end_request: I/O error, dev sdf, sector 265
Nov  4 23:50:52 EEE-Box kernel: EXT3-fs error (device sdf1):
ext3_get_inode_loc: unable to read inode block - inode=2, block=33
Nov  4 23:50:52 EEE-Box kernel: ------------[ cut here ]------------
Nov  4 23:50:52 EEE-Box kernel: WARNING: at fs/buffer.c:1183
mark_buffer_dirty+0x23/0x6a() (Not tainted)
Nov  4 23:50:52 EEE-Box kernel: Modules linked in: joydev cpufreq_stats

xt_tcpudp iptable_filter ip_tables x_tables nfsd lockd nfs_acl auth_rpcgss exportfs bridge bnep rfcomm l2cap bluetooth fuse sunrpc ipv6 loop dm_multipath snd_hda_intel snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device video snd_pcm_oss output snd_mixer_oss snd_pcm i915 snd_timer button snd_page_alloc drm snd_hwdep snd i2c_algo_bit i2c_i801 r8169 pcspkr i2c_core soundcore serio_raw iTCO_wdt iTCO_vendor_support usb_storage sg dm_snapshot dm_zero dm_mirror dm_mod ata_piix pata_acpi ata_generic libata sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]

Nov  4 23:50:52 EEE-Box kernel: Pid: 1760, comm: ls Not tainted
2.6.25-14.fc9.i686 #1
Nov  4 23:50:52 EEE-Box kernel: [<c0427911>] warn_on_slowpath+0x47/0x73
Nov  4 23:50:52 EEE-Box kernel: [<c062d850>] ?
atomic_notifier_call_chain+0xf/0x11
Nov  4 23:50:52 EEE-Box kernel: [<c054aed9>] ? vt_console_print+0x277/0x286
Nov  4 23:50:52 EEE-Box kernel: [<c054ac62>] ? vt_console_print+0x0/0x286
Nov  4 23:50:52 EEE-Box kernel: [<c042799d>] ?
__call_console_drivers+0x56/0x63
Nov  4 23:50:52 EEE-Box kernel: [<c0427a01>] ?
_call_console_drivers+0x57/0x5b
Nov  4 23:50:52 EEE-Box kernel: [<c0427dd5>] ?
release_console_sem+0x195/0x19d
Nov  4 23:50:52 EEE-Box kernel: [<c049f38d>] mark_buffer_dirty+0x23/0x6a
Nov  4 23:50:52 EEE-Box kernel: [<f88ebc36>] ext3_commit_super+0x40/0x53
[ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88ed076>] ext3_handle_error+0x71/0x95
[ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88ed129>] ext3_error+0x39/0x43 [ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88e5598>]
__ext3_get_inode_loc+0x2b1/0x2d8 [ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88e58ed>] ext3_get_inode_loc+0x14/0x16
[ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88e590f>]
ext3_reserve_inode_write+0x20/0x68 [ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88e5986>] ext3_mark_inode_dirty+0x2f/0x46
[ext3]
Nov  4 23:50:52 EEE-Box kernel: [<f88e5aa8>] ext3_dirty_inode+0x53/0x67
[ext3]
Nov  4 23:50:52 EEE-Box kernel: [<c049b785>] __mark_inode_dirty+0x29/0x13c
Nov  4 23:50:52 EEE-Box kernel: [<c04931af>] touch_atime+0xbb/0xbf
Nov  4 23:50:52 EEE-Box kernel: [<c048d74a>] vfs_readdir+0x76/0x8f
Nov  4 23:50:52 EEE-Box kernel: [<c048d490>] ? filldir64+0x0/0xcd
Nov  4 23:50:52 EEE-Box kernel: [<c048d7c6>] sys_getdents64+0x63/0xa1
Nov  4 23:50:52 EEE-Box kernel: [<c0405bf2>] syscall_call+0x7/0xb
Nov  4 23:50:52 EEE-Box kernel: =======================
Nov  4 23:50:52 EEE-Box kernel: ---[ end trace 1a0b4166f7029756 ]---
Nov  4 23:50:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:52 EEE-Box kernel: end_request: I/O error, dev sdf, sector 1
Nov  4 23:50:52 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 0
Nov  4 23:50:52 EEE-Box kernel: lost page write due to I/O error on sdf1
Nov  4 23:50:52 EEE-Box kernel: EXT3-fs error (device sdf1) in
ext3_reserve_inode_write: IO failure
Nov  4 23:50:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:52 EEE-Box kernel: end_request: I/O error, dev sdf, sector 1
Nov  4 23:50:52 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 0
Nov  4 23:50:52 EEE-Box kernel: lost page write due to I/O error on sdf1
Nov  4 23:50:58 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:58 EEE-Box kernel: end_request: I/O error, dev sdf, sector 4401
Nov  4 23:50:58 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 550
Nov  4 23:50:58 EEE-Box kernel: lost page write due to I/O error on sdf1
Nov  4 23:50:58 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:58 EEE-Box kernel: end_request: I/O error, dev sdf, sector 4409
Nov  4 23:50:58 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 551
Nov  4 23:50:58 EEE-Box kernel: lost page write due to I/O error on sdf1
Nov  4 23:50:58 EEE-Box kernel: Aborting journal on device sdf1.
Nov  4 23:50:58 EEE-Box kernel: sd 6:0:0:0: [sdf] Result: hostbyte=DID_ERROR
driverbyte=DRIVER_OK,SUGGEST_OK
Nov  4 23:50:58 EEE-Box kernel: end_request: I/O error, dev sdf, sector 4401
Nov  4 23:50:58 EEE-Box kernel: Buffer I/O error on device sdf1, logical
block 550
Nov  4 23:50:58 EEE-Box kernel: lost page write due to I/O error on sdf1

My current kernel:
Linux EEE-Box 2.6.25-14.fc9.i686 #1 SMP Thu May 1 06:28:41 EDT 2008 i686 i686 i386 GNU/Linux

modinfo usb_storage
filename:
/lib/modules/2.6.25-14.fc9.i686/kernel/drivers/usb/storage/usb-storage.ko

license:        GPL
description:    USB Mass Storage driver for Linux
author:         Matthew Dharm <[email protected]>
srcversion:     38F8E61ECA6C16CA785AA38
--cut aliases--
depends:        scsi_mod
vermagic:       2.6.25-14.fc9.i686 SMP mod_unload 686 4KSTACKS
parm:           delay_use:seconds to delay before using a new device (uint)

modinfo scsi_mod
filename:
/lib/modules/2.6.25-14.fc9.i686/kernel/drivers/scsi/scsi_mod.ko

license:        GPL
description:    SCSI core
srcversion:     DF7CC1670A47C981FCDFA39
depends:
vermagic:       2.6.25-14.fc9.i686 SMP mod_unload 686 4KSTACKS
parm:           dev_flags:Given scsi_dev_flags=vendor:model:flags[,v:m:f]

add black/white list entries for vendor and model with an integer value of flags to the scsi device info list (string)
parm: default_dev_flags:scsi default device flag integer value
(int)
parm: max_luns:last scsi LUN (should be between 1 and 2^32-1)
(uint)

parm:           scan:sync, async or none (string)
parm:           max_report_luns:REPORT LUNS maximum number of LUNS received

(should be between 1 and 16384) (uint)
parm: inq_timeout:Timeout (in seconds) waiting for devices to
answer INQUIRY. Default is 5. Some non-compliant devices need more. (uint)
parm: scsi_logging_level:a bit mask of logging levels (int)

modinfo sd_mod
filename:
/lib/modules/2.6.25-14.fc9.i686/kernel/drivers/scsi/sd_mod.ko --cut aliases--

license:        GPL
description:    SCSI disk (sd) driver
author:         Eric Youngdale
srcversion:     ECC497BE26ACFBB19D2AF29
depends:        scsi_mod
vermagic:       2.6.25-14.fc9.i686 SMP mod_unload 686 4KSTACKS

Dmesg of the device:

Nov  5 00:12:52 EEE-Box kernel: scsi 6:0:0:0: Direct-Access     WD
5000AVV External 1.65 PQ: 0 ANSI: 4
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] 976773168 512-byte
hardware sectors (500108 MB)
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Write Protect is off
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Assuming drive cache:
write through
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] 976773168 512-byte
hardware sectors (500108 MB)
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Write Protect is off
Nov  5 00:12:52 EEE-Box kernel: sd 6:0:0:0: [sdf] Assuming drive cache:

write through

In response to the sg_start question, it does not notify the kernel that the device has been stopped. IMO stopping a disk containing a mounted file system is asking for trouble, however I don't think that it the role of such low level utilities to detect and prevent such things.
As for informing the kernel, if I discovered a way to do that, then I would need to investigate how that might be done on other OSes that sg3_utils is ported to (e.g. Windows). So I would prefer not to go down that path.

Doug Gilbert

You provide a valid point.
However, from an energy/environmental standpoint, I fail to see the problem. Don't harddisks in laptops generally spin down after there's been no activity for a while?
How does the O/S handle those?

These aren't server disks I'm talking about, but disks in a system designed to conserve power.
Why should the disk in my home server be powered up when no-one is using the server?

Western Digital MyBooks and Seagate FreeAgent disks power themselves down automatically (firmware setting).
Does that mean you want people that have these disks to lose their data? How do you expect them to know -in advance- that their disk will spin down, possibly killing their filesystem?

People are alresdy reporting these problems:
http://www.nslu2-linux.org/wiki/FAQ/DealWithAutoSpinDownOnSeagateFreeAgent
http://www.mail-archive.com/[email protected]/msg52952.h
tml

I can understand Linux' UN*X roots imply big servers with SCSI disks that need to be highly available.
It would, however, be nice if the O/S could optionally manage disks the way 'a certain other O/S' does.

If everyone could save those 10 watts per disk with their little server at home, they might even save a few pennies.

- disclaimer-
For safety, I keeep my data on two separate disks and rsync these daily,
so I'm not particularly afraid of a filesystem or O/S crash.
9 times out of 10, I only read from the disk as all media mounts are
readonly and the disk is mounted with 'noatime'.
I would like to unmount the disks before spinning them down,
but I don't think NFS and SMB can auto-mount the disk. Providing the disk in question is automatically spun up...

Regards,

Kit Gerrits

-----Original Message-----
From: Douglas Gilbert [mailto:[email protected]]
Sent: zondag 9 november 2008 23:36
To: Kit Gerrits
Cc: [email protected]
Subject: Re: Question about spun-down USB disk

In response to the sg_start question, it does not notify the kernel that the device has been stopped. IMO stopping a disk containing a mounted file system is asking for trouble, however I don't think that it the role of such low level utilities to detect and prevent such things.
As for informing the kernel, if I discovered a way to do that, then I would need to investigate how that might be done on other OSes that sg3_utils is ported to (e.g. Windows). So I would prefer not to go down that path.

Doug Gilbert

--snip--

(Well, this has of course nothing to do with the sg_start utility.)

The second report (about Seagate Freeagent USB, FireWire, eSATA with Linux 2.6.21-rc7) doesn't have enough detail. The former report (about Seagate Freeagent USB?) suggests a seemingly successful workaround for Seagate Freeagent disks: Switch on the allow_restart flag via sysfs. In Linux 2.6.24 and later, this flag is already enabled by default for all USB HDDs, i.e. the user doesn't have to do anything anymore to activate this fix.

I guess the problem with /your/ disk could be that the kernel cannot easily tell from the type of error status which it gets from the firmware of the disk (of its USB chip, to be precise) that it should issue the START STOP UNIT command to get the disk going again.

About auto-spin-down: Auto-spin-down (and thus auto-spin-up) managed by the kernel for SCSI disks (as opt-in feature; and thus for USB disks too) has been in the works, but I don't know what the status of this work is.

About auto-spin-up: Note that many firmwares of cheap devices don't care a damn about standards; they just barely work with some luck with some versions of Windows if the Windows user doesn't do anything extraordinary with them. From what I understand, proper sense data from the disk would be enough for the kernel to send a START STOP UNIT to switch the motor on and to retry the last request. Somebody correct me if I'm wrong.

BTW, if you stopped the disk with sg_start --stop, can you always successfully start it again with sg_start --start (if you run it before any other accesses to this disk)?

Stefan Richter
-=====-==--- =-== -=-=-
http://arcgraph.de/sr/

The disk spins up automatically, but the O/S threw an error because the disk did not reply in time.
When I access the disks through SMB or NFS, everything is fime. If, for some reason, a program wants to write to a file on there without spinning the drive up in advance,
the error pops up.
Mind you, I haven't had the error in days now.

All I'm looking for is a way to increase the time the O/S waits for the disk to spin up.

Regards,

Kit

-----Original Message-----
From: Stefan Richter [mailto:[email protected]]
Sent: maandag 10 november 2008 18:01
To: Kit Gerrits
Cc: [email protected]; [email protected]
Subject: Re: Question about spun-down USB disk

(Well, this has of course nothing to do with the sg_start utility.)

The second report (about Seagate Freeagent USB, FireWire, eSATA with Linux 2.6.21-rc7) doesn't have enough detail. The former report (about Seagate Freeagent USB?) suggests a seemingly successful workaround for Seagate Freeagent disks: Switch on the allow_restart flag via sysfs.
In Linux 2.6.24 and later, this flag is already enabled by default for all USB HDDs, i.e. the user doesn't have to do anything anymore to activate this fix.

I guess the problem with /your/ disk could be that the kernel cannot easily tell from the type of error status which it gets from the firmware of the disk (of its USB chip, to be precise) that it should issue the START STOP UNIT command to get the disk going again.

About auto-spin-down: Auto-spin-down (and thus auto-spin-up) managed by the kernel for SCSI disks (as opt-in feature; and thus for USB disks too) has been in the works, but I don't know what the status of this work is.

About auto-spin-up: Note that many firmwares of cheap devices don't care a damn about standards; they just barely work with some luck with some versions of Windows if the Windows user doesn't do anything extraordinary with them. From what I understand, proper sense data from the disk would be enough for the kernel to send a START STOP UNIT to switch the motor on and to retry the last request. Somebody correct me if I'm wrong.

BTW, if you stopped the disk with sg_start --stop, can you always successfully start it again with sg_start --start (if you run it before any other accesses to this disk)?
--
Stefan Richter
-=====-==--- =-== -=-=-
http://arcgraph.de/sr/

Kit Gerrits wrote:
...

Try the /sys/bus/scsi/devices/*:*:*:*/timeout attribute.

Of course increasing the timeout isn't a particularly sophisticated method to solve the issue, but simple enough to try. The default timeout is 30 seconds: linux/drivers/scsi/sd.h::SD_TIMEOUT. Can an HDD really take longer than that to receive a request when spun down, spin up, execute the request, and return status?

Stefan Richter
-=====-==--- =-== -=-==
http://arcgraph.de/sr/

OK, that is odd.
The timer on all my disks report 60 (I assume those are seconds).

I'll keep an eye out for that error and see if I can find out where it came from.
If it is not the spin-up time, I must have accidentally pulled a (USB) cable.
(From what I can see in the logs, I just plugged in a different drive before the drive failed)

I thank you very much for the hint, it was exactly what I was looking for,
and I apologoze for my <rant> from earlier on.
(I assumed it was set to 1 or 5 seconds).

Regards,

Kit Gerrits

-----Original Message-----
From: Stefan Richter [mailto:[email protected]]
Sent: dinsdag 11 november 2008 19:28
To: Kit Gerrits
Cc: [email protected]; [email protected]
Subject: Re: Question about spun-down USB disk

Kit Gerrits wrote:
...

Try the /sys/bus/scsi/devices/*:*:*:*/timeout attribute.

Of course increasing the timeout isn't a particularly sophisticated method to solve the issue, but simple enough to try. The default timeout is 30
seconds: linux/drivers/scsi/sd.h::SD_TIMEOUT. Can an HDD really take longer than that to receive a request when spun down, spin up, execute the request, and return status?
--
Stefan Richter
-=====-==--- =-== -=-==
http://arcgraph.de/sr/