SATA hotplug from the user side ?
Dear all,
I'd like to try out SATA hotplugging using a SIL3114. Though I was harvesting the web, I could not find any useful information how this is done in practice.
Well I realized that I can still use scsiadd to print and remove devices, e.g.:
# scsiadd -p
Attached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400832AS Rev: 3.01
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400620AS Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 05
# scsiadd -r 3 0 0 0
Is this all one has to do for hotplugging ? I am asking as I find this in dmesg when I do so (2.6.19.* kernel):
Synchronizing SCSI cache for disk sdb:
ata4.00: disabled
ata4: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: EH complete
ata4: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata4: hard resetting port
ata4: port is slow to respond, please be patient (Status 0xff)
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata4.00: ATA-7, max UDMA/133, 781422768 sectors: LBA48 NCQ (depth 0/32) ata4.00: configured for UDMA/100
ata4: EH complete
scsi 3:0:0:0: rejecting I/O to dead device
scsi 3:0:0:0: rejecting I/O to dead device
Soeren
Signature · 2 lines
For the one fact about the future of which we can be certain is that it will be utterly fantastic. -- Arthur C. Clarke, 1962
Quoted 8 lines · Soeren Sonnenburg
Dear all,
I'd like to try out SATA hotplugging using a SIL3114. Though I was harvesting the web, I could not find any useful information how this is done in practice.
Well I realized that I can still use scsiadd to print and remove devices, e.g.:
For SIL3114, you shouldn't have to run any commands at all. It should notice when you yank the cable, or plug in a new device.
Jeff
Quoted 7 lines · Jeff Garzik
Quoted 6 lines · Soeren Sonnenburg
Dear all,
I'd like to try out SATA hotplugging using a SIL3114. Though I was harvesting the web, I could not find any useful information how this is done in practice.
Quoted 5 lines
Quoted 2 lines
Well I realized that I can still use scsiadd to print and remove devices, e.g.:
For SIL3114, you shouldn't have to run any commands at all. It should notice when you yank the cable, or plug in a new device.
It is true it detects a removal and newly plugged devices immediately... However it still prints warnings and errors that it could not synchronize SCSI cache for the disks. Then it prints regular 'rejects I/O to dead device' warning messages and on replugging the disks puts them to the next free sd device (e.g. sdc -> sdd).
These messages sound eval - so now the question is should I care ? ( On the other hand it did not crash the machine )
What follows is a change between to sata drives attached to port 4/5 of the sil (ata5/ata6 here):
ata6: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata6: hard resetting port
ata6: SATA link down (SStatus 0 SControl 310)
ata6: failed to recover some devices, retrying in 5 secs
ata5: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata5: hard resetting port
ata5: SATA link down (SStatus 0 SControl 310)
ata5: failed to recover some devices, retrying in 5 secs
ata6: hard resetting port
ata6: SATA link down (SStatus 0 SControl 310)
ata6: failed to recover some devices, retrying in 5 secs
ata5: hard resetting port
ata5: SATA link down (SStatus 0 SControl 310)
ata5: failed to recover some devices, retrying in 5 secs
ata6: hard resetting port
ata6: SATA link down (SStatus 0 SControl 310)
ata6.00: disabled
ata6: EH complete
ata6.00: detaching (SCSI 5:0:0:0)
Synchronizing SCSI cache for disk sdd:
FAILED
status = 0, message = 00, host = 4, driver = 00
<6>ata5: hard resetting port
ata5: SATA link down (SStatus 0 SControl 310)
ata5.00: disabled
ata5: EH complete
ata5.00: detaching (SCSI 4:0:0:0)
Synchronizing SCSI cache for disk sdc:
FAILED
status = 0, message = 00, host = 4, driver = 00
<3>ata6: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata6: hard resetting port
ata6: port is slow to respond, please be patient (Status 0xff)
ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata6.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata6.00: configured for UDMA/100
ata6: EH complete
scsi 5:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sdf: 1465149168 512-byte hdwr sectors (750156 MB)
sdf: Write Protect is off
sdf: Mode Sense: 00 3a 00 00
SCSI device sdf: drive cache: write back
SCSI device sdf: 1465149168 512-byte hdwr sectors (750156 MB)
sdf: Write Protect is off
sdf: Mode Sense: 00 3a 00 00
SCSI device sdf: drive cache: write back
sdf: unknown partition table
sd 5:0:0:0: Attached scsi disk sdf
sd 5:0:0:0: Attached scsi generic sg2 type 0
scsi 4:0:0:0: rejecting I/O to dead device
scsi 4:0:0:0: rejecting I/O to dead device
scsi 5:0:0:0: rejecting I/O to dead device
scsi 5:0:0:0: rejecting I/O to dead device
ata5: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata5: hard resetting port
ata5: port is slow to respond, please be patient (Status 0xff)
ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata5.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata5.00: configured for UDMA/100
ata5: EH complete
scsi 4:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 00 3a 00 00
SCSI device sdg: drive cache: write back
SCSI device sdg: 1465149168 512-byte hdwr sectors (750156 MB)
sdg: Write Protect is off
sdg: Mode Sense: 00 3a 00 00
SCSI device sdg: drive cache: write back
sdg: unknown partition table
sd 4:0:0:0: Attached scsi disk sdg
sd 4:0:0:0: Attached scsi generic sg3 type 0
Best,
Soeren
Signature · 2 lines
For the one fact about the future of which we can be certain is that it will be utterly fantastic. -- Arthur C. Clarke, 1962
Quoted 5 lines · Soeren Sonnenburg
It is true it detects a removal and newly plugged devices immediately... However it still prints warnings and errors that it could not synchronize SCSI cache for the disks. Then it prints regular 'rejects I/O to dead device' warning messages and on replugging the disks puts them to the next free sd device (e.g. sdc -> sdd).
You need to stop using the devices before unplugging. If you have no pending IO to the device, there won't be 'rejects IO to dead device' messages. You can ignore the SCSI cache sync failure if the device is properly closed before being unplugged.
Quoted 2 lines
These messages sound eval - so now the question is should I care ? ( On the other hand it did not crash the machine )
So, no, you don't really have to care. Just make sure the device is unmounted prior to unplugging.
Signature · 1 line
tejun
Quoted 11 lines · Tejun Heo
Quoted 5 lines · Soeren Sonnenburg
It is true it detects a removal and newly plugged devices immediately... However it still prints warnings and errors that it could not synchronize SCSI cache for the disks. Then it prints regular 'rejects I/O to dead device' warning messages and on replugging the disks puts them to the next free sd device (e.g. sdc -> sdd).
You need to stop using the devices before unplugging. If you have no pending IO to the device, there won't be 'rejects IO to dead device' messages. You can ignore the SCSI cache sync failure if the device is properly closed before being unplugged.
Jeff & Tejun thanks *a lot* for clarifying this. I am quite happy to see that this is working very reliably!
Quoted 5 lines
Quoted 2 lines
These messages sound eval - so now the question is should I care ? ( On the other hand it did not crash the machine )
So, no, you don't really have to care. Just make sure the device is unmounted prior to unplugging.
OK, but then this really should be in the SATA hotplug FAQ (or can one fix this somehow?)... No user will ignore messages like this. What is especially annoying is that udev on the first remove/insert cycle created a new device node so the disk became /dev/sde (was /dev/sdd):
dmesg output of reinserting the disk 2 times follows:
<remove /dev/sdd>
ata4: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4.00: disabled
ata4: EH complete
ata4.00: detaching (SCSI 3:0:0:0)
Synchronizing SCSI cache for disk sdd:
FAILED
status = 0, message = 00, host = 4, driver = 00
<3>ata4: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
<insert>
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata4.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata4.00: configured for UDMA/100
ata4: EH complete
scsi 3:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
sde: unknown partition table
sd 3:0:0:0: Attached scsi disk sde
sd 3:0:0:0: Attached scsi generic sg3 type 0
<remove now /dev/sde>
ata4: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4.00: disabled
ata4: EH complete
ata4.00: detaching (SCSI 3:0:0:0)
Synchronizing SCSI cache for disk sde:
FAILED
status = 0, message = 00, host = 4, driver = 00
<3>ata4: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
<insert>
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata4.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata4.00: configured for UDMA/100
ata4: EH complete
scsi 3:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
sde: unknown partition table
sd 3:0:0:0: Attached scsi disk sde
sd 3:0:0:0: Attached scsi generic sg3 type 0
remains /dev/sde ...
Soeren
Signature · 2 lines
Sometimes, there's a moment as you're waking, when you become aware of the real world around you, but you're still dreaming.
Quoted 2 lines · Soeren Sonnenburg
Jeff & Tejun thanks *a lot* for clarifying this. I am quite happy to see that this is working very reliably!
You're welcome. :-)
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
These messages sound eval - so now the question is should I care ? ( On the other hand it did not crash the machine )
So, no, you don't really have to care. Just make sure the device is unmounted prior to unplugging.
OK, but then this really should be in the SATA hotplug FAQ (or can one fix this somehow?)... No user will ignore messages like this. What is
Yeah, probably. Care to submit patch to FAQ to Jeff?
Quoted 3 lines
especially annoying is that udev on the first remove/insert cycle created a new device node so the disk became
/dev/sde(was/dev/sdd):
dmesg output of reinserting the disk 2 times follows:
That's because something is still holding onto /dev/sdd. The device remains dangled until all references to it are gone. /dev/sdd is still lingering when the drive is hotplugged; thus, it gets assigned the next device.
I think this is best solved by udev. Think of /dev/sdX as kernel internal name which may change from time to time. Mount devices with LABEL's and use udev-provided persistent names to access removable devices. Also, libata can be improved to tell udev that certain ports are external, I think.
Thanks.
Signature · 1 line
tejun
Quoted 8 lines · Tejun Heo
Quoted 2 lines · Soeren Sonnenburg
Jeff & Tejun thanks *a lot* for clarifying this. I am quite happy to see that this is working very reliably!
You're welcome. :-)
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
These messages sound eval - so now the question is should I care ? ( On the other hand it did not crash the machine )
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
So, no, you don't really have to care. Just make sure the device is unmounted prior to unplugging.
OK, but then this really should be in the SATA hotplug FAQ (or can one fix this somehow?)... No user will ignore messages like this. What is
Yeah, probably. Care to submit patch to FAQ to Jeff?
OK, how about this (please especially check the non SIL part):
SATA Hotplug from the User Side
- For SIL3114 and SIL3124 you don't have to run any commands at all. It should notice when you yank the cable, or plug in a new device. All you have to do is to stop using the devices before unplugging, e.g. unmount partitions on the disk or remove the disk from a dm setup). One can validate which disks are attached using ``scsiadd -p''. Note that the device name may change from e.g. /dev/sdd to /dev/sde on a remove/reinsert cycle (this can be fixed by using udev-provided persistent names). Also note that it is perfectly normal to see messages like this in dmesg:
<removing /dev/sdd>
ata4: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4.00: disabled
ata4: EH complete
ata4.00: detaching (SCSI 3:0:0:0)
Synchronizing SCSI cache for disk sdd:
FAILED
status = 0, message = 00, host = 4, driver = 00
<3>ata4: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
<reinserting the disk>
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata4.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata4.00: configured for UDMA/100
ata4: EH complete
scsi 3:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
sde: unknown partition table
sd 3:0:0:0: Attached scsi disk sde
sd 3:0:0:0: Attached scsi generic sg3 type 0
However if you happen to see messages like
scsi 3:0:0:0: rejecting I/O to dead device
scsi 4:0:0:0: rejecting I/O to dead device
scsi 5:0:0:0: rejecting I/O to dead device
you did not stop using the devices before unplugging (check that you unmounted all partitions, removed the disk from a raid array, dmsetup, cryptsetup). If you have no pending IO to the device, there won't be 'rejects IO to dead device' messages.
- For other chipsets one in addition to stop using the device before unplugging has to call scsiadd -r to fully remove it, e.g. in the following example the disk on scsi host 3 channel 0 id 0 lun 0 will be removed, then on reinserting a disk call scsiadd -s :
# scsiadd -p
Attached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400832AS Rev: 3.01
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400620AS Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 05
# scsiadd -r 3 0 0 0 # scsiadd -s
Soeren
Signature · 2 lines
Sometimes, there's a moment as you're waking, when you become aware of the real world around you, but you're still dreaming.
Quoted 9 lines · Soeren Sonnenburg
- For SIL3114 and SIL3124 you don't have to run any commands at all. It should notice when you yank the cable, or plug in a new device. All you have to do is to stop using the devices before unplugging, e.g. unmount partitions on the disk or remove the disk from a dm setup). One can validate which disks are attached using ``scsiadd -p''. Note that the device name may change from e.g.
/dev/sddto/dev/sdeon a remove/reinsert cycle (this can be fixed by using udev-provided persistent names). Also note that it is perfectly normal to see messages like this in dmesg:
Does SATA electrical conector keying let the disk firmware unload heads before the user manages to pull it out enough to sever power? If it does not, the drive will do an emergency head unload, which is not good and will likely reduce the drive's lifetime.
Using hdparm -Y before the unplug, or scsiadd -r (on a kernel that has Tejun's new patch to optionally issue an START_STOP_UNIT to the SCSI device enabled) is probably a good idea. Unless it is a shared SATA port (I don't know if such a thing exists yet) and another box is talking to the disk, etc.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 2 lines · Henrique de Moraes Holschuh
Does SATA electrical conector keying let the disk firmware unload heads before the user manages to pull it out enough to sever power?
I don't think so.
Quoted 2 lines
If it does not, the drive will do an emergency head unload, which is not good and will likely reduce the drive's lifetime.
Probably.
Quoted 5 lines
Using hdparm -Y before the unplug, or scsiadd -r (on a kernel that has Tejun's new patch to optionally issue an
START_STOP_UNITto the SCSI device enabled) is probably a good idea. Unless it is a shared SATA port (I don't know if such a thing exists yet) and another box is talking to the disk, etc.
Agreed. But it would be *much* better if all these can be taken care of by hald and its minions. Such that the user can just tell the system that the hdd is going to be removed and all these dirty tricks are done automagically.
Signature · 1 line
tejun
Quoted 5 lines · Tejun Heo
Quoted 2 lines · Henrique de Moraes Holschuh
Does SATA electrical conector keying let the disk firmware unload heads before the user manages to pull it out enough to sever power?
I don't think so.
Heh, thought as much. (Good) SCSI hotswap bays notice you pulled the disk release lever and issue an START_STOP_UNIT by themselves to the disk well before you have time to start pulling the disk out. I wonder if the SATA ones do (I kind of doubt that, SATA seems to attract the el-cheapo, el-crapo crowd of manufacturers).
Quoted 4 lines
Quoted 2 lines
If it does not, the drive will do an emergency head unload, which is not good and will likely reduce the drive's lifetime.
Probably.
So, that means it should be explained in the docs that you are to stop the disk first, if you can.
Quoted 10 lines
Quoted 5 lines
Using hdparm -Y before the unplug, or scsiadd -r (on a kernel that has Tejun's new patch to optionally issue an
START_STOP_UNITto the SCSI device enabled) is probably a good idea. Unless it is a shared SATA port (I don't know if such a thing exists yet) and another box is talking to the disk, etc.Agreed. But it would be *much* better if all these can be taken care of by hald and its minions. Such that the user can just tell the system that the hdd is going to be removed and all these dirty tricks are done automagically.
Even if hald does this automatically, it would still be a very good idea to document the proper sequence, IMO...
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 11 lines · Henrique de Moraes Holschuh
Quoted 4 lines · Tejun Heo
Quoted 2 lines · Henrique de Moraes Holschuh
Does SATA electrical conector keying let the disk firmware unload heads before the user manages to pull it out enough to sever power?
I don't think so.
Heh, thought as much. (Good) SCSI hotswap bays notice you pulled the disk release lever and issue an
START_STOP_UNITby themselves to the disk well before you have time to start pulling the disk out. I wonder if the SATA ones do (I kind of doubt that, SATA seems to attract the el-cheapo, el-crapo crowd of manufacturers).
ahci controller spec has places for such thing but I've never seen it actually implemented. I'm thinking about a CLI tool to list & control libata devices including hotplugging, NCQ and other stuff. Eventually, it would be really nice to give the user/admin easy gui/web/whatever tool to watch and control ATA devices.
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
If it does not, the drive will do an emergency head unload, which is not good and will likely reduce the drive's lifetime.
Probably.
So, that means it should be explained in the docs that you are to stop the disk first, if you can.
Yeap, agreed.
Quoted 2 lines
Even if hald does this automatically, it would still be a very good idea to document the proper sequence, IMO...
Agreed.
Signature · 1 line
tejun
Quoted 6 lines · Tejun Heo
Quoted 5 lines · Henrique de Moraes Holschuh
Quoted 4 lines · Tejun Heo
Quoted 2 lines · Henrique de Moraes Holschuh
Does SATA electrical conector keying let the disk firmware unload heads before the user manages to pull it out enough to sever power?
I don't think so.
[...]
Quoted 1 line
Agreed.
OK, so here comes the next revision, I hope it can be put in the docs/on the web page now:
SATA Hotplug from the User Side
In general, before unplugging a drive you must stop using it. This means unmounting all partitions, removing the disk from a potential raid array / device mapper / crypt setup. Also depending on your disk-bay it might be a good idea to power down the drive using hdparm -Y followed by a scsiadd -r.
BIG FAT WARNING: if your SATA bay does not do electrical connector keying to let the disk firmware unload heads before the user manages to pull it out the drive will do an emergency head unload, which is not good and will likely reduce the drive's lifetime.
• For SIL3114 and SIL3124, AHCI and the CK804 flavour of sata_nv you - in principle - don't have to run any commands at all. It should notice when you yank the cable, or plug in a new device. All you have to do is to stop using the devices before unplugging, e.g. unmount partitions on the disk or remove the disk from a dm setup). However it is recommended to power down the drive using hdparm -Y followed by a scsiadd -r as stated above. One can validate which disks are attached using ``scsiadd -p''. In the following example the disk on scsi host 3 channel 0 id 0 lun 0 will be removed:
# scsiadd -p
Attached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400832AS Rev: 3.01
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400620AS Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 05
# scsiadd -r 3 0 0 0
Note that the device name may change from e.g. /dev/sdd to /dev/sde on a remove/reinsert cycle (this can be fixed by using udev-provided persistent names). Also note that it is perfectly normal to see messages like this in dmesg:
<removing /dev/sdd>
ata4: exception Emask 0x10 SAct 0x0 SErr 0x10000 action 0x2 frozen
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4: failed to recover some devices, retrying in 5 secs
ata4: hard resetting port
ata4: SATA link down (SStatus 0 SControl 310)
ata4.00: disabled
ata4: EH complete
ata4.00: detaching (SCSI 3:0:0:0)
Synchronizing SCSI cache for disk sdd:
FAILED
status = 0, message = 00, host = 4, driver = 00
<3>ata4: exception Emask 0x10 SAct 0x0 SErr 0x50000 action 0x2 frozen
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
ata4: COMRESET failed (device not ready)
ata4: hardreset failed, retrying in 5 secs
ata4: hard resetting port
<reinserting the disk>
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310) ata4.00: ATA-7, max UDMA/133, 1465149168 sectors: LBA48 NCQ (depth 0/32) ata4.00: configured for UDMA/100
ata4: EH complete
scsi 3:0:0:0: Direct-Access ATA ST3750640AS 3.AA PQ: 0
ANSI: 5
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
SCSI device sde: 1465149168 512-byte hdwr sectors (750156 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
sde: unknown partition table
sd 3:0:0:0: Attached scsi disk sde
sd 3:0:0:0: Attached scsi generic sg3 type 0
However if you happen to see messages like
scsi 3:0:0:0: rejecting I/O to dead device
scsi 4:0:0:0: rejecting I/O to dead device
scsi 5:0:0:0: rejecting I/O to dead device
you did not stop using the devices before unplugging (check that you unmounted all partitions, removed the disk from a raid array, dmsetup, cryptsetup). If you have no pending IO to the device, there won't be 'rejects IO to dead device' messages.
• For other chipsets one in addition might have to run scsiadd -s on reinserting the disk.
• Note that it might not be such a good idea to do the above on drivers which don't implement the new EH yet. Those are as of January 2007:sata_nv, sata_promise (getting there) and sata_sx4.
Soeren (beeing stuck in a train which again seems to be stuck in a lot of snow somewhere in southern germany...)
Signature · 2 lines
For the one fact about the future of which we can be certain is that it will be utterly fantastic. -- Arthur C. Clarke, 1962
Quoted 2 lines · Soeren Sonnenburg
might be a good idea to power down the drive using hdparm -Y followed by a scsiadd -r.
Unfortunately, I don't think this will be the optimal way to do it for long. Unless I am mistaken, the above will soon have the potential to spin down the device, and then reset the device to wake it up again (through error handling, thus causing error messages in syslog) to send it a spin down command(!). It will work well right now, because scsiadd -r is unable to shutdown a device and just removes it.
In the future, you will need to do *either* one, not both, if you told the kernel to shutdown SCSI devices (which distros will likely enable by default, as it is the right setup for everyone but people with servers with disks attached to multi-initiator SCSI buses and it is required for suspend-to-disk to work).
It will be a mess to make sure everything in userspace is updated to not try to hdparm -y (or worse, hdparm -Y) devices that the kernel will shutdown by itself, but it will make things a lot better for the future.
Unless...
Tejun, is it feasible to teach libata to check the device power management state before issuing any of the sleep, head unload and spin-down commands?
libata would need to block its EH from resetting a channel for this check operation (we don't want to wake up sleeping devices to ask it if it is sleeping...) if it cannot easily check if an device is asleep before issuing
a command to it. Either that, or (for as long as there is no such a thing
as a multi-initiator libata device) just keep track of the device power management state by snooping all the commands sent to it.
The idea would be to do the "optimal" thing if one tries to sleep, unload heads or spin down an device that is already doing one of these things... That would make things MUCH easier on userspace.
Quoted 2 lines
BIG FAT WARNING: if your SATA bay does not do electrical connector keying to let the disk firmware unload heads before the user manages to
This either is in the SATA specification, or it is not, and according to Tejun it probably isn't (if it were, all disks would do the right thing at unplug). I'd reword it to "if your SATA bay does not take extra steps to force the disk firmware to unload heads...".
Quoted 3 lines
the disk or remove the disk from a dm setup). However it is recommended to power down the drive using hdparm -Y followed by a scsiadd -r as stated above. One can validate which disks are attached using ``scsiadd
Again, this might change soon.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 3 lines · Henrique de Moraes Holschuh
Quoted 2 lines · Soeren Sonnenburg
might be a good idea to power down the drive using hdparm -Y followed by a scsiadd -r.
[...]
Quoted 5 lines
Quoted 3 lines
the disk or remove the disk from a dm setup). However it is recommended to power down the drive using hdparm -Y followed by a scsiadd -r as stated above. One can validate which disks are attached using ``scsiadd
Again, this might change soon.
Ok, I think someone more knowledgable than me in this area should do the final polishing und put it in the docs.
I don't anymore see how/that I can help here.
Soeren
Signature · 2 lines
Sometimes, there's a moment as you're waking, when you become aware of the real world around you, but you're still dreaming.
Quoted 9 lines · Soeren Sonnenburg
Quoted 3 lines · Henrique de Moraes Holschuh
Quoted 2 lines · Soeren Sonnenburg
might be a good idea to power down the drive using hdparm -Y followed by a scsiadd -r.
[...]
Quoted 4 lines
Quoted 3 lines
the disk or remove the disk from a dm setup). However it is recommended to power down the drive using hdparm -Y followed by a scsiadd -r as stated above. One can validate which disks are attached using ``scsiadd
Quoted 6 lines
Quoted 1 line
Again, this might change soon.
Ok, I think someone more knowledgable than me in this area should do the final polishing und put it in the docs.
I don't anymore see how/that I can help here.
You did just fine, and in fact your text is 100% correct for 2.6.20 (so it can be taken asis for 2.6.20, and ammended later when the scsi shutdown patch gets accepted).
So please don't fell discouraged. The text is quite good, and if you had not done it, it wouldn't have been improved anytime soon.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 9 lines · Henrique de Moraes Holschuh
Tejun, is it feasible to teach libata to check the device power management state before issuing any of the sleep, head unload and spin-down commands?
libata would need to block its EH from resetting a channel for this check operation (we don't want to wake up sleeping devices to ask it if it is sleeping...) if it cannot easily check if an device is asleep before issuing
a command to it. Either that, or (for as long as there is no such a thing
as a multi-initiator libata device) just keep track of the device power management state by snooping all the commands sent to it.
In the ATA spec, the CHECK POWER MODE command (E5h) specifically states that it shouldn't alter the power mode of the device. Therefore, there should be no harm in checking the power mode of a device in STANDBY or IDLE state, and it shouldn't cause the device to spin up. Checking a drive on my bench I confirmed this behavior.
With Serial ATA you have to consider the SATA bus PM states as well, but that's a different topic of discussion.
--eric
Quoted 5 lines · Eric D. Mudama
In the ATA spec, the CHECK POWER MODE command (E5h) specifically states that it shouldn't alter the power mode of the device. Therefore, there should be no harm in checking the power mode of a device in STANDBY or IDLE state, and it shouldn't cause the device to spin up. Checking a drive on my bench I confirmed this behavior.
Yes, we can but...
1. As I wrote in the other reply, I don't think that's necessary. Devices don't spin up on FLUSH_CACHE followed by STANDBY_IMMEDIATE when they are in Standby state. (Please point out if this isn't true on most drives)
2. As the code currently stands, we have to rely on SCSI HLD for shutting down. We can query power state there and issue shutdown conditionally and all those will be properly translated by libata but it's a bit complex. I'm not sure whether the added complexity is justified.
Quoted 2 lines
With Serial ATA you have to consider the SATA bus PM states as well, but that's a different topic of discussion.
I think that can be dealt with separately.
Thanks.
Signature · 1 line
tejun
Hello, Henrique, sorry about being late.
Quoted 10 lines · Henrique de Moraes Holschuh
Tejun, is it feasible to teach libata to check the device power management state before issuing any of the sleep, head unload and spin-down commands?
libata would need to block its EH from resetting a channel for this check operation (we don't want to wake up sleeping devices to ask it if it is sleeping...) if it cannot easily check if an device is asleep before issuing
a command to it. Either that, or (for as long as there is no such a thing
as a multi-initiator libata device) just keep track of the device power management state by snooping all the commands sent to it.
Quoted 3 lines
The idea would be to do the "optimal" thing if one tries to sleep, unload heads or spin down an device that is already doing one of these things... That would make things MUCH easier on userspace.
For that to be meaningful, the premise is that ATA drives spin up to process STANDBY IMMEDIATE when it's in Standby state. The spec doesn't exactly specify what happens on when STANDBY IMMEDIATE is received while in Standby state other than it exits to Active iff Media access is required.
I've tested several drivers from different vendors and none of them seems to spin up when they receive FLUSH and STANDBY IMMEDIATE while they're in Standby mode. They just ack that the command is complete and stay spun down. Do you have drives which act differently?
So, if most of drives act that way, whether both or anyone of them are being done just doesn't matter as long as at least one gets done.
Thanks.
Signature · 1 line
tejun
Quoted 5 lines · Soeren Sonnenburg
OK, how about this (please especially check the non SIL part):
SATA Hotplug from the User Side
- For SIL3114 and SIL3124 you don't have to run any commands at all. It
ahci and ck804 flavor of sata_nv's can do hotplug without user assistance too.
[--snip--]
Quoted 15 lines
- For other chipsets one in addition to stop using the device before unplugging has to call scsiadd -r to fully remove it, e.g. in the following example the disk on scsi host 3 channel 0 id 0 lun 0 will be removed, then on reinserting a disk call scsiadd -s :
# scsiadd -pAttached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400832AS Rev: 3.01
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi3 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3400620AS Rev: 3.AA
Type: Direct-Access ANSI SCSI revision: 05
Quoted 2 lines
# scsiadd -r 3 0 0 0 # scsiadd -s
Doing the above might not be such a good idea on drivers which don't implement new EH yet. Those are sata_mv, sata_promise (getting there) and sata_sx4.
Thanks.
Signature · 1 line
tejun