restrict legal sdev_state transitions via sysfs
PATCH 1/1 restrict legal sdev_state transitions via sysfs
Userspace can currently write to sysfs to transition sdev_state to RUNNING or OFFLINE from any source state. This causes issues because proper transitioning out of some states involves steps besides just changing sdev_state, so allowing userspace to change sdev_state regardless of the source state can result in inconsistencies; e.g. with iscsi we can end up with sdev_state == SDEV_RUNNING while the device queue is quiesced. Any task attempting IO on the device will then hang, and in more recent kernels, iscsid will hang as well. More detail about this bug is provided in my first attempt:
https://groups.google.com/g/open-iscsi/c/PNKca4HgPDs/m/CXaDkntOAQAJ
---
Looking for feedback on the "allowed source states" list. The bug I hit is solved by prohibiting transitions out of SDEV_BLOCKED, but I think most others shouldn't be allowed either.
drivers/scsi/scsi_sysfs.c | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/scsi/scsi_sysfs.c +8 -0
@@ -822,6 +822,14 @@ store_state_field(struct device *dev, struct device_attribute *attr, }mutex_lock (&sdev->state_mutex); +switch (sdev->sdev_state) { +case SDEV_RUNNING : +case SDEV_OFFLINE : +break ; +default : +mutex_unlock (&sdev->state_mutex); +return -EINVAL ; + }if (sdev->sdev_state ==SDEV_RUNNING && state ==SDEV_RUNNING ) { ret =0 ; }else {
base-commit: 7f615c1b5986ff08a725ee489e838c90f8197bcd
Signature · 1 line
2.25.1
Quoted 4 lines · Uday Shankar
---
Looking for feedback on the "allowed source states" list. The bug I hit is solved by prohibiting transitions out ofSDEV_BLOCKED, but I think most others shouldn't be allowed either.
I think it's ok to be restrictive:
1. BLOCKED is just broken. When the transport classes and scsi_lib transition out of that state they do a lot more than just set the set. We are leaving the kernel in mismatched state where the device state is running but the block layerand transport classes are not ready for IO.
2. CREATED does not happen. We go into RUNNING then do scsi_sysfs_add_sdev so userspace should not see the CREATED state.
3. I'm not 100% sure about SDEV_QUIESCE though. It looks like it has similar issues as BLOCKED where scsi_device_resume will undo things it did in scsi_device_quiesce, so we can't just set the state to RUNNING and expect things to work. I'm not sure about the scsi_transport_spi cases.
It would be best for James or Hannes to comment because they know that code well.
4. The transport classes are the ones that put devs in SDEV_TRANSPORT_OFFLINE and then transition them when they are ready. The block layer is at least in the correct state, but the transport classes may not be ready for IO since they are not expecting IO to be queued to them at that time.
Quoted 25 lines
drivers/scsi/scsi_sysfs.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/scsi/scsi_sysfs.c +8
-0@@ -822,6 +822,14 @@ store_state_field(struct device *dev, struct device_attribute *attr, }mutex_lock (&sdev->state_mutex); +switch (sdev->sdev_state) { +case SDEV_RUNNING : +case SDEV_OFFLINE : +break ; +default : +mutex_unlock (&sdev->state_mutex); +return -EINVAL ; + }if (sdev->sdev_state ==SDEV_RUNNING && state ==SDEV_RUNNING ) { ret =0 ; }else {base-commit:
7f615c1b5986ff08a725ee489e838c90f8197bcd
Quoted 16 lines · Mike Christie
Quoted 4 lines · Uday Shankar
---
Looking for feedback on the "allowed source states" list. The bug I hit is solved by prohibiting transitions out ofSDEV_BLOCKED, but I think most others shouldn't be allowed either.I think it's ok to be restrictive:
1. BLOCKED is just broken. When the transport classes and
scsi_libtransition out of that state they do a lot more than just set the set. We are leaving the kernel in mismatched state where the device state is running but the block layerand transport classes are not ready for IO.2. CREATED does not happen. We go into RUNNING then do
scsi_sysfs_add_sdevso userspace should not see the CREATED state.
Quoted 6 lines
3. I'm not 100% sure about
SDEV_QUIESCEthough. It looks like it has similar issues as BLOCKED wherescsi_device_resumewill undo things it did inscsi_device_quiesce, so we can't just set the state to RUNNING and expect things to work. I'm not sure about thescsi_transport_spicases.It would be best for James or Hannes to comment because they know that code well.
Adding Hannes to CC.
Quoted 31 lines
4. The transport classes are the ones that put devs in
SDEV_TRANSPORT_OFFLINEand then transition them when they are ready. The block layer is at least in the correct state, but the transport classes may not be ready for IO since they are not expecting IO to be queued to them at that time.Quoted 25 lines
drivers/scsi/scsi_sysfs.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/scsi/scsi_sysfs.c +8
-0@@ -822,6 +822,14 @@ store_state_field(struct device *dev, struct device_attribute *attr, }mutex_lock (&sdev->state_mutex); +switch (sdev->sdev_state) { +case SDEV_RUNNING : +case SDEV_OFFLINE : +break ; +default : +mutex_unlock (&sdev->state_mutex); +return -EINVAL ; + }if (sdev->sdev_state ==SDEV_RUNNING && state ==SDEV_RUNNING ) { ret =0 ; }else {base-commit:
7f615c1b5986ff08a725ee489e838c90f8197bcd
Hannes Reinecke
Reviewed-byQuoted 16 lines · Mike Christie
Quoted 4 lines · Uday Shankar
---
Looking for feedback on the "allowed source states" list. The bug I hit is solved by prohibiting transitions out ofSDEV_BLOCKED, but I think most others shouldn't be allowed either.I think it's ok to be restrictive:
1. BLOCKED is just broken. When the transport classes and
scsi_libtransition out of that state they do a lot more than just set the set. We are leaving the kernel in mismatched state where the device state is running but the block layerand transport classes are not ready for IO.2. CREATED does not happen. We go into RUNNING then do
scsi_sysfs_add_sdevso userspace should not see the CREATED state.
Quoted 7 lines
3. I'm not 100% sure about
SDEV_QUIESCEthough. It looks like it has similar issues as BLOCKED wherescsi_device_resumewill undo things it did inscsi_device_quiesce, so we can't just set the state to RUNNING and expect things to work. I'm not sure about thescsi_transport_spicases.It would be best for James or Hannes to comment because they know that code well.
Indeed, you are correct.
The only sensible state transitions to be modified from userland is indeed between SDEV_RUNNING and SDEV_OFFLINE.
All other states are in fact part of the SCSI midlayer operations, and there a quite strict rules on which state transitions are allowed and who should be initiating them.
So yes, I'm fine with this patch.
Cheers,
Hannes
Signature · 5 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew Myers, Andrew McDonald, Martje Boudien Moerman
Quoted 30 lines · Uday Shankar
Userspace can currently write to sysfs to transition
sdev_stateto RUNNING or OFFLINE from any source state. This causes issues because proper transitioning out of some states involves steps besides just changingsdev_state, so allowing userspace to changesdev_stateregardless of the source state can result in inconsistencies; e.g. with iscsi we can end up withsdev_state==SDEV_RUNNINGwhile the device queue is quiesced. Any task attempting IO on the device will then hang, and in more recent kernels, iscsid will hang as well. More detail about this bug is provided in my first attempt:
https://groups.google.com/g/open-iscsi/c/PNKca4HgPDs/m/CXaDkntOAQAJSigned-off-by Uday Shankar <[email protected]>Suggested-by Mike Christie <[email protected]>---
Looking for feedback on the "allowed source states" list. The bug I hit is solved by prohibiting transitions out ofSDEV_BLOCKED, but I think most others shouldn't be allowed either.drivers/scsi/scsi_sysfs.c | 8 ++++++++ 1 file changed, 8 insertions(+)drivers/scsi/scsi_sysfs.c +1
-0@@ -822,6 +822,14 @@ store_state_field(struct device *dev, struct device_attribute *attr, }mutex_lock (&sdev->state_mutex); +switch (sdev->sdev_state) {
drivers/scsi/scsi_sysfs.c store_state_field(struct device *dev, struct device_attribute *attr, · line 835
Quoted 10 lines
drivers/scsi/scsi_sysfs.c +7
-0+case SDEV_RUNNING : +case SDEV_OFFLINE : +break ; +default : +mutex_unlock (&sdev->state_mutex); +return -EINVAL ; + }if (sdev->sdev_state ==SDEV_RUNNING && state ==SDEV_RUNNING ) { ret =0 ; }else {
The return value -EAGAIN might be more appropriate since it is not the value written into sysfs that is invalid but the current state that is inappropriate for the requested transition.
Thanks,
Bart.
Martin K. Petersen
AppliedQuoted 11 lines · Uday Shankar
Userspace can currently write to sysfs to transition
sdev_stateto RUNNING or OFFLINE from any source state. This causes issues because proper transitioning out of some states involves steps besides just changingsdev_state, so allowing userspace to changesdev_stateregardless of the source state can result in inconsistencies; e.g. with iscsi we can end up withsdev_state==SDEV_RUNNINGwhile the device queue is quiesced. Any task attempting IO on the device will then hang, and in more recent kernels, iscsid will hang as well. More detail about this bug is provided in my first attempt:
https://groups.google.com/g/open-iscsi/c/PNKca4HgPDs/m/CXaDkntOAQAJ
Quoted 1 line
[...]
Applied to 6.1/scsi-fixes, thanks!
[1/1] restrict legal sdev_state transitions via sysfs
https://git.kernel.org/mkp/scsi/c/2331ce6126be
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering