Disk errors loop forever in 2.6.29
Sitsofe Wheeler
Tested-byQuoted 6 lines
From: James Bottomley <[email protected]>
Can you try this patch ... it was something I meant to get into 2.6.29 but forgot about. The key problem that you seem to be hitting is that the requeue evades the timeout check. Moving the timeout check to block should fix that.
Even by itself this patch works although the behaviour is now different to that of 2.6.28. Whereas in 2.6.28 it would timeout very quickly (after about 1 or 2 seconds) it now times out after 30 seconds (initially I hadn't realised that it would be so much longer). The other difference is that the failure becomes cached - trying to reread the same area using dd instantly fails from now on without any more disk IO/kernel error messages until echo 1 > /proc/sys/vm/drop_caches is issued. Before it would always go straight to the disk for the unread section and the kernel spew would be repeated before the failure.
I guess if this is new intended behaviour then all is well - it's just different and slower/faster :)
James Bottomley
maintainerQuoted 7 lines · Sitsofe Wheeler
Quoted 6 lines
From: James Bottomley <[email protected]>
Can you try this patch ... it was something I meant to get into 2.6.29 but forgot about. The key problem that you seem to be hitting is that the requeue evades the timeout check. Moving the timeout check to block should fix that.
Quoted 9 lines
Even by itself this patch works although the behaviour is now different to that of 2.6.28. Whereas in 2.6.28 it would timeout very quickly (after about 1 or 2 seconds) it now times out after 30 seconds (initially I hadn't realised that it would be so much longer). The other difference is that the failure becomes cached - trying to reread the same area using dd instantly fails from now on without any more disk IO/kernel error messages until echo 1 >
/proc/sys/vm/drop_cachesis issued. Before it would always go straight to the disk for the unread section and the kernel spew would be repeated before the failure.
Well, the block patch is more of a catch all: without it apparently the scsi_io_completion() rewrite has several cases where it will apparently loop forever.
Your specific behaviour change seems to be a bug in the way the rewrite is handling ABORTED_COMMAND. Could you try the patch below (with or without the block patch, it shouldn't matter) and see if it gets you back to 2.6.28 behaviour?
Thanks,
James
---
drivers/scsi/scsi_lib.c +2 -3
@@ -850,12 +850,11 @@ void scsi_io_completion(struct scsi_cmnd *cmd, unsigned int good_bytes) action =ACTION_FAIL ;break ;case ABORTED_COMMAND : + action =ACTION_FAIL ;if (sshdr.asc ==0x10 ) {/* DIF */ description ="Target Data Integrity Failure" ; - action =ACTION_FAIL ; error = -EILSEQ ; - }else - action =ACTION_RETRY ; + }break ;case NOT_READY :/* If the device is in the process of becoming
Sitsofe Wheeler
Tested-byQuoted 7 lines
From: James Bottomley <[email protected]>
Your specific behaviour change seems to be a bug in the way the rewrite is handling
ABORTED_COMMAND. Could you try the patch below (with or without the block patch, it shouldn't matter) and see if it gets you back to 2.6.28 behaviour?
Yup that patch makes things timeout quickly just like 2.6.28. The whole "it didn't used to cache failures" piece in the previous email seems to have been a fabrication inside my own head - I checked and 2.6.28 caches the failure until the cache is dropped too.
It might be worth turning your original patch into some sort of warning stack trace trigger so that people understand it's just there for unwedging and isn't expected behaviour (it is useful for breaking out of near lockups though).