Fix residual handling in two SCSI LLDs
Subsystems (6)
Bart Van Assche
maintainercover Fix residual handling in two SCSI LLDs
Hi Martin,
This patch series fixes the documentation of scsi_set_resid() and also fixes residual handling in two SCSI LLDs. Please consider this patch series for the next merge window.
Thanks,
Bart.
Bart Van Assche (3):
scsi: core: Fix the scsi_set_resid() documentation
scsi: ufs: Fix residual handling
RDMA/srp: Fix residual handling
Documentation/scsi/scsi_mid_low_api.rst | 4 ++-- drivers/infiniband/ulp/srp/ib_srp.c | 4 ---- drivers/ufs/core/ufshcd.c | 19 ++++++++++++++++--- include/ufs/ufs.h | 6 ++++++ 4 files changed, 24 insertions(+), 9 deletions(-)
Bart Van Assche
maintainerPATCH 1/3 scsi: core: Fix the scsi_set_resid() documentation
Because scsi_finish_command() subtracts the residual from the buffer length, residual overflows must not be reported. Reflect this in the SCSI documentation. See also commit 9237f04e12cc ("scsi: core: Fix scsi_get/set_resid() interface")
--- Documentation/scsi/scsi_mid_low_api.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/scsi/scsi_mid_low_api.rst +2 -2
@@ -1190,11 +1190,11 @@ Members of interest: - pointer to scsi_device object that this command is associated with. resid - - an LLD should set this signed integer to the requested + - an LLD should set this unsigned integer to the requested transfer length (i.e. 'request_bufflen') less the number of bytes that are actually transferred. 'resid' is preset to 0 so an LLD can ignore it if it cannot detect - underruns (overruns should be rare). If possible an LLD + underruns (overruns should not be reported). An LLD should set 'resid' prior to invoking 'done'. The most interesting case is data transfers from a SCSI target device (e.g. READs) that underrun.
Martin K. Petersen
AppliedRe: [PATCH 1/3] scsi: core: Fix the scsi_set_resid() documentation
Bart,
Quoted 4 lines
Because
scsi_finish_command()subtracts the residual from the buffer length, residual overflows must not be reported. Reflect this in the SCSI documentation. See also commit9237f04e12cc("scsi: core: Fix scsi_get/set_resid() interface")
Applied to 6.6/scsi-staging, thanks!
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
Re: [PATCH 1/3] scsi: core: Fix the scsi_set_resid() documentation
In reply to Bart Van AsscheQuoted 19 lines · Bart Van Assche
Because
scsi_finish_command()subtracts the residual from the buffer length, residual overflows must not be reported. Reflect this in the SCSI documentation. See also commit9237f04e12cc("scsi: core: Fix scsi_get/set_resid() interface")Cc Damien Le Moal <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Douglas Gilbert <[email protected]>Signed-off-by Bart Van Assche <[email protected]>--- Documentation/scsi/scsi_mid_low_api.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Documentation/scsi/scsi_mid_low_api.rst +0
-0
Documentation/scsi/scsi_mid_low_api.rst Members of interest: · line 1197
Quoted 10 lines
Documentation/scsi/scsi_mid_low_api.rst +2
-2@@ -1190,11 +1190,11 @@ Members of interest: - pointer to scsi_device object that this command is associated with. resid - - an LLD should set this signed integer to the requested + - an LLD should set this unsigned integer to the requested transfer length (i.e. 'request_bufflen') less the number of bytes that are actually transferred. 'resid' is preset to 0 so an LLD can ignore it if it cannot detect - underruns (overruns should be rare). If possible an LLD + underruns (overruns should not be reported). An LLD
I'm very late to party, sorry. But we have certainly seen at least one overrun reported some years ago on a FC SAN. We've changed some handling in zFCP due to that (a099b7b1fc1f ("scsi: zfcp: add handling for FCP_RESID_OVER to the fcp ingress path") ). The theory back than was, that it was cause by either a faulty ISL in the fabric, or some other "bit-errors" on the wire that caused some FC frames being dropped during transmit.
I added that we mark such commands with DID_ERROR, so they are retried, if that is permissible.
Quoted 3 lines
Documentation/scsi/scsi_mid_low_api.rst +0
-0should set 'resid' prior to invoking 'done'. The most interesting case is data transfers from a SCSI target device (e.g. READs) that underrun.
Signature · 4 lines
Best Regards, Benjamin Block / Linux on IBM Z Kernel Development IBM Deutschland Research & Development GmbH / https://www.ibm.com/privacy Vors. Aufs.-R.: Gregor Pillen / Geschäftsführung: David Faller Sitz der Ges.: Böblingen / Registergericht: AmtsG Stuttgart, HRB 243294
Bart Van Assche
maintainerRe: [PATCH 1/3] scsi: core: Fix the scsi_set_resid() documentation
Quoted 39 lines · Benjamin Block
Quoted 19 lines · Bart Van Assche
Because
scsi_finish_command()subtracts the residual from the buffer length, residual overflows must not be reported. Reflect this in the SCSI documentation. See also commit9237f04e12cc("scsi: core: Fix scsi_get/set_resid() interface")Cc Damien Le Moal <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Douglas Gilbert <[email protected]>Signed-off-by Bart Van Assche <[email protected]>--- Documentation/scsi/scsi_mid_low_api.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Documentation/scsi/scsi_mid_low_api.rst +0
-0Documentation/scsi/scsi_mid_low_api.rst Members of interest: · line 1197
Quoted 10 lines
Documentation/scsi/scsi_mid_low_api.rst +2
-2@@ -1190,11 +1190,11 @@ Members of interest: - pointer to scsi_device object that this command is associated with. resid - - an LLD should set this signed integer to the requested + - an LLD should set this unsigned integer to the requested transfer length (i.e. 'request_bufflen') less the number of bytes that are actually transferred. 'resid' is preset to 0 so an LLD can ignore it if it cannot detect - underruns (overruns should be rare). If possible an LLD + underruns (overruns should not be reported). An LLDI'm very late to party, sorry. But we have certainly seen at least one overrun reported some years ago on a FC SAN. We've changed some handling in zFCP due to that (
a099b7b1fc1f("scsi: zfcp: add handling forFCP_RESID_OVERto the fcp ingress path") ). The theory back than was, that it was cause by either a faulty ISL in the fabric, or some other "bit-errors" on the wire that caused some FC frames being dropped during transmit.
Quoted 7 lines
I added that we mark such commands with
DID_ERROR, so they are retried, if that is permissible.Quoted 3 lines
Documentation/scsi/scsi_mid_low_api.rst +0
-0should set 'resid' prior to invoking 'done'. The most interesting case is data transfers from a SCSI target device (e.g. READs) that underrun.
Thanks, that's interesting feedback. Feel free to submit a patch that refines the scsi_set_resid() documentation further.
Bart.
Bart Van Assche
maintainerPATCH 2/3 scsi: ufs: Fix residual handling
In reply to Bart Van AsscheOnly call scsi_set_resid() in case of an underflow. Do not call scsi_set_resid() in case of an overflow.
--- drivers/ufs/core/ufshcd.c | 19 ++++++++++++++++--- include/ufs/ufs.h | 6 ++++++ 2 files changed, 22 insertions(+), 3 deletions(-)
drivers/ufs/core/ufshcd.c +16 -3
@@ -5241,12 +5241,25 @@ static inline int ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp, struct cq_entry *cqe) { + bool overflow, underflow; +u8 upiu_flags;int result =0 ;int scsi_status;enum utp_ocs ocs; - -scsi_set_resid (lrbp->cmd, -be32_to_cpu (lrbp->ucd_rsp_ptr->sr.residual_transfer_count)); +u32 resid; + + upiu_flags =be32_to_cpu (lrbp->ucd_rsp_ptr->header.dword_0) >>16 ; + overflow = upiu_flags &UPIU_RSP_FLAG_OVERFLOW ; + underflow = upiu_flags &UPIU_RSP_FLAG_UNDERFLOW ; + resid =be32_to_cpu (lrbp->ucd_rsp_ptr->sr.residual_transfer_count); +WARN_ON_ONCE (overflow && underflow); +WARN_ON_ONCE (!overflow && !underflow && resid); +/* +* Test !overflow instead of underflow to support UFS devices that do +* not set either flag. +*/ +if (!overflow) +scsi_set_resid (lrbp->cmd, resid);/* overall command status of utrd */ ocs =ufshcd_get_tr_ocs (lrbp, cqe);
include/ufs/ufs.h +6 -0
@@ -104,6 +104,12 @@ enum { UPIU_CMD_FLAGS_READ = 0x40, }; +/* UPIU response flags */ +enum { +UPIU_RSP_FLAG_UNDERFLOW =0x20 , +UPIU_RSP_FLAG_OVERFLOW =0x40 , +}; +/* UPIU Task Attributes */ enum {UPIU_TASK_ATTR_SIMPLE =0x00 ,
Avri Altman
reviewerRE: [PATCH 2/3] scsi: ufs: Fix residual handling
Quoted 26 lines
Only call
scsi_set_resid()in case of an underflow. Do not callscsi_set_resid()in case of an overflow.Cc Avri Altman <[email protected]>Cc Adrian Hunter <[email protected]>Fixes cb38845d90fc ("scsi: ufs: core: Set the residual byte count")Signed-off-by Bart Van Assche <[email protected]>--- drivers/ufs/core/ufshcd.c | 19 ++++++++++++++++--- include/ufs/ufs.h | 6 ++++++ 2 files changed, 22 insertions(+), 3 deletions(-)drivers/ufs/core/ufshcd.c index renamed from drivers/ufs/core/ufshcd.c +0
-0
e31242fe0518..b27372f9b488100644drivers/ufs/core/ufshcd.c +2
-1@@ -5241,12 +5241,25 @@ static inline int ufshcd_transfer_rsp_status(struct ufs_hba *hba, struct ufshcd_lrb *lrbp, struct cq_entry *cqe) { + bool overflow, underflow; +u8 upiu_flags;int result =0 ;int scsi_status;enum utp_ocs ocs; -
drivers/ufs/core/ufshcd.c static inline int ufshcd_transfer_rsp_status(struct · line 5255
Quoted 10 lines
drivers/ufs/core/ufshcd.c +8
-2-scsi_set_resid (lrbp->cmd, -be32_to_cpu (lrbp->ucd_rsp_ptr->sr.residual_transfer_count)); +u32 resid; + + upiu_flags =be32_to_cpu (lrbp->ucd_rsp_ptr->header.dword_0) >>16 ; + overflow = upiu_flags &UPIU_RSP_FLAG_OVERFLOW ; + underflow = upiu_flags &UPIU_RSP_FLAG_UNDERFLOW ; + resid =be32_to_cpu (lrbp->ucd_rsp_ptr->sr.residual_transfer_count); +WARN_ON_ONCE (overflow && underflow); +WARN_ON_ONCE (!overflow && !underflow && resid);
Do we really need to debug the hw? - see Table 10.16 (3.1 spec).
Thanks,
Avri
Quoted 25 lines
drivers/ufs/core/ufshcd.c +6
-0+/* +* Test !overflow instead of underflow to support UFS devices that do +* not set either flag. +*/ +if (!overflow) +scsi_set_resid (lrbp->cmd, resid);/* overall command status of utrd */ ocs = ufshcd_get_tr_ocs(lrbp, cqe); diff --git a/include/ufs/ufs.h
b/include/ufs/ufs.hindex8316e2408ac3..bee5ccc6e7ce100644include/ufs/ufs.h +6
-0@@ -104,6 +104,12 @@ enum { UPIU_CMD_FLAGS_READ = 0x40, }; +/* UPIU response flags */ +enum { +UPIU_RSP_FLAG_UNDERFLOW =0x20 , +UPIU_RSP_FLAG_OVERFLOW =0x40 , +}; +/* UPIU Task Attributes */ enum {UPIU_TASK_ATTR_SIMPLE =0x00 ,
Bart Van Assche
maintainerRe: [PATCH 2/3] scsi: ufs: Fix residual handling
Quoted 4 lines · Avri Altman
drivers/ufs/core/ufshcd.c static inline int · line 5256
Quoted 3 lines
drivers/ufs/core/ufshcd.c +2
-0+WARN_ON_ONCE (overflow && underflow); +WARN_ON_ONCE (!overflow && !underflow && resid);Do we really need to debug the hw? - see Table 10.16 (3.1 spec).
I was wondering about this too while writing this patch. If nobody objects I will leave the above two WARN_ON_ONCE() statements out when reposting this patch.
Thanks,
Bart.
Re: [PATCH 2/3] scsi: ufs: Fix residual handling
Quoted 8 lines · Bart Van Assche
Quoted 4 lines · Avri Altman
Quoted 3 lines
+
WARN_ON_ONCE(overflow && underflow);
+WARN_ON_ONCE(!overflow && !underflow && resid);Do we really need to debug the hw? - see Table 10.16 (3.1 spec).
I was wondering about this too while writing this patch. If nobody objects I will leave the above two
WARN_ON_ONCE()statements out when reposting this patch.
I agree. Please do leave them out.
Bart Van Assche
maintainerPATCH 3/3 RDMA/srp: Fix residual handling
In reply to Bart Van AsscheAlthough the code for residual handling in the SRP initiator follows the SCSI documentation, that documentation has never been correct. Because scsi_finish_command() starts from the data buffer length and subtracts the residual, scsi_set_resid() must not be called if a residual overflow occurs. Hence remove the scsi_set_resid() calls from the SRP initiator if a residual overflow occurrs.
--- drivers/infiniband/ulp/srp/ib_srp.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/infiniband/ulp/srp/ib_srp.c +0 -4
@@ -1979,12 +1979,8 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)if (unlikely (rsp->flags &SRP_RSP_FLAG_DIUNDER ))scsi_set_resid (scmnd,be32_to_cpu (rsp->data_in_res_cnt)); -else if (unlikely (rsp->flags &SRP_RSP_FLAG_DIOVER )) -scsi_set_resid (scmnd, -be32_to_cpu (rsp->data_in_res_cnt));else if (unlikely (rsp->flags &SRP_RSP_FLAG_DOUNDER ))scsi_set_resid (scmnd,be32_to_cpu (rsp->data_out_res_cnt)); -else if (unlikely (rsp->flags &SRP_RSP_FLAG_DOOVER )) -scsi_set_resid (scmnd, -be32_to_cpu (rsp->data_out_res_cnt));srp_free_req (ch, req, scmnd,be32_to_cpu (rsp->req_lim_delta));
Martin K. Petersen
AppliedQuoted 9 lines · Bart Van Assche
This patch series fixes the documentation of
scsi_set_resid()and also fixes residual handling in two SCSI LLDs. Please consider this patch series for the next merge window.Thanks,
Bart.
[...]
Applied to 6.6/scsi-queue, thanks!
[1/3] scsi: core: Fix the scsi_set_resid() documentation
https://git.kernel.org/mkp/scsi/c/f669b8a683e4
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering