SUNRPC: Fix backchannel reply, again
Subsystems (3)
[email protected]
maintainerPATCH 1/1 SUNRPC: Fix backchannel reply, again
From: Chuck Lever <[email protected]>
I still see "RPC: Could not send backchannel reply error: -110" quite often, along with slow-running tests. Debugging shows that the backchannel is still stumbling when it has to queue a callback reply on a busy transport.
Note that every one of these timeouts causes a connection loss by virtue of the xprt_conditional_disconnect() call in that arm of call_cb_transmit_status().
I found that setting to_maxval is necessary to get the RPC timeout logic to behave whenever to_exponential is not set.
--- net/sunrpc/svc.c | 1 + 1 file changed, 1 insertion(+)
net/sunrpc/svc.c +1 -0
@@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp) timeout.to_initval = req->rq_xprt->timeout->to_initval; timeout.to_retries = req->rq_xprt->timeout->to_retries; } + timeout.to_maxval = timeout.to_initval;memcpy (&req->rq_snd_buf, &rqstp->rq_res,sizeof (req->rq_snd_buf)); task =rpc_run_bc_task (req, &timeout);
Signature · 1 line
2.45.1
Quoted 35 lines · On Fri, Jun 14, 2024 at 10:18:52AM -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
I still see "RPC: Could not send backchannel reply error: -110" quite often, along with slow-running tests. Debugging shows that the backchannel is still stumbling when it has to queue a callback reply on a busy transport.
Note that every one of these timeouts causes a connection loss by virtue of the
xprt_conditional_disconnect()call in that arm ofcall_cb_transmit_status().I found that setting
to_maxvalis necessary to get the RPC timeout logic to behave wheneverto_exponentialis not set.Fixes 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeouts for backchannel")Signed-off-by Chuck Lever <[email protected]>--- net/sunrpc/svc.c | 1 + 1 file changed, 1 insertion(+)net/sunrpc/svc.c +1
-0@@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp) timeout.to_initval = req->rq_xprt->timeout->to_initval; timeout.to_retries = req->rq_xprt->timeout->to_retries; } + timeout.to_maxval = timeout.to_initval;memcpy (&req->rq_snd_buf, &rqstp->rq_res,sizeof (req->rq_snd_buf)); task =rpc_run_bc_task (req, &timeout);Signature · 1 line
2.45.1
Hi - would love to see this in 6.10-rc. Is there a chance that could happen?
Signature · 1 line
Chuck Lever
Quoted 38 lines · Chuck Lever
Quoted 37 lines · On Fri, Jun 14, 2024 at 10:18:52AM -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
I still see "RPC: Could not send backchannel reply error: -110" quite often, along with slow-running tests. Debugging shows that the
backchannel is still stumbling when it has to queue a callback reply
on a busy transport.Note that every one of these timeouts causes a connection loss by virtue of the
xprt_conditional_disconnect()call in that arm ofcall_cb_transmit_status().I found that setting
to_maxvalis necessary to get the RPC timeout logic to behave wheneverto_exponentialis not set.Fixes 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeoutsfor backchannel")
Signed-off-by Chuck Lever <[email protected]>---
net/sunrpc/svc.c | 1 +1 file changed, 1 insertion(+)net/sunrpc/svc.c +0
-0@@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp)timeout.to_initval = req->rq_xprt->timeout-
Quoted 1 line
to_initval;timeout.to_retries = req->rq_xprt->timeout-
Quoted 1 line
to_retries;}
net/sunrpc/svc.c +1
-0+ timeout.to_maxval = timeout.to_initval;memcpy(&req->rq_snd_buf,
&rqstp->rq_res, sizeof(req-Quoted 1 line
rq_snd_buf));
Quoted 8 lines
Quoted 5 lines
task =
rpc_run_bc_task(req, &timeout);Signature · 1 line
2.45.1Hi - would love to see this in 6.10-rc. Is there a chance that could happen?
Hmm... Can we please also set the remaining fields in timeout to 0? Otherwise, we're still playing roulette with what actually ends up happening in xprt_calc_majortimeo(). If to_increment happens to be large enough, we could overflow and end up with a silly small timeout value on a retry.
Signature · 3 lines
Trond Myklebust Linux NFS client maintainer, Hammerspace [email protected]
Quoted 34 lines · On Fri, 2024-06-14 at 10:18 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
I still see "RPC: Could not send backchannel reply error: -110" quite often, along with slow-running tests. Debugging shows that the backchannel is still stumbling when it has to queue a callback reply on a busy transport.
Note that every one of these timeouts causes a connection loss by virtue of the
xprt_conditional_disconnect()call in that arm ofcall_cb_transmit_status().I found that setting
to_maxvalis necessary to get the RPC timeout logic to behave wheneverto_exponentialis not set.Fixes 57331a59ac0d ("NFSv4.1: Use the nfs_client's rpc timeouts forbackchannel")
Signed-off-by Chuck Lever <[email protected]>---
net/sunrpc/svc.c | 1 +1 file changed, 1 insertion(+)net/sunrpc/svc.c +0
-0@@ -1643,6 +1643,7 @@ void svc_process_bc(struct rpc_rqst *req, struct svc_rqst *rqstp)timeout.to_initval = req->rq_xprt->timeout-
Quoted 1 line
to_initval;timeout.to_retries = req->rq_xprt->timeout-
Quoted 1 line
to_retries;}
net/sunrpc/svc.c +1
-0+ timeout.to_maxval = timeout.to_initval;memcpy(&req->rq_snd_buf,
&rqstp->rq_res, sizeof(req-
Quoted 3 lines
Quoted 1 line
rq_snd_buf));task =
rpc_run_bc_task(req, &timeout);