xprtrdma: Decouple req recycling from RPC completion
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest5 patches 0 repliesNew
- v23 patches 0 repliesSuperseded
- v11 patch 1 replySuperseded
Changes in v3 from the cover letter
- While addressing sashiko review comments, substantially reworked to simplify and better align the series with the current architecture of xprtrdma
Patches in v3 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | xprtrdma: Use sendctx DMA state for Send signalingnet/ net/ | +14 | 0 replies | |
| 2/5 | xprtrdma: Decouple req recycling from RPC completionnet/ net/ net/ +2 more | +97 | 0 replies | |
| 3/5 | xprtrdma: Add request-pool slack for delayed recyclingnet/ | +20 | 0 replies | |
| 4/5 | xprtrdma: Clear receive-side ownership pointers on releasenet/ net/ | +14 | 0 replies | |
| 5/5 | xprtrdma: Document and assert reply-handler invariantsnet/ net/ net/ | +84 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.
Cover letter [PATCH v3 0/5] xprtrdma: Decouple req recycling from RPC completion
From: Chuck Lever <[email protected]>
rl_kref currently conflates two lifetimes through one refcount:
it gates when a Reply can wake its RPC task, and it gates when an rpcrdma_req can return to its free pool. The marshal path takes the Send-side reference only when sc_unmap_count > 0, so a Send carrying only pre-registered buffers takes no Send-side reference. When the Reply for such an RPC arrives before its Send completes, the Reply handler drops rl_kref from 1 to 0 and frees the req while the HCA may still be DMA-reading from its send buffer. A retransmission can put different bytes on the wire.
This series narrows rl_kref's job. The RPC layer takes one reference at slot allocation; rpcrdma_prepare_send_sges() takes a Send-side reference unconditionally after WR prep succeeds. A req returns to its free pool only after both owners release. Replies complete the RPC directly, without atomic activity on rl_kref.
Three design choices shape the series.
The Send-side reference is taken only on the success path of rpcrdma_prepare_send_sges(). Marshal failure runs rpcrdma_sendctx_cancel(), which unmaps the SGEs and clears sc_req without touching rl_kref. Sendctx ring walks in rpcrdma_sendctx_put_locked() and rpcrdma_sendctxs_destroy() skip entries whose sc_req is NULL, so a burst of -EIO marshal failures cannot hold reqs off rb_send_bufs.
Connection teardown drains the sendctx ring against pre-reset reqs by ordering rpcrdma_sendctxs_destroy() ahead of rpcrdma_reqs_reset() in rpcrdma_xprt_disconnect(). The drain releases Send-side references whose unsignaled Sends never had a later signaled completion to walk the ring. On the backchannel, releasing a bc_prealloc req re-adds it to bc_pa_list, which xprt_destroy_backchannel() has already emptied; xprt_rdma_destroy() runs xprt_rdma_bc_destroy() a second time after the disconnect to reclaim those reqs.
With recycling now gated on Send completion, completed RPCs can remain pinned by the sendctx ring until the next signaled Send completion. The headroom is bounded: re_send_batch is set to re_max_requests >> 3. The req pool gains max_reqs/8 slack (patch 3) so the recycle delay does not stall a slot allocation that the RPC/RDMA credit window would admit.
Changes since v2:
- While addressing sashiko review comments, substantially reworked to simplify and better align the series with the current architecture of xprtrdma
Link to v2: https://lore.kernel.org/linux-nfs/[email protected]/#r
Changes since v1:
- Split into three patches. A prep patch converts the Send-signaling test from a kref_read to sc_unmap_count, and a separate patch names the request-pool slack at its allocation site.
- Wrap the bc_prealloc release branch in CONFIG_SUNRPC_BACKCHANNEL (kernel test robot, build break on configs without the backchannel).
- Order rpcrdma_sendctxs_destroy() ahead of rpcrdma_reqs_reset() in rpcrdma_xprt_disconnect() so the drain runs against pre-reset reqs.
- Run xprt_rdma_bc_destroy() a second time from xprt_rdma_destroy() to reclaim bc_prealloc reqs returned by the disconnect's drain.
- Add rpcrdma_sendctx_cancel() for the marshal-failure path; sendctx ring walkers skip entries with sc_req == NULL.
Link to v1: https://lore.kernel.org/r/[email protected]
Chuck Lever (5):
xprtrdma: Use sendctx DMA state for Send signaling
xprtrdma: Decouple req recycling from RPC completion
xprtrdma: Add request-pool slack for delayed recycling
xprtrdma: Clear receive-side ownership pointers on release
xprtrdma: Document and assert reply-handler invariants
net/sunrpc/xprtrdma/backchannel.c | 5 +- net/sunrpc/xprtrdma/frwr_ops.c | 2 +- net/sunrpc/xprtrdma/rpc_rdma.c | 134 ++++++++++++++++++++++-------- net/sunrpc/xprtrdma/transport.c | 68 +++++++++++++-- net/sunrpc/xprtrdma/verbs.c | 68 +++++++++++++-- net/sunrpc/xprtrdma/xprt_rdma.h | 2 +- 6 files changed, 225 insertions(+), 54 deletions(-)
Signature · 1 line
2.54.0