svcrdma: fix bounce buffers for non-zero page offsets
Subsystems (4)
PATCH 1/1 svcrdma: fix bounce buffers for non-zero page offsets
This was discovered using O_DIRECT and small unaligned file offsets at the client side.
--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sunrpc/xprtrdma/svc_rdma_sendto.c +1 -1
@@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining); -memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len); remaining -= len; dst += len; pageoff =0 ;
Signature · 1 line
2.26.2
Hi Dan-
Quoted 16 lines · Dan Aloni
This was discovered using
O_DIRECTand small unaligned file offsets at the client side.Fixes e248aa7be86 ("svcrdma: Remove max_sge check at connect time")Signed-off-by Dan Aloni <[email protected]>---
net/sunrpc/xprtrdma/svc_rdma_sendto.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/xprtrdma/svc_rdma_sendto.c +0
-0
net/sunrpc/xprtrdma/svc_rdma_sendto.c static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, · line 641
Quoted 5 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c +1
-1@@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining); -memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len);
I'm assuming the only relevant place that sets xdr->page_base is nfsd_splice_actor() ?
Quoted 6 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c +0
-0remaining -= len; dst += len; pageoff =0 ;Signature · 1 line
2.26.2
--
Chuck Lever
Quoted 21 lines · Chuck Lever
Hi Dan-
Quoted 19 lines · Dan Aloni
This was discovered using
O_DIRECTand small unaligned file offsets at the client side.Fixes e248aa7be86 ("svcrdma: Remove max_sge check at connect time")Signed-off-by Dan Aloni <[email protected]>---
net/sunrpc/xprtrdma/svc_rdma_sendto.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/xprtrdma/svc_rdma_sendto.c +0
-0@@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining);
Quoted 5 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, · line 641
Quoted 2 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c +1
-1-memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len);I'm assuming the only relevant place that sets
xdr->page_baseisnfsd_splice_actor()?
Yes, and traces at the server indeed indicate that splicing happened. This works for both tmpfs and ext4 as host FSes.
Signature · 1 line
Dan Aloni
Quoted 26 lines · Dan Aloni
Quoted 21 lines · Chuck Lever
Hi Dan-
Quoted 19 lines · Dan Aloni
This was discovered using
O_DIRECTand small unaligned file offsets at the client side.Fixes e248aa7be86 ("svcrdma: Remove max_sge check at connect time")Signed-off-by Dan Aloni <[email protected]>---
net/sunrpc/xprtrdma/svc_rdma_sendto.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/xprtrdma/svc_rdma_sendto.c +0
-0@@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining);
Quoted 8 lines
Quoted 5 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, · line 641
Quoted 2 lines
net/sunrpc/xprtrdma/svc_rdma_sendto.c +1
-1-memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len);I'm assuming the only relevant place that sets
xdr->page_baseisnfsd_splice_actor()?Yes, and traces at the server indeed indicate that splicing happened. This works for both tmpfs and ext4 as host FSes.
Seems plausible, thanks. I'll throw it into my test environment.
--
Chuck Lever
PATCH v2 1/1 svcrdma: fix bounce buffers for unaligned offsets and multiple pages
This was discovered using O_DIRECT at the client side, with small unaligned file offsets or IOs that span multiple file pages.
--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Extended testing found another issue with the loop.
net/sunrpc/xprtrdma/svc_rdma_sendto.c +2 -1
@@ -638,10 +638,11 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining); -memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len); remaining -= len; dst += len; pageoff =0 ; + ppages++; } }
Signature · 1 line
2.26.2
Re: [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages
Thanks, applying for 5.10.--b.
Quoted 30 lines · Dan Aloni
This was discovered using
O_DIRECTat the client side, with small unaligned file offsets or IOs that span multiple file pages.Fixes e248aa7be86 ("svcrdma: Remove max_sge check at connect time")Signed-off-by Dan Aloni <[email protected]>--- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)Extended testing found another issue with the loop.
net/sunrpc/xprtrdma/svc_rdma_sendto.c +2
-1@@ -638,10 +638,11 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,while (remaining) { len =min_t (u32,PAGE_SIZE - pageoff, remaining); -memcpy (dst,page_address (*ppages), len); +memcpy (dst,page_address (*ppages) + pageoff, len); remaining -= len; dst += len; pageoff =0 ; + ppages++; } }Signature · 1 line
2.26.2