svcrdma: Advertise the correct max payload

PATCH Superseded Steve Wise 2 participants 5 messages Sep 22, 2014 – Sep 23, 2014 1 message not in the archive

Subsystems (4)

Networking NFS client NFS server InfiniBand

lore

Share & tools

lore

Steve Wise

author

2014-09-22 20:05 UTC #0 lore raw

PATCH v2 1/1 svcrdma: Advertise the correct max payload

Svcrdma currently advertises 1MB, which is too large. The correct value is the minimum of RPCSVC_MAXPAYLOAD and the max scatter-gather allowed in an NFSRDMA IO chunk * the host page size. This bug is usually benign because the Linux X64 NFSRDMA client correctly limits the payload size to the correct value (64*4096 = 256KB). But if the Linux client is PPC64 with a 64KB page size, then the client will indeed use a payload size that will overflow the server.

Signed-off-by Steve Wise <[email protected]>

---

 net/sunrpc/xprtrdma/svc_rdma_transport.c |    2 +-
 net/sunrpc/xprtrdma/xprt_rdma.h          |    4 ++++
 2 files changed, 5 insertions(+), 1 deletions(-)

net/sunrpc/xprtrdma/svc_rdma_transport.c +1 -1

index 374feb4..4e61880 100644
@@ -91,7 +91,7 @@ struct svc_xprt_class svc_rdma_class = {
 	.xcl_name = "rdma",
 	.xcl_owner = THIS_MODULE,
 	.xcl_ops = &svc_rdma_ops,
-	.xcl_max_payload = RPCSVC_MAXPAYLOAD_TCP,
+	.xcl_max_payload = RPCSVC_MAXPAYLOAD_RDMA,
 	.xcl_ident = XPRT_TRANSPORT_RDMA,
 };

net/sunrpc/xprtrdma/xprt_rdma.h +4 -0

index c419498..a9cf5c3 100644
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
 /* Workqueue created in svc_rdma.c */
 extern struct workqueue_struct *svc_rdma_wq;
 
+#define RPCSVC_MAXPAYLOAD_RDMA \
+	(RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+	 RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+
 #endif				/* _LINUX_SUNRPC_XPRT_RDMA_H */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Steve Wise

author

2014-09-23 19:42 UTC #2 lore raw

Quoted 16 lines

net/sunrpc/xprtrdma/xprt_rdma.h extern struct kmem_cache *svc_rdma_ctxt_cachep; · line 398

Quoted 13 lines

net/sunrpc/xprtrdma/xprt_rdma.h +4 -0

index c419498..a9cf5c3 100644
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
 /* Workqueue created in svc_rdma.c */
 extern struct workqueue_struct *svc_rdma_wq;
 
+#define RPCSVC_MAXPAYLOAD_RDMA \
+	(RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+	 RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+

Couldn't you use:

Quoted 7 lines

#if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
#else
#define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #endif
That seems more idiomatic.

Sure. That makes it easier to read in my opinion too.

I'll send out V3 with this change.

Thanks,

Steve.

'J. Bruce Fields'

2014-09-23 19:48 UTC #3 lore raw

Quoted 22 lines · Steve Wise

Quoted 22 lines
net/sunrpc/xprtrdma/xprt_rdma.h extern struct kmem_cache *svc_rdma_ctxt_cachep; · line 398
Quoted 13 lines
net/sunrpc/xprtrdma/xprt_rdma.h +4 -0
index c419498..a9cf5c3 100644
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
 /* Workqueue created in svc_rdma.c */
 extern struct workqueue_struct *svc_rdma_wq;
 
+#define RPCSVC_MAXPAYLOAD_RDMA \
+	(RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+	 RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+
Couldn't you use:
#if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
#else
#define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #endif

Quoted 5 lines

Quoted 1 line
That seems more idiomatic.
Sure. That makes it easier to read in my opinion too.
I'll send out V3 with this change.

While we're bikeshedding, why not use min()?

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Steve Wise

author

2014-09-23 19:53 UTC #4 lore raw

Quoted 34 lines

-----Original Message-----
From: 'J. Bruce Fields' [mailto:[email protected]]
Sent: Tuesday, September 23, 2014 2:48 PM
To: Steve Wise
Cc: 'Chuck Lever'; [email protected]; [email protected]
Subject: Re: [PATCH V2] svcrdma: Advertise the correct max payload
Quoted 24 lines · Steve Wise
Quoted 23 lines
net/sunrpc/xprtrdma/xprt_rdma.h extern struct kmem_cache *svc_rdma_ctxt_cachep; · line 398
Quoted 13 lines
net/sunrpc/xprtrdma/xprt_rdma.h +4 -0
index c419498..a9cf5c3 100644
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
 /* Workqueue created in svc_rdma.c */
 extern struct workqueue_struct *svc_rdma_wq;
 
+#define RPCSVC_MAXPAYLOAD_RDMA \
+	(RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+	 RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+
Couldn't you use:
#if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
#else
#define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #endif
That seems more idiomatic.

Quoted 7 lines

Quoted 3 lines
Sure. That makes it easier to read in my opinion too.
I'll send out V3 with this change.
While we're bikeshedding, why not use min()?
--b.

I tried that initially. But min() and min_t() don't work because of the way we use the #define. With it defined thusly:

#define RPCSVC_MAXPAYLOAD_RDMA min(RPC_MAXPAYLOAD, RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)

I see this error:

CC [M] net/sunrpc/xprtrdma/svc_rdma_transport.o
net/sunrpc/xprtrdma/svc_rdma_transport.c:94: error: braced-group within expression allowed only inside a function make[3]: *** [net/sunrpc/xprtrdma/svc_rdma_transport.o] Error 1
make[2]: *** [net/sunrpc/xprtrdma] Error 2
make[1]: *** [net/sunrpc] Error 2
make: *** [net] Error 2

min() looks like this:

#define min(x, y) ({                            \
        typeof(x) _min1 = (x);                  \
        typeof(y) _min2 = (y);                  \
        (void) (&_min1 == &_min2);              \
        _min1 < _min2 ? _min1 : _min2; })

'J. Bruce Fields'

2014-09-23 19:55 UTC #5 lore raw

Quoted 36 lines · Steve Wise

Quoted 36 lines
-----Original Message-----
From: 'J. Bruce Fields' [mailto:[email protected]]
Sent: Tuesday, September 23, 2014 2:48 PM
To: Steve Wise
Cc: 'Chuck Lever'; [email protected]; [email protected]
Subject: Re: [PATCH V2] svcrdma: Advertise the correct max payload
Quoted 26 lines · Steve Wise
Quoted 16 lines
net/sunrpc/xprtrdma/xprt_rdma.h extern struct kmem_cache *svc_rdma_ctxt_cachep; · line 398
Quoted 13 lines
net/sunrpc/xprtrdma/xprt_rdma.h +4 -0
index c419498..a9cf5c3 100644
@@ -392,4 +392,8 @@ extern struct kmem_cache *svc_rdma_ctxt_cachep;
 /* Workqueue created in svc_rdma.c */
 extern struct workqueue_struct *svc_rdma_wq;
 
+#define RPCSVC_MAXPAYLOAD_RDMA \
+	(RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) ? \
+	 RPCSVC_MAXPAYLOAD : (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT))
+
Couldn't you use:
Quoted 7 lines
#if RPCSVC_MAXPAYLOAD < (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #define RPCSVC_MAXPAYLOAD_RDMA RPC_MAXPAYLOAD
#else
#define RPCSVC_MAXPAYLOAD_RDMA (RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT) #endif
That seems more idiomatic.
Sure. That makes it easier to read in my opinion too.

Quoted 7 lines

Quoted 5 lines
Quoted 1 line
I'll send out V3 with this change.
While we're bikeshedding, why not use min()?
--b.
I tried that initially. But min() and min_t() don't work because of the way we use the #define. With it defined thusly:

OK, OK. Whatever works.--b.

Quoted 12 lines

#define RPCSVC_MAXPAYLOAD_RDMA min(RPC_MAXPAYLOAD, RPCRDMA_MAX_DATA_SEGS << PAGE_SHIFT)
I see this error:
CC [M] net/sunrpc/xprtrdma/svc_rdma_transport.o
net/sunrpc/xprtrdma/svc_rdma_transport.c:94: error: braced-group within expression allowed only inside a function make[3]: *** [net/sunrpc/xprtrdma/svc_rdma_transport.o] Error 1
make[2]: *** [net/sunrpc/xprtrdma] Error 2
make[1]: *** [net/sunrpc] Error 2
make: *** [net] Error 2

Quoted 8 lines

min() looks like this:

#define min(x, y) ({                            \
        typeof(x) _min1 = (x);                  \
        typeof(y) _min2 = (y);                  \
        (void) (&_min1 == &_min2);              \
        _min1 < _min2 ? _min1 : _min2; })

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html