sunrpc: raise kernel RPC channel buffer size
Subsystems (3)
PATCH 1/1 sunrpc: raise kernel RPC channel buffer size
Its possible that using AUTH_SYS and mountd manage-gids option a user may hit the 8k RPC channel buffer limit. This have been observed on field, causing unanswered RPCs on clients after mountd fails to write on channel :
rpc.mountd[11231]: auth_unix_gid: error writing reply
Userland nfs-utils uses a buffer size of 32k (RPC_CHAN_BUF_SIZE), so lets match those two.
--- net/sunrpc/cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sunrpc/cache.c +1 -1
@@ -908,7 +908,7 @@ static ssize_t cache_do_downcall(char *kaddr, const char __user *buf,static ssize_t cache_slow_downcall (const char __user *buf,size_t count,struct cache_detail *cd) { -static char write_buf[8192 ];/* protected by queue_io_mutex */ +static char write_buf[32768 ];/* protected by queue_io_mutex */ ssize_t ret = -EINVAL ;if (count >=sizeof (write_buf))
Signature · 1 line
2.21.0
Quoted 9 lines · Roberto Bergantinos Corpas
Its possible that using
AUTH_SYSand mountd manage-gids option a user may hit the 8k RPC channel buffer limit. This have been observed on field, causing unanswered RPCs on clients after mountd fails to write on channel :rpc.mountd[11231]:
auth_unix_gid: error writing replyUserland nfs-utils uses a buffer size of 32k (
RPC_CHAN_BUF_SIZE), so lets match those two.
Thanks, applying.
That should allow about 4000 group memberships. If that doesn't do it then maybe it's time to rethink....
--b.
Quoted 21 lines
Signed-off-by Roberto Bergantinos Corpas <[email protected]>--- net/sunrpc/cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/cache.c +1
-1@@ -908,7 +908,7 @@ static ssize_t cache_do_downcall(char *kaddr, const char __user *buf,static ssize_t cache_slow_downcall (const char __user *buf,size_t count,struct cache_detail *cd) { -static char write_buf[8192 ];/* protected by queue_io_mutex */ +static char write_buf[32768 ];/* protected by queue_io_mutex */ ssize_t ret = -EINVAL ;if (count >=sizeof (write_buf))Signature · 1 line
2.21.0
Hi Bruce, Roberto,
Quoted 36 lines · J. Bruce Fields
Quoted 9 lines · Roberto Bergantinos Corpas
Its possible that using
AUTH_SYSand mountd manage-gids option a user may hit the 8k RPC channel buffer limit. This have been observed on field, causing unanswered RPCs on clients after mountd fails to write on channel :rpc.mountd[11231]:
auth_unix_gid: error writing replyUserland nfs-utils uses a buffer size of 32k (
RPC_CHAN_BUF_SIZE), so lets match those two.Thanks, applying.
That should allow about 4000 group memberships. If that doesn't do it then maybe it's time to rethink....
--b.
Quoted 18 lines
Signed-off-by Roberto Bergantinos Corpas <[email protected]>--- net/sunrpc/cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/cache.c +1
-1@@ -908,7 +908,7 @@ static ssize_t cache_do_downcall(char *kaddr, const char __user *buf,static ssize_t cache_slow_downcall (const char __user *buf,size_t count,struct cache_detail *cd) { -static char write_buf[8192 ];/* protected by queue_io_mutex */ +static char write_buf[32768 ];/* protected by queue_io_mutex */ ssize_t ret = -EINVAL ;
Quoted 1 line
net/sunrpc/cache.c static ssize_t cache_do_downcall(char *kaddr, const char __user *buf, · line 914
Quoted 1 line
net/sunrpc/cache.c +0
-0if (count >=sizeof (write_buf))
This is now commit 27a1e8a0f79e643d ("sunrpc: raise kernel RPC channel buffer size") upstream, and increases kernel size by 24 KiB, even if RPC is not used.
Can this buffer allocated dynamically instead? This code path seems to be a slow path anyway. If it's critical, perhaps this buffer can be allocated on first use?
Thanks!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Quoted 45 lines · Geert Uytterhoeven
Hi Bruce, Roberto,
Quoted 36 lines · J. Bruce Fields
Quoted 9 lines · Roberto Bergantinos Corpas
Its possible that using
AUTH_SYSand mountd manage-gids option a user may hit the 8k RPC channel buffer limit. This have been observed on field, causing unanswered RPCs on clients after mountd fails to write on channel :rpc.mountd[11231]:
auth_unix_gid: error writing replyUserland nfs-utils uses a buffer size of 32k (
RPC_CHAN_BUF_SIZE), so lets match those two.Thanks, applying.
That should allow about 4000 group memberships. If that doesn't do it then maybe it's time to rethink....
--b.
Quoted 18 lines
Signed-off-by Roberto Bergantinos Corpas <[email protected]>---
net/sunrpc/cache.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)net/sunrpc/cache.c +1
-1@@ -908,7 +908,7 @@ static ssize_t cache_do_downcall(char *kaddr, const char __user *buf,static ssize_t cache_slow_downcall (const char __user *buf,size_t count,struct cache_detail *cd) { -static char write_buf[8192 ];/* protected by queue_io_mutex */ +static char write_buf[32768 ];/* protected by queue_io_mutex */ ssize_t ret = -EINVAL ;Quoted 1 line
net/sunrpc/cache.c static ssize_t cache_do_downcall(char *kaddr, const char __user *buf, · line 914
Quoted 1 line
net/sunrpc/cache.c +0
-0if (count >=sizeof (write_buf))This is now commit
27a1e8a0f79e643d("sunrpc: raise kernel RPC channel buffer size") upstream, and increases kernel size by 24 KiB, even if RPC is not used.
Quoted 3 lines
Can this buffer allocated dynamically instead? This code path seems to be a slow path anyway. If it's critical, perhaps this buffer can be allocated on first use?
Sure. Actually it is using an allocated buffer typically, see cache_downcall().
Looking back at the history.... That was added by Trond in 2009 (da77005f0d64 "SUNRPC: Remove the global temporary write buffer in net/sunrpc/cache.c").
Before that there's a pre-git change from 2004 which replaced a simple kmalloc(PAGE_SIZE).
So I guess the point was to be careful about higher-order allocations, but probably it was overkill.
How about making it a kvmalloc?
--b.
Good point Geert !
Quoted 1 line
How about making it a kvmalloc?
I can post a new patch using kvmalloc, Bruce looks we can also prescind of queue_io_mutex, what do you think ?
Quoted 3 lines
--b.
Quoted 6 lines · Roberto Bergantinos Corpas
Good point Geert !
Quoted 1 line
How about making it a kvmalloc?
I can post a new patch using kvmalloc, Bruce looks we can also prescind of
queue_io_mutex, what do you think ?
And revert da77005f0d64, I think.
Maybe there's something I'm missing, but I don't think we need all that complexity.
--b.