NFSD: Don't start nfsd threads from a pool_threads write
Subsystems (2)
Chuck Lever
maintainerPATCH 1/1 NFSD: Don't start nfsd threads from a pool_threads write
Writing to pool_threads calls nfsd_set_nrthreads() directly, which starts nfsd threads as soon as a svc_serv exists. A portlist write creates the svc_serv without running nfsd_startup_net(), so a pool_threads write that follows it starts threads in a namespace whose NFSv4 state has not been set up. If the server in that namespace has been started and shut down before, the hash table pointers in struct nfsd_net still point at memory that nfs4_state_destroy_net() freed, and the first EXCHANGE_ID those threads handle reads it:
BUG: KASAN: slab-use-after-free in add_to_unconfirmed nfsd4_exchange_id nfsd4_proc_compound nfsd_dispatch
A namespace that has never been started dereferences NULL instead.
write_pool_threads() already starts nothing when no svc_serv exists, and its comment documents that only the threads file starts NFS. Ignore the thread counts in a pool_threads write while NFSD_NET_UP is clear as well, so a pool_threads write never starts threads before nfsd_startup_net() has run. The reply still reports the current thread count for each pool.
--- fs/nfsd/nfsctl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfsd/nfsctl.c +2 -1
@@ -497,7 +497,8 @@ static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)if (nthreads ==NULL ) goto out_free; -if (size >0 ) { +/* Threads started now would run without NFSv4 state. */ +if (size >0 &&test_bit (NFSD_NET_UP , &nn->flags)) {for (i =0 ; i < npools; i++) { rv =get_int (&mesg, &nthreads[i]);if (rv == -ENOENT )
Signature · 1 line
2.54.0
Quoted 37 lines · Chuck Lever
Writing to
pool_threadscallsnfsd_set_nrthreads()directly, which starts nfsd threads as soon as asvc_servexists. A portlist write creates thesvc_servwithout runningnfsd_startup_net(), so apool_threadswrite that follows it starts threads in a namespace whose NFSv4 state has not been set up. If the server in that namespace has been started and shut down before, the hash table pointers in structnfsd_netstill point at memory thatnfs4_state_destroy_net()freed, and the firstEXCHANGE_IDthose threads handle reads it:BUG: KASAN: slab-use-after-free in add_to_unconfirmed nfsd4_exchange_id nfsd4_proc_compound nfsd_dispatchA namespace that has never been started dereferences NULL instead.
write_pool_threads()already starts nothing when nosvc_servexists, and its comment documents that only the threads file starts NFS. Ignore the thread counts in apool_threadswrite whileNFSD_NET_UPis clear as well, so apool_threadswrite never starts threads beforenfsd_startup_net()has run. The reply still reports the current thread count for each pool.Fixes 0a7ec37727dc ("nfsd: make unconf_id_hashtbl allocated per net")Reported-by [email protected]Signed-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfsctl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)fs/nfsd/nfsctl.c +0
-0
fs/nfsd/nfsctl.c static ssize_t write_pool_threads(struct file *file, char *buf, size_t size) · line 504
Quoted 9 lines
fs/nfsd/nfsctl.c +2
-1@@ -497,7 +497,8 @@ static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)if (nthreads ==NULL ) goto out_free; -if (size >0 ) { +/* Threads started now would run without NFSv4 state. */ +if (size >0 &&test_bit (NFSD_NET_UP , &nn->flags)) {for (i =0 ; i < npools; i++) { rv =get_int (&mesg, &nthreads[i]);if (rv == -ENOENT )
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Chuck,
kernel test robot noticed the following build errors:
[auto build test ERROR on brauner-vfs/vfs.all]
[also build test ERROR on linus/master v7.3-rc6 next-20261008] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Chuck-Lever/NFSD-Don-t-start-nfsd-threads-from-a-pool_threads-write/20261009-104017 base: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git vfs.all patch link: https://lore.kernel.org/r/20261009144017.1284586-1-cel%40kernel.org patch subject: [PATCH v1] NFSD: Don't start nfsd threads from a pool_threads write config: parisc64-defconfig (https://download.01.org/0day-ci/archive/20261010/[email protected]/config) compiler: hppa64-linux-gcc (GCC) 16.1.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261010/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from include/linux/log2.h:12,
from include/asm-generic/div64.h:55,
from ./arch/parisc/include/generated/asm/div64.h:1,
from include/linux/math.h:6,
from include/linux/math64.h:6,
from include/linux/time64.h:5,
from include/linux/restart_block.h:9,
from include/linux/thread_info.h:14,
from include/asm-generic/preempt.h:5,
from ./arch/parisc/include/generated/asm/preempt.h:1,
from include/linux/preempt.h:107,
from include/linux/spinlock.h:56,
from include/linux/mmzone.h:8,
from include/linux/gfp.h:7,
from include/linux/slab.h:17,
from fs/nfsd/nfsctl.c:8:
fs/nfsd/nfsctl.c: In function 'write_pool_threads':
>> fs/nfsd/nfsctl.c:504:48: error: 'nn' undeclared (first use in this function)
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~
include/linux/bitops.h:44:44: note: in definition of macro 'bitop'
44 | __builtin_constant_p((uintptr_t)(addr) != (uintptr_t)NULL) && \
| ^~~~
fs/nfsd/nfsctl.c:504:25: note: in expansion of macro 'test_bit'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~~~~~~~
fs/nfsd/nfsctl.c:504:48: note: each undeclared identifier is reported only once for each function it appears in
include/linux/bitops.h:44:44: note: in definition of macro 'bitop'
44 | __builtin_constant_p((uintptr_t)(addr) != (uintptr_t)NULL) && \
| ^~~~
fs/nfsd/nfsctl.c:504:25: note: in expansion of macro 'test_bit'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~~~~~~~vim +/nn +504 fs/nfsd/nfsctl.c
449
450 /*
451 * write_pool_threads - Set or report the configured number of threads per pool
452 *
453 * Input:
454 * buf: ignored
455 * size: zero
456 *
457 * OR
458 *
459 * Input:
460 * buf: C string containing whitespace-
461 * separated unsigned integer values
462 * representing the number of NFSD
463 * threads to start in each pool
464 * size: non-zero length of C string in @buf
465 * Output:
466 * On success: passed-in buffer filled with '\n'-terminated C
467 * string containing integer values representing the
468 * configured number of NFSD threads in each pool;
469 * return code is the size in bytes of the string
470 * On error: return code is zero or a negative errno value
471 */
472 static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)
473 {
474 /* if size > 0, look for an array of number of threads per node
475 * and apply them then write out number of threads per node as reply
476 */
477 char *mesg = buf;
478 int i;
479 int rv;
480 int len;
481 int npools;
482 int *nthreads;
483 struct net *net = netns(file);
484
485 mutex_lock(&nfsd_mutex);
486 npools = nfsd_nrpools(net);
487 if (npools == 0) {
488 /*
489 * NFS is shut down. The admin can start it by
490 * writing to the threads file but NOT the pool_threads
491 * file, sorry. Report zero threads.
492 */
493 mutex_unlock(&nfsd_mutex);
494 strcpy(buf, "0\n");
495 return strlen(buf);
496 }
497
498 nthreads = kzalloc_objs(int, npools);
499 rv = -ENOMEM;
500 if (nthreads == NULL)
501 goto out_free;
502
503 /* Threads started now would run without NFSv4 state. */
> 504 if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) { 505 for (i = 0; i < npools; i++) {
506 rv = get_int(&mesg, &nthreads[i]);
507 if (rv == -ENOENT)
508 break; /* fewer numbers than pools */
509 if (rv)
510 goto out_free; /* syntax error */
511 rv = -EINVAL;
512 if (nthreads[i] < 0)
513 goto out_free;
514 trace_nfsd_ctl_pool_threads(net, i, nthreads[i]);
515 }
516
517 /*
518 * There must always be a thread in pool 0; the admin
519 * can't shut down NFS completely using pool_threads.
520 */
521 if (nthreads[0] == 0)
522 nthreads[0] = 1;
523
524 rv = nfsd_set_nrthreads(i, nthreads, net);
525 if (rv)
526 goto out_free;
527 }
528
529 rv = nfsd_get_nrthreads(npools, nthreads, net);
530 if (rv)
531 goto out_free;
532
533 mesg = buf;
534 size = SIMPLE_TRANSACTION_LIMIT;
535 for (i = 0; i < npools && size > 0; i++) {
536 snprintf(mesg, size, "%d%c", nthreads[i], (i == npools-1 ? '\n' : ' '));
537 len = strlen(mesg);
538 size -= len;
539 mesg += len;
540 }
541 rv = mesg - buf;
542 out_free:
543 kfree(nthreads);
544 mutex_unlock(&nfsd_mutex);
545 return rv;
546 }
547
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Chuck,
kernel test robot noticed the following build errors:
[auto build test ERROR on brauner-vfs/vfs.all]
[also build test ERROR on linus/master v7.3-rc6 next-20261008] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Chuck-Lever/NFSD-Don-t-start-nfsd-threads-from-a-pool_threads-write/20261009-104017 base: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git vfs.all patch link: https://lore.kernel.org/r/20261009144017.1284586-1-cel%40kernel.org patch subject: [PATCH v1] NFSD: Don't start nfsd threads from a pool_threads write config: loongarch-defconfig (https://download.01.org/0day-ci/archive/20261010/[email protected]/config) compiler: clang version 24.0.0git (https://github.com/llvm/llvm-project 242243a5814307de825db3428662d67b064e2cbc) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261010/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> fs/nfsd/nfsctl.c:504:41: error: use of undeclared identifier 'nn'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~
>> fs/nfsd/nfsctl.c:504:41: error: use of undeclared identifier 'nn'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~
>> fs/nfsd/nfsctl.c:504:41: error: use of undeclared identifier 'nn'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~
>> fs/nfsd/nfsctl.c:504:41: error: use of undeclared identifier 'nn'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~
>> fs/nfsd/nfsctl.c:504:41: error: use of undeclared identifier 'nn'
504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) {
| ^~5 errors generated.
vim +/nn +504 fs/nfsd/nfsctl.c
449
450 /*
451 * write_pool_threads - Set or report the configured number of threads per pool
452 *
453 * Input:
454 * buf: ignored
455 * size: zero
456 *
457 * OR
458 *
459 * Input:
460 * buf: C string containing whitespace-
461 * separated unsigned integer values
462 * representing the number of NFSD
463 * threads to start in each pool
464 * size: non-zero length of C string in @buf
465 * Output:
466 * On success: passed-in buffer filled with '\n'-terminated C
467 * string containing integer values representing the
468 * configured number of NFSD threads in each pool;
469 * return code is the size in bytes of the string
470 * On error: return code is zero or a negative errno value
471 */
472 static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)
473 {
474 /* if size > 0, look for an array of number of threads per node
475 * and apply them then write out number of threads per node as reply
476 */
477 char *mesg = buf;
478 int i;
479 int rv;
480 int len;
481 int npools;
482 int *nthreads;
483 struct net *net = netns(file);
484
485 mutex_lock(&nfsd_mutex);
486 npools = nfsd_nrpools(net);
487 if (npools == 0) {
488 /*
489 * NFS is shut down. The admin can start it by
490 * writing to the threads file but NOT the pool_threads
491 * file, sorry. Report zero threads.
492 */
493 mutex_unlock(&nfsd_mutex);
494 strcpy(buf, "0\n");
495 return strlen(buf);
496 }
497
498 nthreads = kzalloc_objs(int, npools);
499 rv = -ENOMEM;
500 if (nthreads == NULL)
501 goto out_free;
502
503 /* Threads started now would run without NFSv4 state. */
> 504 if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) { 505 for (i = 0; i < npools; i++) {
506 rv = get_int(&mesg, &nthreads[i]);
507 if (rv == -ENOENT)
508 break; /* fewer numbers than pools */
509 if (rv)
510 goto out_free; /* syntax error */
511 rv = -EINVAL;
512 if (nthreads[i] < 0)
513 goto out_free;
514 trace_nfsd_ctl_pool_threads(net, i, nthreads[i]);
515 }
516
517 /*
518 * There must always be a thread in pool 0; the admin
519 * can't shut down NFS completely using pool_threads.
520 */
521 if (nthreads[0] == 0)
522 nthreads[0] = 1;
523
524 rv = nfsd_set_nrthreads(i, nthreads, net);
525 if (rv)
526 goto out_free;
527 }
528
529 rv = nfsd_get_nrthreads(npools, nthreads, net);
530 if (rv)
531 goto out_free;
532
533 mesg = buf;
534 size = SIMPLE_TRANSACTION_LIMIT;
535 for (i = 0; i < npools && size > 0; i++) {
536 snprintf(mesg, size, "%d%c", nthreads[i], (i == npools-1 ? '\n' : ' '));
537 len = strlen(mesg);
538 size -= len;
539 mesg += len;
540 }
541 rv = mesg - buf;
542 out_free:
543 kfree(nthreads);
544 mutex_unlock(&nfsd_mutex);
545 return rv;
546 }
547
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Chuck Lever
maintainerQuoted 177 lines · kernel test robot
Hi Chuck,
kernel test robot noticed the following build errors:
[auto build test ERROR on brauner-vfs/vfs.all]
[also build test ERROR on linus/master v7.3-rc6 next-20261008] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]url:
https://github.com/intel-lab-lkp/linux/commits/Chuck-Lever/NFSD-Don-t-start-nfsd-threads-from-a-pool_threads-write/20261009-104017
base: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
vfs.all
patch link:
https://lore.kernel.org/r/20261009144017.1284586-1-cel%40kernel.org
patch subject: [PATCH v1] NFSD: Don't start nfsd threads from apool_threadswrite
config: parisc64-defconfig
(https://download.01.org/0day-ci/archive/20261010/[email protected]/config)
compiler: hppa64-linux-gcc (GCC) 16.1.0
reproduce (this is a W=1 build):
(https://download.01.org/0day-ci/archive/20261010/[email protected]/reproduce)If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes:
https://lore.kernel.org/oe-kbuild-all/[email protected]/All errors (new ones prefixed by >>):
In file included from include/linux/log2.h:12, from include/asm-generic/div64.h:55, from ./arch/parisc/include/generated/asm/div64.h:1, from include/linux/math.h:6, from include/linux/math64.h:6, from include/linux/time64.h:5, from include/linux/restart_block.h:9, from include/linux/thread_info.h:14, from include/asm-generic/preempt.h:5, from ./arch/parisc/include/generated/asm/preempt.h:1, from include/linux/preempt.h:107, from include/linux/spinlock.h:56, from include/linux/mmzone.h:8, from include/linux/gfp.h:7, from include/linux/slab.h:17, from fs/nfsd/nfsctl.c:8: fs/nfsd/nfsctl.c: In function 'write_pool_threads':Quoted 1 line
Quoted 1 line
fs/nfsd/nfsctl.c:504:48: error: 'nn' undeclared (first use in this function)504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) { | ^~ include/linux/bitops.h:44:44: note: in definition of macro 'bitop' 44 | __builtin_constant_p((uintptr_t)(addr) != (uintptr_t)NULL) && \ | ^~~~ fs/nfsd/nfsctl.c:504:25: note: in expansion of macro 'test_bit' 504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) { | ^~~~~~~~ fs/nfsd/nfsctl.c:504:48: note: each undeclared identifier is reported only once for each function it appears in include/linux/bitops.h:44:44: note: in definition of macro 'bitop' 44 | __builtin_constant_p((uintptr_t)(addr) != (uintptr_t)NULL) && \ | ^~~~ fs/nfsd/nfsctl.c:504:25: note: in expansion of macro 'test_bit' 504 | if (size > 0 && test_bit(NFSD_NET_UP, &nn->flags)) { | ^~~~~~~~vim +/nn +504
fs/nfsd/nfsctl.c449 450 /* 451 * write_pool_threads - Set or report the configured number of threads per pool 452 * 453 * Input: 454 * buf: ignored 455 * size: zero 456 * 457 * OR 458 * 459 * Input: 460 * buf: C string containing whitespace- 461 * separated unsigned integer values 462 * representing the number of NFSD 463 * threads to start in each pool 464 * size: non-zero length of C string in @buf 465 * Output: 466 * On success: passed-in buffer filled with '\n'-terminated C 467 * string containing integer values representing the 468 * configured number of NFSD threads in each pool; 469 * return code is the size in bytes of the string 470 * On error: return code is zero or a negative errno value 471 */ 472 static ssize_t write_pool_threads(struct file *file, char *buf,
size_tsize)
473 {
474 /* if size > 0, look for an array of number of threads per node
475 * and apply them then write out number of threads per node as
reply
476 */477 char *mesg = buf; 478 int i; 479 int rv; 480 int len; 481 int npools; 482 int *nthreads; 483 struct net *net = netns(file); 484 485 mutex_lock(&nfsd_mutex); 486 npools = nfsd_nrpools(net); 487 if (npools == 0) { 488 /* 489 * NFS is shut down. The admin can start it by 490 * writing to the threads file but NOT the pool_threads 491 * file, sorry. Report zero threads. 492 */ 493 mutex_unlock(&nfsd_mutex); 494 strcpy(buf, "0\n"); 495 return strlen(buf); 496 } 497 498 nthreads = kzalloc_objs(int, npools); 499 rv = -ENOMEM; 500 if (nthreads == NULL) 501 goto out_free; 502 503 /* Threads started now would run without NFSv4 state. */Quoted 1 line
504 if (size > 0 &&
test_bit(NFSD_NET_UP, &nn->flags)) {505 for (i = 0; i < npools; i++) { 506 rv = get_int(&mesg, &nthreads[i]); 507 if (rv == -ENOENT) 508 break; /* fewer numbers than pools */ 509 if (rv) 510 goto out_free; /* syntax error */ 511 rv = -EINVAL; 512 if (nthreads[i] < 0) 513 goto out_free; 514 trace_nfsd_ctl_pool_threads(net, i, nthreads[i]); 515 } 516 517 /* 518 * There must always be a thread in pool 0; the admin 519 * can't shut down NFS completely using pool_threads. 520 */ 521 if (nthreads[0] == 0) 522 nthreads[0] = 1; 523 524 rv = nfsd_set_nrthreads(i, nthreads, net); 525 if (rv) 526 goto out_free; 527 } 528 529 rv = nfsd_get_nrthreads(npools, nthreads, net); 530 if (rv) 531 goto out_free; 532 533 mesg = buf; 534 size = SIMPLE_TRANSACTION_LIMIT; 535 for (i = 0; i < npools && size > 0; i++) { 536 snprintf(mesg, size, "%d%c", nthreads[i], (i == npools-1 ? '\n' : ' ')); 537 len = strlen(mesg); 538 size -= len; 539 mesg += len; 540 } 541 rv = mesg - buf; 542 out_free: 543 kfree(nthreads); 544 mutex_unlock(&nfsd_mutex); 545 return rv; 546 } 547
Quoted 3 lines
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
The build bot applied this patch to trees that do not include the work that's pending in the cel/nfsd-testing branch. It applied cleanly, but those trees are missing the pre-requisite nfsd-per-net-mutex series from Jeff Layton.
Interestingly, sashiko reported the same failure yesterday, but I was not able to reproduce it because I didn't realize where it was trying to apply the patch.
Since this patch is marked Cc: stable, I'll make it easy to backport by sinking it on my nfsd-testing branch below the pre-requisites and then adjusting those patches.
Signature · 1 line
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)