Skip to contentSkip to conversation

NFS: Fix directory delegation verifier checks

PATCH In review Anna Schumaker 6 participants 14 messages Dec 19, 2025 – Apr 5

Subsystems (3)
lore
Share & tools
lore

Anna Schumaker

author maintainer
#0 lore raw

PATCH 1/1 NFS: Fix directory delegation verifier checks

From: Anna Schumaker <[email protected]>

Doing this check in nfs_check_verifier() resulted in many, many more lookups on the wire when running Christoph's delegation benchmarking script. After some experimentation, I found that we can treat directory delegations exactly the same as having a delegated verifier when we reach nfs4_lookup_revalidate() for the best performance.

Fixes 156b09482933 ("NFS: Request a directory delegation on ACCESS, CREATE, and UNLINK")
---
 fs/nfs/dir.c | 21 ++-------------------
 1 file changed, 2 insertions(+), 19 deletions(-)
fs/nfs/dir.c +2 -19
index 23a78a742b61..c0e9d5a45cd0 100644
@@ -1516,14 +1516,6 @@ static int nfs_check_verifier(struct inode *dir, struct dentry *dentry,
 	if (!nfs_dentry_verify_change(dir, dentry))
 		return 0;
 
-	/*
-	 * If we have a directory delegation then we don't need to revalidate
-	 * the directory. The delegation will either get recalled or we will
-	 * receive a notification when it changes.
-	 */
-	if (nfs_have_directory_delegation(dir))
-		return 0;
-
 	/* Revalidate nfsi->cache_change_attribute before we declare a match */
 	if (nfs_mapping_need_revalidate_inode(dir)) {
 		if (rcu_walk)
@@ -2216,13 +2208,6 @@ int nfs_atomic_open(struct inode *dir, struct dentry *dentry,
 }
 EXPORT_SYMBOL_GPL(nfs_atomic_open);
 
-static int
-nfs_lookup_revalidate_delegated_parent(struct inode *dir, struct dentry *dentry,
-				       struct inode *inode)
-{
-	return nfs_lookup_revalidate_done(dir, dentry, inode, 1);
-}
-
 static int
 nfs4_lookup_revalidate(struct inode *dir, const struct qstr *name,
 		       struct dentry *dentry, unsigned int flags)
@@ -2247,12 +2232,10 @@ nfs4_lookup_revalidate(struct inode *dir, const struct qstr *name,
 	if (inode == NULL)
 		goto full_reval;
 
-	if (nfs_verifier_is_delegated(dentry))
+	if (nfs_verifier_is_delegated(dentry) ||
+	    nfs_have_directory_delegation(inode))
 		return nfs_lookup_revalidate_delegated(dir, dentry, inode);
 
-	if (nfs_have_directory_delegation(dir))
-		return nfs_lookup_revalidate_delegated_parent(dir, dentry, inode);
-
 	/* NFS only supports OPEN on regular files */
 	if (!S_ISREG(inode->i_mode))
 		goto full_reval;
Signature · 1 line
2.52.0
Tested-by on v1 1/1
#1 lore raw
Quoted 11 lines · Anna Schumaker

From: Anna Schumaker <[email protected]>

Doing this check in nfs_check_verifier() resulted in many, many more lookups on the wire when running Christoph's delegation benchmarking script. After some experimentation, I found that we can treat directory delegations exactly the same as having a delegated verifier when we reach nfs4_lookup_revalidate() for the best performance.

Fixes 156b09482933 ("NFS: Request a directory delegation on ACCESS, CREATE, and UNLINK")

I wish I could actually review this, but I don't actually understand the lookup revalidation logic enough for that. But it does fix the problem I saw, so at least:

#2 lore raw
Quoted 3 lines · Christoph Hellwig

I wish I could actually review this, but I don't actually understand the lookup revalidation logic enough for that. But it does fix the problem I saw, so at least:

Actually - I have to take this back. This patch makes generic/786 when run on NFS v4.2 go from just failing with:

"Server reported failure (1)"

to actually crashing the kernel:

[   30.195133] run fstests generic/786 at 2025-12-23 01:04:02
[   36.956689] Oops: general protection fault, probably for non-canonical address 0xcccccccccccccd0c: 0000 [#1] SMP NOPTI
[   36.958741] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted: G                 N  6.19.0-rc2+ #4523 PREEMPT(full) 
[   36.960855] Tainted: [N]=TEST
[   36.961443] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   36.963167] RIP: 0010:nfs_end_delegation_return+0xda/0x390
[   36.964321] Code: 49 89 ce e8 18 c1 bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   36.967856] RSP: 0018:ffffc900018e7cc0 EFLAGS: 00010286
[   36.968862] RAX: ffff88810efc5e20 RBX: ffff888104fa6970 RCX: ffff888112570170
[   36.970200] RDX: ffff888105780040 RSI: ffff8881189120e8 RDI: ffff888112570210
[   36.971676] RBP: ffff888112570210 R08: 0000000000000000 R09: 0000000000000000
[   36.973041] R10: ffffc900018e7d78 R11: fefefefefefefeff R12: cccccccccccccccc
[   36.974313] R13: ffff88810efc5da0 R14: ffff888112570170 R15: ffff888104fa6940
[   36.975635] FS:  00007f0df5dc4740(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   36.977106] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   36.978187] CR2: 00007f0df5e26450 CR3: 0000000118e1b001 CR4: 0000000000772ef0
[   36.979629] PKRU: 55555554
[   36.980205] Call Trace:
[   36.980706]  <TASK>
[   36.981131]  ? nfs_clear_verifier_delegated+0x50/0x70
[   36.982109]  nfs4_proc_setattr+0xff/0x110
[   36.982894]  nfs_setattr+0x1c8/0x410
[   36.983634]  notify_change+0x373/0x510
[   36.984376]  ? chmod_common+0xad/0x160
[   36.985055]  chmod_common+0xad/0x160
[   36.985702]  __x64_sys_chmod+0x56/0xb0
[   36.986388]  do_syscall_64+0x50/0xf80
[   36.987097]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   36.988171] RIP: 0033:0x7f0df5ec6707
[   36.988880] Code: 73 01 c3 48 8b 0d f1 76 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 5a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 c1 76 0e 00 f7 d8 64 89 02 b8
[   36.992424] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX: 000000000000005a
[   36.993747] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX: 00007f0df5ec6707
[   36.995045] RDX: 0000000000000008 RSI: 00000000000001fd RDI: 00007fff3cd5cbc0
[   36.996499] RBP: 0000000000000000 R08: 00000000000001fd R09: 0000000000000000
[   36.997855] R10: 0000000000000003 R11: 0000000000000206 R12: 0000000000000035
[   36.999218] R13: 0000000000000007 R14: 0000000000000000 R15: 0000558071f1ebc0
[   37.000618]  </TASK>
[   37.001039] Modules linked in: kvm_intel kvm irqbypass
[   37.002029] ---[ end trace 0000000000000000 ]---
[   37.003342] RIP: 0010:nfs_end_delegation_return+0xda/0x390
[   37.004372] Code: 49 89 ce e8 18 c1 bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   37.007875] RSP: 0018:ffffc900018e7cc0 EFLAGS: 00010286
[   37.008902] RAX: ffff88810efc5e20 RBX: ffff888104fa6970 RCX: ffff888112570170
[   37.010307] RDX: ffff888105780040 RSI: ffff8881189120e8 RDI: ffff888112570210
[   37.011702] RBP: ffff888112570210 R08: 0000000000000000 R09: 0000000000000000
[   37.012975] R10: ffffc900018e7d78 R11: fefefefefefefeff R12: cccccccccccccccc
[   37.014271] R13: ffff88810efc5da0 R14: ffff888112570170 R15: ffff888104fa6940
[   37.015600] FS:  00007f0df5dc4740(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   37.017070] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   37.018190] CR2: 00007f0df5e26450 CR3: 0000000118e1b002 CR4: 0000000000772ef0
[   37.019525] PKRU: 55555554
[   37.020028] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:287
[   37.021516] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid: 3837, name: locktest
[   37.022851] preempt_count: 0, expected: 0
[   37.023565] RCU nest depth: 1, expected: 0
[   37.024235] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted: G      D          N  6.19.0-rc2+ #4523 PREEMPT(full) 
[   37.024240] Tainted: [D]=DIE, [N]=TEST
[   37.024241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   37.024242] Call Trace:
[   37.024243]  <TASK>
[   37.024244]  dump_stack_lvl+0x4b/0x70
[   37.024250]  __might_resched.cold+0xd3/0x10e
[   37.024255]  mutex_lock+0x19/0x80
[   37.024259]  sched_mm_cid_exit+0x51/0x1e0
[   37.024263]  do_exit+0xb0/0xa40
[   37.024265]  ? __x64_sys_chmod+0x56/0xb0
[   37.024269]  make_task_dead+0x87/0x90
[   37.024271]  rewind_stack_and_make_dead+0x16/0x20
[   37.024275] RIP: 0033:0x7f0df5ec6707
[   37.024277] Code: 73 01 c3 48 8b 0d f1 76 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 5a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 c1 76 0e 00 f7 d8 64 89 02 b8
[   37.024278] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX: 000000000000005a
[   37.024280] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX: 00007f0df5ec6707
[   37.024281] RDX: 0000000000000008 RSI: 00000000000001fd RDI: 00007fff3cd5cbc0
[   37.024282] RBP: 0000000000000000 R08: 00000000000001fd R09: 0000000000000000
[   37.024283] R10: 0000000000000003 R11: 0000000000000206 R12: 0000000000000035
[   37.024284] R13: 0000000000000007 R14: 0000000000000000 R15: 0000558071f1ebc0
[   37.024286]  </TASK>
[   37.024761] ------------[ cut here ]------------
[   37.041739] Voluntary context switch within RCU read-side critical section!
[   37.041740] WARNING: kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x39b/0x5e0, CPU#0: locktest/3837
[   37.043557] Modules linked in: kvm_intel kvm irqbypass
[   37.044135] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted: G      D W        N  6.19.0-rc2+ #4523 PREEMPT(full) 
[   37.045192] Tainted: [D]=DIE, [W]=WARN, [N]=TEST
[   37.045680] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   37.046655] RIP: 0010:rcu_note_context_switch+0x39b/0x5e0
[   37.047211] Code: ef e8 f9 a7 34 01 c6 45 11 00 48 8b 55 28 4c 89 ef 48 89 c6 b9 01 00 00 00 e8 91 97 ff ff e9 d6 fc ff ff 48 8d 3d 25 4f 86 02 <67> 48 0f b9 3a e9 a6 fc ff ff 4c 8b 6d 20 4c 89 ef e8 bf a7 34 01
[   37.049166] RSP: 0018:ffffc900018e7c18 EFLAGS: 00010002
[   37.049714] RAX: 0000000000000001 RBX: ffff888105780040 RCX: 0000000000000000
[   37.050451] RDX: 0000000000000001 RSI: ffffffff8304f78e RDI: ffffffff83c34c20
[   37.051193] RBP: ffff888237c2a300 R08: ffff88810c4208f7 R09: 0000000000000002
[   37.051957] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
[   37.052702] R13: ffffffff846e55c0 R14: ffff888105780040 R15: ffffffff82608cb0
[   37.053437] FS:  0000000000000000(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   37.054272] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   37.054876] CR2: 00007f0df5e26450 CR3: 000000000325a003 CR4: 0000000000772ef0
[   37.055627] PKRU: 55555554
[   37.055892] Call Trace:
[   37.056138]  <TASK>
[   37.056349]  ? _raw_spin_unlock_irqrestore+0x1d/0x40
[   37.056819]  ? __pfx_rpc_wait_bit_killable+0x10/0x10
[   37.057291]  __schedule+0xa1/0xe30
[   37.057622]  ? preempt_count_add+0x73/0xb0
[   37.058013]  ? __pfx_rpc_wait_bit_killable+0x10/0x10
[   37.058494]  schedule+0x29/0xe0
[   37.058839]  rpc_wait_bit_killable+0xc/0x60
[   37.059285]  __wait_on_bit+0x2c/0x90
[   37.059740]  out_of_line_wait_on_bit+0x8e/0xb0
[   37.060210]  ? __pfx_wake_bit_function+0x10/0x10
[   37.060700]  nfs4_do_close+0x289/0x310
[   37.061101]  __put_nfs_open_context+0xc9/0x140
[   37.061572]  nfs_file_release+0x38/0x50
[   37.061980]  ? security_file_release+0x17/0x30
[   37.062442]  __fput+0xf2/0x2b0
[   37.062773]  task_work_run+0x57/0xa0
[   37.063153]  do_exit+0x273/0xa40
[   37.063524]  ? __x64_sys_chmod+0x56/0xb0
[   37.063937]  make_task_dead+0x87/0x90
[   37.064331]  rewind_stack_and_make_dead+0x16/0x20
[   37.064819] RIP: 0033:0x7f0df5ec6707
[   37.065200] Code: Unable to access opcode bytes at 0x7f0df5ec66dd.
[   37.065828] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX: 000000000000005a
[   37.066601] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX: 00007f0df5ec6707
[   37.067328] RDX: 0000000000000008 RSI: 00000000000001fd RDI: 00007fff3cd5cbc0
[   37.068095] RBP: 0000000000000000 R08: 00000000000001fd R09: 0000000000000000
[   37.068848] R10: 0000000000000003 R11: 0000000000000206 R12: 0000000000000035
[   37.069585] R13: 0000000000000007 R14: 0000000000000000 R15: 0000558071f1ebc0
[   37.070319]  </TASK>
[   37.070560] ---[ end trace 0000000000000000 ]---
#3 lore raw

Hi Christoph,

Quoted 198 lines · Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

I wish I could actually review this, but I don't actually understand
the lookup revalidation logic enough for that.  But it does fix the problem I saw, so at least:

Actually - I have to take this back.  This patch makes generic/786 when run on NFS v4.2 go from just failing with:

"Server reported failure (1)"

to actually crashing the kernel:

[   30.195133] run fstests generic/786 at 2025-12-23 01:04:02 [   36.956689] Oops: general protection fault, probably for non- canonical address 0xcccccccccccccd0c: 0000 [#1] SMP NOPTI [   36.958741] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted:
G                 N  6.19.0-rc2+ #4523 PREEMPT(full)
[   36.960855] Tainted: [N]=TEST
[   36.961443] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   36.963167] RIP: 0010:nfs_end_delegation_return+0xda/0x390 [   36.964321] Code: 49 89 ce e8 18 c1 bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   36.967856] RSP: 0018:ffffc900018e7cc0 EFLAGS: 00010286 [   36.968862] RAX: ffff88810efc5e20 RBX: ffff888104fa6970 RCX:
ffff888112570170
[   36.970200] RDX: ffff888105780040 RSI: ffff8881189120e8 RDI:
ffff888112570210
[   36.971676] RBP: ffff888112570210 R08: 0000000000000000 R09:
0000000000000000
[   36.973041] R10: ffffc900018e7d78 R11: fefefefefefefeff R12:
cccccccccccccccc
[   36.974313] R13: ffff88810efc5da0 R14: ffff888112570170 R15:
ffff888104fa6940
[   36.975635] FS:  00007f0df5dc4740(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   36.977106] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   36.978187] CR2: 00007f0df5e26450 CR3: 0000000118e1b001 CR4:
0000000000772ef0
[   36.979629] PKRU: 55555554
[   36.980205] Call Trace:
[   36.980706]  <TASK>
[   36.981131]  ? nfs_clear_verifier_delegated+0x50/0x70
[   36.982109]  nfs4_proc_setattr+0xff/0x110
[   36.982894]  nfs_setattr+0x1c8/0x410
[   36.983634]  notify_change+0x373/0x510
[   36.984376]  ? chmod_common+0xad/0x160
[   36.985055]  chmod_common+0xad/0x160
[   36.985702]  __x64_sys_chmod+0x56/0xb0
[   36.986388]  do_syscall_64+0x50/0xf80
[   36.987097]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   36.988171] RIP: 0033:0x7f0df5ec6707
[   36.988880] Code: 73 01 c3 48 8b 0d f1 76 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 5a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 c1 76 0e 00 f7 d8 64 89 02 b8
[   36.992424] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX:
000000000000005a
[   36.993747] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX:
00007f0df5ec6707
[   36.995045] RDX: 0000000000000008 RSI: 00000000000001fd RDI:
00007fff3cd5cbc0
[   36.996499] RBP: 0000000000000000 R08: 00000000000001fd R09:
0000000000000000
[   36.997855] R10: 0000000000000003 R11: 0000000000000206 R12:
0000000000000035
[   36.999218] R13: 0000000000000007 R14: 0000000000000000 R15:
0000558071f1ebc0
[   37.000618]  </TASK>
[   37.001039] Modules linked in: kvm_intel kvm irqbypass [   37.002029] ---[ end trace 0000000000000000 ]---
[   37.003342] RIP: 0010:nfs_end_delegation_return+0xda/0x390 [   37.004372] Code: 49 89 ce e8 18 c1 bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   37.007875] RSP: 0018:ffffc900018e7cc0 EFLAGS: 00010286 [   37.008902] RAX: ffff88810efc5e20 RBX: ffff888104fa6970 RCX:
ffff888112570170
[   37.010307] RDX: ffff888105780040 RSI: ffff8881189120e8 RDI:
ffff888112570210
[   37.011702] RBP: ffff888112570210 R08: 0000000000000000 R09:
0000000000000000
[   37.012975] R10: ffffc900018e7d78 R11: fefefefefefefeff R12:
cccccccccccccccc
[   37.014271] R13: ffff88810efc5da0 R14: ffff888112570170 R15:
ffff888104fa6940
[   37.015600] FS:  00007f0df5dc4740(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   37.017070] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   37.018190] CR2: 00007f0df5e26450 CR3: 0000000118e1b002 CR4:
0000000000772ef0
[   37.019525] PKRU: 55555554
[   37.020028] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:287
[   37.021516] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid:
3837, name: locktest
[   37.022851] preempt_count: 0, expected: 0
[   37.023565] RCU nest depth: 1, expected: 0
[   37.024235] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted: G D          N  6.19.0-rc2+ #4523 PREEMPT(full)
[   37.024240] Tainted: [D]=DIE, [N]=TEST
[   37.024241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   37.024242] Call Trace:
[   37.024243]  <TASK>
[   37.024244]  dump_stack_lvl+0x4b/0x70
[   37.024250]  __might_resched.cold+0xd3/0x10e
[   37.024255]  mutex_lock+0x19/0x80
[   37.024259]  sched_mm_cid_exit+0x51/0x1e0
[   37.024263]  do_exit+0xb0/0xa40
[   37.024265]  ? __x64_sys_chmod+0x56/0xb0
[   37.024269]  make_task_dead+0x87/0x90
[   37.024271]  rewind_stack_and_make_dead+0x16/0x20
[   37.024275] RIP: 0033:0x7f0df5ec6707
[   37.024277] Code: 73 01 c3 48 8b 0d f1 76 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 5a 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 c1 76 0e 00 f7 d8 64 89 02 b8
[   37.024278] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX:
000000000000005a
[   37.024280] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX:
00007f0df5ec6707
[   37.024281] RDX: 0000000000000008 RSI: 00000000000001fd RDI:
00007fff3cd5cbc0
[   37.024282] RBP: 0000000000000000 R08: 00000000000001fd R09:
0000000000000000
[   37.024283] R10: 0000000000000003 R11: 0000000000000206 R12:
0000000000000035
[   37.024284] R13: 0000000000000007 R14: 0000000000000000 R15:
0000558071f1ebc0
[   37.024286]  </TASK>
[   37.024761] ------------[ cut here ]------------
[   37.041739] Voluntary context switch within RCU read-side critical section!
[   37.041740] WARNING: kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x39b/0x5e0, CPU#0: locktest/3837 [   37.043557] Modules linked in: kvm_intel kvm irqbypass [   37.044135] CPU: 0 UID: 0 PID: 3837 Comm: locktest Tainted: G D W        N  6.19.0-rc2+ #4523 PREEMPT(full)
[   37.045192] Tainted: [D]=DIE, [W]=WARN, [N]=TEST
[   37.045680] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   37.046655] RIP: 0010:rcu_note_context_switch+0x39b/0x5e0 [   37.047211] Code: ef e8 f9 a7 34 01 c6 45 11 00 48 8b 55 28 4c 89 ef 48 89 c6 b9 01 00 00 00 e8 91 97 ff ff e9 d6 fc ff ff 48 8d 3d 25 4f 86 02 <67> 48 0f b9 3a e9 a6 fc ff ff 4c 8b 6d 20 4c 89 ef e8 bf a7 34 01
[   37.049166] RSP: 0018:ffffc900018e7c18 EFLAGS: 00010002 [   37.049714] RAX: 0000000000000001 RBX: ffff888105780040 RCX:
0000000000000000
[   37.050451] RDX: 0000000000000001 RSI: ffffffff8304f78e RDI:
ffffffff83c34c20
[   37.051193] RBP: ffff888237c2a300 R08: ffff88810c4208f7 R09:
0000000000000002
[   37.051957] R10: 0000000000000001 R11: 0000000000000001 R12:
0000000000000000
[   37.052702] R13: ffffffff846e55c0 R14: ffff888105780040 R15:
ffffffff82608cb0
[   37.053437] FS:  0000000000000000(0000) GS:ffff8882b3544000(0000) knlGS:0000000000000000
[   37.054272] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   37.054876] CR2: 00007f0df5e26450 CR3: 000000000325a003 CR4:
0000000000772ef0
[   37.055627] PKRU: 55555554
[   37.055892] Call Trace:
[   37.056138]  <TASK>
[   37.056349]  ? _raw_spin_unlock_irqrestore+0x1d/0x40
[   37.056819]  ? __pfx_rpc_wait_bit_killable+0x10/0x10
[   37.057291]  __schedule+0xa1/0xe30
[   37.057622]  ? preempt_count_add+0x73/0xb0
[   37.058013]  ? __pfx_rpc_wait_bit_killable+0x10/0x10
[   37.058494]  schedule+0x29/0xe0
[   37.058839]  rpc_wait_bit_killable+0xc/0x60
[   37.059285]  __wait_on_bit+0x2c/0x90
[   37.059740]  out_of_line_wait_on_bit+0x8e/0xb0
[   37.060210]  ? __pfx_wake_bit_function+0x10/0x10
[   37.060700]  nfs4_do_close+0x289/0x310
[   37.061101]  __put_nfs_open_context+0xc9/0x140
[   37.061572]  nfs_file_release+0x38/0x50
[   37.061980]  ? security_file_release+0x17/0x30
[   37.062442]  __fput+0xf2/0x2b0
[   37.062773]  task_work_run+0x57/0xa0
[   37.063153]  do_exit+0x273/0xa40
[   37.063524]  ? __x64_sys_chmod+0x56/0xb0
[   37.063937]  make_task_dead+0x87/0x90
[   37.064331]  rewind_stack_and_make_dead+0x16/0x20
[   37.064819] RIP: 0033:0x7f0df5ec6707
[   37.065200] Code: Unable to access opcode bytes at 0x7f0df5ec66dd. [   37.065828] RSP: 002b:00007fff3cd5bad8 EFLAGS: 00000206 ORIG_RAX:
000000000000005a
[   37.066601] RAX: ffffffffffffffda RBX: 0000558071f19b90 RCX:
00007f0df5ec6707
[   37.067328] RDX: 0000000000000008 RSI: 00000000000001fd RDI:
00007fff3cd5cbc0

Quoted 9 lines

[   37.068095] RBP: 0000000000000000 R08: 00000000000001fd R09:
0000000000000000
[   37.068848] R10: 0000000000000003 R11: 0000000000000206 R12:
0000000000000035
[   37.069585] R13: 0000000000000007 R14: 0000000000000000 R15:
0000558071f1ebc0
[   37.070319]  </TASK>
[   37.070560] ---[ end trace 0000000000000000 ]---

Does applying the following on top of Anna's patch fix the Oops?

Cheers
Trond

8<--------------------------------------------------------------
From 3485794e16f68659b26918ccb1c57b6210bd0048 Mon Sep 17 00:00:00 2001
Message-ID: <3485794e16f68659b26918ccb1c57b6210bd0048.1767217813.git.trond.myklebust@hammerspace.com>
From: Trond Myklebust <[email protected]>
Date: Wed, 31 Dec 2025 16:41:15 -0500
Subject: [PATCH] NFSv4: Fix nfs_clear_verifier_delegated() for delegated
directories

If the client returns a directory delegation, then look up all the child dentries, and clear their 'verifier delegated' bit, unless subject to a file delegation.

Similarly, if a file delegation is being returned, check if there is a directory delegation before clearing a 'verifier delegated' bit.

Fixes 156b09482933 ("NFS: Request a directory delegation on ACCESS, CREATE, and UNLINK")
---
 fs/nfs/dir.c | 59 +++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 51 insertions(+), 8 deletions(-)
fs/nfs/dir.c +51 -8
index 23a78a742b61..d6222c6c3165 100644
@@ -1440,7 +1440,8 @@ static void nfs_set_verifier_locked(struct dentry *dentry, unsigned long verf)
 
 	if (!dir || !nfs_verify_change_attribute(dir, verf))
 		return;
-	if (inode && NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
+	if (NFS_PROTO(dir)->have_delegation(dir, FMODE_READ, 0) ||
+	    (inode && NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0)))
 		nfs_set_verifier_delegated(&verf);
 	dentry->d_time = verf;
 }
@@ -1465,6 +1466,49 @@ void nfs_set_verifier(struct dentry *dentry, unsigned long verf)
 EXPORT_SYMBOL_GPL(nfs_set_verifier);
 
 #if IS_ENABLED(CONFIG_NFS_V4)
+static void nfs_clear_verifier_file(struct inode *inode)
+{
+	struct dentry *alias;
+	struct inode *dir;
+
+	hlist_for_each_entry(alias, &inode->i_dentry, d_u.d_alias) {
+		spin_lock(&alias->d_lock);
+		dir = d_inode_rcu(alias->d_parent);
+		if (!dir ||
+		    !NFS_PROTO(dir)->have_delegation(dir, FMODE_READ, 0))
+			nfs_unset_verifier_delegated(&alias->d_time);
+		spin_unlock(&alias->d_lock);
+	}
+}
+
+static void nfs_clear_verifier_directory(struct inode *dir)
+{
+	struct dentry *this_parent;
+	struct dentry *dentry;
+	struct inode *inode;
+
+	if (hlist_empty(&dir->i_dentry))
+		return;
+	this_parent =
+		hlist_entry(dir->i_dentry.first, struct dentry, d_u.d_alias);
+
+	spin_lock(&this_parent->d_lock);
+	nfs_unset_verifier_delegated(&this_parent->d_time);
+	dentry = d_first_child(this_parent);
+	hlist_for_each_entry_from(dentry, d_sib) {
+		if (unlikely(dentry->d_flags & DCACHE_DENTRY_CURSOR))
+			continue;
+		inode = d_inode_rcu(dentry);
+		if (inode &&
+		    NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
+			continue;
+		spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
+		nfs_unset_verifier_delegated(&dentry->d_time);
+		spin_unlock(&dentry->d_lock);
+	}
+	spin_unlock(&this_parent->d_lock);
+}
+
 /**
  * nfs_clear_verifier_delegated - clear the dir verifier delegation tag
  * @inode: pointer to inode
@@ -1477,17 +1521,16 @@ EXPORT_SYMBOL_GPL(nfs_set_verifier);
  */
 void nfs_clear_verifier_delegated(struct inode *inode)
 {
-	struct dentry *alias;
-
 	if (!inode)
 		return;
+	rcu_read_lock();
 	spin_lock(&inode->i_lock);
-	hlist_for_each_entry(alias, &inode->i_dentry, d_u.d_alias) {
-		spin_lock(&alias->d_lock);
-		nfs_unset_verifier_delegated(&alias->d_time);
-		spin_unlock(&alias->d_lock);
-	}
+	if (S_ISREG(inode->i_mode))
+		nfs_clear_verifier_file(inode);
+	else if (S_ISDIR(inode->i_mode))
+		nfs_clear_verifier_directory(inode);
 	spin_unlock(&inode->i_lock);
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(nfs_clear_verifier_delegated);
 #endif /* IS_ENABLED(CONFIG_NFS_V4) */
Signature · 1 line
2.52.0
#4 lore raw
Quoted 1 line · Trond Myklebust

Does applying the following on top of Anna's patch fix the Oops?

It does. But now generic/633 crashes reliably:

generic/633 2s ... [ 58.670535] run fstests generic/633 at 2026-01-02 02:00:17

[   58.865568] process 'vfstest' launched '/dev/fd/4/file1' with NULL argv: empty string added
[   58.897522] Oops: general protection fault, probably for non-canonical address 0xcccccccccccccd0c: 0000 [#1] SMP NOPTI
[   58.898234] CPU: 0 UID: 0 PID: 3852 Comm: vfstest Tainted: G                 N  6.19.0-rc2+ #4535 PREEMPT(full) 
[   58.898829] Tainted: [N]=TEST
[   58.899013] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   58.899594] RIP: 0010:nfs_end_delegation_return+0xda/0x390
[   58.899922] Code: 49 89 ce e8 58 be bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   58.901063] RSP: 0018:ffffc90001947c90 EFLAGS: 00010286
[   58.901419] RAX: ffff88811a3122c0 RBX: ffff888105268970 RCX: ffff888111a9a210
[   58.901827] RDX: ffff8881039320c0 RSI: ffff888105268940 RDI: ffff888111a9a2b0
[   58.902236] RBP: ffff888111a9a2b0 R08: 0000000000000000 R09: 0000000000000000
[   58.902696] R10: ffffc90001947d48 R11: fefefefefefefeff R12: cccccccccccccccc
[   58.903112] R13: ffff88811a312240 R14: ffff888111a9a210 R15: ffff888105268940
[   58.903594] FS:  00007f04f92c6740(0000) GS:ffff8882b353d000(0000) knlGS:0000000000000000
[   58.904125] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.904483] CR2: 000055c55fe522d8 CR3: 000000011a3f1004 CR4: 0000000000772ef0
[   58.904927] PKRU: 55555554
[   58.905132] Call Trace:
[   58.905294]  <TASK>
[   58.905434]  ? _raw_spin_unlock+0x13/0x30
[   58.905689]  nfs4_proc_setattr+0xff/0x110
[   58.905947]  nfs_setattr+0x1c8/0x410
[   58.906175]  notify_change+0x373/0x510
[   58.906415]  ? init_object+0x5a/0xc0
[   58.906643]  ? chown_common+0x1ec/0x220
[   58.906885]  chown_common+0x1ec/0x220
[   58.907126]  do_fchownat+0xc3/0xf0
[   58.907358]  __x64_sys_fchownat+0x1a/0x30
[   58.907611]  do_syscall_64+0x50/0xf80
[   58.907848]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   58.908171] RIP: 0033:0x7f04f93c8e4a
[   58.908390] Code: 48 8b 0d b1 6f 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 04 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7e 6f 0e 00 f7 d8 64 89 01 48
[   58.909467] RSP: 002b:00007ffe4cd3ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000104
[   58.909893] RAX: ffffffffffffffda RBX: 00007ffe4cd3ecb0 RCX: 00007f04f93c8e4a
[   58.910336] RDX: 0000000000002710 RSI: 000055c54836500e RDI: 0000000000000003
[   58.910872] RBP: 000055c55fd522a0 R08: 0000000000000100 R09: 0000000000000001
[   58.911350] R10: 0000000000002710 R11: 0000000000000246 R12: 0000000000000006
[   58.911797] R13: 0000000000002710 R14: 0000000000002710 R15: 000055c55fd52313
[   58.912234]  </TASK>
[   58.912373] Modules linked in: kvm_intel kvm irqbypass
[   58.912717] ---[ end trace 0000000000000000 ]---
[   58.913829] RIP: 0010:nfs_end_delegation_return+0xda/0x390
[   58.914177] Code: 49 89 ce e8 58 be bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   58.915541] RSP: 0018:ffffc90001947c90 EFLAGS: 00010286
[   58.915871] RAX: ffff88811a3122c0 RBX: ffff888105268970 RCX: ffff888111a9a210
[   58.916317] RDX: ffff8881039320c0 RSI: ffff888105268940 RDI: ffff888111a9a2b0
[   58.916766] RBP: ffff888111a9a2b0 R08: 0000000000000000 R09: 0000000000000000
[   58.917299] R10: ffffc90001947d48 R11: fefefefefefefeff R12: cccccccccccccccc
[   58.917709] R13: ffff88811a312240 R14: ffff888111a9a210 R15: ffff888105268940
[   58.918109] FS:  00007f04f92c6740(0000) GS:ffff8882b353d000(0000) knlGS:0000000000000000
[   58.918560] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   58.918944] CR2: 00007fbf39569000 CR3: 000000011a3f1004 CR4: 0000000000772ef0
[   58.919622] PKRU: 55555554
maintainer
#5 lore raw
Quoted 85 lines · On Mon, 2026-01-05 at 22:21 -0800, [email protected] wrote:
Quoted 1 line · Trond Myklebust

Does applying the following on top of Anna's patch fix the Oops?

It does.  But now generic/633 crashes reliably:

generic/633  2s ... [   58.670535] run fstests generic/633 at 2026- 01-02 02:00:17
[   58.865568] process 'vfstest' launched '/dev/fd/4/file1' with NULL
argv: empty string added
[   58.897522] Oops: general protection fault, probably for non- canonical address 0xcccccccccccccd0c: 0000 [#1] SMP NOPTI [   58.898234] CPU: 0 UID: 0 PID: 3852 Comm: vfstest Tainted:
G                 N  6.19.0-rc2+ #4535 PREEMPT(full)
[   58.898829] Tainted: [N]=TEST
[   58.899013] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   58.899594] RIP: 0010:nfs_end_delegation_return+0xda/0x390 [   58.899922] Code: 49 89 ce e8 58 be bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   58.901063] RSP: 0018:ffffc90001947c90 EFLAGS: 00010286 [   58.901419] RAX: ffff88811a3122c0 RBX: ffff888105268970 RCX:
ffff888111a9a210
[   58.901827] RDX: ffff8881039320c0 RSI: ffff888105268940 RDI:
ffff888111a9a2b0
[   58.902236] RBP: ffff888111a9a2b0 R08: 0000000000000000 R09:
0000000000000000
[   58.902696] R10: ffffc90001947d48 R11: fefefefefefefeff R12:
cccccccccccccccc
[   58.903112] R13: ffff88811a312240 R14: ffff888111a9a210 R15:
ffff888105268940
[   58.903594] FS:  00007f04f92c6740(0000) GS:ffff8882b353d000(0000) knlGS:0000000000000000
[   58.904125] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   58.904483] CR2: 000055c55fe522d8 CR3: 000000011a3f1004 CR4:
0000000000772ef0
[   58.904927] PKRU: 55555554
[   58.905132] Call Trace:
[   58.905294]  <TASK>
[   58.905434]  ? _raw_spin_unlock+0x13/0x30
[   58.905689]  nfs4_proc_setattr+0xff/0x110
[   58.905947]  nfs_setattr+0x1c8/0x410
[   58.906175]  notify_change+0x373/0x510
[   58.906415]  ? init_object+0x5a/0xc0
[   58.906643]  ? chown_common+0x1ec/0x220
[   58.906885]  chown_common+0x1ec/0x220
[   58.907126]  do_fchownat+0xc3/0xf0
[   58.907358]  __x64_sys_fchownat+0x1a/0x30
[   58.907611]  do_syscall_64+0x50/0xf80
[   58.907848]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   58.908171] RIP: 0033:0x7f04f93c8e4a
[   58.908390] Code: 48 8b 0d b1 6f 0e 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 04 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 7e 6f 0e 00 f7 d8 64 89 01 48
[   58.909467] RSP: 002b:00007ffe4cd3ec98 EFLAGS: 00000246 ORIG_RAX:
0000000000000104
[   58.909893] RAX: ffffffffffffffda RBX: 00007ffe4cd3ecb0 RCX:
00007f04f93c8e4a
[   58.910336] RDX: 0000000000002710 RSI: 000055c54836500e RDI:
0000000000000003
[   58.910872] RBP: 000055c55fd522a0 R08: 0000000000000100 R09:
0000000000000001
[   58.911350] R10: 0000000000002710 R11: 0000000000000246 R12:
0000000000000006
[   58.911797] R13: 0000000000002710 R14: 0000000000002710 R15:
000055c55fd52313
[   58.912234]  </TASK>
[   58.912373] Modules linked in: kvm_intel kvm irqbypass [   58.912717] ---[ end trace 0000000000000000 ]---
[   58.913829] RIP: 0010:nfs_end_delegation_return+0xda/0x390 [   58.914177] Code: 49 89 ce e8 58 be bf ff 48 8b 85 60 ff ff ff 4c 8d 68 80 49 39 c6 74 50 4c 8b 7c 24 08 48 8b 1c 24 4d 8b 65 60 4d 85 e4 74 2e <49> 8b 44 24 40 a8 02 74 25 49 8b 44 24 40 f6 c4 02 75 1b 41 8b 47
[   58.915541] RSP: 0018:ffffc90001947c90 EFLAGS: 00010286 [   58.915871] RAX: ffff88811a3122c0 RBX: ffff888105268970 RCX:
ffff888111a9a210
[   58.916317] RDX: ffff8881039320c0 RSI: ffff888105268940 RDI:
ffff888111a9a2b0
[   58.916766] RBP: ffff888111a9a2b0 R08: 0000000000000000 R09:
0000000000000000
[   58.917299] R10: ffffc90001947d48 R11: fefefefefefefeff R12:
cccccccccccccccc

Quoted 8 lines

[   58.917709] R13: ffff88811a312240 R14: ffff888111a9a210 R15:
ffff888105268940
[   58.918109] FS:  00007f04f92c6740(0000) GS:ffff8882b353d000(0000) knlGS:0000000000000000
[   58.918560] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   58.918944] CR2: 00007fbf39569000 CR3: 000000011a3f1004 CR4:
0000000000772ef0
[   58.919622] PKRU: 55555554

Sigh... One last patch on top of all the previous ones, but if we hit another issue I think we need to consider just disabling directory delegations on the client until all the remaining issues can be fixed in the next release.

Anna, are you able to reproduce these bugs?

8<-----------------------------------------------------------------
From cbf97626edbf8c0b619d37aca8d6da77f46e69d6 Mon Sep 17 00:00:00 2001
Message-ID: <cbf97626edbf8c0b619d37aca8d6da77f46e69d6.1767719343.git.trond.myklebust@hammerspace.com>
From: Trond Myklebust <[email protected]>
Date: Tue, 6 Jan 2026 11:54:32 -0500
Subject: [PATCH] NFSv4.x: Directory delegations don't require any state
recovery

The state recovery code in nfs_end_delegation_return() is intended to allow regular files to recover cached open and lock state. It has no function for directory delegations, and may cause corruption.

Fixes 156b09482933 ("NFS: Request a directory delegation on ACCESS, CREATE, and UNLINK")
---
 fs/nfs/delegation.c | 5 +++++
 1 file changed, 5 insertions(+)
fs/nfs/delegation.c +5 -0
index 2248e3ad089a..c9fa4c1f68fc 100644
@@ -581,6 +581,10 @@ static int nfs_end_delegation_return(struct inode *inode, struct nfs_delegation
 	if (delegation == NULL)
 		return 0;
 
+	/* Directory delegations don't require any state recovery */
+	if (!S_ISREG(inode->i_mode))
+		goto out_return;
+
 	if (!issync)
 		mode |= O_NONBLOCK;
 	/* Recall of any remaining application leases */
@@ -604,6 +608,7 @@ static int nfs_end_delegation_return(struct inode *inode, struct nfs_delegation
 		goto out;
 	}
 
+out_return:
 	err = nfs_do_return_delegation(inode, delegation, issync);
 out:
 	/* Refcount matched in nfs_start_delegation_return_locked() */
Signature · 7 lines
2.52.0


-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected], [email protected]
#6 lore raw
Quoted 4 lines · Trond Myklebust

Sigh... One last patch on top of all the previous ones, but if we hit another issue I think we need to consider just disabling directory delegations on the client until all the remaining issues can be fixed in the next release.

This still crasheѕ generic/633:

generic/633 4s ... [ 73.075526] run fstests generic/633 at 2026-01-07 05:21:37

[   73.286318] process 'vfstest' launched '/dev/fd/4/file1' with NULL argv: empty string added
[   73.314625] Oops: general protection fault, probably for non-canonical address 0xcccccccccccccd50: 0000 [#1] SMP NOPTI
[   73.315391] CPU: 1 UID: 0 PID: 100 Comm: kworker/u8:3 Tainted: G                 N  6.19.0-rc4+ #4540 PREEMPT(full) 
[   73.316043] Tainted: [N]=TEST
[   73.316229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   73.316786] Workqueue: rpciod rpc_async_schedule
[   73.317066] RIP: 0010:nfs_inode_find_state_and_recover+0x9c/0x260
[   73.317430] Code: 01 0f 85 83 00 00 00 49 8b 84 24 80 00 00 00 4c 8d 60 80 48 39 c3 0f 84 68 01 00 00 4d 8b 7c 24 60 4d 85 ff 74 e1 48 8b 7d 00 <49> 39 bf 84 00 00 00 75 af 8b 4d 08 41 39 8f 8c 00 00 00 75 a3 48
[   73.318513] RSP: 0018:ffffc900001bfd00 EFLAGS: 00010286
[   73.318868] RAX: ffff88810cfb98e0 RBX: ffff8881d68d8c50 RCX: 0000000000000000
[   73.319354] RDX: ffff88810330a0c0 RSI: ffff88811a86d914 RDI: b6162427695ded30
[   73.319832] RBP: ffff88811a86d918 R08: ffff88810c685100 R09: ffff88810c685130
[   73.320270] R10: 0000000000000003 R11: fefefefefefefeff R12: ffff88810cfb9860
[   73.320698] R13: ffff8881179b2800 R14: 0000000000000000 R15: cccccccccccccccc
[   73.321135] FS:  0000000000000000(0000) GS:ffff8882b363d000(0000) knlGS:0000000000000000
[   73.321640] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   73.321994] CR2: 0000559277cef238 CR3: 0000000161792002 CR4: 0000000000772ef0
[   73.322422] PKRU: 55555554
[   73.322592] Call Trace:
[   73.322749]  <TASK>
[   73.322889]  nfs4_delegreturn_done+0x1b7/0x380
[   73.323173]  ? __pfx_rpc_exit_task+0x10/0x10
[   73.323475]  rpc_exit_task+0x5c/0x170
[   73.323755]  __rpc_execute+0xb1/0x490
[   73.324000]  rpc_async_schedule+0x2a/0x40
[   73.324257]  process_one_work+0x16c/0x330
[   73.324515]  worker_thread+0x254/0x3a0
[   73.324762]  ? __pfx_worker_thread+0x10/0x10
[   73.325042]  kthread+0x117/0x230
[   73.325303]  ? __pfx_kthread+0x10/0x10
[   73.325557]  ? __pfx_kthread+0x10/0x10
[   73.325812]  ret_from_fork+0x1b6/0x200
[   73.326045]  ? __pfx_kthread+0x10/0x10
[   73.326278]  ret_from_fork_asm+0x1a/0x30
[   73.326535]  </TASK>
[   73.326681] Modules linked in: kvm_intel kvm irqbypass
[   73.327046] ---[ end trace 0000000000000000 ]---
maintainer
#7 lore raw
Quoted 55 lines · On Tue, 2026-01-06 at 21:23 -0800, [email protected] wrote:
Quoted 6 lines · Trond Myklebust

Sigh... One last patch on top of all the previous ones, but if we hit
another issue I think we need to consider just disabling directory delegations on the client until all the remaining issues can be fixed
in the next release.

This still crasheѕ generic/633:

generic/633  4s ... [   73.075526] run fstests generic/633 at 2026- 01-07 05:21:37
[   73.286318] process 'vfstest' launched '/dev/fd/4/file1' with NULL
argv: empty string added
[   73.314625] Oops: general protection fault, probably for non- canonical address 0xcccccccccccccd50: 0000 [#1] SMP NOPTI [   73.315391] CPU: 1 UID: 0 PID: 100 Comm: kworker/u8:3 Tainted:
G                 N  6.19.0-rc4+ #4540 PREEMPT(full)
[   73.316043] Tainted: [N]=TEST
[   73.316229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   73.316786] Workqueue: rpciod rpc_async_schedule
[   73.317066] RIP: 0010:nfs_inode_find_state_and_recover+0x9c/0x260 [   73.317430] Code: 01 0f 85 83 00 00 00 49 8b 84 24 80 00 00 00 4c 8d 60 80 48 39 c3 0f 84 68 01 00 00 4d 8b 7c 24 60 4d 85 ff 74 e1 48 8b 7d 00 <49> 39 bf 84 00 00 00 75 af 8b 4d 08 41 39 8f 8c 00 00 00 75 a3 48
[   73.318513] RSP: 0018:ffffc900001bfd00 EFLAGS: 00010286 [   73.318868] RAX: ffff88810cfb98e0 RBX: ffff8881d68d8c50 RCX:
0000000000000000
[   73.319354] RDX: ffff88810330a0c0 RSI: ffff88811a86d914 RDI:
b6162427695ded30
[   73.319832] RBP: ffff88811a86d918 R08: ffff88810c685100 R09:
ffff88810c685130
[   73.320270] R10: 0000000000000003 R11: fefefefefefefeff R12:
ffff88810cfb9860
[   73.320698] R13: ffff8881179b2800 R14: 0000000000000000 R15:
cccccccccccccccc
[   73.321135] FS:  0000000000000000(0000) GS:ffff8882b363d000(0000) knlGS:0000000000000000
[   73.321640] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   73.321994] CR2: 0000559277cef238 CR3: 0000000161792002 CR4:
0000000000772ef0
[   73.322422] PKRU: 55555554
[   73.322592] Call Trace:
[   73.322749]  <TASK>
[   73.322889]  nfs4_delegreturn_done+0x1b7/0x380
[   73.323173]  ? __pfx_rpc_exit_task+0x10/0x10
[   73.323475]  rpc_exit_task+0x5c/0x170
[   73.323755]  __rpc_execute+0xb1/0x490
[   73.324000]  rpc_async_schedule+0x2a/0x40
[   73.324257]  process_one_work+0x16c/0x330
[   73.324515]  worker_thread+0x254/0x3a0
[   73.324762]  ? __pfx_worker_thread+0x10/0x10
[   73.325042]  kthread+0x117/0x230

Quoted 8 lines

[   73.325303]  ? __pfx_kthread+0x10/0x10
[   73.325557]  ? __pfx_kthread+0x10/0x10
[   73.325812]  ret_from_fork+0x1b6/0x200
[   73.326045]  ? __pfx_kthread+0x10/0x10
[   73.326278]  ret_from_fork_asm+0x1a/0x30
[   73.326535]  </TASK>
[   73.326681] Modules linked in: kvm_intel kvm irqbypass [   73.327046] ---[ end trace 0000000000000000 ]---

Thanks for doing all this testing, Christoph. I really appreciate it. The previous patch was incomplete. This is incremental to yesterday's patch, but I'll squash them together in the testing branch, since they're both about blocking state recovery in the non-regular file case.

8<-----------------------------------------------------------------
From 534676d290af6fae6ad1b067b81c13523340bc83 Mon Sep 17 00:00:00 2001
Message-ID: <534676d290af6fae6ad1b067b81c13523340bc83.1767798220.git.trond.myklebust@hammerspace.com>
From: Trond Myklebust <[email protected]>
Date: Wed, 7 Jan 2026 10:01:58 -0500
Subject: [PATCH] NFSv4.x: Directory delegations don't require any state
recovery (part 2)

This patch will be squashed with part 1.

---
 fs/nfs/nfs4state.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nfs/nfs4state.c +5 -1
index 01179f7de322..dba51c622cf3 100644
@@ -1445,6 +1445,8 @@ void nfs_inode_find_state_and_recover(struct inode *inode,
 	struct nfs4_state *state;
 	bool found = false;
 
+	if (!S_ISREG(inode->i_mode))
+		goto out;
 	rcu_read_lock();
 	list_for_each_entry_rcu(ctx, &nfsi->open_files, list) {
 		state = ctx->state;
@@ -1466,7 +1468,7 @@ void nfs_inode_find_state_and_recover(struct inode *inode,
 			found = true;
 	}
 	rcu_read_unlock();
-
+out:
 	nfs_inode_find_delegation_state_and_recover(inode, stateid);
 	if (found)
 		nfs4_schedule_state_manager(clp);
@@ -1478,6 +1480,8 @@ static void nfs4_state_mark_open_context_bad(struct nfs4_state *state, int err)
 	struct nfs_inode *nfsi = NFS_I(inode);
 	struct nfs_open_context *ctx;
 
+	if (!S_ISREG(inode->i_mode))
+		return;
 	rcu_read_lock();
 	list_for_each_entry_rcu(ctx, &nfsi->open_files, list) {
 		if (ctx->state != state)
Signature · 1 line
2.52.0
#8 lore raw
Quoted 5 lines · Trond Myklebust

Thanks for doing all this testing, Christoph. I really appreciate it. The previous patch was incomplete. This is incremental to yesterday's patch, but I'll squash them together in the testing branch, since they're both about blocking state recovery in the non-regular file case.

This fixes generic/633. I've kicked off a full xfstests run and will report back.

#10 lore raw
In reply to Trond Myklebust

Hi Trond,

Quoted 79 lines · Trond Myklebust
Quoted 55 lines · On Tue, 2026-01-06 at 21:23 -0800, [email protected] wrote:
Quoted 6 lines · Trond Myklebust

Sigh... One last patch on top of all the previous ones, but if we hit
another issue I think we need to consider just disabling directory delegations on the client until all the remaining issues can be fixed
in the next release.

This still crasheѕ generic/633:

generic/633  4s ... [   73.075526] run fstests generic/633 at 2026- 01-07 05:21:37
[   73.286318] process 'vfstest' launched '/dev/fd/4/file1' with NULL
argv: empty string added
[   73.314625] Oops: general protection fault, probably for non- canonical address 0xcccccccccccccd50: 0000 [#1] SMP NOPTI [   73.315391] CPU: 1 UID: 0 PID: 100 Comm: kworker/u8:3 Tainted:
G                 N  6.19.0-rc4+ #4540 PREEMPT(full)
[   73.316043] Tainted: [N]=TEST
[   73.316229] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[   73.316786] Workqueue: rpciod rpc_async_schedule
[   73.317066] RIP: 0010:nfs_inode_find_state_and_recover+0x9c/0x260 [   73.317430] Code: 01 0f 85 83 00 00 00 49 8b 84 24 80 00 00 00 4c 8d 60 80 48 39 c3 0f 84 68 01 00 00 4d 8b 7c 24 60 4d 85 ff 74 e1 48 8b 7d 00 <49> 39 bf 84 00 00 00 75 af 8b 4d 08 41 39 8f 8c 00 00 00 75 a3 48
[   73.318513] RSP: 0018:ffffc900001bfd00 EFLAGS: 00010286 [   73.318868] RAX: ffff88810cfb98e0 RBX: ffff8881d68d8c50 RCX:
0000000000000000
[   73.319354] RDX: ffff88810330a0c0 RSI: ffff88811a86d914 RDI:
b6162427695ded30
[   73.319832] RBP: ffff88811a86d918 R08: ffff88810c685100 R09:
ffff88810c685130
[   73.320270] R10: 0000000000000003 R11: fefefefefefefeff R12:
ffff88810cfb9860
[   73.320698] R13: ffff8881179b2800 R14: 0000000000000000 R15:
cccccccccccccccc
[   73.321135] FS:  0000000000000000(0000) GS:ffff8882b363d000(0000) knlGS:0000000000000000
[   73.321640] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [   73.321994] CR2: 0000559277cef238 CR3: 0000000161792002 CR4:
0000000000772ef0
[   73.322422] PKRU: 55555554
[   73.322592] Call Trace:
[   73.322749]  <TASK>
[   73.322889]  nfs4_delegreturn_done+0x1b7/0x380
[   73.323173]  ? __pfx_rpc_exit_task+0x10/0x10
[   73.323475]  rpc_exit_task+0x5c/0x170
[   73.323755]  __rpc_execute+0xb1/0x490
[   73.324000]  rpc_async_schedule+0x2a/0x40
[   73.324257]  process_one_work+0x16c/0x330
[   73.324515]  worker_thread+0x254/0x3a0
[   73.324762]  ? __pfx_worker_thread+0x10/0x10
[   73.325042]  kthread+0x117/0x230

Quoted 8 lines

[   73.325303]  ? __pfx_kthread+0x10/0x10
[   73.325557]  ? __pfx_kthread+0x10/0x10
[   73.325812]  ret_from_fork+0x1b6/0x200
[   73.326045]  ? __pfx_kthread+0x10/0x10
[   73.326278]  ret_from_fork_asm+0x1a/0x30
[   73.326535]  </TASK>
[   73.326681] Modules linked in: kvm_intel kvm irqbypass [   73.327046] ---[ end trace 0000000000000000 ]---

Thanks for doing all this testing, Christoph. I really appreciate it. The previous patch was incomplete. This is incremental to yesterday's patch, but I'll squash them together in the testing branch, since they're both about blocking state recovery in the non-regular file case.

8<-----------------------------------------------------------------
From 534676d290af6fae6ad1b067b81c13523340bc83 Mon Sep 17 00:00:00 2001
Message-ID: <534676d290af6fae6ad1b067b81c13523340bc83.1767798220.git.trond.myklebust@hammerspace.com>
From: Trond Myklebust <[email protected]>
Date: Wed, 7 Jan 2026 10:01:58 -0500
Subject: [PATCH] NFSv4.x: Directory delegations don't require any state
recovery (part 2)

Quoted 1 line

This patch will be squashed with part 1.

I finally hit Christoph's crasher this morning, and this patch did help when running with NFS v4.1 but I still see the problem with v4.2. I haven't dug into it enough to know if it's just harder to hit now, though.

Do you want me to take back over debugging this from here? Anna

Quoted 37 lines
---
 fs/nfs/nfs4state.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
fs/nfs/nfs4state.c +5 -1
index 01179f7de322..dba51c622cf3 100644
@@ -1445,6 +1445,8 @@ void nfs_inode_find_state_and_recover(struct inode *inode,
 	struct nfs4_state *state;
 	bool found = false;
 
+	if (!S_ISREG(inode->i_mode))
+		goto out;
 	rcu_read_lock();
 	list_for_each_entry_rcu(ctx, &nfsi->open_files, list) {
 		state = ctx->state;
@@ -1466,7 +1468,7 @@ void nfs_inode_find_state_and_recover(struct inode *inode,
 			found = true;
 	}
 	rcu_read_unlock();
-
+out:
 	nfs_inode_find_delegation_state_and_recover(inode, stateid);
 	if (found)
 		nfs4_schedule_state_manager(clp);
@@ -1478,6 +1480,8 @@ static void nfs4_state_mark_open_context_bad(struct nfs4_state *state, int err)
 	struct nfs_inode *nfsi = NFS_I(inode);
 	struct nfs_open_context *ctx;
 
+	if (!S_ISREG(inode->i_mode))
+		return;
 	rcu_read_lock();
 	list_for_each_entry_rcu(ctx, &nfsi->open_files, list) {
 		if (ctx->state != state)
#9 lore raw
In reply to [email protected]
Quoted 9 lines · On Wed, Jan 07, 2026 at 07:30:48AM -0800, [email protected] wrote:
Quoted 5 lines · Trond Myklebust

Thanks for doing all this testing, Christoph. I really appreciate it. The previous patch was incomplete. This is incremental to yesterday's patch, but I'll squash them together in the testing branch, since they're both about blocking state recovery in the non-regular file case.

This fixes generic/633. I've kicked off a full xfstests run and will report back.

xfstests survives without crashing. However since 6.19-rc a bunch of timestamp related tests keep faіling.

maintainer
#11 lore raw
In reply to Trond Myklebust

[cc to fsdevel added]

Quoted 11 lines · Trond Myklebust
fs/nfs/dir.c +11 -0
+static void nfs_clear_verifier_directory(struct inode *dir)
+{
+	struct dentry *this_parent;
+	struct dentry *dentry;
+	struct inode *inode;
+
+	if (hlist_empty(&dir->i_dentry))
+		return;
+	this_parent =
+		hlist_entry(dir->i_dentry.first, struct dentry, d_u.d_alias);
+

fs/nfs/dir.c void nfs_set_verifier(struct dentry *dentry, unsigned long verf) · line 1504

Quoted 10 lines
fs/nfs/dir.c +10 -0
+	spin_lock(&this_parent->d_lock);
+	nfs_unset_verifier_delegated(&this_parent->d_time);
+	dentry = d_first_child(this_parent);
+	hlist_for_each_entry_from(dentry, d_sib) {
+		if (unlikely(dentry->d_flags & DCACHE_DENTRY_CURSOR))
+			continue;
+		inode = d_inode_rcu(dentry);
+		if (inode &&
+		    NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
+			continue;

What's to stop the inode from being freed right under you? You are *not* guaranteed to be holding rcu_read_lock(), unless I'm missing something in the call chain, so... what's going on here?

maintainer
#12 lore raw
Quoted 21 lines · Al Viro

[cc to fsdevel added]

Quoted 17 lines · Trond Myklebust
fs/nfs/dir.c +17 -0
+static void nfs_clear_verifier_directory(struct inode *dir)
+{
+	struct dentry *this_parent;
+	struct dentry *dentry;
+	struct inode *inode;
+
+	if (hlist_empty(&dir->i_dentry))
+		return;
+	this_parent =
+		hlist_entry(dir->i_dentry.first, struct dentry, d_u.d_alias);
+
+	spin_lock(&this_parent->d_lock);
+	nfs_unset_verifier_delegated(&this_parent->d_time);
+	dentry = d_first_child(this_parent);
+	hlist_for_each_entry_from(dentry, d_sib) {
+		if (unlikely(dentry->d_flags & DCACHE_DENTRY_CURSOR))
+			continue;
Quoted 8 lines

fs/nfs/dir.c void nfs_set_verifier(struct dentry *dentry, unsigned long verf) · line 1504

Quoted 4 lines
fs/nfs/dir.c +4 -0
+		inode = d_inode_rcu(dentry);
+		if (inode &&
+		    NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
+			continue;

What's to stop the inode from being freed right under you? You are *not* guaranteed to be holding rcu_read_lock(), unless I'm missing something in the call chain, so... what's going on here?

While we are at it, what are cursors doing on NFS directories? AFAICS, you are not using simple_dir_operations there or anything that would call dcache_dir_open(), for that matter. Confused...

maintainer
#13 lore raw
In reply to Al Viro
Quoted 21 lines · Al Viro

[cc to fsdevel added]

Quoted 17 lines · Trond Myklebust
fs/nfs/dir.c +17 -0
+static void nfs_clear_verifier_directory(struct inode *dir)
+{
+	struct dentry *this_parent;
+	struct dentry *dentry;
+	struct inode *inode;
+
+	if (hlist_empty(&dir->i_dentry))
+		return;
+	this_parent =
+		hlist_entry(dir->i_dentry.first, struct dentry, d_u.d_alias);
+
+	spin_lock(&this_parent->d_lock);
+	nfs_unset_verifier_delegated(&this_parent->d_time);
+	dentry = d_first_child(this_parent);
+	hlist_for_each_entry_from(dentry, d_sib) {
+		if (unlikely(dentry->d_flags & DCACHE_DENTRY_CURSOR))
+			continue;
Quoted 8 lines

fs/nfs/dir.c void nfs_set_verifier(struct dentry *dentry, unsigned long verf) · line 1504

Quoted 4 lines
fs/nfs/dir.c +4 -0
+		inode = d_inode_rcu(dentry);
+		if (inode &&
+		    NFS_PROTO(inode)->have_delegation(inode, FMODE_READ, 0))
+			continue;

What's to stop the inode from being freed right under you? You are *not* guaranteed to be holding rcu_read_lock(), unless I'm missing something in the call chain, so... what's going on here?

Incidentally, nfs_clear_verifier_file() doesn't need to bother with checking for NULL dir - alias is found in ->i_dentry of inode with ->i_lock held, which means it can't have progressed through __dentry_kill() to dropping the reference to parent. So holding alias->d_lock (which stabilizes ->d_parent) is enough - ->d_parent is still pinned by alias and it can't have become negative.

"Can't have progressed through __dentry_kill()" is critical here - that's the difference from nfs_set_verifier_locked() case.

Folks, this stuff can get seriously subtle; _please_ ask around on fsdevel if you are doing something non-trivial with it.