nouveau: fix instmem race condition around ptr stores
Subsystems (4)
Dave Airlie
maintainerPATCH 1/1 nouveau: fix instmem race condition around ptr stores
From: Dave Airlie <[email protected]>
Running a lot of VK CTS in parallel against nouveau, once every few hours you might see something like this crash.
BUG: kernel NULL pointer dereference, address: 0000000000000008 PGD 8000000114e6e067 P4D 8000000114e6e067 PUD 109046067 PMD 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 7 PID: 53891 Comm: deqp-vk Not tainted 6.8.0-rc6+ #27 Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 RIP: 0010:gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] Code: c7 48 01 c8 49 89 45 58 85 d2 0f 84 95 00 00 00 41 0f b7 46 12 49 8b 7e 08 89 da 42 8d 2c f8 48 8b 47 08 41 83 c7 01 48 89 ee <48> 8b 40 08 ff d0 0f 1f 00 49 8b 7e 08 48 89 d9 48 8d 75 04 48 c1 RSP: 0000:ffffac20c5857838 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 00000000004d8001 RCX: 0000000000000001 RDX: 00000000004d8001 RSI: 00000000000006d8 RDI: ffffa07afe332180 RBP: 00000000000006d8 R08: ffffac20c5857ad0 R09: 0000000000ffff10 R10: 0000000000000001 R11: ffffa07af27e2de0 R12: 000000000000001c R13: ffffac20c5857ad0 R14: ffffa07a96fe9040 R15: 000000000000001c FS: 00007fe395eed7c0(0000) GS:ffffa07e2c980000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000011febe001 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:
...
? gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] ? gp100_vmm_pgt_mem+0x37/0x180 [nouveau] nvkm_vmm_iter+0x351/0xa20 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __lock_acquire+0x3ed/0x2170 ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_ptes_get_map+0xc2/0x100 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_map_locked+0x224/0x3a0 [nouveau]
Adding any sort of useful debug usually makes it go away, so I hand wrote the function in a line, and debugged the asm.
Every so often pt->memory->ptrs is NULL. This ptrs ptr is set in the nv50_instobj_acquire called from nvkm_kmap.
If Thread A and Thread B both get to nv50_instobj_acquire around the same time, and Thread A hits the refcount_set line, and in lockstep thread B succeeds at refcount_inc_not_zero, there is a chance the ptrs value won't have been stored since refcount_set is unordered. Force a memory barrier here, I picked smp_mb, since we want it on all CPUs and it's write followed by a read.
--- drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c +3 -0
@@ -250,6 +250,9 @@ nv50_instobj_acquire(struct nvkm_memory *memory) iobj->base.memory.ptrs = &nv50_instobj_fast; else iobj->base.memory.ptrs = &nv50_instobj_slow; +/* barrier to ensure ptrs is written before another thread +does refcount_inc_not_zero successfully. */ +smp_mb ();refcount_set (&iobj->maps,1 ); }
Signature · 1 line
2.43.2
Am Dienstag, dem 09.04.2024 um 10:34 +1000 schrieb Dave Airlie:
Quoted 64 lines
From: Dave Airlie <[email protected]>
Running a lot of VK CTS in parallel against nouveau, once every few hours you might see something like this crash.
BUG: kernel NULL pointer dereference, address: 0000000000000008 PGD 8000000114e6e067 P4D 8000000114e6e067 PUD 109046067 PMD 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 7 PID: 53891 Comm: deqp-vk Not tainted 6.8.0-rc6+ #27 Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 RIP: 0010:gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] Code: c7 48 01 c8 49 89 45 58 85 d2 0f 84 95 00 00 00 41 0f b7 46 12 49 8b 7e 08 89 da 42 8d 2c f8 48 8b 47 08 41 83 c7 01 48 89 ee <48> 8b 40 08 ff d0 0f 1f 00 49 8b 7e 08 48 89 d9 48 8d 75 04 48 c1 RSP: 0000:ffffac20c5857838 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 00000000004d8001 RCX: 0000000000000001 RDX: 00000000004d8001 RSI: 00000000000006d8 RDI: ffffa07afe332180 RBP: 00000000000006d8 R08: ffffac20c5857ad0 R09: 0000000000ffff10 R10: 0000000000000001 R11: ffffa07af27e2de0 R12: 000000000000001c R13: ffffac20c5857ad0 R14: ffffa07a96fe9040 R15: 000000000000001c FS: 00007fe395eed7c0(0000) GS:ffffa07e2c980000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000011febe001 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:...
? gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] ? gp100_vmm_pgt_mem+0x37/0x180 [nouveau] nvkm_vmm_iter+0x351/0xa20 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __lock_acquire+0x3ed/0x2170 ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_ptes_get_map+0xc2/0x100 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_map_locked+0x224/0x3a0 [nouveau]Adding any sort of useful debug usually makes it go away, so I hand wrote the function in a line, and debugged the asm.
Every so often
pt->memory->ptrsis NULL. This ptrs ptr is set in thenv50_instobj_acquirecalled fromnvkm_kmap.If Thread A and Thread B both get to
nv50_instobj_acquirearound the same time, and Thread A hits therefcount_setline, and in lockstep thread B succeeds atrefcount_inc_not_zero, there is a chance the ptrs value won't have been stored sincerefcount_setis unordered. Force a memory barrier here, I pickedsmp_mb, since we want it on all CPUs and it's write followed by a read.Cc linux-stableSigned-off-by Dave Airlie <[email protected]>--- drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c | 3 +++ 1 file changed, 3 insertions(+)drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c +0
-0
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c nv50_instobj_acquire(struct nvkm_memory *memory) · line 255
Quoted 6 lines
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c +3
-0@@ -250,6 +250,9 @@ nv50_instobj_acquire(struct nvkm_memory *memory) iobj->base.memory.ptrs = &nv50_instobj_fast; else iobj->base.memory.ptrs = &nv50_instobj_slow; +/* barrier to ensure ptrs is written before another thread +does refcount_inc_not_zero successfully. */ +smp_mb ();
Doesn't this miss the corresponding smp_rmb after refcount_inc_not_zero? Without it a sufficiently speculating CPU might still hoist the NULL ptr load across the refcount increase.
Regards,
Lucas
Quoted 53 lines · Lucas Stach
Am Dienstag, dem 09.04.2024 um 10:34 +1000 schrieb Dave Airlie:
Quoted 52 lines
From: Dave Airlie <[email protected]>
Running a lot of VK CTS in parallel against nouveau, once every few hours you might see something like this crash.
BUG: kernel NULL pointer dereference, address: 0000000000000008 PGD 8000000114e6e067 P4D 8000000114e6e067 PUD 109046067 PMD 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 7 PID: 53891 Comm: deqp-vk Not tainted 6.8.0-rc6+ #27 Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 RIP: 0010:gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] Code: c7 48 01 c8 49 89 45 58 85 d2 0f 84 95 00 00 00 41 0f b7 46 12 49 8b 7e 08 89 da 42 8d 2c f8 48 8b 47 08 41 83 c7 01 48 89 ee <48> 8b 40 08 ff d0 0f 1f 00 49 8b 7e 08 48 89 d9 48 8d 75 04 48 c1 RSP: 0000:ffffac20c5857838 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 00000000004d8001 RCX: 0000000000000001 RDX: 00000000004d8001 RSI: 00000000000006d8 RDI: ffffa07afe332180 RBP: 00000000000006d8 R08: ffffac20c5857ad0 R09: 0000000000ffff10 R10: 0000000000000001 R11: ffffa07af27e2de0 R12: 000000000000001c R13: ffffac20c5857ad0 R14: ffffa07a96fe9040 R15: 000000000000001c FS: 00007fe395eed7c0(0000) GS:ffffa07e2c980000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000011febe001 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:...
? gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] ? gp100_vmm_pgt_mem+0x37/0x180 [nouveau] nvkm_vmm_iter+0x351/0xa20 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __lock_acquire+0x3ed/0x2170 ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_ptes_get_map+0xc2/0x100 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_map_locked+0x224/0x3a0 [nouveau]Adding any sort of useful debug usually makes it go away, so I hand wrote the function in a line, and debugged the asm.
Every so often
pt->memory->ptrsis NULL. This ptrs ptr is set in thenv50_instobj_acquirecalled fromnvkm_kmap.If Thread A and Thread B both get to
nv50_instobj_acquirearound the same time, and Thread A hits therefcount_setline, and in lockstep thread B succeeds atrefcount_inc_not_zero, there is a chance the ptrs value won't have been stored sincerefcount_setis unordered. Force a memory barrier here, I pickedsmp_mb, since we want it on all CPUs and it's write followed by a read.
Good catch!
Quoted 7 lines
Quoted 7 lines
Cc linux-stableSigned-off-by Dave Airlie <[email protected]>--- drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c | 3 +++ 1 file changed, 3 insertions(+)
Quoted 15 lines
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c nv50_instobj_acquire(struct nvkm_memory *memory) · line 255
Quoted 11 lines
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c +3
-0@@ -250,6 +250,9 @@ nv50_instobj_acquire(struct nvkm_memory *memory) iobj->base.memory.ptrs = &nv50_instobj_fast; else iobj->base.memory.ptrs = &nv50_instobj_slow; +/* barrier to ensure ptrs is written before another thread +does refcount_inc_not_zero successfully. */ +smp_mb ();Doesn't this miss the corresponding
smp_rmbafterrefcount_inc_not_zero? Without it a sufficiently speculating CPU might still hoist the NULL ptr load across the refcount increase.
Agree, also think this one could be smp_wmb() only.
I also think it's reasonable to keep "the fast path refcount_inc_not_zero that doesn't take the lock", since the scope for this being potentially racy is limited to this function only.
Quoted 3 lines
Regards,
Lucas
Dave Airlie
maintainerQuoted 81 lines · Danilo Krummrich
Quoted 53 lines · Lucas Stach
Am Dienstag, dem 09.04.2024 um 10:34 +1000 schrieb Dave Airlie:
Quoted 52 lines
From: Dave Airlie <[email protected]>
Running a lot of VK CTS in parallel against nouveau, once every few hours you might see something like this crash.
BUG: kernel NULL pointer dereference, address: 0000000000000008 PGD 8000000114e6e067 P4D 8000000114e6e067 PUD 109046067 PMD 0 Oops: 0000 [#1] PREEMPT SMP PTI CPU: 7 PID: 53891 Comm: deqp-vk Not tainted 6.8.0-rc6+ #27 Hardware name: Gigabyte Technology Co., Ltd. Z390 I AORUS PRO WIFI/Z390 I AORUS PRO WIFI-CF, BIOS F8 11/05/2021 RIP: 0010:gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] Code: c7 48 01 c8 49 89 45 58 85 d2 0f 84 95 00 00 00 41 0f b7 46 12 49 8b 7e 08 89 da 42 8d 2c f8 48 8b 47 08 41 83 c7 01 48 89 ee <48> 8b 40 08 ff d0 0f 1f 00 49 8b 7e 08 48 89 d9 48 8d 75 04 48 c1 RSP: 0000:ffffac20c5857838 EFLAGS: 00010202 RAX: 0000000000000000 RBX: 00000000004d8001 RCX: 0000000000000001 RDX: 00000000004d8001 RSI: 00000000000006d8 RDI: ffffa07afe332180 RBP: 00000000000006d8 R08: ffffac20c5857ad0 R09: 0000000000ffff10 R10: 0000000000000001 R11: ffffa07af27e2de0 R12: 000000000000001c R13: ffffac20c5857ad0 R14: ffffa07a96fe9040 R15: 000000000000001c FS: 00007fe395eed7c0(0000) GS:ffffa07e2c980000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000008 CR3: 000000011febe001 CR4: 00000000003706f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace:...
? gp100_vmm_pgt_mem+0xe3/0x180 [nouveau] ? gp100_vmm_pgt_mem+0x37/0x180 [nouveau] nvkm_vmm_iter+0x351/0xa20 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] ? __lock_acquire+0x3ed/0x2170 ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_ptes_get_map+0xc2/0x100 [nouveau] ? __pfx_nvkm_vmm_ref_ptes+0x10/0x10 [nouveau] ? __pfx_gp100_vmm_pgt_mem+0x10/0x10 [nouveau] nvkm_vmm_map_locked+0x224/0x3a0 [nouveau]Adding any sort of useful debug usually makes it go away, so I hand wrote the function in a line, and debugged the asm.
Every so often
pt->memory->ptrsis NULL. This ptrs ptr is set in thenv50_instobj_acquirecalled fromnvkm_kmap.If Thread A and Thread B both get to
nv50_instobj_acquirearound the same time, and Thread A hits therefcount_setline, and in lockstep thread B succeeds atrefcount_inc_not_zero, there is a chance the ptrs value won't have been stored sincerefcount_setis unordered. Force a memory barrier here, I pickedsmp_mb, since we want it on all CPUs and it's write followed by a read.Good catch!
Quoted 7 lines
Quoted 7 lines
Cc linux-stableSigned-off-by Dave Airlie <[email protected]>--- drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c | 3 +++ 1 file changed, 3 insertions(+)Quoted 15 lines
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c nv50_instobj_acquire(struct nvkm_memory *memory) · line 255
Quoted 11 lines
drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c +3
-0@@ -250,6 +250,9 @@ nv50_instobj_acquire(struct nvkm_memory *memory) iobj->base.memory.ptrs = &nv50_instobj_fast; else iobj->base.memory.ptrs = &nv50_instobj_slow; +/* barrier to ensure ptrs is written before another thread +does refcount_inc_not_zero successfully. */ +smp_mb ();Doesn't this miss the corresponding
smp_rmbafterrefcount_inc_not_zero? Without it a sufficiently speculating CPU might still hoist the NULL ptr load across the refcount increase.
Quoted 5 lines
Agree, also think this one could be
smp_wmb()only.I also think it's reasonable to keep "the fast path
refcount_inc_not_zerothat doesn't take the lock", since the scope for this being potentially racy is limited to this function only.
I've been retesting with just barrier() here, since this seems at least to be compiler related, but probably the smp_rmb/smp_wmb combo is the safest answer across arches.
Dave.