RDMA/core: Fix use-after-free when netns exit races compat dev removal
PATCH 1/1 RDMA/core: Fix use-after-free when netns exit races compat dev removal
A compat device is removed from two places: disable_device() during the ib device unregistration, and rdma_dev_exit_net() when the netns it belongs to dies. remove_one_compat_dev() lets the xa_erase() decide which of the two performs the removal, but drops compat_devs_mutex before device_del(), so the caller that finds nothing to erase returns without waiting for the removal the other one is running.
When that caller is rdma_dev_exit_net(), cleanup_net() carries on while the compat device is still alive. Its kobject is tagged with the dying net, so the device_del() still in progress dereferences net->uevent_sock after uevent_net_exit() has freed it:
CPU0 (ib-unreg-wq) CPU1 (netns wq)
------------------ ---------------
__ib_unregister_device()
disable_device()
remove_compat_devs()
remove_one_compat_dev()
xa_erase()
device_del(cdev)
...
cleanup_net()
rdma_dev_exit_net()
remove_one_compat_dev()
xa_erase()
// returns without waiting
uevent_net_exit()
kfree(net->uevent_sock)
kobject_uevent_env()
net->uevent_sock->sk // UAFBUG: KASAN: slab-use-after-free in kobject_uevent_env+0xb6f/0xc80 Read of size 8 at addr ffff888103806490 by task kworker/u16:1/41 Workqueue: ib-unreg-wq ib_unregister_work Call Trace: kobject_uevent_env+0xb6f/0xc80 device_del+0x737/0xc10 disable_device+0x1ad/0x230 __ib_unregister_device+0x229/0x3f0 Freed by task 12: kfree+0x1b3/0x550 ops_undo_list+0x273/0x8c0 cleanup_net+0x3b3/0x720
Fix by holding compat_devs_mutex across the whole removal, so that once remove_one_compat_dev() returns the compat device is gone no matter which caller removed it.
---
With this patch applied the report no longer shows up.
drivers/infiniband/core/device.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/infiniband/core/device.c +7 -1
@@ -1005,14 +1005,20 @@ static void remove_one_compat_dev(struct ib_device *device, u32 id) {struct ib_core_device *cdev; +/* +* Hold the lock across device_del(): the other remover may have won +* the xa_erase() and still be inside device_del(), and the netns exit +* path has to wait for it instead of letting cleanup_net() free the +* netns state the compat device is still tagged with. +*/ mutex_lock (&device->compat_devs_mutex); cdev =xa_erase (&device->compat_devs, id); -mutex_unlock (&device->compat_devs_mutex);if (cdev) {ib_free_port_attrs (cdev);device_del (&cdev->dev);put_device (&cdev->dev); } +mutex_unlock (&device->compat_devs_mutex); }static void remove_compat_devs (struct ib_device *device)
Signature · 1 line
2.53.0
Quoted 31 lines · Serhat Kumral
A compat device is removed from two places:
disable_device()during the ib device unregistration, andrdma_dev_exit_net()when the netns it belongs to dies.remove_one_compat_dev()lets thexa_erase()decide which of the two performs the removal, but dropscompat_devs_mutexbeforedevice_del(), so the caller that finds nothing to erase returns without waiting for the removal the other one is running.When that caller is
rdma_dev_exit_net(),cleanup_net()carries on while the compat device is still alive. Its kobject is tagged with the dying net, so thedevice_del()still in progress dereferencesnet->uevent_sockafteruevent_net_exit()has freed it:CPU0 (ib-unreg-wq) CPU1 (netns wq)
------------------ ---------------
__ib_unregister_device()
disable_device()remove_compat_devs() remove_one_compat_dev() xa_erase() device_del(cdev) ... cleanup_net() rdma_dev_exit_net() remove_one_compat_dev() xa_erase() // returns without waiting uevent_net_exit() kfree(net->uevent_sock) kobject_uevent_env() net->uevent_sock->sk // UAF
Quoted 12 lines
BUG: KASAN: slab-use-after-free in kobject_uevent_env+0xb6f/0xc80 Read of size 8 at addr ffff888103806490 by task kworker/u16:1/41 Workqueue: ib-unreg-wq ib_unregister_work Call Trace: kobject_uevent_env+0xb6f/0xc80 device_del+0x737/0xc10 disable_device+0x1ad/0x230 __ib_unregister_device+0x229/0x3f0 Freed by task 12: kfree+0x1b3/0x550 ops_undo_list+0x273/0x8c0 cleanup_net+0x3b3/0x720
Please don't trim kernel panic logs. Please post the full output, including the steps required to reproduce the issue.
Thanks
Quoted 43 lines
Fix by holding
compat_devs_mutexacross the whole removal, so that onceremove_one_compat_dev()returns the compat device is gone no matter which caller removed it.Fixes 4e0f7b907072 ("RDMA/core: Implement compat device/sysfs tree in net namespace")Assisted-by Claude:claude-opus-5Signed-off-by Serhat Kumral <[email protected]>---
With this patch applied the report no longer shows up.drivers/infiniband/core/device.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/infiniband/core/device.c +7
-1@@ -1005,14 +1005,20 @@ static void remove_one_compat_dev(struct ib_device *device, u32 id) {struct ib_core_device *cdev; +/* +* Hold the lock across device_del(): the other remover may have won +* the xa_erase() and still be inside device_del(), and the netns exit +* path has to wait for it instead of letting cleanup_net() free the +* netns state the compat device is still tagged with. +*/ mutex_lock (&device->compat_devs_mutex); cdev =xa_erase (&device->compat_devs, id); -mutex_unlock (&device->compat_devs_mutex);if (cdev) {ib_free_port_attrs (cdev);device_del (&cdev->dev);put_device (&cdev->dev); } +mutex_unlock (&device->compat_devs_mutex); }static void remove_compat_devs (struct ib_device *device)Signature · 1 line
2.53.0
Quoted 2 lines
Please don't trim kernel panic logs. Please post the full output, including the steps required to reproduce the issue.
sure,
-----------------------------------------------------------------------------
i=1
while [ "$i" -le 200 ]; do
dev="rxed$i"
unshare -n /bin/sh -c "
ip link add ${dev} type dummy || exit 1
ip link set ${dev} up || exit 1
ip addr add 198.51.100.1/24 dev ${dev} || exit 1
rdma link add rxe${i} type rxe netdev ${dev} || exit 1
sleep 0.2
ip link set ${dev} netns 1 || exit 1
" || echo "iteration $i failed" sleep 0.05
ip link del "$dev" 2>/dev/null
i=$((i + 1))
done
-----------------------------------------------------------------------------
Run in a qemu guest with -enable-kvm.
-----------------------------------------------------------------------------
[ 50.325096] ================================================================== [ 50.325099] BUG: KASAN: slab-use-after-free in kobject_uevent_env+0xb6f/0xc80 [ 50.325118] Read of size 8 at addr ffff8881084d0250 by task kworker/u32:1/62 [ 50.325120] [ 50.325123] CPU: 3 UID: 0 PID: 62 Comm: kworker/u32:1 Not tainted 7.3.0-rc1 #1 PREEMPT(lazy) [ 50.325126] Hardware name: QEMU Ubuntu 26.04 PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1ubuntu1 04/01/2014 [ 50.325128] Workqueue: ib-unreg-wq ib_unregister_work [ 50.325133] Call Trace: [ 50.325134] <TASK> [ 50.325135] dump_stack_lvl+0x5d/0x80 [ 50.325140] print_report+0x153/0x4b1 [ 50.325143] ? kobject_uevent_env+0xb6f/0xc80 [ 50.325145] ? __virt_addr_valid+0x221/0x4c0 [ 50.325148] ? kobject_uevent_env+0xb6f/0xc80 [ 50.325150] kasan_report+0xe4/0x1a0 [ 50.325153] ? kobject_uevent_env+0xb6f/0xc80 [ 50.325155] ? __pfx_device_namespace+0x10/0x10 [ 50.325159] kobject_uevent_env+0xb6f/0xc80 [ 50.325161] device_del+0x737/0xc10 [ 50.325164] ? __pfx_device_del+0x10/0x10 [ 50.325166] ? __kobject_del+0xcb/0x310 [ 50.325168] disable_device+0x1ad/0x230 [ 50.325169] ? __pfx_disable_device+0x10/0x10 [ 50.325172] __ib_unregister_device+0x229/0x3f0 [ 50.325174] ? process_one_work+0x790/0x1630 [ 50.325177] ib_unregister_work+0x14/0x30 [ 50.325178] process_one_work+0x8e1/0x1630 [ 50.325181] ? __pfx_process_one_work+0x10/0x10 [ 50.325183] ? lock_acquire+0x18c/0x300 [ 50.325186] ? lock_is_held_type+0x87/0xf0 [ 50.325189] worker_thread+0x4af/0xd20 [ 50.325192] ? __pfx_worker_thread+0x10/0x10 [ 50.325193] kthread+0x2ce/0x3a0 [ 50.325195] ? _raw_spin_unlock_irq+0x23/0x40 [ 50.325199] ? __pfx_kthread+0x10/0x10 [ 50.325200] ret_from_fork+0x520/0x770 [ 50.325203] ? __pfx_ret_from_fork+0x10/0x10 [ 50.325205] ? __switch_to+0x58a/0xf60 [ 50.325207] ? __pfx_kthread+0x10/0x10 [ 50.325209] ret_from_fork_asm+0x1a/0x30 [ 50.325212] </TASK> [ 50.325213] [ 50.325213] Allocated by task 1409: [ 50.325215] kasan_save_stack+0x30/0x50 [ 50.325217] kasan_save_track+0x14/0x30 [ 50.325218] __kasan_kmalloc+0x7f/0x90 [ 50.325221] __kmalloc_cache_noprof+0x223/0x510 [ 50.325224] uevent_net_init+0xcf/0x2f0 [ 50.325225] ops_init+0x185/0x560 [ 50.325228] setup_net+0xf7/0x320 [ 50.325229] copy_net_ns+0x225/0x400 [ 50.325231] create_new_namespaces+0x358/0x9e0 [ 50.325234] unshare_nsproxy_namespaces+0x9c/0x140 [ 50.325239] ksys_unshare+0x4dd/0x770 [ 50.325241] __x64_sys_unshare+0x2f/0x50 [ 50.325243] do_syscall_64+0x106/0x5f0 [ 50.325245] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 50.325248] [ 50.325249] Freed by task 12: [ 50.325250] kasan_save_stack+0x30/0x50 [ 50.325251] kasan_save_track+0x14/0x30 [ 50.325252] kasan_save_free_info+0x3b/0x70 [ 50.325254] __kasan_slab_free+0x47/0x70 [ 50.325256] kfree+0x1b3/0x550 [ 50.325257] ops_undo_list+0x273/0x8c0 [ 50.325259] cleanup_net+0x3b3/0x720 [ 50.325261] process_one_work+0x8e1/0x1630 [ 50.325262] worker_thread+0x4af/0xd20 [ 50.325264] kthread+0x2ce/0x3a0 [ 50.325265] ret_from_fork+0x520/0x770 [ 50.325267] ret_from_fork_asm+0x1a/0x30 [ 50.325269] [ 50.325269] The buggy address belongs to the object at ffff8881084d0240 [ 50.325269] which belongs to the cache kmalloc-32 of size 32 [ 50.325271] The buggy address is located 16 bytes inside of [ 50.325271] freed 32-byte region [ffff8881084d0240, ffff8881084d0260) [ 50.325273] [ 50.325273] The buggy address belongs to the physical page: [ 50.325275] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0xffff8881084d0c80 pfn:0x1084d0 [ 50.325277] flags: 0x200000000000200(workingset|node=0|zone=2) [ 50.325279] page_type: f5(slab) [ 50.325282] raw: 0200000000000200 ffff888100042780 ffffea000427e690 ffffea0004039fd0 [ 50.325284] raw: ffff8881084d0c80 000000000040002a 00000000f5000000 0000000000000000 [ 50.325285] page dumped because: kasan: bad access detected [ 50.325286] [ 50.325286] Memory state around the buggy address: [ 50.325287] ffff8881084d0100: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc [ 50.325288] ffff8881084d0180: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc [ 50.325289] >ffff8881084d0200: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc [ 50.325290] ^ [ 50.325291] ffff8881084d0280: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc [ 50.325292] ffff8881084d0300: fa fb fb fb fc fc fc fc fa fb fb fb fc fc fc fc [ 50.325293] ==================================================================