block: unregister holder kobject on late add_disk failure
Subsystems (2)
PATCH 1/2 block: unregister holder kobject on late add_disk failure
When device_add_disk fails after the registering the delayed holders, the kobjects for them are leaked. Fix this by unregistering them for these failure cases.
--- block/genhd.c | 4 +++- block/holder.c | 10 ++++++++++ include/linux/blkdev.h | 4 ++++ 3 files changed, 17 insertions(+), 1 deletion(-)
block/genhd.c +3 -1
@@ -484,7 +484,7 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk, ret =blk_register_queue (disk);if (ret) - goto out_put_slave_dir; + goto out_unregister_holders;if (!(disk->flags &GENHD_FL_HIDDEN )) { ret = bdi_register(disk->bdi, "%u:%u",
@@ -526,6 +526,8 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk,bdi_unregister (disk->bdi); out_unregister_queue:blk_unregister_queue (disk); +out_unregister_holders: +bd_unregister_all_holders (disk); out_put_slave_dir:kobject_put (disk->slave_dir); out_put_holder_dir:
block/holder.c +10 -0
@@ -173,3 +173,13 @@ int bd_register_pending_holders(struct gendisk *disk)mutex_unlock (&disk->open_mutex);return ret; } + +void bd_unregister_all_holders (struct gendisk *disk) +{ +struct bd_holder_disk *holder; + +mutex_lock (&disk->open_mutex); +list_for_each_entry (holder, &disk->slave_bdevs, list) +__unlink_disk_holder (holder->bdev, disk); +mutex_unlock (&disk->open_mutex); +}
include/linux/blkdev.h +4 -0
@@ -840,6 +840,7 @@ void set_capacity(struct gendisk *disk, sector_t size);int bd_link_disk_holder (struct block_device *bdev,struct gendisk *disk);void bd_unlink_disk_holder (struct block_device *bdev,struct gendisk *disk);int bd_register_pending_holders (struct gendisk *disk); +void bd_unregister_all_holders (struct gendisk *disk);#else static inline int bd_link_disk_holder (struct block_device *bdev,struct gendisk *disk)
@@ -854,6 +855,9 @@ static inline int bd_register_pending_holders(struct gendisk *disk) {return 0 ; } +static inline void bd_unregister_all_holders (struct gendisk *disk) +{ +}#endif /* CONFIG_BLOCK_HOLDER_DEPRECATED */ dev_t part_devt (struct gendisk *disk,u8 partno);
Signature · 1 line
2.30.2
PATCH 2/2 block: clear the holder releated fields on late disk_add failure
Zero out the pointers to the holder related kobjects so that bd_unlink_disk_holder does the right thing when dm cleans up the delayed holders after add_disk fails.
--- block/genhd.c | 2 ++ 1 file changed, 2 insertions(+)
block/genhd.c +2 -0
@@ -530,8 +530,10 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk,bd_unregister_all_holders (disk); out_put_slave_dir:kobject_put (disk->slave_dir); + disk->slave_dir =NULL ; out_put_holder_dir:kobject_put (disk->part0->bd_holder_dir); + disk->part0->bd_holder_dir =NULL ; out_del_integrity:blk_integrity_del (disk); out_del_block_link:
Signature · 1 line
2.30.2
Re: [PATCH 2/2] block: clear the holder releated fields on late disk_add failure
Hi, Christoph!
Quoted 4 lines · Christoph Hellwig
Zero out the pointers to the holder related kobjects so that
bd_unlink_disk_holderdoes the right thing when dm cleans up the delayed holders afteradd_diskfails.
I'm planning to do the similar thing in the normal path:
1) in del_gendisk: (add a new api kobject_put_and_test)
if (kobject_put_and_test(bd_holder_dir/slave_dir))
bd_holder_dir/slave_dir = NULL;
2) in bd_link_disk_holder, get bd_holder_dir first:
if (!kobject_get_unless_zero(bd_holder_dir))
return -ENODEV;
...bd_find_holder_disk()
Do you think this is ok?
Thanks,
Kuai
Quoted 23 lines
Fixes 89f871af1b26 ("dm: delay registering the gendisk")Reported-by Yu Kuai <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>--- block/genhd.c | 2 ++ 1 file changed, 2 insertions(+)block/genhd.c +2
-0@@ -530,8 +530,10 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk,bd_unregister_all_holders (disk); out_put_slave_dir:kobject_put (disk->slave_dir); + disk->slave_dir =NULL ; out_put_holder_dir:kobject_put (disk->part0->bd_holder_dir); + disk->part0->bd_holder_dir =NULL ; out_del_integrity:blk_integrity_del (disk); out_del_block_link:
Re: [PATCH 2/2] block: clear the holder releated fields on late disk_add failure
Quoted 5 lines · Yu Kuai
1) in
del_gendisk: (add a new apikobject_put_and_test)if (
kobject_put_and_test(bd_holder_dir/slave_dir))bd_holder_dir/slave_dir = NULL;
Quoted 8 lines
2) in
bd_link_disk_holder, getbd_holder_dirfirst:if (!kobject_get_unless_zero(bd_holder_dir))
return -ENODEV;...
bd_find_holder_disk()Do you think this is ok?
I'm not quite sure what the point is.
If you want to really clean this up a good thing would be to remove the delayed holder registration entirely and just do them in dm after add_disk and remove them before del_gendisk. I've been wanting to do that a few times but always gave up due to the mess in dm.
Re: [PATCH 2/2] block: clear the holder releated fields on late disk_add failure
Hi,
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Yu Kuai
1) in
del_gendisk: (add a new apikobject_put_and_test)if (
kobject_put_and_test(bd_holder_dir/slave_dir))bd_holder_dir/slave_dir = NULL;2) in
bd_link_disk_holder, getbd_holder_dirfirst:
Quoted 9 lines
Quoted 6 lines
if (!kobject_get_unless_zero(bd_holder_dir))
return -ENODEV;...
bd_find_holder_disk()Do you think this is ok?
I'm not quite sure what the point is.
Because I'm afraid bd_link_disk_holder() just take bdev as paramater, (bdev is got by blkdev_get_by_dev), but there is no guarantee that disk can't be remove(del_gendisk is called for the disk that bdev belongs to).
Thus I think the interface bd_link_disk_holder() itself is problematic, current caller drbd/md/dm might all be affected.
Thanks,
Kuai
Quoted 7 lines
If you want to really clean this up a good thing would be to remove the delayed holder registration entirely and just do them in dm after
add_diskand remove them beforedel_gendisk. I've been wanting to do that a few times but always gave up due to the mess in dm..
Re: [PATCH 2/2] block: clear the holder releated fields on late disk_add failure
Quoted 21 lines · Yu Kuai
Hi,
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Yu Kuai
1) in
del_gendisk: (add a new apikobject_put_and_test)if (
kobject_put_and_test(bd_holder_dir/slave_dir)) bd_holder_dir/slave_dir = NULL;2) in
bd_link_disk_holder, getbd_holder_dirfirst:Quoted 9 lines
Quoted 6 lines
if (!kobject_get_unless_zero(bd_holder_dir))
return-ENODEV;
...bd_find_holder_disk()Do you think this is ok?
I'm not quite sure what the point is.
Quoted 8 lines
Because I'm afraid
bd_link_disk_holder()just take bdev as paramater, (bdev is got byblkdev_get_by_dev), but there is no guarantee that disk can't beremove(del_gendisk is called for the disk that bdev belongs to).Thus I think the interface
bd_link_disk_holder()itself is problematic, current caller drbd/md/dm might all be affected.
I just verified that by the following thest:
1) add some delay before calling bd_link_disk_holder:
drivers/md/dm.c +2 -0
@@ -745,6 +745,8 @@ static int open_table_device(struct table_device *td, dev_t dev,if (IS_ERR (bdev))return PTR_ERR (bdev); +printk ("%s: delay 5s\n" , __func__); +msleep (5000 ); r =bd_link_disk_holder (bdev,dm_disk (md));if (r) {blkdev_put (bdev, td->dm_dev.mode |FMODE_EXCL );
2) run the cmd:
dmsetup create test1 --table "0 100000 linear /dev/sda 0" & sleep 1
echo 1 > /sys/block/sda/device/delete
And the follwing uaf is triggered:
[ 164.085806] ================================================================== [ 164.087942] BUG: KASAN: use-after-free in kobject_get+0x20/0x120 [ 164.089350] Read of size 1 at addr ffff8881021624bc by task dmsetup/915 [ 164.090837] [ 164.091227] CPU: 19 PID: 915 Comm: dmsetup Not tainted 6.0.0-next-20221017-00006-ga50117a47 [ 164.093327] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ?-20190727_073836-4 [ 164.096194] Call Trace: [ 164.096577] <TASK> [ 164.096761] ? dump_stack_lvl+0x73/0x9f [ 164.097072] ? print_report+0x249/0x746 [ 164.097421] ? __virt_addr_valid+0xd4/0x200 [ 164.097815] ? kobject_get+0x20/0x120 [ 164.098135] ? kasan_report+0xc0/0x120 [ 164.098491] ? kobject_get+0x20/0x120 [ 164.098824] ? __asan_load1+0x68/0xb0 [ 164.099159] ? kobject_get+0x20/0x120 [ 164.099500] ? bd_link_disk_holder+0x203/0x270 [ 164.099872] ? dm_get_table_device.cold+0x59/0xbf [ 164.100295] ? dm_get_device+0x35c/0x4f0 [ 164.100651] ? dm_put_device+0x230/0x230 [ 164.101011] ? vsscanf+0x1360/0x1360 [ 164.101374] ? __kmem_cache_alloc_node+0x1ce/0x490 [ 164.101836] ? kasan_set_track+0x29/0x40 [ 164.102187] ? kasan_save_alloc_info+0x1f/0x40 [ 164.102588] ? __kasan_kmalloc+0xcb/0xe0 [ 164.102959] ? linear_ctr+0x190/0x260 [ 164.103290] ? linear_dtr+0x50/0x50 [ 164.103606] ? dm_table_destroy+0x280/0x280 [ 164.103976] ? __kasan_check_write+0x20/0x30 [ 164.104369] ? up_read+0x25/0xf0 [ 164.104667] ? dm_table_add_target+0x2f5/0x840 [ 164.105058] ? dm_split_args+0x2f0/0x2f0 [ 164.105440] ? __kasan_check_write+0x20/0x30 [ 164.105886] ? mutex_lock+0xa6/0x110 [ 164.106230] ? memset+0x4a/0x70 [ 164.106524] ? kvfree+0x44/0x50 [ 164.106816] ? dm_table_create+0x1a3/0x240 [ 164.107202] ? table_load+0x2b5/0x710 [ 164.107543] ? list_devices+0x4c0/0x4c0 [ 164.107900] ? kvmalloc_node+0x7d/0x160 [ 164.108853] ? ctl_ioctl+0x388/0x7b0 [ 164.109191] ? list_devices+0x4c0/0x4c0 [ 164.109568] ? free_params+0x50/0x50 [ 164.109900] ? do_vfs_ioctl+0x931/0x10d0 [ 164.110282] ? handle_mm_fault+0x3aa/0x610 [ 164.110667] ? __kasan_check_read+0x1d/0x30 [ 164.111054] ? __fget_light+0xc2/0x370 [ 164.111408] ? dm_ctl_ioctl+0x12/0x20 [ 164.111744] ? __x64_sys_ioctl+0xd5/0x150 [ 164.112125] ? do_syscall_64+0x35/0x80 [ 164.112489] ? entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 164.112978] </TASK> [ 164.113183] [ 164.113327] Allocated by task 179: [ 164.113647] kasan_save_stack+0x26/0x60 [ 164.113977] kasan_set_track+0x29/0x40 [ 164.114316] kasan_save_alloc_info+0x1f/0x40 [ 164.114709] __kasan_kmalloc+0xcb/0xe0 [ 164.115063] kmalloc_trace+0x7e/0x150 [ 164.115399] kobject_create_and_add+0x3d/0xc0 [ 164.115806] device_add_disk+0x3bf/0x800 [ 164.116164] sd_probe+0x603/0x8e0 [ 164.116482] really_probe+0x4f2/0x730 [ 164.116832] __driver_probe_device+0x223/0x320 [ 164.117243] driver_probe_device+0x69/0x140 [ 164.117627] __device_attach_driver+0x10a/0x200 [ 164.118032] bus_for_each_drv+0x10e/0x1b0 [ 164.118410] __device_attach_async_helper+0x175/0x230 [ 164.118877] async_run_entry_fn+0x73/0x300 [ 164.119254] process_one_work+0x47b/0x9a0 [ 164.119641] worker_thread+0x30c/0x8b0 [ 164.119979] kthread+0x1e5/0x250 [ 164.120283] ret_from_fork+0x1f/0x30 [ 164.120611] [ 164.120755] Freed by task 821: [ 164.121043] kasan_save_stack+0x26/0x60 [ 164.121420] kasan_set_track+0x29/0x40 [ 164.121805] kasan_save_free_info+0x32/0x60 [ 164.122205] __kasan_slab_free+0x172/0x2c0 [ 164.122588] __kmem_cache_free+0x11c/0x560 [ 164.122975] kfree+0xd3/0x240 [ 164.123252] dynamic_kobj_release+0x1e/0x60 [ 164.123646] kobject_put+0x192/0x410 [ 164.123972] del_gendisk+0x22a/0x640 [ 164.124314] sd_remove+0x65/0xa0 [ 164.124610] device_remove+0xbe/0xe0 [ 164.124930] device_release_driver_internal+0x161/0x2e0 [ 164.125401] device_release_driver+0x16/0x20 [ 164.125809] bus_remove_device+0x1d3/0x310 [ 164.126162] device_del+0x310/0x7e0 [ 164.126475] __scsi_remove_device+0x26c/0x360 [ 164.126854] scsi_remove_device+0x38/0x60 [ 164.127214] sdev_store_delete+0x73/0xf0 [ 164.127569] dev_attr_store+0x40/0x70 [ 164.127903] sysfs_kf_write+0x89/0xc0 [ 164.128242] kernfs_fop_write_iter+0x21d/0x330 [ 164.128637] vfs_write+0x60e/0x840 [ 164.128942] ksys_write+0xcd/0x1e0 [ 164.129255] __x64_sys_write+0x46/0x60 [ 164.129598] do_syscall_64+0x35/0x80 [ 164.129915] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 164.130370] [ 164.130518] The buggy address belongs to the object at ffff888102162480 [ 164.130518] which belongs to the cache kmalloc-64 of size 64 [ 164.131590] The buggy address is located 60 bytes inside of [ 164.131590] 64-byte region [ffff888102162480, ffff8881021624c0) [ 164.132629] [ 164.132775] The buggy address belongs to the physical page: [ 164.133277] page:0000000072f285df refcount:1 mapcount:0 mapping:0000000000000000 index:0x02 [ 164.134136] flags: 0x2fffff80000200(slab|node=0|zone=2|lastcpupid=0x1fffff) [ 164.134793] raw: 002fffff80000200 ffff88810004c640 dead000000200020 0000000000000000 [ 164.135493] raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000 [ 164.136186] page dumped because: kasan: bad access detected [ 164.136697] [ 164.136849] Memory state around the buggy address: [ 164.137300] ffff888102162380: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 164.137980] ffff888102162400: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 164.138638] >ffff888102162480: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 164.139305] ^ [ 164.139780] ffff888102162500: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 164.140444] ffff888102162580: fa fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc [ 164.141086] ==================================================================
Quoted 12 lines
Thanks,
KuaiQuoted 7 lines
If you want to really clean this up a good thing would be to remove the delayed holder registration entirely and just do them in dm after
add_diskand remove them before del_gendisk. I've been wanting to do that a few times but always gave up due to the mess in dm..
.
Re: [PATCH 2/2] block: clear the holder releated fields on late disk_add failure
Quoted 7 lines · Yu Kuai
2) run the cmd:
dmsetup create test1 --table "0 100000 linear
/dev/sda0" & sleep 1
echo 1 >/sys/block/sda/device/deleteAnd the follwing uaf is triggered:
Yes, for that we also need to clear the pointer and unregister all holder in del_gedisk (or even better move this mess to dm :()