blktrace: fix use after free
Subsystems (6)
cover blktrace: fix use after free
After two iterations of RFCs I think this is ready now. I've taken the feedback from the last series, both on code and commit log. I've also extended the commit log on the last patch to also explain how the original shift to async request_queue removal turned out to actually be a userspace regression and added a respective fixes tag for it.
You can find these patches on my 20200414-dev-blkqueue-defer-removal-patch-v1 branch based on linux-next tag next-20200414 on kernel.org [0].
Further review and rants are appreciated.
Luis Chamberlain (5):
block: move main block debugfs initialization to its own file
blktrace: fix debugfs use after free
blktrace: refcount the request_queue during ioctl mm/swapfile: refcount block and queue before using
blkcg_schedule_throttle()
block: revert back to synchronous request_queue removal
block/Makefile | 1 + block/blk-core.c | 28 ++++++++++++++++-------- block/blk-debugfs.c | 27 ++++++++++++++++++++++++ block/blk-mq-debugfs.c | 5 ----- block/blk-sysfs.c | 41 ++++++++++++++++++------------------ block/blk.h | 17 +++++++++++++++ include/linux/blkdev.h | 7 +++--- include/linux/blktrace_api.h | 1 - kernel/trace/blktrace.c | 25 ++++++++++++---------- mm/swapfile.c | 14 ++++++++++-- 10 files changed, 114 insertions(+), 52 deletions(-) create mode 100644 block/blk-debugfs.c
Signature · 1 line
2.25.1
PATCH 1/5 block: move main block debugfs initialization to its own file
make_request-based drivers and and request-based drivers share some some debugfs code. By moving this into its own file it makes it easier to expand and audit this shared code.
This patch contains no functional changes.
--- block/Makefile | 1 + block/blk-core.c | 9 +-------- block/blk-debugfs.c | 15 +++++++++++++++ block/blk.h | 7 +++++++ 4 files changed, 24 insertions(+), 8 deletions(-) create mode 100644 block/blk-debugfs.c
block/Makefile +1 -0
@@ -10,6 +10,7 @@ obj-$(CONFIG_BLOCK) := bio.o elevator.o blk-core.o blk-sysfs.o \ blk-mq-sysfs.o blk-mq-cpumap.o blk-mq-sched.o ioctl.o \ genhd.o ioprio.o badblocks.o partitions/ blk-rq-qos.o +obj-$(CONFIG_DEBUG_FS) += blk-debugfs.o obj-$(CONFIG_BOUNCE) += bounce.o obj-$(CONFIG_BLK_SCSI_REQUEST) += scsi_ioctl.o obj-$(CONFIG_BLK_DEV_BSG) += bsg.o
block/blk-core.c +1 -8
@@ -48,10 +48,6 @@#include "blk-pm.h" #include "blk-rq-qos.h" -#ifdef CONFIG_DEBUG_FS -struct dentry *blk_debugfs_root; -#endif -EXPORT_TRACEPOINT_SYMBOL_GPL (block_bio_remap);EXPORT_TRACEPOINT_SYMBOL_GPL (block_rq_remap);EXPORT_TRACEPOINT_SYMBOL_GPL (block_bio_complete);
@@ -1796,10 +1792,7 @@ int __init blk_dev_init(void) blk_requestq_cachep =kmem_cache_create ("request_queue" ,sizeof (struct request_queue ),0 ,SLAB_PANIC ,NULL ); - -#ifdef CONFIG_DEBUG_FS - blk_debugfs_root =debugfs_create_dir ("block" ,NULL ); -#endif +blk_debugfs_register ();return 0 ; }
block/blk-debugfs.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* +* Shared request-based / make_request-based functionality +*/ +#include <linux/kernel.h> +#include <linux/blkdev.h> +#include <linux/debugfs.h> + +struct dentry *blk_debugfs_root; + +void blk_debugfs_register (void ) +{ + blk_debugfs_root =debugfs_create_dir ("block" ,NULL ); +}
block/blk.h +7 -0
@@ -487,5 +487,12 @@ struct request_queue *__blk_alloc_queue(int node_id);int __bio_add_pc_page (struct request_queue *q,struct bio *bio,struct page *page,unsigned int len,unsigned int offset,bool *same_page); +#ifdef CONFIG_DEBUG_FS +void blk_debugfs_register (void ); +#else +static inline void blk_debugfs_register (void ) +{ +} +#endif /* CONFIG_DEBUG_FS */ #endif /* BLK_INTERNAL_H */
Signature · 1 line
2.25.1
Re: [PATCH 1/5] block: move main block debugfs initialization to its own file
Quoted 6 lines · Luis Chamberlain
make_request-based drivers and and request-based drivers share some some debugfs code. By moving this into its own file it makes it easier to expand and audit this shared code.
This patch contains no functional changes.
Quoted 8 lines
Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>
Re: [PATCH 1/5] block: move main block debugfs initialization to its own file
In reply to Luis ChamberlainQuoted 3 lines · Luis Chamberlain
make_request-based drivers and and request-based drivers share some some debugfs code. By moving this into its own file it makes it easier to expand and audit this shared code.
PATCH 2/5 blktrace: fix debugfs use after free
In reply to Luis ChamberlainOn commit 6ac93117ab00 ("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.
Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
[ 107.193134] debugfs: Directory 'loop0' with parent 'block' already present! [ 107.254615] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 107.258785] #PF: supervisor write access in kernel mode [ 107.262035] #PF: error_code(0x0002) - not-present page [ 107.264106] PGD 0 P4D 0 [ 107.264404] Oops: 0002 [#1] SMP NOPTI [ 107.264803] CPU: 8 PID: 674 Comm: kworker/8:2 Tainted: G E 5.6.0-rc7-next-20200327 #1 [ 107.265712] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 107.266553] Workqueue: events __blk_release_queue [ 107.267051] RIP: 0010:down_write+0x15/0x40 [ 107.267488] Code: eb ca e8 ee a5 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 107.269300] RSP: 0018:ffff9927c06efda8 EFLAGS: 00010246 [ 107.269841] RAX: 0000000000000000 RBX: ffff8be7e73b0600 RCX: ffffff8100000000 [ 107.270559] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 107.271281] RBP: 00000000000000a0 R08: ffff8be7ebc80fa8 R09: ffff8be7ebc80fa8 [ 107.272001] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 107.272722] R13: ffff8be7efc30400 R14: ffff8be7e0571200 R15: 00000000000000a0 [ 107.273475] FS: 0000000000000000(0000) GS:ffff8be7efc00000(0000) knlGS:0000000000000000 [ 107.274346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 107.274968] CR2: 00000000000000a0 CR3: 000000042abee003 CR4: 0000000000360ee0 [ 107.275710] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 107.276465] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 107.277214] Call Trace: [ 107.277532] simple_recursive_removal+0x4e/0x2e0 [ 107.278049] ? debugfs_remove+0x60/0x60 [ 107.278493] debugfs_remove+0x40/0x60 [ 107.278922] blk_trace_free+0xd/0x50 [ 107.279339] __blk_trace_remove+0x27/0x40 [ 107.279797] blk_trace_shutdown+0x30/0x40 [ 107.280256] __blk_release_queue+0xab/0x110 [ 107.280734] process_one_work+0x1b4/0x380 [ 107.281194] worker_thread+0x50/0x3c0 [ 107.281622] kthread+0xf9/0x130 [ 107.281994] ? process_one_work+0x380/0x380 [ 107.282467] ? kthread_park+0x90/0x90 [ 107.282895] ret_from_fork+0x1f/0x40 [ 107.283316] Modules linked in: loop(E) <etc> [ 107.288562] CR2: 00000000000000a0 [ 107.288957] ---[ end trace b885d243d441bbce ]---
This splat happens to be very similar to the one reported via kernel.org korg#205713, only that korg#205713 was for v4.19.83 and the above now includes the simple_recursive_removal() introduced via commit a3d1e7eb5abe ("simple_recursive_removal(): kernel-side rm -rf for ramfs-style filesystems") merged on v5.6.
korg#205713 then was used to create CVE-2019-19770 and claims that the bug is in a use-after-free in the debugfs core code. The implications of this being a generic UAF on debugfs would be much more severe, as it would imply parent dentries can sometimes not be positive, which we hold by design is just not possible.
Below is the splat explained with a bit more details, explaining what is happening in userspace, kernel, and a print of the CPU on, which the code runs on:
load loopback module
[ 13.603371] == blk_mq_debugfs_register(12) start [ 13.604040] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.604934] == blk_mq_debugfs_register(12) end [ 13.627382] == blk_mq_debugfs_register(12) start [ 13.628041] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.629240] == blk_mq_debugfs_register(12) end [ 13.651667] == blk_mq_debugfs_register(12) start [ 13.652836] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.655107] == blk_mq_debugfs_register(12) end [ 13.684917] == blk_mq_debugfs_register(12) start [ 13.687876] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.691588] == blk_mq_debugfs_register(13) end [ 13.707320] == blk_mq_debugfs_register(13) start [ 13.707863] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.708856] == blk_mq_debugfs_register(13) end [ 13.735623] == blk_mq_debugfs_register(13) start [ 13.736656] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.738411] == blk_mq_debugfs_register(13) end [ 13.763326] == blk_mq_debugfs_register(13) start [ 13.763972] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.765167] == blk_mq_debugfs_register(13) end [ 13.779510] == blk_mq_debugfs_register(13) start [ 13.780522] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.782338] == blk_mq_debugfs_register(13) end [ 13.783521] loop: module loaded
LOOP_CTL_DEL(loop0) #1
[ 13.803550] = __blk_release_queue(4) start [ 13.807772] == blk_trace_shutdown(4) start [ 13.810749] == blk_trace_shutdown(4) end [ 13.813437] = __blk_release_queue(4) calling blk_mq_debugfs_unregister() [ 13.817593] ==== blk_mq_debugfs_unregister(4) begin [ 13.817621] ==== blk_mq_debugfs_unregister(4) debugfs_remove_recursive(q->debugfs_dir) [ 13.821203] ==== blk_mq_debugfs_unregister(4) end q->debugfs_dir is NULL [ 13.826166] = __blk_release_queue(4) blk_mq_debugfs_unregister() end [ 13.832992] = __blk_release_queue(4) end
LOOP_CTL_ADD(loop0) #1
[ 13.843742] == blk_mq_debugfs_register(7) start [ 13.845569] == blk_mq_debugfs_register(7) q->debugfs_dir created [ 13.848628] == blk_mq_debugfs_register(7) end
BLKTRACE_SETUP(loop0) #1
[ 13.850924] == blk_trace_ioctl(7, BLKTRACESETUP) start [ 13.852852] === do_blk_trace_setup(7) start [ 13.854580] === do_blk_trace_setup(7) creating directory [ 13.856620] === do_blk_trace_setup(7) using what debugfs_lookup() gave [ 13.860635] === do_blk_trace_setup(7) end with ret: 0 [ 13.862615] == blk_trace_ioctl(7, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0) #2
[ 13.883304] = __blk_release_queue(7) start [ 13.885324] == blk_trace_shutdown(7) start [ 13.887197] == blk_trace_shutdown(7) calling __blk_trace_remove() [ 13.889807] == __blk_trace_remove(7) start [ 13.891669] === blk_trace_cleanup(7) start [ 13.911656] ====== blk_trace_free(7) start
LOOP_CTL_ADD(loop0) #2
[ 13.912709] == blk_mq_debugfs_register(2) start
---> From LOOP_CTL_DEL(loop0) #2
[ 13.915887] ====== blk_trace_free(7) end
---> From LOOP_CTL_ADD(loop0) #2
[ 13.918359] debugfs: Directory 'loop0' with parent 'block' already present! [ 13.926433] == blk_mq_debugfs_register(2) q->debugfs_dir created [ 13.930373] == blk_mq_debugfs_register(2) end
BLKTRACE_SETUP(loop0) #2
[ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start [ 13.936758] === do_blk_trace_setup(2) start [ 13.938944] === do_blk_trace_setup(2) creating directory [ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave
---> From LOOP_CTL_DEL(loop0) #2
[ 13.971046] === blk_trace_cleanup(7) end [ 13.973175] == __blk_trace_remove(7) end [ 13.975352] == blk_trace_shutdown(7) end [ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister() [ 13.980645] ==== blk_mq_debugfs_unregister(7) begin [ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir) [ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL [ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end [ 13.993155] = __blk_release_queue(7) end
---> From BLKTRACE_SETUP(loop0) #2
[ 13.995928] === do_blk_trace_setup(2) end with ret: 0 [ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0) #3
[ 14.035119] = __blk_release_queue(2) start [ 14.036925] == blk_trace_shutdown(2) start [ 14.038518] == blk_trace_shutdown(2) calling __blk_trace_remove() [ 14.040829] == __blk_trace_remove(2) start [ 14.042413] === blk_trace_cleanup(2) start
LOOP_CTL_ADD(loop0) #3
[ 14.072522] == blk_mq_debugfs_register(6) start
---> From LOOP_CTL_DEL(loop0) #3
[ 14.075151] ====== blk_trace_free(2) start
---> From LOOP_CTL_ADD(loop0) #3
[ 14.075882] == blk_mq_debugfs_register(6) q->debugfs_dir created
---> From LOOP_CTL_DEL(loop0) #3
[ 14.078624] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 14.084332] == blk_mq_debugfs_register(6) end [ 14.086971] #PF: supervisor write access in kernel mode [ 14.086974] #PF: error_code(0x0002) - not-present page [ 14.086977] PGD 0 P4D 0 [ 14.086984] Oops: 0002 [#1] SMP NOPTI [ 14.086990] CPU: 2 PID: 287 Comm: kworker/2:2 Tainted: G E 5.6.0-next-20200403+ #54 [ 14.086991] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 14.087002] Workqueue: events __blk_release_queue [ 14.087011] RIP: 0010:down_write+0x15/0x40 [ 14.090300] == blk_trace_ioctl(6, BLKTRACESETUP) start [ 14.093277] Code: eb ca e8 3e 34 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 14.093280] RSP: 0018:ffffc28a00533da8 EFLAGS: 00010246 [ 14.093284] RAX: 0000000000000000 RBX: ffff9f7a24d07980 RCX: ffffff8100000000 [ 14.093286] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 14.093287] RBP: 00000000000000a0 R08: 0000000000000000 R09: 0000000000000019 [ 14.093289] R10: 0000000000000774 R11: 0000000000000000 R12: 0000000000000000 [ 14.093291] R13: ffff9f7a2fab0400 R14: ffff9f7a21dd1140 R15: 00000000000000a0 [ 14.093294] FS: 0000000000000000(0000) GS:ffff9f7a2fa80000(0000) knlGS:0000000000000000 [ 14.093296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.093298] CR2: 00000000000000a0 CR3: 00000004293d2003 CR4: 0000000000360ee0 [ 14.093307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 14.093308] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 14.093310] Call Trace: [ 14.093324] simple_recursive_removal+0x4e/0x2e0 [ 14.093330] ? debugfs_remove+0x60/0x60 [ 14.093334] debugfs_remove+0x40/0x60 [ 14.093339] blk_trace_free+0x20/0x70 [ 14.093346] __blk_trace_remove+0x54/0x90 [ 14.096704] === do_blk_trace_setup(6) start [ 14.098534] blk_trace_shutdown+0x74/0x80 [ 14.100958] === do_blk_trace_setup(6) creating directory [ 14.104575] __blk_release_queue+0xbe/0x160 [ 14.104580] process_one_work+0x1b4/0x380 [ 14.104585] worker_thread+0x50/0x3c0 [ 14.104589] kthread+0xf9/0x130 [ 14.104593] ? process_one_work+0x380/0x380 [ 14.104596] ? kthread_park+0x90/0x90 [ 14.104599] ret_from_fork+0x1f/0x40 [ 14.104603] Modules linked in: loop(E) xfs(E) libcrc32c(E)
crct10dif_pclmul(E) crc32_pclmul(E) ghash_clmulni_intel(E) joydev(E) serio_raw(E) aesni_intel(E) glue_helper(E) virtio_balloon(E) evdev(E) crypto_simd(E) pcspkr(E) cryptd(E) i6300esb(E) button(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) virtio_net(E) net_failover(E) failover(E) virtio_blk(E) ata_generic(E) uhci_hcd(E) ata_piix(E) ehci_hcd(E) nvme(E) libata(E) crc32c_intel(E) usbcore(E) psmouse(E) nvme_core(E) virtio_pci(E) scsi_mod(E) virtio_ring(E) t10_pi(E) virtio(E) i2c_piix4(E) floppy(E)
[ 14.107400] === do_blk_trace_setup(6) using what debugfs_lookup() gave [ 14.108939] CR2: 00000000000000a0 [ 14.110589] === do_blk_trace_setup(6) end with ret: 0 [ 14.111592] ---[ end trace 7a783b33b9614db9 ]---
The root cause to this issue is that debugfs_lookup() can find a previous incarnation's dir of the same name which is about to get removed from a not yet schedule work.
We can fix the UAF by simply using a debugfs directory which moving forward will always be accessible if debugfs is enabled, this way, its allocated and avaialble always for both request-based block drivers or make_request drivers (multiqueue) block drivers.
This simplifies the code considerably, with the only penalty now being that we're always creating the request queue debugfs directory for the request-based block device drivers.
The UAF then is not a core debugfs issue, but instead a misuse of debugfs, and this issue can only be triggered if you are root, and misuse blktrace.
This issue can be reproduced with break-blktrace [2] using:
break-blktrace -c 10 -d -s
This patch fixes this issue. Note that there is also another respective UAF but from the ioctl path [3], this should also fix that issue.
This patch then also disputes the severity of CVE-2019-19770 as this issue is only possible by being root and using blktrace.
It is not a core debugfs issue.
[0] https://bugzilla.kernel.org/show_bug.cgi?id=205713
[1] https://nvd.nist.gov/vuln/detail/CVE-2019-19770
[2] https://github.com/mcgrof/break-blktrace
[3] https://lore.kernel.org/lkml/[email protected]/
--- block/blk-debugfs.c | 12 ++++++++++++ block/blk-mq-debugfs.c | 5 ----- block/blk-sysfs.c | 3 +++ block/blk.h | 10 ++++++++++ include/linux/blkdev.h | 5 ++++- include/linux/blktrace_api.h | 1 - kernel/trace/blktrace.c | 19 ++++++++----------- 7 files changed, 37 insertions(+), 18 deletions(-)
block/blk-debugfs.c +12 -0
@@ -13,3 +13,15 @@ void blk_debugfs_register(void) { blk_debugfs_root =debugfs_create_dir ("block" ,NULL ); } + +void blk_queue_debugfs_register (struct request_queue *q) +{ + q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root); +} + +void blk_queue_debugfs_unregister (struct request_queue *q) +{ +debugfs_remove_recursive (q->debugfs_dir); + q->debugfs_dir =NULL ; +}
block/blk-mq-debugfs.c +0 -5
@@ -823,9 +823,6 @@ void blk_mq_debugfs_register(struct request_queue *q)struct blk_mq_hw_ctx *hctx;int i; - q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root); -debugfs_create_files (q->debugfs_dir, q, blk_mq_debugfs_queue_attrs);/*
@@ -856,9 +853,7 @@ void blk_mq_debugfs_register(struct request_queue *q)void blk_mq_debugfs_unregister (struct request_queue *q) { -debugfs_remove_recursive (q->debugfs_dir); q->sched_debugfs_dir =NULL ; - q->debugfs_dir =NULL ; } static void blk_mq_debugfs_register_ctx(struct blk_mq_hw_ctx *hctx,
block/blk-sysfs.c +3 -0
@@ -895,6 +895,7 @@ static void __blk_release_queue(struct work_struct *work)blk_trace_shutdown (q); +blk_queue_debugfs_unregister (q);if (queue_is_mq (q))blk_mq_debugfs_unregister (q);
@@ -975,6 +976,8 @@ int blk_register_queue(struct gendisk *disk) goto unlock; } +blk_queue_debugfs_register (q); +if (queue_is_mq (q)) {__blk_mq_register_dev (dev, q);blk_mq_debugfs_register (q);
block/blk.h +10 -0
@@ -489,10 +489,20 @@ int __bio_add_pc_page(struct request_queue *q, struct bio *bio, bool *same_page);#ifdef CONFIG_DEBUG_FS void blk_debugfs_register (void ); +void blk_queue_debugfs_register (struct request_queue *q); +void blk_queue_debugfs_unregister (struct request_queue *q);#else static inline void blk_debugfs_register (void ) { } + +static inline void blk_queue_debugfs_register (struct request_queue *q) +{ +} + +static inline void blk_queue_debugfs_unregister (struct request_queue *q) +{ +}#endif /* CONFIG_DEBUG_FS */ #endif /* BLK_INTERNAL_H */
include/linux/blkdev.h +4 -1
@@ -569,8 +569,11 @@ struct request_queue {struct list_head tag_set_list;struct bio_set bio_split; -#ifdef CONFIG_BLK_DEBUG_FS +#ifdef CONFIG_DEBUG_FS +/* Used by block/blk-*debugfs.c and kernel/trace/blktrace.c */ struct dentry *debugfs_dir; +#endif +#ifdef CONFIG_BLK_DEBUG_FS struct dentry *sched_debugfs_dir;struct dentry *rqos_debugfs_dir;#endif
include/linux/blktrace_api.h +0 -1
@@ -22,7 +22,6 @@ struct blk_trace {u64 end_lba;u32 pid;u32 dev; -struct dentry *dir;struct dentry *dropped_file;struct dentry *msg_file;struct list_head running_list;
kernel/trace/blktrace.c +8 -11
@@ -311,7 +311,6 @@ static void blk_trace_free(struct blk_trace *bt)debugfs_remove (bt->msg_file);debugfs_remove (bt->dropped_file);relay_close (bt->rchan); -debugfs_remove (bt->dir);free_percpu (bt->sequence);free_percpu (bt->msg_data);kfree (bt);
@@ -476,7 +475,6 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, struct blk_user_trace_setup *buts) {struct blk_trace *bt =NULL ; -struct dentry *dir =NULL ;int ret;if (!buts->buf_size || !buts->buf_nr)
@@ -485,6 +483,9 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev,if (!blk_debugfs_root)return -ENOENT ; +if (!q->debugfs_dir) +return -ENOENT ; +strncpy (buts->name, name,BLKTRACE_BDEV_SIZE ); buts->name[BLKTRACE_BDEV_SIZE -1 ] ='\0' ;
@@ -509,21 +510,19 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, ret = -ENOENT ; - dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); - bt->dev = dev;atomic_set (&bt->dropped,0 );INIT_LIST_HEAD (&bt->running_list); ret = -EIO ; - bt->dropped_file =debugfs_create_file ("dropped" ,0444 , dir, bt, + bt->dropped_file =debugfs_create_file ("dropped" ,0444 , + q->debugfs_dir, bt, &blk_dropped_fops); - bt->msg_file =debugfs_create_file ("msg" ,0222 , dir, bt, &blk_msg_fops); + bt->msg_file =debugfs_create_file ("msg" ,0222 , q->debugfs_dir, + bt, &blk_msg_fops); - bt->rchan =relay_open ("trace" , dir, buts->buf_size, + bt->rchan =relay_open ("trace" , q->debugfs_dir, buts->buf_size, buts->buf_nr, &blk_relay_callbacks, bt);if (!bt->rchan) goto err;
@@ -551,8 +550,6 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, ret =0 ; err: -if (dir && !bt->dir) -dput (dir);if (ret)blk_trace_free (bt);return ret;
Signature · 1 line
2.25.1
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 263 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
[ 107.193134] debugfs: Directory 'loop0' with parent 'block' already present! [ 107.254615] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 107.258785] #PF: supervisor write access in kernel mode [ 107.262035] #PF: error_code(0x0002) - not-present page [ 107.264106] PGD 0 P4D 0 [ 107.264404] Oops: 0002 [#1] SMP NOPTI [ 107.264803] CPU: 8 PID: 674 Comm: kworker/8:2 Tainted: G E 5.6.0-rc7-next-20200327 #1 [ 107.265712] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 107.266553] Workqueue: events __blk_release_queue [ 107.267051] RIP: 0010:down_write+0x15/0x40 [ 107.267488] Code: eb ca e8 ee a5 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 107.269300] RSP: 0018:ffff9927c06efda8 EFLAGS: 00010246 [ 107.269841] RAX: 0000000000000000 RBX: ffff8be7e73b0600 RCX: ffffff8100000000 [ 107.270559] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 107.271281] RBP: 00000000000000a0 R08: ffff8be7ebc80fa8 R09: ffff8be7ebc80fa8 [ 107.272001] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 107.272722] R13: ffff8be7efc30400 R14: ffff8be7e0571200 R15: 00000000000000a0 [ 107.273475] FS: 0000000000000000(0000) GS:ffff8be7efc00000(0000) knlGS:0000000000000000 [ 107.274346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 107.274968] CR2: 00000000000000a0 CR3: 000000042abee003 CR4: 0000000000360ee0 [ 107.275710] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 107.276465] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 107.277214] Call Trace: [ 107.277532] simple_recursive_removal+0x4e/0x2e0 [ 107.278049] ? debugfs_remove+0x60/0x60 [ 107.278493] debugfs_remove+0x40/0x60 [ 107.278922] blk_trace_free+0xd/0x50 [ 107.279339] __blk_trace_remove+0x27/0x40 [ 107.279797] blk_trace_shutdown+0x30/0x40 [ 107.280256] __blk_release_queue+0xab/0x110 [ 107.280734] process_one_work+0x1b4/0x380 [ 107.281194] worker_thread+0x50/0x3c0 [ 107.281622] kthread+0xf9/0x130 [ 107.281994] ? process_one_work+0x380/0x380 [ 107.282467] ? kthread_park+0x90/0x90 [ 107.282895] ret_from_fork+0x1f/0x40 [ 107.283316] Modules linked in: loop(E) <etc> [ 107.288562] CR2: 00000000000000a0 [ 107.288957] ---[ end trace b885d243d441bbce ]---This splat happens to be very similar to the one reported via kernel.org korg#205713, only that korg#205713 was for v4.19.83 and the above now includes the
simple_recursive_removal()introduced via commita3d1e7eb5abe("simple_recursive_removal(): kernel-side rm -rf for ramfs-style filesystems") merged on v5.6.korg#205713 then was used to create CVE-2019-19770 and claims that the bug is in a use-after-free in the debugfs core code. The implications of this being a generic UAF on debugfs would be much more severe, as it would imply parent dentries can sometimes not be positive, which we hold by design is just not possible.
Below is the splat explained with a bit more details, explaining what is happening in userspace, kernel, and a print of the CPU on, which the code runs on:
load loopback module
[ 13.603371] == blk_mq_debugfs_register(12) start [ 13.604040] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.604934] == blk_mq_debugfs_register(12) end [ 13.627382] == blk_mq_debugfs_register(12) start [ 13.628041] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.629240] == blk_mq_debugfs_register(12) end [ 13.651667] == blk_mq_debugfs_register(12) start [ 13.652836] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.655107] == blk_mq_debugfs_register(12) end [ 13.684917] == blk_mq_debugfs_register(12) start [ 13.687876] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.691588] == blk_mq_debugfs_register(13) end [ 13.707320] == blk_mq_debugfs_register(13) start [ 13.707863] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.708856] == blk_mq_debugfs_register(13) end [ 13.735623] == blk_mq_debugfs_register(13) start [ 13.736656] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.738411] == blk_mq_debugfs_register(13) end [ 13.763326] == blk_mq_debugfs_register(13) start [ 13.763972] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.765167] == blk_mq_debugfs_register(13) end [ 13.779510] == blk_mq_debugfs_register(13) start [ 13.780522] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.782338] == blk_mq_debugfs_register(13) end [ 13.783521] loop: module loaded
LOOP_CTL_DEL(loop0)#1[ 13.803550] = __blk_release_queue(4) start [ 13.807772] == blk_trace_shutdown(4) start [ 13.810749] == blk_trace_shutdown(4) end [ 13.813437] = __blk_release_queue(4) calling blk_mq_debugfs_unregister() [ 13.817593] ==== blk_mq_debugfs_unregister(4) begin [ 13.817621] ==== blk_mq_debugfs_unregister(4) debugfs_remove_recursive(q->debugfs_dir) [ 13.821203] ==== blk_mq_debugfs_unregister(4) end q->debugfs_dir is NULL [ 13.826166] = __blk_release_queue(4) blk_mq_debugfs_unregister() end [ 13.832992] = __blk_release_queue(4) end
LOOP_CTL_ADD(loop0)#1[ 13.843742] == blk_mq_debugfs_register(7) start [ 13.845569] == blk_mq_debugfs_register(7) q->debugfs_dir created [ 13.848628] == blk_mq_debugfs_register(7) end
BLKTRACE_SETUP(loop0)#1[ 13.850924] == blk_trace_ioctl(7, BLKTRACESETUP) start [ 13.852852] === do_blk_trace_setup(7) start [ 13.854580] === do_blk_trace_setup(7) creating directory [ 13.856620] === do_blk_trace_setup(7) using what debugfs_lookup() gave [ 13.860635] === do_blk_trace_setup(7) end with ret: 0 [ 13.862615] == blk_trace_ioctl(7, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0)#2[ 13.883304] = __blk_release_queue(7) start [ 13.885324] == blk_trace_shutdown(7) start [ 13.887197] == blk_trace_shutdown(7) calling __blk_trace_remove() [ 13.889807] == __blk_trace_remove(7) start [ 13.891669] === blk_trace_cleanup(7) start [ 13.911656] ====== blk_trace_free(7) start
LOOP_CTL_ADD(loop0)#2[ 13.912709] == blk_mq_debugfs_register(2) start---> From
LOOP_CTL_DEL(loop0)#2[ 13.915887] ====== blk_trace_free(7) end---> From
LOOP_CTL_ADD(loop0)#2[ 13.918359] debugfs: Directory 'loop0' with parent 'block' already present! [ 13.926433] == blk_mq_debugfs_register(2) q->debugfs_dir created [ 13.930373] == blk_mq_debugfs_register(2) end
BLKTRACE_SETUP(loop0)#2[ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start [ 13.936758] === do_blk_trace_setup(2) start [ 13.938944] === do_blk_trace_setup(2) creating directory [ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave---> From
LOOP_CTL_DEL(loop0)#2[ 13.971046] === blk_trace_cleanup(7) end [ 13.973175] == __blk_trace_remove(7) end [ 13.975352] == blk_trace_shutdown(7) end [ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister() [ 13.980645] ==== blk_mq_debugfs_unregister(7) begin [ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir) [ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL [ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end [ 13.993155] = __blk_release_queue(7) end---> From
BLKTRACE_SETUP(loop0)#2[ 13.995928] === do_blk_trace_setup(2) end with ret: 0 [ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0)#3[ 14.035119] = __blk_release_queue(2) start [ 14.036925] == blk_trace_shutdown(2) start [ 14.038518] == blk_trace_shutdown(2) calling __blk_trace_remove() [ 14.040829] == __blk_trace_remove(2) start [ 14.042413] === blk_trace_cleanup(2) start
LOOP_CTL_ADD(loop0)#3[ 14.072522] == blk_mq_debugfs_register(6) start---> From
LOOP_CTL_DEL(loop0)#3[ 14.075151] ====== blk_trace_free(2) start---> From
LOOP_CTL_ADD(loop0)#3[ 14.075882] == blk_mq_debugfs_register(6) q->debugfs_dir created---> From
LOOP_CTL_DEL(loop0)#3[ 14.078624] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 14.084332] == blk_mq_debugfs_register(6) end [ 14.086971] #PF: supervisor write access in kernel mode [ 14.086974] #PF: error_code(0x0002) - not-present page [ 14.086977] PGD 0 P4D 0 [ 14.086984] Oops: 0002 [#1] SMP NOPTI [ 14.086990] CPU: 2 PID: 287 Comm: kworker/2:2 Tainted: G E 5.6.0-next-20200403+ #54 [ 14.086991] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 14.087002] Workqueue: events __blk_release_queue [ 14.087011] RIP: 0010:down_write+0x15/0x40 [ 14.090300] == blk_trace_ioctl(6, BLKTRACESETUP) start [ 14.093277] Code: eb ca e8 3e 34 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 14.093280] RSP: 0018:ffffc28a00533da8 EFLAGS: 00010246 [ 14.093284] RAX: 0000000000000000 RBX: ffff9f7a24d07980 RCX: ffffff8100000000 [ 14.093286] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 14.093287] RBP: 00000000000000a0 R08: 0000000000000000 R09: 0000000000000019 [ 14.093289] R10: 0000000000000774 R11: 0000000000000000 R12: 0000000000000000 [ 14.093291] R13: ffff9f7a2fab0400 R14: ffff9f7a21dd1140 R15: 00000000000000a0 [ 14.093294] FS: 0000000000000000(0000) GS:ffff9f7a2fa80000(0000) knlGS:0000000000000000 [ 14.093296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.093298] CR2: 00000000000000a0 CR3: 00000004293d2003 CR4: 0000000000360ee0 [ 14.093307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 14.093308] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 14.093310] Call Trace: [ 14.093324] simple_recursive_removal+0x4e/0x2e0 [ 14.093330] ? debugfs_remove+0x60/0x60 [ 14.093334] debugfs_remove+0x40/0x60 [ 14.093339] blk_trace_free+0x20/0x70 [ 14.093346] __blk_trace_remove+0x54/0x90 [ 14.096704] === do_blk_trace_setup(6) start [ 14.098534] blk_trace_shutdown+0x74/0x80 [ 14.100958] === do_blk_trace_setup(6) creating directory [ 14.104575] __blk_release_queue+0xbe/0x160 [ 14.104580] process_one_work+0x1b4/0x380 [ 14.104585] worker_thread+0x50/0x3c0 [ 14.104589] kthread+0xf9/0x130 [ 14.104593] ? process_one_work+0x380/0x380 [ 14.104596] ? kthread_park+0x90/0x90 [ 14.104599] ret_from_fork+0x1f/0x40 [ 14.104603] Modules linked in: loop(E) xfs(E) libcrc32c(E)
crct10dif_pclmul(E)crc32_pclmul(E)ghash_clmulni_intel(E)joydev(E)serio_raw(E)aesni_intel(E)glue_helper(E)virtio_balloon(E)evdev(E)crypto_simd(E)pcspkr(E) cryptd(E) i6300esb(E) button(E)ip_tables(E)x_tables(E)autofs4(E) ext4(E)crc32c_generic(E)crc16(E) mbcache(E) jbd2(E)virtio_net(E)net_failover(E)failover(E)virtio_blk(E)ata_generic(E)uhci_hcd(E)ata_piix(E)ehci_hcd(E)nvme(E) libata(E)crc32c_intel(E)usbcore(E) psmouse(E)nvme_core(E)virtio_pci(E)scsi_mod(E)virtio_ring(E)t10_pi(E)virtio(E)i2c_piix4(E)floppy(E)[ 14.107400] === do_blk_trace_setup(6) using what debugfs_lookup() gave [ 14.108939] CR2: 00000000000000a0 [ 14.110589] === do_blk_trace_setup(6) end with ret: 0 [ 14.111592] ---[ end trace 7a783b33b9614db9 ]---The root cause to this issue is that
debugfs_lookup()can find a previous incarnation's dir of the same name which is about to get removed from a not yet schedule work.We can fix the UAF by simply using a debugfs directory which moving forward will always be accessible if debugfs is enabled, this way, its allocated and avaialble always for both request-based block drivers or
make_requestdrivers (multiqueue) block drivers.This simplifies the code considerably, with the only penalty now being that we're always creating the request queue debugfs directory for the request-based block device drivers.
The UAF then is not a core debugfs issue, but instead a misuse of debugfs, and this issue can only be triggered if you are root, and misuse blktrace.
This issue can be reproduced with break-blktrace [2] using:
break-blktrace -c 10 -d -s
This patch fixes this issue. Note that there is also another respective UAF but from the ioctl path [3], this should also fix that issue.
This patch then also disputes the severity of CVE-2019-19770 as this issue is only possible by being root and using blktrace.
It is not a core debugfs issue.
[0] https://bugzilla.kernel.org/show_bug.cgi?id=205713
[1] https://nvd.nist.gov/vuln/detail/CVE-2019-19770
[2] https://github.com/mcgrof/break-blktrace
[3] https://lore.kernel.org/lkml/[email protected]/
Quoted 10 lines
Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Reported-by [email protected]Fixes 6ac93117ab00 ("blktrace: use existing disk debugfs directory")Signed-off-by Luis Chamberlain <[email protected]>
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Luis ChamberlainLooks good:
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Luis ChamberlainQuoted 4 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Luis ChamberlainQuoted 8 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
I think this patch makes this all cleaner anyway, but - without the apparent loop bug mentioned by Bart which allows removal of the loop device while blktrace is active (if I read that right), can this still happen?
-Eric
Quoted 39 lines
[ 107.193134] debugfs: Directory 'loop0' with parent 'block' already present! [ 107.254615] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 107.258785] #PF: supervisor write access in kernel mode [ 107.262035] #PF: error_code(0x0002) - not-present page [ 107.264106] PGD 0 P4D 0 [ 107.264404] Oops: 0002 [#1] SMP NOPTI [ 107.264803] CPU: 8 PID: 674 Comm: kworker/8:2 Tainted: G E 5.6.0-rc7-next-20200327 #1 [ 107.265712] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 107.266553] Workqueue: events __blk_release_queue [ 107.267051] RIP: 0010:down_write+0x15/0x40 [ 107.267488] Code: eb ca e8 ee a5 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 107.269300] RSP: 0018:ffff9927c06efda8 EFLAGS: 00010246 [ 107.269841] RAX: 0000000000000000 RBX: ffff8be7e73b0600 RCX: ffffff8100000000 [ 107.270559] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 107.271281] RBP: 00000000000000a0 R08: ffff8be7ebc80fa8 R09: ffff8be7ebc80fa8 [ 107.272001] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 107.272722] R13: ffff8be7efc30400 R14: ffff8be7e0571200 R15: 00000000000000a0 [ 107.273475] FS: 0000000000000000(0000) GS:ffff8be7efc00000(0000) knlGS:0000000000000000 [ 107.274346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 107.274968] CR2: 00000000000000a0 CR3: 000000042abee003 CR4: 0000000000360ee0 [ 107.275710] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 107.276465] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 107.277214] Call Trace: [ 107.277532] simple_recursive_removal+0x4e/0x2e0 [ 107.278049] ? debugfs_remove+0x60/0x60 [ 107.278493] debugfs_remove+0x40/0x60 [ 107.278922] blk_trace_free+0xd/0x50 [ 107.279339] __blk_trace_remove+0x27/0x40 [ 107.279797] blk_trace_shutdown+0x30/0x40 [ 107.280256] __blk_release_queue+0xab/0x110 [ 107.280734] process_one_work+0x1b4/0x380 [ 107.281194] worker_thread+0x50/0x3c0 [ 107.281622] kthread+0xf9/0x130 [ 107.281994] ? process_one_work+0x380/0x380 [ 107.282467] ? kthread_park+0x90/0x90 [ 107.282895] ret_from_fork+0x1f/0x40 [ 107.283316] Modules linked in: loop(E) <etc> [ 107.288562] CR2: 00000000000000a0 [ 107.288957] ---[ end trace b885d243d441bbce ]---
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 6 lines · Eric Sandeen
Quoted 5 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.
Quoted 7 lines
Quoted 3 lines
Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
I think this patch makes this all cleaner anyway, but - without the apparent loop bug mentioned by Bart which allows removal of the loop device while blktrace is active (if I read that right), can this still happen?
That's a great question. Even if the loop driver fix would be sufficient to fix the blktrace debugfs use-after free I think the block layer patches from this series are still very valuable. As explained in the cover letter this patch series fixes more than only the blktrace debugfs use-after-free.
Thanks,
Bart.
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Eric SandeenQuoted 6 lines · Eric Sandeen
Quoted 5 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.
Quoted 7 lines
Quoted 3 lines
Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
I think this patch makes this all cleaner anyway, but - without the apparent loop bug mentioned by Bart which allows removal of the loop device while blktrace is active (if I read that right), can this still happen?
I have not tested that, but some modifications of the break-blktrace program could enable us to test that, however I don't think the race would be possible after patch 3/5 "blktrace: refcount the request_queue during ioctl" is merged, as removal then a pending blktrace would refcount the request_queue and the removal would have to wait until the refcount is decremeneted, until after the blktrace ioctl.
Luis
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 11 lines · Luis Chamberlain
Quoted 10 lines · Eric Sandeen
Quoted 8 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
Quoted 6 lines
Quoted 3 lines
I think this patch makes this all cleaner anyway, but - without the apparent loop bug mentioned by Bart which allows removal of the loop device while blktrace is active (if I read that right), can this still happen?
I have not tested that, but some modifications of the break-blktrace program could enable us to test that
FWIW, I modified it to modprobe & rmmod scsi_debug instead of the loop ioctls, and the module can't be unloaded after the blktrace is started since it's busy.
Not sure that's equivalent tho.
Quoted 5 lines
however I don't think the race
would be possible after patch 3/5 "blktrace: refcount therequest_queueduring ioctl" is merged, as removal then a pending blktrace would refcount therequest_queueand the removal would have to wait until the refcount is decremeneted, until after the blktrace ioctl.
I'm out of my depth in the block layer, not sure who's supposed to take refs on what. ;)
-Eric
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 18 lines · Eric Sandeen
Quoted 15 lines · Luis Chamberlain
Quoted 13 lines · Eric Sandeen
Quoted 8 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
I think this patch makes this all cleaner anyway, but - without the apparent loop bug mentioned by Bart which allows removal of the loop device while blktrace is active (if I read that right), can this still happen?
Quoted 7 lines
Quoted 2 lines
I have not tested that, but some modifications of the break-blktrace program could enable us to test that
FWIW, I modified it to modprobe & rmmod
scsi_debuginstead of the loop ioctls, and the module can't be unloaded after the blktrace is started since it's busy.Not sure that's equivalent tho.
Given what Bart mentioned about sd_open, that might be the saving grace for blocking async behaviour on scsi. If it only refcounted the request_queue as the loop driver it would have been exposed to the same bug.
Quoted 8 lines
Quoted 5 lines
however I don't think the race
would be possible after patch 3/5 "blktrace: refcount therequest_queueduring ioctl" is merged, as removal then a pending blktrace would refcount therequest_queueand the removal would have to wait until the refcount is decremeneted, until after the blktrace ioctl.I'm out of my depth in the block layer, not sure who's supposed to take refs on what. ;)
I'm new to all this code, only been a few weeks looking into it, but am trying to do my best ot make sense of it. So the above is what I can tell would be needed.
Luis
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Luis ChamberlainQuoted 232 lines · Luis Chamberlain
On commit
6ac93117ab00("blktrace: use existing disk debugfs directory") merged on v4.12 Omar fixed the original blktrace code for request-based drivers (multiqueue). This however left in place a possible crash, if you happen to abuse blktrace in a way it was not intended.Namely, if you loop adding a device, setup the blktrace with BLKTRACESETUP, forget to BLKTRACETEARDOWN, and then just remove the device you end up with a panic:
[ 107.193134] debugfs: Directory 'loop0' with parent 'block' already present! [ 107.254615] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 107.258785] #PF: supervisor write access in kernel mode [ 107.262035] #PF: error_code(0x0002) - not-present page [ 107.264106] PGD 0 P4D 0 [ 107.264404] Oops: 0002 [#1] SMP NOPTI [ 107.264803] CPU: 8 PID: 674 Comm: kworker/8:2 Tainted: G E 5.6.0-rc7-next-20200327 #1 [ 107.265712] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 107.266553] Workqueue: events __blk_release_queue [ 107.267051] RIP: 0010:down_write+0x15/0x40 [ 107.267488] Code: eb ca e8 ee a5 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 107.269300] RSP: 0018:ffff9927c06efda8 EFLAGS: 00010246 [ 107.269841] RAX: 0000000000000000 RBX: ffff8be7e73b0600 RCX: ffffff8100000000 [ 107.270559] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 107.271281] RBP: 00000000000000a0 R08: ffff8be7ebc80fa8 R09: ffff8be7ebc80fa8 [ 107.272001] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 107.272722] R13: ffff8be7efc30400 R14: ffff8be7e0571200 R15: 00000000000000a0 [ 107.273475] FS: 0000000000000000(0000) GS:ffff8be7efc00000(0000) knlGS:0000000000000000 [ 107.274346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 107.274968] CR2: 00000000000000a0 CR3: 000000042abee003 CR4: 0000000000360ee0 [ 107.275710] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 107.276465] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 107.277214] Call Trace: [ 107.277532] simple_recursive_removal+0x4e/0x2e0 [ 107.278049] ? debugfs_remove+0x60/0x60 [ 107.278493] debugfs_remove+0x40/0x60 [ 107.278922] blk_trace_free+0xd/0x50 [ 107.279339] __blk_trace_remove+0x27/0x40 [ 107.279797] blk_trace_shutdown+0x30/0x40 [ 107.280256] __blk_release_queue+0xab/0x110 [ 107.280734] process_one_work+0x1b4/0x380 [ 107.281194] worker_thread+0x50/0x3c0 [ 107.281622] kthread+0xf9/0x130 [ 107.281994] ? process_one_work+0x380/0x380 [ 107.282467] ? kthread_park+0x90/0x90 [ 107.282895] ret_from_fork+0x1f/0x40 [ 107.283316] Modules linked in: loop(E) <etc> [ 107.288562] CR2: 00000000000000a0 [ 107.288957] ---[ end trace b885d243d441bbce ]---This splat happens to be very similar to the one reported via kernel.org korg#205713, only that korg#205713 was for v4.19.83 and the above now includes the
simple_recursive_removal()introduced via commita3d1e7eb5abe("simple_recursive_removal(): kernel-side rm -rf for ramfs-style filesystems") merged on v5.6.korg#205713 then was used to create CVE-2019-19770 and claims that the bug is in a use-after-free in the debugfs core code. The implications of this being a generic UAF on debugfs would be much more severe, as it would imply parent dentries can sometimes not be positive, which we hold by design is just not possible.
Below is the splat explained with a bit more details, explaining what is happening in userspace, kernel, and a print of the CPU on, which the code runs on:
load loopback module
[ 13.603371] == blk_mq_debugfs_register(12) start [ 13.604040] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.604934] == blk_mq_debugfs_register(12) end [ 13.627382] == blk_mq_debugfs_register(12) start [ 13.628041] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.629240] == blk_mq_debugfs_register(12) end [ 13.651667] == blk_mq_debugfs_register(12) start [ 13.652836] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.655107] == blk_mq_debugfs_register(12) end [ 13.684917] == blk_mq_debugfs_register(12) start [ 13.687876] == blk_mq_debugfs_register(12) q->debugfs_dir created [ 13.691588] == blk_mq_debugfs_register(13) end [ 13.707320] == blk_mq_debugfs_register(13) start [ 13.707863] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.708856] == blk_mq_debugfs_register(13) end [ 13.735623] == blk_mq_debugfs_register(13) start [ 13.736656] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.738411] == blk_mq_debugfs_register(13) end [ 13.763326] == blk_mq_debugfs_register(13) start [ 13.763972] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.765167] == blk_mq_debugfs_register(13) end [ 13.779510] == blk_mq_debugfs_register(13) start [ 13.780522] == blk_mq_debugfs_register(13) q->debugfs_dir created [ 13.782338] == blk_mq_debugfs_register(13) end [ 13.783521] loop: module loaded
LOOP_CTL_DEL(loop0)#1[ 13.803550] = __blk_release_queue(4) start [ 13.807772] == blk_trace_shutdown(4) start [ 13.810749] == blk_trace_shutdown(4) end [ 13.813437] = __blk_release_queue(4) calling blk_mq_debugfs_unregister() [ 13.817593] ==== blk_mq_debugfs_unregister(4) begin [ 13.817621] ==== blk_mq_debugfs_unregister(4) debugfs_remove_recursive(q->debugfs_dir) [ 13.821203] ==== blk_mq_debugfs_unregister(4) end q->debugfs_dir is NULL [ 13.826166] = __blk_release_queue(4) blk_mq_debugfs_unregister() end [ 13.832992] = __blk_release_queue(4) end
LOOP_CTL_ADD(loop0)#1[ 13.843742] == blk_mq_debugfs_register(7) start [ 13.845569] == blk_mq_debugfs_register(7) q->debugfs_dir created [ 13.848628] == blk_mq_debugfs_register(7) end
BLKTRACE_SETUP(loop0)#1[ 13.850924] == blk_trace_ioctl(7, BLKTRACESETUP) start [ 13.852852] === do_blk_trace_setup(7) start [ 13.854580] === do_blk_trace_setup(7) creating directory [ 13.856620] === do_blk_trace_setup(7) using what debugfs_lookup() gave [ 13.860635] === do_blk_trace_setup(7) end with ret: 0 [ 13.862615] == blk_trace_ioctl(7, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0)#2[ 13.883304] = __blk_release_queue(7) start [ 13.885324] == blk_trace_shutdown(7) start [ 13.887197] == blk_trace_shutdown(7) calling __blk_trace_remove() [ 13.889807] == __blk_trace_remove(7) start [ 13.891669] === blk_trace_cleanup(7) start [ 13.911656] ====== blk_trace_free(7) start
LOOP_CTL_ADD(loop0)#2[ 13.912709] == blk_mq_debugfs_register(2) start---> From
LOOP_CTL_DEL(loop0)#2[ 13.915887] ====== blk_trace_free(7) end---> From
LOOP_CTL_ADD(loop0)#2[ 13.918359] debugfs: Directory 'loop0' with parent 'block' already present! [ 13.926433] == blk_mq_debugfs_register(2) q->debugfs_dir created [ 13.930373] == blk_mq_debugfs_register(2) end
BLKTRACE_SETUP(loop0)#2[ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start [ 13.936758] === do_blk_trace_setup(2) start [ 13.938944] === do_blk_trace_setup(2) creating directory [ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave---> From
LOOP_CTL_DEL(loop0)#2[ 13.971046] === blk_trace_cleanup(7) end [ 13.973175] == __blk_trace_remove(7) end [ 13.975352] == blk_trace_shutdown(7) end [ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister() [ 13.980645] ==== blk_mq_debugfs_unregister(7) begin [ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir) [ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL [ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end [ 13.993155] = __blk_release_queue(7) end---> From
BLKTRACE_SETUP(loop0)#2[ 13.995928] === do_blk_trace_setup(2) end with ret: 0 [ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) end
LOOP_CTL_DEL(loop0)#3[ 14.035119] = __blk_release_queue(2) start [ 14.036925] == blk_trace_shutdown(2) start [ 14.038518] == blk_trace_shutdown(2) calling __blk_trace_remove() [ 14.040829] == __blk_trace_remove(2) start [ 14.042413] === blk_trace_cleanup(2) start
LOOP_CTL_ADD(loop0)#3[ 14.072522] == blk_mq_debugfs_register(6) start---> From
LOOP_CTL_DEL(loop0)#3[ 14.075151] ====== blk_trace_free(2) start---> From
LOOP_CTL_ADD(loop0)#3[ 14.075882] == blk_mq_debugfs_register(6) q->debugfs_dir created---> From
LOOP_CTL_DEL(loop0)#3[ 14.078624] BUG: kernel NULL pointer dereference, address: 00000000000000a0 [ 14.084332] == blk_mq_debugfs_register(6) end [ 14.086971] #PF: supervisor write access in kernel mode [ 14.086974] #PF: error_code(0x0002) - not-present page [ 14.086977] PGD 0 P4D 0 [ 14.086984] Oops: 0002 [#1] SMP NOPTI [ 14.086990] CPU: 2 PID: 287 Comm: kworker/2:2 Tainted: G E 5.6.0-next-20200403+ #54 [ 14.086991] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014 [ 14.087002] Workqueue: events __blk_release_queue [ 14.087011] RIP: 0010:down_write+0x15/0x40 [ 14.090300] == blk_trace_ioctl(6, BLKTRACESETUP) start [ 14.093277] Code: eb ca e8 3e 34 8d ff cc cc cc cc cc cc cc cc cc cc cc cc cc cc 0f 1f 44 00 00 55 48 89 fd e8 52 db ff ff 31 c0 ba 01 00 00 00 <f0> 48 0f b1 55 00 75 0f 65 48 8b 04 25 c0 8b 01 00 48 89 45 08 5d [ 14.093280] RSP: 0018:ffffc28a00533da8 EFLAGS: 00010246 [ 14.093284] RAX: 0000000000000000 RBX: ffff9f7a24d07980 RCX: ffffff8100000000 [ 14.093286] RDX: 0000000000000001 RSI: ffffff8100000000 RDI: 00000000000000a0 [ 14.093287] RBP: 00000000000000a0 R08: 0000000000000000 R09: 0000000000000019 [ 14.093289] R10: 0000000000000774 R11: 0000000000000000 R12: 0000000000000000 [ 14.093291] R13: ffff9f7a2fab0400 R14: ffff9f7a21dd1140 R15: 00000000000000a0 [ 14.093294] FS: 0000000000000000(0000) GS:ffff9f7a2fa80000(0000) knlGS:0000000000000000 [ 14.093296] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.093298] CR2: 00000000000000a0 CR3: 00000004293d2003 CR4: 0000000000360ee0 [ 14.093307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 14.093308] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 14.093310] Call Trace: [ 14.093324] simple_recursive_removal+0x4e/0x2e0 [ 14.093330] ? debugfs_remove+0x60/0x60 [ 14.093334] debugfs_remove+0x40/0x60 [ 14.093339] blk_trace_free+0x20/0x70 [ 14.093346] __blk_trace_remove+0x54/0x90 [ 14.096704] === do_blk_trace_setup(6) start [ 14.098534] blk_trace_shutdown+0x74/0x80 [ 14.100958] === do_blk_trace_setup(6) creating directory [ 14.104575] __blk_release_queue+0xbe/0x160 [ 14.104580] process_one_work+0x1b4/0x380 [ 14.104585] worker_thread+0x50/0x3c0 [ 14.104589] kthread+0xf9/0x130 [ 14.104593] ? process_one_work+0x380/0x380 [ 14.104596] ? kthread_park+0x90/0x90 [ 14.104599] ret_from_fork+0x1f/0x40 [ 14.104603] Modules linked in: loop(E) xfs(E) libcrc32c(E)
crct10dif_pclmul(E)crc32_pclmul(E)ghash_clmulni_intel(E)joydev(E)serio_raw(E)aesni_intel(E)glue_helper(E)virtio_balloon(E)evdev(E)crypto_simd(E)pcspkr(E) cryptd(E) i6300esb(E) button(E)ip_tables(E)x_tables(E)autofs4(E) ext4(E)crc32c_generic(E)crc16(E) mbcache(E) jbd2(E)virtio_net(E)net_failover(E)failover(E)virtio_blk(E)ata_generic(E)uhci_hcd(E)ata_piix(E)ehci_hcd(E)nvme(E) libata(E)crc32c_intel(E)usbcore(E) psmouse(E)nvme_core(E)virtio_pci(E)scsi_mod(E)virtio_ring(E)t10_pi(E)virtio(E)i2c_piix4(E)floppy(E)[ 14.107400] === do_blk_trace_setup(6) using what debugfs_lookup() gave [ 14.108939] CR2: 00000000000000a0 [ 14.110589] === do_blk_trace_setup(6) end with ret: 0 [ 14.111592] ---[ end trace 7a783b33b9614db9 ]---The root cause to this issue is that
debugfs_lookup()can find a previous incarnation's dir of the same name which is about to get removed from a not yet schedule work.
Quoted 8 lines
We can fix the UAF by simply using a debugfs directory which moving forward will always be accessible if debugfs is enabled, this way, its allocated and avaialble always for both request-based block drivers or
make_requestdrivers (multiqueue) block drivers.This simplifies the code considerably, with the only penalty now being that we're always creating the request queue debugfs directory for the request-based block device drivers.
This patch is indeed a big simplification, just a minor comment.
Quoted 207 lines
The UAF then is not a core debugfs issue, but instead a misuse of debugfs, and this issue can only be triggered if you are root, and misuse blktrace.
This issue can be reproduced with break-blktrace [2] using:
break-blktrace -c 10 -d -s
This patch fixes this issue. Note that there is also another respective UAF but from the ioctl path [3], this should also fix that issue.
This patch then also disputes the severity of CVE-2019-19770 as this issue is only possible by being root and using blktrace.
It is not a core debugfs issue.
[0] https://bugzilla.kernel.org/show_bug.cgi?id=205713
[1] https://nvd.nist.gov/vuln/detail/CVE-2019-19770
[2] https://github.com/mcgrof/break-blktrace
[3] https://lore.kernel.org/lkml/[email protected]/Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Reported-by [email protected]Fixes 6ac93117ab00 ("blktrace: use existing disk debugfs directory")Signed-off-by Luis Chamberlain <[email protected]>--- block/blk-debugfs.c | 12 ++++++++++++ block/blk-mq-debugfs.c | 5 ----- block/blk-sysfs.c | 3 +++ block/blk.h | 10 ++++++++++ include/linux/blkdev.h | 5 ++++- include/linux/blktrace_api.h | 1 - kernel/trace/blktrace.c | 19 ++++++++----------- 7 files changed, 37 insertions(+), 18 deletions(-)block/blk-debugfs.c +12
-0@@ -13,3 +13,15 @@ void blk_debugfs_register(void) { blk_debugfs_root =debugfs_create_dir ("block" ,NULL ); } + +void blk_queue_debugfs_register (struct request_queue *q) +{ + q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root); +} + +void blk_queue_debugfs_unregister (struct request_queue *q) +{ +debugfs_remove_recursive (q->debugfs_dir); + q->debugfs_dir =NULL ; +}block/blk-mq-debugfs.c +0
-5@@ -823,9 +823,6 @@ void blk_mq_debugfs_register(struct request_queue *q)struct blk_mq_hw_ctx *hctx;int i; - q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root); -debugfs_create_files (q->debugfs_dir, q, blk_mq_debugfs_queue_attrs);/* @@ -856,9 +853,7 @@ void blk_mq_debugfs_register(struct request_queue *q)void blk_mq_debugfs_unregister (struct request_queue *q) { -debugfs_remove_recursive (q->debugfs_dir); q->sched_debugfs_dir =NULL ; - q->debugfs_dir =NULL ; } static void blk_mq_debugfs_register_ctx(struct blk_mq_hw_ctx *hctx,block/blk-sysfs.c +3
-0@@ -895,6 +895,7 @@ static void __blk_release_queue(struct work_struct *work)blk_trace_shutdown (q); +blk_queue_debugfs_unregister (q);if (queue_is_mq (q))blk_mq_debugfs_unregister (q);@@ -975,6 +976,8 @@ int blk_register_queue(struct gendisk *disk) goto unlock; } +blk_queue_debugfs_register (q); +if (queue_is_mq (q)) {__blk_mq_register_dev (dev, q);blk_mq_debugfs_register (q);block/blk.h +10
-0@@ -489,10 +489,20 @@ int __bio_add_pc_page(struct request_queue *q, struct bio *bio, bool *same_page);#ifdef CONFIG_DEBUG_FS void blk_debugfs_register (void ); +void blk_queue_debugfs_register (struct request_queue *q); +void blk_queue_debugfs_unregister (struct request_queue *q);#else static inline void blk_debugfs_register (void ) { } + +static inline void blk_queue_debugfs_register (struct request_queue *q) +{ +} + +static inline void blk_queue_debugfs_unregister (struct request_queue *q) +{ +}#endif /* CONFIG_DEBUG_FS */ #endif /* BLK_INTERNAL_H */ include/linux/blkdev.h +4
-1@@ -569,8 +569,11 @@ struct request_queue {struct list_head tag_set_list;struct bio_set bio_split; -#ifdef CONFIG_BLK_DEBUG_FS +#ifdef CONFIG_DEBUG_FS +/* Used by block/blk-*debugfs.c and kernel/trace/blktrace.c */ struct dentry *debugfs_dir; +#endif +#ifdef CONFIG_BLK_DEBUG_FS struct dentry *sched_debugfs_dir;struct dentry *rqos_debugfs_dir;#endif include/linux/blktrace_api.h +0
-1@@ -22,7 +22,6 @@ struct blk_trace {u64 end_lba;u32 pid;u32 dev; -struct dentry *dir;struct dentry *dropped_file;struct dentry *msg_file;struct list_head running_list;kernel/trace/blktrace.c +4
-7@@ -311,7 +311,6 @@ static void blk_trace_free(struct blk_trace *bt)debugfs_remove (bt->msg_file);debugfs_remove (bt->dropped_file);relay_close (bt->rchan); -debugfs_remove (bt->dir);free_percpu (bt->sequence);free_percpu (bt->msg_data);kfree (bt);@@ -476,7 +475,6 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, struct blk_user_trace_setup *buts) {struct blk_trace *bt =NULL ; -struct dentry *dir =NULL ;int ret;if (!buts->buf_size || !buts->buf_nr)@@ -485,6 +483,9 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev,if (!blk_debugfs_root)return -ENOENT ; +if (!q->debugfs_dir) +return -ENOENT ; +strncpy (buts->name, name,BLKTRACE_BDEV_SIZE ); buts->name[BLKTRACE_BDEV_SIZE -1 ] ='\0' ;@@ -509,21 +510,19 @@ static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, ret = -ENOENT ; - dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); - bt->dev = dev;atomic_set (&bt->dropped,0 );INIT_LIST_HEAD (&bt->running_list); ret = -EIO ; - bt->dropped_file = debugfs_create_file("dropped", 0444, dir, bt, + bt->dropped_file = debugfs_create_file("dropped", 0444,
kernel/trace/blktrace.c static int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, · line 526
Quoted 10 lines
kernel/trace/blktrace.c +4
-2+ q->debugfs_dir, bt, &blk_dropped_fops); - bt->msg_file =debugfs_create_file ("msg" ,0222 , dir, bt, &blk_msg_fops); + bt->msg_file =debugfs_create_file ("msg" ,0222 , q->debugfs_dir, + bt, &blk_msg_fops); - bt->rchan =relay_open ("trace" , dir, buts->buf_size, + bt->rchan =relay_open ("trace" , q->debugfs_dir, buts->buf_size, buts->buf_nr, &blk_relay_callbacks, bt);
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
Thanks,
Ming
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
- q->debugfs_dir = debugfs_create_dir(kobject_name(q->kobj.parent),
- blk_debugfs_root);
To this:
+ q->debugfs_dir = debugfs_create_dir(kobject_name(q->kobj.parent),
+ blk_debugfs_root);
Maybe I am overlooking something.
Luis
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 19 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
block/blk-mq-debugfs.c +0
-2- q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root);To this:
block/blk-debugfs.c +2
-0+ q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root);
Quoted 1 line
Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()
- dir = debugfs_lookup(buts->name, blk_debugfs_root);
- if (!dir)
- bt->dir = dir = debugfs_create_dir(buts->name, blk_debugfs_root);
-
Then create blktrace attributes under the dir of q->debugfs_dir.
However, buts->name could be one partition device name, but q->debugfs_dir has to be disk name.
This change is visible to blktrace utilities.
Thanks,
Ming
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 31 lines · Ming Lei
Quoted 19 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
block/blk-mq-debugfs.c +0
-2- q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root);To this:
block/blk-debugfs.c +2
-0+ q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root);Quoted 1 line
Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()kernel/trace/blktrace.c +0
-4- dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); -
Quoted 6 lines
Then create blktrace attributes under the dir of
q->debugfs_dir.However,
buts->namecould be one partition device name, butq->debugfs_dirhas to be disk name.This change is visible to blktrace utilities.
Just test the 1st two patches via "blktrace /dev/sda2", follows the result, so this way can't be accepted.
[root@ktest-01 ~]# blktrace /dev/sda2
Thread 0 failed open /sys/kernel/debug/block/sda2/trace0: 2/No such file or directory Thread 4 failed open /sys/kernel/debug/block/sda2/trace4: 2/No such file or directory Thread 1 failed open /sys/kernel/debug/block/sda2/trace1: 2/No such file or directory Thread 2 failed open /sys/kernel/debug/block/sda2/trace2: 2/No such file or directory Thread 5 failed open /sys/kernel/debug/block/sda2/trace5: 2/No such file or directory Thread 3 failed open /sys/kernel/debug/block/sda2/trace3: 2/No such file or directory Thread 6 failed open /sys/kernel/debug/block/sda2/trace6: 2/No such file or directory Thread 7 failed open /sys/kernel/debug/block/sda2/trace7: 2/No such file or directory FAILED to start thread on CPU 0: 1/Operation not permitted
FAILED to start thread on CPU 1: 1/Operation not permitted
FAILED to start thread on CPU 2: 1/Operation not permitted
FAILED to start thread on CPU 3: 1/Operation not permitted
FAILED to start thread on CPU 4: 1/Operation not permitted
FAILED to start thread on CPU 5: 1/Operation not permitted
FAILED to start thread on CPU 6: 1/Operation not permitted
FAILED to start thread on CPU 7: 1/Operation not permitted
Thanks,
Ming
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 51 lines · Ming Lei
Quoted 31 lines · Ming Lei
Quoted 19 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
block/blk-mq-debugfs.c +0
-2- q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root);To this:
block/blk-debugfs.c +2
-0+ q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root);Quoted 1 line
Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()kernel/trace/blktrace.c +0
-4- dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); -Quoted 6 lines
Then create blktrace attributes under the dir of
q->debugfs_dir.However,
buts->namecould be one partition device name, butq->debugfs_dirhas to be disk name.This change is visible to blktrace utilities.
Just test the 1st two patches via "blktrace
/dev/sda2", follows the result, so this way can't be accepted.[root@ktest-01 ~]# blktrace
/dev/sda2
Thread 0 failed open/sys/kernel/debug/block/sda2/trace0: 2/No such file or directory Thread 4 failed open/sys/kernel/debug/block/sda2/trace4: 2/No such file or directory Thread 1 failed open/sys/kernel/debug/block/sda2/trace1: 2/No such file or directory Thread 2 failed open/sys/kernel/debug/block/sda2/trace2: 2/No such file or directory Thread 5 failed open/sys/kernel/debug/block/sda2/trace5: 2/No such file or directory Thread 3 failed open/sys/kernel/debug/block/sda2/trace3: 2/No such file or directory Thread 6 failed open/sys/kernel/debug/block/sda2/trace6: 2/No such file or directory Thread 7 failed open/sys/kernel/debug/block/sda2/trace7: 2/No such file or directory
Quoted 8 lines
FAILED to start thread on CPU 0: 1/Operation not permitted FAILED to start thread on CPU 1: 1/Operation not permitted FAILED to start thread on CPU 2: 1/Operation not permitted FAILED to start thread on CPU 3: 1/Operation not permitted FAILED to start thread on CPU 4: 1/Operation not permitted FAILED to start thread on CPU 5: 1/Operation not permitted FAILED to start thread on CPU 6: 1/Operation not permitted FAILED to start thread on CPU 7: 1/Operation not permitted
Thanks, as I noted, I think we'd need to pre-create the directories per parition. Let me know if you think of a better alternative.
Luis
Re: [PATCH 2/5] blktrace: fix debugfs use after free
In reply to Ming LeiQuoted 31 lines · Ming Lei
Quoted 19 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
block/blk-mq-debugfs.c +0
-2- q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root);To this:
block/blk-debugfs.c +2
-0+ q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root);Quoted 1 line
Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()kernel/trace/blktrace.c +0
-4- dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); -
Quoted 3 lines
Then create blktrace attributes under the dir of
q->debugfs_dir.However,
buts->namecould be one partition device name, but
I can see how buts->name is set to bdevname() which expands to disk_name(bdev->bd_disk, bdev->bd_part->partno, buf).
Quoted 1 line
q->debugfs_dirhas to be disk name.
I can't see this, can you point me to where it is clear the request_queue kobject's parent is sure to be the disk name?
If it is different, the issue I don't think should be debugfs, but the bigger issue would be that blktrace on two different partitions would clash.
Also, the *old* lookup intent on partitions always would fail on mq and we'd end up creating a directory.
I think we'd need to create a directory per partition, even when we don't use blktrace. That makes this more complex than I'd hope for.
Luis
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 36 lines · Luis Chamberlain
Quoted 31 lines · Ming Lei
Quoted 19 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
block/blk-mq-debugfs.c +0
-2- q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), - blk_debugfs_root);To this:
block/blk-debugfs.c +2
-0+ q->debugfs_dir =debugfs_create_dir (kobject_name (q->kobj.parent), + blk_debugfs_root);Quoted 1 line
Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()kernel/trace/blktrace.c +0
-4- dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); -Quoted 3 lines
Then create blktrace attributes under the dir of
q->debugfs_dir.However,
buts->namecould be one partition device name, but
Quoted 7 lines
I can see how
buts->nameis set tobdevname()which expands todisk_name(bdev->bd_disk, bdev->bd_part->partno, buf).Quoted 1 line
q->debugfs_dirhas to be disk name.I can't see this, can you point me to where it is clear the
request_queuekobject's parent is sure to be the disk name?
blk_register_queue():
... ret = kobject_add(&q->kobj, kobject_get(&dev->kobj), "%s", "queue"); ...
Quoted 10 lines
If it is different, the issue I don't think should be debugfs, but the bigger issue would be that blktrace on two different partitions would clash.
Also, the *old* lookup intent on partitions always would fail on mq and we'd end up creating a directory.
I think we'd need to create a directory per partition, even when we don't use blktrace. That makes this more complex than I'd hope for.
Anyway, the current ABI can't be broken, also I'd suggest to understand how the userpace utility uses blktrace syscall interfaces first before figuring any new approach.
Thanks,
Ming
Re: [PATCH 2/5] blktrace: fix debugfs use after free
Quoted 43 lines · Ming Lei
Quoted 41 lines · Luis Chamberlain
Quoted 31 lines · Ming Lei
Quoted 13 lines · Luis Chamberlain
Quoted 6 lines · Ming Lei
In theory, multiple partitions can be traced concurrently, but looks it never works, so it won't cause trouble for multiple partition trace.
One userspace visible change is that blktrace debugfs dir name is switched to disk name from partition name in case of partition trace, will it break some utilities?
How is this possible, its not clear to me, we go from:
-
q->debugfs_dir= debugfs_create_dir(kobject_name(q->kobj.parent),
-blk_debugfs_root);Quoted 7 lines
To this:
+
q->debugfs_dir= debugfs_create_dir(kobject_name(q->kobj.parent),
+blk_debugfs_root);Maybe I am overlooking something.
Your patch removes the blktrace debugfs dir:
do_blk_trace_setup()kernel/trace/blktrace.c +0
-4- dir =debugfs_lookup (buts->name, blk_debugfs_root); -if (!dir) - bt->dir = dir =debugfs_create_dir (buts->name, blk_debugfs_root); -Quoted 3 lines
Then create blktrace attributes under the dir of
q->debugfs_dir.However,
buts->namecould be one partition device name, butI can see how
buts->nameis set tobdevname()which expands todisk_name(bdev->bd_disk, bdev->bd_part->partno, buf).Quoted 1 line
q->debugfs_dirhas to be disk name.
Quoted 7 lines
Quoted 2 lines
I can't see this, can you point me to where it is clear the
request_queuekobject's parent is sure to be the disk name?
blk_register_queue():... ret = kobject_add(&q->kobj, kobject_get(&dev->kobj), "%s", "queue"); ...
Alright, I have a fix for this now, and I do have also a further explanation as to *why* the debugfs_lookup() doesn't help us here. I'll follow up with more patches.
Luis
PATCH 3/5 blktrace: refcount the request_queue during ioctl
In reply to Luis ChamberlainEnsure that the request_queue is refcounted during its full ioctl cycle. This avoids possible races against removal, given blk_get_queue() also checks to ensure the queue is not dying.
This small race is possible if you defer removal of the request_queue and userspace fires off an ioctl for the device in the meantime.
--- kernel/trace/blktrace.c | 6 ++++++ 1 file changed, 6 insertions(+)
kernel/trace/blktrace.c +6 -0
@@ -701,6 +701,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg)if (!q)return -ENXIO ; +if (!blk_get_queue (q)) +return -ENXIO ; +mutex_lock (&q->blk_trace_mutex);switch (cmd) {
@@ -729,6 +732,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg) }mutex_unlock (&q->blk_trace_mutex); + +blk_put_queue (q); +return ret; }
Signature · 1 line
2.25.1
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 6 lines · Luis Chamberlain
Ensure that the
request_queueis refcounted during its full ioctl cycle. This avoids possible races against removal, givenblk_get_queue()also checks to ensure the queue is not dying.This small race is possible if you defer removal of the
request_queueand userspace fires off an ioctl for the device in the meantime.
Hmm, where exactly does the race come in so that it can only happen after where you take the reference, but not before it? I'm probably missing something, but that just means it needs to be explained a little better :)
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Luis Chamberlain
Ensure that the
request_queueis refcounted during its full ioctl cycle. This avoids possible races against removal, givenblk_get_queue()also checks to ensure the queue is not dying.
Quoted 7 lines
Quoted 2 lines
This small race is possible if you defer removal of the
request_queueand userspace fires off an ioctl for the device in the meantime.Hmm, where exactly does the race come in so that it can only happen after where you take the reference, but not before it? I'm probably missing something, but that just means it needs to be explained a little better :)
From the trace on patch 2/5:
BLKTRACE_SETUP(loop0) #2
[ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start
[ 13.936758] === do_blk_trace_setup(2) start
[ 13.938944] === do_blk_trace_setup(2) creating directory
[ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave
---> From LOOP_CTL_DEL(loop0) #2
[ 13.971046] === blk_trace_cleanup(7) end
[ 13.973175] == __blk_trace_remove(7) end
[ 13.975352] == blk_trace_shutdown(7) end
[ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister()
[ 13.980645] ==== blk_mq_debugfs_unregister(7) begin
[ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir)
[ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL
[ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end
[ 13.993155] = __blk_release_queue(7) end
---> From BLKTRACE_SETUP(loop0) #2
[ 13.995928] === do_blk_trace_setup(2) end with ret: 0
[ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) endThe BLKTRACESETUP above works on request_queue which later LOOP_CTL_DEL races on and sweeps the debugfs dir underneath us. If you use this commit alone though, this doesn't fix the race issue however, and that's because of both still the debugfs_lookup() use and that we're still using asynchronous removal at this point.
refcounting will just ensure we don't take the request_queue underneath our noses.
Should I just add this to the commit log?
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 10 lines · Luis Chamberlain
The BLKTRACESETUP above works on
request_queuewhich laterLOOP_CTL_DELraces on and sweeps the debugfs dir underneath us. If you use this commit alone though, this doesn't fix the race issue however, and that's because of both still thedebugfs_lookup()use and that we're still using asynchronous removal at this point.refcounting will just ensure we don't take the
request_queueunderneath our noses.Should I just add this to the commit log?
That sounds much more useful than the trace.
Btw, Isn't blk_get_queue racy as well? Shouldn't we check blk_queue_dying after getting the reference and undo it if the queue is indeeed dying?
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Luis Chamberlain
The BLKTRACESETUP above works on
request_queuewhich laterLOOP_CTL_DELraces on and sweeps the debugfs dir underneath us. If you use this commit alone though, this doesn't fix the race issue however, and that's because of both still thedebugfs_lookup()use and that we're still using asynchronous removal at this point.refcounting will just ensure we don't take the
request_queueunderneath our noses.
Quoted 7 lines
Quoted 1 line
Should I just add this to the commit log?
That sounds much more useful than the trace.
Btw, Isn't
blk_get_queueracy as well? Shouldn't we checkblk_queue_dyingafter getting the reference and undo it if the queue is indeeed dying?
Yes that race should be possible:
bool blk_get_queue(struct request_queue *q)
{
if (likely(!blk_queue_dying(q))) {
----------> we can get the queue to go dying here <---------
__blk_get_queue(q);
return true;
}
return false;
} EXPORT_SYMBOL(blk_get_queue);
I'll pile up a fix. I've also considered doing a full review of callers outside of the core block layer using it, and maybe just unexporting this. It was originally exported due to commit d86e0e83b ("block: export blk_{get,put}_queue()") to fix a scsi bug, but I can't find such respective fix. I suspec that using bdgrab()/bdput() seems more likely what drivers should be using. That would allow us to keep this functionality internal.
Think that's worthy review?
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 9 lines · Luis Chamberlain
I'll pile up a fix. I've also considered doing a full review of callers outside of the core block layer using it, and maybe just unexporting this. It was originally exported due to commit
d86e0e83b("block: export blk_{get,put}_queue()") to fix a scsi bug, but I can't find such respective fix. I suspec that using bdgrab()/bdput() seems more likely what drivers should be using. That would allow us to keep this functionality internal.Think that's worthy review?
Probably. I did in fact very quickly look into that but then gave up due to the fair amount of modular users.
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Luis Chamberlain
I'll pile up a fix. I've also considered doing a full review of callers outside of the core block layer using it, and maybe just unexporting this. It was originally exported due to commit
d86e0e83b("block: export blk_{get,put}_queue()") to fix a scsi bug, but I can't find such respective fix. I suspec that using bdgrab()/bdput() seems more likely what drivers should be using. That would allow us to keep this functionality internal.
Quoted 4 lines
Quoted 1 line
Think that's worthy review?
Probably. I did in fact very quickly look into that but then gave up due to the fair amount of modular users.
Alright, then might as well then verify if the existing practice of bdgrab()/bdput() is indeed valid logic, as otherwise we'd be puting the atomic context / sleep concern to bdput(). As noted earlier I am able to confirm easily that bdgrab() can be called in atomic contex, however I cannot easily yet vet for *why* this was a safe assumption for bdput().
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
In reply to Luis ChamberlainQuoted 19 lines · Luis Chamberlain
Quoted 3 lines · Christoph Hellwig
Btw, Isn't
blk_get_queueracy as well? Shouldn't we checkblk_queue_dyingafter getting the reference and undo it if the queue is indeeed dying?Yes that race should be possible:
bool
blk_get_queue(struct request_queue *q){ if (likely(!blk_queue_dying(q))) { ----------> we can get the queue to go dying here <--------- __blk_get_queue(q); return true; } return false; }
EXPORT_SYMBOL(blk_get_queue);
Quoted 7 lines
I'll pile up a fix. I've also considered doing a full review of callers outside of the core block layer using it, and maybe just unexporting this. It was originally exported due to commit
d86e0e83b("block: export blk_{get,put}_queue()") to fix a scsi bug, but I can't find such respective fix. I suspec that using bdgrab()/bdput() seems more likely what drivers should be using. That would allow us to keep this functionality internal.
blk_get_queue() prevents concurrent freeing of struct request_queue but does not prevent concurrent blk_cleanup_queue() calls. Callers of blk_get_queue() may encounter a change of the queue state from normal into dying any time during the blk_get_queue() call or after blk_get_queue() has finished. Maybe I'm overlooking something but I doubt that modifying blk_get_queue() will help.
Bart.
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 28 lines · Bart Van Assche
Quoted 19 lines · Luis Chamberlain
Quoted 3 lines · Christoph Hellwig
Btw, Isn't
blk_get_queueracy as well? Shouldn't we checkblk_queue_dyingafter getting the reference and undo it if the queue is indeeed dying?Yes that race should be possible:
bool
blk_get_queue(struct request_queue *q){ if (likely(!blk_queue_dying(q))) { ----------> we can get the queue to go dying here <--------- __blk_get_queue(q); return true; } return false; }
EXPORT_SYMBOL(blk_get_queue);Quoted 7 lines
I'll pile up a fix. I've also considered doing a full review of callers outside of the core block layer using it, and maybe just unexporting this. It was originally exported due to commit
d86e0e83b("block: export blk_{get,put}_queue()") to fix a scsi bug, but I can't find such respective fix. I suspec that using bdgrab()/bdput() seems more likely what drivers should be using. That would allow us to keep this functionality internal.
Quoted 2 lines
blk_get_queue()prevents concurrent freeing of structrequest_queuebut does not prevent concurrentblk_cleanup_queue()calls.
Wouldn't concurrent blk_cleanup_queue() calls be a bug? If so should I make it clear that it would be or simply prevent it?
Quoted 5 lines
Callers of
blk_get_queue()may encounter a change of the queue state from normal into dying any time during theblk_get_queue()call or afterblk_get_queue()has finished. Maybe I'm overlooking something but I doubt that modifyingblk_get_queue()will help.
Good point, to fix that race described by Christoph we'd have to take into consideration refcounts of the request_queue to prevent queues from changing state to dying if the refcount is > 1, however that'd also would mean not allowing the request_queue from ever dying.
One way we could resolve this could be to to keep track of a quiesce/dying request, then at that point prevent blk_get_queue() from allowing increments, and once the refcount is down to 1, flip the switch to dying.
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines · Bart Van Assche
blk_get_queue()prevents concurrent freeing of structrequest_queuebut does not prevent concurrentblk_cleanup_queue()calls.Wouldn't concurrent
blk_cleanup_queue()calls be a bug? If so should I make it clear that it would be or simply prevent it?
I think calling blk_cleanup_queue() while the queue refcount > 0 is well established behavior. At least the SCSI core triggers that behavior since a very long time. I prefer not to change that behavior.
Regarding patch 3/5: how about dropping that patch? If the queue refcount can drop to zero while blk_trace_ioctl() is in progress I think that should be fixed in the block_device_operations.open callback instead of in blk_trace_ioctl().
Thanks,
Bart.
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 11 lines · Bart Van Assche
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines · Bart Van Assche
blk_get_queue()prevents concurrent freeing of structrequest_queuebut does not prevent concurrentblk_cleanup_queue()calls.Wouldn't concurrent
blk_cleanup_queue()calls be a bug? If so should I make it clear that it would be or simply prevent it?I think calling
blk_cleanup_queue()while the queue refcount > 0 is well established behavior. At least the SCSI core triggers that behavior since a very long time. I prefer not to change that behavior.
I see. An alternative is to simply check if we already are cleaning up and if so abort early on the blk_cleanup_queue(). That would allow re-entrant calls, and just be a no-op to the additional calls. Or is the re-entrant, two attemps to really do all the work blk_cleanup_queue() expected functionality already?
Quoted 4 lines
Regarding patch 3/5: how about dropping that patch? If the queue refcount can drop to zero while
blk_trace_ioctl()is in progress I think that should be fixed in the block_device_operations.open callback instead of inblk_trace_ioctl().
I'll take a look, thanks!
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
In reply to Luis ChamberlainQuoted 29 lines · Luis Chamberlain
Quoted 6 lines · Christoph Hellwig
Hmm, where exactly does the race come in so that it can only happen after where you take the reference, but not before it? I'm probably missing something, but that just means it needs to be explained a little better :)
From the trace on patch 2/5:
BLKTRACE_SETUP(loop0) #2 [ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start [ 13.936758] === do_blk_trace_setup(2) start [ 13.938944] === do_blk_trace_setup(2) creating directory [ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave ---> From LOOP_CTL_DEL(loop0) #2 [ 13.971046] === blk_trace_cleanup(7) end [ 13.973175] == __blk_trace_remove(7) end [ 13.975352] == blk_trace_shutdown(7) end [ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister() [ 13.980645] ==== blk_mq_debugfs_unregister(7) begin [ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir) [ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL [ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end [ 13.993155] = __blk_release_queue(7) end ---> From BLKTRACE_SETUP(loop0) #2 [ 13.995928] === do_blk_trace_setup(2) end with ret: 0 [ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) end
Quoted 8 lines
The BLKTRACESETUP above works on
request_queuewhich laterLOOP_CTL_DELraces on and sweeps the debugfs dir underneath us. If you use this commit alone though, this doesn't fix the race issue however, and that's because of both still thedebugfs_lookup()use and that we're still using asynchronous removal at this point.refcounting will just ensure we don't take the
request_queueunderneath our noses.
I think the above trace reveals a bug in the loop driver. The loop driver shouldn't allow the associated request queue to disappear while the loop device is open. One may want to have a look at sd_open() in the sd driver. The scsi_disk_get() call in that function not only increases the reference count of the SCSI disk but also of the underlying SCSI device.
Thanks,
Bart.
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 36 lines · Bart Van Assche
Quoted 35 lines · Luis Chamberlain
Quoted 6 lines · Christoph Hellwig
Hmm, where exactly does the race come in so that it can only happen after where you take the reference, but not before it? I'm probably missing something, but that just means it needs to be explained a little better :)
From the trace on patch 2/5:
BLKTRACE_SETUP(loop0) #2 [ 13.933961] == blk_trace_ioctl(2, BLKTRACESETUP) start [ 13.936758] === do_blk_trace_setup(2) start [ 13.938944] === do_blk_trace_setup(2) creating directory [ 13.941029] === do_blk_trace_setup(2) using what debugfs_lookup() gave ---> From LOOP_CTL_DEL(loop0) #2 [ 13.971046] === blk_trace_cleanup(7) end [ 13.973175] == __blk_trace_remove(7) end [ 13.975352] == blk_trace_shutdown(7) end [ 13.977415] = __blk_release_queue(7) calling blk_mq_debugfs_unregister() [ 13.980645] ==== blk_mq_debugfs_unregister(7) begin [ 13.980696] ==== blk_mq_debugfs_unregister(7) debugfs_remove_recursive(q->debugfs_dir) [ 13.983118] ==== blk_mq_debugfs_unregister(7) end q->debugfs_dir is NULL [ 13.986945] = __blk_release_queue(7) blk_mq_debugfs_unregister() end [ 13.993155] = __blk_release_queue(7) end ---> From BLKTRACE_SETUP(loop0) #2 [ 13.995928] === do_blk_trace_setup(2) end with ret: 0 [ 13.997623] == blk_trace_ioctl(2, BLKTRACESETUP) endThe BLKTRACESETUP above works on
request_queuewhich laterLOOP_CTL_DELraces on and sweeps the debugfs dir underneath us. If you use this commit alone though, this doesn't fix the race issue however, and that's because of both still thedebugfs_lookup()use and that we're still using asynchronous removal at this point.
Quoted 6 lines
Quoted 2 lines
refcounting will just ensure we don't take the
request_queueunderneath our noses.I think the above trace reveals a bug in the loop driver. The loop driver shouldn't allow the associated request queue to disappear while the loop device is open.
The bug was *not* in the driver, the bug was in that deferal of removal was allowed to be asynchronous, therefore the removal from a userspace perspective *finishes*, but its not actually really done. Back when the removal was synchronous, the loop driver waited on cleanup, and didn't return to userspace until it was really removed.
This is why I annotated that the move to asynch removal turns out to actually be a userspace API regression.
Quoted 3 lines
One may want to have a look at
sd_open()in the
sd driver. Thescsi_disk_get()call in that function not only increases the reference count of the SCSI disk but also of the underlying SCSI device.
Are you saying to use this as a template for what a driver should do or do you suspect there is a bug there? Not sure what you mean here.
Luis
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
In reply to Luis ChamberlainQuoted 35 lines · Luis Chamberlain
Ensure that the
request_queueis refcounted during its full ioctl cycle. This avoids possible races against removal, givenblk_get_queue()also checks to ensure the queue is not dying.This small race is possible if you defer removal of the
request_queueand userspace fires off an ioctl for the device in the meantime.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Reviewed-by Bart Van Assche <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- kernel/trace/blktrace.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/trace/blktrace.c +3
-0@@ -701,6 +701,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg)if (!q)return -ENXIO ; +if (!blk_get_queue (q)) +return -ENXIO ; +mutex_lock (&q->blk_trace_mutex);switch (cmd) {
kernel/trace/blktrace.c int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg) · line 739
Quoted 8 lines
kernel/trace/blktrace.c +3
-0@@ -729,6 +732,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg) }mutex_unlock (&q->blk_trace_mutex); + +blk_put_queue (q); +return ret; }
Actually when bdev is opened, one extra refcount is held on gendisk, so gendisk won't go away. And __device_add_disk() does grab one extra refcount on request queue, so request queue shouldn't go away when ioctl is running.
Can you describe a bit more what the issue is to be addressed by this patch?
Thanks,
Ming
Re: [PATCH 3/5] blktrace: refcount the request_queue during ioctl
Quoted 38 lines · Ming Lei
Quoted 37 lines · Luis Chamberlain
Ensure that the
request_queueis refcounted during its full ioctl cycle. This avoids possible races against removal, givenblk_get_queue()also checks to ensure the queue is not dying.This small race is possible if you defer removal of the
request_queueand userspace fires off an ioctl for the device in the meantime.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Reviewed-by Bart Van Assche <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- kernel/trace/blktrace.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/trace/blktrace.c +3
-0@@ -701,6 +701,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg)if (!q)return -ENXIO ; +if (!blk_get_queue (q)) +return -ENXIO ; +mutex_lock (&q->blk_trace_mutex);switch (cmd) {@@ -729,6 +732,9 @@ int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg) }
Quoted 11 lines
kernel/trace/blktrace.c int blk_trace_ioctl(struct block_device *bdev, unsigned cmd, char __user *arg) · line 739
Quoted 6 lines
kernel/trace/blktrace.c +3
-0mutex_unlock (&q->blk_trace_mutex); + +blk_put_queue (q); +return ret; }Actually when bdev is opened, one extra refcount is held on gendisk, so gendisk won't go away. And
__device_add_disk()does grab one extra refcount on request queue, so request queue shouldn't go away when ioctl is running.
Alright, then yes, this should not be needed.
Luis
PATCH 4/5 mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
In reply to Luis Chamberlainblock devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's request_queue structure is also refcounted, however, if the last blk_put_queue() is called under atomic context the block layer has to defer removal.
By refcounting the block device during the use of blkcg_schedule_throttle(), we ensure ensure two things:
1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using the request_queue structure in atomic context, since the last blk_put_queue() will be called upon disk_release(), *after* our own bdput().
This means this code path is *not* going to remove the request_queue structure, as we are ensuring some later upper layer disk_release() will be the one to release the request_queue structure for us.
--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
mm/swapfile.c +12 -2
@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;
@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev); +if (!bdev) +continue ; +/* +* By adding our own bdgrab() we ensure the queue +* sticks around until disk_release(), and so we ensure +* our release of the request_queue does not happen in +* atomic context. +*/ +blkcg_schedule_throttle (bdev_get_queue (bdev), true); +bdput (bdev);break ; } }
Signature · 1 line
2.25.1
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 50 lines · Luis Chamberlain
block devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's
request_queuestructure is also refcounted, however, if the lastblk_put_queue()is called under atomic context the block layer has to defer removal.By refcounting the block device during the use of
blkcg_schedule_throttle(), we ensure ensure two things:1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using therequest_queuestructure in atomic context, since the lastblk_put_queue()will be called upondisk_release(), *after* our ownbdput().This means this code path is *not* going to remove the
request_queuestructure, as we are ensuring some later upper layerdisk_release()will be the one to release therequest_queuestructure for us.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/swapfile.c +2
-2@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev);
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3785
Quoted 10 lines
mm/swapfile.c +10
-0+if (!bdev) +continue ; +/* +* By adding our own bdgrab() we ensure the queue +* sticks around until disk_release(), and so we ensure +* our release of the request_queue does not happen in +* atomic context. +*/ +blkcg_schedule_throttle (bdev_get_queue (bdev), true); +bdput (bdev);
I don't understand the atomic part of the comment. How does bdgrab/bdput help us there?
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 56 lines · Christoph Hellwig
Quoted 55 lines · Luis Chamberlain
block devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's
request_queuestructure is also refcounted, however, if the lastblk_put_queue()is called under atomic context the block layer has to defer removal.By refcounting the block device during the use of
blkcg_schedule_throttle(), we ensure ensure two things:1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using therequest_queuestructure in atomic context, since the lastblk_put_queue()will be called upondisk_release(), *after* our ownbdput().This means this code path is *not* going to remove the
request_queuestructure, as we are ensuring some later upper layerdisk_release()will be the one to release therequest_queuestructure for us.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/swapfile.c +7
-2@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev); +if (!bdev) +continue ; +/* +* By adding our own bdgrab() we ensure the queue +* sticks around until disk_release(), and so we ensure
Quoted 8 lines
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3785
Quoted 5 lines
mm/swapfile.c +5
-0+* our release of the request_queue does not happen in +* atomic context. +*/ +blkcg_schedule_throttle (bdev_get_queue (bdev), true); +bdput (bdev);I don't understand the atomic part of the comment. How does bdgrab/bdput help us there?
The commit log above did a better job at explaining this in terms of our goal to use the request_queue and how this use would prevent the risk of releasing the request_queue, which could sleep.
If its not clear still, at least why we'd escape the sleep potential of the request_queue, we can just see its up to the disk_release() to call the last blk_put_queue():
static void __device_add_disk(struct device *parent, struct gendisk disk,
const struct attribute_group **groups, bool register_queue)
{
...
/*
* Take an extra ref on queue which will be put on disk_release()
* so that it sticks around as long as @disk is there.
*/
WARN_ON_ONCE(!blk_get_queue(disk->queue));
disk_add_events(disk);
blk_integrity_add(disk);
}static void disk_release(struct device *dev)
{
struct gendisk *disk = dev_to_disk(dev);
blk_free_devt(dev->devt);
disk_release_events(disk);
kfree(disk->random);
disk_replace_part_tbl(disk, NULL);
hd_free_part(&disk->part0);
if (disk->queue)
blk_put_queue(disk->queue);
kfree(disk);
} I admit that all this however it did a poor job at explaining why bdgrab()/bdput() was safe in atomic context other than the implicit reasoning that we already do that elsewhere in atomic context.
bdgrab() specifically was added to be able to refcount a block device in atomic context via commit dddac6a7b445 ("("PM / Hibernate: Replace bdget call with simple atomic_inc of i_count"). In its latest incarnation we have:
/**
* bdgrab -- Grab a reference to an already referenced block device
* @bdev: Block device to grab a reference to.
*/
struct block_device *bdgrab(struct block_device *bdev)
{
ihold(bdev->bd_inode);
return bdev;
} EXPORT_SYMBOL(bdgrab);
And this in turn:
/*
* get additional reference to inode; caller must already hold one. */
void ihold(struct inode *inode)
{
WARN_ON(atomic_inc_return(&inode->i_count) < 2);
} EXPORT_SYMBOL(ihold);
However... I'd eventure to say we don't have tribal knowledge documented about why bdput() is safe in atomic context when used with bdgrab(), including the commit log which added it. So the only thing backing its safety is that we already use this combo in atomic context, and if its incorrect, other areas would be incorrect as well.
But looking underneath the hood, I see at the end of __blkdev_get():
static int __blkdev_get(struct block_device *bdev, fmode_t mode, int for_part)
{
...
disk = bdev_get_gendisk(bdev, &partno);
...
/* only one opener holds refs to the module and disk */
if (!first_open)
put_disk_and_module(disk);
...
}So ihold() seems like a way to ensure the caller of bdgrab() won't be this first opener. If only one opener holds the ref to the disk and it was not us, we musn't be the one to decrease it, and if the disk is held, it should mean the block device should be refcounted by it as well. More review on this later part is appreciated though.
Luis
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
I don't understand the atomic part of the comment. How does bdgrab/bdput help us there?
The commit log above did a better job at explaining this in terms of our goal to use the
request_queueand how this use would prevent the risk of releasing therequest_queue, which could sleep.
So bdput eventually does and iput, but what leads to an out of context offload?
But anyway, isn't the original problem better solved by simply not releasing the queue from atomic context to start with? There isn't really any good reason we keep holding the spinlock once we have a reference on the queue, so something like this (not even compile tested) should do the work:
block/blk-cgroup.c +13 -10
@@ -1673,6 +1673,17 @@ void blkcg_maybe_throttle_current(void)blk_put_queue (q); } +/* consumes a reference on q */ +void __blkcg_schedule_throttle (struct request_queue *q,bool use_memdelay) +{ +if (current->throttle_queue) +blk_put_queue (current->throttle_queue); + current->throttle_queue = q; +if (use_memdelay) + current->use_memdelay = use_memdelay; +set_notify_resume (current); +} +/** * blkcg_schedule_throttle - this task needs to check for throttling * @q: the request queue IO was submitted on
@@ -1694,16 +1705,8 @@ void blkcg_schedule_throttle(struct request_queue *q, bool use_memdelay) {if (unlikely (current->flags &PF_KTHREAD ))return ; - -if (!blk_get_queue (q)) -return ; - -if (current->throttle_queue) -blk_put_queue (current->throttle_queue); - current->throttle_queue = q; -if (use_memdelay) - current->use_memdelay = use_memdelay; -set_notify_resume (current); +if (blk_get_queue (q)) +__blkcg_schedule_throttle (q, use_memdelay); }/**
include/linux/blk-cgroup.h +1 -0
@@ -679,6 +679,7 @@ static inline void blkcg_clear_delay(struct blkcg_gq *blkg)void blkcg_add_delay (struct blkcg_gq *blkg,u64 now,u64 delta);void blkcg_schedule_throttle (struct request_queue *q,bool use_memdelay); +void __blkcg_schedule_throttle (struct request_queue *q,bool use_memdelay);void blkcg_maybe_throttle_current (void );#else /* CONFIG_BLK_CGROUP */
mm/swapfile.c +8 -3
@@ -3749,9 +3749,10 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, gfp_t gfp_mask) {struct swap_info_struct *si, *next; +struct request_queue *q =NULL ; +if (!(gfp_mask & __GFP_IO) || !memcg)return ; -if (!blk_cgroup_congested ())return ;
@@ -3761,17 +3762,21 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,*/ if (current->throttle_queue)return ; +if (unlikely (current->flags &PF_KTHREAD )) +return ;spin_lock (&swap_avail_lock);plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); +if (blk_get_queue (dev_get_queue (si->bdev))) + q =dev_get_queue (si->bdev);break ; } }spin_unlock (&swap_avail_lock); +if (q) + __blkcg_schedule_throttle(q, true); }#endif
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
I don't understand the atomic part of the comment. How does bdgrab/bdput help us there?
The commit log above did a better job at explaining this in terms of our goal to use the
request_queueand how this use would prevent the risk of releasing therequest_queue, which could sleep.
Quoted 8 lines
So bdput eventually does and iput, but what leads to an out of context offload?
But anyway, isn't the original problem better solved by simply not releasing the queue from atomic context to start with? There isn't really any good reason we keep holding the spinlock once we have a reference on the queue, so something like this (not even compile tested) should do the work:
Actually - mem_cgroup_throttle_swaprate already checks for a non-NULL current->throttle_queue above, so we should never even call blk_put_queue here. Was this found by code inspection, or was there a real report?
In the latter case we need to figure out what protects >throttle_queue, as the way blkcg_schedule_throttle first put the reference and only then assign a value to it already looks like it introduces a tiny race window.
Otherwise just open coding the applicable part ofblkcg_schedule_throttle in mem_cgroup_throttle_swaprate seems easiest:
mm/swapfile.c +9 -4
@@ -3761,15 +3761,20 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,*/ if (current->throttle_queue)return ; +if (unlikely (current->flags &PF_KTHREAD )) +return ;spin_lock (&swap_avail_lock);plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) { -if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); -break ; +if (!si->bdev) +continue ; +if (blk_get_queue (dev_get_queue (si->bdev))) { + current->throttle_queue =dev_get_queue (si->bdev); + current->use_memdelay = true; +set_notify_resume (current); } +break ; }spin_unlock (&swap_avail_lock); }
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 18 lines · Christoph Hellwig
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
I don't understand the atomic part of the comment. How does bdgrab/bdput help us there?
The commit log above did a better job at explaining this in terms of our goal to use the
request_queueand how this use would prevent the risk of releasing therequest_queue, which could sleep.Quoted 8 lines
So bdput eventually does and iput, but what leads to an out of context offload?
But anyway, isn't the original problem better solved by simply not releasing the queue from atomic context to start with? There isn't really any good reason we keep holding the spinlock once we have a reference on the queue, so something like this (not even compile tested) should do the work:
Quoted 4 lines
Actually -
mem_cgroup_throttle_swapratealready checks for a non-NULLcurrent->throttle_queueabove, so we should never even callblk_put_queuehere. Was this found by code inspection, or was there a real report?
No but report, this code path came up as a possible use of blk_put_queue() which already exists in atomic context. So yes, it already uses blk_put_queue(), but how are we *sure* its not going to be the last one? Because if it is that mean the release will happen in atomic context, defeating the goal of the last patch in this series.
Using bdgrab() however ensures that during the lifecycle of this path, blk_put_queue() won't be the last if used after, but instead we are sure it will be upon disk release.
In fact, with bdgrab() isn't the blk_get_queue() on blkcg_schedule_throttle() no longer needed?
Quoted 27 lines
In the latter case we need to figure out what protects >throttle_queue, as the way
blkcg_schedule_throttlefirst put the reference and only then assign a value to it already looks like it introduces a tiny race window.Otherwise just open coding the applicable part of
blkcg_schedule_throttleinmem_cgroup_throttle_swaprateseems easiest:mm/swapfile.c +3
-4@@ -3761,15 +3761,20 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,*/ if (current->throttle_queue)return ; +if (unlikely (current->flags &PF_KTHREAD )) +return ;spin_lock (&swap_avail_lock);plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) { -if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); -break ; +if (!si->bdev)
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3780
Quoted 10 lines
mm/swapfile.c +6
-0+continue ; +if (blk_get_queue (dev_get_queue (si->bdev))) { + current->throttle_queue =dev_get_queue (si->bdev); + current->use_memdelay = true; +set_notify_resume (current); } +break ; }spin_unlock (&swap_avail_lock); }
Sorry, its not clear to me who calls the respective blk_put_queue() here?
Luis
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 18 lines · Luis Chamberlain
Quoted 18 lines
mm/swapfile.c +8
-4if (current->throttle_queue)return ; +if (unlikely (current->flags &PF_KTHREAD )) +return ;spin_lock (&swap_avail_lock);plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) { -if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); -break ; +if (!si->bdev) +continue ; +if (blk_get_queue (dev_get_queue (si->bdev))) { + current->throttle_queue =dev_get_queue (si->bdev); + current->use_memdelay = true; +set_notify_resume (current);
Quoted 8 lines
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3780
Quoted 5 lines
mm/swapfile.c +1
-0} +break ; }spin_unlock (&swap_avail_lock); }Sorry, its not clear to me who calls the respective
blk_put_queue()here?
If you look at blkcg_schedule_throttle, it only puts the queue that was in current->throttle_queue. But mem_cgroup_throttle_swaprate exits early when current->throttle_queue is non-zero (first two lines quote above). So when called from mem_cgroup_throttle_swaprate, blkcg_schedule_throttle should never actually put a queue. Open coding the few relevant lines from blkcg_schedule_throttle in mem_cgroup_throttle_swaprate makes that obvious.
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
In reply to Luis ChamberlainQuoted 44 lines · Luis Chamberlain
block devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's
request_queuestructure is also refcounted, however, if the lastblk_put_queue()is called under atomic context the block layer has to defer removal.By refcounting the block device during the use of
blkcg_schedule_throttle(), we ensure ensure two things:1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using therequest_queuestructure in atomic context, since the lastblk_put_queue()will be called upondisk_release(), *after* our ownbdput().This means this code path is *not* going to remove the
request_queuestructure, as we are ensuring some later upper layerdisk_release()will be the one to release therequest_queuestructure for us.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/swapfile.c +1
-0@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3775
Quoted 6 lines
mm/swapfile.c +1
-2@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node,plist_for_each_entry_safe (si, next, &swap_avail_heads[node], avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev);
When swapon, the block_device has been opened in claim_swapfile(), so no need to worry about the queue being gone here.
Thanks,
Ming
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 46 lines · Ming Lei
Quoted 45 lines · Luis Chamberlain
block devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's
request_queuestructure is also refcounted, however, if the lastblk_put_queue()is called under atomic context the block layer has to defer removal.By refcounting the block device during the use of
blkcg_schedule_throttle(), we ensure ensure two things:1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using therequest_queuestructure in atomic context, since the lastblk_put_queue()will be called upondisk_release(), *after* our ownbdput().This means this code path is *not* going to remove the
request_queuestructure, as we are ensuring some later upper layerdisk_release()will be the one to release therequest_queuestructure for us.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/swapfile.c +1
-0@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, plist_for_each_entry_safe(si, next, &swap_avail_heads[node],
Quoted 8 lines
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3775
Quoted 5 lines
mm/swapfile.c +1
-2avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev);When swapon, the
block_devicehas been opened inclaim_swapfile(), so no need to worry about the queue being gone here.
Thanks, so why bdev_get_queue() before?
Luis
Re: [PATCH 4/5] mm/swapfile: refcount block and queue before using blkcg_schedule_throttle()
Quoted 56 lines · Luis Chamberlain
Quoted 46 lines · Ming Lei
Quoted 45 lines · Luis Chamberlain
block devices are refcounted so to ensure once its final user goes away it can be cleaned up by the lower layers properly. The block device's
request_queuestructure is also refcounted, however, if the lastblk_put_queue()is called under atomic context the block layer has to defer removal.By refcounting the block device during the use of
blkcg_schedule_throttle(), we ensure ensure two things:1) the block device remains available during the call
2) we ensure avoid having to deal with the fact we're using therequest_queuestructure in atomic context, since the lastblk_put_queue()will be called upondisk_release(), *after* our ownbdput().This means this code path is *not* going to remove the
request_queuestructure, as we are ensuring some later upper layerdisk_release()will be the one to release therequest_queuestructure for us.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Signed-off-by Luis Chamberlain <[email protected]>--- mm/swapfile.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/swapfile.c +1
-0@@ -3753,6 +3753,7 @@ static void free_swap_count_continuations(struct swap_info_struct *si)void mem_cgroup_throttle_swaprate (struct mem_cgroup *memcg,int node,gfp_t gfp_mask) { +struct block_device *bdev;struct swap_info_struct *si, *next;if (!(gfp_mask & __GFP_IO) || !memcg)return ;@@ -3771,8 +3772,17 @@ void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, plist_for_each_entry_safe(si, next, &swap_avail_heads[node],Quoted 8 lines
mm/swapfile.c void mem_cgroup_throttle_swaprate(struct mem_cgroup *memcg, int node, · line 3775
Quoted 5 lines
mm/swapfile.c +1
-2avail_lists[node]) {if (si->bdev) { -blkcg_schedule_throttle (bdev_get_queue (si->bdev), - true); + bdev =bdgrab (si->bdev);When swapon, the
block_devicehas been opened inclaim_swapfile(), so no need to worry about the queue being gone here.
Quoted 1 line
Thanks, so why
bdev_get_queue()before?
bdev_get_queue() returns the request queue associated with the the block device, and it is just that blkcg_schedule_throttle() needs it.
Maybe I misunderstood your question, if yes, please explain it in a bit detail.
Thanks,
Ming
PATCH 5/5 block: revert back to synchronous request_queue removal
In reply to Luis ChamberlainCommit dc9edc44de6c ("block: Fix a blk_exit_rl() regression") merged on v4.12 moved the work behind blk_release_queue() into a workqueue after a splat floated around which indicated some work on blk_release_queue() could sleep in blk_exit_rl(). This splat would be possible when a driver called blk_put_queue() or blk_cleanup_queue() (which calls blk_put_queue() as its final call) from an atomic context.
blk_put_queue() decrements the refcount for the request_queue kobject, and upon reaching 0 blk_release_queue() is called. Although blk_exit_rl() is now removed through commit db6d9952356 ("block: remove request_list code"), we reserve the right to be able to sleep within blk_release_queue() context. If you see no other way and *have* be in atomic context when you driver calls the last blk_put_queue() you can always just increase your block device's reference count with bdgrab() as this can be done in atomic context and the request_queue removal would be left to upper layers later. We document this bit of tribal knowledge as well now, and adjust kdoc format a bit.
We revert back to synchronous request_queue removal because asynchronous removal creates a regression with expected userspace interaction with several drivers. An example is when removing the loopback driver and issues ioctl from userspace to do so, upon return and if successful one expects the device to be removed. Moving to asynchronous request_queue removal could have broken many scripts which relied on the removal to have been completed if there was no error.
Using asynchronous request_queue removal however has helped us find other bugs, in the future we can test what could break with this arrangement by enabling CONFIG_DEBUG_KOBJECT_RELEASE.
--- block/blk-core.c | 19 ++++++++++++++++++- block/blk-sysfs.c | 38 +++++++++++++++++--------------------- include/linux/blkdev.h | 2 -- 3 files changed, 35 insertions(+), 24 deletions(-)
block/blk-core.c +18 -1
@@ -301,6 +301,17 @@ void blk_clear_pm_only(struct request_queue *q) }EXPORT_SYMBOL_GPL (blk_clear_pm_only); +/** +* blk_put_queue - decrement the request_queue refcount +* +* Decrements the refcount to the request_queue kobject, when this reaches +* 0 we'll have blk_release_queue() called. You should avoid calling +* this function in atomic context but if you really have to ensure you +* first refcount the block device with bdgrab() / bdput() so that the +* last decrement happens in blk_cleanup_queue(). +* +* @q: the request_queue structure to decrement the refcount for +*/ void blk_put_queue (struct request_queue *q) {kobject_put (&q->kobj);
@@ -328,10 +339,16 @@ EXPORT_SYMBOL_GPL(blk_set_queue_dying);/** * blk_cleanup_queue - shutdown a request queue -* @q: request queue to shutdown * * Mark @q DYING, drain all pending requests, mark @q DEAD, destroy and * put it. All future requests will be failed immediately with -ENODEV. +* +* You should not call this function in atomic context. If you need to +* refcount a request_queue in atomic context, instead refcount the +* block device with bdgrab() / bdput(). +* +* @q: request queue to shutdown +* */ void blk_cleanup_queue (struct request_queue *q) {
block/blk-sysfs.c +17 -21
@@ -860,22 +860,27 @@ static void blk_exit_queue(struct request_queue *q)bdi_put (q->backing_dev_info); } -/** -* __blk_release_queue - release a request queue -* @work: pointer to the release_work member of the request queue to be released +* blk_release_queue - release a request queue +* +* This function is called as part of the process when a block device is being +* unregistered. Releasing a request queue starts with blk_cleanup_queue(), +* which set the appropriate flags and then calls blk_put_queue() as the last +* step. blk_put_queue() decrements the reference counter of the request queue +* and once the reference counter reaches zero, this function is called to +* release all allocated resources of the request queue. * -* Description: -* This function is called when a block device is being unregistered. The -* process of releasing a request queue starts with blk_cleanup_queue, which -* set the appropriate flags and then calls blk_put_queue, that decrements -* the reference counter of the request queue. Once the reference counter -* of the request queue reaches zero, blk_release_queue is called to release -* all allocated resources of the request queue. +* This function can sleep, and so we must ensure that the very last +* blk_put_queue() is never called from atomic context. +* +* @kobj: pointer to a kobject, who's container is a request_queue */ -static void __blk_release_queue (struct work_struct *work) +static void blk_release_queue (struct kobject *kobj) { -struct request_queue *q =container_of (work,typeof (*q), release_work); +struct request_queue *q = + container_of(kobj, struct request_queue, kobj); + +might_sleep ();if (test_bit (QUEUE_FLAG_POLL_STATS , &q->queue_flags))blk_stat_remove_callback (q, q->poll_cb);
@@ -905,15 +910,6 @@ static void __blk_release_queue(struct work_struct *work)call_rcu (&q->rcu_head, blk_free_queue_rcu); } -static void blk_release_queue (struct kobject *kobj) -{ -struct request_queue *q = - container_of(kobj, struct request_queue, kobj); - -INIT_WORK (&q->release_work, __blk_release_queue); -schedule_work (&q->release_work); -} -static const struct sysfs_ops queue_sysfs_ops = { .show = queue_attr_show, .store = queue_attr_store,
include/linux/blkdev.h +0 -2
@@ -582,8 +582,6 @@ struct request_queue {size_t cmd_size; -struct work_struct release_work; -#define BLK_MAX_WRITE_HINTS 5u64 write_hints[BLK_MAX_WRITE_HINTS ]; };
Signature · 1 line
2.25.1
Re: [PATCH 5/5] block: revert back to synchronous request_queue removal
Quoted 70 lines · Luis Chamberlain
Commit
dc9edc44de6c("block: Fix ablk_exit_rl()regression") merged on v4.12 moved the work behindblk_release_queue()into a workqueue after a splat floated around which indicated some work onblk_release_queue()could sleep inblk_exit_rl(). This splat would be possible when a driver calledblk_put_queue()orblk_cleanup_queue()(which callsblk_put_queue()as its final call) from an atomic context.
blk_put_queue()decrements the refcount for therequest_queuekobject, and upon reaching 0blk_release_queue()is called. Althoughblk_exit_rl()is now removed through commitdb6d9952356("block: removerequest_listcode"), we reserve the right to be able to sleep withinblk_release_queue()context. If you see no other way and *have* be in atomic context when you driver calls the lastblk_put_queue()you can always just increase your block device's reference count withbdgrab()as this can be done in atomic context and therequest_queueremoval would be left to upper layers later. We document this bit of tribal knowledge as well now, and adjust kdoc format a bit.We revert back to synchronous
request_queueremoval because asynchronous removal creates a regression with expected userspace interaction with several drivers. An example is when removing the loopback driver and issues ioctl from userspace to do so, upon return and if successful one expects the device to be removed. Moving to asynchronousrequest_queueremoval could have broken many scripts which relied on the removal to have been completed if there was no error.Using asynchronous
request_queueremoval however has helped us find other bugs, in the future we can test what could break with this arrangement by enablingCONFIG_DEBUG_KOBJECT_RELEASE.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Suggested-by Nicolai Stange <[email protected]>Fixes dc9edc44de6c ("block: Fix a blk_exit_rl() regression")Signed-off-by Luis Chamberlain <[email protected]>--- block/blk-core.c | 19 ++++++++++++++++++- block/blk-sysfs.c | 38 +++++++++++++++++--------------------- include/linux/blkdev.h | 2 -- 3 files changed, 35 insertions(+), 24 deletions(-)block/blk-core.c +11
-0@@ -301,6 +301,17 @@ void blk_clear_pm_only(struct request_queue *q) }EXPORT_SYMBOL_GPL (blk_clear_pm_only); +/** +* blk_put_queue - decrement the request_queue refcount +* +* Decrements the refcount to the request_queue kobject, when this reaches +* 0 we'll have blk_release_queue() called. You should avoid calling +* this function in atomic context but if you really have to ensure you +* first refcount the block device with bdgrab() / bdput() so that the +* last decrement happens in blk_cleanup_queue(). +* +* @q: the request_queue structure to decrement the refcount for +*/ void blk_put_queue (struct request_queue *q) {kobject_put (&q->kobj);@@ -328,10 +339,16 @@ EXPORT_SYMBOL_GPL(blk_set_queue_dying);
block/blk-core.c EXPORT_SYMBOL_GPL(blk_set_queue_dying); · line 348
Quoted 10 lines
block/blk-core.c +4
-1/** * blk_cleanup_queue - shutdown a request queue -* @q: request queue to shutdown * * Mark @q DYING, drain all pending requests, mark @q DEAD, destroy and * put it. All future requests will be failed immediately with -ENODEV. +* +* You should not call this function in atomic context. If you need to +* refcount a request_queue in atomic context, instead refcount the +* block device with bdgrab() / bdput().
I think this needs a WARN_ON thrown in to enforece the calling context.
block/blk-core.c EXPORT_SYMBOL_GPL(blk_set_queue_dying); · line 350
Quoted 2 lines
block/blk-core.c +2
-0+* +* @q: request queue to shutdown
Moving the argument documentation seems against the usual kerneldoc style.
Otherwise this look good, I hope it sticks :)
Re: [PATCH 5/5] block: revert back to synchronous request_queue removal
Quoted 71 lines · Christoph Hellwig
Quoted 70 lines · Luis Chamberlain
Commit
dc9edc44de6c("block: Fix ablk_exit_rl()regression") merged on v4.12 moved the work behindblk_release_queue()into a workqueue after a splat floated around which indicated some work onblk_release_queue()could sleep inblk_exit_rl(). This splat would be possible when a driver calledblk_put_queue()orblk_cleanup_queue()(which callsblk_put_queue()as its final call) from an atomic context.
blk_put_queue()decrements the refcount for therequest_queuekobject, and upon reaching 0blk_release_queue()is called. Althoughblk_exit_rl()is now removed through commitdb6d9952356("block: removerequest_listcode"), we reserve the right to be able to sleep withinblk_release_queue()context. If you see no other way and *have* be in atomic context when you driver calls the lastblk_put_queue()you can always just increase your block device's reference count withbdgrab()as this can be done in atomic context and therequest_queueremoval would be left to upper layers later. We document this bit of tribal knowledge as well now, and adjust kdoc format a bit.We revert back to synchronous
request_queueremoval because asynchronous removal creates a regression with expected userspace interaction with several drivers. An example is when removing the loopback driver and issues ioctl from userspace to do so, upon return and if successful one expects the device to be removed. Moving to asynchronousrequest_queueremoval could have broken many scripts which relied on the removal to have been completed if there was no error.Using asynchronous
request_queueremoval however has helped us find other bugs, in the future we can test what could break with this arrangement by enablingCONFIG_DEBUG_KOBJECT_RELEASE.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Suggested-by Nicolai Stange <[email protected]>Fixes dc9edc44de6c ("block: Fix a blk_exit_rl() regression")Signed-off-by Luis Chamberlain <[email protected]>--- block/blk-core.c | 19 ++++++++++++++++++- block/blk-sysfs.c | 38 +++++++++++++++++--------------------- include/linux/blkdev.h | 2 -- 3 files changed, 35 insertions(+), 24 deletions(-)block/blk-core.c +11
-0@@ -301,6 +301,17 @@ void blk_clear_pm_only(struct request_queue *q) }EXPORT_SYMBOL_GPL (blk_clear_pm_only); +/** +* blk_put_queue - decrement the request_queue refcount +* +* Decrements the refcount to the request_queue kobject, when this reaches +* 0 we'll have blk_release_queue() called. You should avoid calling +* this function in atomic context but if you really have to ensure you +* first refcount the block device with bdgrab() / bdput() so that the +* last decrement happens in blk_cleanup_queue(). +* +* @q: the request_queue structure to decrement the refcount for +*/ void blk_put_queue (struct request_queue *q) {kobject_put (&q->kobj);@@ -328,10 +339,16 @@ EXPORT_SYMBOL_GPL(blk_set_queue_dying);
Quoted 12 lines
block/blk-core.c EXPORT_SYMBOL_GPL(blk_set_queue_dying); · line 348
Quoted 10 lines
block/blk-core.c +4
-1/** * blk_cleanup_queue - shutdown a request queue -* @q: request queue to shutdown * * Mark @q DYING, drain all pending requests, mark @q DEAD, destroy and * put it. All future requests will be failed immediately with -ENODEV. +* +* You should not call this function in atomic context. If you need to +* refcount a request_queue in atomic context, instead refcount the +* block device with bdgrab() / bdput(). I think this needs a
WARN_ONthrown in to enforece the calling context.
I considered adding a might_sleep() but upon review with Bart, he noted that this function already has a mutex_lock(), and if you look under the hood of mutex_lock(), it has a might_sleep() at the very top. The warning then is implicit.
Quoted 5 lines
block/blk-core.c EXPORT_SYMBOL_GPL(blk_set_queue_dying); · line 350
Quoted 2 lines
block/blk-core.c +2
-0+* +* @q: request queue to shutdown Moving the argument documentation seems against the usual kerneldoc style.
Would you look at that, Documentation/doc-guide/kernel-doc.rst does say to keep the argument at the top as it was in place before, OK will revert that. Sorry, I used include/net/mac80211.h as my base for style.
Quoted 1 line
Otherwise this look good, I hope it sticks :)
I hope that the kdocs / might_sleep() sprinkled should make it stick now. But hey, this uncovered wonderful obscure bugs, it was fun. I'll add a selftest also later to ensure we don't regress on some of this later once again.
Luis
Re: [PATCH 5/5] block: revert back to synchronous request_queue removal
Quoted 6 lines · Luis Chamberlain
Quoted 1 line
I think this needs a
WARN_ONthrown in to enforece the calling context.I considered adding a
might_sleep()but upon review with Bart, he noted that this function already has amutex_lock(), and if you look under the hood ofmutex_lock(), it has amight_sleep()at the very top. The warning then is implicit.
It might just be a personal preference, but I think the documentation value of a WARN_ON_ONCE or might_sleep with a comment at the top of the function is much higher than a blurb in a long kerneldoc text and a later mutex_lock.
Re: [PATCH 5/5] block: revert back to synchronous request_queue removal
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · Luis Chamberlain
Quoted 1 line
I think this needs a
WARN_ONthrown in to enforece the calling context.I considered adding a
might_sleep()but upon review with Bart, he noted that this function already has amutex_lock(), and if you look under the hood ofmutex_lock(), it has amight_sleep()at the very top. The warning then is implicit.
Quoted 4 lines
It might just be a personal preference, but I think the documentation value of a
WARN_ON_ONCEormight_sleepwith a comment at the top of the function is much higher than a blurb in a long kerneldoc text and a latermutex_lock.
Well I'm a fan of making this explicit, so sure will just sprinkle a might_sleep(), even though we have a mutex_lock().
Luis
Re: [PATCH 5/5] block: revert back to synchronous request_queue removal
In reply to Luis ChamberlainQuoted 162 lines · Luis Chamberlain
Commit
dc9edc44de6c("block: Fix ablk_exit_rl()regression") merged on v4.12 moved the work behindblk_release_queue()into a workqueue after a splat floated around which indicated some work onblk_release_queue()could sleep inblk_exit_rl(). This splat would be possible when a driver calledblk_put_queue()orblk_cleanup_queue()(which callsblk_put_queue()as its final call) from an atomic context.
blk_put_queue()decrements the refcount for therequest_queuekobject, and upon reaching 0blk_release_queue()is called. Althoughblk_exit_rl()is now removed through commitdb6d9952356("block: removerequest_listcode"), we reserve the right to be able to sleep withinblk_release_queue()context. If you see no other way and *have* be in atomic context when you driver calls the lastblk_put_queue()you can always just increase your block device's reference count withbdgrab()as this can be done in atomic context and therequest_queueremoval would be left to upper layers later. We document this bit of tribal knowledge as well now, and adjust kdoc format a bit.We revert back to synchronous
request_queueremoval because asynchronous removal creates a regression with expected userspace interaction with several drivers. An example is when removing the loopback driver and issues ioctl from userspace to do so, upon return and if successful one expects the device to be removed. Moving to asynchronousrequest_queueremoval could have broken many scripts which relied on the removal to have been completed if there was no error.Using asynchronous
request_queueremoval however has helped us find other bugs, in the future we can test what could break with this arrangement by enablingCONFIG_DEBUG_KOBJECT_RELEASE.Cc Bart Van Assche <[email protected]>Cc Omar Sandoval <[email protected]>Cc Hannes Reinecke <[email protected]>Cc Nicolai Stange <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Michal Hocko <[email protected]>Cc yu kuai <[email protected]>Suggested-by Nicolai Stange <[email protected]>Fixes dc9edc44de6c ("block: Fix a blk_exit_rl() regression")Signed-off-by Luis Chamberlain <[email protected]>--- block/blk-core.c | 19 ++++++++++++++++++- block/blk-sysfs.c | 38 +++++++++++++++++--------------------- include/linux/blkdev.h | 2 -- 3 files changed, 35 insertions(+), 24 deletions(-)block/blk-core.c +18
-1@@ -301,6 +301,17 @@ void blk_clear_pm_only(struct request_queue *q) }EXPORT_SYMBOL_GPL (blk_clear_pm_only); +/** +* blk_put_queue - decrement the request_queue refcount +* +* Decrements the refcount to the request_queue kobject, when this reaches +* 0 we'll have blk_release_queue() called. You should avoid calling +* this function in atomic context but if you really have to ensure you +* first refcount the block device with bdgrab() / bdput() so that the +* last decrement happens in blk_cleanup_queue(). +* +* @q: the request_queue structure to decrement the refcount for +*/ void blk_put_queue (struct request_queue *q) {kobject_put (&q->kobj);@@ -328,10 +339,16 @@ EXPORT_SYMBOL_GPL(blk_set_queue_dying);/** * blk_cleanup_queue - shutdown a request queue -* @q: request queue to shutdown * * Mark @q DYING, drain all pending requests, mark @q DEAD, destroy and * put it. All future requests will be failed immediately with -ENODEV. +* +* You should not call this function in atomic context. If you need to +* refcount a request_queue in atomic context, instead refcount the +* block device with bdgrab() / bdput(). +* +* @q: request queue to shutdown +* */ void blk_cleanup_queue (struct request_queue *q) {block/blk-sysfs.c +17
-21@@ -860,22 +860,27 @@ static void blk_exit_queue(struct request_queue *q)bdi_put (q->backing_dev_info); } -/** -* __blk_release_queue - release a request queue -* @work: pointer to the release_work member of the request queue to be released +* blk_release_queue - release a request queue +* +* This function is called as part of the process when a block device is being +* unregistered. Releasing a request queue starts with blk_cleanup_queue(), +* which set the appropriate flags and then calls blk_put_queue() as the last +* step. blk_put_queue() decrements the reference counter of the request queue +* and once the reference counter reaches zero, this function is called to +* release all allocated resources of the request queue. * -* Description: -* This function is called when a block device is being unregistered. The -* process of releasing a request queue starts with blk_cleanup_queue, which -* set the appropriate flags and then calls blk_put_queue, that decrements -* the reference counter of the request queue. Once the reference counter -* of the request queue reaches zero, blk_release_queue is called to release -* all allocated resources of the request queue. +* This function can sleep, and so we must ensure that the very last +* blk_put_queue() is never called from atomic context. +* +* @kobj: pointer to a kobject, who's container is a request_queue */ -static void __blk_release_queue (struct work_struct *work) +static void blk_release_queue (struct kobject *kobj) { -struct request_queue *q =container_of (work,typeof (*q), release_work); +struct request_queue *q = + container_of(kobj, struct request_queue, kobj); + +might_sleep ();if (test_bit (QUEUE_FLAG_POLL_STATS , &q->queue_flags))blk_stat_remove_callback (q, q->poll_cb);@@ -905,15 +910,6 @@ static void __blk_release_queue(struct work_struct *work)call_rcu (&q->rcu_head, blk_free_queue_rcu); } -static void blk_release_queue (struct kobject *kobj) -{ -struct request_queue *q = - container_of(kobj, struct request_queue, kobj); - -INIT_WORK (&q->release_work, __blk_release_queue); -schedule_work (&q->release_work); -} -static const struct sysfs_ops queue_sysfs_ops = { .show = queue_attr_show, .store = queue_attr_store,include/linux/blkdev.h +0
-2@@ -582,8 +582,6 @@ struct request_queue {size_t cmd_size; -struct work_struct release_work; -#define BLK_MAX_WRITE_HINTS 5u64 write_hints[BLK_MAX_WRITE_HINTS ]; };Signature · 1 line
2.25.1
Thanks,
Ming
Quoted 14 lines · Luis Chamberlain
After two iterations of RFCs I think this is ready now. I've taken the feedback from the last series, both on code and commit log. I've also extended the commit log on the last patch to also explain how the original shift to async
request_queueremoval turned out to actually be a userspace regression and added a respective fixes tag for it.You can find these patches on my 20200414-dev-blkqueue-defer-removal-patch-v1 branch based on linux-next tag next-20200414 on kernel.org [0].
Further review and rants are appreciated.
Quoted 7 lines
Luis Chamberlain (5):
block: move main block debugfs initialization to its own file
blktrace: fix debugfs use after free
blktrace: refcount therequest_queueduring ioctl mm/swapfile: refcount block and queue before using
blkcg_schedule_throttle()
block: revert back to synchronousrequest_queueremoval
Looks good from a debugfs point of view, thanks for doing this cleanup.
greg k-h