Skip to contentSkip to conversation

block: aoe: fix page fault in freedev()

PATCH In review Valentin Kleibel 2 participants 10 messages Mar 10, 2022 – Apr 11, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 block: aoe: fix page fault in freedev()

There is a bug in the aoe driver module where every forcible removal of an aoe device (eg. "rmmod aoe" with aoe devices available or "aoe-flush ex.x") leads to a page fault. The code in freedev() calls blk_mq_free_tag_set() before running blk_cleanup_queue() which leads to this issue (drivers/block/aoe/aoedev.c L281ff). This issue was fixed upstream in commit 6560ec9 (aoe: use blk_mq_alloc_disk and blk_cleanup_disk) with the introduction and use of the function blk_cleanup_disk().

This patch applies to kernels 5.4 and 5.10.

The function calls are reordered to match the behavior of blk_cleanup_disk() to mitigate this issue.

Fixes 3582dd2 (aoe: convert aoeblk to blk-mq)

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215647

---
 drivers/block/aoe/aoedev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/block/aoe/aoedev.c +1 -1
index e2ea2356da06..08c98ea724ea 100644
@@ -277,9 +277,9 @@ freedev(struct aoedev *d)
        if (d->gd) {
                aoedisk_rm_debugfs(d);
                del_gendisk(d->gd);
+               blk_cleanup_queue(d->blkq);
                put_disk(d->gd);
                blk_mq_free_tag_set(&d->tag_set);
-               blk_cleanup_queue(d->blkq);
        }
        t = d->targets;
        e = t + d->ntargets;
#1 lore raw
Quoted 11 lines · Valentin Kleibel

There is a bug in the aoe driver module where every forcible removal of an aoe device (eg. "rmmod aoe" with aoe devices available or "aoe-flush ex.x") leads to a page fault.
The code in freedev() calls blk_mq_free_tag_set() before running blk_cleanup_queue() which leads to this issue (drivers/block/aoe/aoedev.c L281ff).
This issue was fixed upstream in commit 6560ec9 (aoe: use blk_mq_alloc_disk and blk_cleanup_disk) with the introduction and use of the function blk_cleanup_disk().

This patch applies to kernels 5.4 and 5.10.

We need a fix for Linus's tree first before we can backport anything to older kernels. Does this also work there?

Quoted 5 lines

The function calls are reordered to match the behavior of blk_cleanup_disk() to mitigate this issue.

Fixes 3582dd2 (aoe: convert aoeblk to blk-mq)

A few more digits in the sha1 here would be good, otherwise our tools will complain. It should look like:

Fixes 3582dd291788 ("aoe: convert aoeblk to blk-mq")

thanks,

greg k-h

#2 lore raw
Quoted 4 lines · Greg Kroah-Hartman
Quoted 1 line

This patch applies to kernels 5.4 and 5.10.

We need a fix for Linus's tree first before we can backport anything to older kernels. Does this also work there?

It is fixed in Linus' tree starting with 5.14.
The patch reproduces the behavior of the current master introduced in commit 6560ec961a08 (aoe: use blk_mq_alloc_disk and blk_cleanup_disk).

Quoted 5 lines
Quoted 1 line
Fixes 3582dd2 (aoe: convert aoeblk to blk-mq)

A few more digits in the sha1 here would be good, otherwise our tools will complain. It should look like:

Fixes 3582dd291788 ("aoe: convert aoeblk to blk-mq")

thanks for the hint, i will do so in the future.

cheers,
valentin

#3 lore raw
Quoted 7 lines · Valentin Kleibel
Quoted 4 lines · Greg Kroah-Hartman
Quoted 1 line

This patch applies to kernels 5.4 and 5.10.

We need a fix for Linus's tree first before we can backport anything to older kernels. Does this also work there?

It is fixed in Linus' tree starting with 5.14.

What commit fixes it there? Why not just backport that one only?

thanks,

greg k-h

#4 lore raw
Quoted 10 lines · Greg Kroah-Hartman
Quoted 7 lines · Valentin Kleibel
Quoted 4 lines · Greg Kroah-Hartman
Quoted 1 line

This patch applies to kernels 5.4 and 5.10.

We need a fix for Linus's tree first before we can backport anything to older kernels. Does this also work there?

It is fixed in Linus' tree starting with 5.14.

What commit fixes it there? Why not just backport that one only?

commit 6560ec961a08 (aoe: use blk_mq_alloc_disk and blk_cleanup_disk)
This commit uses the function blk_cleanup_disk() in freedev() in drivers/block/aoe/aoedev.c which fixes the issue. The function was introduced in f525464a8000 (block: add blk_alloc_disk and blk_cleanup_disk APIs):
void blk_cleanup_disk(struct gendisk *disk)

{
	blk_cleanup_queue(disk->queue);
	put_disk(disk);
}

EXPORT_SYMBOL(blk_cleanup_disk);

I tried to backport the fix to the lts kernels without introducing a new API by just adjusting the order of the two function calls. Is it preferable to introduce and use the function blk_cleanup_disk()?

cheers,
valentin

#5 lore raw
Quoted 22 lines · Valentin Kleibel
Quoted 10 lines · Greg Kroah-Hartman
Quoted 7 lines · Valentin Kleibel
Quoted 4 lines · Greg Kroah-Hartman
Quoted 1 line

This patch applies to kernels 5.4 and 5.10.

We need a fix for Linus's tree first before we can backport anything to older kernels. Does this also work there?

It is fixed in Linus' tree starting with 5.14.

What commit fixes it there? Why not just backport that one only?

commit 6560ec961a08 (aoe: use blk_mq_alloc_disk and blk_cleanup_disk) This commit uses the function blk_cleanup_disk() in freedev() in drivers/block/aoe/aoedev.c which fixes the issue.
The function was introduced in f525464a8000 (block: add blk_alloc_disk and blk_cleanup_disk APIs):
void blk_cleanup_disk(struct gendisk *disk)

{
	blk_cleanup_queue(disk->queue);
	put_disk(disk);
}
Quoted 5 lines

EXPORT_SYMBOL(blk_cleanup_disk);

I tried to backport the fix to the lts kernels without introducing a new API by just adjusting the order of the two function calls.
Is it preferable to introduce and use the function blk_cleanup_disk()?

I do not know, sorry. But we prefer the upstream commits as much as possible as one-off changes like this are almost always buggy and wrong in the end.

Try doing the backports and see what that looks like please.

thanks,

greg k-h

#6 lore raw
Quoted 5 lines

I do not know, sorry. But we prefer the upstream commits as much as possible as one-off changes like this are almost always buggy and wrong in the end.

Try doing the backports and see what that looks like please.

I did the backports now but Christoph Hellwig already pointed out:

Quoted 2 lines

No idea what is hidden in bugzilla, but the blk_mq_alloc_disk changes aren't easily backportable.

[https://lore.kernel.org/all/[email protected]/]

Therefore I cherry-picked only the changes for blk_cleanup_disk as they are much simpler than the changes to blk_mq_alloc_disk.

I kept the original commit messages, please tell me if you feel they should be changed or do so yourself.

Please apply to 5.4 and 5.10.

Cheers,
Valentin

changelog:
v2:
* cherry pick from upstream commits instead of creating a new patch

#7 lore raw

PATCH v2 1/2 block: add blk_alloc_disk and blk_cleanup_disk APIs

Add two new APIs to allocate and free a gendisk including the request_queue for use with BIO based drivers. This is to avoid boilerplate code in drivers.

(cherry picked from commit f525464a8000f092c20b00eead3eaa9d849c599e)
Fixes 3582dd291788 (aoe: convert aoeblk to blk-mq)

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215647

---
 block/genhd.c         | 15 +++++++++++++++
 include/linux/genhd.h |  1 +
 2 files changed, 16 insertions(+)
block/genhd.c +15 -0
index 796baf761202..421cad085502 100644
@@ -1836,6 +1836,21 @@ void put_disk_and_module(struct gendisk *disk)
        }
 }
 EXPORT_SYMBOL(put_disk_and_module);
+/**
+ * blk_cleanup_disk - shutdown a gendisk allocated by blk_alloc_disk
+ * @disk: gendisk to shutdown
+ *
+ * Mark the queue hanging off @disk DYING, drain all pending requests, then mark
+ * the queue DEAD, destroy and put it and the gendisk structure.
+ *
+ * Context: can sleep
+ */
+void blk_cleanup_disk(struct gendisk *disk)
+{
+       blk_cleanup_queue(disk->queue);
+       put_disk(disk);
+}
+EXPORT_SYMBOL(blk_cleanup_disk);
 
 static void set_disk_ro_uevent(struct gendisk *gd, int ro)
 {
include/linux/genhd.h +1 -0
index 03da3f603d30..b7b180d3734a 100644
@@ -369,6 +369,7 @@ extern void blk_unregister_region(dev_t devt, unsigned long range);
 #define alloc_disk(minors) alloc_disk_node(minors, NUMA_NO_NODE)
 
 int register_blkdev(unsigned int major, const char *name);
+void blk_cleanup_disk(struct gendisk *disk);
 void unregister_blkdev(unsigned int major, const char *name);
 
 void revalidate_disk_size(struct gendisk *disk, bool verbose);
#8 lore raw

Re: [PATCH v2 1/2] block: add blk_alloc_disk and blk_cleanup_disk APIs

Quoted 57 lines · Valentin Kleibel

Add two new APIs to allocate and free a gendisk including the request_queue for use with BIO based drivers. This is to avoid boilerplate code in drivers.

(cherry picked from commit f525464a8000f092c20b00eead3eaa9d849c599e)
Fixes 3582dd291788 (aoe: convert aoeblk to blk-mq)

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215647

---
 block/genhd.c         | 15 +++++++++++++++
 include/linux/genhd.h |  1 +
 2 files changed, 16 insertions(+)
block/genhd.c +5 -0
index 796baf761202..421cad085502 100644
@@ -1836,6 +1836,21 @@ void put_disk_and_module(struct gendisk *disk)
        }
 }
 EXPORT_SYMBOL(put_disk_and_module);
+/**
+ * blk_cleanup_disk - shutdown a gendisk allocated by blk_alloc_disk
+ * @disk: gendisk to shutdown
+ *
+ * Mark the queue hanging off @disk DYING, drain all pending requests, then

mark
+ * the queue DEAD, destroy and put it and the gendisk structure.
+ *
+ * Context: can sleep
+ */

block/genhd.c +6 -0
+void blk_cleanup_disk(struct gendisk *disk)
+{
+       blk_cleanup_queue(disk->queue);
+       put_disk(disk);
+}
+EXPORT_SYMBOL(blk_cleanup_disk);
 
 static void set_disk_ro_uevent(struct gendisk *gd, int ro)
 {
include/linux/genhd.h +1 -0
index 03da3f603d30..b7b180d3734a 100644
@@ -369,6 +369,7 @@ extern void blk_unregister_region(dev_t devt, unsigned
 long range);
 #define alloc_disk(minors) alloc_disk_node(minors, NUMA_NO_NODE)
 
 int register_blkdev(unsigned int major, const char *name);
+void blk_cleanup_disk(struct gendisk *disk);
 void unregister_blkdev(unsigned int major, const char *name);
 
Quoted 1 line

void revalidate_disk_size(struct gendisk *disk, bool verbose);

This backport looks to be incomplete, and is also totally whitespace damaged and can not be applied at all :(

Please fix both up and resend.

thanks,

greg k-h

#9 lore raw

PATCH v2 2/2 aoe: use blk_mq_alloc_disk and blk_cleanup_disk

In reply to Valentin Kleibel

Use blk_mq_alloc_disk and blk_cleanup_disk to simplify the gendisk and request_queue allocation.

(cherry picked from commit 6560ec961a080944f8d5e1fef17b771bfaf189cb)
Fixes 3582dd291788 (aoe: convert aoeblk to blk-mq)

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215647

---
 drivers/block/aoe/aoedev.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/block/aoe/aoedev.c +1 -2
index e2ea2356da06..c5753c6bfe80 100644
@@ -277,9 +277,8 @@ freedev(struct aoedev *d)
        if (d->gd) {
                aoedisk_rm_debugfs(d);
                del_gendisk(d->gd);
-               put_disk(d->gd);
+               blk_cleanup_disk(d->gd);
                blk_mq_free_tag_set(&d->tag_set);
-               blk_cleanup_queue(d->blkq);
        }
        t = d->targets;
        e = t + d->ntargets;