md/raid1: stop mdx_raid1 thread when raid1 array run failed
PATCH v3 1/1 md/raid1: stop mdx_raid1 thread when raid1 array run failed
From: Jiang Li <[email protected]>
fail run raid1 array when we assemble array with the inactive disk only, but the mdx_raid1 thread were not stop, Even if the associated resources have been released. it will caused a NULL dereference when we do poweroff.
This causes the following Oops:
[ 287.587787] BUG: kernel NULL pointer dereference, address: 0000000000000070
[ 287.594762] #PF: supervisor read access in kernel mode
[ 287.599912] #PF: error_code(0x0000) - not-present page
[ 287.605061] PGD 0 P4D 0
[ 287.607612] Oops: 0000 [#1] SMP NOPTI
[ 287.611287] CPU: 3 PID: 5265 Comm: md0_raid1 Tainted: G U 5.10.146 #0
[ 287.619029] Hardware name: xxxxxxx/To be filled by O.E.M, BIOS 5.19 06/16/2022
[ 287.626775] RIP: 0010:md_check_recovery+0x57/0x500 [md_mod]
[ 287.632357] Code: fe 01 00 00 48 83 bb 10 03 00 00 00 74 08 48 89 ......
[ 287.651118] RSP: 0018:ffffc90000433d78 EFLAGS: 00010202
[ 287.656347] RAX: 0000000000000000 RBX: ffff888105986800 RCX: 0000000000000000
[ 287.663491] RDX: ffffc90000433bb0 RSI: 00000000ffffefff RDI: ffff888105986800
[ 287.670634] RBP: ffffc90000433da0 R08: 0000000000000000 R09: c0000000ffffefff
[ 287.677771] R10: 0000000000000001 R11: ffffc90000433ba8 R12: ffff888105986800
[ 287.684907] R13: 0000000000000000 R14: fffffffffffffe00 R15: ffff888100b6b500
[ 287.692052] FS: 0000000000000000(0000) GS:ffff888277f80000(0000) knlGS:0000000000000000
[ 287.700149] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 287.705897] CR2: 0000000000000070 CR3: 000000000320a000 CR4: 0000000000350ee0
[ 287.713033] Call Trace:
[ 287.715498] raid1d+0x6c/0xbbb [raid1]
[ 287.719256] ? __schedule+0x1ff/0x760
[ 287.722930] ? schedule+0x3b/0xb0
[ 287.726260] ? schedule_timeout+0x1ed/0x290
[ 287.730456] ? __switch_to+0x11f/0x400
[ 287.734219] md_thread+0xe9/0x140 [md_mod]
[ 287.738328] ? md_thread+0xe9/0x140 [md_mod]
[ 287.742601] ? wait_woken+0x80/0x80
[ 287.746097] ? md_register_thread+0xe0/0xe0 [md_mod]
[ 287.751064] kthread+0x11a/0x140
[ 287.754300] ? kthread_park+0x90/0x90
[ 287.757974] ret_from_fork+0x1f/0x30In fact, when raid1 array run fail, we need to domd_unregister_thread() before raid1_free().
---
Hi,
my full name is Jiang Li. Thanks!
The following steps can be used to reproduce the problem:
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[a-b] mdadm -A /dev/md0 /dev/sd[a-b]
mkfs.ext4 /dev/md0
mount /dev/md0 /mnt/md0
mdadm -S /dev/md0
mdadm -A /dev/md0 /dev/sda --run
mount /dev/md0 /mnt/md0
#Store a certain amount of data, so that the rebuilding will not be completed in a short time dd if=/dev/urandom of=/mnt/md0/test.db bs=1M count=40960
mdadm --add /dev/md0 /dev/sdb
sleep 1
#when the array were rebuilding, we stop the array. umount /dev/md0
mdadm -S /dev/md0
#assemble array with the sdb only in --run insist mode:
mdadm -A /dev/md0 /dev/sdb --run
#md run fail with kernel message as follow:
[ 850.593441] md: md0 stopped. [ 850.672071] md: pers->run() failed ... [ 850.686913] md: md0 stopped.
#we can see the md0_raid1 thread were not stop:
$ps -ef | grep raid1
root 5672 2 0 Oct31 ? 00:00:04 [md0_raid1]
#now, do poweroff can reproduce the problem.
drivers/md/raid1.c | 1 + 1 file changed, 1 insertion(+)
drivers/md/raid1.c +1 -0
@@ -3159,6 +3159,7 @@ static int raid1_run(struct mddev *mddev)* RAID1 needs at least one disk in active */ if (conf->raid_disks - mddev->degraded <1 ) { +md_unregister_thread (&conf->thread); ret = -EINVAL ; goto abort; }
Signature · 1 line
2.25.1
Quoted 40 lines · Jiang Li
From: Jiang Li <[email protected]>
fail run raid1 array when we assemble array with the inactive disk only, but the
mdx_raid1thread were not stop, Even if the associated resources have been released. it will caused a NULL dereference when we do poweroff.This causes the following Oops:
[ 287.587787] BUG: kernel NULL pointer dereference, address: 0000000000000070 [ 287.594762] #PF: supervisor read access in kernel mode [ 287.599912] #PF: error_code(0x0000) - not-present page [ 287.605061] PGD 0 P4D 0 [ 287.607612] Oops: 0000 [#1] SMP NOPTI [ 287.611287] CPU: 3 PID: 5265 Comm: md0_raid1 Tainted: G U 5.10.146 #0 [ 287.619029] Hardware name: xxxxxxx/To be filled by O.E.M, BIOS 5.19 06/16/2022 [ 287.626775] RIP: 0010:md_check_recovery+0x57/0x500 [md_mod] [ 287.632357] Code: fe 01 00 00 48 83 bb 10 03 00 00 00 74 08 48 89 ...... [ 287.651118] RSP: 0018:ffffc90000433d78 EFLAGS: 00010202 [ 287.656347] RAX: 0000000000000000 RBX: ffff888105986800 RCX: 0000000000000000 [ 287.663491] RDX: ffffc90000433bb0 RSI: 00000000ffffefff RDI: ffff888105986800 [ 287.670634] RBP: ffffc90000433da0 R08: 0000000000000000 R09: c0000000ffffefff [ 287.677771] R10: 0000000000000001 R11: ffffc90000433ba8 R12: ffff888105986800 [ 287.684907] R13: 0000000000000000 R14: fffffffffffffe00 R15: ffff888100b6b500 [ 287.692052] FS: 0000000000000000(0000) GS:ffff888277f80000(0000) knlGS:0000000000000000 [ 287.700149] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 287.705897] CR2: 0000000000000070 CR3: 000000000320a000 CR4: 0000000000350ee0 [ 287.713033] Call Trace: [ 287.715498] raid1d+0x6c/0xbbb [raid1] [ 287.719256] ? __schedule+0x1ff/0x760 [ 287.722930] ? schedule+0x3b/0xb0 [ 287.726260] ? schedule_timeout+0x1ed/0x290 [ 287.730456] ? __switch_to+0x11f/0x400 [ 287.734219] md_thread+0xe9/0x140 [md_mod] [ 287.738328] ? md_thread+0xe9/0x140 [md_mod] [ 287.742601] ? wait_woken+0x80/0x80 [ 287.746097] ? md_register_thread+0xe0/0xe0 [md_mod] [ 287.751064] kthread+0x11a/0x140 [ 287.754300] ? kthread_park+0x90/0x90 [ 287.757974] ret_from_fork+0x1f/0x30
Quoted 4 lines
In fact, when raid1 array run fail, we need to do
md_unregister_thread()beforeraid1_free().Signed-off-by Jiang Li <[email protected]>
Applied to md-next. Thanks!
Song
Quoted 55 lines
---
Hi,
my full name is Jiang Li. Thanks!The following steps can be used to reproduce the problem:
mdadm --create/dev/md0--level=1 --raid-devices=2 /dev/sd[a-b] mdadm -A/dev/md0/dev/sd[a-b]
mkfs.ext4/dev/md0
mount/dev/md0/mnt/md0mdadm -S
/dev/md0mdadm -A
/dev/md0/dev/sda--run
mount/dev/md0/mnt/md0
#Store a certain amount of data, so that the rebuilding will not be completed in a short time dd if=/dev/urandom of=/mnt/md0/test.db bs=1M count=40960mdadm --add
/dev/md0/dev/sdb
sleep 1
#when the array were rebuilding, we stop the array. umount/dev/md0
mdadm -S/dev/md0#assemble array with the sdb only in --run insist mode:
mdadm -A/dev/md0/dev/sdb--run#md run fail with kernel message as follow:
[ 850.593441] md: md0 stopped. [ 850.672071] md: pers->run() failed ... [ 850.686913] md: md0 stopped.#we can see the
md0_raid1thread were not stop:
$ps -ef | grep raid1
root 5672 2 0 Oct31 ? 00:00:04 [md0_raid1]#now, do poweroff can reproduce the problem.
drivers/md/raid1.c | 1 + 1 file changed, 1 insertion(+)drivers/md/raid1.c +1
-1@@ -3159,6 +3159,7 @@ static int raid1_run(struct mddev *mddev)* RAID1 needs at least one disk in active */ if (conf->raid_disks - mddev->degraded <1 ) { +md_unregister_thread (&conf->thread); ret = -EINVAL ; goto abort; } --2.25.1