Skip to contentSkip to conversation

Revert "Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d""

PATCH In review Song Liu 5 participants 6 messages Jan 25, 2024 – Feb 14, 2024

Subsystems (3)
lore
Share & tools
lore

Song Liu

author maintainer
#0 lore raw

PATCH 1/1 Revert "Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d""

This reverts commit bed9e27baf52a09b7ba2a3714f1e24e17ced386d.

The original set [1][2] was expected to undo a suboptimal fix in [2], and replace it with a better fix [1]. However, as reported by Dan Moulding [2] causes an issue with raid5 with journal device.

Revert [2] for now to close the issue. We will follow up on another issue reported by Juxiao Bi, as [2] is expected to fix it. We believe this is a good trade-off, because the latter issue happens less freqently.

In the meanwhile, we will NOT revert [1], as it contains the right logic.

Fixes bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")
Cc [email protected] # v5.19+
Cc Junxiao Bi <[email protected]>
Cc Yu Kuai <[email protected]>

[1] commit d6e035aad6c0 ("md: bypass block throttle for superblock update")
[2] commit bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")

---
 drivers/md/raid5.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/md/raid5.c +12 -0
index 8497880135ee..2b2f03705990 100644
@@ -36,6 +36,7 @@
  */
 
 #include <linux/blkdev.h>
+#include <linux/delay.h>
 #include <linux/kthread.h>
 #include <linux/raid/pq.h>
 #include <linux/async_tx.h>
@@ -6773,7 +6774,18 @@ static void raid5d(struct md_thread *thread)
 			spin_unlock_irq(&conf->device_lock);
 			md_check_recovery(mddev);
 			spin_lock_irq(&conf->device_lock);
+
+			/*
+			 * Waiting on MD_SB_CHANGE_PENDING below may deadlock
+			 * seeing md_check_recovery() is needed to clear
+			 * the flag when using mdmon.
+			 */
+			continue;
 		}
+
+		wait_event_lock_irq(mddev->sb_wait,
+			!test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags),
+			conf->device_lock);
 	}
 	pr_debug("%d stripes handled\n", handled);
 
Signature · 1 line
2.34.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 20 lines · Song Liu

This reverts commit bed9e27baf52a09b7ba2a3714f1e24e17ced386d.

The original set [1][2] was expected to undo a suboptimal fix in [2], and replace it with a better fix [1]. However, as reported by Dan Moulding [2] causes an issue with raid5 with journal device.

Revert [2] for now to close the issue. We will follow up on another issue reported by Juxiao Bi, as [2] is expected to fix it. We believe this is a good trade-off, because the latter issue happens less freqently.

In the meanwhile, we will NOT revert [1], as it contains the right logic.

Fixes bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")
Cc [email protected] # v5.19+
Cc Junxiao Bi<[email protected]>
Cc Yu Kuai<[email protected]>
Quoted 2 lines

[1] commit d6e035aad6c0 ("md: bypass block throttle for superblock update")
[2] commit bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")

LGTM

#2 lore raw

PATCH 1/1 Revert "Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d""

In reply to Song Liu

Thank you Song. Let me know if there is any more information I can provide to help diagnose or reproduce this.

-- Dan

#3 lore raw
In reply to Song Liu

Should we get some understanding what is the issue before reverting the commit? I am not clear what is the issue, already asked Dan in another thread.

Thanks,

Junxiao.

Quoted 57 lines · Song Liu

This reverts commit bed9e27baf52a09b7ba2a3714f1e24e17ced386d.

The original set [1][2] was expected to undo a suboptimal fix in [2], and replace it with a better fix [1]. However, as reported by Dan Moulding [2] causes an issue with raid5 with journal device.

Revert [2] for now to close the issue. We will follow up on another issue reported by Juxiao Bi, as [2] is expected to fix it. We believe this is a good trade-off, because the latter issue happens less freqently.

In the meanwhile, we will NOT revert [1], as it contains the right logic.

Fixes bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")
Cc [email protected] # v5.19+
Cc Junxiao Bi <[email protected]>
Cc Yu Kuai <[email protected]>

[1] commit d6e035aad6c0 ("md: bypass block throttle for superblock update")
[2] commit bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")

---
  drivers/md/raid5.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
drivers/md/raid5.c +12 -0
index 8497880135ee..2b2f03705990 100644
@@ -36,6 +36,7 @@
   */
 
  #include <linux/blkdev.h>
+#include <linux/delay.h>
  #include <linux/kthread.h>
  #include <linux/raid/pq.h>
  #include <linux/async_tx.h>
@@ -6773,7 +6774,18 @@ static void raid5d(struct md_thread *thread)
  			spin_unlock_irq(&conf->device_lock);
  			md_check_recovery(mddev);
  			spin_lock_irq(&conf->device_lock);
+
+			/*
+			 * Waiting on MD_SB_CHANGE_PENDING below may deadlock
+			 * seeing md_check_recovery() is needed to clear
+			 * the flag when using mdmon.
+			 */
+			continue;
  		}
+
+		wait_event_lock_irq(mddev->sb_wait,
+			!test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags),
+			conf->device_lock);
  	}
  	pr_debug("%d stripes handled\n", handled);
 
#4 lore raw
In reply to Song Liu
Quoted 10 lines · Song Liu

This reverts commit bed9e27baf52a09b7ba2a3714f1e24e17ced386d.

The original set [1][2] was expected to undo a suboptimal fix in [2], and replace it with a better fix [1]. However, as reported by Dan Moulding [2] causes an issue with raid5 with journal device.

Revert [2] for now to close the issue. We will follow up on another issue reported by Juxiao Bi, as [2] is expected to fix it. We believe this is a good trade-off, because the latter issue happens less freqently.

Quoted 6 lines

In the meanwhile, we will NOT revert [1], as it contains the right logic.

Fixes bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")
Cc [email protected] # v5.19+

#regzbot ^introduced bed9e27baf52a09b7ba2a3714f1
#regzbot title md/raid5: issue with raid5 with journal device #regzbot fix Revert "Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d""
#regzbot ignore-activity

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) --
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.

#5 lore raw
Quoted 18 lines · Linux regression tracking #adding (Thorsten Leemhuis)
Quoted 10 lines · Song Liu

This reverts commit bed9e27baf52a09b7ba2a3714f1e24e17ced386d.

The original set [1][2] was expected to undo a suboptimal fix in [2], and replace it with a better fix [1]. However, as reported by Dan Moulding [2] causes an issue with raid5 with journal device.

Revert [2] for now to close the issue. We will follow up on another issue reported by Juxiao Bi, as [2] is expected to fix it. We believe this is a good trade-off, because the latter issue happens less freqently.

Quoted 6 lines

In the meanwhile, we will NOT revert [1], as it contains the right logic.

Fixes bed9e27baf52 ("Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d"")
Cc [email protected] # v5.19+
Quoted 5 lines

#regzbot ^introduced bed9e27baf52a09b7ba2a3714f1
#regzbot title md/raid5: issue with raid5 with journal device #regzbot fix Revert "Revert "md/raid5: Wait for MD_SB_CHANGE_PENDING in raid5d""
#regzbot ignore-activity

I mixed something up here, let me fix this:

#regzbot introduced 0de40f76d56
#regzbot fix 87165c64fe1a98bbab
#regzbot ignore-activity

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) --
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.