PM: hibernate: Freeze kernel threads in software_resume()

PATCH Applied Dexuan Cui 5 participants 6 messages Apr 24, 2020 – Apr 27, 2020

Subsystems (3)

Stable branch Suspend to RAM Hibernation

lore

Share & tools

lore

Dexuan Cui

author

2020-04-24 03:41 UTC #0 lore raw

PATCH 1/1 PM: hibernate: Freeze kernel threads in software_resume()

Currently the kernel threads are not frozen in software_resume(), so between dpm_suspend_start(PMSG_QUIESCE) and resume_target_kernel(), system_freezable_power_efficient_wq can still try to submit SCSI commands and this can cause a panic since the low level SCSI driver (e.g. hv_storvsc) has quiesced the SCSI adapter and can not accept any SCSI commands: https://lkml.org/lkml/2020/4/10/47

At first I posted a fix (https://lkml.org/lkml/2020/4/21/1318) trying to resolve the issue from hv_storvsc, but with the help of Bart Van Assche, I realized it's better to fix software_resume(), since this looks like a generic issue, not only pertaining to SCSI.

Cc Bart Van Assche <[email protected]>

Cc [email protected]

Signed-off-by Dexuan Cui <[email protected]>

---
 kernel/power/hibernate.c | 7 +++++++
 1 file changed, 7 insertions(+)

kernel/power/hibernate.c +7 -0

index 86aba8706b16..30bd28d1d418 100644
@@ -898,6 +898,13 @@ static int software_resume(void)
 	error = freeze_processes();
 	if (error)
 		goto Close_Finish;
+
+	error = freeze_kernel_threads();
+	if (error) {
+		thaw_processes();
+		goto Close_Finish;
+	}
+
 	error = load_image_and_restore();
 	thaw_processes();
  Finish:

Signature · 1 line

2.19.1

Sasha Levin

maintainer

2020-04-26 15:03 UTC #1 lore raw

[This is an automated email]

This commit has been processed because it contains a -stable tag. The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.6.7, v5.4.35, v4.19.118, v4.14.177, v4.9.220, v4.4.220.

v5.6.7: Build OK!
v5.4.35: Build OK!
v4.19.118: Build OK!
v4.14.177: Build OK!
v4.9.220: Build OK!
v4.4.220: Failed to apply! Possible dependencies:
ea00f4f4f00c ("PM / sleep: make PM notifiers called symmetrically") fe12c00d21bb ("PM / hibernate: Introduce test_resume mode for hibernation")

NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

Signature · 2 lines

Thanks
Sasha

Rafael J. Wysocki

Applied

2020-04-26 16:24 UTC #2 lore raw

In reply to Dexuan Cui

Quoted 28 lines · CEST Dexuan Cui

Currently the kernel threads are not frozen in software_resume(), so between dpm_suspend_start(PMSG_QUIESCE) and resume_target_kernel(), system_freezable_power_efficient_wq can still try to submit SCSI commands and this can cause a panic since the low level SCSI driver (e.g. hv_storvsc) has quiesced the SCSI adapter and can not accept any SCSI commands: https://lkml.org/lkml/2020/4/10/47
At first I posted a fix (https://lkml.org/lkml/2020/4/21/1318) trying to resolve the issue from hv_storvsc, but with the help of Bart Van Assche, I realized it's better to fix software_resume(), since this looks like a generic issue, not only pertaining to SCSI.
Cc Bart Van Assche <[email protected]>
Cc [email protected]
Signed-off-by Dexuan Cui <[email protected]>
---
 kernel/power/hibernate.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/power/hibernate.c +0 -0
index 86aba8706b16..30bd28d1d418 100644
@@ -898,6 +898,13 @@ static int software_resume(void)
 	error = freeze_processes();
 	if (error)
 		goto Close_Finish;

kernel/power/hibernate.c static int software_resume(void) · line 910

Quoted 10 lines

kernel/power/hibernate.c +7 -0

+
+	error = freeze_kernel_threads();
+	if (error) {
+		thaw_processes();
+		goto Close_Finish;
+	}
+
 	error = load_image_and_restore();
 	thaw_processes();
  Finish:

Applied as a fix for 5.7-rc4, thanks!

Bart Van Assche

2020-04-26 18:34 UTC #3 lore raw

Quoted 32 lines · Rafael J. Wysocki

Quoted 31 lines · CEST Dexuan Cui
Currently the kernel threads are not frozen in software_resume(), so between dpm_suspend_start(PMSG_QUIESCE) and resume_target_kernel(), system_freezable_power_efficient_wq can still try to submit SCSI commands and this can cause a panic since the low level SCSI driver (e.g. hv_storvsc) has quiesced the SCSI adapter and can not accept any SCSI commands: https://lkml.org/lkml/2020/4/10/47
At first I posted a fix (https://lkml.org/lkml/2020/4/21/1318) trying to resolve the issue from hv_storvsc, but with the help of Bart Van Assche, I realized it's better to fix software_resume(), since this looks like a generic issue, not only pertaining to SCSI.
Cc Bart Van Assche <[email protected]>
Cc [email protected]
Signed-off-by Dexuan Cui <[email protected]>
---
 kernel/power/hibernate.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/power/hibernate.c +4 -0
index 86aba8706b16..30bd28d1d418 100644
@@ -898,6 +898,13 @@ static int software_resume(void)
 	error = freeze_processes();
 	if (error)
 		goto Close_Finish;
+
+	error = freeze_kernel_threads();
+	if (error) {
+		thaw_processes();

Quoted 8 lines

kernel/power/hibernate.c static int software_resume(void) · line 910
Quoted 6 lines
kernel/power/hibernate.c +3 -0
+		goto Close_Finish;
+	}
+
 	error = load_image_and_restore();
 	thaw_processes();
  Finish:
Applied as a fix for 5.7-rc4, thanks!

Hi Rafael,

What is not clear to me is how kernel threads are thawed after load_image_and_restore() has finished? Should a comment perhaps be added above the freeze_kernel_threads() call that explains how thaw_kernel_threads() is invoked after load_image_and_restore() has finished?

Thanks,

Bart.

Dexuan Cui

author

2020-04-27 00:58 UTC #4 lore raw

Quoted 24 lines

From: Bart Van Assche <[email protected]>
Sent: Sunday, April 26, 2020 11:34 AM
To: Rafael J. Wysocki <[email protected]>; Dexuan Cui <[email protected]>
Quoted 10 lines
Quoted 10 lines
kernel/power/hibernate.c +4 -0
@@ -898,6 +898,13 @@ static int software_resume(void)
 	error = freeze_processes();
 	if (error)
 		goto Close_Finish;
+
+	error = freeze_kernel_threads();
+	if (error) {
+		thaw_processes();
Quoted 8 lines
kernel/power/hibernate.c static int software_resume(void) · line 910
Quoted 6 lines
kernel/power/hibernate.c +3 -0
+		goto Close_Finish;
+	}
+
 	error = load_image_and_restore();
 	thaw_processes();
  Finish:
Applied as a fix for 5.7-rc4, thanks!
Hi Rafael,

Quoted 7 lines

What is not clear to me is how kernel threads are thawed after load_image_and_restore() has finished? Should a comment perhaps be added above the freeze_kernel_threads() call that explains how thaw_kernel_threads() is invoked after load_image_and_restore() has finished?
Bart.

Hi Bart, Rafael, I would suggest the below comment:

If load_image_and_restore() succeeds, it won't return, and the execution will be restored from the 'old' kernel's hibernate() -> hibernation_snapshot() -> create_image() -> swsusp_arch_suspend(), and later hibernate() -> thaw_processes() will thaw every frozen kernel process and userspace process of the 'old' kernel.

Thanks,
-- Dexuan

Rafael J. Wysocki

maintainer

2020-04-27 08:44 UTC #5 lore raw

In reply to Bart Van Assche

Quoted 43 lines · Bart Van Assche

Quoted 32 lines · Rafael J. Wysocki
Quoted 31 lines · CEST Dexuan Cui
Currently the kernel threads are not frozen in software_resume(), so between dpm_suspend_start(PMSG_QUIESCE) and resume_target_kernel(), system_freezable_power_efficient_wq can still try to submit SCSI commands and this can cause a panic since the low level SCSI driver (e.g. hv_storvsc) has quiesced the SCSI adapter and can not accept any SCSI commands: https://lkml.org/lkml/2020/4/10/47
At first I posted a fix (https://lkml.org/lkml/2020/4/21/1318) trying to resolve the issue from hv_storvsc, but with the help of Bart Van Assche, I realized it's better to fix software_resume(), since this looks like a generic issue, not only pertaining to SCSI.
Cc Bart Van Assche <[email protected]>
Cc [email protected]
Signed-off-by Dexuan Cui <[email protected]>
---
 kernel/power/hibernate.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/power/hibernate.c +4 -0
index 86aba8706b16..30bd28d1d418 100644
@@ -898,6 +898,13 @@ static int software_resume(void)
     error = freeze_processes();
     if (error)
             goto Close_Finish;
+
+    error = freeze_kernel_threads();
+    if (error) {
+            thaw_processes();
Quoted 8 lines
kernel/power/hibernate.c static int software_resume(void) · line 910
Quoted 6 lines
kernel/power/hibernate.c +3 -0
+            goto Close_Finish;
+    }
+
     error = load_image_and_restore();
     thaw_processes();
  Finish:
Applied as a fix for 5.7-rc4, thanks!

Quoted 7 lines

Hi Rafael,
What is not clear to me is how kernel threads are thawed after load_image_and_restore() has finished? Should a comment perhaps be added above the freeze_kernel_threads() call that explains how thaw_kernel_threads() is invoked after load_image_and_restore() has finished?

It isn't, because that is not necessary.

thaw_processes() will thaw them along with the user space.

Cheers!