Skip to contentSkip to conversation

reboot: enable IRQs before do_exit in the halt and power off fallback

PATCH In review Bradley Morgan 3 participants 4 messages Jul 12 – Jul 13 Stable branch

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 reboot: enable IRQs before do_exit in the halt and power off fallback

The reboot syscall calls do_exit(0) after kernel_halt() or kernel_power_off(). Those are expected to stop the machine and not return. When they do return (no PM info, power off failed), the shutdown path has already disabled interrupts: native_machine_shutdown() calls local_irq_disable() on x86, and do_exit() then hits its WARN_ON(irqs_disabled()) at kernel/exit.c:930.

do_exit only warns by design; make_task_dead() is the path that fixes the IRQs disabled state (commit 001c28e57187 ("exit: Detect and fix irq disabled state in oops")). The reboot fallback is not an oops and wants a clean do_exit, so enable IRQs at the two call sites instead, matching the make_task_dead pattern.

Splat from syzbot:

ACPI: PM: Preparing to enter system sleep state S5
kvm: exiting hardware virtualization
reboot: Power down

  ------------[ cut here ]------------
  irqs_disabled()
  WARNING: kernel/exit.c:930 at do_exit+0x1cf7/0x2ae0 kernel/exit.c:930, CPU#0: init/6193
  CPU: 0 UID: 0 PID: 6193 Comm: init Tainted: G             L      syzkaller #0 PREEMPT(full)
  Tainted: [L]=SOFTLOCKUP
  Call Trace:
   <TASK>
   __do_sys_reboot+0x36e/0x400 kernel/reboot.c:784
   do_syscall_64+0x115/0x840 arch/x86/entry/syscall_64.c:94
   entry_SYSCALL_64_after_hwframe+0x77/0x7f
   </TASK>
---
 kernel/reboot.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/reboot.c +10 -0
index bed6967bfa96..7e8ebb470721 100644
@@ -777,10 +777,20 @@ SYSCALL_DEFINE4(reboot, int, magic1, int, magic2, unsigned int, cmd,
 
 	case LINUX_REBOOT_CMD_HALT:
 		kernel_halt();
+		/* kernel_halt() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: halt returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 
 	case LINUX_REBOOT_CMD_POWER_OFF:
 		kernel_power_off();
+		/* kernel_power_off() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: power off returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 		break;
 
Signature · 1 line
2.53.0
#1 lore raw
Quoted 7 lines · Bradley Morgan

The reboot syscall calls do_exit(0) after kernel_halt() or kernel_power_off(). Those are expected to stop the machine and not return. When they do return (no PM info, power off failed), the shutdown path has already disabled interrupts: native_machine_shutdown() calls local_irq_disable() on x86, and do_exit() then hits its WARN_ON(irqs_disabled()) at kernel/exit.c:930.

Quoted 5 lines

do_exit only warns by design; make_task_dead() is the path that fixes the IRQs disabled state (commit 001c28e57187 ("exit: Detect and fix irq disabled state in oops")). The reboot fallback is not an oops and wants a clean do_exit, so enable IRQs at the two call sites instead, matching the make_task_dead pattern.

I think this is fixing symptoms not the actual cause.

How does kernel_halt or kernel_power_off manage to return?

If they aren't supposed to return changing the code to call make_task_dead to indicate you are on an error path is probably the better fix.

Eric

Quoted 51 lines

Splat from syzbot:

ACPI: PM: Preparing to enter system sleep state S5
kvm: exiting hardware virtualization
reboot: Power down

  ------------[ cut here ]------------
  irqs_disabled()
  WARNING: kernel/exit.c:930 at do_exit+0x1cf7/0x2ae0 kernel/exit.c:930, CPU#0: init/6193
  CPU: 0 UID: 0 PID: 6193 Comm: init Tainted: G             L      syzkaller #0 PREEMPT(full)
  Tainted: [L]=SOFTLOCKUP
  Call Trace:
   <TASK>
   __do_sys_reboot+0x36e/0x400 kernel/reboot.c:784
   do_syscall_64+0x115/0x840 arch/x86/entry/syscall_64.c:94
   entry_SYSCALL_64_after_hwframe+0x77/0x7f
   </TASK>
---
 kernel/reboot.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/reboot.c +10 -0
index bed6967bfa96..7e8ebb470721 100644
@@ -777,10 +777,20 @@ SYSCALL_DEFINE4(reboot, int, magic1, int, magic2, unsigned int, cmd,
 
 	case LINUX_REBOOT_CMD_HALT:
 		kernel_halt();
+		/* kernel_halt() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: halt returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 
 	case LINUX_REBOOT_CMD_POWER_OFF:
 		kernel_power_off();
+		/* kernel_power_off() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: power off returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 		break;
#2 lore raw
Quoted 17 lines · Eric W. Biederman
Quoted 7 lines · Bradley Morgan

The reboot syscall calls do_exit(0) after kernel_halt() or kernel_power_off(). Those are expected to stop the machine and not return. When they do return (no PM info, power off failed), the shutdown path has already disabled interrupts: native_machine_shutdown() calls local_irq_disable() on x86, and do_exit() then hits its WARN_ON(irqs_disabled()) at kernel/exit.c:930.

Quoted 5 lines

do_exit only warns by design; make_task_dead() is the path that fixes the IRQs disabled state (commit 001c28e57187 ("exit: Detect and fix irq disabled state in oops")). The reboot fallback is not an oops and wants a clean do_exit, so enable IRQs at the two call sites instead, matching the make_task_dead pattern.

I think this is fixing symptoms not the actual cause.

Quoted 7 lines

How does kernel_halt or kernel_power_off manage to return?

If they aren't supposed to return changing the code to call make_task_dead to indicate you are on an error path is probably the better fix.

Eric

Doh! Nice catch!

I'll do a V2 with some other things I needed to address too, e.g:

Quoted 19 lines
Quoted 8 lines

Splat from syzbot:

ACPI: PM: Preparing to enter system sleep state S5
kvm: exiting hardware virtualization
reboot: Power down

  ------------[ cut here ]------------
  irqs_disabled()
  WARNING: kernel/exit.c:930 at do_exit+0x1cf7/0x2ae0 kernel/exit.c:930,

CPU#0: init/6193

Quoted 1 line
  CPU: 0 UID: 0 PID: 6193 Comm: init Tainted: G             L     

syzkaller #0 PREEMPT(full)

Quoted 8 lines
  Tainted: [L]=SOFTLOCKUP
  Call Trace:
   <TASK>
   __do_sys_reboot+0x36e/0x400 kernel/reboot.c:784
   do_syscall_64+0x115/0x840 arch/x86/entry/syscall_64.c:94
   entry_SYSCALL_64_after_hwframe+0x77/0x7f
   </TASK>
Quoted 5 lines
Quoted 4 lines
Fixes 001c28e57187 ("exit: Detect and fix irq disabled state in oops")
Reported-by [email protected]

Closes:

https://lore.kernel.org/all/[email protected]/T/

The lore links wrong, I'll fix that bug later.

Quoted 22 lines

kernel/reboot.c SYSCALL_DEFINE4(reboot, int, magic1, int, magic2,

Quoted 11 lines
---
 kernel/reboot.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/reboot.c +0 -0
index bed6967bfa96..7e8ebb470721 100644
@@ -777,10 +777,20 @@ SYSCALL_DEFINE4(reboot, int, magic1, int, magic2,

unsigned int, cmd,

Quoted 10 lines
kernel/reboot.c +5 -0
 	case LINUX_REBOOT_CMD_HALT:
 		kernel_halt();
+		/* kernel_halt() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: halt returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 
Quoted 10 lines

kernel/reboot.c SYSCALL_DEFINE4(reboot, int, magic1, int, magic2, unsigned int, cmd, · line 795

Quoted 9 lines
kernel/reboot.c +5 -0
 	case LINUX_REBOOT_CMD_POWER_OFF:
 		kernel_power_off();
+		/* kernel_power_off() was expected to not return. */
+		if (irqs_disabled()) {
+			pr_info("reboot: power off returned with irqs disabled\n");
+			local_irq_enable();
+		}
 		do_exit(0);
 		break;

Thanks!

#3 lore raw
In reply to Eric W. Biederman
Quoted 16 lines · Eric W. Biederman
Quoted 7 lines · Bradley Morgan

The reboot syscall calls do_exit(0) after kernel_halt() or kernel_power_off(). Those are expected to stop the machine and not return. When they do return (no PM info, power off failed), the shutdown path has already disabled interrupts: native_machine_shutdown() calls local_irq_disable() on x86, and do_exit() then hits its WARN_ON(irqs_disabled()) at kernel/exit.c:930.

Quoted 5 lines

do_exit only warns by design; make_task_dead() is the path that fixes the IRQs disabled state (commit 001c28e57187 ("exit: Detect and fix irq disabled state in oops")). The reboot fallback is not an oops and wants a clean do_exit, so enable IRQs at the two call sites instead, matching the make_task_dead pattern.

Quoted 3 lines

I think this is fixing symptoms not the actual cause.

How does kernel_halt or kernel_power_off manage to return?

Agreed... and this was already reported twice at least. See

	https://lore.kernel.org/all/[email protected]/

	https://lore.kernel.org/all/[email protected]/

Oleg.