Skip to contentSkip to conversation

sleeping functions in invalid context on panic fixes

PATCH Applied Nicholas Piggin 2 participants 5 messages May 14, 2018 – May 21, 2018

Subsystems (2)
lore
Share & tools
lore

Nicholas Piggin

author reviewer
#0 lore raw

cover sleeping functions in invalid context on panic fixes

Here's a couple of fixes which seem to solve a problem where panics can hang forever rather than reboot after 10 seconds. The symptoms are that a CPU calls panic(), but later it is found in idle.

Nicholas Piggin (2):
powerpc/powernv: Fix opal_event_shutdown() called with interrupts
disabled
powerpc/powernv: Fix NVRAM sleep in invalid context when crashing

 arch/powerpc/platforms/powernv/opal-irqchip.c |  2 +-
 arch/powerpc/platforms/powernv/opal-nvram.c   | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)
Signature · 1 line
2.17.0
author reviewer
#1 lore raw

PATCH 1/2 powerpc/powernv: Fix opal_event_shutdown() called with interrupts disabled

A kernel crash in process context that calls emergency_restart from panic will end up calling opal_event_shutdown with interrupts disabled but not in interrupt. This causes a sleeping function to be called which gives the following warning with sysrq+c:

    Rebooting in 10 seconds..
    BUG: sleeping function called from invalid context at kernel/locking/mutex.c:238
    in_atomic(): 0, irqs_disabled(): 1, pid: 7669, name: bash
    CPU: 20 PID: 7669 Comm: bash Tainted: G      D W         4.17.0-rc5+ #3
    Call Trace:
    dump_stack+0xb0/0xf4 (unreliable)
    ___might_sleep+0x174/0x1a0
    mutex_lock+0x38/0xb0
    __free_irq+0x68/0x460
    free_irq+0x70/0xc0
    opal_event_shutdown+0xb4/0xf0
    opal_shutdown+0x24/0xa0
    pnv_shutdown+0x28/0x40
    machine_shutdown+0x44/0x60
    machine_restart+0x28/0x80
    emergency_restart+0x30/0x50
    panic+0x2a0/0x328
    oops_end+0x1ec/0x1f0
    bad_page_fault+0xe8/0x154
    handle_page_fault+0x34/0x38
    --- interrupt: 300 at sysrq_handle_crash+0x44/0x60
    LR = __handle_sysrq+0xfc/0x260
    flag_spec.62335+0x12b844/0x1e8db4 (unreliable)
    __handle_sysrq+0xfc/0x260
    write_sysrq_trigger+0xa8/0xb0
    proc_reg_write+0xac/0x110
    __vfs_write+0x6c/0x240
    vfs_write+0xd0/0x240
    ksys_write+0x6c/0x110
Fixes 9f0fd0499d30 ("powerpc/powernv: Add a virtual irqchip for opal events")
---
 arch/powerpc/platforms/powernv/opal-irqchip.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/platforms/powernv/opal-irqchip.c +1 -1
index 9d1b8c0aaf93..05ffe05f0fdc 100644
@@ -177,7 +177,7 @@ void opal_event_shutdown(void)
 		if (!opal_irqs[i])
 			continue;
 
-		if (in_interrupt())
+		if (in_interrupt() || irqs_disabled())
 			disable_irq_nosync(opal_irqs[i]);
 		else
 			free_irq(opal_irqs[i], NULL);
Signature · 1 line
2.17.0
#2 lore raw

Re: [1/2] powerpc/powernv: Fix opal_event_shutdown() called with interrupts disabled

Quoted 35 lines · Nicholas Piggin

A kernel crash in process context that calls emergency_restart from panic will end up calling opal_event_shutdown with interrupts disabled but not in interrupt. This causes a sleeping function to be called which gives the following warning with sysrq+c:

    Rebooting in 10 seconds..
    BUG: sleeping function called from invalid context at kernel/locking/mutex.c:238
    in_atomic(): 0, irqs_disabled(): 1, pid: 7669, name: bash
    CPU: 20 PID: 7669 Comm: bash Tainted: G      D W         4.17.0-rc5+ #3
    Call Trace:
    dump_stack+0xb0/0xf4 (unreliable)
    ___might_sleep+0x174/0x1a0
    mutex_lock+0x38/0xb0
    __free_irq+0x68/0x460
    free_irq+0x70/0xc0
    opal_event_shutdown+0xb4/0xf0
    opal_shutdown+0x24/0xa0
    pnv_shutdown+0x28/0x40
    machine_shutdown+0x44/0x60
    machine_restart+0x28/0x80
    emergency_restart+0x30/0x50
    panic+0x2a0/0x328
    oops_end+0x1ec/0x1f0
    bad_page_fault+0xe8/0x154
    handle_page_fault+0x34/0x38
    --- interrupt: 300 at sysrq_handle_crash+0x44/0x60
    LR = __handle_sysrq+0xfc/0x260
    flag_spec.62335+0x12b844/0x1e8db4 (unreliable)
    __handle_sysrq+0xfc/0x260
    write_sysrq_trigger+0xa8/0xb0
    proc_reg_write+0xac/0x110
    __vfs_write+0x6c/0x240
    vfs_write+0xd0/0x240
    ksys_write+0x6c/0x110
Quoted 2 lines
Fixes 9f0fd0499d30 ("powerpc/powernv: Add a virtual irqchip for opal events")

Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/c0beffc4f4c658fde86d52c837e784

cheers

author reviewer
#3 lore raw

PATCH 2/2 powerpc/powernv: Fix NVRAM sleep in invalid context when crashing

In reply to Nicholas Piggin

Similarly to opal_event_shutdown, opal_nvram_write can be called in the crash path with irqs disabled. Special case the delay to avoid sleeping in invalid context.

Fixes 3b8070335f ("powerpc/powernv: Fix OPAL NVRAM driver OPAL_BUSY loops")
---
 arch/powerpc/platforms/powernv/opal-nvram.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
arch/powerpc/platforms/powernv/opal-nvram.c +12 -2
index 1bceb95f422d..5584247f5029 100644
@@ -44,6 +44,10 @@ static ssize_t opal_nvram_read(char *buf, size_t count, loff_t *index)
 	return count;
 }
 
+/*
+ * This can be called in the panic path with interrupts off, so use
+ * mdelay in that case.
+ */
 static ssize_t opal_nvram_write(char *buf, size_t count, loff_t *index)
 {
 	s64 rc = OPAL_BUSY;
@@ -58,10 +62,16 @@ static ssize_t opal_nvram_write(char *buf, size_t count, loff_t *index)
 	while (rc == OPAL_BUSY || rc == OPAL_BUSY_EVENT) {
 		rc = opal_write_nvram(__pa(buf), count, off);
 		if (rc == OPAL_BUSY_EVENT) {
-			msleep(OPAL_BUSY_DELAY_MS);
+			if (in_interrupt() || irqs_disabled())
+				mdelay(OPAL_BUSY_DELAY_MS);
+			else
+				msleep(OPAL_BUSY_DELAY_MS);
 			opal_poll_events(NULL);
 		} else if (rc == OPAL_BUSY) {
-			msleep(OPAL_BUSY_DELAY_MS);
+			if (in_interrupt() || irqs_disabled())
+				mdelay(OPAL_BUSY_DELAY_MS);
+			else
+				msleep(OPAL_BUSY_DELAY_MS);
 		}
 	}
 
Signature · 1 line
2.17.0
#4 lore raw

Re: [2/2] powerpc/powernv: Fix NVRAM sleep in invalid context when crashing

Quoted 7 lines · Nicholas Piggin

Similarly to opal_event_shutdown, opal_nvram_write can be called in the crash path with irqs disabled. Special case the delay to avoid sleeping in invalid context.

Fixes 3b8070335f ("powerpc/powernv: Fix OPAL NVRAM driver OPAL_BUSY loops")

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/c1d2a31397ec51f0370f6bd17b19b3

cheers