Skip to contentSkip to conversation

kdump hangs up by Sysrq+C trigger in high load.

PATCH Yoshihiro Takahashi 2 participants 3 messages Dec 16, 2008 – Dec 17, 2008

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi.

kdump hangs up by Sysrq+C trigger once in about 10 times in high load. After the above occurs, kdump cannot collect vmcores with NMI button. When waiting_for_crash_ipi does case more than 0, mdelay seem to make a stall.

While nmi_shootdown_cpus(), mdelay may not work since interruption is disable. (ex. when crashing kernel by NMI interruption, all interruption may be disabled.).

So I think that it should use other way (TSC) instead of mdelay() in nmi_shootdown_cpus().

Here is the log.
--
SysRq : Trigger a crashdump

BUG: warning at arch/x86/kernel/crash.c:107/nmi_shootdown_cpus() (Not tainted)
---
 crash.c |   26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
arch/x86/kernel/crash.c +21 -5
index 2685538..fc63636 100644
@@ -89,7 +89,8 @@ static struct notifier_block crash_nmi_nb = {
 
 static void nmi_shootdown_cpus(void)
 {
-	unsigned long msecs;
+	int i, tsc_wait, tsc_count;
+	unsigned long tsc_start, tsc_now, tsc_delay;
 
 	atomic_set(&waiting_for_crash_ipi, num_online_cpus() - 1);
 	/* Would it be better to replace the trap vector here? */
@@ -102,10 +103,25 @@ static void nmi_shootdown_cpus(void)
 
 	smp_send_nmi_allbutself();
 
-	msecs = 1000; /* Wait at most a second for the other cpus to stop */
-	while ((atomic_read(&waiting_for_crash_ipi) > 0) && msecs) {
-		mdelay(1);
-		msecs--;
+	tsc_delay = HZ; /* Wait at most a second for the other cpus to stop */
+	while ((atomic_read(&waiting_for_crash_ipi) > 0) && tsc_delay) {
+		tsc_wait = 1000000; /* msec */
+		tsc_count = 16384;
+		while (tsc_wait > 0) {
+			if (tsc_count-- < 0) {
+				break;
+			}
+			tsc_start = get_cycles();
+			for (i = 0; i < 100000; i++) {
+				;
+			}
+			tsc_now = get_cycles();
+			tsc_wait -= tsc_now - tsc_start;
+		}
+		if (tsc_count < 0) {
+			break;
+		}
+		tsc_delay--;
 	}
 
 	/* Leave the nmi callback set */
#1 lore raw
Quoted 7 lines · Yoshihiro Takahashi

Hi.

kdump hangs up by Sysrq+C trigger once in about 10 times in high load. After the above occurs, kdump cannot collect vmcores with NMI button. When waiting_for_crash_ipi does case more than 0, mdelay seem to make a stall.

Quoted 6 lines

While nmi_shootdown_cpus(), mdelay may not work since interruption is disable. (ex. when crashing kernel by NMI interruption, all interruption may be disabled.).

So I think that it should use other way (TSC) instead of mdelay() in nmi_shootdown_cpus().

mdelay uses either the tsc or CPU loops - neither stop while IRQs are disabled.

Could you figure out exactly why it hangs in your case?

Ingo

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw

Ingo. Thank you, your reply.

I will serch it again.

Quoted 17 lines · On Tue, 16 Dec 2008 12:09:49 +0100 Ingo Molnar <[email protected]> wrote:
Quoted 7 lines · Yoshihiro Takahashi

Hi.

kdump hangs up by Sysrq+C trigger once in about 10 times in high load. After the above occurs, kdump cannot collect vmcores with NMI button. When waiting_for_crash_ipi does case more than 0, mdelay seem to make a stall.

Quoted 6 lines

While nmi_shootdown_cpus(), mdelay may not work since interruption is disable. (ex. when crashing kernel by NMI interruption, all interruption may be disabled.).

So I think that it should use other way (TSC) instead of mdelay() in nmi_shootdown_cpus().

Quoted 6 lines

mdelay uses either the tsc or CPU loops - neither stop while IRQs are disabled.

Could you figure out exactly why it hangs in your case?

Ingo

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec