Skip to contentSkip to conversation

clockevents: fix resume logic

DISCUSSION Andrew Morton 5 participants 31 messages Sep 10, 2007 – Oct 2, 2007 1 message not in the archive

Subsystems (11)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 13 lines · On Sun, 22 Jul 2007 01:59:11 GMT Linux Kernel Mailing List…
Gitweb:     http://git.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6
Commit:     18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6
Parent:     93da56efcf8c6a111f0349f6b7651172d4745ca0
Author:     Thomas Gleixner <[email protected]>
AuthorDate: Sat Jul 21 04:37:34 2007 -0700
Committer:  Linus Torvalds <[email protected]>
CommitDate: Sat Jul 21 17:49:15 2007 -0700

clockevents: fix resume logic

We need to make sure, that the clockevent devices are resumed, before the tick is resumed. The current resume logic does not guarantee this.

Quoted 8 lines

Add CLOCK_EVT_MODE_RESUME and call the set mode functions of the clock event devices before resuming the tick / oneshot functionality.

Fixup the existing users.

Thanks to Nigel Cunningham for tracking down a long standing thinko, which affected the jinxed VAIO.

This patch broke the jinxed vaio.

Which is a bit odd, considering that I must have tested it at the time. But I bisected it right down to this commit, and the below revert patch fixed it up.

The symptoms are that with this patch applied, resume-from-RAM will get stuck partway through doing its stuff. If I then repeatedly hit a key on the keyboard, resume will struggle through and complete. The system time is then a few seconds behind the time which `hwclock' says, so it looks like we're also not restoring the time correctly.

Also, a `reboot -f' get stuck in the same way. No progress until I start hitting a key.

With the below revert patch against current-Linus-mainline, resume-from-RAM does work correctly. However suspend-to-disk is still busted, in the same
way: I need to repeatedly hit a key to make progress.

From: Andrew Morton <[email protected]>

Revert:

commit 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6

Author: Thomas Gleixner <[email protected]>
Date:   Sat Jul 21 04:37:34 2007 -0700

clockevents: fix resume logic

We need to make sure, that the clockevent devices are resumed, before the tick is resumed. The current resume logic does not guarantee this.

Add CLOCK_EVT_MODE_RESUME and call the set mode functions of the clock event devices before resuming the tick / oneshot functionality.

Fixup the existing users.

Thanks to Nigel Cunningham for tracking down a long standing thinko, which affected the jinxed VAIO.

It causes the following on the Vaio:

- resume-from-ram gets stuck and requires repeated hitting of a key for it to make progress

- `reboot -f' fails in the same way, with the same fix

- the system time after resume is wrong: it is behind the hwclock time by a period which appears to be equal to the time spent in suspend

Cc Thomas Gleixner <[email protected]>
Cc john stultz <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc "Rafael J. Wysocki" <[email protected]>

---

 arch/arm/mach-davinci/time.c      |    2 
 arch/arm/mach-imx/time.c          |    1 
 arch/arm/mach-ixp4xx/common.c     |    2 
 arch/arm/mach-omap1/time.c        |    1 
 arch/arm/plat-omap/timer32k.c     |    2 
 arch/i386/kernel/apic.c           |    3 -
 arch/i386/kernel/hpet.c           |   71 ++++++++++++++++++++++++++--
 arch/i386/kernel/i8253.c          |   29 +++++------
 arch/i386/kernel/vmiclock.c       |    1 
 arch/i386/xen/time.c              |    3 -
 arch/sh/kernel/timers/timer-tmu.c |    1 
 arch/sparc64/kernel/time.c        |    1 
 drivers/lguest/lguest.c           |    2 
 include/linux/clockchips.h        |    1 
 kernel/time/tick-broadcast.c      |    6 --
 kernel/time/tick-common.c         |   16 ++----
 16 files changed, 88 insertions(+), 54 deletions(-)
arch/arm/mach-davinci/time.c renamed from arch/arm/mach-davinci/time.c~a +0 -2
diff -puN arch/arm/mach-davinci/time.c~a arch/arm/mach-davinci/time.c
@@ -285,8 +285,6 @@ static void davinci_set_mode(enum clock_
 	case CLOCK_EVT_MODE_SHUTDOWN:
 		t->opts = TIMER_OPTS_DISABLED;
 		break;
-	case CLOCK_EVT_MODE_RESUME:
-		break;
 	}
 }
 
arch/arm/mach-imx/time.c renamed from arch/arm/mach-imx/time.c~a +0 -1
diff -puN arch/arm/mach-imx/time.c~a arch/arm/mach-imx/time.c
@@ -159,7 +159,6 @@ static void imx_set_mode(enum clock_even
 		break;
 	case CLOCK_EVT_MODE_SHUTDOWN:
 	case CLOCK_EVT_MODE_UNUSED:
-	case CLOCK_EVT_MODE_RESUME:
 		/* Left event sources disabled, no more interrupts appears */
 		break;
 	}
arch/arm/mach-ixp4xx/common.c renamed from arch/arm/mach-ixp4xx/common.c~a +0 -2
diff -puN arch/arm/mach-ixp4xx/common.c~a arch/arm/mach-ixp4xx/common.c
@@ -459,8 +459,6 @@ static void ixp4xx_set_mode(enum clock_e
 	default:
 		osrt = opts = 0;
 		break;
-	case CLOCK_EVT_MODE_RESUME:
-		break;
 	}
 
 	*IXP4XX_OSRT1 = osrt | opts;
arch/arm/mach-omap1/time.c renamed from arch/arm/mach-omap1/time.c~a +0 -1
diff -puN arch/arm/mach-omap1/time.c~a arch/arm/mach-omap1/time.c
@@ -156,7 +156,6 @@ static void omap_mpu_set_mode(enum clock
 		break;
 	case CLOCK_EVT_MODE_UNUSED:
 	case CLOCK_EVT_MODE_SHUTDOWN:
-	case CLOCK_EVT_MODE_RESUME:
 		break;
 	}
 }
arch/arm/plat-omap/timer32k.c renamed from arch/arm/plat-omap/timer32k.c~a +0 -2
diff -puN arch/arm/plat-omap/timer32k.c~a arch/arm/plat-omap/timer32k.c
@@ -157,8 +157,6 @@ static void omap_32k_timer_set_mode(enum
 	case CLOCK_EVT_MODE_UNUSED:
 	case CLOCK_EVT_MODE_SHUTDOWN:
 		break;
-	case CLOCK_EVT_MODE_RESUME:
-		break;
 	}
 }
 
arch/i386/kernel/apic.c renamed from arch/i386/kernel/apic.c~a +0 -3
diff -puN arch/i386/kernel/apic.c~a arch/i386/kernel/apic.c
@@ -264,9 +264,6 @@ static void lapic_timer_setup(enum clock
 		v |= (APIC_LVT_MASKED | LOCAL_TIMER_VECTOR);
 		apic_write_around(APIC_LVTT, v);
 		break;
-	case CLOCK_EVT_MODE_RESUME:
-		/* Nothing to do here */
-		break;
 	}
 
 	local_irq_restore(flags);
arch/i386/kernel/hpet.c renamed from arch/i386/kernel/hpet.c~a +66 -5

92 lines not on this page · raw message

arch/i386/kernel/i8253.c renamed from arch/i386/kernel/i8253.c~a +14 -15
diff -puN arch/i386/kernel/i8253.c~a arch/i386/kernel/i8253.c
@@ -3,11 +3,11 @@
  *
  */
 #include <linux/clockchips.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
+#include <linux/spinlock.h>
 #include <linux/jiffies.h>
+#include <linux/sysdev.h>
 #include <linux/module.h>
-#include <linux/spinlock.h>
+#include <linux/init.h>
 
 #include <asm/smp.h>
 #include <asm/delay.h>
@@ -40,27 +40,26 @@ static void init_pit_timer(enum clock_ev
 	case CLOCK_EVT_MODE_PERIODIC:
 		/* binary, mode 2, LSB/MSB, ch 0 */
 		outb_p(0x34, PIT_MODE);
+		udelay(10);
 		outb_p(LATCH & 0xff , PIT_CH0);	/* LSB */
+		udelay(10);
 		outb(LATCH >> 8 , PIT_CH0);	/* MSB */
 		break;
 
+	/*
+	 * Avoid unnecessary state transitions, as it confuses
+	 * Geode / Cyrix based boxen.
+	 */
 	case CLOCK_EVT_MODE_SHUTDOWN:
+		if (evt->mode == CLOCK_EVT_MODE_UNUSED)
+			break;
 	case CLOCK_EVT_MODE_UNUSED:
-		if (evt->mode == CLOCK_EVT_MODE_PERIODIC ||
-		    evt->mode == CLOCK_EVT_MODE_ONESHOT) {
-			outb_p(0x30, PIT_MODE);
-			outb_p(0, PIT_CH0);
-			outb_p(0, PIT_CH0);
-		}
-		break;
-
+		if (evt->mode == CLOCK_EVT_MODE_SHUTDOWN)
+			break;
 	case CLOCK_EVT_MODE_ONESHOT:
 		/* One shot setup */
 		outb_p(0x38, PIT_MODE);
-		break;
-
-	case CLOCK_EVT_MODE_RESUME:
-		/* Nothing to do here */
+		udelay(10);
 		break;
 	}
 	spin_unlock_irqrestore(&i8253_lock, flags);
arch/i386/kernel/vmiclock.c renamed from arch/i386/kernel/vmiclock.c~a +0 -1
diff -puN arch/i386/kernel/vmiclock.c~a arch/i386/kernel/vmiclock.c
@@ -143,7 +143,6 @@ static void vmi_timer_set_mode(enum cloc
 
 	switch (mode) {
 	case CLOCK_EVT_MODE_ONESHOT:
-	case CLOCK_EVT_MODE_RESUME:
 		break;
 	case CLOCK_EVT_MODE_PERIODIC:
 		cycles_per_hz = vmi_timer_ops.get_cycle_frequency();
arch/i386/xen/time.c renamed from arch/i386/xen/time.c~a +0 -3
diff -puN arch/i386/xen/time.c~a arch/i386/xen/time.c
@@ -412,7 +412,6 @@ static void xen_timerop_set_mode(enum cl
 		break;
 
 	case CLOCK_EVT_MODE_ONESHOT:
-	case CLOCK_EVT_MODE_RESUME:
 		break;
 
 	case CLOCK_EVT_MODE_UNUSED:
@@ -475,8 +474,6 @@ static void xen_vcpuop_set_mode(enum clo
 		    HYPERVISOR_vcpu_op(VCPUOP_stop_periodic_timer, cpu, NULL))
 			BUG();
 		break;
-	case CLOCK_EVT_MODE_RESUME:
-		break;
 	}
 }
 
arch/sh/kernel/timers/timer-tmu.c renamed from arch/sh/kernel/timers/timer-tmu.c~a +0 -1
diff -puN arch/sh/kernel/timers/timer-tmu.c~a arch/sh/kernel/timers/timer-tmu.c
@@ -80,7 +80,6 @@ static void tmu_set_mode(enum clock_even
 		break;
 	case CLOCK_EVT_MODE_UNUSED:
 	case CLOCK_EVT_MODE_SHUTDOWN:
-	case CLOCK_EVT_MODE_RESUME:
 		break;
 	}
 }
arch/sparc64/kernel/time.c renamed from arch/sparc64/kernel/time.c~a +0 -1
diff -puN arch/sparc64/kernel/time.c~a arch/sparc64/kernel/time.c
@@ -882,7 +882,6 @@ static void sparc64_timer_setup(enum clo
 {
 	switch (mode) {
 	case CLOCK_EVT_MODE_ONESHOT:
-	case CLOCK_EVT_MODE_RESUME:
 		break;
 
 	case CLOCK_EVT_MODE_SHUTDOWN:
drivers/lguest/lguest.c renamed from drivers/lguest/lguest.c~a +0 -2
diff -puN drivers/lguest/lguest.c~a drivers/lguest/lguest.c
@@ -730,8 +730,6 @@ static void lguest_clockevent_set_mode(e
 		break;
 	case CLOCK_EVT_MODE_PERIODIC:
 		BUG();
-	case CLOCK_EVT_MODE_RESUME:
-		break;
 	}
 }
 
include/linux/clockchips.h renamed from include/linux/clockchips.h~a +0 -1
diff -puN include/linux/clockchips.h~a include/linux/clockchips.h
@@ -23,7 +23,6 @@ enum clock_event_mode {
 	CLOCK_EVT_MODE_SHUTDOWN,
 	CLOCK_EVT_MODE_PERIODIC,
 	CLOCK_EVT_MODE_ONESHOT,
-	CLOCK_EVT_MODE_RESUME,
 };
 
 /* Clock event notification values */
kernel/time/tick-broadcast.c renamed from kernel/time/tick-broadcast.c~a +2 -4
diff -puN kernel/time/tick-broadcast.c~a kernel/time/tick-broadcast.c
@@ -55,7 +55,7 @@ cpumask_t *tick_get_broadcast_mask(void)
  */
 static void tick_broadcast_start_periodic(struct clock_event_device *bc)
 {
-	if (bc)
+	if (bc && bc->mode == CLOCK_EVT_MODE_SHUTDOWN)
 		tick_setup_periodic(bc, 1);
 }
 
@@ -316,7 +316,7 @@ void tick_suspend_broadcast(void)
 	spin_lock_irqsave(&tick_broadcast_lock, flags);
 
 	bc = tick_broadcast_device.evtdev;
-	if (bc)
+	if (bc && tick_broadcast_device.mode == TICKDEV_MODE_PERIODIC)
 		clockevents_set_mode(bc, CLOCK_EVT_MODE_SHUTDOWN);
 
 	spin_unlock_irqrestore(&tick_broadcast_lock, flags);
@@ -333,8 +333,6 @@ int tick_resume_broadcast(void)
 	bc = tick_broadcast_device.evtdev;
 
 	if (bc) {
-		clockevents_set_mode(bc, CLOCK_EVT_MODE_RESUME);
-
 		switch (tick_broadcast_device.mode) {
 		case TICKDEV_MODE_PERIODIC:
 			if(!cpus_empty(tick_broadcast_mask))
kernel/time/tick-common.c renamed from kernel/time/tick-common.c~a +6 -10
diff -puN kernel/time/tick-common.c~a kernel/time/tick-common.c
@@ -318,17 +318,12 @@ static void tick_resume(void)
 {
 	struct tick_device *td = &__get_cpu_var(tick_cpu_device);
 	unsigned long flags;
-	int broadcast = tick_resume_broadcast();
 
 	spin_lock_irqsave(&tick_device_lock, flags);
-	clockevents_set_mode(td->evtdev, CLOCK_EVT_MODE_RESUME);
-
-	if (!broadcast) {
-		if (td->mode == TICKDEV_MODE_PERIODIC)
-			tick_setup_periodic(td->evtdev, 0);
-		else
-			tick_resume_oneshot();
-	}
+	if (td->mode == TICKDEV_MODE_PERIODIC)
+		tick_setup_periodic(td->evtdev, 0);
+	else
+		tick_resume_oneshot();
 	spin_unlock_irqrestore(&tick_device_lock, flags);
 }
 
@@ -365,7 +360,8 @@ static int tick_notify(struct notifier_b
 		break;
 
 	case CLOCK_EVT_NOTIFY_RESUME:
-		tick_resume();
+		if (!tick_resume_broadcast())
+			tick_resume();
 		break;
 
 	default:

_

#2 lore raw
Quoted 15 lines · Andrew Morton
Quoted 6 lines

clockevents: fix resume logic

We need to make sure, that the clockevent devices are resumed, before the tick is resumed. The current resume logic does not guarantee this.

Quoted 8 lines

Add CLOCK_EVT_MODE_RESUME and call the set mode functions of the clock event devices before resuming the tick / oneshot functionality.

Fixup the existing users.

Thanks to Nigel Cunningham for tracking down a long standing thinko, which affected the jinxed VAIO.

Quoted 5 lines

This patch broke the jinxed vaio.

Which is a bit odd, considering that I must have tested it at the time. But I bisected it right down to this commit, and the below revert patch fixed it up.

I just looked up, that you confirmed earlier that the patch does _not_ break the VAIO.

I think, that some other suspend/resume/ACPI or whatever change went in before this patch got merged into Linus tree. Sigh. That's pretty hard to find out.

tglx

#3 lore raw
Quoted 22 lines · On Tue, 11 Sep 2007 08:37:16 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 15 lines · Andrew Morton
Quoted 6 lines

clockevents: fix resume logic

We need to make sure, that the clockevent devices are resumed, before the tick is resumed. The current resume logic does not guarantee this.

Quoted 8 lines

Add CLOCK_EVT_MODE_RESUME and call the set mode functions of the clock event devices before resuming the tick / oneshot functionality.

Fixup the existing users.

Thanks to Nigel Cunningham for tracking down a long standing thinko, which affected the jinxed VAIO.

Quoted 5 lines

This patch broke the jinxed vaio.

Which is a bit odd, considering that I must have tested it at the time. But I bisected it right down to this commit, and the below revert patch fixed it up.

Quoted 7 lines

I just looked up, that you confirmed earlier that the patch does _not_ break the VAIO.

I think, that some other suspend/resume/ACPI or whatever change went in before this patch got merged into Linus tree. Sigh. That's pretty hard to find out.

I tried just 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 on top of 2.6.22 (threw away the lguest and xen hunks). The same problem occurs. 2.6.22 is OK.

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

#4 lore raw
Quoted 9 lines · Andrew Morton
Quoted 9 lines
Quoted 5 lines

This patch broke the jinxed vaio.

Which is a bit odd, considering that I must have tested it at the time. But I bisected it right down to this commit, and the below revert patch fixed it up.

I just looked up, that you confirmed earlier that the patch does _not_ break the VAIO.

Quoted 8 lines
Quoted 4 lines

I think, that some other suspend/resume/ACPI or whatever change went in before this patch got merged into Linus tree. Sigh. That's pretty hard to find out.

I tried just 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 on top of 2.6.22 (threw away the lguest and xen hunks). The same problem occurs. 2.6.22 is OK.

Hmm.

Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

tglx

#5 lore raw
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

It get's even worse. Reverting the patch breaks _my_ VAIO.

/me cries and looks for a job which does not involve computers

tglx

#6 lore raw
Quoted 8 lines · On Tue, 11 Sep 2007 09:34:31 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 7 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

Quoted 8 lines
Quoted 3 lines

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

It get's even worse. Reverting the patch breaks _my_ VAIO.

/me cries and looks for a job which does not involve computers

heh.

I've confirmed that 2.6.22-rc6+thatpatch fails, but 2.6.22-rc6 plus the three hundred patches up to and including that patch doesn't fail.

So there's something between "origin.patch" and "clockevents-fix-resume-logic.patch" in the 2.6.22-rc6-mm1 lineup which fixes thatpatch, once it is applied.

So I'm doing an inverted reverse polish bisection search to find out which patch preemptively fixes clockevents-fix-resume-logic.patch. Try doing that with git, suckers.

gimme twenty minutes...

#7 lore raw
In reply to Thomas Gleixner
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Can you please try the following command line options:

nolapic_timer
nohz=off
highres=off
nohz=off highres=off

tglx

#8 lore raw
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug. It's a quarter megabyte :( Presumably clockevents-fix-resume-logic.patch depended on something which was in that git-acpi.patch but which for some reason didn't get merged. Ho hum, I'll take a look, see if I can spot it.

Quoted 3 lines

Can you please try the following command line options:

nolapic_timer

Works

Quoted 1 line

nohz=off

Fails

Quoted 1 line

highres=off

Fails

Quoted 1 line

nohz=off highres=off

Works

#9 lore raw
Quoted 8 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 6 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 5 lines · Thomas Gleixner
Quoted 5 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too. Maybe we just need to merge the acpi tree?

 46 files changed, 4168 insertions(+), 2224 deletions(-)

I'll let Len do that ;)

Ho hum. I guess I could do a git-bisect on the acpi tree. Getting a bit dull.

#10 lore raw
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 8 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 7 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too. Maybe we just need to merge the acpi tree?

Did you enable CONFIG_CPU_IDLE after you applied git-acpi to rc6 ?

tglx

#11 lore raw
Quoted 17 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 16 lines · Andrew Morton
Quoted 14 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too. Maybe we just need to merge the acpi tree?

Did you enable CONFIG_CPU_IDLE after you applied git-acpi to rc6 ?

Nope.

Anyway, I've had it. git seems a bit weird when it's trying to bisect down to a point where code started to work correctly, rather than finding when it failed (I just reversed "good" and "bad"). The acpi tree is littered with build errors at various points and I just don't know which patch fixed it - git-bisect seems to keep turning up junk.

Ho hum. Maybe tomorrow I'll have a go at turning the acpi tree into a series of flat patches and do it sanely.

#12 lore raw
Quoted 15 lines · Andrew Morton
Quoted 13 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 8 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 7 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too.

Do you mean this one:

http://git.kernel.org/?p=linux/kernel/git/lenb/linux-acpi-2.6.git

#13 lore raw
Quoted 20 lines · On Tue, 11 Sep 2007 13:23:55 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 19 lines · Andrew Morton
Quoted 17 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 16 lines · Andrew Morton
Quoted 14 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too.

Do you mean this one:

http://git.kernel.org/?p=linux/kernel/git/lenb/linux-acpi-2.6.git

Nope.

Here's the algorthm: go to the latest -mm tree and look at the first line of broken-out/git-acpi.patch:

GIT f94aac9883f9b02700270cf286577a9bccf98f47 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git#test

that gives the URL, the branch and the top-level commit. All the -mm git trees have that first line.

Anyway, I was able to extract all the diffs and generate a patch series. The below patch fixes current mainline on the Vaio.

It doesn't compile (natch), so there's a second hackpatch at the end which I used to test it.

commit ef5f15a8b79123a047285ec2e3899108661df779

Author: Venkatesh Pallipadi <[email protected]>
Date:   Thu Feb 22 13:54:03 2007 -0800

cpuidle take2: Hookup ACPI C-states driver with cpuidle

Hookup ACPI C-states onto generic cpuidle infrastructure.

drivers/acpi/procesor_idle.c is now a ACPI C-states driver that registers as a driver in cpuidle infrastructure and the policy part is removed from drivers/acpi/processor_idle.c. We use governor in cpuidle instead.

drivers/acpi/processor_core.c +3 -0
index f7de02a..7352254 100644
@@ -44,6 +44,7 @@ #include <linux/proc_fs.h>
 #include <linux/seq_file.h>
 #include <linux/dmi.h>
 #include <linux/moduleparam.h>
+#include <linux/cpuidle.h>
 
 #include <asm/io.h>
 #include <asm/system.h>
@@ -1022,11 +1023,13 @@ #endif
 
 	acpi_processor_ppc_init();
 
+	cpuidle_register_driver(&acpi_idle_driver);
 	return 0;
 }
 
 static void __exit acpi_processor_exit(void)
 {
+	cpuidle_unregister_driver(&acpi_idle_driver);
 
 	acpi_processor_ppc_exit();
 
drivers/acpi/processor_idle.c +332 -537

962 lines not on this page · raw message

include/acpi/processor.h +2 -0
index b4b0ffd..dcd6aca 100644
@@ -161,6 +161,7 @@ struct acpi_processor_flags {
 	u8 bm_check:1;
 	u8 has_cst:1;
 	u8 power_setup_done:1;
+	u8 bm_rld_set:1;
 };
 
 struct acpi_processor {
@@ -279,6 +280,7 @@ int acpi_processor_power_init(struct acp
 int acpi_processor_cst_has_changed(struct acpi_processor *pr);
 int acpi_processor_power_exit(struct acpi_processor *pr,
 			      struct acpi_device *device);
+extern struct cpuidle_driver acpi_idle_driver;
 
 /* in processor_thermal.c */
 int acpi_processor_get_limit_info(struct acpi_processor *pr);

hackpatch:

 acpi/processor.h              |    0 
 drivers/acpi/processor_core.c |    4 ++--
 drivers/acpi/processor_idle.c |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/acpi/processor_core.c renamed from drivers/acpi/processor_core.c~ef5f15a8b79123a047285ec2e3899108661df779-fix +2 -2
diff -puN drivers/acpi/processor_core.c~ef5f15a8b79123a047285ec2e3899108661df779-fix drivers/acpi/processor_core.c
@@ -1039,13 +1039,13 @@ static int __init acpi_processor_init(vo
 
 	acpi_processor_ppc_init();
 
-	cpuidle_register_driver(&acpi_idle_driver);
+//	cpuidle_register_driver(&acpi_idle_driver);
 	return 0;
 }
 
 static void __exit acpi_processor_exit(void)
 {
-	cpuidle_unregister_driver(&acpi_idle_driver);
+//	cpuidle_unregister_driver(&acpi_idle_driver);
 
 	acpi_processor_ppc_exit();
 
drivers/acpi/processor_idle.c renamed from drivers/acpi/processor_idle.c~ef5f15a8b79123a047285ec2e3899108661df779-fix +1 -1
diff -puN drivers/acpi/processor_idle.c~ef5f15a8b79123a047285ec2e3899108661df779-fix drivers/acpi/processor_idle.c
@@ -633,7 +633,7 @@ int acpi_processor_cst_has_changed(struc
 		return -ENODEV;
 
 	acpi_processor_get_power_info(pr);
-	return cpuidle_force_redetect(&per_cpu(cpuidle_devices, pr->id));
+	return 0;
 }
 
 /* proc interface */

diff -puN include/acpi/processor.h~ef5f15a8b79123a047285ec2e3899108661df779-fix include/acpi/processor.h _

#14 lore raw
Quoted 37 lines · Andrew Morton
Quoted 20 lines · On Tue, 11 Sep 2007 13:23:55 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 19 lines · Andrew Morton
Quoted 17 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 16 lines · Andrew Morton
Quoted 14 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too.

Do you mean this one:

http://git.kernel.org/?p=linux/kernel/git/lenb/linux-acpi-2.6.git

Nope.

Here's the algorthm: go to the latest -mm tree and look at the first line of broken-out/git-acpi.patch:

GIT f94aac9883f9b02700270cf286577a9bccf98f47 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git#test

Quoted 7 lines

that gives the URL, the branch and the top-level commit. All the -mm git trees have that first line.

Anyway, I was able to extract all the diffs and generate a patch series. The below patch fixes current mainline on the Vaio.

It evidently assumes cpuidle to be present, which is not in the mainline.

It seems to me that the total effect of this one and the hackpatch is that the C states are not handled any more.

#15 lore raw
Quoted 43 lines · On Tue, 11 Sep 2007 14:09:12 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 42 lines · Andrew Morton
Quoted 20 lines · On Tue, 11 Sep 2007 13:23:55 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 19 lines · Andrew Morton
Quoted 17 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 16 lines · Andrew Morton
Quoted 14 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too.

Do you mean this one:

http://git.kernel.org/?p=linux/kernel/git/lenb/linux-acpi-2.6.git

Nope.

Here's the algorthm: go to the latest -mm tree and look at the first line of broken-out/git-acpi.patch:

GIT f94aac9883f9b02700270cf286577a9bccf98f47 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git#test

that gives the URL, the branch and the top-level commit. All the -mm git trees have that first line.

Quoted 4 lines
Quoted 2 lines

Anyway, I was able to extract all the diffs and generate a patch series. The below patch fixes current mainline on the Vaio.

It evidently assumes cpuidle to be present, which is not in the mainline.

Bear in mind that the cpuidle patch fixes resume-from-ram when cpuidle is disabled in config.

Quoted 2 lines

It seems to me that the total effect of this one and the hackpatch is that the C states are not handled any more.

hm.

dmesg without the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-bad.txt
dmesg with the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-good.txt
difference: http://userweb.kernel.org/~akpm/dmesg-diff.txt

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

#16 lore raw
Quoted 8 lines · Andrew Morton
Quoted 1 line

It evidently assumes cpuidle to be present, which is not in the mainline.

Bear in mind that the cpuidle patch fixes resume-from-ram when cpuidle is disabled in config.

Quoted 2 lines

It seems to me that the total effect of this one and the hackpatch is that the C states are not handled any more.

Quoted 8 lines

hm.

dmesg without the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-bad.txt
dmesg with the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-good.txt
difference: http://userweb.kernel.org/~akpm/dmesg-diff.txt

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

The question is whether the system goes into C2 with the patch applied.

Can you please provide the output of /proc/acpi/processor/CPU0/power for both the bad and the good one ?

tglx

#17 lore raw
Quoted 15 lines · On Tue, 11 Sep 2007 20:38:03 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 14 lines · Andrew Morton
Quoted 1 line

It evidently assumes cpuidle to be present, which is not in the mainline.

Bear in mind that the cpuidle patch fixes resume-from-ram when cpuidle is disabled in config.

Quoted 2 lines

It seems to me that the total effect of this one and the hackpatch is that the C states are not handled any more.

hm.

dmesg without the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-bad.txt
dmesg with the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-good.txt
difference: http://userweb.kernel.org/~akpm/dmesg-diff.txt

Quoted 8 lines
Quoted 2 lines

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

The question is whether the system goes into C2 with the patch applied.

Can you please provide the output of /proc/acpi/processor/CPU0/power for both the bad and the good one ?

good:

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C0
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]
    C2:                  type[C2] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]

bad:

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C2
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

#18 lore raw
Quoted 22 lines · Andrew Morton
Quoted 8 lines
Quoted 2 lines

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

The question is whether the system goes into C2 with the patch applied.

Can you please provide the output of /proc/acpi/processor/CPU0/power for both the bad and the good one ?

good:

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C0
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]
    C2:                  type[C2] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]

bad:

Quoted 8 lines

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C2
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

tglx

#19 lore raw
Quoted 9 lines · Thomas Gleixner
Quoted 2 lines
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

Summary of the oddities we are seing:

1.) disabling local apic timer makes the problem go away 2.) disabling nohz and highres makes the problem go away 3.) adding the cpuidle patches from the acpi tree makes the problem go away

The obvious conclusion is, that in all other cases we run into a state, where the local apic timer is not working.

1) we do not use it
2) it is used in periodic mode
3) the cpu does not enter C2 (which turns the lapic timer off on the VAIO)

While 1) and 3) are understandable, the reason why 2) is working is a mystery because the periodic mode is affected by the C state as well.

Andrew, can you please provide the output of /proc/timer_list when you boot the kernel with "nohz=off highres=off", but honestly I do not expect a lot of enlightenment from it.

The Sysrq-Q output from the point where the box is stuck without keystrokes might give us more information.

tglx

#20 lore raw
Quoted 11 lines · On Tue, 11 Sep 2007 23:49:47 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 9 lines · Thomas Gleixner
Quoted 2 lines
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

Quoted 6 lines

Summary of the oddities we are seing:

1.) disabling local apic timer makes the problem go away 2.) disabling nohz and highres makes the problem go away 3.) adding the cpuidle patches from the acpi tree makes the problem go away

Only the fist cpuidle patch, actually. I'd have though this was a big hint?

Quoted 8 lines

The obvious conclusion is, that in all other cases we run into a state, where the local apic timer is not working.

1) we do not use it
2) it is used in periodic mode
3) the cpu does not enter C2 (which turns the lapic timer off on the VAIO)

Quoted 6 lines

While 1) and 3) are understandable, the reason why 2) is working is a mystery because the periodic mode is affected by the C state as well.

Andrew, can you please provide the output of /proc/timer_list when you boot the kernel with "nohz=off highres=off", but honestly I do not expect a lot of enlightenment from it.

Timer List Version: v0.3
HRTIMER_MAX_CLOCK_BASES: 2
now at 91231842576 nsecs

cpu: 0
clock 0:
.index: 0
.resolution: 4000250 nsecs
.get_time: ktime_get_real
.offset: 0 nsecs
active timers:
clock 1:
.index: 1
.resolution: 4000250 nsecs
.get_time: ktime_get
.offset: 0 nsecs
active timers:
#0: <f67d5eb0>, hrtimer_wakeup, S:01, do_nanosleep, cpuspeed/1685

 # expires at 91415336182 nsecs [in 183493606 nsecs]
 #1: <f67d5eb0>, it_real_fn, S:01, do_setitimer, ntpd/2182
 # expires at 91859626754 nsecs [in 627784178 nsecs]
 #2: <f67d5eb0>, hrtimer_wakeup, S:01, do_nanosleep, hald-addon-stor/2321
 # expires at 92102929614 nsecs [in 871087038 nsecs]
 #3: <f67d5eb0>, it_real_fn, S:01, do_setitimer, automount/2115
 # expires at 105148251993 nsecs [in 13916409417 nsecs]
 #4: <f67d5eb0>, hrtimer_wakeup, S:01, do_nanosleep, crond/2228
 # expires at 111446499472 nsecs [in 20214656896 nsecs]
 #5: <f67d5eb0>, it_real_fn, S:01, do_setitimer, syslogd/1886
 # expires at 114439083561 nsecs [in 23207240985 nsecs]
 #6: <f67d5eb0>, hrtimer_wakeup, S:01, do_nanosleep, atd/2280
 # expires at 343127174592 nsecs [in 251895332016 nsecs]
 #7: <f67d5eb0>, it_real_fn, S:01, do_setitimer, sendmail/2917
 # expires at 3659006407026 nsecs [in 3567774564450 nsecs]
 #8: <f67d5eb0>, it_real_fn, S:01, do_setitimer, sendmail/2927
 # expires at 3659252445429 nsecs [in 3568020602853 nsecs]
 #9: <f67d5eb0>, it_real_fn, S:01, do_setitimer, anacron/2272
 # expires at 3941983748056 nsecs [in 3850751905480 nsecs]
  .expires_next   : 9223372036854775807 nsecs
  .hres_active    : 0
  .nr_events      : 0
  .nohz_mode      : 0
  .idle_tick      : 0 nsecs
  .tick_stopped   : 0
  .idle_jiffies   : 0
  .idle_calls     : 0
  .idle_sleeps    : 0
  .idle_entrytime : 0 nsecs
  .idle_sleeptime : 0 nsecs
  .last_jiffies   : 0
  .next_jiffies   : 0
  .idle_expires   : 0 nsecs
jiffies: 4294915081

Tick Device: mode: 0
Clock Event Device: pit

 max_delta_ns:   27461866
 min_delta_ns:   12571
 mult:           5124677
 shift:          32
 mode:           2
 next_event:     9223372036854775807 nsecs

set_next_event: pit_next_event
set_mode: init_pit_timer
event_handler: tick_handle_periodic_broadcast
tick_broadcast_mask: 00000001
tick_broadcast_oneshot_mask: 00000000

Tick Device: mode: 0
Clock Event Device: lapic

 max_delta_ns:   1009158122
 min_delta_ns:   1804
 mult:           35701831
 shift:          32
 mode:           1
 next_event:     0 nsecs

set_next_event: lapic_next_event
set_mode: lapic_timer_setup
event_handler: tick_handle_periodic

#21 lore raw
Quoted 13 lines · Andrew Morton
Quoted 11 lines · On Tue, 11 Sep 2007 23:49:47 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 9 lines · Thomas Gleixner
Quoted 2 lines
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

Quoted 8 lines
Quoted 6 lines

Summary of the oddities we are seing:

1.) disabling local apic timer makes the problem go away 2.) disabling nohz and highres makes the problem go away 3.) adding the cpuidle patches from the acpi tree makes the problem go away

Only the fist cpuidle patch, actually. I'd have though this was a big hint?

Yes, it prevents the cpu to enter C2, which keeps the local APIC alive.

Quoted 23 lines
Quoted 8 lines

The obvious conclusion is, that in all other cases we run into a state, where the local apic timer is not working.

1) we do not use it
2) it is used in periodic mode
3) the cpu does not enter C2 (which turns the lapic timer off on the VAIO)

Quoted 6 lines

While 1) and 3) are understandable, the reason why 2) is working is a mystery because the periodic mode is affected by the C state as well.

Andrew, can you please provide the output of /proc/timer_list when you boot the kernel with "nohz=off highres=off", but honestly I do not expect a lot of enlightenment from it.

Tick Device: mode: 0
Clock Event Device: pit

 max_delta_ns:   27461866
 min_delta_ns:   12571
 mult:           5124677
 shift:          32
 mode:           2
 next_event:     9223372036854775807 nsecs
Quoted 5 lines

set_next_event: pit_next_event
set_mode: init_pit_timer
event_handler: tick_handle_periodic_broadcast
tick_broadcast_mask: 00000001
tick_broadcast_oneshot_mask: 00000000

Ok, I got my brain together and read through the code. The periodic mode switches to broadcast right away when the acpi code discovers that there is a lapic stops C-State available.

Does the test hack below fix the problem for nohz/highres enabled kernels ?

tglx

kernel/time/tick-broadcast.c +2 -0
@@ -382,6 +382,8 @@ static int tick_broadcast_set_event(ktime_t expires, int force)
 
 int tick_resume_broadcast_oneshot(struct clock_event_device *bc)
 {
+	cpu_set(smp_processor_id(), tick_broadcast_oneshot_mask);
+
 	clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
 
 	if(!cpus_empty(tick_broadcast_oneshot_mask))
#22 lore raw
Quoted 8 lines · On Wed, 12 Sep 2007 18:57:42 +0200 Thomas Gleixner <[email protected]> wrote:

Does the test hack below fix the problem for nohz/highres enabled kernels ?

tglx

kernel/time/tick-broadcast.c static int tick_broadcast_set_event(ktime_t expires, int force) · line 389

Quoted 8 lines
kernel/time/tick-broadcast.c +2 -0
@@ -382,6 +382,8 @@ static int tick_broadcast_set_event(ktime_t expires, int force)
 
 int tick_resume_broadcast_oneshot(struct clock_event_device *bc)
 {
+	cpu_set(smp_processor_id(), tick_broadcast_oneshot_mask);
+
 	clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
 
 	if(!cpus_empty(tick_broadcast_oneshot_mask))

yup

#23 lore raw
In reply to Thomas Gleixner
Quoted 32 lines · On Tue, 11 Sep 2007 21:52:01 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 22 lines · Andrew Morton
Quoted 8 lines
Quoted 2 lines

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

The question is whether the system goes into C2 with the patch applied.

Can you please provide the output of /proc/acpi/processor/CPU0/power for both the bad and the good one ?

good:

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C0
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]
    C2:                  type[C2] promotion[--] demotion[--] latency[001] usage[00000000] duration[00000000000000000000]

bad:

Quoted 8 lines

sony:/home/akpm> cat /proc/acpi/processor/CPU0/power

active state:            C2
max_cstate:              C8
bus master activity:     00000000
maximum allowed latency: 8000 usec
states:
    C1:                  type[C1] promotion[C2] demotion[--] latency[001] usage[00000010] duration[00000000000000000000]

*C2: type[C2] promotion[--] demotion[C1] latency[001] usage[00008316] duration[00000000000170717293]

Quoted 7 lines

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

That's a bit tricky because hitting the keyboard is what unsticks things. And the video is black after resume-from-RAM (has always been thus) and we broke netconsole-over-e100-during-resume back in 2.6.21 or thereabouts.

<struggles a bit>

Here: http://userweb.kernel.org/~akpm/x.txt

#24 lore raw

Hi!

Quoted 10 lines
Quoted 7 lines

Ok, here we are. The bad one uses C2 which stops the local apic on the VAIO. I suspect we end up in the suspend/resume with going into C2 without the broadcast active.

Can you try to get the output of SysRq-Q during the "it needs help from keyboard" period ?

That's a bit tricky because hitting the keyboard is what unsticks things. And the video is black after resume-from-RAM (has always been thus) and we

Ok, can we try to fix the video issue for you? That should make the development easier... I assume you tried s2ram from suspend.sf.net, and no combination of switches helped?
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#25 lore raw
Quoted 6 lines · Pavel Machek
Quoted 2 lines

That's a bit tricky because hitting the keyboard is what unsticks things. And the video is black after resume-from-RAM (has always been thus) and we

Ok, can we try to fix the video issue for you? That should make the development easier... I assume you tried s2ram from suspend.sf.net, and no combination of switches helped?

I have the same issue. Blank screen after suspend to ram. Hibernate works.

Do you have a debug patch or something ?

tglx

#26 lore raw
Quoted 4 lines · Thomas Gleixner
Quoted 3 lines · Pavel Machek
Quoted 2 lines

That's a bit tricky because hitting the keyboard is what unsticks things. And the video is black after resume-from-RAM (has always been thus) and we

Quoted 8 lines
Quoted 3 lines

Ok, can we try to fix the video issue for you? That should make the development easier... I assume you tried s2ram from suspend.sf.net, and no combination of switches helped?

I have the same issue. Blank screen after suspend to ram. Hibernate works.

Do you have a debug patch or something ?

IIRC, Andrew said that the X server could restore the graphics after a resume from RAM on his Vaio.

Greetings,
Rafael

#27 lore raw
In reply to Thomas Gleixner
Quoted 10 lines · Pavel Machek
Quoted 6 lines
Quoted 2 lines

That's a bit tricky because hitting the keyboard is what unsticks things. And the video is black after resume-from-RAM (has always been thus) and we

Ok, can we try to fix the video issue for you? That should make the development easier... I assume you tried s2ram from suspend.sf.net, and no combination of switches helped?

I have the same issue. Blank screen after suspend to ram. Hibernate works.

So, you have tried s2ram with all the different options, and none of them can restore the screen? (s2ram is available at suspend.sf.net.) i386 emulator is needed to restore the screen on many machines, that's why we can't do it from kernel :-(.

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#28 lore raw
In reply to Andrew Morton
Quoted 9 lines · On Tue, 11 Sep 2007 11:25:31 -0700 Andrew Morton <[email protected]>…

dmesg without the cpuidle patch:
http://userweb.kernel.org/~akpm/dmesg-bad.txt dmesg with the cpuidle
patch: http://userweb.kernel.org/~akpm/dmesg-good.txt difference:
http://userweb.kernel.org/~akpm/dmesg-diff.txt

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

powertop can show you which C-states are actually used; it could be a theory that in the working case the deeper C-states aren't used (which then means lapic keeps working etc)...
powertop would show this change ....

#29 lore raw
In reply to Andrew Morton
Quoted 56 lines · Andrew Morton
Quoted 43 lines · On Tue, 11 Sep 2007 14:09:12 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 42 lines · Andrew Morton
Quoted 20 lines · On Tue, 11 Sep 2007 13:23:55 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 19 lines · Andrew Morton
Quoted 17 lines · On Tue, 11 Sep 2007 11:16:04 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 16 lines · Andrew Morton
Quoted 14 lines · On Tue, 11 Sep 2007 01:20:05 -0700 Andrew Morton <[email protected]>…
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug.

Len's current tree fixes it too.

Do you mean this one:

http://git.kernel.org/?p=linux/kernel/git/lenb/linux-acpi-2.6.git

Nope.

Here's the algorthm: go to the latest -mm tree and look at the first line of broken-out/git-acpi.patch:

GIT f94aac9883f9b02700270cf286577a9bccf98f47 git+ssh://master.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git#test

that gives the URL, the branch and the top-level commit. All the -mm git trees have that first line.

Quoted 4 lines
Quoted 2 lines

Anyway, I was able to extract all the diffs and generate a patch series. The below patch fixes current mainline on the Vaio.

It evidently assumes cpuidle to be present, which is not in the mainline.

Bear in mind that the cpuidle patch fixes resume-from-ram when cpuidle is disabled in config.

Quoted 2 lines

It seems to me that the total effect of this one and the hackpatch is that the C states are not handled any more.

Quoted 8 lines

hm.

dmesg without the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-bad.txt
dmesg with the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-good.txt
difference: http://userweb.kernel.org/~akpm/dmesg-diff.txt

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

Hm, these things look like they may be related to the suspend/resume problems:

+Marking TSC unstable due to: possible TSC halt in C2.

+Clocksource tsc unstable (delta = -458965792 ns)

#30 lore raw

Review

Quoted 12 lines · On Tue, 11 Sep 2007 21:35:15 +0200 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 6 lines

dmesg without the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-bad.txt
dmesg with the cpuidle patch: http://userweb.kernel.org/~akpm/dmesg-good.txt
difference: http://userweb.kernel.org/~akpm/dmesg-diff.txt

there doesn't seem to be a lot of difference in the time handling, except there are large changes in when things happen in the bootup sequence.

Hm, these things look like they may be related to the suspend/resume problems:

(quoted diff) +2 -0
+Marking TSC unstable due to: possible TSC halt in C2.
 
+Clocksource tsc unstable (delta = -458965792 ns)

sort-of. But look:

akpm:/home/akpm> grep -i tsc dmesg-diff.txt +Marking TSC unstable due to: possible TSC halt in C2. +Clocksource tsc unstable (delta = -458965792 ns) -Marking TSC unstable due to: cpufreq changes. -Clocksource tsc unstable (delta = -150503296 ns)

the good and bad kernels seem to be doing quite similar things with the time management. But they're doing it at quite different places in the boot process.

#31 lore raw
In reply to Andrew Morton
Quoted 30 lines · Andrew Morton
Quoted 11 lines · On Tue, 11 Sep 2007 09:47:20 +0200 Thomas Gleixner <[email protected]> wrote:
Quoted 10 lines · Thomas Gleixner
Quoted 6 lines

I went back to the original patch which I sent to Linus and it matches 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6. So all I can think is that there must have been something else in the tree which I tested which fixed the bug which 18de5bc4c1f1f1fa5e14f354a7603bd6e9d4e3b6 introduced. argh.

Can you think what would cause the symptoms which I described?

It seems that time is not updated. Timer interrupt not active or some other odd thing. I figure out what's going on when I find a box which exposes the problem.

2.6.22-rc6-mm1's git-acpi.patch contains something which fixes this bug. It's a quarter megabyte :( Presumably clockevents-fix-resume-logic.patch depended on something which was in that git-acpi.patch but which for some reason didn't get merged. Ho hum, I'll take a look, see if I can spot it.

Quoted 3 lines

Can you please try the following command line options:

nolapic_timer

Works

Quoted 1 line

nohz=off

Fails

Quoted 7 lines
Quoted 1 line

highres=off

Fails

Quoted 1 line

nohz=off highres=off

Works

Ah. Adding your observation of the missing bit from the acpi stuff, I think this must be related to the broadcast functionality.

Btw, 23-rc6 does not resume the Intel Video card on one of my laptops. Sigh !

tglx