Skip to contentSkip to conversation

regression in 2.6.23-rc8 - power off failed

DISCUSSION Wolfgang Erig 7 participants 18 messages Sep 29, 2007 – Oct 8, 2007

lore
Share & tools
lore
#0 lore raw

Hi,

the latest kernel does not power off my system.

2.6.22 succeeded
2.6.23-rc8 failed

$ git bisect bad  
Bisecting: 0 revisions left to test after this 
[f216cc3748a3a22c2b99390fddcdafa0583791a2] ACPI: suspend: consolidate handling of Sx states.

Which more info is needed?

Wolfgang

#1 lore raw
Quoted 7 lines · Wolfgang Erig

Hi,

the latest kernel does not power off my system.

2.6.22 succeeded
2.6.23-rc8 failed

Quoted 7 lines
$ git bisect bad  
Bisecting: 0 revisions left to test after this 
[f216cc3748a3a22c2b99390fddcdafa0583791a2] ACPI: suspend: consolidate handling of Sx states.
$ git bisect good 
Bisecting: 0 revisions left to test after this 
[626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

-hpa

#2 lore raw

Both are bad.
Two different systems and two different bisections. I sent the last step of each.

Quoted 11 lines · H. Peter Anvin
Quoted 10 lines · Wolfgang Erig

Hi,

the latest kernel does not power off my system.

2.6.22 succeeded
2.6.23-rc8 failed

$ git bisect bad  
Bisecting: 0 revisions left to test after this 
[f216cc3748a3a22c2b99390fddcdafa0583791a2] ACPI: suspend: consolidate handling of Sx states.

This is fixed for me after pulling from
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
Thankyou and sorry for the noise.

Quoted 6 lines
Quoted 4 lines
$ git bisect good 
Bisecting: 0 revisions left to test after this 
[626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

This problem (no power off) persists after pull some minutes ago. Sorry for the confusion.

Wolfgang

#3 lore raw
Quoted 4 lines · Wolfgang Erig

Both are bad.
Two different systems and two different bisections. I sent the last step of each.

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
$ git bisect good 
Bisecting: 0 revisions left to test after this 
[626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

This problem (no power off) persists after pull some minutes ago. Sorry for the confusion.

I believe there must have been something wrong here (possibly inconsistent experiments?) This checkin has *zero code changes* from the previous one (and next one) -- the kernel should have been binarily identical to the previous one. The code introduced in this checkin doesn't even get compiled until two checkins later, 4fd06960f120e02e9abc802a09f9511c400042a5.

-hpa

#4 lore raw
Quoted 14 lines · H. Peter Anvin
Quoted 4 lines · Wolfgang Erig

Both are bad.
Two different systems and two different bisections. I sent the last step of each.

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
$ git bisect good 
Bisecting: 0 revisions left to test after this 
[626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

This problem (no power off) persists after pull some minutes ago. Sorry for the confusion.

Quoted 6 lines

I believe there must have been something wrong here (possibly inconsistent experiments?) This checkin has *zero code changes* from the previous one (and next one) -- the kernel should have been binarily identical to the previous one. The code introduced in this checkin doesn't even get compiled until two checkins later, 4fd06960f120e02e9abc802a09f9511c400042a5.

I have done two bisections simultanously and it was late at night. I start again with a fresh tree and better controlled experiments.

Wolfgang

Acked-by Thomas Gleixner
#5 lore raw
Quoted 17 lines · Wolfgang Erig
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Wolfgang Erig

Both are bad.
Two different systems and two different bisections. I sent the last step of each.

Quoted 2 lines
Quoted 1 line
$ git bisect good Bisecting: 0 revisions left to test after this [626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

This problem (no power off) persists after pull some minutes ago. Sorry for the confusion.

Quoted 6 lines

I believe there must have been something wrong here (possibly inconsistent experiments?) This checkin has *zero code changes* from the previous one (and next one) -- the kernel should have been binarily identical to the previous one. The code introduced in this checkin doesn't even get compiled until two checkins later, 4fd06960f120e02e9abc802a09f9511c400042a5.

Quoted 2 lines

I have done two bisections simultanously and it was late at night. I start again with a fresh tree and better controlled experiments.

If this is an SMP system, then you could just be getting random results, depending upon which CPU is attempting the poweroff.

I have a newish patch in Andrew's tree now to fix SMP poweroff (has been broken forever), reproduced here below in case you missed it.

* * *
We need to disable all CPUs other than the boot CPU (usually 0) before attempting to power-off modern SMP machines. This fixes the hang-on-poweroff issue on my MythTV SMP box, and also on Thomas Gleixner's new toybox.

---

kernel/sys.c renamed from kernel/sys.c.orig +0 -0
@@ -32,6 +32,7 @@

#include <linux/getcpu.h>
#include <linux/task_io_accounting_ops.h>
#include <linux/seccomp.h>
+#include <linux/cpu.h>

#include <linux/compat.h>
#include <linux/syscalls.h>
@@ -878,6 +879,7 @@

	kernel_shutdown_prepare(SYSTEM_POWER_OFF);
	if (pm_power_off_prepare)
		pm_power_off_prepare();
+	disable_nonboot_cpus();
	sysdev_shutdown();
	printk(KERN_EMERG "Power down.\n");
	machine_power_off();
#6 lore raw
Quoted 48 lines · Mark Lord
Quoted 17 lines · Wolfgang Erig
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Wolfgang Erig

Both are bad.
Two different systems and two different bisections. I sent the last step of each.

Quoted 2 lines
Quoted 1 line
$ git bisect good Bisecting: 0 revisions left to test after this [626073132b381684c4983e0d911e9aceb32e2cbc] Assembly header and main routine for new x86 setup code 

OK, so which one is the bad one?

This problem (no power off) persists after pull some minutes ago. Sorry for the confusion.

Quoted 6 lines

I believe there must have been something wrong here (possibly inconsistent experiments?) This checkin has *zero code changes* from the previous one (and next one) -- the kernel should have been binarily identical to the previous one. The code introduced in this checkin doesn't even get compiled until two checkins later, 4fd06960f120e02e9abc802a09f9511c400042a5.

Quoted 2 lines

I have done two bisections simultanously and it was late at night. I start again with a fresh tree and better controlled experiments.

If this is an SMP system, then you could just be getting random results, depending upon which CPU is attempting the poweroff.

I have a newish patch in Andrew's tree now to fix SMP poweroff (has been broken forever), reproduced here below in case you missed it.

* * *
We need to disable all CPUs other than the boot CPU (usually 0) before attempting to power-off modern SMP machines. This fixes the hang-on-poweroff issue on my MythTV SMP box, and also on Thomas Gleixner's new toybox.

---

kernel/sys.c 2007-09-28 15:48:54.000000000 -0400 renamed from kernel/sys.c.orig 2007-09-13 09:49:11.000000000 -0400 +1 -0
@@ -32,6 +32,7 @@
 #include <linux/getcpu.h>
 #include <linux/task_io_accounting_ops.h>
 #include <linux/seccomp.h>
+#include <linux/cpu.h>
 
 #include <linux/compat.h>
 #include <linux/syscalls.h>

kernel/sys.c 2007-09-28 15:48:54.000000000 -0400 line 885

Quoted 7 lines
kernel/sys.c 2007-09-28 15:48:54.000000000 -0400 renamed from kernel/sys.c.orig 2007-09-13 09:49:11.000000000 -0400 +1 -0
@@ -878,6 +879,7 @@
    kernel_shutdown_prepare(SYSTEM_POWER_OFF);
    if (pm_power_off_prepare)
        pm_power_off_prepare();
+    disable_nonboot_cpus();
    sysdev_shutdown();
    printk(KERN_EMERG "Power down.\n");
    machine_power_off();

-static void
-acpi_power_off (void)
-{

-       printk("%s called\n",__FUNCTION__);
-       /* Some SMP machines only can poweroff in boot CPU */
-       set_cpus_allowed(current, cpumask_of_cpu(0));

ACPI in kernel 2.6.12 did disable non-boot cpus too in powe_off. Later only comment was left for some reason... Regards,
Alex.

#7 lore raw
Quoted 9 lines · Alexey Starikovskiy
(quoted diff) +0 -6
-static void
-acpi_power_off (void)
-{
-       printk("%s called\n",__FUNCTION__);
-       /* Some SMP machines only can poweroff in boot CPU */
-       set_cpus_allowed(current, cpumask_of_cpu(0));

ACPI in kernel 2.6.12 did disable non-boot cpus too in powe_off. Later only comment was left for some reason...

Am I midreading that code, or does it really assume that the boot cpu is always zero? Or just that zero will be able to do the power off?

In any case I have had an SMP machine which did not have a CPU zero, and it was discussed here, I believe. Wonder what happens if you set affinity to a CPU you don't have...

Signature · 3 lines
Bill Davidsen <[email protected]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot
#8 lore raw
Quoted 14 lines · Bill Davidsen
Quoted 9 lines · Alexey Starikovskiy
(quoted diff) +0 -6
-static void
-acpi_power_off (void)
-{
-       printk("%s called\n",__FUNCTION__);
-       /* Some SMP machines only can poweroff in boot CPU */
-       set_cpus_allowed(current, cpumask_of_cpu(0));

ACPI in kernel 2.6.12 did disable non-boot cpus too in powe_off. Later only comment was left for some reason...

Am I midreading that code, or does it really assume that the boot cpu is always zero? Or just that zero will be able to do the power off?

Quoted 3 lines

In any case I have had an SMP machine which did not have a CPU zero, and it was discussed here, I believe. Wonder what happens if you set affinity to a CPU you don't have...

Good question, but it also caused other problems to appear, IIRC.

IMHO, it's better to call disable_nonboot_cpus() in an appropriate place anyway.

Greetings,
Rafael

#9 lore raw
Quoted 16 lines · Rafael J. Wysocki
Quoted 13 lines · Bill Davidsen
Quoted 9 lines · Alexey Starikovskiy
(quoted diff) +0 -6
-static void
-acpi_power_off (void)
-{
-       printk("%s called\n",__FUNCTION__);
-       /* Some SMP machines only can poweroff in boot CPU */
-       set_cpus_allowed(current, cpumask_of_cpu(0));

ACPI in kernel 2.6.12 did disable non-boot cpus too in powe_off. Later only comment was left for some reason...

Am I midreading that code, or does it really assume that the boot cpu is always zero? Or just that zero will be able to do the power off?

Quoted 1 line

In any case I have had an SMP machine which did not have a CPU zero, and it was discussed here, I believe. Wonder what happens if you set affinity to a CPU you don't have...

Quoted 7 lines

Good question, but it also caused other problems to appear, IIRC.

IMHO, it's better to call disable_nonboot_cpus() in an appropriate place anyway.

Greetings,
Rafael

Ok, here is commit which removed the code in question from acpi_power_off:

commit 6660316cb7a1a2c59a73a52870490c0f782f45c1

Author: Eric W. Biederman <[email protected]>
Date:   Tue Jul 26 12:16:00 2005 -0600

[PATCH] acpi_power_off: Don't switch to the boot cpu

machine_power_off on i386 and x86_64 now switch to the boot cpu out of paranoia and because the MP Specification indicates it is a good idea on reboot, so for those architectures it is a noop. I can't see anything in the acpi spec that requires you to be on the boot cpu to power off the system, so this should not be an issue for ia64. In addition ia64 has the altix a massive multi-node system where switching to the boot cpu sounds insane as we may hot removed the boot cpu.

Regards,
Alex.

#10 lore raw
Quoted 44 lines · Alexey Starikovskiy
Quoted 19 lines · Rafael J. Wysocki
Quoted 14 lines · Bill Davidsen
Quoted 9 lines · Alexey Starikovskiy
(quoted diff) +0 -6
-static void
-acpi_power_off (void)
-{
-       printk("%s called\n",__FUNCTION__);
-       /* Some SMP machines only can poweroff in boot CPU */
-       set_cpus_allowed(current, cpumask_of_cpu(0));

ACPI in kernel 2.6.12 did disable non-boot cpus too in powe_off. Later only comment was left for some reason...

Am I midreading that code, or does it really assume that the boot cpu is always zero? Or just that zero will be able to do the power off?

Quoted 3 lines

In any case I have had an SMP machine which did not have a CPU zero, and it was discussed here, I believe. Wonder what happens if you set affinity to a CPU you don't have...

Quoted 7 lines

Good question, but it also caused other problems to appear, IIRC.

IMHO, it's better to call disable_nonboot_cpus() in an appropriate place anyway.

Greetings,
Rafael

Ok, here is commit which removed the code in question from acpi_power_off:

commit 6660316cb7a1a2c59a73a52870490c0f782f45c1

Author: Eric W. Biederman <[email protected]>
Date:   Tue Jul 26 12:16:00 2005 -0600

[PATCH] acpi_power_off: Don't switch to the boot cpu

machine_power_off on i386 and x86_64 now switch to the boot cpu out of paranoia and because the MP Specification indicates it is a good idea on reboot, so for those architectures it is a noop. I can't see anything in the acpi spec that requires you to be on the boot cpu to power off the system, so this should not be an issue for ia64. In addition ia64 has the altix a massive multi-node system where switching to the boot cpu sounds insane as we may hot removed the boot cpu.

Quoted 2 lines

I see. :-)

Anyway, I think we should atually go UP before executing sysdev_shutdown().

How we are going to do that is another matter.

Greetings,
Rafael

#11 lore raw
In reply to Wolfgang Erig

Hi Peter,

Quoted 2 lines · Wolfgang Erig

I start again with a fresh tree and better controlled experiments.

now the result of bisection seems to be consistent.

The last good commit is
f2d98ae63dc64dedb00499289e13a50677f771f9 Linker script for the new x86 setup code

The first bad commit (no power off) is 4fd06960f120e02e9abc802a09f9511c400042a5 Use the new x86 setup code for i386

Now I try the things written in
http://marc.info/[email protected]

Wolfgang

#12 lore raw

Hi Peter,

Quoted 3 lines · Wolfgang Erig

Now I try the things written in
http://marc.info/[email protected]

I have dumped a memory region which is my understanding what you want to see. The difference between the good and the bad case is only the patch
"4fd06960f120e02e9abc802a09f9511c400042a5 Use the new x86 setup code for i386"

I modified arch/i386/kernel/setup.c and dumped 4K of /dev/kmem at the address of boot_params.

Good case:

c0457340 > 00 08 fc ff 00 00 03 50  8c c8 03 00 8e c0 19 01 < .......P........
c0457350 > 10 00 7c fb fc be 31 00  ac 20 c0 74 09 b4 0e bb < ..|...1.. .t....
c0457360 > 07 00 cd 10 eb f2 31 c0  cd 16 cd 19 ea f0 ff 00 < ......1.........

c0457370 > f0 < .
c0457371 "Direct booting "
c0457380 > 02 01 00 f0 ff 96 00 00 00 f0 40 00 03 00 ff ff < ..........@.....
c0457390 > ff ff ff ff < ....
c0457394 "nger supported."
c04573a3 > 0d 0a < ..
c04573a5 "Please use a boot loader prp4"
c04573c2 > 0f 00 00 00 00 00 08 00 00 00 70 34 3f < ..........p4?
c04573cf 11 * 00
c04573e0 > 08 00 fc 01 00 74 00 00 00 00 < .....t....
c04573ea " key to reboot . . ."
c04573fe > 0d 0a < ..
c0457400 121 * 00
c0457521 > fc 01 00 00 00 09 00 05 00 00 00 00 00 00 00 00 < ................ c0457531 > 0e 01 00 f4 a4 01 00 00 00 00 0f 02 08 55 aa eb < .............U.. c0457541 ":HdrS"

c0457546 > 06 02 00 00 00 00 00 10  7f 11 71 81 00 80 00 00 < ..........q.....
c0457556 > 10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 8e < ................
c0457566 > 00 00 00 90 09 00 ff ff  ff 1f 00 00 10 00 00 00 < ................
c0457576 > 00 00 ff 07 00 00 e8 c1  0c 90                   < ..........
c0457580    99 * 00
c0457619 > fc 09 00 00 00 00 00 01  00 00 00 00 fc 09 00 00 < ................
c0457629 > 00 00 00 00 04 00 00 00  00 00 00 02 00 00 00 3a < ...............:
c0457639 > 12 0f 00 00 00 00 00 c6  ed 00 00 00 00 00 00 02 < ................
c0457649 > 00 00 00 00 00 10 00 00  00 00 00 00 00 f0 07 00 < ................
c0457659 > 00 00 00 01 00 00 00 3a  12 ff ff 00 00 00 00 c6 < .......:........

c0457669 > ed 00 00 00 00 00 00 02 < ........
c0457671 bcf * 00
c0458240 > b8 00 15 b2 81 cd 13 8c c8 8e d8 81 3e a5 1b 55 < ............>..U
c0458250 > aa 75 4c 81 3e a7 1b < .uL.>..
c0458257 "ZZuD"
c045825b > eb 40 ac 20 c0 74 05 e8 08 00 eb f6 c3 e8 00 00 < .@. .t.......... c045826b > b0 20 50 51 bb 07 00 b9 01 00 b4 0e cd 10 59 58 < . PQ..........YX
c045827b > c3 b0 07 eb ed < .....
c0458280 "No setup signature found ..."

c045829c > 00 eb 4f 8c c8 83 e8 20  8e d8 30 ff 8a 1e f1 01 < ..O.... ..0.....
c04582ac > 83 eb 04 c1 e3 08 89 d9  c1 eb 03 81 c3 00 10 2e < ................
c04582bc > 89 1e 0c 00 bf 00 08 29  f6 0e 07 b8 00 10 8e d8 < .......)........
c04582cc > f3 a5 8c c8 8e d8 81 3e  a5 1b 55 aa 75 0a 81 3e < .......>..U.u..>
c04582dc > a7 1b 5a 5a 75 02 eb 0a  8d 36 40 0d e8 72 ff f4 < [email protected]..
c04582ec > eb fd 8c c8 83 e8 20 8e  d8 2e f6 06 11 00 01 74 < ...... ........t
c04582fc > 2e 2e 80 3e 10 00 00 75  26 0e 1f 8d 36 d0 0d e8 < ...>...u&...6...

c045830c > 4f ff eb db < O...
c0458310 "Wrong loader, giving up..."
c045832a > 00 e8 36 00 66 85 c0 74 61 8c c8 8e d8 8d 36 00 < ..6.f..ta.....6.
c045833a > 0e e8 1f ff eb fe < ......

Bad case:
c0457340 > 00 08 00 00 00 00 03 50 00 00 03 00 00 00 19 01 < .......P........
c0457350 > 10 < .
c0457351 4f * 00
c04573a0 > 80 86 00 00 00 00 00 00 00 00 00 00 < ............
c04573ac "CISG"
c04573b0 30 * 00
c04573e0 > 08 00 fc 01 00 74 < .....t
c04573e6 13e * 00
c0457524 > e0 29 09 00 05 00 00 00 00 00 00 00 00 13 01 00 < .)..............
c0457534 > fa a4 01 00 00 00 ff ff 02 08 55 aa eb < ..........U..
c0457541 ":HdrS"

c0457546 > 06 02 00 00 00 00 00 10  3c 23 71 81 00 80 00 00 < ........<#q.....
c0457556 > 10 00 00 00 00 00 00 00  00 00 00 00 00 00 00 8e < ................
c0457566 > 00 00 00 90 09 00 ff ff  ff 1f 00 00 10          < .............
c0457573    a6 * 00
c0457619 > fc 09 00 00 00 00 00 01  00 00 00 00 fc 09 00 00 < ................
c0457629 > 00 00 00 00 04 00 00 00  00 00 00 02 00 00 00 3a < ...............:
c0457639 > 12 0f 00 00 00 00 00 c6  ed 00 00 00 00 00 00 02 < ................
c0457649 > 00 00 00 00 00 10 00 00  00 00 00 00 00 f0 07 00 < ................
c0457659 > 00 00 00 01 00 00 00 3a  12 ff ff 00 00 00 00 c6 < .......:........

c0457669 > ed 00 00 00 00 00 00 02 < ........
c0457671 ccf * 00

$ cat /proc/cmdline
root=/dev/hda1

Wolfgang

#13 lore raw
In reply to Wolfgang Erig
Quoted 7 lines · Wolfgang Erig

Hi Peter,

Quoted 1 line · Wolfgang Erig

I start again with a fresh tree and better controlled experiments.

now the result of bisection seems to be consistent.

Quoted 9 lines

The last good commit is
f2d98ae63dc64dedb00499289e13a50677f771f9 Linker script for the new x86 setup code

The first bad commit (no power off) is 4fd06960f120e02e9abc802a09f9511c400042a5 Use the new x86 setup code for i386

Now I try the things written in
http://marc.info/[email protected]

OK, that makes more sense. I'll look at this on Monday.

-hpa

#14 lore raw

Re: regression in 2.6.23-rc8 - power off failed: solved

Quoted 3 lines · H. Peter Anvin

Does your .config file have CONFIG_APM=y? Otherwise, that's your problem right there (since your old laptop doesn't appear to have ACPI.)

no CONFIG_APM and yes, this is my problem.
A very hard way to figure this out,
and too much noise. Sorry for this.

It is not the first time, I had to change my .config to maintain a feature of the kernel after an upgrade. I think, you are aware of this an I have no suggestion for improvement. Now this old laptop is running perfectly again with the latest kernel.

Thankyou for this,
Wolfgang

#15 lore raw
In reply to Wolfgang Erig
Quoted 1 line · Wolfgang Erig

the latest kernel does not power off my system.

This is a known regression in rc8. See this mail and thread for details:
http://lkml.org/lkml/2007/9/25/239

The issue has already been fixed in Linus' git tree.
Please try again with that, or apply the patches included in the posts referenced in the message linked above.

Cheers,
Frans Pop

#16 lore raw

My apologies for the two bogus addresses in the "To" of my previous message. Script error, won't happen again.

Frans Pop

#17 lore raw
In reply to Wolfgang Erig

Hi,
This is known issue.
Please try latest rc8-git2, it should contain the fix.

Regards,
Alex.

Quoted 12 lines · Wolfgang Erig

Hi,

the latest kernel does not power off my system.

2.6.22 succeeded
2.6.23-rc8 failed

$ git bisect bad  Bisecting: 0 revisions left to test after this [f216cc3748a3a22c2b99390fddcdafa0583791a2] ACPI: suspend: consolidate handling of Sx states.

Which more info is needed?

Wolfgang