Use correct x86 reboot vector
PATCH 1/1 Use correct x86 reboot vector
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
See the "DPMI on AMD64" and "Warm reboot for x86-64 linux" threads on http://www.x86-64.org/mailing_lists/list?listname=discuss&listnum=0
for more details.
This patch fixes the 2.5.69 i386 reboot code to use this too.
arch/i386/kernel/reboot.c renamed from arch/i386/kernel/reboot.c-o +1 -1
@@ -123,7 +123,7 @@ };static unsigned char jump_to_bios [] = { -0xea ,0x00 ,0x00 ,0xff ,0xff /* ljmp $0xffff,$0x0000 */ +0xea ,0xf0 ,0xff ,0x00 ,0xf0 /* ljmp $0xf000:0xfff0 */ };/*
Quoted 4 lines · Andi Kleen
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
Could this bug, by any chance, cause a system to shutdown instead of rebooting? This is what happens to me at the moment but not each and every time.
Signature · 6 lines
Martin's distress was in contrast to the bitter satisfaction of some of his fellow marines as they surveyed the scene. "The Iraqis are sick people and we are the chemotherapy," said Corporal Ryan Dupre. "I am starting to hate this country. Wait till I get hold of a friggin' Iraqi. No, I won't get hold of one. I'll just kill him." - http://www.informationclearinghouse.info/article2479.htm
Followup to: <[email protected]> By author: CaT <[email protected]> In newsgroup: linux.dev.kernel
Quoted 11 lines · Andi Kleen
Quoted 4 lines
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
Could this bug, by any chance, cause a system to shutdown instead of rebooting? This is what happens to me at the moment but not each and every time.
No, it wouldn't.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
Quoted 9 lines · CaT
Quoted 4 lines · Andi Kleen
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
Could this bug, by any chance, cause a system to shutdown instead of rebooting? This is what happens to me at the moment but not each and every time.
Unlikely. But try it and see 8)
At least some SMP boxes freak if you do a poweroff request on CPU != 0
Quoted 4 lines · Andi Kleen
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
You are right. That's what a 286 does when the RESET signal is asserted.
Which is amazing, because I wrote that ffff:0000 and I was reading from the Phoenix BIOS book at the time. It was long ago but I'm fairly sure I got that address from the book.
I just did some Googling and found that there examples of DOS code fragments using both vectors. Also, the original IBM BIOS (as they say) had a long jump at the vector, which is presumably one of the many de facto ABIs which real mode programmers grew to depend on.
Quoted 3 lines
See the "DPMI on AMD64" and "Warm reboot for x86-64 linux" threads on http://www.x86-64.org/mailing_lists/list?listname=discuss&listnum=0
for more details.
One would hope that AMD64 systems, being a new design and all, offer a documented and reliable method of rebooting.
It should never be necessary to have to write "reboot=..." on the kernel command line to choose which legacy method works on different AMD64 motherboards. Am I too idealistic?
-- Jamie
Quoted 1 line · Alan Cox
At least some SMP boxes freak if you do a poweroff request on CPU != 0
Power-off works on some SMP boxes?
-- Jamie
Quoted 4 lines · Jamie Lokier
Quoted 1 line · Alan Cox
At least some SMP boxes freak if you do a poweroff request on CPU != 0
Power-off works on some SMP boxes?
With ACPI kernels, my Dual PII 333 / Intel 440 LX powers down without pressing the button.
Jos
Quoted 8 lines
Quoted 4 lines · Andi Kleen
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
You are right. That's what a 286 does when the RESET signal is asserted.
Quoted 8 lines
Which is amazing, because I wrote that ffff:0000 and I was reading from the Phoenix BIOS book at the time. It was long ago but I'm
fairly sure I got that address from the book.I just did some Googling and found that there examples of DOS code fragments using both vectors. Also, the original IBM BIOS (as they say) had a long jump at the vector, which is presumably one of the many de facto ABIs which real mode programmers grew to depend on.
This seems to be a difference from 8086/8088 to the 286.
My iAPX 286 Hardware Reference Manual says that the RESET signal initializes CS to 0FF0000H and IP to 0FFF0H, while my iAPX 86,88 User's Manual says that RESET sets CS to 0FFFFh and IP to 0.
~Randy
Quoted 4 lines · Jamie Lokier
I just did some Googling and found that there examples of DOS code fragments using both vectors. Also, the original IBM BIOS (as they say) had a long jump at the vector, which is presumably one of the many de facto ABIs which real mode programmers grew to depend on.
The 16 byte code space is very small, and usually only contains that LONG jump to an usable address space.
When the vector f000:fff0 is used, we can survive BIOSes that use relative jumps with negative offsets or indirect short jumps instead.
When the vector ffff:0000 is used, the code segment effectively contains only 16 bytes (or someone must abuse the 8086 wraparound), can't think of negative offset short jumps there. As the code is read-only in this early stage, (BIOS code is RW after the BIOS copied itself to RAM) self modifying code (which uses absolute addressing) can be excluded too.
Okay... now, as 386 and newer cpus need a far jump to unlock A20-A31, I think it is safe to assume all BIOSes will do a far jump as soon as possible, which means it doesn't matter which vector is used.
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
Jos
Quoted 2 lines · Jos Hulzink
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
I agree, for the simple reason that it is what the chip does on a hardware reset signal.
-- Jamie
Quoted 6 lines · Jamie Lokier
Quoted 2 lines · Jos Hulzink
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
I agree, for the simple reason that it is what the chip does on a hardware reset signal.
Hmzz... this seems indeed true for the 386, that's the only doc I got at hands here. Willing to believe that this is the hardware behaviour of all 386 and newer 32 bit procs.
If this really fixes some issues, I'm eager to see that BIOS code....
Jos
Quoted 6 lines · Jamie Lokier
Quoted 2 lines · Jos Hulzink
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
I agree, for the simple reason that it is what the chip does on a hardware reset signal.
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains 0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address 0xfffffff0.
At least that was true on an original 386. It's something that could easily have changed since.
In other words, you're all wrong. Nyaah, nyaah.
Linus
Quoted 14 lines · Linus Torvalds
Quoted 6 lines · Jamie Lokier
Quoted 2 lines · Jos Hulzink
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
I agree, for the simple reason that it is what the chip does on a hardware reset signal.
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains
0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address0xfffffff0.
Quoted 6 lines
At least that was true on an original 386. It's something that could easily have changed since.
In other words, you're all wrong. Nyaah, nyaah.
Linus
Source: 80386 Programmers Reference Manual, Intel (1986)
EIP is set 0000FFF0H
CS is set F000H
After RESET, lines A31-A20 are FORCED high till a far JMP is done.
So, unfortunately we have to say Linus is right once again. Damn ;-) My conclusion is that we are unable to use the CPU reset as the reference for warm boots, for we can't control A312-A20 in real mode. But as far as I can see, my arguments still hold...
Jos
Jos
Quoted 9 lines · Jos Hulzink
Quoted 8 lines · Linus Torvalds
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains
0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address0xfffffff0.At least that was true on an original 386. It's something that could easily have changed since.
I got my info from an article on the net which says that a 386 does behave as you say, but it is possible for the system designer to arrange that it boots into the 286-compatible vector at physical address 0x000ffff0. It states that the feature is specifically so that system designers don't have to create a "memory hole" (that's as much detail as it gives).
I can't be arsed to look in a real 386 manual though :)
Quoted 11 lines
Source: 80386 Programmers Reference Manual, Intel (1986)
EIP is set 0000FFF0H
CS is set F000HAfter RESET, lines A31-A20 are FORCED high till a far JMP is done.
So, unfortunately we have to say Linus is right once again. Damn ;-) My conclusion is that we are unable to use the CPU reset as the reference for warm boots, for we can't control A312-A20 in real mode. But as far as I can see, my arguments still hold...
You can set up unreal mode but it is quite fiddly.
-- Jamie
Quoted 11 lines · Jamie Lokier
Quoted 9 lines · Jos Hulzink
Quoted 8 lines · Linus Torvalds
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains
0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address0xfffffff0.At least that was true on an original 386. It's something that could easily have changed since.
Quoted 6 lines
I got my info from an article on the net which says that a 386 does behave as you say, but it is possible for the system designer to arrange that it boots into the 286-compatible vector at physical address
0x000ffff0. It states that the feature is specifically so that system designers don't have to create a "memory hole" (that's as much detail as it gives).
Guys, mem[0xfffffff0,...] == mem[0x000ffff0,...] since the hw remaps the bios. Being picky about Intel specs, it should be f000:fff0 though.
- Davide
Quoted 8 lines · Linus Torvalds
Quoted 6 lines · Jamie Lokier
Quoted 2 lines · Jos Hulzink
For the sake of bad behaving BIOSes however, I'd vote for the f000:fff0 vector, unless someone can hand me a paper that says it is wrong.
I agree, for the simple reason that it is what the chip does on a hardware reset signal.
Quoted 8 lines
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains
0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address0xfffffff0.At least that was true on an original 386. It's something that could easily have changed since.
Correct. And no one has changed it since. I use that all of the time.
Quoted 1 line
In other words, you're all wrong. Nyaah, nyaah.
However 0xf000:fff0 is as close as you can get, and since that is usually RAM it can do something different on a reset vs a reboot if it wants to.
Using 0xf000 is just polite as it allows relative jumps.
Eric
Quoted 20 lines · Davide Libenzi
Quoted 11 lines · Jamie Lokier
Quoted 9 lines · Jos Hulzink
Quoted 8 lines · Linus Torvalds
Hmm.. Doesnt' a _real_ hardware reset actually use a magic segment that isn't even really true real mode? I have this memory that the reset value for a i386 has CS=0xf000, but the shadow base register actually contains
0xffff0000. In other words, the CPU actually starts up in "unreal" mode, and will fetch the first instruction from physical address0xfffffff0.At least that was true on an original 386. It's something that could easily have changed since.
Quoted 6 lines
I got my info from an article on the net which says that a 386 does behave as you say, but it is possible for the system designer to arrange that it boots into the 286-compatible vector at physical address
0x000ffff0. It states that the feature is specifically so that system designers don't have to create a "memory hole" (that's as much detail as it gives).
Quoted 2 lines
Guys, mem[0xfffffff0,...] == mem[0x000ffff0,...] since the hw remaps the bios. Being picky about Intel specs, it should be f000:fff0 though.
The remapping is quite common but it usually happens that after bootup:
0xf0000-0xfffff is shadowed RAM. While 0xffff0000-0xffffffff still points to the rom chip.
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
Eric
Quoted 7 lines · Alan Cox
Quoted 6 lines · CaT
Quoted 4 lines · Andi Kleen
Extensive discussion by various experts on the [email protected] mailing list concluded that the correct vector to restart an 286+ CPU is f000:fff0, not ffff:0000. Both seem to work on current systems, but the first is correct.
Quoted 7 lines
Quoted 3 lines
Could this bug, by any chance, cause a system to shutdown instead of rebooting? This is what happens to me at the moment but not each and every time.
Unlikely. But try it and see 8)
At least some SMP boxes freak if you do a poweroff request on CPU != 0
As per the MP spec. The system should reboot on the bootstrap cpu. smp_processor_id() == 0 on x86. apicid??
I have a patch for this as part my kexec stuff as the kernel freaks when it doesn't start up on the bootstrap cpu as well. I am busily cleaning it up so it works in interrupt context as well.
Alan if you want it holler and I can send it to you as well.
Eric
Quoted 6 lines · Eric W. Biederman
The remapping is quite common but it usually happens that after bootup:
0xf0000-0xfffff is shadowed RAM. While 0xffff0000-0xffffffff still points to the rom chip.Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
Have you ever heard about unreal mode ? But I do not think that a reset has to start over there. I do not think that exist hw/sw that expect that reset address to be 0xfffffff0 instead of 0x000ffff0, since they map the same content.
- Davide
Quoted 4 lines · Eric W. Biederman
Quoted 1 line
At least some SMP boxes freak if you do a poweroff request on CPU != 0
As per the MP spec. The system should reboot on the bootstrap cpu.
smp_processor_id()== 0 on x86. apicid??
APM now makes its calls on CPU#0 which was the trigger for these problems
Quoted 3 lines · [email protected] (Eric W. Biederman) writes:
Davide Libenzi <[email protected]> writes:
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
Well, it should be possible to use a trick similar to the BIG REAL or UNREAL mode. Just load CS with a segment that has a base of 0xffff0000 in protected mode and then jump back to real mode. Something like this, completely untested of course, should do it:
.align 4reset_gdt:
.word reset_gdt_end - reset_gdt -1
.long reset_gdt
.word 0/* 16 bit code segment starting at 0xffff0000 */
.word 0xffff, 0x0000
.byte 0xff, 0x9b, 0x00, 0xff#endif
reset_gdt_end:
lgdt %cs:reset_gdt
ljmp $ROM_CODE_SEG, 0xfff0/Christer
Signature · 4 lines
"Just how much can I get away with and still go to heaven?" Freelance consultant specializing in device driver programming for Linux Christer Weinigel <[email protected]> http://www.weinigel.se
Quoted 3 lines · Eric W. Biederman
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
You should be able to do it the same way as you enter unreal mode, ie:
- in protected mode cpl0, crate a segment that has index 0xf000 (ie you need a large GDT for this to work), and has the right attributes (ie base 0xffff0000, 16-bit, etc).
Make sure you reload the other segments with something sanish and be 16-bit clean.
- clear the PE bit, but do _not_ do the long jump to reload the segment that intel says you should do - just do a short jump to 0xfff0.
One problem is that the code segment you create this way will have the right base and size, but it will be non-writeable (no way to create a writable code segment in protected mode), so it will be different in other ways.
And because you'll have to do some of the the setup with that new and inconvenient CS, you'll either have to make the limit be big (and wrap around EIP in order to first execute code that is in low memory), or you'll have to play even more tricks and clear both PE and PG at the same time and just "fall through" to the code at 0xfffffff0.
Sounds like it might work, at least on a few CPU's.
Linus
Quoted 23 lines · Linus Torvalds
Quoted 3 lines · Eric W. Biederman
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
You should be able to do it the same way as you enter unreal mode, ie:
- in protected mode cpl0, crate a segment that has index
0xf000(ie you need a large GDT for this to work), and has the right attributes (ie base0xffff0000, 16-bit, etc).Make sure you reload the other segments with something sanish and be 16-bit clean.
- clear the PE bit, but do _not_ do the long jump to reload the segment that intel says you should do - just do a short jump to
0xfff0.One problem is that the code segment you create this way will have the right base and size, but it will be non-writeable (no way to create a writable code segment in protected mode), so it will be different in other ways.
Quoted 7 lines
And because you'll have to do some of the the setup with that new and inconvenient CS, you'll either have to make the limit be big (and wrap around EIP in order to first execute code that is in low memory), or you'll have to play even more tricks and clear both PE and PG at the same time and just "fall through" to the code at
0xfffffff0.Sounds like it might work, at least on a few CPU's.
There's a missing piece of behavior here that's probably fatal. Namely, the next time the CS descriptor is loaded, even with the same value, the high bits are lost. So, for example, if you're running BIOS out of ROM, decompressing it into the top of 20-bit address space, then long jumping to your uncompressed code, you don't want to find yourself back in ROM.
Perhaps there's a trick that can be played with loading the descriptor into CS and then clearing the descriptor table without flushing, but it sounds rather dubious..
Signature · 1 line
Matt Mackall : http://www.selenic.com : of or relating to the moon
Quoted 8 lines · Alan Cox
Quoted 4 lines · Eric W. Biederman
Quoted 1 line
At least some SMP boxes freak if you do a poweroff request on CPU != 0
As per the MP spec. The system should reboot on the bootstrap cpu.
smp_processor_id()== 0 on x86. apicid??APM now makes its calls on CPU#0 which was the trigger for these problems
I have a couple of issues with the current state of affairs.
1) We should always do this to be safe.
2) Reboot has this issue as well.
3) The way APM does it overrides the kernel command line option, and apm_power_off forces the cpu twice.
4) We have this implemented in 3 different ways in 3 different places.
5) machine_reboot needs this to do this in interrupt context for Sys-req-B and certain cases of panic and that is not currently handled.
On a related note do you know why machine_halt and machine_power_off return? After shutting everything down that seems very much like the wrong thing to do.
Eric
Quoted 14 lines · Linus Torvalds
Quoted 3 lines · Eric W. Biederman
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
You should be able to do it the same way as you enter unreal mode, ie:
- in protected mode cpl0, crate a segment that has index
0xf000(ie you need a large GDT for this to work), and has the right attributes (ie base0xffff0000, 16-bit, etc).Make sure you reload the other segments with something sanish and be 16-bit clean.
Quoted 7 lines
- clear the PE bit, but do _not_ do the long jump to reload the segment that intel says you should do - just do a short jump to
0xfff0.One problem is that the code segment you create this way will have the right base and size, but it will be non-writeable (no way to create a writable code segment in protected mode), so it will be different in other ways.
I suspect the fact it is unwritable won't be a real problem. ROM chips are essentially unwritable. But there will be the behavioral difference between discarding writes and causing an exception.
Quoted 7 lines
And because you'll have to do some of the the setup with that new and inconvenient CS, you'll either have to make the limit be big (and wrap around EIP in order to first execute code that is in low memory), or you'll have to play even more tricks and clear both PE and PG at the same time and just "fall through" to the code at
0xfffffff0.Sounds like it might work, at least on a few CPU's.
I will have to try it one of these times. I keep wondering if I can call a BIOS without trigger a reset line.
At the same time I think this is very much not what we want in the reboot path. As I suspect the existing BIOS if it does anything different will freak out on us. Jumping to a known location where there is ram sounds much safer.
Eric
Quoted 6 lines · Davide Libenzi
Quoted 4 lines · Eric W. Biederman
The remapping is quite common but it usually happens that after bootup:
0xf0000-0xfffff is shadowed RAM. While 0xffff0000-0xffffffff still points to the rom chip.
Quoted 7 lines
Quoted 2 lines
Now if someone could tell me how to do a jump to 0xffff0000:0xfff0 in real mode I would find that very interesting.
Have you ever heard about unreal mode ? But I do not think that a reset has to start over there. I do not think that exist hw/sw that expect that reset address to be
0xfffffff0instead of0x000ffff0, since they map the same content.
There is some software at least that knows the difference. I have seen short jumps in a couple of BIOS's. But a reset is very different from a reboot. As memory must be reinitialized etc. So I think going to 0xffff0000:0xfff0 would be a very bad idea if the intent is to get a reliable reboot.
Eric
Quoted 10 lines · Matt Mackall
There's a missing piece of behavior here that's probably fatal. Namely, the next time the CS descriptor is loaded, even with the same value, the high bits are lost. So, for example, if you're running BIOS out of ROM, decompressing it into the top of 20-bit address space, then long jumping to your uncompressed code, you don't want to find yourself back in ROM.
Perhaps there's a trick that can be played with loading the descriptor into CS and then clearing the descriptor table without flushing, but it sounds rather dubious..
If PE is really disabled that should bit should come for free. And it is why it is so hard to fake this behavior.
Eric
[following up to myself]
Quoted 7 lines · Christer Weinigel
Well, it should be possible to use a trick similar to the BIG REAL or UNREAL mode. Just load CS with a segment that has a base of
0xffff0000in protected mode and then jump back to real mode. Something like this, completely untested of course, should do it:.align 4
reset_gdt:
Quoted 7 lines
.word reset_gdt_end - reset_gdt -1 .long reset_gdt .word 0/* 16 bit code segment starting at
0xffff0000*/.word 0xffff, 0x0000 .byte 0xff, 0x9b, 0x00, 0xff
better add the following too:
move.l %cr0, %eax
and.l $~1, %eax
move.l %eax, %cr0Quoted 4 lines
reset_gdt_end:lgdt %cs:reset_gdt ljmp $ROM_CODE_SEG, 0xfff0
BTW, what does Windows do here? Whatever Windows is using should work with Linux too.
/Christer
Signature · 4 lines
"Just how much can I get away with and still go to heaven?" Freelance consultant specializing in device driver programming for Linux Christer Weinigel <[email protected]> http://www.weinigel.se
Quoted 1 line · On Sun, 11 May 2003 [email protected] wrote:
BTW, what does Windows do here?
I just happen to have the sources under my nose, let me look it up ... :)
- Davide
Followup to: <[email protected]> By author: Matt Mackall <[email protected]> In newsgroup: linux.dev.kernel
Quoted 8 lines
There's a missing piece of behavior here that's probably fatal. Namely, the next time the CS descriptor is loaded, even with the same value, the high bits are lost. So, for example, if you're running BIOS out of ROM, decompressing it into the top of 20-bit address space, then long jumping to your uncompressed code, you don't want to find yourself back in ROM.
Nope, that's *exactly* the desired behaviour.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
PATCH 1/1 always shutdown on the bootstrap processor
In reply to Alan CoxQuoted 8 lines · Alan Cox
Quoted 4 lines · Eric W. Biederman
Quoted 1 line
At least some SMP boxes freak if you do a poweroff request on CPU != 0
As per the MP spec. The system should reboot on the bootstrap cpu.
smp_processor_id()== 0 on x86. apicid??APM now makes its calls on CPU#0 which was the trigger for these problems
We have the APM case, we have the reboot case, we have the specifications that say you should use the bootstrap processor, and I have problems with kexec if we do anything different.
And looking at other architectures this problem also happens on the alpha, and I don't know how many others.
reboot/shutdown/halt/kexec all of these are a slow path so is there any good reason not to unify all of these and always switch back to the bootstrap processor on shutdown? Just to avoid this kinds of problems?
Things I have taken into consideration.
1) Running multiple processors on x86 is intricately tied to apics so they both need to be shutdown together. And apic shutdown needs to happen last because devices potentially need interrupts (at least timer interrupts for timeouts) to shutdown properly. So I cannot use the device mode to perform the shutdown.
2) The reboot case needs to be callable from an irq handler.
3) We may have off lined the bootstrap cpu for some reason.
4) The user may have specified an override cpu to shut down on.
With a net of 2 additional lines of code I am able to do what the current code does today, in one central location, and much more consistency.
For the author the code compiles and runs, and looks obviously correct. So obviously it is perfect and ready to go in the kernel :)
This patch also makes machine_restart, machine_halt, and machine_power_off, all no return functions, so the core kernel does not have to figure out how to deal with a kernel that is half shutdown. Turning off a cpu and then calling flush_tlbs can be entertaining.
Eric
arch/i386/kernel/apic.c | 60 +++++++++++++++++++++ arch/i386/kernel/apm.c | 9 --- arch/i386/kernel/dmi_scan.c | 27 --------- arch/i386/kernel/io_apic.c | 2 arch/i386/kernel/reboot.c | 78 ++++++++++++---------------- arch/i386/kernel/smp.c | 26 --------- include/asm-i386/apic.h | 13 ++++ include/asm-i386/mach-default/mach_reboot.h | 2 include/linux/reboot.h | 7 +- kernel/panic.c | 2 kernel/sys.c | 4 - 11 files changed, 116 insertions, 114 deletions
arch/i386/kernel/apic.c +60 -0
@@ -25,6 +25,7 @@#include <linux/interrupt.h> #include <linux/mc146818rtc.h> #include <linux/kernel_stat.h> +#include <linux/reboot.h> #include <asm/atomic.h> #include <asm/smp.h>
@@ -37,6 +38,7 @@#include <mach_apic.h> #include "io_ports.h" +#include "mach_reboot.h" void __init apic_intr_init (void ) {
@@ -1114,6 +1116,64 @@printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n" ,smp_processor_id (), v , v1);irq_exit (); +} + + +struct stop_apics { + NORET_TYPE void (*rest)(void *info) ATTRIB_NORET; +void *info; +int reboot_cpu_id; +}; + +static void cpu_stop_apics (void *ptr) +{ +struct stop_apics *arg = ptr; +if (smp_processor_id () != arg->reboot_cpu_id) { +local_irq_disable (); +disable_local_APIC (); +stop_this_cpu (); + } +local_irq_disable (); +disable_local_APIC (); +local_irq_enable (); + +#if defined(CONFIG_X86_IO_APIC ) +if (smp_found_config) { +disable_IO_APIC (); + } +#endif +disconnect_bsp_APIC (); + arg->rest (arg->info); +} + +void stop_apics(NORET_TYPE void(*rest)(void *)ATTRIB_NORET, void *info) +{ +/* By resetting the APIC's we disable the nmi watchdog */ +extern int reboot_cpu; +struct stop_apics arg; + +/* The boot cpu is always logical cpu 0 */ + arg.rest = rest; + arg.info = info; + arg.reboot_cpu_id =0 ; + +/* See if there has been give a command line override . +*/ +if ((reboot_cpu !=-1 ) &&cpu_possible (reboot_cpu)) { + arg.reboot_cpu_id = reboot_cpu; + } + +/* Make certain the the cpu I'm rebooting on is online */ +if (!cpu_online (arg.reboot_cpu_id)) { + arg.reboot_cpu_id =smp_processor_id (); + } +/* If we aren't in interrupt context use the scheduler, +* so rest will not be called in an interrupt context either. +*/ +if (!in_interrupt ()) { +set_cpus_allowed (current,1 << arg.reboot_cpu_id); + } +on_each_cpu (cpu_stop_apics, &arg,1 ,0 ); }/*
arch/i386/kernel/apm.c +0 -9
@@ -911,17 +911,8 @@/* * This may be called on an SMP machine. */ -#ifdef CONFIG_SMP -/* Some bioses don't like being called from CPU != 0 */ -if (smp_processor_id () !=0 ) { -set_cpus_allowed (current,1 <<0 ); -if (unlikely (smp_processor_id () !=0 )) -BUG (); - } -#endif if (apm_info.realmode_power_off) { - (void )apm_save_cpus ();machine_real_restart (po_bios_call,sizeof (po_bios_call)); }else
arch/i386/kernel/dmi_scan.c +1 -26
@@ -220,31 +220,6 @@return 0 ; } -/* -* Some machines require the "reboot=s" commandline option, this quirk makes that automatic. -*/ -static __init int set_smp_reboot (struct dmi_blacklist *d) -{ -#ifdef CONFIG_SMP -extern int reboot_smp; -if (reboot_smp ==0 ) - { - reboot_smp =1 ; -printk (KERN_INFO "%s series board detected. Selecting SMP-method for reboots.\n" , d->ident); - } -#endif -return 0 ; -} - -/* -* Some machines require the "reboot=b,s" commandline option, this quirk makes that automatic. -*/ -static __init int set_smp_bios_reboot (struct dmi_blacklist *d) -{ -set_smp_reboot (d); -set_bios_reboot (d); -return 0 ; -}/* * Some bioses have a broken protected mode poweroff and need to use realmode
@@ -554,7 +529,7 @@MATCH (DMI_BIOS_VERSION ,"4.60 PGMA" ), MATCH(DMI_BIOS_DATE, "134526184"), NO_MATCH } }, - { set_smp_bios_reboot, "Dell PowerEdge 1300", { /* Handle problems with rebooting on Dell 1300's */ + { set_bios_reboot, "Dell PowerEdge 1300", { /* Handle problems with rebooting on Dell 1300's */MATCH (DMI_SYS_VENDOR ,"Dell Computer Corporation" ),MATCH (DMI_PRODUCT_NAME ,"PowerEdge 1300/" ),NO_MATCH ,NO_MATCH
arch/i386/kernel/io_apic.c +0 -2
@@ -1545,8 +1545,6 @@* Clear the IO-APIC before rebooting: */ clear_IO_APIC (); - -disconnect_bsp_APIC (); }/*
arch/i386/kernel/reboot.c +35 -43
127 lines not on this page · raw message
arch/i386/kernel/smp.c +0 -26
@@ -539,32 +539,6 @@return 0 ; } -static void stop_this_cpu (void * dummy) -{ -/* -* Remove this CPU: -*/ -clear_bit (smp_processor_id (), &cpu_online_map); -local_irq_disable (); -disable_local_APIC (); -if (cpu_data[smp_processor_id ()].hlt_works_ok) -for (;;) __asm__("hlt" ); -for (;;); -} - -/* -* this function calls the 'stop' function on all other CPUs in the system. -*/ - -void smp_send_stop (void ) -{ -smp_call_function (stop_this_cpu,NULL ,1 ,0 ); - -local_irq_disable (); -disable_local_APIC (); -local_irq_enable (); -} -/* * Reschedule call back. Nothing to do, * all the work is done automatically when
include/asm-i386/apic.h +13 -0
@@ -3,6 +3,7 @@#include <linux/config.h> #include <linux/pm.h> +#include <linux/linkage.h> #include <asm/fixmap.h> #include <asm/apicdef.h> #include <asm/system.h>
@@ -99,6 +100,18 @@#define NMI_LOCAL_APIC 2#define NMI_INVALID 3 +extern NORET_TYPE void +stop_apics(NORET_TYPE void (*rest)(void *info) ATTRIB_NORET, void *info) +ATTRIB_NORET; +#else +static inline NORET_TYPE void +stop_apics(NORET_TYPE void (*rest)(void *info) ATTRIB_NORET, void *info) +ATTRIB_NORET; +static inline void +stop_apics(NORET_TYPE void (*rest)(void *info) ATTRIB_NORET, void *info) +{ +rest (info); +}#endif /* CONFIG_X86_LOCAL_APIC */ #endif /* __ASM_APIC_H */
include/asm-i386/mach-default/mach_reboot.h +2 -0
@@ -27,4 +27,6 @@ } } +void stop_this_cpu (void ); +#endif /* !_MACH_REBOOT_H */
include/linux/reboot.h +4 -3
@@ -35,6 +35,7 @@#ifdef __KERNEL__#include <linux/notifier.h> +#include <linux/linkage.h> extern int register_reboot_notifier (struct notifier_block *);extern int unregister_reboot_notifier (struct notifier_block *);
@@ -44,9 +45,9 @@* Architecture-specific implementations of sys_reboot commands. */ -extern void machine_restart (char *cmd); -extern void machine_halt (void ); -extern void machine_power_off (void ); +NORET_TYPE void machine_restart(char *cmd) ATTRIB_NORET; +NORET_TYPE void machine_halt(void) ATTRIB_NORET; +NORET_TYPE void machine_power_off(void) ATTRIB_NORET;#endif
kernel/panic.c +1 -1
@@ -63,7 +63,7 @@sys_sync ();bust_spinlocks (0 ); -#ifdef CONFIG_SMP +#if defined(CONFIG_SMP ) && !defined(__i386__)smp_send_stop ();#endif
kernel/sys.c +0 -4
@@ -415,8 +415,6 @@device_shutdown ();printk (KERN_EMERG "System halted.\n" );machine_halt (); -unlock_kernel (); -do_exit (0 );break ;case LINUX_REBOOT_CMD_POWER_OFF :
@@ -425,8 +423,6 @@device_shutdown ();printk (KERN_EMERG "Power down.\n" );machine_power_off (); -unlock_kernel (); -do_exit (0 );break ;case LINUX_REBOOT_CMD_RESTART2 :
Quoted 5 lines · Eric W. Biederman
There is some software at least that knows the difference. I have seen short jumps in a couple of BIOS's. But a reset is very different from a reboot. As memory must be reinitialized etc. So I think going to 0xffff0000:0xfff0 would be a very bad idea if the intent is to get a reliable reboot.
You may change a bit in the i8042 controller to make a BIOS assume that's
a cold boot. The bit is zeroed (IIRC; apply a complement if my memory is bad) upon a system RESET that's propagated to the i8042 (i.e. a power-on or a button reset, but not a triple-fault or i8042 output port or port 0x92, etc. one). The bit is set to one by a BIOS during POST and never zeroed afterwards, but it's r/w, so there is no problem to clear it if needed. This should be quite a reliable way to reboot as a BIOS is assumed to initialize hardware from scratch (regardless of the reset vector used).
This assumes 100% PC/AT compatibility, of course, which need not be true
these days any longer.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Followup to: <[email protected]> By author: [email protected] (Eric W. Biederman) In newsgroup: linux.dev.kernel
Quoted 7 lines
There is some software at least that knows the difference. I have seen short jumps in a couple of BIOS's. But a reset is very different from a reboot. As memory must be reinitialized etc. So I think going to 0xffff0000:0xfff0 would be a very bad idea if the intent is to get a reliable reboot.
I agree.
Jumping to 0xf000:0xfff0 is widely accepted to be a standard warm reboot (as *should* an INIT, e.g. triplefault, be, as well -- make sure A20 is enabled before tripping, though.) For quite a few (most?) BIOSes, the vector that is stored at 0xf000:0xfff0 in the running (BIOS decompressed and shadowed) configuration is *not* the same as the one at the RESET vector.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64