Pentium Pro - sysenter - doublefault
PATCH 1/1 Pentium Pro - sysenter - doublefault
Hi Everyone,
I upgraded my Pentium Pro system to Redhat 9, installed a linux-2.6.0-test3 kernel, and it fails with a double-fault when init starts.
The code which decides if it is o.k. to use sysenter is broken for some Pentium Pro cpus ,in particular, this bit of code from arch/i386/kernel/cpu/intel.c:
/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */ if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 ) clear_bit(X86_FEATURE_SEP, c->x86_capability);
On my cpu model=1 and mask=9, it doesn't clear 86_FEATURE_SEP. This results in a double-fault when init starts. The double-fault happens on the sysexit. The new double-fault handler caught this nicely, and I was able to debug this with kgdb.
The logic above is exactly what Intel says to do in "IA-32 Intel® Architecture Software Developer's Manual, Volume 2: Instruction Set Reference" on page 3-767. It also says that sysenter was added to the Pentium II.
I checked the Pentium Pro and Pentium II Specifications Update manuals hoping to find the details to justify the "mask < 3" portion of the test above. They both describe sysenter related errata but none which was fixed in mask 3.
The attached patch avoids using sysenter on all Pentium Pro systems.
Jim Houston - Concurrent Computer Corp.
diff -urN linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c
--- linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c 2003-08-20
10:30:14.000000000 -0400
+++ linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c 2003-08-21
14:39:35.000000000 -0400
@@ -246,7 +246,7 @@
}
/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */
- if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 )
+ if ( c->x86 == 6 && c->x86_model < 3)
clear_bit(X86_FEATURE_SEP, c->x86_capability);
/* Names for the Pentium II/Celeron processors
Quoted 19 lines · Jim Houston
Hi Everyone,
I upgraded my Pentium Pro system to Redhat 9, installed a linux-2.6.0-test3 kernel, and it fails with a double-fault when init starts.
The code which decides if it is o.k. to use sysenter is broken for some Pentium Pro cpus ,in particular, this bit of code from
arch/i386/kernel/cpu/intel.c:/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */ if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 ) clear_bit(X86_FEATURE_SEP, c->x86_capability);On my cpu model=1 and mask=9, it doesn't clear 86_FEATURE_SEP. This results in a double-fault when init starts. The double-fault happens on the sysexit. The new double-fault handler caught this nicely, and I was able to debug this with kgdb.
Quoted 4 lines
The logic above is exactly what Intel says to do in "IA-32 Intel® Architecture Software Developer's Manual, Volume 2: Instruction Set Reference" on page 3-767. It also says that sysenter was added to the Pentium II.
I double-checked AP-485 (24161823.pdf, the "real" reference to CPUID), and it says (section 3.4) that SEP is unsupported when the signature as a whole is less that 0x633. This means all PPros, and PII Model 3s with steppings less than 3.
The problem is that the kernel check you quoted above is buggy: the c->x86_model < 3 && c->x86_mask < 3 part fails for late-stepping PPros since c->x86_mask >= 3 for them. The test should be rewritten as:
if (c->x86 == 6 && (c->x86_model < 3 ||
(c->x86_model == 3 && c->x86_mask < 3)))
clear_bit(X86_FEATURE_SEP, c->x86_capability);/Mikael
Quoted 10 lines · Mikael Pettersson
Quoted 4 lines
The logic above is exactly what Intel says to do in "IA-32 Intel® Architecture Software Developer's Manual, Volume 2: Instruction Set Reference" on page 3-767. It also says that sysenter was added to the Pentium II.
I double-checked AP-485 (24161823.pdf, the "real" reference to CPUID), and it says (section 3.4) that SEP is unsupported when the signature as a whole is less that
0x633. This means all PPros, and PII Model 3s with steppings less than 3.
Hi Dave, Everyone,
This make sense. Here is Mikael's suggested code as a patch.
Dave, I picked your name from the maintainers list. Please feed this patch up the chain.
Jim Houston - Concurrent Computer Corp.
diff -urN linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c
--- linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c 2003-08-20
10:30:14.000000000 -0400
+++ linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c 2003-08-21
21:34:40.000000000 -0400
@@ -246,7 +246,8 @@
}
/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */
- if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 )
+ if ( c->x86 == 6 && ((c->x86_model < 3) ||
+ (c->x86_model == 3 && c->x86_mask < 3)))
clear_bit(X86_FEATURE_SEP, c->x86_capability);
/* Names for the Pentium II/Celeron processors
Quoted 9 lines · Mikael Pettersson
Quoted 4 lines
The logic above is exactly what Intel says to do in "IA-32 Intel® Architecture Software Developer's Manual, Volume 2: Instruction Set Reference" on page 3-767. It also says that sysenter was added to the Pentium II.
I double-checked AP-485 (24161823.pdf, the "real" reference to CPUID), and it says (section 3.4) that SEP is unsupported when the signature as a whole is less that
0x633. This means all PPros, and PII Model 3s with steppings less than 3.
So (double-checking) the pseudo-code in "IA-32: Intel Architecture Software Development Manual, Volume 2: Instruction Set Reference" is buggy?
Oh my! Perhaps there are other bugs in that behemoth of a manual, too :/
-- Jamie
Quoted 4 lines · Mikael Pettersson
I double-checked AP-485 (24161823.pdf, the "real" reference to CPUID), and it says (section 3.4) that SEP is unsupported when the signature as a whole is less that
0x633. This means all PPros, and PII Model 3s with steppings less than 3.
"SEP is unsupported". It's interesting that Pentium Pro erratum #82 is "SYSENTER/SYSEXIT instructions can implicitly load 'null segment selector' to SS and CS registers", implying that SYSENTER does _something_ useful on PPros.
-- Jamie
Hi!
Quoted 9 lines
Quoted 4 lines
I double-checked AP-485 (24161823.pdf, the "real" reference to CPUID), and it says (section 3.4) that SEP is unsupported when the signature as a whole is less that
0x633. This means all PPros, and PII Model 3s with steppings less than 3."SEP is unsupported". It's interesting that Pentium Pro erratum #82 is "SYSENTER/SYSEXIT instructions can implicitly load 'null segment selector' to SS and CS registers", implying that SYSENTER does _something_ useful on PPros.
Well, with CS==0 machine is not going to survive too long. If it only happens sometimes you might catch the double fault and fixup, but....
Pavel
Signature · 2 lines
Pavel Written on sharp zaurus, because my Velo1 broke. If you have Velo you don't need...
Quoted 8 lines · Pavel Machek
Quoted 4 lines
"SEP is unsupported". It's interesting that Pentium Pro erratum #82 is "SYSENTER/SYSEXIT instructions can implicitly load 'null segment selector' to SS and CS registers", implying that SYSENTER does _something_ useful on PPros.
Well, with CS==0 machine is not going to survive too long. If it only happens sometimes you might catch the double fault and fixup, but....
The erratum only applies when you load CS==0 _deliberately_, by setting the MSR to that.
I'm wondering what happens when you don't do silly things - what is the undocumented behaviour of SYSENTER/SYSEXIT on those chips?
I vaguely recall reading details about the behaviour change made by Intel, around the time it was done, but I can't see to find it anywhere.
-- Jamie
Quoted 4 lines · Jim Houston
On my cpu model=1 and mask=9, it doesn't clear 86_FEATURE_SEP. This results in a double-fault when init starts. The double-fault happens on the sysexit. The new double-fault handler caught this nicely, and I was able to debug this with kgdb.
Does anyone know what the syenter & sysexit instructions do on these early PPro CPUs?
The Intel documentation is vague, saying only to avoid using them. I'd like to know what happens if userspace does "sysenter" on one of these systems. Does it issue Invalid Opcode, General Protection fault, or something else?
Jim you can answer this as you have such a Ppro. Could you please run this very simple userspace program for me, and report the result?
int main() { __asm__ ("sysenter"); return 0; }I expect it to die with SIGILL on Pentium and earlier chips, and SIGSEGV on "good" PPro and later chips running kernels which don't enable the sysenter instruction.
But what does it do on your early Intel PPro, the one which is the subject of this thread?
Thanks,
-- Jamie
On Mon, Aug 25, 2003 at 05:05:14AM +0100, Jamie Lokier wrote:
...
Quoted 4 lines
Jim you can answer this as you have such a Ppro. Could you please run this very simple userspace program for me, and report the result?
int main() { __asm__ ("sysenter"); return 0; }
I tested on two boxes:
Stepping 1 ppro: SIGSEGV Stepping 7 ppro: SIGSEGV
If you need additional info, please just ask.
Signature · 6 lines
................................................................ : [email protected] : And I see the elder races, : :.........................: putrid forms of man : : Jakob Østergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............:
Quoted 9 lines · Jakob Oestergaard
Quoted 4 lines
Jim you can answer this as you have such a Ppro. Could you please run this very simple userspace program for me, and report the result?
int main() { __asm__ ("sysenter"); return 0; }I tested on two boxes:
Stepping 1 ppro: SIGSEGV Stepping 7 ppro: SIGSEGV
Thank you!
So that means the sysenter instruction _does_ exist on the PPro and early Pentium II, but it isn't usable.
It's safe so long as it is disabled, but as you found, when you enable and use it, it doesn't work as expected. (I wonder what the actual behaviour of sysenter/sysexit are on these CPUs.)
-- Jamie
Quoted 2 lines · Jamie Lokier
So that means the sysenter instruction _does_ exist on the PPro and early Pentium II, but it isn't usable.
If anyone has information on what the SYSENTER and SYSEXIT instructions actually do on Intel Pentium Pro or stepping<3 Pentium II processors, I am very interested.
I'm intrigued to know if the buggy behaviour of these instructions is really unsafe, or simply hard to use so Intel changed the behaviour. (An example of hard to use would be SYSENTER not disabling interrupts). If they are safe but hard to use, perhaps the ingenuity of kernel hackers can work around the hardness >:)
Does anyone have a contact at Intel for this question?
Thanks,
-- Jamie
Quoted 7 lines · Jamie Lokier
Quoted 2 lines · Jamie Lokier
So that means the sysenter instruction _does_ exist on the PPro and early Pentium II, but it isn't usable.
If anyone has information on what the SYSENTER and SYSEXIT instructions actually do on Intel Pentium Pro or stepping<3 Pentium II processors, I am very interested.
I dug up a little more from the archeological site (machine room - sigh)...
-------------------------------------------
model name : Pentium II (Deschutes)
stepping : 2
$ ./syse Segmentation fault ------------------------------------------- model name : Celeron (Mendocino) stepping : 0 cpu MHz : 334.097 $ ./syse Segmentation fault ------------------------------------------- CPU: Pentium II/Pentium II Xeon/Celeron (299.94-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x651 Stepping = 1 $ ./syse Bus error (core dumped) -------------------------------------------
The last one is a FreeBSD box.
Signature · 6 lines
................................................................ : [email protected] : And I see the elder races, : :.........................: putrid forms of man : : Jakob Østergaard : See him rise and claim the earth, : : OZ9ABN : his downfall is at hand. : :.........................:............{Konkhra}...............:
Quoted 8 lines · Jamie Lokier
Quoted 2 lines · Jamie Lokier
So that means the sysenter instruction _does_ exist on the PPro and early Pentium II, but it isn't usable.
If anyone has information on what the SYSENTER and SYSEXIT instructions actually do on Intel Pentium Pro or stepping<3 Pentium II processors, I am very interested.
Quoted 5 lines
I'm intrigued to know if the buggy behaviour of these instructions is really unsafe, or simply hard to use so Intel changed the behaviour. (An example of hard to use would be SYSENTER not disabling interrupts). If they are safe but hard to use, perhaps the ingenuity of kernel hackers can work around the hardness >:)
Hi Jamie,
I tried your test on my machine. It fails with a segmentation fault. I noticed that the Pentium II specifications update manual starts with rev C0 stepping (ignoring mask rev < 3).
I'm inclined to forgive Intel for not publishing the scary errata that goes with the first few mask revs, particularly for an old product.
When I was chasing the original problem, I added tracing code (compiling the kernel with finstrument-functions) so that when I got into kgdb after the double-fault I could see that it had just completed a umask system call. I'm assuming that it failed on the sysexit.
I keep the old Pentium Pro around because it has an NMI interrupt button.
I'm happy that Linus has merged the fix to disable correctly disable sysenter for these machines.
Jim Houston - Concurrent Computer Corp
Quoted 8 lines · Jamie Lokier
Could you please run this very simple userspace program for me, and report the result?
int main() { __asm__ ("sysenter"); return 0; }I expect it to die with SIGILL on Pentium and earlier chips, and SIGSEGV on "good" PPro and later chips running kernels which don't enable the sysenter instruction.
OK, since I get something different to the other reports I saw:
1:20PM-malvern-0-534-% ./sysenter
1:20PM-malvern-STKFLT-535-% echo $?
144
/proc/cpuinfo
processor : 0vendor_id : GenuineIntel
cpu family : 6
model : 5
model name : Pentium II (Deschutes)
stepping : 0
cpu MHz : 333.495
cache size : 512 KBfdiv_bug : nohlt_bug : nof00f_bug : nocoma_bug : no
fpu : yesfpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 657.40
uname -a:
Linux malvern 2.6.0-test2-mm5 #2 Fri Aug 8 12:06:50 BST 2003 i686 unknown
HTH
Signature · 3 lines
Richard \\\ SuperH Core+Debug Architect /// .. At home .. P. /// [email protected] /// [email protected] Curnow \\\ http://www.superh.com/ /// www.rc0.org.uk
Quoted 5 lines · Richard Curnow
OK, since I get something different to the other reports I saw:
1:20PM-malvern-0-534-% ./sysenter
1:20PM-malvern-STKFLT-535-% echo $?
144
Hi Richard,
That's because you ran it on a 2.5/2.6 kernel, right? The test code is meant for 2.4 kernels and earlier :)
Here is a more universal test:
int main () {
asm ("movl %%esp,%%ebp;sysenter" : : "a" (1), "b" (0));
return 0;
}I expect it to do the first of these which is applicable:
- raise SIGILL on Pentium and earlier Intel CPUs
- raise SIGILL on non-Intel CPUs which don't have the SEP capability
- raise SIGSEGV on Pentium Pro CPUs
- raise SIGSEGV on Pentium II CPUs with model == 3 and stepping < 3
- raise SIGSEGV on 2.4 kernels
- exit with status 0 on 2.6 kernels
Enjoy,
-- Jamie
Quoted 2 lines · Jamie Lokier
Here is a more universal test:
[snip]
Quoted 1 line
- exit with status 0 on 2.6 kernels
Yes, confirmed, that's what it did on 2.6.0-test2-mm5 here. Sorry, I can't try 2.4 right now.
Signature · 3 lines
Richard \\\ SuperH Core+Debug Architect /// .. At home .. P. /// [email protected] /// [email protected] Curnow \\\ http://www.superh.com/ /// www.rc0.org.uk
did you notice the announcement from www.apache.org and www.debian.org?
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jamie Lokier
Sent: Wednesday, August 27, 2003 10:01 PM
To: Jim Houston; [email protected]; [email protected]
Subject: Re: [PATCH] Pentium Pro - sysenter - doublefault
Quoted 4 lines · Richard Curnow
OK, since I get something different to the other reports I saw:
1:20PM-malvern-0-534-% ./sysenter 1:20PM-malvern-STKFLT-535-% echo
$? 144
Hi Richard,
That's because you ran it on a 2.5/2.6 kernel, right? The test code is meant for 2.4 kernels and earlier :)
Here is a more universal test:
int main () {
asm ("movl %%esp,%%ebp;sysenter" : : "a" (1), "b" (0));
return 0;
}I expect it to do the first of these which is applicable:
- raise SIGILL on Pentium and earlier Intel CPUs
- raise SIGILL on non-Intel CPUs which don't have the SEP
capability
- raise SIGSEGV on Pentium Pro CPUs
- raise SIGSEGV on Pentium II CPUs with model == 3 and stepping
< 3
- raise SIGSEGV on 2.4 kernels
- exit with status 0 on 2.6 kernels
Enjoy,
-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Jamie Lokier wrote:
<SNIP>
Quoted 11 lines
I expect it to do the first of these which is applicable:
- raise SIGILL on Pentium and earlier Intel CPUs
- raise SIGILL on non-Intel CPUs which don't have the SEP capability
- raise SIGSEGV on Pentium Pro CPUs
- raise SIGSEGV on Pentium II CPUs with model == 3 and stepping < 3
- raise SIGSEGV on 2.4 kernels
- exit with status 0 on 2.6 kernelsEnjoy,
-- Jamie
As expected I get a SIGILL on P166 with 2.4.22
-sb
-----------------------------------------------
The price of freedom? Ask your Senator how much
the RIAA gave him for his Lexus.
* Jamie Lokier <[email protected]>:
Quoted 11 lines
Quoted 5 lines · Richard Curnow
OK, since I get something different to the other reports I saw:
1:20PM-malvern-0-534-% ./sysenter
1:20PM-malvern-STKFLT-535-% echo $?
144Hi Richard,
That's because you ran it on a 2.5/2.6 kernel, right? The test code is meant for 2.4 kernels and earlier :)
If this is of any help..
- -
pvsavola@a11a:~/code$ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 1 model name : Pentium Pro stepping : 6 cpu MHz : 199.312 cache size : 256 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov bogomips : 397.31
pvsavola@a11a:~/code$ ./sysent ; echo $? Segmentation fault 139 pvsavola@a11a:~/code$ uname -a Linux a11a 2.4.19-ck3-rmap #1 Mon Aug 26 21:38:49 EEST 2002 i686 GNU/Linux - -
Signature · 1 line
Psi -- <http://www.iki.fi/pasi.savolainen>