PROBLEM: 2.6 kernels on x86 do not preserve FPU flags across context switches

Hi Andi,

you asked:
| On what CPUs does the failure occur? Linux uses different paths | depending on if the CPU supports SSE or not.

Travis responded:

| We run on both AMDs (Durons and Athlons) as well as PII, PIII, and 
| PIV's. Our kernels are all compiled as generic 586+. Though when we were 
| testing for this, we did try the more generic 486+ option, as well as 
| exact processor matches for the AMD at least. I don't remember it making 
| a difference.

+-----------------------------
| Date:	Wed, 16 Jun 2004 23:40:18 +0200
| From:	Andi Kleen <[email protected]>
| Subject:	Re: PROBLEM: 2.6 kernels on x86 do not preserve FPU flags across context switches

| [email protected] writes:

| > 	I am the team lead and chief VM developer for a Smaltalk
| > 	implementation based on a JIT execution engine.  Our customers
| > 	have been seeing rare incorrect floating-point results in
| > 	intensive fp applications on 2.6 kernels using various x86
| > 	compatible processors.  These problems do not occur on
| > 	previous kernel versons.  We recently had occasion to
| > 	reimplement our fp primitives to avoid severe performance
| > 	problems on Xeon processors that were traced to Xeon's
| > 	relatively slow implementation of fnclex and fstsw.  The older

| Funny, Linux just added fnclex to a critical path on popular request.
| But I guess it will need to be removed again, we already discussed
| that. 

| > I don't know whether any action on your part is appropriate.  The
| > use of the FPU status flags is presumably rare on linux (I believe
| > that neither gcc nor glibc make use of them).  But "exotic"
| > execution machinery such as runtimes for dynamic or functional
| > languages (language implementations that may not use IEEE arithmetic
| > and instead flag Infs and NaNs as an error) may fall foul of this
| > issue.  Since previous versions of the kernel on x86 apparently do
| > preserve the FPU status flags perhaps its simple to preserve the old
| > behaviour.  At the very least let me suggest you document the
| > limitation.

| This sounds like a serious kernel bug that should be fixed if
| true. Can you perhaps create a simple demo program that shows the
| problem and post it?

| On what CPUs does the failure occur? Linux uses different paths | depending on if the CPU supports SSE or not.

| Does your program receive signals? Could it be related to them?

| -Andi
---
Eliot Miranda ,,,^..^,,, mailto:[email protected]
VisualWorks Engineering, Cincom Smalltalk: scene not herd Tel +1 408 216 4581 3350 Scott Blvd, Bldg 36 Suite B, Santa Clara, CA 95054 USA Fax +1 408 216 4500

And you saw it on all of them? (in particular both on PII and on PIV?)

I actually doubt the problem happens on a context switch - the kernel just uses FXSAVE/FNSAVE for this and this is extremly hard to get wrong. Either you have no FPU state saved at all or you have all, since the CPU handles it completely in microcode.

Most likely candidate would be signal context saving. When a signal happens and the process used floating point then the i386 kernel converts the internal FXSAVE/FNSAVE image to another image derived from iBCS on the signal stack (and then later back). If any problems with subtle corruptions happen I would expect them in this process.

This would be more likely on SSE enabled CPUs though, on pre SSE CPUs this code is much simpler.

Do you know which status bit gets corrupted exactly?

-Andi