Skip to contentSkip to conversation

PPC math-emu multiply problem

BUG Greg Weeks 5 participants 15 messages Jul 29, 2004 – Aug 16, 2004

lore
Share & tools
lore

Greg Weeks

author
#0 lore raw

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is. The value in mindble is the smallest that can be represented in a double. When we try to divide it by two we should see an underflow and a return value of 0. We see this when using soft-float in gcc, or when there is HW floating point support, but it fails when the kernel trap emulator is used.

If anyone can verify this on a PPC other than an 8560 without hardware floating point I'd appreciate it. I did all of these tests with a 2.6.X based kernels. The x86 was 2.6.6 vanilla, 8560 is 2.6.6 with lots of stuff added and support for 8560. The 8260 was 2.6.0 with changes. I bumped into this with the LSB ldexp test. A simple multiply shows the problem though.

Greg Weeks

mulbug.c file
------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <math.h>
#include <errno.h>

int main()

{
       double  x, rtval;
       double mindble = 4.9406564584124654418e-324;

       x = mindble;

       printf("x = %.20g\n", x);

       errno = 0;
       rtval = ldexp(x, -1);

       printf("using ldexp(x, -1) ERRNO = %d - %s,  %.20g\n",
           errno, strerror(errno), rtval);

       printf("using (x * .5) %.20g\n", (x * .5));

exit(0);
}
-----------------------------------------

compile with:
gcc mulbug.c -lm -o mulbug

on an 8260 ppc with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

on an x86 with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

on an 8560 ppc with kernel trap float emulator.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 0 - Success, 4.9406564584124654418e-324 using (x * .5) 4.9406564584124654418e-324

on an 8260 with soft-float in the gcc

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

#1 lore raw
Quoted 52 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is. The value in mindble is the smallest that can be represented in a double. When we try to divide it by two we should see an underflow and a return value of 0. We see this when using soft-float in gcc, or when there is HW floating point support, but it fails when the kernel trap emulator is used.

If anyone can verify this on a PPC other than an 8560 without hardware floating point I'd appreciate it. I did all of these tests with a 2.6.X based kernels. The x86 was 2.6.6 vanilla, 8560 is 2.6.6 with lots of stuff added and support for 8560. The 8260 was 2.6.0 with changes. I bumped into this with the LSB ldexp test. A simple multiply shows the problem though.

Greg Weeks

mulbug.c file
------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <math.h>
#include <errno.h>

int main()

{
       double  x, rtval;
       double mindble = 4.9406564584124654418e-324;

       x = mindble;

       printf("x = %.20g\n", x);

       errno = 0;
       rtval = ldexp(x, -1);

       printf("using ldexp(x, -1) ERRNO = %d - %s,  %.20g\n",
           errno, strerror(errno), rtval);

       printf("using (x * .5) %.20g\n", (x * .5));

exit(0);
}
-----------------------------------------

compile with:
gcc mulbug.c -lm -o mulbug

on an 8260 ppc with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

on an x86 with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

Quoted 5 lines

on an 8560 ppc with kernel trap float emulator.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 0 - Success, 4.9406564584124654418e-324 using (x * .5) 4.9406564584124654418e-324

I get the same results on an 8560, with 2.6.8-rc2. I do not think this is an issue with 8560, but the kernel math emulation's precision. It is most likely the case that the gcc soft-float code is 'more correct'.
What version of gcc were you using? It might be possible to replace
the kernel fp emulation with gcc's (never looked at how gcc does it).

Quoted 12 lines

on an 8260 with soft-float in the gcc

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#2 lore raw
Quoted 55 lines · Kumar Gala
Quoted 52 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is. The value in mindble is the smallest that can be represented in a double. When we try to divide it by two we should see an underflow and a return value of 0. We see this when using soft-float in gcc, or when there is HW floating point support, but it fails when the kernel trap emulator is used.

If anyone can verify this on a PPC other than an 8560 without hardware floating point I'd appreciate it. I did all of these tests with a 2.6.X based kernels. The x86 was 2.6.6 vanilla, 8560 is 2.6.6 with lots of stuff added and support for 8560. The 8260 was 2.6.0 with changes. I bumped into this with the LSB ldexp test. A simple multiply shows the problem though.

Greg Weeks

mulbug.c file
------------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <math.h>
#include <errno.h>

int main()

{
       double  x, rtval;
       double mindble = 4.9406564584124654418e-324;

       x = mindble;

       printf("x = %.20g\n", x);

       errno = 0;
       rtval = ldexp(x, -1);

       printf("using ldexp(x, -1) ERRNO = %d - %s,  %.20g\n",
           errno, strerror(errno), rtval);

       printf("using (x * .5) %.20g\n", (x * .5));

exit(0);
}
-----------------------------------------

compile with:
gcc mulbug.c -lm -o mulbug

on an 8260 ppc with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

on an x86 with HW float.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

Quoted 9 lines
Quoted 5 lines

on an 8560 ppc with kernel trap float emulator.

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 0 - Success, 4.9406564584124654418e-324 using (x * .5) 4.9406564584124654418e-324

I get the same results on an 8560, with 2.6.8-rc2. I do not think this is an issue with 8560, but the kernel math emulation's precision. It is most likely the case that the gcc soft-float code is 'more correct'. What version of gcc were you using? It might be possible to replace the kernel fp emulation with gcc's (never looked at how gcc does it).

That's what I suspect as well. I'm using a gcc 3.4 snapshot and a 3.4 release. The kernel math-emu code is based on gcc anyway. It appears the multiply is working ok, it's when we go to pack the fp value back up that we lose it.

This is the debug output from the multiply.

fmul: dfc20368 dfc203d0 dfc20368
A: 0 8388608 0 -1074 (0) [00800000.00000000 ffffffcd]
B: 0 8388608 0 -1 (0) [00800000.00000000 3fe]
D: 0 8388608 0 -1075 (0) [00800000.00000000 ffffffcc]

Quoted 6 lines
Quoted 5 lines

on an 8260 with soft-float in the gcc

x = 4.9406564584124654418e-324
using ldexp(x, -1) ERRNO = 34 - Numerical result out of range, 0 using (x * .5) 0

#3 lore raw
In reply to Kumar Gala
Quoted 6 lines · Kumar Gala
Quoted 3 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is.

I'm not surprised. I lifted this code from Sparc, glibc, and adapted it as best I could for PPC years ago for the 8xx. I was happy when it appeared to work for the general cases. :-)

Due to its overhead, I never expected it to be _the_ solution for processors that don't have floating point hardware. Recompiling the libraries with soft-float and using that option when compiling is the way to go.

Remember, don't mix soft-float compilation with libraries compiled with HW floating point, and trap emulations. They are not compatible and will return erroneous results.

Thanks.

	-- Dan
#4 lore raw
Quoted 6 lines · Dan Malek
Quoted 3 lines · Kumar Gala

On Jul 29, 2004, at 8:14 AM, Greg Weeks wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is.

I'm not surprised. I lifted this code from Sparc, glibc, and adapted it as best I could for PPC years ago for the 8xx. I was happy when it appeared to work for the general cases. :-)

Thanks for doing the initial work!

Quoted 4 lines

Due to its overhead, I never expected it to be _the_ solution for processors that don't have floating point hardware. Recompiling the libraries with soft-float and using that option when compiling is the way to go.

Indeed, when using this board for real we generally recommend exactly the same thing.

Quoted 3 lines

Remember, don't mix soft-float compilation with libraries compiled with HW floating point, and trap emulations. They are not compatible and will return erroneous results.

Unfortunately, LSB does not provide a set of test binaries which use the softfloat ABI, so in order to get an LSB-comformant distribution on this CPU, we need to use floating point emulation. Interestingly, this is the only LSB floating point test which fails.

Signature · 1 line
 Anyone who quotes me in their sig is an idiot. -- Rusty Russell's sig
author
#5 lore raw
In reply to Dan Malek
Quoted 11 lines · Dan Malek
Quoted 7 lines · Kumar Gala
Quoted 3 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is.

Quoted 8 lines

I'm not surprised. I lifted this code from Sparc, glibc, and adapted it as best I could for PPC years ago for the 8xx. I was happy when it appeared to work for the general cases. :-)

Due to its overhead, I never expected it to be _the_ solution for processors that don't have floating point hardware. Recompiling the libraries with soft-float and using that option when compiling is the way to go.

OK, this patch fixes my multiply problem with the LSB test. I still need to test to make sure I didn't break anything else, but it appears the rounding is only used when converting back to IEEE format. Is there some reason this is something really dumb to do?

Signed-off-by Greg Weeks <[email protected]> under TS0087
arch/ppc/math-emu/soft-fp.h 2004-07-30 10:31:57.000000000 -0400 renamed from arch/ppc/math-emu/soft-fp.h.orig 2004-07-30 10:31:34.000000000 -0400 +0 -0
@@ -15,7 +15,7 @@

# define FP_RND_PINF        2
# define FP_RND_MINF        3
#ifndef FP_ROUNDMODE
-# define FP_ROUNDMODE        FP_RND_NEAREST
+# define FP_ROUNDMODE        FP_RND_ZERO
#endif
#endif
author
#6 lore raw
Quoted 18 lines · Greg Weeks
Quoted 16 lines · Dan Malek
Quoted 8 lines · Kumar Gala
Quoted 3 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is.

I'm not surprised. I lifted this code from Sparc, glibc, and adapted it as best I could for PPC years ago for the 8xx. I was happy when it appeared to work for the general cases. :-)

Quoted 8 lines
Quoted 4 lines

Due to its overhead, I never expected it to be _the_ solution for processors that don't have floating point hardware. Recompiling the libraries with soft-float and using that option when compiling is the way to go.

OK, this patch fixes my multiply problem with the LSB test. I still need to test to make sure I didn't break anything else, but it appears the rounding is only used when converting back to IEEE format. Is there some reason this is something really dumb to do?

When I actually built a kernel rather than just my test code the FP_ROUNDMODE is picked up from the linux/math-emu/soft-fp.h. I don't want to change the common definition unless I'm sure this is the correct solution.

Signed-off-by Greg Weeks <[email protected]> under TS0087
arch/ppc/math-emu/soft-fp.h 2004-07-30 11:18:59.000000000 -0400 renamed from arch/ppc/math-emu/soft-fp.h.orig 2004-07-30 10:31:34.000000000 -0400 +0 -0
@@ -14,9 +14,8 @@

# define FP_RND_ZERO        1
# define FP_RND_PINF        2
# define FP_RND_MINF        3
-#ifndef FP_ROUNDMODE
-# define FP_ROUNDMODE        FP_RND_NEAREST
-#endif
+#undef FP_ROUNDMODE
+# define FP_ROUNDMODE        FP_RND_ZERO
#endif

#define _FP_ROUND_NEAREST(wc, X) \

#7 lore raw
Quoted 31 lines · Greg Weeks
Quoted 24 lines · Greg Weeks
Quoted 12 lines · Dan Malek
Quoted 8 lines · Kumar Gala
Quoted 3 lines · Greg Weeks

I'm seeing what appears to be a bug in the ppc kernel trap math emulator. An extreme case for multiplies isn't working the way gcc soft-float or hardware floating point is.

Quoted 8 lines

I'm not surprised. I lifted this code from Sparc, glibc, and adapted it as best I could for PPC years ago for the 8xx. I was happy when it appeared to work for the general cases. :-)

Due to its overhead, I never expected it to be _the_ solution for processors that don't have floating point hardware. Recompiling the libraries with soft-float and using that option when compiling is the way to go.

Quoted 5 lines

OK, this patch fixes my multiply problem with the LSB test. I still need to test to make sure I didn't break anything else, but it appears the rounding is only used when converting back to IEEE format. Is there some reason this is something really dumb to do?

Quoted 6 lines

When I actually built a kernel rather than just my test code the FP_ROUNDMODE is picked up from the linux/math-emu/soft-fp.h. I don't want to change the common definition unless I'm sure this is the correct solution.

Signed-off-by Greg Weeks <[email protected]> under TS0087

Has anyone had a problem with this? If not, I'll go and pass it along...

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
#8 lore raw
Quoted 2 lines · Tom Rini

Has anyone had a problem with this? If not, I'll go and pass it along...

The default rounding mode should be whatever is defined by IEEE. I thought the emulator used the proper default value and if want something different it should be selected by the control register. Maybe the emulator isn't implementing the control register properly.

Thanks.

	-- Dan
#9 lore raw
Quoted 13 lines · Dan Malek
Quoted 2 lines · Tom Rini

Has anyone had a problem with this? If not, I'll go and pass it along...

The default rounding mode should be whatever is defined by IEEE. I thought the emulator used the proper default value and if want something different it should be selected by the control register. Maybe the emulator isn't implementing the control register properly.

Quoted 3 lines

Thanks.

    -- Dan

The submitted patch changing the default rounding mode caused several regressions in the LSB suite, so it was NOT the correct fix.

Greg made a finer-grained change which seems to have made LSB happy. Unfortunately, he's on vacation this week, or I'd have him submit the patch he finally used.

I THINK he changed the rouding mode only for a single corner case in the normalization macro, but that's about all I remember...

Signature · 1 line
 Anyone who quotes me in their sig is an idiot. -- Rusty Russell's sig
#10 lore raw
In reply to Dan Malek
Quoted 11 lines · Dan Malek
Quoted 2 lines · Tom Rini

Has anyone had a problem with this? If not, I'll go and pass it along...

The default rounding mode should be whatever is defined by IEEE. I thought the emulator used the proper default value and if want something different it should be selected by the control register. Maybe the emulator isn't implementing the control register properly.

Or we had the wrong default? Greg, any chance you've looked into this more? Thanks.

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
author
#11 lore raw
Quoted 20 lines · Tom Rini
Quoted 8 lines · Dan Malek
Quoted 4 lines · Tom Rini

Has anyone had a problem with this? If not, I'll go and pass it along...

Quoted 7 lines

The default rounding mode should be whatever is defined by IEEE. I thought the emulator used the proper default value and if want something different it should be selected by the control register. Maybe the emulator isn't implementing the control register properly.

Quoted 5 lines

Or we had the wrong default? Greg, any chance you've looked into this more? Thanks.

I'm back.

The round mode for the emulator is compiled in. Changing the round mode caused failures in some of the other LSB float tests. I had intended to say something about this before taking off on vacation. Sorry.

The way I got the LSB tests to pass was to remove the round in the denormalised underflow case. This appears to match the hardware behavior. I've not looked at the PPC floating point model close enough to know if this is proper behavior. It is what the LSB tests are expecting and doesn't cause a failure in any of the other LSB tests.

Greg Weeks

Signed-off-by Greg Weeks <[email protected]> under TS0087
arch/ppc/math-emu/op-common.h 2004-08-04 10:04:45.000000000 -0400 renamed from arch/ppc/math-emu/op-common.h.orig 2004-08-16 08:56:07.000000000 -0400 +0 -1
@@ -82,7 +82,6 @@ do {
       \
       if (X##_e <= _FP_WFRACBITS_##fs)                        \
         {                                                     \
           _FP_FRAC_SRS_##wc(X, X##_e, _FP_WFRACBITS_##fs);    \
-           __ret |= _FP_ROUND(wc, X);                          \
           _FP_FRAC_SLL_##wc(X, 1);                            \
           if (_FP_FRAC_OVERP_##wc(fs, X))                     \

             {                                                 \
#12 lore raw
Quoted 12 lines · Greg Weeks
Quoted 11 lines · Tom Rini
Quoted 9 lines · Dan Malek
Quoted 3 lines · Tom Rini

Has anyone had a problem with this? If not, I'll go and pass it along...

The default rounding mode should be whatever is defined by IEEE. I thought the emulator used the proper default value and if want something different it should be selected by the control register. Maybe the emulator isn't implementing the control register properly.

Quoted 8 lines
Quoted 3 lines

Or we had the wrong default? Greg, any chance you've looked into this more? Thanks.

I'm back.

The round mode for the emulator is compiled in. Changing the round mode caused failures in some of the other LSB float tests. I had intended to say something about this before taking off on vacation. Sorry.

S'alright.

Quoted 5 lines

The way I got the LSB tests to pass was to remove the round in the denormalised underflow case. This appears to match the hardware behavior. I've not looked at the PPC floating point model close enough to know if this is proper behavior. It is what the LSB tests are expecting and doesn't cause a failure in any of the other LSB tests.

Have you guys run the LSB tests on some PPC with hw floating point (is that what you mean by 'matches the hardware behavior' ?) to see if the test also passes there as-is? And does anyone object to this patch? Now that 2.6.8.1 is out I'm gonna start committing in a bunch of stuff I've had queued up and see if I can get Linus to pull. Thanks.

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/
author
#13 lore raw
Quoted 4 lines · Tom Rini
Quoted 3 lines

The way I got the LSB tests to pass was to remove the round in the denormalised underflow case. This appears to match the hardware behavior. I've not looked at the PPC floating point model close enough to know if this is proper behavior. It is what the LSB tests are expecting and doesn't cause a failure in any of the other LSB tests.

Quoted 8 lines

Have you guys run the LSB tests on some PPC with hw floating point (is that what you mean by 'matches the hardware behavior' ?) to see if the test also passes there as-is? And does anyone object to this patch? Now that 2.6.8.1 is out I'm gonna start committing in a bunch of stuff I've had queued up and see if I can get Linus to pull. Thanks.

I didn't run the entire LSB, just some of the math tests. I had an 8260 and the 8560 we found the problem on and also a normal x86 box. I think this is the correct fix. At least all of the LSB math tests pass now and the LTP float tests don't complain.

Greg Weeks

#14 lore raw
Quoted 18 lines · Greg Weeks
Quoted 8 lines · Tom Rini
Quoted 7 lines

The way I got the LSB tests to pass was to remove the round in the denormalised underflow case. This appears to match the hardware behavior. I've not looked at the PPC floating point model close enough to know if this is proper behavior. It is what the LSB tests are expecting and doesn't cause a failure in any of the other LSB tests.

Quoted 8 lines

Have you guys run the LSB tests on some PPC with hw floating point (is that what you mean by 'matches the hardware behavior' ?) to see if the test also passes there as-is? And does anyone object to this patch? Now that 2.6.8.1 is out I'm gonna start committing in a bunch of stuff I've had queued up and see if I can get Linus to pull. Thanks.

Quoted 4 lines

I didn't run the entire LSB, just some of the math tests. I had an 8260 and the 8560 we found the problem on and also a normal x86 box. I think this is the correct fix. At least all of the LSB math tests pass now and the LTP float tests don't complain.

Just to be clear, with that patch, 8260, 8560 and x86 all agree on results ?

Signature · 2 lines
Tom Rini
http://gate.crashing.org/~trini/