2.6.0-test3-mm1: scheduling while atomic (ext3?)
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 1060772769.8009.4.camel@localhost.localdomain.suse.lists.linux.kernel
Quoted 1 line · Alan Cox
Put the likely(pos) in the asm/prefetch for Athlon until someone can
out what is going on with some specific Athlons, 2.6 and certain
Quoted 1 line
kernels (notably 4G/4G)
You can use the same workaround as x86-64. add an exception handler and just jump back. Advantage is that it is completely outside the fast path.
But note you also have to add runtime sorting of __ex_table when you do this, otherwise the __ex_table becomes unsorted when someone uses list_for_each (which does prefetch) in a __init function
(all code is available in x86-64, just needs to be ported over)
-Andi
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 1060774803.8008.24.camel@localhost.localdomain.suse.lists.linux.kernel
Quoted 6 lines · Alan Cox
Quoted 4 lines · Andrew Morton
Like this?
What happens if someone runs a K6 kernel on a K7? Or various other CPU types? What is the matrix here?
Quoted 6 lines
Beats me, but then the prefetch code in 2.6 seems broken from 5 seconds of inspection anyway. We are testing the XMM feature and using prefetchnta for Athlon, thats wrong for lots of athlon processors that dont have XMM but do have prefetch/prefetchw, (which btw also seem to work properly on all these processors
while prefetchnta seems to do funky things)
The early Athlon Specific test was not done to avoid too much bloat. (three alternatives instead of two)
Most Athlons in existence should have XMM already and the rest works.
You can hardly call that broken.
I would be surprised if prefetch behaves differently than prefetchnta on Athlon. If the bug is similar to what happens on Opteron then I bet it won't make a difference.
Quoted 2 lines
For Athlon we should be testing 3Dnow, and using prefetch/prefetchw for Intel cases we want to go for prefetchnta if XMM is set (PIII, PIV)
That's done for write prefetches correctly.
(as Intel does not have a write prefetch)
-Andi
Quoted 9 lines · Andi Kleen
Quoted 6 lines
Beats me, but then the prefetch code in 2.6 seems broken from 5 seconds of inspection anyway. We are testing the XMM feature and using prefetchnta for Athlon, thats wrong for lots of athlon processors that dont have XMM but do have prefetch/prefetchw, (which btw also seem to work properly on all these processors
while prefetchnta seems to do funky things)The early Athlon Specific test was not done to avoid too much bloat. (three alternatives instead of two)
Lets replace working code with broken macros whoooo.. progress. Lots of Athlons don't have XMM, most of the older ones where prefetch has the most impact in fact. (The XMM using ones have the hw prefetcher too).
Quoted 1 line
Most Athlons in existence should have XMM already and the rest works.
Lots don't have XMM
Quoted 1 line
You can hardly call that broken.
I just did. Its worse than 2.4 behaviour.
Quoted 2 lines
That's done for write prefetches correctly.
(as Intel does not have a write prefetch)
Actually its iffy too. 3Dnow doesnt imply prefetchw. You must test 3Dnow && vendor==AMD && Athlon. (K6 prefetchw is slower than not using it, other 3Dnow chips dont have it eg the Cyrix MII which may explain a couple of things. I don't see anywhere we mask the 3Dnow property by these but I've not dug through the CPU code right now to see if we have a "3Dnowplus" type definition we can check.
I suspect the best way to do prefetch cleanly would be something like this
#if defined(CONFIG_MK7)
alternative_input("prefetch" or "prefetchnta")
#else
alternative_input(ASM_NOP4 or "prefetchnta");#endif
Ideally we want a 3 way patch table to fix up at boot time but the if case at least gets us back to desirable situations. Also if I remember the prefetch exception thing rightly you can misalign the prefetch instruction as a workaround.
Alan
Quoted 8 lines · Alan Cox
Quoted 7 lines · Andi Kleen
Quoted 6 lines
Beats me, but then the prefetch code in 2.6 seems broken from 5 seconds of inspection anyway. We are testing the XMM feature and using prefetchnta for Athlon, thats wrong for lots of athlon processors that dont have XMM but do have prefetch/prefetchw, (which btw also seem to work properly on all these processors
while prefetchnta seems to do funky things)
Quoted 6 lines
Quoted 2 lines
The early Athlon Specific test was not done to avoid too much bloat. (three alternatives instead of two)
Lets replace working code with broken macros whoooo.. progress. Lots of Athlons don't have XMM, most of the older ones where prefetch has the most impact in fact. (The XMM using ones have the hw prefetcher too).
hw prefetch has nothing to do with how the linux kernel uses prefetch. It's only using it for data structures that cannot be handled by the auto prefetcher.
[except the broken 3dnow! copy that was never enabled]
Quoted 4 lines
Quoted 2 lines
Most Athlons in existence should have XMM already and the rest works.
Lots don't have XMM
All XPs have.
Quoted 4 lines
Quoted 2 lines
You can hardly call that broken.
I just did. Its worse than 2.4 behaviour.
In 2.4 distribution users never got anything. That is what was really broken.
Quoted 5 lines
Quoted 3 lines
That's done for write prefetches correctly.
(as Intel does not have a write prefetch)Actually its iffy too. 3Dnow doesnt imply prefetchw. You
My AMD manual lists it as part of 3dnow. If an CPU advertises 3dnow! but doesn't have the instruction it's broken.
Quoted 3 lines
must test 3Dnow && vendor==AMD && Athlon. (K6 prefetchw is slower than not using it, other 3Dnow chips dont have it eg the Cyrix MII which may explain a couple of things. I don't
I would consider the MII broken then. setup should clear the 3dnow bit.
Quoted 3 lines
see anywhere we mask the 3Dnow property by these but I've not dug through the CPU code right now to see if we have a "3Dnowplus" type definition we can check.
there is 3dnowext, set on Athlons, but K6 has prefetchw too and it
But if you only want Athlon you can check for X86_FEATURE_K7. The problem is that it doesn't include K8 and K8
has prefetchw too (alternative currently only allows a single bit, not a bitmask). Better is to either clear 3dnow on the MII or define a new pseudo bit that defines working and useful prefetchw
Quoted 9 lines
I suspect the best way to do prefetch cleanly would be something like this
#if
defined(CONFIG_MK7)
alternative_input("prefetch" or "prefetchnta")#else alternative_input(ASM_NOP4 or "prefetchnta");#endif
No for weird combinations you define a new pseudo CPUID capability bit, check for that in the CPU detection and use that in the alternative.
If you really want 3 way alternative you can just define a macro for it. The basic data structure supports it - the macro
just needs to have two .altinstructions records and two replacement codes. But I have my doubt it is worth it for this case.
No stinkin' ifdefs please, that would break the whole concept.
Quoted 5 lines
Ideally we want a 3 way patch table to fix up at boot time but the if case at least gets us back to desirable situations. Also if I remember the prefetch exception thing rightly you can misalign the prefetch instruction as a workaround.
Nope, no misalignment. All it does is to just handle the exception using __ex_table and jumps to the next instruction.
[the exceptions are very rare, they need very specific circumstances in the CPU to trigger, so it's ok to make it slow]
Only trap is that you have to add the exception table sorting too...
-Andi
Quoted 5 lines · Andi Kleen
hw prefetch has nothing to do with how the linux kernel uses prefetch. It's only using it for data structures that cannot be handled by the auto prefetcher.
[except the broken 3dnow! copy that was never enabled]
Not broken in 2.4, although the 2.4-ac kernel uses movntq instead for Athlon as it is faster than mmx_memcpy, which we use for Cyrix/VIA/IDT processors where it is a win.
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Most Athlons in existence should have XMM already and the rest works.
Lots don't have XMM
All XPs have.
And what about all the pre MP/XP ones, lots of those.
Quoted 2 lines
My AMD manual lists it as part of 3dnow. If an CPU advertises 3dnow! but doesn't have the instruction it's broken.
My AMD docs list it as part of the AMD extended 3dnow. The original 3dnow as done by AMD/Cyrix does not have it
Quoted 2 lines
I would consider the MII broken then. setup should clear the 3dnow bit.
"Mummy it doesnt work like I personally have decreed it shall lets break it and screw all the users". Thats the Dan Bernstein school of charm theory of software development.
Quoted 2 lines
there is 3dnowext, set on Athlons, but K6 has prefetchw too and it
3dnowext is what we want here. It might end up doing a prefetchw on K6 but at least K6 actually has the instruction...
Quoted 6 lines
But if you only want Athlon you can check for
X86_FEATURE_K7. The problem is that it doesn't include K8 and K8
has prefetchw too (alternative currently only allows a single bit, not a bitmask). Better is to either clear 3dnow on the MII or define a new pseudo bit that defines working and useful prefetchw
We want a pseudobit - otherwise we'll break other code that checks 3dnow is present properly.
Quoted 8 lines
Quoted 5 lines
#if
defined(CONFIG_MK7)
alternative_input("prefetch" or "prefetchnta")#else alternative_input(ASM_NOP4 or "prefetchnta");#endif
No for weird combinations you define a new pseudo CPUID capability bit, check for that in the CPU detection and use that in the alternative.
Ok
Quoted 4 lines
If you really want 3 way alternative you can just define a macro for it. The basic data structure supports it - the macro
just needs to have two .altinstructions records and two replacement codes. But I have my doubt it is worth it for this case.
prefetching is a big win on older Athlon because the CPU is fast and the chipset/ram sucks hugely relative to it
Quoted 1 line
No stinkin' ifdefs please, that would break the whole concept.
Ok
Quoted 6 lines
Quoted 3 lines
case at least gets us back to desirable situations. Also if I remember the prefetch exception thing rightly you can misalign the prefetch instruction as a workaround.
Nope, no misalignment. All it does is to just handle the exception using
__ex_tableand jumps to the next instruction.
If you misalign the instruction you don't seem to get the exception on Athlon, dunno about the Opteron errata or if the opteron errata bites in 32bit. If it does I guess we should clear mmx, xmm for Opteron by your arguments ;)
Quoted 10 lines · Alan Cox
Quoted 5 lines · Andi Kleen
hw prefetch has nothing to do with how the linux kernel uses prefetch. It's only using it for data structures that cannot be handled by the auto prefetcher.
[except the broken 3dnow! copy that was never enabled]
Not broken in 2.4, although the 2.4-ac kernel uses movntq instead for Athlon as it is faster than
mmx_memcpy, which we use for Cyrix/VIA/IDT processors where it is a win.
movntq for a memcpy? That ise a very bad idea. It wins in micro benchmarks, but the destination is pushed out of cache and the next code accessing the destination will suffer badly from the cache misses. All the NT stuff is basically useless in the kernel because it only helps with data sets significantly bigger than your cache, and we usually only deal with 4K chunks of everything.
[I did the same mistake early on Opteron/x86-64 for copy_page, but later fixed it]
Quoted 5 lines
Quoted 2 lines
My AMD manual lists it as part of 3dnow. If an CPU advertises 3dnow! but doesn't have the instruction it's broken.
My AMD docs list it as part of the AMD extended 3dnow. The original 3dnow as done by AMD/Cyrix does not have it
The x86-64 manuals lists it as part of the 3dnow feature set.
The K6 has it, right?
Is there a "more original" 3dnow that what has been in the K6?
Quoted 5 lines
Quoted 2 lines
I would consider the MII broken then. setup should clear the 3dnow bit.
"Mummy it doesnt work like I personally have decreed it shall lets break it and screw all the users". Thats the Dan Bernstein school of charm
It doesn't work like the AMD instruction reference manual describes it.
Quoted 1 line
theory of software development.
Being a bit touchy from the heat today ? @)
Of course it should be fixed, but the fix as it is a bug workaround doesn't have to be very fast. So it would be ok to just clear the 3dnow bit. But then to handle the K6 case (which is interesting, I didn't know) too it would be probably better to define a separate bit.
Quoted 8 lines
Quoted 5 lines
The problem is that it doesn't include K8 and K8
has prefetchw too (alternative currently only allows a single bit, not a bitmask). Better is to either clear 3dnow on the MII or define a new pseudo bit that defines working and useful prefetchwWe want a pseudobit - otherwise we'll break other code that checks 3dnow is present properly.
Ok. I will do that when I'm back next week unless someone beats me to it ;-)
Quoted 7 lines
Quoted 4 lines
If you really want 3 way alternative you can just define a macro for it. The basic data structure supports it - the macro
just needs to have two .altinstructions records and two replacement codes. But I have my doubt it is worth it for this case.prefetching is a big win on older Athlon because the CPU is fast and the chipset/ram sucks hugely relative to it
Hmm ok. So it probably needs an alternative3().
It's not hard to do, just a bit ugly because the macro will have a lot of arguments.
Quoted 7 lines
Quoted 2 lines
Nope, no misalignment. All it does is to just handle the exception using
__ex_tableand jumps to the next instruction.If you misalign the instruction you don't seem to get the exception on Athlon, dunno about the Opteron errata or if the opteron errata bites in 32bit. If it does I guess we should clear mmx, xmm for Opteron by your arguments ;)
I didn't know about the misalignment bit. Interesting. Misalignment to what boundary?
But is it slower than an aligned execution? If yes I would prefer my solution because it keeps the fast path as fast as possible.
-Andi
Quoted 3 lines · Andi Kleen
stuff is basically useless in the kernel because it only helps with data sets significantly bigger than your cache, and we usually only deal with 4K chunks of everything.
Could be. I didnt write that code. I think Manfred also played with the copy tricks that came from the AMD slides.
Quoted 2 lines
The K6 has it, right?
Is there a "more original" 3dnow that what has been in the K6?
K6-II/III does. I don't know about original K6. but I believe it doesn't. The original 3Dnow was a joint Cyrix/AMD thing and it lacks several instructions later added (including prefetch). The later Cyrix also has a couple of the additional ones but not prefetch.
Quoted 4 lines
Quoted 2 lines
"Mummy it doesnt work like I personally have decreed it shall lets break it and screw all the users". Thats the Dan Bernstein school of charm
It doesn't work like the AMD instruction reference manual describes it.
Well there is a suprise, AMD didn't design it 8)
Quoted 4 lines
Of course it should be fixed, but the fix as it is a bug workaround doesn't have to be very fast. So it would be ok to just clear the 3dnow bit. But then to handle the K6 case (which is interesting, I didn't know) too it would be probably better to define a separate bit.
What else checks the 3Dnow bit ?
Quoted 5 lines
Quoted 2 lines
We want a pseudobit - otherwise we'll break other code that checks 3dnow is present properly.
Ok. I will do that when I'm back next week unless someone beats me to it ;-)
Some kind of "has prefetch and its actually useful" 8)
Quoted 7 lines
Quoted 4 lines
If you misalign the instruction you don't seem to get the exception on Athlon, dunno about the Opteron errata or if the opteron errata bites in 32bit. If it does I guess we should clear mmx, xmm for Opteron by your arguments ;)
I didn't know about the misalignment bit. Interesting. Misalignment to what boundary?
I'll have to go check again. Its something RH internal testing found when people were going "uh what the hell is going on here" 8)
Quoted 2 lines
But is it slower than an aligned execution? If yes I would prefer my solution because it keeps the fast path as fast as possible.
Has AMD confirmed that your solution is ok for the K7 as well as K8 - ie that if we hit the errata the fixup recovers the CPU from whatever lunatic state it is now in ?
Quoted 7 lines · Alan Cox
Quoted 3 lines · Andi Kleen
stuff is basically useless in the kernel because it only helps with data sets significantly bigger than your cache, and we usually only deal with 4K chunks of everything.
Could be. I didnt write that code. I think Manfred also played with the copy tricks that came from the AMD slides.
The AMD slides assume all very big data sets ;-)
I would recommend to remove it.
Quoted 6 lines
Quoted 4 lines
Of course it should be fixed, but the fix as it is a bug workaround doesn't have to be very fast. So it would be ok to just clear the 3dnow bit. But then to handle the K6 case (which is interesting, I didn't know) too it would be probably better to define a separate bit.
What else checks the 3Dnow bit ?
Nothing in kernel AFAIK, but it's possible that it is used by user space reading /proc/cpuinfo.
Quoted 5 lines
Quoted 3 lines
Ok. I will do that when I'm back next week unless someone beats me to it ;-)
Some kind of "has prefetch and its actually useful" 8)
X86_FEATURE_PREFETCHWX86_FEATURE_PREFETCH3DNOW
(note I didn't volunteer to write alternative3 for the later, someone else has to do that if they want it ;-)
Quoted 6 lines
Quoted 2 lines
But is it slower than an aligned execution? If yes I would prefer my solution because it keeps the fast path as fast as possible.
Has AMD confirmed that your solution is ok for the K7 as well as K8 - ie that if we hit the errata the fixup recovers the CPU from whatever lunatic state it is now in ?
My solution is a fix as the problem is described in the Opteron Specification Update (and also as our own testing showed - we discovered the problem originally)
The Errata is basically: When there is a prefetch and a load for the same address in flight and the load faults and the CPU is in a very specific complicated state then the Exception is reported on the prefetch, not the fault.
The fix just handles the exception and doesn't crash.
At least on Opteron it can be also fixed with a magic bit in the BIOS, maybe that's possible on XP too. But I opted to work around it in the kernel to not force all people to get a new BIOS.
BTW we saw it mainly in the x86-64 copy_*_user and csum_copy_* functions which do also prefetches. LTP would sometimes trigger it when it tests how the kernel behaves with invalid addresses. But it happened very rarely in the dcache hash too. But still it's hard to trigger, the linked list one is very hard to hit. I tried to reproduce it in user space, but failed. The LTP one is much easier, but still not that common.
-Andi
Quoted 3 lines · Andi Kleen
The AMD slides assume all very big data sets ;-)
I would recommend to remove it.
I'll do some timings when I get a moment - the prefetching mmx copy was a win (and faster than the others for small data as well as large on the K7-550 (really a K7 not "Athlon" 8)) way back when.
Quoted 4 lines
Quoted 1 line
What else checks the 3Dnow bit ?
Nothing in kernel AFAIK, but it's possible that it is used by user space reading
/proc/cpuinfo.
DaveJ and your docs are right on 3dnow it turns out so sorry about that and ignore me on prefetchw, its just the prefetch side thats 3 way.
Quoted 6 lines
BTW we saw it mainly in the x86-64 copy_*_user and csum_copy_* functions which do also prefetches. LTP would sometimes trigger it when it tests how the kernel behaves with invalid addresses. But it happened very rarely in the dcache hash too. But still it's hard to trigger, the linked list one is very hard to hit. I tried to reproduce it in user space, but failed. The LTP one is much easier, but still not that common.
Thanks
Quoted 6 lines · Alan Cox
Quoted 3 lines · Andi Kleen
The AMD slides assume all very big data sets ;-)
I would recommend to remove it.
I'll do some timings when I get a moment - the prefetching mmx copy
Microbenchmarks are useless for this. You have to bench the users too, otherwise you don't recognize the additional cache misses.
Quoted 2 lines
was a win (and faster than the others for small data as well as large on the K7-550 (really a K7 not "Athlon" 8)) way back when.
Possible. In my experience the best copy functions vary widly between different steppings. Just optimizing for a single one is probably not a good idea, especially not for an very old one (except when you add dynamic patches for the different steppings, but then it quickly gets ugly with too many variants)
When in doubt use the more simple function.
-Andi
Quoted 4 lines · Alan Cox
K6-II/III does. I don't know about original K6. but I believe it doesn't. The original 3Dnow was a joint Cyrix/AMD thing and it lacks several instructions later added (including prefetch). The later Cyrix also has a couple of the additional ones but not prefetch.
Which Cyrixen are you talking about ?
C3's up to and including Ezra-T should DTRT when it comes to 3dnow prefetch instruction, and pre-VIA Cyrixen didn't have 3dnow at all iirc.
Dave
Signature · 1 line
Dave Jones http://www.codemonkey.org.uk
Quoted 2 lines · Dave Jones
Quoted 2 lines
several instructions later added (including prefetch). The later Cyrix also has a couple of the additional ones but not prefetch.
Ok I got this crossed - the Cyrix/AMD thing was the extended MMX stuff, they both did 3Dnow! but the Jalapeno old style Cyrix CPU with 3dnow was canned. Ok so there is a different reason why my Cyrix crashes on boot with 2.6test. Andi is right that 3Dnow safely implies prefetch. My docs list it as part of extended MMX not 3dnow although the Cyrix seems to not posess the instruction anyway.
So 3dnow == prefetch/prefetchw is ok but not useful on K6.
Quoted 4 lines
Which Cyrixen are you talking about ?
C3's up to and including Ezra-T should DTRT when it comes to 3dnow prefetch instruction, and pre-VIA Cyrixen didn't have 3dnow at all iirc.
pre VIA Cyrixen have MMX and CXMMX. The CPU also set bit 31 but doesn't have 3dnow (which fooled me but the kernel does know about). C3's seem to have prefetch/prefetchw (but not prefetchnta). I don't have a nemeiah but I assume Nemeiah has prefetchnta too ?
I've tried building a summary list. Additional contributions welcomed
MMX: Pentium (later only), Cyrix MediaGX (later only), Cyrix 6x86/MII
Intel PII/PIII/PIV, AMD K6/Athlon/Opteron, VIA Cyrix III, VIA C3
CXMMX: Extended MMX - Cyrix MII/AMD K6(II+ ?)/K7/Opteron 3DNOW: AMD K6-II/III(not original K6),K7/,Opteron, VIA Cyrix III, VIA C3 (pre Nemiah only ??)
"Enhanced" 3DNow: Athlon Tbird
SSE: Intel PII, PIII, Athlon (XP, Duron >=1Gz only)
SSE2: Pentium IV
So the prefetch fallback is needed for pre Nemiah C3, Duron < 1Ghz and pre T-Bird Athlon if my table is right.
Quoted 4 lines · Alan Cox
pre VIA Cyrixen have MMX and CXMMX. The CPU also set bit 31 but doesn't have 3dnow (which fooled me but the kernel does know about). C3's seem to have prefetch/prefetchw (but not prefetchnta). I don't have a nemeiah but I assume Nemeiah has prefetchnta too ?
With Nehemiah, they dropped 3dnow, and went with SSE.
Quoted 5 lines
MMX: Pentium (later only), Cyrix MediaGX (later only), Cyrix 6x86/MII
Intel PII/PIII/PIV, AMD K6/Athlon/Opteron, VIA Cyrix III, VIA C3
CXMMX: Extended MMX - Cyrix MII/AMD K6(II+ ?)/K7/Opteron 3DNOW: AMD K6-II/III(not original K6),K7/,Opteron, VIA Cyrix III, VIA C3 (pre Nemiah only ??)
"Nehemiah".
+ Winchip-2A (though as mentioned, prefetch is a nop, the rest of 3dnow worked though iirc).
Quoted 3 lines
"Enhanced" 3DNow: Athlon Tbird
SSE: Intel PII, PIII, Athlon (XP, Duron >=1Gz only)
SSE2: Pentium IV
Dave
Signature · 1 line
Dave Jones http://www.codemonkey.org.uk
And one other update
Winchip C6 - MMX, extended MMX
Winchip II+ , MMX, extended MMX, 3Dnow (dunno if it has prefetch I don't have one of these)