no more MTRRs available ?
Hello all,
what exactly does
mtrr: no more MTRRs available
mtrr: no more MTRRs available
during boot mean? What can I do against this? This comes up while booting a system with 6GB and P-III 1.4 GHz (Serverworks chipset). Kernel is 2.4.20.
Thanks for any hints.
Signature · 2 lines
Regards, Stephan
A message that is not in the archive (sent off-list or not yet received): Pine.LNX.4.44.0301291025240.18828-100000@coffee.psychology.mcmaster.ca
Quoted 9 lines · On Wed, 29 Jan 2003 10:25:57 -0500 (EST) Mark Hahn <[email protected]>…
Quoted 7 lines
what exactly does
mtrr: no more MTRRs available
mtrr: no more MTRRs availableduring boot mean? What can I do against this? This comes up while booting a system with 6GB and P-III 1.4 GHz (Serverworks chipset). Kernel is 2.4.20.
you need to look at
/proc/mtrr.
Thanks for your hint, but what does this tell me?
# cat /proc/mtrr reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1 reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1 reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1 reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1 reg04: base=0xf0000000 (3840MB), size= 128MB: write-back, count=1 reg05: base=0xf7000000 (3952MB), size= 16MB: uncachable, count=1 reg06: base=0x100000000 (4096MB), size=4096MB: write-back, count=1 reg07: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
Signature · 2 lines
Regards, Stephan
A message that is not in the archive (sent off-list or not yet received): Pine.LNX.4.44.0301291046490.18828-100000@coffee.psychology.mcmaster.ca
Quoted 11 lines · On Wed, 29 Jan 2003 10:51:03 -0500 (EST) Mark Hahn <[email protected]>…
Quoted 11 lines
Quoted 10 lines
Quoted 8 lines
what exactly does
mtrr: no more MTRRs available
mtrr: no more MTRRs availableduring boot mean? What can I do against this? This comes up while booting a system with 6GB and P-III 1.4 GHz (Serverworks chipset). Kernel is 2.4.20.
you need to look at
/proc/mtrr.
Quoted 7 lines
Quoted 1 line
Thanks for your hint, but what does this tell me?
that your bios is stupid, I think. mtrr's handle areas that are powers of two in size (and >= 1M, I think). the problem here is that the bios is trying to represent 4G of write-back ram and a 16M of uncachable IO area (AGP aperture, I'm guessing). the correct way to do this is a single 4G mtrr with an overlapping 16M one.
Ah, I see. So getting the above two messages would simply mean that there is no table space left to add yet another two entries kernel gets from bios?
Quoted 1 line
do you have >4G ram? that would explain the latter two.
Yes, this box has 6GB.
Quoted 3 lines
note that you can fairly freely add/delete mtrr's from userspace. as long as you do it infrequently, I don't see why there would be any risk or performance problem.
So the conclusion is that it does not look nice currently, but does not do any harm (performance loss) either. I can live with that.
Thanks for your comments.
Quoted 9 lines
Quoted 9 lines
# cat /proc/mtrr reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1 reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1 reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1 reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1 reg04: base=0xf0000000 (3840MB), size= 128MB: write-back, count=1 reg05: base=0xf7000000 (3952MB), size= 16MB: uncachable, count=1 reg06: base=0x100000000 (4096MB), size=4096MB: write-back, count=1 reg07: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
Signature · 2 lines
Regards, Stephan
Quoted 1 line · Stephan von Krawczynski
Mark Hahn <[email protected]> wrote:
(odd, I'm not getting Mark's mails to l-k).
Quoted 5 lines
Quoted 5 lines
that your bios is stupid, I think. mtrr's handle areas that are powers of two in size (and >= 1M, I think). the problem here is that the bios is trying to represent 4G of write-back ram and a 16M of uncachable IO area (AGP aperture, I'm guessing). the correct way to do this is a single 4G mtrr with an overlapping 16M one.
That does seem to make more sense. Perhaps too much sense. ISTR there were ordering rules in how you layer MTRRs on top of each other.
Dave
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs
Quoted 5 lines · On Wed, 29 Jan 2003 16:20:13 +0000 Dave Jones <[email protected]> wrote:
Quoted 1 line · Stephan von Krawczynski
Mark Hahn <[email protected]> wrote:
(odd, I'm not getting Mark's mails to l-k).
He does not send them there. I know it is a no-no to forward his writings to the list, but I thought it may be of use to others, too (and he did not shoot me for it so far ;-)
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
that your bios is stupid, I think. mtrr's handle areas that are powers of two in size (and >= 1M, I think). the problem here is that the bios is trying to represent 4G of write-back ram and a 16M of uncachable IO area (AGP aperture, I'm guessing). the correct way to do this is a single 4G mtrr with an overlapping 16M one.
That does seem to make more sense. Perhaps too much sense. ISTR there were ordering rules in how you layer MTRRs on top of each other.
Ok, after reading your thoughts and re-reading the bios stuff I found and "DRAM alignment" option, and voila:
reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1
reg01: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg02: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
reg03: base=0x400000000 (16384MB), size=16384MB: write-back, count=1
reg04: base=0x800000000 (32768MB), size=32768MB: write-back, count=1
reg05: base=0xfb000000 (4016MB), size= 8MB: write-combining, count=1
The table got obviously shorter and significantly "better". It seems the onboard VGA stuff is now mapped out of the normal DRAM range.
I must admit though that some of this bios looks fishy, because now I cannot find the original DRAM caching entries any longer, no matter what I switch on or off (you know those "write-back, write-through" stuff).
Now I think one can really be content with this setup. Thanks a lot to Mark and Dave.
In case anyone is interested, this is an Asus TRL-DLS mb.
Signature · 2 lines
Regards, Stephan
Quoted 7 lines · Stephan von Krawczynski
# cat /proc/mtrr reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1 reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1 reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1 reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1 reg04: base=0xf0000000 (3840MB), size= 128MB: write-back, count=1 reg05: base=0xf7000000 (3952MB), size= 16MB: uncachable, count=1
Due to this 16MB hole, your BIOS has to set up a write-back range covering 2048-16 (2032MB). Due to limitations on the way MTRRs work, you can't do this all in 1, so you have to split it up over several. The result is that you use up 5 MTRRs covering 1 range + 1 for the unmapped 16MB. Leaving just two for..
Quoted 2 lines
reg06: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg07: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
So you really are out of MTRRs (There are only 8 on this CPU). Does your BIOS have any option to disable this hole in low memory ? I've seen similar things on laptops that use shared memory between CPU & Graphics card, but I was unaware of anything like this on desktop/server motherboards.
If this has onboard VGA, and theres a way to disable it, perhaps that will work. (Even if it means plugging in a PCI/AGP video card)
Dave
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs
Quoted 9 lines · Dave Jones
Quoted 7 lines
# cat /proc/mtrr reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1 reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1 reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1 reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1 reg04: base=0xf0000000 (3840MB), size= 128MB: write-back, count=1 reg05: base=0xf7000000 (3952MB), size= 16MB: uncachable, count=1Due to this 16MB hole, your BIOS has to set up a write-back range covering 2048-16 (2032MB).
Dummies guide to maths. This should be 4096-16=4080 of course
Dave
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs
Quoted 9 lines · Stephan von Krawczynski
# cat /proc/mtrr reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1 reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1 reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1 reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1 reg04: base=0xf0000000 (3840MB), size= 128MB: write-back, count=1 reg05: base=0xf7000000 (3952MB), size= 16MB: uncachable, count=1 reg06: base=0x100000000 (4096MB), size=4096MB: write-back, count=1 reg07: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
Looks better than what I'm getting on 2.5.59:
curly:~# cat /proc/mtrr reg00: base=0xc0000000 (49152MB), size=16384MB: uncachable, count=1 reg01: base=0x00000000 ( 0MB), size=524288MB: write-back, count=1 reg02: base=0x800000000 (524288MB), size=262144MB: write-back, count=1
Yes, this is standard ia32 (P-III/Coppermine cpus), and hence the numbers here are utter garbage.
-- wli
Quoted 8 lines · William Lee Irwin III
Looks better than what I'm getting on 2.5.59:
curly:~# cat /proc/mtrr reg00: base=0xc0000000 (49152MB), size=16384MB: uncachable, count=1 reg01: base=0x00000000 ( 0MB), size=524288MB: write-back, count=1 reg02: base=0x800000000 (524288MB), size=262144MB: write-back, count=1 Yes, this is standard ia32 (P-III/Coppermine cpus), and hence the numbers here are utter garbage.
Bizarre. The size field isn't being shifted, and your base is somewhere off in 64bit land.
See Andi's "RED-PEN" comments in various parts of arch/i386/kernel/cpu/mtrr/ They need fixing at some point, and could be the cause of your problems.
Dave
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs
Quoted 7 lines · William Lee Irwin III
Quoted 7 lines
Looks better than what I'm getting on 2.5.59:
curly:~# cat /proc/mtrr reg00: base=0xc0000000 (49152MB), size=16384MB: uncachable, count=1 reg01: base=0x00000000 ( 0MB), size=524288MB: write-back, count=1 reg02: base=0x800000000 (524288MB), size=262144MB: write-back, count=1 Yes, this is standard ia32 (P-III/Coppermine cpus), and hence the numbers here are utter garbage.
Quoted 4 lines · Dave Jones
Bizarre. The size field isn't being shifted, and your base is somewhere off in 64bit land.
See Andi's "RED-PEN" comments in various parts ofarch/i386/kernel/cpu/mtrr/They need fixing at some point, and could be the cause of your problems.
OTOH since 0-512GB are in there this explains why the (massive) perf. decrease only happens sometimes. The MTRR corruption issues are only visible with 48GB atm. I haven't been focusing on MTRR's but I may arrange to trace the codepaths etc. in the eventual future to find where the bits are going bad esp. as benchmark time approaches.
-- wli
Quoted 12 lines · William Lee Irwin III
Quoted 5 lines
Quoted 5 lines
reg00: base=0xc0000000 (49152MB), size=16384MB: uncachable, count=1
reg01: base=0x00000000 ( 0MB), size=524288MB: write-back, count=1
reg02: base=0x800000000 (524288MB), size=262144MB: write-back, count=1 Yes, this is standard ia32 (P-III/Coppermine cpus), and hence the numbers here are utter garbage.Quoted 4 lines · Dave Jones
Bizarre. The size field isn't being shifted, and your base is somewhere off in 64bit land.
See Andi's "RED-PEN" comments in various parts ofarch/i386/kernel/cpu/mtrr/They need fixing at some point, and could be the cause of your problems.
Quoted 5 lines
OTOH since 0-512GB are in there this explains why the (massive) perf. decrease only happens sometimes. The MTRR corruption issues are only visible with 48GB atm. I haven't been focusing on MTRR's but I may arrange to trace the codepaths etc. in the eventual future to find where the bits are going bad esp. as benchmark time approaches.
ohhh, 48GB. I forgot you were doing the silly-amounts-of-mem thing. Its extremely likely you'll have to fix up those comments Andi made, and possibly some other parts too. That code should be 64bit clean now (due to x86-64 sharing it), but there may still be some gotchas, especially on weird-ass systems like what you've been playing with 8)
Dave
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs