Intel-Quad on GA-P35-S3 motherboard with 4*2GB
I can't see whole 8GB of ram.
With F2 BIOS release i can only work with kernel param mem=4G. After updating to F4 BIOS release I can work with mem=8G, but I see this:
# free -m
total used free shared buffers cached
Mem: 6473 474 5999 0 29 278Without mem=8G kernel starts to slow down and hangs while starting filesystem. There is no message.
My whole memory was tested with memtest. Full test took about 48hour. Is it wrong?
I have: Intel(R) Core(TM)2 Quad CPU @ 2.40GHz
There are 4 * 2GB / 667MHz DIMMs.
# cat /proc/iomem |grep RAM 00000000-0009f7ff : System RAM 00100000-9fedffff : System RAM 100000000-25fffffff : System RAM
# dmesg|grep BIOS-e820
BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000009fee0000 (usable)
BIOS-e820: 000000009fee0000 - 000000009fee3000 (ACPI NVS)
BIOS-e820: 000000009fee3000 - 000000009fef0000 (ACPI data)
BIOS-e820: 000000009fef0000 - 000000009ff00000 (reserved)
BIOS-e820: 00000000c0000000 - 00000000c4000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
I can see that higer memory range was remmaped,
by why accesing something greater than 100000000 is problematic? Do I need to change something i kernel configuration?
Quoted 39 lines · Wojciech Kromer
I can't see whole 8GB of ram.
With F2 BIOS release i can only work with kernel param mem=4G. After updating to F4 BIOS release I can work with mem=8G, but I see this:
# free -m total used free shared buffers cached Mem: 6473 474 5999 0 29 278Without mem=8G kernel starts to slow down and hangs while starting filesystem.
There is no message.My whole memory was tested with memtest. Full test took about 48hour. Is it wrong?
I have: Intel(R) Core(TM)2 Quad CPU @ 2.40GHz
There are 4 * 2GB / 667MHz DIMMs.# cat /proc/iomem |grep RAM 00000000-0009f7ff : System RAM 00100000-9fedffff : System RAM 100000000-25fffffff : System RAM# dmesg|grep BIOS-e820
BIOS-e820: 0000000000000000 -000000000009f800(usable)
BIOS-e820:000000000009f800-00000000000a0000(reserved)
BIOS-e820:00000000000f0000- 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 -000000009fee0000(usable)
BIOS-e820:000000009fee0000-000000009fee3000(ACPI NVS)
BIOS-e820:000000009fee3000-000000009fef0000(ACPI data)
BIOS-e820:000000009fef0000-000000009ff00000(reserved)
BIOS-e820:00000000c0000000-00000000c4000000(reserved)
BIOS-e820:00000000fec00000- 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
Quoted 3 lines
I can see that higer memory range was remmaped,
by why accesing something greater than 100000000 is problematic? Do I need to change something i kernel configuration?
What does your mtrr look like? How about dmesg? Might be the stupid mtrr setup some bioses have been doing on intel chips where it would do:
4GB range at 4GB
1GB range at 8GB
512MB range at 9GB
256MB range at 9.5GB
etc.
And then it runs out of entries (which pisses of X).
The simple solution would have been to assign an 8GB range at 4GB or maybe a 4GB range at 4GB and a 2GB range at 8GB to cover all the ram. Without the mtrr coverage you get no caching of that memory which makes use that that section of memory slow. To make the system useable there is code in the kernel to discard any memory the BIOS didn't correctly cover with an mtrr cachable entry.
--
Len Sorensen
Quoted 9 lines
What does your mtrr look like? How about dmesg? Might be the stupid mtrr setup some bioses have been doing on intel chips where it would do:
4GB range at 4GB
1GB range at 8GB
512MB range at 9GB
256MB range at 9.5GB
etc.
Quoted 11 lines
And then it runs out of entries (which pisses of X).
The simple solution would have been to assign an 8GB range at 4GB or maybe a 4GB range at 4GB and a 2GB range at 8GB to cover all the ram. Without the mtrr coverage you get no caching of that memory which makes use that that section of memory slow. To make the system useable there is code in the kernel to discard any memory the BIOS didn't correctly cover with an mtrr cachable entry.
I have 2.6.22.6 kernel
First MTRR looks good for me.
Why it was rejected?
And how to force using it?
here are more details
[root@kromblack ~]# cat /proc/mtrr
reg00: base=0x100000000 (4096MB), size=8192MB: write-back, count=1
reg01: base=0x280000000 (10240MB), size=2048MB: uncachable, count=1
reg02: base=0x260000000 (9728MB), size= 512MB: uncachable, count=1
reg03: base=0x00000000 ( 0MB), size=4096MB: write-back, count=1
reg04: base=0xc0000000 (3072MB), size=1024MB: uncachable, count=1
reg05: base=0xa0000000 (2560MB), size= 512MB: uncachable, count=1
reg06: base=0x9ff00000 (2559MB), size= 1MB: write-through, count=1
[root@kromblack ~]# dmesg|grep mtrr
mtrr: your CPUs had inconsistent variable MTRR settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.
and other interesting dmesg fragments:
Entering add_active_range(0, 0, 159) 0 entries of 3200 used Entering add_active_range(0, 256, 655072) 1 entries of 3200 used Entering add_active_range(0, 1048576, 2097152) 2 entries of 3200 used end_pfn_map = 2097152
DMI 2.4 present.
ACPI: RSDP 000F6F10, 0014 (r0 GBT )
ACPI: RSDT 9FEE3040, 003C (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: FACP 9FEE30C0, 0074 (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: DSDT 9FEE3180, 4A88 (r1 GBT GBTUACPI 1000 MSFT 100000C)
ACPI: FACS 9FEE0000, 0040
ACPI: HPET 9FEE7D80, 0038 (r1 GBT GBTUACPI 42302E31 GBTU 98)
ACPI: MCFG 9FEE7E00, 003C (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: APIC 9FEE7C80, 0084 (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: SSDT 9FEE7E80, 015C (r1 PmRef Cpu0Ist 3000 INTL 20040311)
ACPI: SSDT 9FEE8430, 0275 (r1 PmRef CpuPm 3000 INTL 20040311)
No NUMA configuration found
Faking a node at 0000000000000000-0000000200000000 Entering add_active_range(0, 0, 159) 0 entries of 3200 used Entering add_active_range(0, 256, 655072) 1 entries of 3200 used Entering add_active_range(0, 1048576, 2097152) 2 entries of 3200 used Bootmem setup node 0 0000000000000000-0000000200000000 Zone PFN ranges:
DMA 0 -> 4096
DMA32 4096 -> 1048576
Normal 1048576 -> 2097152
early_node_map[3] active PFN ranges
0: 0 -> 159
0: 256 -> 655072
0: 1048576 -> 2097152
On node 0 totalpages: 1703551
DMA zone: 56 pages used for memmap
DMA zone: 1125 pages reserved
DMA zone: 2818 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 636696 pages, LIFO batch:31
Normal zone: 14336 pages used for memmap
Normal zone: 1034240 pages, LIFO batch:31
ACPI: HPET id: 0x8086a201 base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
swsusp: Registered nosave memory region: 000000000009f000 - 00000000000a0000
swsusp: Registered nosave memory region: 00000000000a0000 - 00000000000f0000
swsusp: Registered nosave memory region: 00000000000f0000 - 0000000000100000
swsusp: Registered nosave memory region: 000000009fee0000 - 000000009fee3000
swsusp: Registered nosave memory region: 000000009fee3000 - 000000009fef0000
swsusp: Registered nosave memory region: 000000009fef0000 - 000000009ff00000
swsusp: Registered nosave memory region: 000000009ff00000 - 00000000c0000000
swsusp: Registered nosave memory region: 00000000c0000000 - 00000000c4000000
swsusp: Registered nosave memory region: 00000000c4000000 - 00000000fec00000
swsusp: Registered nosave memory region: 00000000fec00000 - 0000000100000000 Allocating PCI resources starting at c8000000 (gap: c4000000:3ac00000)
SMP: Allowing 4 CPUs, 0 hotplug CPUs
PERCPU: Allocating 36744 bytes of per cpu data
Built 1 zonelists. Total pages: 1673754
Quoted 8 lines · Wojciech Kromer
I have 2.6.22.6 kernel
First MTRR looks good for me.
Why it was rejected?
And how to force using it?here are more details
Quoted 8 lines
[root@kromblack ~]# cat
/proc/mtrr
reg00: base=0x100000000 (4096MB), size=8192MB: write-back, count=1
reg01: base=0x280000000 (10240MB), size=2048MB: uncachable, count=1
reg02: base=0x260000000 (9728MB), size= 512MB: uncachable, count=1
reg03: base=0x00000000 ( 0MB), size=4096MB: write-back, count=1
reg04: base=0xc0000000 (3072MB), size=1024MB: uncachable, count=1
reg05: base=0xa0000000 (2560MB), size= 512MB: uncachable, count=1
reg06: base=0x9ff00000 (2559MB), size= 1MB: write-through, count=1
So first it says there is 8GB ram at 4GB. Then it deletes the top 2GB of that and then another 512MB. So now there is 5.5GB at 4GB. It then says there is 4GB at 0GB, then deletes the top 1GB of that, and then another 512MB, leaving 2.5GB at 0GB. So overall that sounds like 8GB total. It also seems that if it had mapped 2GB at 0GB and 6GB at 4GB the MTRR would have been a heck of a lot simpler (but a 32bit OS would have had 512MB less ram of course).
Quoted 4 lines
[root@kromblack ~]# dmesg|grep mtrr
mtrr: your CPUs had inconsistent variable MTRR settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.
Now that does sound odd. How many CPUs does the machine have?
Quoted 59 lines
and other interesting dmesg fragments:
Entering
add_active_range(0, 0, 159)0 entries of 3200 used Enteringadd_active_range(0, 256, 655072)1 entries of 3200 used Enteringadd_active_range(0, 1048576, 2097152)2 entries of 3200 usedend_pfn_map= 2097152
DMI 2.4 present.
ACPI: RSDP 000F6F10, 0014 (r0 GBT )
ACPI: RSDT 9FEE3040, 003C (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: FACP 9FEE30C0, 0074 (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: DSDT 9FEE3180, 4A88 (r1 GBT GBTUACPI 1000 MSFT 100000C)
ACPI: FACS 9FEE0000, 0040
ACPI: HPET 9FEE7D80, 0038 (r1 GBT GBTUACPI 42302E31 GBTU 98)
ACPI: MCFG 9FEE7E00, 003C (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: APIC 9FEE7C80, 0084 (r1 GBT GBTUACPI 42302E31 GBTU 1010101)
ACPI: SSDT 9FEE7E80, 015C (r1 PmRef Cpu0Ist 3000 INTL 20040311)
ACPI: SSDT 9FEE8430, 0275 (r1 PmRef CpuPm 3000 INTL 20040311)
No NUMA configuration found
Faking a node at 0000000000000000-0000000200000000 Enteringadd_active_range(0, 0, 159)0 entries of 3200 used Enteringadd_active_range(0, 256, 655072)1 entries of 3200 used Enteringadd_active_range(0, 1048576, 2097152)2 entries of 3200 used Bootmem setup node 0 0000000000000000-0000000200000000 Zone PFN ranges:
DMA 0 -> 4096
DMA32 4096 -> 1048576
Normal 1048576 -> 2097152
early_node_map[3] active PFN ranges
0: 0 -> 159
0: 256 -> 655072
0: 1048576 -> 2097152
On node 0 totalpages: 1703551
DMA zone: 56 pages used for memmap
DMA zone: 1125 pages reserved
DMA zone: 2818 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 636696 pages, LIFO batch:31
Normal zone: 14336 pages used for memmap
Normal zone: 1034240 pages, LIFO batch:31ACPI: HPET id:
0x8086a201base:0xfed00000
Using ACPI (MADT) for SMP configuration information
swsusp: Registered nosave memory region:000000000009f000-00000000000a0000
swsusp: Registered nosave memory region:00000000000a0000-00000000000f0000
swsusp: Registered nosave memory region:00000000000f0000- 0000000000100000
swsusp: Registered nosave memory region:000000009fee0000-000000009fee3000
swsusp: Registered nosave memory region:000000009fee3000-000000009fef0000
swsusp: Registered nosave memory region:000000009fef0000-000000009ff00000
swsusp: Registered nosave memory region:000000009ff00000-00000000c0000000
swsusp: Registered nosave memory region:00000000c0000000-00000000c4000000
Quoted 8 lines
swsusp: Registered nosave memory region:
00000000c4000000-00000000fec00000
swsusp: Registered nosave memory region:00000000fec00000- 0000000100000000
Allocating PCI resources starting atc8000000(gap: c4000000:3ac00000)
SMP: Allowing 4 CPUs, 0 hotplug CPUs
PERCPU: Allocating 36744 bytes of per cpu data
Built 1 zonelists. Total pages: 1673754
So what does the dmesg say about the memory table it got from the bios? That is the e820 table.
--
Len Sorensen
Quoted 12 lines
Quoted 10 lines
[root@kromblack ~]# cat
/proc/mtrr
reg00: base=0x100000000 (4096MB), size=8192MB: write-back, count=1
reg01: base=0x280000000 (10240MB), size=2048MB: uncachable, count=1
reg02: base=0x260000000 (9728MB), size= 512MB: uncachable, count=1
reg03: base=0x00000000 ( 0MB), size=4096MB: write-back, count=1
reg04: base=0xc0000000 (3072MB), size=1024MB: uncachable, count=1
reg05: base=0xa0000000 (2560MB), size= 512MB: uncachable, count=1
reg06: base=0x9ff00000 (2559MB), size= 1MB: write-through, count=1
Yes, it's a bit strange, but there should be a way to configure it in linux. I found only memmap option which does selection of regions, but not remapping.
Quoted 9 lines
So first it says there is 8GB ram at 4GB. Then it deletes the top 2GB of that and then another 512MB. So now there is 5.5GB at 4GB. It then says there is 4GB at 0GB, then deletes the top 1GB of that, and then another 512MB, leaving 2.5GB at 0GB. So overall that sounds like 8GB total. It also seems that if it had mapped 2GB at 0GB and 6GB at 4GB the MTRR would have been a heck of a lot simpler (but a 32bit OS would have had 512MB less ram of course).
I have 64 bit compilation of course.
Quoted 9 lines
Quoted 5 lines
[root@kromblack ~]# dmesg|grep mtrr
mtrr: your CPUs had inconsistent variable MTRR settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.Now that does sound odd. How many CPUs does the machine have?
It's Intel-quad, so there are 4 CPUs.
Quoted 3 lines
So what does the dmesg say about the memory table it got from the bios? That is the e820 table.
It's in prevoius mail:
# dmesg|grep BIOS-e820
Quoted 10 lines
BIOS-e820: 0000000000000000 -
000000000009f800(usable)
BIOS-e820:000000000009f800-00000000000a0000(reserved)
BIOS-e820:00000000000f0000- 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 -000000009fee0000(usable)
BIOS-e820:000000009fee0000-000000009fee3000(ACPI NVS)
BIOS-e820:000000009fee3000-000000009fef0000(ACPI data)
BIOS-e820:000000009fef0000-000000009ff00000(reserved)
BIOS-e820:00000000c0000000-00000000c4000000(reserved)
BIOS-e820:00000000fec00000- 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
Quoted 3 lines · Wojciech Kromer
Yes, it's a bit strange, but there should be a way to configure it in linux. I found only memmap option which does selection of regions, but not remapping.
No it is entirely the BIOS's job to do that. It would be very hard for linux to safely change that.
Quoted 1 line
I have 64 bit compilation of course.
So in your case it would have been just fine with a simpler mapping.
Quoted 1 line
It's Intel-quad, so there are 4 CPUs.
Weird. The message about the MTRR on all the CPUs not maching sound like a strange bios bug. Are you running the latest bios version?
Quoted 13 lines
It's in prevoius mail:
# dmesg|grep BIOS-e820
Quoted 10 lines
BIOS-e820: 0000000000000000 -
000000000009f800(usable)
BIOS-e820:000000000009f800-00000000000a0000(reserved)
BIOS-e820:00000000000f0000- 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 -000000009fee0000(usable)
BIOS-e820:000000009fee0000-000000009fee3000(ACPI NVS)
BIOS-e820:000000009fee3000-000000009fef0000(ACPI data)
BIOS-e820:000000009fef0000-000000009ff00000(reserved)
BIOS-e820:00000000c0000000-00000000c4000000(reserved)
BIOS-e820:00000000fec00000- 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
Looks to match the MTRR setup at least. So how much ram does the kernel actually want to use?
--
Len Sorensen
Quoted 9 lines
Quoted 4 lines
It's Intel-quad, so there are 4 CPUs.
Weird. The message about the MTRR on all the CPUs not maching sound like a strange bios bug. Are you running the latest bios version?
See the story below.
Quoted 4 lines
Looks to match the MTRR setup at least. So how much ram does the kernel actually want to use?
a)With mem=8GB parameter I had:
#free -m
total used free shared buffers cached
Mem: 6473 474 5999 0 29 278
b) Without mem=8GM system slows down while booting and sometimes restart...
==== So the story begins ===========================
It's strange GIGABYTE missinformation, but:
- my board came with F2 release
- after detecting memory problem, I've upgraded it to F4 (latest at this moment)
- next day came with F5 release ,
there was only one info: "Fix PS2 keyboard compatibility issues", so I decided *not* to upgrade BIOS
But yesterday I've finally upgraded to F5 release. ... and here is *surrprise*: MTRR changed to:
#cat /proc/mtrr
reg00: base=0x00000000 ( 0MB), size=4096MB: write-back, count=1
reg01: base=0xc0000000 (3072MB), size=1024MB: uncachable, count=1
reg02: base=0xa0000000 (2560MB), size= 512MB: uncachable, count=1
reg03: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg04: base=0x200000000 (8192MB), size=2048MB: write-back, count=1
reg05: base=0x260000000 (9728MB), size= 512MB: uncachable, count=1
reg06: base=0x9ff00000 (2559MB), size= 1MB: write-through, count=1
Now there are no complains about MTRR in dmesg.
E820 seems to be the same:
BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000009fee0000 (usable)
BIOS-e820: 000000009fee0000 - 000000009fee3000 (ACPI NVS)
BIOS-e820: 000000009fee3000 - 000000009fef0000 (ACPI data)
BIOS-e820: 000000009fef0000 - 000000009ff00000 (reserved)
BIOS-e820: 00000000c0000000 - 00000000c4000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
And finally I have my 8GB working without any kernel parameter,
# free -m total used free shared buffers cached Mem: 7988 1120 6868 0 88 612
Now i need to rerun memory tests.
Thank you for helping me with this stuff.
Quoted 10 lines
# free -m total used free shared buffers cached Mem: 7988 1120 6868 0 88 612Now i need to rerun memory tests.
Thank you for helping me with this stuff.
Uff. Now full memtest takes less than 2 hours.
Quoted 20 lines · Wojciech Kromer
a)With mem=8GB parameter I had:
#free -m
total used free shared buffers cached
Mem: 6473 474 5999 0 29 278b) Without mem=8GM system slows down while booting and sometimes restart...
==== So the story begins ===========================
It's strange GIGABYTE missinformation, but:
- my board came with F2 release
- after detecting memory problem, I've upgraded it to F4 (latest at this moment)
- next day came with F5 release ,
there was only one info: "Fix PS2 keyboard compatibility issues", so I decided *not* to upgrade BIOS
Quoted 2 lines
But yesterday I've finally upgraded to F5 release. ... and here is *surrprise*: MTRR changed to:
Never ever trust the "changelog" of the BIOS release to contain all the changes. That would be admiting to having made mistakes after all. :)
Quoted 8 lines
#cat
/proc/mtrr
reg00: base=0x00000000 ( 0MB), size=4096MB: write-back, count=1
reg01: base=0xc0000000 (3072MB), size=1024MB: uncachable, count=1
reg02: base=0xa0000000 (2560MB), size= 512MB: uncachable, count=1
reg03: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg04: base=0x200000000 (8192MB), size=2048MB: write-back, count=1
reg05: base=0x260000000 (9728MB), size= 512MB: uncachable, count=1
reg06: base=0x9ff00000 (2559MB), size= 1MB: write-through, count=1
Well they are in sane order now (the previous order was weird) And they don't do as much setting a range as one thing and then deleting small pieces from it afterwards.
Quoted 4 lines
Now there are no complains about MTRR in dmesg.
E820 seems to be the same:
Quoted 10 lines
BIOS-e820: 0000000000000000 -
000000000009f800(usable)
BIOS-e820:000000000009f800-00000000000a0000(reserved)
BIOS-e820:00000000000f0000- 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 -000000009fee0000(usable)
BIOS-e820:000000009fee0000-000000009fee3000(ACPI NVS)
BIOS-e820:000000009fee3000-000000009fef0000(ACPI data)
BIOS-e820:000000009fef0000-000000009ff00000(reserved)
BIOS-e820:00000000c0000000-00000000c4000000(reserved)
BIOS-e820:00000000fec00000- 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000260000000 (usable)
Well that part always did look sane.
Quoted 8 lines
And finally I have my 8GB working without any kernel parameter,
# free -m total used free shared buffers cached Mem: 7988 1120 6868 0 88 612Now i need to rerun memory tests.
Thank you for helping me with this stuff.
Well now things look good. Nice to know BIOS updates occationally fix things, even if they don't admit to it.
--
Len Sorensen
Quoted 1 line · Wojciech Kromer
Uff. Now full memtest takes less than 2 hours.
Being able to cache memory helps a lot.
--
Len Sorensen