trim memory not covered by WB MTRRs

On some machines, buggy BIOSes don't properly setup WB MTRRs to cover all available RAM, meaning the last few megs (or even gigs) of memory will be marked uncached. Since Linux tends to allocate from high memory addresses first, this causes the machine to be unusably slow as soon as the kernel starts really using memory (i.e. right around init time).

This patch works around the problem by scanning the MTRRs at boot and figuring out whether the current end_pfn value (setup by early e820 code) goes beyond the highest WB MTRR range, and if so, trimming it to match. A fairly obnoxious KERN_WARNING is printed too, letting the user know that not all of their memory is available due to a likely BIOS bug.

Something similar could be done on i386 if needed, but the boot ordering would be slightly different, since the MTRR code on i386 depends on the boot_cpu_data structure being setup.

Justin, can you please test and make sure this patch works for you too? It'll only work around the problem, but it's better than having to do mem= by hand or waiting for a fix from your BIOS vendor.

Thanks,
Jesse

index c4ebb51..71fc768 100644
@@ -13,7 +13,7 @@
 #include "mtrr.h"
 
 struct mtrr_state {
-	struct mtrr_var_range *var_ranges;
+	struct mtrr_var_range var_ranges[NUM_VAR_RANGES];
 	mtrr_type fixed_ranges[NUM_FIXED_RANGES];
 	unsigned char enabled;
 	unsigned char have_fixed;

@@ -84,12 +84,6 @@ void get_mtrr_state(void)
 	struct mtrr_var_range *vrs;
 	unsigned lo, dummy;
 
-	if (!mtrr_state.var_ranges) {
-		mtrr_state.var_ranges = kmalloc(num_var_ranges * sizeof (struct mtrr_var_range), 
-						GFP_KERNEL);
-		if (!mtrr_state.var_ranges)
-			return;
-	} 
 	vrs = mtrr_state.var_ranges;
 
 	rdmsr(MTRRcap_MSR, lo, dummy);

index c7d8f17..d7922ce 100644
@@ -12,7 +12,7 @@
 #include "mtrr.h"
 
 /* RED-PEN: this is accessed without any locking */
-extern unsigned int *usage_table;
+extern unsigned int usage_table[];
 
 
 #define FILE_FCOUNT(f) (((struct seq_file *)((f)->private_data))->private)

index 289dfe6..a29dcba 100644
@@ -14,6 +14,7 @@
 #define MTRRphysMask_MSR(reg) (0x200 + 2 * (reg) + 1)
 
 #define NUM_FIXED_RANGES 88
+#define NUM_VAR_RANGES 256
 #define MTRRfix64K_00000_MSR 0x250
 #define MTRRfix16K_80000_MSR 0x258
 #define MTRRfix16K_A0000_MSR 0x259

index c3c6b91..c138eac 100644
@@ -14,7 +14,6 @@
 void __init check_bugs(void)
 {
 	identify_cpu(&boot_cpu_data);
-	mtrr_bp_init();
 #if !defined(CONFIG_SMP)
 	printk("CPU: ");
 	print_cpu_info(&boot_cpu_data);

index eb6524f..409b63c 100644
@@ -266,6 +266,10 @@ void __init setup_arch(char **cmdline_p)
 	 * we are rounding upwards:
 	 */
 	end_pfn = e820_end_of_ram();
+	/* Trim memory not covered by WB MTRRs */
+	mtrr_bp_init();
+	mtrr_trim_uncached_memory();
+
 	num_physpages = end_pfn;
 
 	check_efer();

index b557c48..cc62bd8 100644
@@ -78,6 +78,7 @@ extern int mtrr_add_page (unsigned long base, unsigned long size,
 		     unsigned int type, char increment);
 extern int mtrr_del (int reg, unsigned long base, unsigned long size);
 extern int mtrr_del_page (int reg, unsigned long base, unsigned long size);
+extern void mtrr_trim_uncached_memory(void);
 #  else
 static __inline__ int mtrr_add (unsigned long base, unsigned long size,
 				unsigned int type, char increment)

Against what kernel version does this patch apply?

linux-2.6.21# patch -p1 < ../mtrr.patch
patching file arch/i386/kernel/cpu/mtrr/generic.c
Hunk #2 succeeded at 66 (offset -18 lines).
patching file arch/i386/kernel/cpu/mtrr/if.c
patching file arch/i386/kernel/cpu/mtrr/main.c
patching file arch/i386/kernel/cpu/mtrr/mtrr.h
can't find file to patch at input line 160
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------

|diff --git a/arch/x86_64/kernel/bugs.c b/arch/x86_64/kernel/bugs.c
|index c3c6b91..c138eac 100644
|--- a/arch/x86_64/kernel/bugs.c
|+++ b/arch/x86_64/kernel/bugs.c

--------------------------
File to patch:

Um... git as of b4946ffb1860597b187d78d61ac6504177eb0ff8. Sorry I should have updated before spinning the patch (will do now).

Jesse

Appears to apply cleanly to git head as of a minute ago too.

Jesse

Can you produce a patch against 2.6.22-rc4 or 2.6.21 so I and other people can easily try it? I do not have git installed on this machine, thanks.

Justin.

Seems to apply cleanly to 2.6.22-rc4 too. Haven't tested that though.

Jesse

Will give it a shot.

Jesse, it worked.

With mem=8832M (without your patch): 2.6.22-rc4:

top - 17:39:02 up 1 day, 8:07, 25 users, load average: 2.33, 0.76, 0.30
Tasks: 325 total, 11 running, 314 sleeping, 0 stopped, 0 zombie
Cpu(s): 80.0%us, 20.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8039620k total, 7936472k used, 103148k free, 708k buffers
Swap: 16787768k total, 128k used, 16787640k free, 6646248k cached

With no mem= in append line (with your patch): 2.6.22-rc4:

top - 17:44:01 up 1 min, 1 user, load average: 0.97, 0.25, 0.08

Tasks: 145 total,   1 running, 144 sleeping,   0 stopped,   0 zombie
Cpu(s):  5.2%us,  3.0%sy,  1.2%ni, 86.8%id,  3.8%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8039608k total,   969380k used,  7070228k free,     1232k buffers
Swap: 16787768k total,        0k used, 16787768k free,   109448k cached

Odd, remote netconsole did not capture the dmesg the E820 memory map.

Jun  6 17:43:03 p34 [   53.598611] ahci 0000:00:1f.2: AHCI 0001.0100 32 slots 6 ports 3 Gbps 0x3f impl SATA mode Jun  6 17:43:03 p34 [   53.598683] ahci 0000:00:1f.2: flags: 64bit ncq led clo pio slum part Jun  6 17:43:03 p34 [   53.598986] scsi0 : ahci Jun  6 17:43:03 p34 [   53.599131] scsi1 : ahci Jun  6 17:43:03 p34 [   53.599239] scsi2 : ahci Jun  6 17:43:03 p34 [   53.599340] scsi3 : ahci Jun  6 17:43:03 p34 [   53.599438] scsi4 : ahci

I will run with this patch for a while but so far, no issues, everything looks great.

Will it make it into 2.6.22-rc5? :)

Justin.

Mem:   8039620k total,  7936472k used,   103148k free,      708k buffers
Mem:   8039608k total,   969380k used,  7070228k free,     1232k buffers

I am curious, why does the patch != the mem=8832M?

Justin.

I'm not sure... can you post your e820 map from boot and the contents of /proc/mtrr? Maybe my patch is trimming off a few too many pages, or maybe 8832M isn't quite right and actually ends up leaving you with a few uncached pages.

Jesse

I cannot post the E820 memory map, I have no way to capture it, I cannot get anything from netconsole and dmesg does not show it either.

BEFORE:
reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1
reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
reg02: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1
reg03: base=0xcf800000 (3320MB), size= 8MB: uncachable, count=1
reg04: base=0xcf700000 (3319MB), size= 1MB: uncachable, count=1
reg05: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg06: base=0x200000000 (8192MB), size= 512MB: write-back, count=1
reg07: base=0x220000000 (8704MB), size= 128MB: write-back, count=1

AFTER:

$ cat /proc/mtrr
reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
reg02: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1
reg03: base=0xcf800000 (3320MB), size=   8MB: uncachable, count=1
reg04: base=0xcf700000 (3319MB), size=   1MB: uncachable, count=1
reg05: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg06: base=0x200000000 (8192MB), size= 512MB: write-back, count=1
reg07: base=0x220000000 (8704MB), size= 128MB: write-back, count=1

But that still works.

Justin.

Unless you know of some other way I can capture the output, it only starts showing the dmesg from [50..] onward.

Justin.

Did you boot the kernel with the 'debug' option? Maybe your dmesg buffer is too small (there's a config option for that iirc).

Jesse

Nope, I booted with only netconsole= options. I have a lot of HW in the box and I guess the buffer is too small. Not sure where to change it in the kernel. Looking..

It's called "kernel log buffer size" and it's in "General setup".

Jesse

I was reviewing some OLD mailing list archives then!

--------------

Per: http://www.edlug.ed.ac.uk/archive/Aug2003/msg00270.html

The kernel has a cyclic buffer of length LOG_BUF_LEN (4096, since 1.3.54:
8192, since 2.1.113: 16384; in
recent kernels the size can be set at compile time) in which
messages
given as argument to the kernel
function printk() are stored (regardless of their loglevel).

So it doesn't look like you can without recompiling...

Bruce

--------------

Getting you the E820 memory map in a few moments.

Changed from 15 -> 16.

| | (16) Kernel log buffer size (16 => 64KB, 17 => 128KB)

Justin.

Here ya go (and full dmesg attached)

[    0.000000] Linux version 2.6.22-rc4 ([email protected]) (gcc
version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)) #3 SMP Wed Jun 6
18:30:43 EDT 2007
[    0.000000] Command line: auto BOOT_IMAGE=2.6.22-rc4-3 ro root=902
[email protected]/eth0,[email protected]/00:50:8D:ED:3C:E7
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
[    0.000000]  BIOS-e820: 000000000008f000 - 00000000000a0000
(reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000
(reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000cf58f000 (usable)
[    0.000000]  BIOS-e820: 00000000cf58f000 - 00000000cf59c000
(reserved)
[    0.000000]  BIOS-e820: 00000000cf59c000 - 00000000cf653000 (usable)
[    0.000000]  BIOS-e820: 00000000cf653000 - 00000000cf6a5000 (ACPI
NVS)
[    0.000000]  BIOS-e820: 00000000cf6a5000 - 00000000cf6a8000 (ACPI
data)
[    0.000000]  BIOS-e820: 00000000cf6a8000 - 00000000cf6ef000 (ACPI
NVS)
[    0.000000]  BIOS-e820: 00000000cf6ef000 - 00000000cf6f1000 (ACPI
data)
[    0.000000]  BIOS-e820: 00000000cf6f1000 - 00000000cf6f2000 (usable)
[    0.000000]  BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI
data)
[    0.000000]  BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usable)
[    0.000000]  BIOS-e820: 00000000cf700000 - 00000000d0000000
(reserved)
[    0.000000]  BIOS-e820: 00000000fff00000 - 0000000100000000
(reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000022c000000 (usable)
[    0.000000] Entering add_active_range(0, 0, 143) 0 entries of 256
used
[    0.000000] Entering add_active_range(0, 256, 849295) 1 entries of
256 used
[    0.000000] Entering add_active_range(0, 849308, 849491) 2 entries of
256 used
[    0.000000] Entering add_active_range(0, 849649, 849650) 3 entries of
256 used
[    0.000000] Entering add_active_range(0, 849663, 849664) 4 entries of
256 used
[    0.000000] Entering add_active_range(0, 1048576, 2277376) 5 entries
of 256 used
[    0.000000] end_pfn_map = 2277376
[    0.000000] ***************
[    0.000000] **** WARNING: likely BIOS bug
[    0.000000] **** MTRRs don't cover all of memory, trimmed 16384 pages
[    0.000000] ***************
[    0.000000] DMI 2.4 present.
[    0.000000] ACPI: RSDP 000FE020, 0014 (r0 INTEL )
[    0.000000] ACPI: RSDT CF6FD038, 0050 (r1 INTEL  DG965WH       64C
1000013)
[    0.000000] ACPI: FACP CF6FC000, 0074 (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: DSDT CF6F7000, 40E9 (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: FACS CF6A8000, 0040
[    0.000000] ACPI: APIC CF6F6000, 0078 (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: WDDT CF6F5000, 0040 (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: MCFG CF6F4000, 003C (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: ASF! CF6F3000, 00A6 (r32 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: HPET CF6F2000, 0038 (r1 INTEL  DG965WH       64C
MSFT  1000013)
[    0.000000] ACPI: SSDT CF6F0000, 01BC (r1 INTEL     CpuPm      64C
MSFT  1000013)
[    0.000000] ACPI: SSDT CF6EF000, 0175 (r1 INTEL   Cpu0Ist      64C
MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A7000, 0175 (r1 INTEL   Cpu1Ist      64C
MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A6000, 0175 (r1 INTEL   Cpu2Ist      64C
MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A5000, 0175 (r1 INTEL   Cpu3Ist      64C
MSFT  1000013)
[    0.000000] Entering add_active_range(0, 0, 143) 0 entries of 256
used
[    0.000000] Entering add_active_range(0, 256, 849295) 1 entries of
256 used
[    0.000000] Entering add_active_range(0, 849308, 849491) 2 entries of
256 used
[    0.000000] Entering add_active_range(0, 849649, 849650) 3 entries of
256 used
[    0.000000] Entering add_active_range(0, 849663, 849664) 4 entries of
256 used
[    0.000000] Entering add_active_range(0, 1048576, 2260992) 5 entries
of 256 used
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA             0 ->     4096
[    0.000000]   DMA32        4096 ->  1048576
[    0.000000]   Normal    1048576 ->  2260992
[    0.000000] early_node_map[6] active PFN ranges
[    0.000000]     0:        0 ->      143
[    0.000000]     0:      256 ->   849295
[    0.000000]     0:   849308 ->   849491
[    0.000000]     0:   849649 ->   849650
[    0.000000]     0:   849663 ->   849664
[    0.000000]     0:  1048576 ->  2260992
[    0.000000] On node 0 totalpages: 2061783
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 1395 pages reserved
[    0.000000]   DMA zone: 2532 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 831104 pages, LIFO batch:31
[    0.000000]   Normal zone: 16576 pages used for memmap
[    0.000000]   Normal zone: 1195840 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] Linux version 2.6.22-rc4 ([email protected]) (gcc version 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)) #3 SMP Wed Jun 6 18:30:43 EDT 2007
[    0.000000] Command line: auto BOOT_IMAGE=2.6.22-rc4-3 ro root=902 [email protected]/eth0,[email protected]/00:50:8D:ED:3C:E7
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
[    0.000000]  BIOS-e820: 000000000008f000 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000cf58f000 (usable)
[    0.000000]  BIOS-e820: 00000000cf58f000 - 00000000cf59c000 (reserved)
[    0.000000]  BIOS-e820: 00000000cf59c000 - 00000000cf653000 (usable)
[    0.000000]  BIOS-e820: 00000000cf653000 - 00000000cf6a5000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000cf6a5000 - 00000000cf6a8000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000cf6a8000 - 00000000cf6ef000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000cf6ef000 - 00000000cf6f1000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000cf6f1000 - 00000000cf6f2000 (usable)
[    0.000000]  BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usable)
[    0.000000]  BIOS-e820: 00000000cf700000 - 00000000d0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000022c000000 (usable)
[    0.000000] Entering add_active_range(0, 0, 143) 0 entries of 256 used
[    0.000000] Entering add_active_range(0, 256, 849295) 1 entries of 256 used
[    0.000000] Entering add_active_range(0, 849308, 849491) 2 entries of 256 used
[    0.000000] Entering add_active_range(0, 849649, 849650) 3 entries of 256 used
[    0.000000] Entering add_active_range(0, 849663, 849664) 4 entries of 256 used
[    0.000000] Entering add_active_range(0, 1048576, 2277376) 5 entries of 256 used
[    0.000000] end_pfn_map = 2277376
[    0.000000] ***************
[    0.000000] **** WARNING: likely BIOS bug
[    0.000000] **** MTRRs don't cover all of memory, trimmed 16384 pages
[    0.000000] ***************
[    0.000000] DMI 2.4 present.
[    0.000000] ACPI: RSDP 000FE020, 0014 (r0 INTEL )
[    0.000000] ACPI: RSDT CF6FD038, 0050 (r1 INTEL  DG965WH       64C       1000013)
[    0.000000] ACPI: FACP CF6FC000, 0074 (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: DSDT CF6F7000, 40E9 (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: FACS CF6A8000, 0040
[    0.000000] ACPI: APIC CF6F6000, 0078 (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: WDDT CF6F5000, 0040 (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: MCFG CF6F4000, 003C (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: ASF! CF6F3000, 00A6 (r32 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: HPET CF6F2000, 0038 (r1 INTEL  DG965WH       64C MSFT  1000013)
[    0.000000] ACPI: SSDT CF6F0000, 01BC (r1 INTEL     CpuPm      64C MSFT  1000013)
[    0.000000] ACPI: SSDT CF6EF000, 0175 (r1 INTEL   Cpu0Ist      64C MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A7000, 0175 (r1 INTEL   Cpu1Ist      64C MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A6000, 0175 (r1 INTEL   Cpu2Ist      64C MSFT  1000013)
[    0.000000] ACPI: SSDT CF6A5000, 0175 (r1 INTEL   Cpu3Ist      64C MSFT  1000013)
[    0.000000] Entering add_active_range(0, 0, 143) 0 entries of 256 used
[    0.000000] Entering add_active_range(0, 256, 849295) 1 entries of 256 used
[    0.000000] Entering add_active_range(0, 849308, 849491) 2 entries of 256 used
[    0.000000] Entering add_active_range(0, 849649, 849650) 3 entries of 256 used
[    0.000000] Entering add_active_range(0, 849663, 849664) 4 entries of 256 used
[    0.000000] Entering add_active_range(0, 1048576, 2260992) 5 entries of 256 used
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA             0 ->     4096
[    0.000000]   DMA32        4096 ->  1048576
[    0.000000]   Normal    1048576 ->  2260992
[    0.000000] early_node_map[6] active PFN ranges
[    0.000000]     0:        0 ->      143
[    0.000000]     0:      256 ->   849295
[    0.000000]     0:   849308 ->   849491
[    0.000000]     0:   849649 ->   849650
[    0.000000]     0:   849663 ->   849664
[    0.000000]     0:  1048576 ->  2260992
[    0.000000] On node 0 totalpages: 2061783
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 1395 pages reserved
[    0.000000]   DMA zone: 2532 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 831104 pages, LIFO batch:31
[    0.000000]   Normal zone: 16576 pages used for memmap
[    0.000000]   Normal zone: 1195840 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] Processor #0 (Bootup-CPU)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
[    0.000000] Processor #2
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
[    0.000000] Processor #1
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
[    0.000000] Processor #3
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 2, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Setting APIC routing to flat
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] Allocating PCI resources starting at d4000000 (gap: d0000000:2ff00000)
[    0.000000] PERCPU: Allocating 24744 bytes of per cpu data
[    0.000000] Built 1 zonelists.  Total pages: 2029476
[    0.000000] Kernel command line: auto BOOT_IMAGE=2.6.22-rc4-3 ro root=902 [email protected]/eth0,[email protected]/00:50:8D:ED:3C:E7
[    0.000000] netconsole: local port 4444
[    0.000000] netconsole: local IP 192.168.168.253
[    0.000000] netconsole: interface eth0
[    0.000000] netconsole: remote port 514
[    0.000000] netconsole: remote IP 192.168.168.254
[    0.000000] netconsole: remote ethernet address 00:50:8d:ed:3c:e7
[    0.000000] Initializing CPU#0
[    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
[   48.079506] time.c: Detected 2397.607 MHz processor.
[   48.080578] Console: colour VGA+ 80x25
[   48.088448] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[   48.094647] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[   48.095721] Checking aperture...
[   48.095768] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[   48.132061] Placing software IO TLB between 0x951a000 - 0xd51a000
[   48.205318] Memory: 8039012k/9043968k available (3549k kernel code, 207808k reserved, 1317k data, 216k init)
[   48.264754] Calibrating delay using timer specific routine.. 4797.72 BogoMIPS (lpj=2398863)
[   48.264878] Mount-cache hash table entries: 256
[   48.265005] CPU: L1 I cache: 32K, L1 D cache: 32K
[   48.265068] CPU: L2 cache: 4096K
[   48.265108] using mwait in idle threads.
[   48.265150] CPU: Physical Processor ID: 0
[   48.265191] CPU: Processor Core ID: 0
[   48.265237] CPU0: Thermal monitoring enabled (TM2)
[   48.265284] Freeing SMP alternatives: 33k freed
[   48.265350] ACPI: Core revision 20070126
[   48.278788] Using local APIC timer interrupts.
[   48.320532] result 16650035
[   48.320571] Detected 16.650 MHz APIC timer.
[   48.320815] Booting processor 1/4 APIC 0x2
[   48.331205] Initializing CPU#1
[   48.391721] Calibrating delay using timer specific routine.. 4795.28 BogoMIPS (lpj=2397640)
[   48.391727] CPU: L1 I cache: 32K, L1 D cache: 32K
[   48.391728] CPU: L2 cache: 4096K
[   48.391730] CPU: Physical Processor ID: 0
[   48.391731] CPU: Processor Core ID: 2
[   48.391736] CPU1: Thermal monitoring enabled (TM2)
[   48.392159] Intel(R) Core(TM)2 Quad CPU           @ 2.40GHz stepping 07
[   48.392222] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
[   48.412781] Booting processor 2/4 APIC 0x1
[   48.423528] Initializing CPU#2
[   48.483700] Calibrating delay using timer specific routine.. 4795.23 BogoMIPS (lpj=2397616)
[   48.483706] CPU: L1 I cache: 32K, L1 D cache: 32K
[   48.483708] CPU: L2 cache: 4096K
[   48.483710] CPU: Physical Processor ID: 0
[   48.483711] CPU: Processor Core ID: 1
[   48.483716] CPU2: Thermal monitoring enabled (TM2)
[   48.484157] Intel(R) Core(TM)2 Quad CPU           @ 2.40GHz stepping 07
[   48.484186] checking TSC synchronization [CPU#0 -> CPU#2]: passed.
[   48.504779] Booting processor 3/4 APIC 0x3
[   48.515172] Initializing CPU#3
[   48.575678] Calibrating delay using timer specific routine.. 4795.27 BogoMIPS (lpj=2397638)
[   48.575684] CPU: L1 I cache: 32K, L1 D cache: 32K
[   48.575686] CPU: L2 cache: 4096K
[   48.575688] CPU: Physical Processor ID: 0
[   48.575689] CPU: Processor Core ID: 3
[   48.575694] CPU3: Thermal monitoring enabled (TM2)
[   48.576114] Intel(R) Core(TM)2 Quad CPU           @ 2.40GHz stepping 07
[   48.576188] checking TSC synchronization [CPU#0 -> CPU#3]: passed.
[   48.596679] Brought up 4 CPUs
[   49.216748] migration_cost=17,3461
[   49.217116] NET: Registered protocol family 16
[   49.217217] ACPI: bus type pci registered
[   49.217266] PCI: BIOS Bug: MCFG area at f0000000 is not E820-reserved
[   49.217316] PCI: Not using MMCONFIG.
[   49.217368] PCI: Using configuration type 1
[   49.220054] ACPI: Interpreter enabled
[   49.220096] ACPI: Using IOAPIC for interrupt routing
[   49.223600] ACPI: PCI Root Bridge [PCI0] (0000:00)
[   49.223659] PCI: Probing PCI hardware (bus 00)
[   49.224313] PCI quirk: region 0400-047f claimed by ICH6 ACPI/GPIO/TCO
[   49.224361] PCI quirk: region 0500-053f claimed by ICH6 GPIO
[   49.225260] PCI: Transparent bridge - 0000:00:1e.0
[   49.225372] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[   49.225650] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P32_._PRT]
[   49.225851] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0._PRT]
[   49.225931] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX1._PRT]
[   49.226010] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX2._PRT]
[   49.226090] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX3._PRT]
[   49.226169] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX4._PRT]
[   49.229348] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 7 9 10 *11 12)
[   49.229638] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 9 *10 11 12)
[   49.229924] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 9 10 *11 12)
[   49.230250] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 9 10 *11 12)
[   49.230548] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 7 *9 10 11 12)
[   49.230834] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 7 9 *10 11 12)
[   49.231141] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 7 *9 10 11 12)
[   49.231448] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 7 9 10 *11 12)
[   49.231825] Linux Plug and Play Support v0.97 (c) Adam Belay
[   49.231875] pnp: PnP ACPI init
[   49.231921] ACPI: bus type pnp registered
[   49.234206] pnp: PnP ACPI: found 12 devices
[   49.234261] ACPI: ACPI bus type pnp unregistered
[   49.234441] SCSI subsystem initialized
[   49.234541] libata version 2.21 loaded.
[   49.234634] usbcore: registered new interface driver usbfs
[   49.234730] usbcore: registered new interface driver hub
[   49.234848] usbcore: registered new device driver usb
[   49.234931] PCI: Using ACPI for IRQ routing
[   49.234973] PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
[   49.235126] PCI-GART: No AMD northbridge found.
[   49.235172] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[   49.235286] hpet0: 3 64-bit timers, 14318180 Hz
[   49.236430] pnp: 00:01: iomem range 0xf0000000-0xf7ffffff has been reserved
[   49.236491] pnp: 00:01: iomem range 0xfed13000-0xfed13fff has been reserved
[   49.236541] Time: tsc clocksource has been installed.
[   49.236597] pnp: 00:01: iomem range 0xfed14000-0xfed17fff has been reserved
[   49.236655] pnp: 00:01: iomem range 0xfed18000-0xfed18fff has been reserved
[   49.236706] pnp: 00:06: ioport range 0x500-0x53f has been reserved
[   49.236773] pnp: 00:06: ioport range 0x400-0x47f has been reserved
[   49.237372] pnp: 00:06: ioport range 0x680-0x6ff has been reserved
[   49.237848] ACPI: PCI Interrupt 0000:06:03.0[A] -> GSI 19 (level, low) -> IRQ 19
[   49.288159] ohci1394: fw-host0: OHCI-1394 1.1 (PCI): IRQ=[19]  MMIO=[e0084000-e00847ff]  Max Packet=[2048]  IR/IT contexts=[4/8]
[   49.288289] PCI: Bridge: 0000:00:1c.0
[   49.288330]   IO window: disabled.
[   49.288373]   MEM window: e0700000-e07fffff
[   49.288416]   PREFETCH window: disabled.
[   49.288460] PCI: Bridge: 0000:00:1c.1
[   49.288501]   IO window: 5000-5fff
[   49.288545]   MEM window: e0400000-e04fffff
[   49.288599]   PREFETCH window: disabled.
[   49.288652] PCI: Bridge: 0000:00:1c.2
[   49.288694]   IO window: 4000-4fff
[   49.288736]   MEM window: e0300000-e03fffff
[   49.288800]   PREFETCH window: e0800000-e08fffff
[   49.288846] PCI: Bridge: 0000:00:1c.3
[   49.288887]   IO window: 3000-3fff
[   49.288930]   MEM window: e0200000-e02fffff
[   49.288973]   PREFETCH window: e0900000-e09fffff
[   49.289018] PCI: Bridge: 0000:00:1c.4
[   49.289060]   IO window: 2000-2fff
[   49.289102]   MEM window: e0100000-e01fffff
[   49.289146]   PREFETCH window: e0a00000-e0afffff
[   49.289192] PCI: Bridge: 0000:00:1e.0
[   49.289233]   IO window: 1000-1fff
[   49.289276]   MEM window: e0000000-e00fffff
[   49.289319]   PREFETCH window: e0b00000-e0bfffff
[   49.289376] ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 17 (level, low) -> IRQ 17
[   49.289461] PCI: Setting latency timer of device 0000:00:1c.0 to 64
[   49.289474] ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 16 (level, low) -> IRQ 16
[   49.289559] PCI: Setting latency timer of device 0000:00:1c.1 to 64
[   49.289573] ACPI: PCI Interrupt 0000:00:1c.2[C] -> GSI 18 (level, low) -> IRQ 18
[   49.289677] PCI: Setting latency timer of device 0000:00:1c.2 to 64
[   49.289690] ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 19 (level, low) -> IRQ 19
[   49.289773] PCI: Setting latency timer of device 0000:00:1c.3 to 64
[   49.289785] ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 17 (level, low) -> IRQ 17
[   49.289889] PCI: Setting latency timer of device 0000:00:1c.4 to 64
[   49.289897] PCI: Setting latency timer of device 0000:00:1e.0 to 64
[   49.289944] NET: Registered protocol family 2
[   49.300806] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
[   49.301356] TCP established hash table entries: 131072 (order: 9, 3145728 bytes)
[   49.302625] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[   49.303110] TCP: Hash tables configured (established 131072 bind 65536)
[   49.303158] TCP reno registered
[   49.307229] Installing knfsd (copyright (C) 1996 [email protected]).
[   49.307421] SGI XFS with large block/inode numbers, no debug enabled
[   49.307761] io scheduler noop registered
[   49.307813] io scheduler anticipatory registered (default)
[   49.307863] Boot video device is 0000:00:02.0
[   49.308278] PCI: Setting latency timer of device 0000:00:1c.0 to 64
[   49.308309] assign_interrupt_mode Found MSI capability
[   49.308380] Allocate Port Service[0000:00:1c.0:pcie00]
[   49.308402] Allocate Port Service[0000:00:1c.0:pcie02]
[   49.308460] PCI: Setting latency timer of device 0000:00:1c.1 to 64
[   49.308491] assign_interrupt_mode Found MSI capability
[   49.308569] Allocate Port Service[0000:00:1c.1:pcie00]
[   49.308602] Allocate Port Service[0000:00:1c.1:pcie02]
[   49.308658] PCI: Setting latency timer of device 0000:00:1c.2 to 64
[   49.308689] assign_interrupt_mode Found MSI capability
[   49.308757] Allocate Port Service[0000:00:1c.2:pcie00]
[   49.308777] Allocate Port Service[0000:00:1c.2:pcie02]
[   49.308835] PCI: Setting latency timer of device 0000:00:1c.3 to 64
[   49.308866] assign_interrupt_mode Found MSI capability
[   49.308933] Allocate Port Service[0000:00:1c.3:pcie00]
[   49.308956] Allocate Port Service[0000:00:1c.3:pcie02]
[   49.309015] PCI: Setting latency timer of device 0000:00:1c.4 to 64
[   49.309045] assign_interrupt_mode Found MSI capability
[   49.309134] Allocate Port Service[0000:00:1c.4:pcie00]
[   49.309156] Allocate Port Service[0000:00:1c.4:pcie02]
[   49.310808] lp: driver loaded but no devices found
[   49.310888] Real Time Clock Driver v1.12ac
[   49.311009] hpet_resources: 0xfed00000 is busy
[   49.311027] Linux agpgart interface v0.102 (c) Dave Jones
[   49.311105] agpgart: Detected an Intel 965G Chipset.
[   49.312363] agpgart: Detected 7676K stolen memory.
[   49.324807] agpgart: AGP aperture is 256M @ 0xd0000000
[   49.324885] [drm] Initialized drm 1.1.0 20060810
[   49.324944] ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16
[   49.325066] [drm] Initialized i915 1.6.0 20060119 on minor 0
[   49.325167] parport_pc 00:07: reported by Plug and Play ACPI
[   49.325285] parport0: PC-style at 0x378 (0x778), irq 7, using FIFO [PCSPP,TRISTATE,COMPAT,ECP]
[   49.408131] lp0: using parport0 (interrupt-driven).
[   49.408298] loop: module loaded
[   49.408338] Intel(R) PRO/1000 Network Driver - version 7.3.20-k2-NAPI
[   49.408385] Copyright (c) 1999-2006 Intel Corporation.
[   49.408457] ACPI: PCI Interrupt 0000:00:19.0[A] -> GSI 20 (level, low) -> IRQ 20
[   49.408565] PCI: Setting latency timer of device 0000:00:19.0 to 64
[   49.427147] e1000: 0000:00:19.0: e1000_probe: (PCI Express:2.5Gb/s:Width x1) 00:19:d1:6e:9d:43
[   49.502455] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
[   49.502543] ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI 21 (level, low) -> IRQ 21
[   49.764949] e1000: 0000:06:00.0: e1000_probe: (PCI:33MHz:32-bit) 00:0e:0c:00:cd:b1
[   49.911684] e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
[   49.911744] ACPI: PCI Interrupt 0000:06:01.0[A] -> GSI 22 (level, low) -> IRQ 22
[   50.158693] e1000: 0000:06:01.0: e1000_probe: (PCI:33MHz:32-bit) 00:07:e9:29:37:db
[   50.184398] e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
[   50.184493] netconsole: device eth0 not up yet, forcing it
[   50.544713] ieee1394: Host added: ID:BUS[0-00:1023]  GUID[0090270001c5be9f]
[   52.444887] e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   52.468712] netconsole: network logging started
[   52.468869] ahci 0000:00:1f.2: version 2.2
[   52.468883] ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 19 (level, low) -> IRQ 19
[   53.470122] ahci 0000:00:1f.2: AHCI 0001.0100 32 slots 6 ports 3 Gbps 0x3f impl SATA mode
[   53.470195] ahci 0000:00:1f.2: flags: 64bit ncq led clo pio slum part 
[   53.470269] PCI: Setting latency timer of device 0000:00:1f.2 to 64
[   53.470484] scsi0 : ahci
[   53.470599] scsi1 : ahci
[   53.470694] scsi2 : ahci
[   53.470792] scsi3 : ahci
[   53.470892] scsi4 : ahci
[   53.471005] scsi5 : ahci
[   53.471098] ata1: SATA max UDMA/133 cmd 0xffffc2000002e100 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.471178] ata2: SATA max UDMA/133 cmd 0xffffc2000002e180 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.471256] ata3: SATA max UDMA/133 cmd 0xffffc2000002e200 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.471334] ata4: SATA max UDMA/133 cmd 0xffffc2000002e280 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.471412] ata5: SATA max UDMA/133 cmd 0xffffc2000002e300 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.471498] ata6: SATA max UDMA/133 cmd 0xffffc2000002e380 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   53.981909] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   53.985309] ata1.00: ata_hpa_resize 1: sectors = 145226112, hpa_sectors = 145226112
[   53.985379] ata1.00: ATA-6: WDC WD740GD-00FLC0, 33.08F33, max UDMA/133
[   53.985436] ata1.00: 145226112 sectors, multi 0: LBA48 
[   53.989871] ata1.00: ata_hpa_resize 1: sectors = 145226112, hpa_sectors = 145226112
[   53.989943] ata1.00: configured for UDMA/133
[   54.448246] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   54.451681] ata2.00: ata_hpa_resize 1: sectors = 145226112, hpa_sectors = 145226112
[   54.451751] ata2.00: ATA-6: WDC WD740GD-00FLC0, 33.08F33, max UDMA/133
[   54.451806] ata2.00: 145226112 sectors, multi 0: LBA48 
[   54.456220] ata2.00: ata_hpa_resize 1: sectors = 145226112, hpa_sectors = 145226112
[   54.456291] ata2.00: configured for UDMA/133
[   54.915523] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   54.917712] ata3.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   54.917783] ata3.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   54.917856] ata3.00: 293046768 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   54.920699] ata3.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   54.920771] ata3.00: configured for UDMA/133
[   55.378761] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   55.380949] ata4.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   55.381029] ata4.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   55.381100] ata4.00: 293046768 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   55.383936] ata4.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   55.384008] ata4.00: configured for UDMA/133
[   55.843019] ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   55.845093] ata5.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   55.845164] ata5.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   55.845230] ata5.00: 293046768 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   55.847887] ata5.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   55.847960] ata5.00: configured for UDMA/133
[   56.307338] ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   56.309375] ata6.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   56.309446] ata6.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   56.309502] ata6.00: 293046768 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   56.312166] ata6.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   56.312238] ata6.00: configured for UDMA/133
[   56.312366] scsi 0:0:0:0: Direct-Access     ATA      WDC WD740GD-00FL 33.0 PQ: 0 ANSI: 5
[   56.312519] sd 0:0:0:0: [sda] 145226112 512-byte hardware sectors (74356 MB)
[   56.312587] sd 0:0:0:0: [sda] Write Protect is off
[   56.312643] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   56.312654] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.312764] sd 0:0:0:0: [sda] 145226112 512-byte hardware sectors (74356 MB)
[   56.312830] sd 0:0:0:0: [sda] Write Protect is off
[   56.312888] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   56.312899] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.312989]  sda: sda1 sda2 sda3
[   56.322761] sd 0:0:0:0: [sda] Attached SCSI disk
[   56.322978] scsi 1:0:0:0: Direct-Access     ATA      WDC WD740GD-00FL 33.0 PQ: 0 ANSI: 5
[   56.323159] sd 1:0:0:0: [sdb] 145226112 512-byte hardware sectors (74356 MB)
[   56.323221] sd 1:0:0:0: [sdb] Write Protect is off
[   56.323274] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   56.323289] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.323398] sd 1:0:0:0: [sdb] 145226112 512-byte hardware sectors (74356 MB)
[   56.323462] sd 1:0:0:0: [sdb] Write Protect is off
[   56.323513] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   56.323528] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.323632]  sdb: sdb1 sdb2 sdb3
[   56.330482] sd 1:0:0:0: [sdb] Attached SCSI disk
[   56.330653] scsi 2:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   56.330826] sd 2:0:0:0: [sdc] 293046768 512-byte hardware sectors (150040 MB)
[   56.330886] sd 2:0:0:0: [sdc] Write Protect is off
[   56.330942] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   56.330958] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.331061] sd 2:0:0:0: [sdc] 293046768 512-byte hardware sectors (150040 MB)
[   56.331128] sd 2:0:0:0: [sdc] Write Protect is off
[   56.331176] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[   56.331191] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.331295]  sdc: sdc1
[   56.335795] sd 2:0:0:0: [sdc] Attached SCSI disk
[   56.335938] scsi 3:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   56.336132] sd 3:0:0:0: [sdd] 293046768 512-byte hardware sectors (150040 MB)
[   56.336206] sd 3:0:0:0: [sdd] Write Protect is off
[   56.336253] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   56.336263] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.336374] sd 3:0:0:0: [sdd] 293046768 512-byte hardware sectors (150040 MB)
[   56.336431] sd 3:0:0:0: [sdd] Write Protect is off
[   56.336478] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[   56.336488] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.336580]  sdd: sdd1
[   56.343788] sd 3:0:0:0: [sdd] Attached SCSI disk
[   56.343935] scsi 4:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   56.344106] sd 4:0:0:0: [sde] 293046768 512-byte hardware sectors (150040 MB)
[   56.344166] sd 4:0:0:0: [sde] Write Protect is off
[   56.344220] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[   56.344235] sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.344339] sd 4:0:0:0: [sde] 293046768 512-byte hardware sectors (150040 MB)
[   56.344403] sd 4:0:0:0: [sde] Write Protect is off
[   56.344454] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00
[   56.344469] sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.344573]  sde: sde1
[   56.355511] sd 4:0:0:0: [sde] Attached SCSI disk
[   56.355657] scsi 5:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   56.355824] sd 5:0:0:0: [sdf] 293046768 512-byte hardware sectors (150040 MB)
[   56.355884] sd 5:0:0:0: [sdf] Write Protect is off
[   56.355937] sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
[   56.355952] sd 5:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.356059] sd 5:0:0:0: [sdf] 293046768 512-byte hardware sectors (150040 MB)
[   56.356126] sd 5:0:0:0: [sdf] Write Protect is off
[   56.356174] sd 5:0:0:0: [sdf] Mode Sense: 00 3a 00 00
[   56.356189] sd 5:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   56.356293]  sdf: sdf1
[   56.361893] sd 5:0:0:0: [sdf] Attached SCSI disk
[   56.362122] sata_sil24 0000:03:00.0: version 0.9
[   56.362153] ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 18 (level, low) -> IRQ 18
[   56.362327] PCI: Setting latency timer of device 0000:03:00.0 to 64
[   56.362382] scsi6 : sata_sil24
[   56.362501] scsi7 : sata_sil24
[   56.362616] ata7: SATA max UDMA/100 cmd 0xffffc20000038000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   56.362703] ata8: SATA max UDMA/100 cmd 0xffffc2000003a000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   56.766751] ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   56.769241] ata7.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   56.769313] ata7.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   56.769369] ata7.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   56.772597] ata7.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   56.772670] ata7.00: configured for UDMA/100
[   57.176868] ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   57.179393] ata8.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   57.179460] ata8.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   57.179511] ata8.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   57.182775] ata8.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   57.182847] ata8.00: configured for UDMA/100
[   57.182955] scsi 6:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   57.183126] sd 6:0:0:0: [sdg] 293046768 512-byte hardware sectors (150040 MB)
[   57.183191] sd 6:0:0:0: [sdg] Write Protect is off
[   57.183238] sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00
[   57.183248] sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   57.183361] sd 6:0:0:0: [sdg] 293046768 512-byte hardware sectors (150040 MB)
[   57.183418] sd 6:0:0:0: [sdg] Write Protect is off
[   57.183485] sd 6:0:0:0: [sdg] Mode Sense: 00 3a 00 00
[   57.183495] sd 6:0:0:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   57.183566]  sdg: sdg1
[   57.194888] sd 6:0:0:0: [sdg] Attached SCSI disk
[   57.195063] scsi 7:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   57.195213] sd 7:0:0:0: [sdh] 293046768 512-byte hardware sectors (150040 MB)
[   57.195275] sd 7:0:0:0: [sdh] Write Protect is off
[   57.195327] sd 7:0:0:0: [sdh] Mode Sense: 00 3a 00 00
[   57.195342] sd 7:0:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   57.195470] sd 7:0:0:0: [sdh] 293046768 512-byte hardware sectors (150040 MB)
[   57.195526] sd 7:0:0:0: [sdh] Write Protect is off
[   57.195593] sd 7:0:0:0: [sdh] Mode Sense: 00 3a 00 00
[   57.195603] sd 7:0:0:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   57.195675]  sdh: sdh1
[   57.206857] sd 7:0:0:0: [sdh] Attached SCSI disk
[   57.206966] ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 19 (level, low) -> IRQ 19
[   57.207106] PCI: Setting latency timer of device 0000:04:00.0 to 64
[   57.207188] scsi8 : sata_sil24
[   57.207305] scsi9 : sata_sil24
[   57.207422] ata9: SATA max UDMA/100 cmd 0xffffc20000068000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   57.207507] ata10: SATA max UDMA/100 cmd 0xffffc2000006a000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   57.612628] ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   57.614742] ata9.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   57.614823] ata9.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   57.614874] ata9.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   57.617628] ata9.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   57.617700] ata9.00: configured for UDMA/100
[   58.022845] ata10: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   58.024964] ata10.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.025035] ata10.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   58.025100] ata10.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   58.027875] ata10.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.027949] ata10.00: configured for UDMA/100
[   58.028056] scsi 8:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   58.028242] sd 8:0:0:0: [sdi] 293046768 512-byte hardware sectors (150040 MB)
[   58.028302] sd 8:0:0:0: [sdi] Write Protect is off
[   58.028354] sd 8:0:0:0: [sdi] Mode Sense: 00 3a 00 00
[   58.028370] sd 8:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.028474] sd 8:0:0:0: [sdi] 293046768 512-byte hardware sectors (150040 MB)
[   58.028540] sd 8:0:0:0: [sdi] Write Protect is off
[   58.028591] sd 8:0:0:0: [sdi] Mode Sense: 00 3a 00 00
[   58.028606] sd 8:0:0:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.028710]  sdi: sdi1
[   58.038904] sd 8:0:0:0: [sdi] Attached SCSI disk
[   58.039042] scsi 9:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   58.039193] sd 9:0:0:0: [sdj] 293046768 512-byte hardware sectors (150040 MB)
[   58.039253] sd 9:0:0:0: [sdj] Write Protect is off
[   58.039305] sd 9:0:0:0: [sdj] Mode Sense: 00 3a 00 00
[   58.039320] sd 9:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.039454] sd 9:0:0:0: [sdj] 293046768 512-byte hardware sectors (150040 MB)
[   58.039519] sd 9:0:0:0: [sdj] Write Protect is off
[   58.039566] sd 9:0:0:0: [sdj] Mode Sense: 00 3a 00 00
[   58.039576] sd 9:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.039668]  sdj: sdj1
[   58.047213] sd 9:0:0:0: [sdj] Attached SCSI disk
[   58.047352] ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 16 (level, low) -> IRQ 16
[   58.047504] PCI: Setting latency timer of device 0000:05:00.0 to 64
[   58.047552] scsi10 : sata_sil24
[   58.047668] scsi11 : sata_sil24
[   58.047787] ata11: SATA max UDMA/100 cmd 0xffffc20000070000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   58.047872] ata12: SATA max UDMA/100 cmd 0xffffc20000072000 ctl 0x0000000000000000 bmdma 0x0000000000000000 irq 0
[   58.452485] ata11: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   58.454979] ata11.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.455050] ata11.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   58.455135] ata11.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   58.458386] ata11.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.458459] ata11.00: configured for UDMA/100
[   58.862662] ata12: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   58.865167] ata12.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.865238] ata12.00: ATA-7: WDC WD1500ADFD-00NLR1, 20.07P20, max UDMA/133
[   58.865294] ata12.00: 293046768 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   58.868544] ata12.00: ata_hpa_resize 1: sectors = 293046768, hpa_sectors = 293046768
[   58.868617] ata12.00: configured for UDMA/100
[   58.868723] scsi 10:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   58.868881] sd 10:0:0:0: [sdk] 293046768 512-byte hardware sectors (150040 MB)
[   58.868983] sd 10:0:0:0: [sdk] Write Protect is off
[   58.869031] sd 10:0:0:0: [sdk] Mode Sense: 00 3a 00 00
[   58.869041] sd 10:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.869152] sd 10:0:0:0: [sdk] 293046768 512-byte hardware sectors (150040 MB)
[   58.869237] sd 10:0:0:0: [sdk] Write Protect is off
[   58.869292] sd 10:0:0:0: [sdk] Mode Sense: 00 3a 00 00
[   58.869302] sd 10:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.869374]  sdk: sdk1
[   58.877168] sd 10:0:0:0: [sdk] Attached SCSI disk
[   58.877349] scsi 11:0:0:0: Direct-Access     ATA      WDC WD1500ADFD-0 20.0 PQ: 0 ANSI: 5
[   58.877517] sd 11:0:0:0: [sdl] 293046768 512-byte hardware sectors (150040 MB)
[   58.877593] sd 11:0:0:0: [sdl] Write Protect is off
[   58.877645] sd 11:0:0:0: [sdl] Mode Sense: 00 3a 00 00
[   58.877661] sd 11:0:0:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.877767] sd 11:0:0:0: [sdl] 293046768 512-byte hardware sectors (150040 MB)
[   58.877847] sd 11:0:0:0: [sdl] Write Protect is off
[   58.877898] sd 11:0:0:0: [sdl] Mode Sense: 00 3a 00 00
[   58.877913] sd 11:0:0:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   58.878019]  sdl: sdl1
[   58.885496] sd 11:0:0:0: [sdl] Attached SCSI disk
[   58.885641] ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 17 (level, low) -> IRQ 17
[   58.885756] PCI: Setting latency timer of device 0000:02:00.0 to 64
[   58.885790] scsi12 : pata_marvell
[   58.885892] scsi13 : pata_marvell
[   58.886030] ata13: PATA max UDMA/100 cmd 0x0000000000015018 ctl 0x0000000000015026 bmdma 0x0000000000015000 irq 0
[   58.886113] ata14: DUMMY
[   58.886182] BAR5:00:00 01:7F 02:22 03:CA 04:00 05:00 06:00 07:00 08:00 09:00 0A:00 0B:00 0C:01 0D:00 0E:00 0F:00 
[   59.192728] ata13.00: ATAPI, max UDMA/33
[   59.349867] ata13.00: configured for UDMA/33
[   59.350932] scsi 12:0:0:0: CD-ROM            _NEC     DVD_RW ND-3520A  1.04 PQ: 0 ANSI: 5
[   59.355097] sr0: scsi3-mmc drive: 48x/48x writer cd/rw xa/form2 cdda tray
[   59.355179] Uniform CD-ROM driver Revision: 3.20
[   59.355295] sr 12:0:0:0: Attached scsi CD-ROM sr0
[   59.355382] ACPI: PCI Interrupt 0000:00:1a.7[C] -> GSI 18 (level, low) -> IRQ 18
[   59.355485] PCI: Setting latency timer of device 0000:00:1a.7 to 64
[   59.355489] ehci_hcd 0000:00:1a.7: EHCI Host Controller
[   59.355585] ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
[   59.355679] ehci_hcd 0000:00:1a.7: debug port 1
[   59.355733] PCI: cache line size of 32 is not supported by device 0000:00:1a.7
[   59.355738] ehci_hcd 0000:00:1a.7: irq 18, io mem 0xe0625c00
[   59.359701] ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
[   59.359916] usb usb1: configuration #1 chosen from 1 choice
[   59.360043] hub 1-0:1.0: USB hub found
[   59.360100] hub 1-0:1.0: 4 ports detected
[   59.460812] ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 23 (level, low) -> IRQ 23
[   59.460915] PCI: Setting latency timer of device 0000:00:1d.7 to 64
[   59.460940] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[   59.461058] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
[   59.461166] ehci_hcd 0000:00:1d.7: debug port 1
[   59.461227] PCI: cache line size of 32 is not supported by device 0000:00:1d.7
[   59.461232] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xe0625800
[   59.465147] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
[   59.465349] usb usb2: configuration #1 chosen from 1 choice
[   59.465475] hub 2-0:1.0: USB hub found
[   59.465531] hub 2-0:1.0: 6 ports detected
[   59.566560] USB Universal Host Controller Interface driver v3.0
[   59.566659] ACPI: PCI Interrupt 0000:00:1a.0[A] -> GSI 16 (level, low) -> IRQ 16
[   59.566756] PCI: Setting latency timer of device 0000:00:1a.0 to 64
[   59.566759] uhci_hcd 0000:00:1a.0: UHCI Host Controller
[   59.566879] uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
[   59.566973] uhci_hcd 0000:00:1a.0: irq 16, io base 0x000060c0
[   59.567170] usb usb3: configuration #1 chosen from 1 choice
[   59.567298] hub 3-0:1.0: USB hub found
[   59.567937] hub 3-0:1.0: 2 ports detected
[   59.668727] ACPI: PCI Interrupt 0000:00:1a.1[B] -> GSI 21 (level, low) -> IRQ 21
[   59.668825] PCI: Setting latency timer of device 0000:00:1a.1 to 64
[   59.668829] uhci_hcd 0000:00:1a.1: UHCI Host Controller
[   59.668968] uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
[   59.669063] uhci_hcd 0000:00:1a.1: irq 21, io base 0x000060a0
[   59.669258] usb usb4: configuration #1 chosen from 1 choice
[   59.669386] hub 4-0:1.0: USB hub found
[   59.669442] hub 4-0:1.0: 2 ports detected
[   59.770816] ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 23 (level, low) -> IRQ 23
[   59.770915] PCI: Setting latency timer of device 0000:00:1d.0 to 64
[   59.770929] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[   59.771061] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 5
[   59.771144] uhci_hcd 0000:00:1d.0: irq 23, io base 0x00006080
[   59.771355] usb usb5: configuration #1 chosen from 1 choice
[   59.771481] hub 5-0:1.0: USB hub found
[   59.771537] hub 5-0:1.0: 2 ports detected
[   59.872869] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 19
[   59.872966] PCI: Setting latency timer of device 0000:00:1d.1 to 64
[   59.872970] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[   59.873098] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 6
[   59.873201] uhci_hcd 0000:00:1d.1: irq 19, io base 0x00006060
[   59.873394] usb usb6: configuration #1 chosen from 1 choice
[   59.873522] hub 6-0:1.0: USB hub found
[   59.873579] hub 6-0:1.0: 2 ports detected
[   59.974945] ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 18
[   59.975042] PCI: Setting latency timer of device 0000:00:1d.2 to 64
[   59.975046] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[   59.975185] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 7
[   59.975288] uhci_hcd 0000:00:1d.2: irq 18, io base 0x00006040
[   59.975488] usb usb7: configuration #1 chosen from 1 choice
[   59.975613] hub 7-0:1.0: USB hub found
[   59.975670] hub 7-0:1.0: 2 ports detected
[   60.077046] Initializing USB Mass Storage driver...
[   60.080010] usb 5-1: new low speed USB device using uhci_hcd and address 2
[   60.244227] usb 5-1: configuration #1 chosen from 1 choice
[   60.247286] usbcore: registered new interface driver usb-storage
[   60.247337] USB Mass Storage support registered.
[   60.247477] PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12
[   60.250401] serio: i8042 KBD port at 0x60,0x64 irq 1
[   60.250448] serio: i8042 AUX port at 0x60,0x64 irq 12
[   60.250547] mice: PS/2 mouse device common for all mice
[   60.250886] input: PC Speaker as /class/input/input0
[   60.250976] i2c /dev entries driver
[   60.251125] coretemp coretemp.0: Using undocumented features, absolute temperature might be wrong!
[   60.251289] coretemp coretemp.1: Using undocumented features, absolute temperature might be wrong!
[   60.251442] coretemp coretemp.2: Using undocumented features, absolute temperature might be wrong!
[   60.251595] coretemp coretemp.3: Using undocumented features, absolute temperature might be wrong!
[   60.251733] md: raid1 personality registered for level 1
[   60.268013] raid6: int64x1   2027 MB/s
[   60.285002] raid6: int64x2   2781 MB/s
[   60.301998] raid6: int64x4   3101 MB/s
[   60.318996] raid6: int64x8   1914 MB/s
[   60.335979] raid6: sse2x1    3824 MB/s
[   60.352979] raid6: sse2x2    4222 MB/s
[   60.369967] raid6: sse2x4    6777 MB/s
[   60.370029] raid6: using algorithm sse2x4 (6777 MB/s)
[   60.370091] md: raid6 personality registered for level 6
[   60.370148] md: raid5 personality registered for level 5
[   60.370216] md: raid4 personality registered for level 4
[   60.370262] raid5: automatically using best checksumming function: generic_sse
[   60.374963]    generic_sse:  8524.000 MB/sec
[   60.375006] raid5: using function: generic_sse (8524.000 MB/sec)
[   60.375269] usbcore: registered new interface driver hiddev
[   60.823087] hiddev96: USB HID v1.00 Device [        UPS] on usb-0000:00:1d.0-1
[   60.823210] usbcore: registered new interface driver usbhid
[   60.823260] drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
[   60.823328] Advanced Linux Sound Architecture Driver Version 1.0.14 (Thu May 31 09:03:25 2007 UTC).
[   60.823664] ACPI: PCI Interrupt 0000:00:1b.0[A] -> GSI 22 (level, low) -> IRQ 22
[   60.823775] PCI: Setting latency timer of device 0000:00:1b.0 to 64
[   60.892415] input: ImPS/2 Generic Wheel Mouse as /class/input/input1
[   60.916271] input: AT Translated Set 2 keyboard as /class/input/input2
[   60.957332] ALSA device list:
[   60.957377]   #0: HDA Intel at 0xe0620000 irq 22
[   60.957429] u32 classifier
[   60.957482]     Actions configured 
[   60.957527] nf_conntrack version 0.5.0 (8192 buckets, 65536 max)
[   60.957781] ip_tables: (C) 2000-2006 Netfilter Core Team
[   60.957880] TCP cubic registered
[   60.957937] NET: Registered protocol family 1
[   60.957994] NET: Registered protocol family 17
[   60.958370] md: Autodetecting RAID arrays.
[   61.110568] md: autorun ...
[   61.110608] md: considering sdl1 ...
[   61.110657] md:  adding sdl1 ...
[   61.110699] md:  adding sdk1 ...
[   61.110764] md:  adding sdj1 ...
[   61.110811] md:  adding sdi1 ...
[   61.110855] md:  adding sdh1 ...
[   61.110910] md:  adding sdg1 ...
[   61.110962] md:  adding sdf1 ...
[   61.111007] md:  adding sde1 ...
[   61.111052] md:  adding sdd1 ...
[   61.111095] md:  adding sdc1 ...
[   61.111139] md: sdb3 has different UUID to sdl1
[   61.111186] md: sdb2 has different UUID to sdl1
[   61.111233] md: sdb1 has different UUID to sdl1
[   61.111279] md: sda3 has different UUID to sdl1
[   61.111326] md: sda2 has different UUID to sdl1
[   61.111374] md: sda1 has different UUID to sdl1
[   61.111489] md: created md3
[   61.111542] md: bind<sdc1>
[   61.111588] md: bind<sdd1>
[   61.111632] md: bind<sde1>
[   61.111678] md: bind<sdf1>
[   61.111724] md: bind<sdg1>
[   61.111770] md: bind<sdh1>
[   61.111841] md: bind<sdi1>
[   61.111886] md: bind<sdj1>
[   61.111932] md: bind<sdk1>
[   61.111999] md: bind<sdl1>
[   61.112045] md: running: <sdl1><sdk1><sdj1><sdi1><sdh1><sdg1><sdf1><sde1><sdd1><sdc1>
[   61.112350] raid5: device sdl1 operational as raid disk 2
[   61.112401] raid5: device sdk1 operational as raid disk 3
[   61.112448] raid5: device sdj1 operational as raid disk 0
[   61.112496] raid5: device sdi1 operational as raid disk 1
[   61.112543] raid5: device sdh1 operational as raid disk 6
[   61.112591] raid5: device sdg1 operational as raid disk 5
[   61.112639] raid5: device sdf1 operational as raid disk 4
[   61.112686] raid5: device sde1 operational as raid disk 7
[   61.112734] raid5: device sdd1 operational as raid disk 8
[   61.112782] raid5: device sdc1 operational as raid disk 9
[   61.113360] raid5: allocated 10562kB for md3
[   61.113406] raid5: raid level 5 set md3 active with 10 out of 10 devices, algorithm 2
[   61.113474] RAID5 conf printout:
[   61.113517]  --- rd:10 wd:10
[   61.113559]  disk 0, o:1, dev:sdj1
[   61.113602]  disk 1, o:1, dev:sdi1
[   61.113646]  disk 2, o:1, dev:sdl1
[   61.113687]  disk 3, o:1, dev:sdk1
[   61.113730]  disk 4, o:1, dev:sdf1
[   61.113774]  disk 5, o:1, dev:sdg1
[   61.113819]  disk 6, o:1, dev:sdh1
[   61.113862]  disk 7, o:1, dev:sde1
[   61.113926]  disk 8, o:1, dev:sdd1
[   61.113970]  disk 9, o:1, dev:sdc1
[   61.114083] md: considering sdb3 ...
[   61.114140] md:  adding sdb3 ...
[   61.114184] md: sdb2 has different UUID to sdb3
[   61.114230] md: sdb1 has different UUID to sdb3
[   61.114278] md:  adding sda3 ...
[   61.114320] md: sda2 has different UUID to sdb3
[   61.114366] md: sda1 has different UUID to sdb3
[   61.114482] md: created md2
[   61.114534] md: bind<sda3>
[   61.114577] md: bind<sdb3>
[   61.114624] md: running: <sdb3><sda3>
[   61.114765] raid1: raid set md2 active with 2 out of 2 mirrors
[   61.114873] md: considering sdb2 ...
[   61.114929] md:  adding sdb2 ...
[   61.114984] md: sdb1 has different UUID to sdb2
[   61.115040] md:  adding sda2 ...
[   61.115084] md: sda1 has different UUID to sdb2
[   61.115219] md: created md1
[   61.115271] md: bind<sda2>
[   61.115317] md: bind<sdb2>
[   61.115361] md: running: <sdb2><sda2>
[   61.115499] raid1: raid set md1 active with 2 out of 2 mirrors
[   61.115609] md: considering sdb1 ...
[   61.115665] md:  adding sdb1 ...
[   61.115710] md:  adding sda1 ...
[   61.115753] md: created md0
[   61.115794] md: bind<sda1>
[   61.115840] md: bind<sdb1>
[   61.115887] md: running: <sdb1><sda1>
[   61.116029] raid1: raid set md0 active with 2 out of 2 mirrors
[   61.116720] md: ... autorun DONE.
[   61.146509] UDF-fs: No VRS found
[   61.146710] Filesystem "md2": Disabling barriers, not supported by the underlying device
[   61.156442] XFS mounting filesystem md2
[   61.237815] Ending clean XFS mount for filesystem: md2
[   61.237835] VFS: Mounted root (xfs filesystem) readonly.
[   61.237927] Freeing unused kernel memory: 216k freed
[   63.224910] Adding 16787768k swap on /dev/md0.  Priority:-1 extents:1 across:16787768k
[   63.273053] Filesystem "md2": Disabling barriers, not supported by the underlying device
[   64.303771] kjournald starting.  Commit interval 5 seconds
[   64.309323] EXT3 FS on md1, internal journal
[   64.309396] EXT3-fs: mounted filesystem with ordered data mode.
[   64.310221] Filesystem "md3": Disabling barriers, not supported by the underlying device
[   64.310483] XFS mounting filesystem md3
[   64.563068] Ending clean XFS mount for filesystem: md3
[   82.790183] e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX
[   84.590210] process `syslogd' is using obsolete setsockopt SO_BSDCOMPAT
[  106.449661] mtrr: no more MTRRs available

Seems stable & fast so far, except when it hit swap :)

top - 18:38:13 up 5 min, 6 users, load average: 38.56, 12.65, 4.52

Tasks: 227 total,  64 running, 163 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us, 23.9%sy,  0.0%ni,  0.0%id, 72.7%wa,  0.0%hi,  3.4%si,  0.0%st
Cpu1  :  0.0%us, 23.5%sy,  0.0%ni,  1.0%id, 75.5%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us, 24.1%sy,  0.0%ni,  0.0%id, 75.9%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us, 23.6%sy,  0.0%ni,  0.0%id, 76.4%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8039576k total,  7998988k used,    40588k free,       12k buffers
Swap: 16787768k total,   879324k used, 15908444k free,    16316k cached

Tested with

$ stress --vm 32 --vm-bytes 250M
and
$ stress --vm 64 --vm-bytes 250M

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  248 root      10  -5     0    0    0 D   18  0.0   0:07.81 kswapd0
 3788 war       18   0  256m 115m  164 D    3  1.5   0:01.32 stress
 3800 war       18   0  256m 7300  160 R    3  0.1   0:00.58 stress
 3801 war       18   0  256m  34m  160 R    3  0.4   0:02.48 stress
 3808 war       18   0  256m 217m  164 R    3  2.8   0:01.16 stress
  559 root      10  -5     0    0    0 D    2  0.0   0:00.15 md0_raid1
 3771 war       18   0  256m 174m  164 R    2  2.2   0:01.30 stress
 3785 war       18   0  256m  30m  164 R    2  0.4   0:01.42 stress
 3813 war       18   0  256m  34m  164 R    2  0.4   0:01.06 stress
 3787 war       18   0  256m  24m  160 R    2  0.3   0:00.25 stress
 3794 war       18   0  256m  40m  160 R    2  0.5   0:00.14 stress
 3778 war       18   0  256m 109m  164 R    2  1.4   0:01.23 stress
 3783 war       18   0  256m 185m  164 R    2  2.4   0:01.29 stress
 3799 war       18   0  256m 172m  164 R    2  2.2   0:01.07 stress
 3816 war       18   0  256m 203m  160 R    2  2.6   0:00.39 stress
 3819 war       18   0  256m 144m  160 R    2  1.8   0:00.40 stress
 3820 war       18   0  256m 147m  160 R    2  1.9   0:00.39 stress
 3825 war       18   0  256m  91m  164 R    2  1.2   0:01.16 stress
 3777 war       18   0  256m  29m  164 R    1  0.4   0:01.05 stress
 3779 war       18   0  256m 197m  164 R    1  2.5   0:01.29 stress
 3805 war       18   0  256m 216m  160 R    1  2.8   0:00.36 stress
 3812 war       18   0  256m  30m  164 R    1  0.4   0:01.14 stress
 3830 war       18   0  256m 176m  164 R    1  2.3   0:01.17 stress
 3831 war       18   0  256m  64m  164 R    1  0.8   0:01.40 stress
 3769 war       18   0  256m  43m  164 R    1  0.6   0:01.23 stress
 3770 war       18   0  256m 169m  164 R    1  2.2   0:01.18 stress
 3773 war       18   0  256m  44m  160 R    1  0.6   0:00.29 stress
 3774 war       18   0  256m  21m  160 R    1  0.3   0:00.21 stress
 3780 war       18   0  256m 167m  164 R    1  2.1   0:01.33 stress
 3781 war       18   0  256m 143m  164 R    1  1.8   0:01.35 stress
 3798 war       18   0  256m 153m  164 R    1  1.9   0:01.16 stress
 3807 war       18   0  256m  31m  164 R    1  0.4   0:01.17 stress
 3817 war       18   0  256m 185m  160 R    1  2.4   0:00.40 stress
 3822 war       18   0  256m 200m  160 R    1  2.6   0:00.41 stress
 3824 war       18   0  256m 8684  160 R    1  0.1   0:00.16 stress
 3827 war       18   0  256m 103m  164 R    1  1.3   0:00.95 stress
 3828 war       18   0  256m 1976  160 R    1  0.0   0:00.19 stress
 2026 daemon    30  15 30580  436  372 D    1  0.0   0:02.10 bindgraph.pl
 3768 war       18   0  256m  81m  164 R    1  1.0   0:01.84 stress
 3772 war       18   0  256m 158m  164 R    1  2.0   0:01.11 stress
 3782 war       18   0  256m 180m  164 R    1  2.3   0:01.25 stress
 3786 war       18   0  256m 173m  164 R    1  2.2   0:01.13 stress
 3790 war       18   0  256m  20m  164 R    1  0.3   0:01.10 stress
 3791 war       18   0  256m  36m  164 R    1  0.5   0:01.80 stress
 3796 war       18   0  256m 111m  164 R    1  1.4   0:01.04 stress
 3797 war       18   0  256m 120m  164 R    1  1.5   0:01.12 stress
 3803 war       18   0  256m 129m  160 R    1  1.6   0:01.98 stress
 3804 war       18   0  256m 234m  160 R    1  3.0   0:00.40 stress
 3806 war       18   0  256m  49m  164 R    1  0.6   0:00.99 stress
 3809 war       18   0  256m  54m  164 R    1  0.7   0:01.16 stress
 3814 war       18   0  256m  38m  164 R    1  0.5   0:00.97 stress
 3815 war       18   0  256m  69m  164 R    1  0.9   0:01.06 stress
  246 root      15   0     0    0    0 D    0  0.0   0:00.01 pdflush
 3687 war       15   0 18012  896  560 R    0  0.0   0:00.88 top
 3775 war       18   0  256m 153m  164 R    0  2.0   0:01.14 stress
 3776 war       18   0  256m  55m  164 R    0  0.7   0:01.34 stress
 3784 war       18   0  256m  13m  164 R    0  0.2   0:00.96 stress
 3789 war       18   0  256m  46m  164 R    0  0.6   0:01.54 stress
 3792 war       18   0  256m  12m  164 R    0  0.2   0:01.00 stress
 3793 war       18   0  256m  44m  164 R    0  0.6   0:00.96 stress
 3795 war       18   0  256m 196m  164 R    0  2.5   0:01.06 stress
 3802 war       18   0  256m  22m  160 R    0  0.3   0:02.70 stress
 3810 war       18   0  256m  40m  164 R    0  0.5   0:01.06 stress
 3818 war       18   0  256m 146m  160 R    0  1.9   0:00.34 stress
 3829 war       18   0  256m 130m  164 R    0  1.7   0:01.11 stress
  247 root      15   0     0    0    0 D    0  0.0   0:00.01 pdflush
  554 root      10  -5     0    0    0 D    0  0.0   0:00.01 md2_raid1
 3811 war       18   0  256m  61m  160 R    0  0.8   0:05.85 stress
 3821 war       18   0  256m 136m  160 R    0  1.7   0:00.37 stress
 3823 war       18   0  256m 204m  160 R    0  2.6   0:00.34 stress
 3826 war       18   0  256m  95m  164 R    0  1.2   0:01.03 stress

or you can just boot with "log_buf_len=256k" on the kernel boot line (e.g.)

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Thanks, very useful note!

$ stress --vm 32 --vm-bytes 250M
stress: info: [4087] dispatching hogs: 0 cpu, 0 io, 32 vm, 0 hdd

Mem:   8039576k total,  5288036k used,  2751540k free,       12k buffers
Swap:        0k total,        0k used,        0k free,    37944k cached

$ stress --vm 40 --vm-bytes 250M
stress: info: [4120] dispatching hogs: 0 cpu, 0 io, 40 vm, 0 hdd
Mem:   8039576k total,  6661484k used,  1378092k free,       12k buffers
Swap:        0k total,        0k used,        0k free,    38000k cached

$ stress --vm 42 --vm-bytes 250M
stress: info: [4398] dispatching hogs: 0 cpu, 0 io, 42 vm, 0 hdd
Mem:   8039576k total,  6447048k used,  1592528k free,        8k buffers
Swap:        0k total,        0k used,        0k free,    11148k cached

$ stress --vm 45 --vm-bytes 250M
stress: info: [4352] dispatching hogs: 0 cpu, 0 io, 45 vm, 0 hdd
stress: FAIL: [4372] (494) hogvm malloc failed: Cannot allocate memory
stress: FAIL: [4352] (395) <-- worker 4372 returned error 1
stress: WARN: [4352] (397) now reaping child worker processes
stress: FAIL: [4352] (452) failed run completed in 4s

$ stress --vm 48 --vm-bytes 250M
stress: info: [4303] dispatching hogs: 0 cpu, 0 io, 48 vm, 0 hdd
stress: FAIL: [4323] (494) hogvm malloc failed: Cannot allocate memory
stress: FAIL: [4322] (494) hogvm malloc failed: Cannot allocate memory
stress: FAIL: [4303] (395) <-- worker 4323 returned error 1
stress: WARN: [4303] (397) now reaping child worker processes
stress: FAIL: [4303] (395) <-- worker 4322 returned error 1
stress: WARN: [4303] (397) now reaping child worker processes
stress: FAIL: [4303] (452) failed run completed in 4s

Is there a better way to verify I can use all the available memory?

Justin.

Hm, not sure if it was from the patch or what but I ran this:

1. swapoff -a
2. ./eatmem

The machine responded to ping and alt-sysrq-b but the box remain unresponsive, I guess the kernel did not kill the process? :(

The moments before it 'froze'

top - 18:48:01 up 15 min, 7 users, load average: 6.61, 18.50, 13.31

Tasks: 200 total,  18 running, 182 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us, 90.7%sy,  0.0%ni,  5.9%id,  3.3%wa,  0.0%hi,  0.0%si, 0.0%st
Mem:   8039576k total,  7998860k used,    40716k free,        8k buffers
Swap:        0k total,        0k used,        0k free,     1664k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  248 root      11  -5     0    0    0 R   85  0.0   0:16.05 kswapd0
 2265 nut       18   0 13320  244    4 R   40  0.0   0:03.13 newhidups
 2267 nut       18   0 12216  168    4 R   40  0.0   0:02.04 upsd
 2474 ntp       18   0 22192  400    8 R   39  0.0   0:02.00 ntpd
 3563 jpiszcz   18   0 41964 1264    4 R   38  0.0   0:02.20 pine
 3530 root      18   0 96240 3132   36 R   37  0.0   0:02.09 kdm_greet
 2052 root      18   0  6080  112    4 R   37  0.0   0:02.00 hald-addon-stor
 4479 war       17   0 18012  700  252 R   33  0.0   0:01.81 top
 4480 war       19   0 6948m 6.8g    4 R   22 88.4   0:05.81 eatmem
 2095 root      18   0 13128  216    8 R   10  0.0   0:00.50 dirmngr
 2545 root      18   0 95788 2488    4 R    5  0.0   0:00.25 apache2
 3564 war       18   0 41620  832    4 R    5  0.0   0:00.34 pine
 2270 nut       15   0 12212  144    4 S    1  0.0   0:00.05 upsmon
  561 root      10  -5     0    0    0 S    0  0.0   0:00.02 xfsbufd

Very simply program:

#include <iostream>
using namespace std;

int main()
{
long int interations = 10000000;
int counter = 1;

  for(counter;counter<interations;counter++)
  {
     double *d = new double[100];
  }

return 0;
}

Any idea why the OOM killer can or does not kill it?

Justin.

You usually have to access the allocated memory, like:

	*d = 1.0;

for it to actually be allocated (AFAIK).

What are the values of /proc/sys/vm/overcommit* ?

See Documentation/vm/overcommit-accounting .

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

They should be the defaults as I do not change them:

p34:~# find /proc/|grep -i overcommit
/proc/sys/vm/overcommit_memory
/proc/sys/vm/overcommit_ratio
find: /proc/5128: No such file or directory
p34:~# cat /proc/sys/vm/overcommit_memory
0
p34:~# cat /proc/sys/vm/overcommit_ratio
50
p34:~#

Comments?

You can be sure your memory is available if reported in /proc/meminfo or at boot, since those represent the actual kernel data structures used for memory allocation:

[    0.000000] On node 0 totalpages: 2061783

That corresponds to 2061783*4k = 8445063168 bytes or ~8053M. Is that fairly close to what's actually installed in the machine?

Note that your boot also mentions this:

[  106.449661] mtrr: no more MTRRs available

which indicates that things like X may not be able to map the framebuffer with the 'write-combine' attribute, which will hurt performance. I've heard reports that turning of 'Intel QST fan control' in your BIOS settings will prevent all your MTRRs from being used (improperly, probably another BIOS bug) so that X will perform well. But if you don't use X on this machine, you don't have to worry about it. The other option would be to remap your MTRRs by hand to free one up for X, you can do that by combining the last one or two entries into a single MTRR using the API described in Documentation/mtrr.txt before you start X.

Jesse

FYI--

[  106.449661] mtrr: no more MTRRs available

This has always occurred, even with mem=8832M setting.

Justin.

Did the dmesg output get you what you needed? Why the few KB difference? :)

Justin.

Yeah, looked at your e820 and your MTRR settings and I think my patch is doing the right thing (i.e. trimming just the right amount of memory, leaving you with as much as possible).

The mem= approach though looks slightly off, but I haven't looked at x86_64's mem= handling to see why. From a high level though, adjusting end_pfn is the right thing to do, since theoretically mem= could choose to make holes in your low memory and keep your high memory in the allocation pools (though it's not generally implemented this way).

Jesse

Ahh, ok! Sounds great, I will keep running the kernel with your patch without mem= and let you know if I see any issues.

Chances of getting this into 2.6.22-rc5?

Justin.

I'm not sure it's appropriate for -rc5 since it mucks around with some early boot ordering, but I'll leave that to Andi, since it does address some real bugs people have been seeing.

Can we add your "Tested-by: Justin Piszcz <[email protected]>" to the patch? :)

Thanks,
Jesse

I don't think the patch is suitable for merging at this time. Perhaps if it survives some time in -mm* / 2.6.23* it could be backported in a later 2.6.22 stable release. But right now it definitely needs more testing and addressing of my review comments.

All such headers are only for the trail of blame and do you want to blame Justin if anything goes wrong? Perhaps it should rather have a
Blame-to: <whoever wrote Justin/Jesse's BIOS> but that also wouldn't help without concrete contact points.

-ANdi

Hah! Again, I'll keep runnihg with Jesse's patch and as long as I can keep patching newer kernels I can continue to run with it. So far, overnight with backups and the like, I have not noticed any problems. Also tested logging in to X/KDE, no issues. [yet]

Justin.

BTW an unrelated/happy side-effect of the patch is that it removes the zero-size-guilty kmalloc()'s from the mtrr code. -mm does have some fixes that prevent those zero-size allocations, which could likely be retired if this gets merged in? ...

[ On another unrelated note, could someone tell me who maintains i386? I've looked around, but surprisingly can't seem to find anybody listed anywhere. ]

Satyam

I think that header would be Lame-workaround-needed-because-of:
<[email protected]>. :) The idea of tested-by is to give people a clue about who would be able to test any changes in the area affected. So far from blaming Justin, it would give him credit for all his testing, and let people know that he might be able to test similar patches in the future. I think it's worthwhile to track that...

Jesse

Ok. Overall this feels good but a few nits below. Would it make sense to split this into two patches. The first to just do the cleanup that removes the allocations for holding the mttr ranges?

I suppose we could split it, but it's small, and the only reason for removing the allocations was so that we could init it earlier.

You're right, I should have kept the old name with MAX_ in it. I'll fix it up.

Oops, yeah, this should just be in mtrr.h.

Overlapping should be ok, since that's usually intentional (e.g. one big wb range with a portion of uc space due to another mtrr).

Holes definitely aren't dealt with, but then we haven't seen any yet...

That would be useful, but only if we moved the check to a little earlier, prior to the addition of the active ranges from the e820. Might be a little nicer than adjusting end_pfn, but will ultimately achieve the same thing...

Jesse

I'm not say overlapping was a bug. I was saying that you don't handle overlapping mtrrs in figuring the last cached addresses. Therefore when a UC range overlaps a WB range we might thing the last page in the WB range is cached when it is not.

Yes, with perhaps a little more consistency.

Eric

Oh right, that might be an issue, but we haven't seen it yet...

Jesse

In theory -- while not recommended -- a BIOS could also
use a default fallback MTRR for cached and use explicit MTRRs to map the non existing ranges uncached. Would it make sense to handle this case?

Right now if someone used a default WC MTRR to make the memory cached you would clip all memory.

Perhaps a fail safe would be good -- always leave some memory left over even if it looks wrong.

Should also probably have some command line option to disable the check in case something bad happens with it.

Another thing that might be sense to investigate in relationship to this patch is large page mappings with MTRRs. iirc P4 and also K8 splits pages internally with MTRR boundaries and might have some other bad side effects. Should we use this as hints to use 4K pages for the boundary areas?

-Andi

Probably. I could just check the default memory type and bail out if it's cacheable.

Sure.

Or I could trim to the nearest large page boundary... We'd lose a little more memory but it would keep things simple.

Jesse

How much more memory are we going to lose? Is mem= a better option if its going to keep decreasing?

Hey, this was already there... I didn't want to rewrite the whole thing at once. :) Patch looks fine though.

Thanks,
Jesse

Hi!

Way too obnoxious warning, I'd say. Just drop the *s.

(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

i386 allmodconfig:

arch/i386/kernel/cpu/mtrr/main.c: In function 'mtrr_trim_uncached_memory':
arch/i386/kernel/cpu/mtrr/main.c:655: error: 'end_pfn' undeclared (first use in this function) arch/i386/kernel/cpu/mtrr/main.c:655: error: (Each undeclared identifier is reported only once arch/i386/kernel/cpu/mtrr/main.c:655: error: for each function it appears in.)

I'll poke some ifdefs in there.

Oops, sorry about that, the code is only called on x86_64 but compiled even for i386...

Jesse

A quick update. This patch is horribly incorrect on a socket F opteron/Athlon 64 with memory above 4GB.

In particular those cpus are capable of mapping all of memory above 4GB as write back without using a single MTRR.

So examining MTRRs is insufficient.

Eric

Hm, yuck. What do you suggest? Should we only run this check when Intel chips are present? Checking only the bottom 4G isn't sufficient since we've seen platforms that have issues above that range...

Thanks,
Jesse

My gut feel says that we need to call a function that is potentially cpu specific, older AMD cpus and Intel cpus can just use the generic mtrr code.

I would also suggest we build a list of ranges of write-back memory. Which until we handle overlapping MTRRs in the generic MTRR case is just the write-back MTRRs.

Then we get the data in a linux specific form we can check the linux specific data structure against the e820 map.

I don't think that is going to much harder and it allows for creative cpu designers.

Although this suggests that we want to worry about all memory holes as well. Because I have seen at least one system which failed to cover the lower 4G with MTRRs. While everything above 4G was fine.

Eric

I assume this cannot be fixed by the simple approach of echoing some useful numbers into /proc/mtrr like we used to do for video memory? (Before X did this automatically?)

An extra bootscript seems better than loosing memory.

Helge Hafting

In some cases it probably can, in other cases not because the memory controller is misconfigured or not the right bits are set in the PCI bridges to enable DAC IO or ... or ...

There are also definite limits on how much quirks can do to fix this -- Linux is a generic kernel, not a BIOS replacement, and can never be as intimate with the current setup
as the BIOS is.

It's definitely far safer to not use the memory. You're running in a situation never tested or considered by the motherboard vendor and everything is possible.

-Andi