x86/amd: Restore early_fill_mp_bus_to_node]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · On Mon, 27 Feb 2012 10:44:06 +0100 Ingo Molnar <[email protected]> wrote:
Yours?
----- Forwarded message from Andreas Herrmann <[email protected]> -----
Date: Fri, 24 Feb 2012 16:19:12 +0100
From: Andreas Herrmann <[email protected]>
To: Ingo Molnar <[email protected]>, Thomas Gleixner <[email protected]>, "H. Peter Anvin"<[email protected]>CC: [email protected]
Subject: [PATCH 1/2] x86/amd: Restoreearly_fill_mp_bus_to_nodeOnce upon a time this function was overloaded with quirky stuff to fix resource detection on systems w/
_CRSdefects (seems that some Sun and HP systems were affected).See commit
30a18d6c3f1e774de656ebd8ff219d53e2ba4029
(x86: multi pci root bus with different io resource range, on 64-bit)Restore the old function and thus decouple it from the quirk that is CPU family specific (e.g. it won't work on AMD family 15h CPUs). BTW, I assume that the
_CRSstuff is working on current systems.
Quoted 6 lines
This is required to properly initilize the
numa_nodeinformation of PCI busses and associated devices.Signed-off-by Andreas Herrmann <[email protected]>--- arch/x86/pci/amd_bus.c | 84 +++++++++++++++++++++++++++++++----------------
ISTR this function getting things wrong on some Nehalem based systems, and assigning all IO devices to node 0, resulting in bad NIC performance.
Jesse, are things working ok now? Care to test this patch to make sure it doesn't regress anything on one of the previously affected systems?
Thanks,
Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
Quoted 30 lines · Jesse Barnes
Quoted 27 lines · On Mon, 27 Feb 2012 10:44:06 +0100 Ingo Molnar <[email protected]> wrote:
Yours?
----- Forwarded message from Andreas Herrmann <[email protected]> -----
Date: Fri, 24 Feb 2012 16:19:12 +0100
From: Andreas Herrmann <[email protected]>
To: Ingo Molnar <[email protected]>, Thomas Gleixner <[email protected]>, "H. Peter Anvin"<[email protected]>CC: [email protected]
Subject: [PATCH 1/2] x86/amd: Restoreearly_fill_mp_bus_to_nodeOnce upon a time this function was overloaded with quirky stuff to fix resource detection on systems w/
_CRSdefects (seems that some Sun and HP systems were affected).See commit
30a18d6c3f1e774de656ebd8ff219d53e2ba4029
(x86: multi pci root bus with different io resource range, on 64-bit)Restore the old function and thus decouple it from the quirk that is CPU family specific (e.g. it won't work on AMD family 15h CPUs). BTW, I assume that the
_CRSstuff is working on current systems.This is required to properly initilize the
numa_nodeinformation of PCI busses and associated devices.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Andreas Herrmann <[email protected]>--- arch/x86/pci/amd_bus.c | 84 +++++++++++++++++++++++++++++++----------------ISTR this function getting things wrong on some Nehalem based systems, and assigning all IO devices to node 0, resulting in bad NIC performance.
Are you referring to the original early_fill_mp_bus_to_node function? I don't see how this can happen as it's only called in case of existence of AMD CPU NB devices. Those won't be detected on Intel at all.
Quoted 3 lines
Jesse, are things working ok now? Care to test this patch to make sure it doesn't regress anything on one of the previously affected systems?
Andreas
PATCH RESEND 1/2 x86/amd: Restore early_fill_mp_bus_to_node
In reply to Jesse BarnesOnce upon a time this function was overloaded with quirky stuff to fix resource detection on systems w/ _CRS defects (seems that some Sun and HP systems were affected).
See commit 30a18d6c3f1e774de656ebd8ff219d53e2ba4029
(x86: multi pci root bus with different io resource range, on 64-bit)
Restore the old function and thus decouple it from the quirk that is CPU family specific (e.g. it won't work on AMD family 15h CPUs). BTW, I assume that the _CRS stuff is working on current systems.
This is required to properly initilize the numa_node information of PCI busses and associated devices.
--- arch/x86/pci/amd_bus.c | 84 +++++++++++++++++++++++++++++++---------------- 1 files changed, 55 insertions(+), 29 deletions(-)
Patch didn't show up in any tree so far.
Thus resending it to get this finally fixed.
Please apply.
Thanks,
Andreas
arch/x86/pci/amd_bus.c +55 -29
@@ -30,36 +30,19 @@ static struct pci_hostbridge_probe pci_probes[] __initdata = { { 0, 0x18, PCI_VENDOR_ID_AMD, 0x1300 }, }; -#define RANGE_NUM 16 -/** * early_fill_mp_bus_to_node() * called before pcibios_scan_root and pci_scan_bus * fills the mp_bus_to_cpumask array based according to the LDT Bus Number * Registers found in the K8 northbridge */ -static int __init early_fill_mp_bus_info (void ) +static int __init early_fill_mp_bus_to_node (void ) { -int i; -int j; -unsigned bus; -unsigned slot; -int node; -int link; -int def_node; -int def_link; +int i, j, node, link; +unsigned bus, slot;struct pci_root_info *info;u32 reg; -struct resource *res; -u64 start; -u64 end; -struct range range[RANGE_NUM ]; -u64 val; -u32 address;bool found; -struct resource fam10h_mmconf_res, *fam10h_mmconf; -u64 fam10h_mmconf_start; -u64 fam10h_mmconf_end;if (!early_pci_allowed ())return -1 ;
@@ -67,8 +50,7 @@ static int __init early_fill_mp_bus_info(void) found = false;for (i =0 ; i <ARRAY_SIZE (pci_probes); i++) {u32 id; -u16 device; -u16 vendor; +u16 device, vendor; bus = pci_probes[i].bus; slot = pci_probes[i].slot;
@@ -88,8 +70,7 @@ static int __init early_fill_mp_bus_info(void) pci_root_num =0 ;for (i =0 ; i <4 ; i++) { -int min_bus; -int max_bus; +int min_bus, max_bus; reg =read_pci_config (bus, slot,1 ,0xe0 + (i <<2 ));/* Check if that register is enabled for bus range */
@@ -111,9 +92,50 @@ static int __init early_fill_mp_bus_info(void) info->node = node; info->link = link;sprintf (info->name,"PCI Bus #%02x" , min_bus); +printk (KERN_DEBUG "bus: [%02x, %02x] on node %x link %x\n" , + info->bus_min, info->bus_max, info->node, info->link); pci_root_num++; } +return 0 ; +} + + +#define RANGE_NUM 16 +static int __init early_fill_mp_bus_info (void ) +{ +int i, j, node, link, def_node, def_link; +unsigned bus, slot; +struct pci_root_info *info; +struct resource *res; +struct resource fam10h_mmconf_res, *fam10h_mmconf; +struct range range[RANGE_NUM ]; +u64 fam10h_mmconf_start, fam10h_mmconf_end; +u64 start, end, val; +u32 reg, address; +bool found; + + found = false; +for (i =0 ; i <ARRAY_SIZE (pci_probes); i++) { +u32 id; +u16 device, vendor; + + bus = pci_probes[i].bus; + slot = pci_probes[i].slot; + id =read_pci_config (bus, slot,0 ,PCI_VENDOR_ID ); + + vendor = id &0xffff ; + device = (id>>16 ) &0xffff ; +if (pci_probes[i].vendor == vendor && + pci_probes[i].device == device) { + found = true; +break ; + } + } + +if (!found) +return 0 ; +/* get the default node and link for left over res */ reg =read_pci_config (bus, slot,0 ,0x60 ); def_node = (reg >>8 ) &0x07 ;
@@ -310,14 +332,11 @@ static int __init early_fill_mp_bus_info(void) }for (i =0 ; i < pci_root_num; i++) { -int res_num; -int busnum; +int res_num, busnum; info = &pci_root_info[i]; res_num = info->res_num; busnum = info->bus_min; -printk (KERN_DEBUG "bus: [%02x, %02x] on node %x link %x\n" , - info->bus_min, info->bus_max, info->node, info->link);for (j =0 ; j < res_num; j++) { res = &info->res[j]; printk(KERN_DEBUG "bus: %02x index %x %pR\n",
@@ -412,7 +431,14 @@ static int __init amd_postcore_init(void)if (boot_cpu_data.x86_vendor !=X86_VENDOR_AMD )return 0 ; -early_fill_mp_bus_info (); +if ((early_fill_mp_bus_to_node () ==0 ) && + (boot_cpu_data.x86 <0x11 )) { +/* +* call this only on older systems w/o _CRS for "multi +* pci root bus" +*/ +early_fill_mp_bus_info (); + }pci_io_ecs_init ();return 0 ;
Signature · 1 line
1.7.8.5
PATCH RESEND 2/2 x86/amd: Enable early_fill_mp_bus_to_node on AMD family 15h models 0-0xf
While at it avoid calling this function on family 11h (aka Griffin) which was a mobile part and doesn't support NUMA.
--- arch/x86/pci/amd_bus.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/pci/amd_bus.c +1 -1
@@ -27,7 +27,7 @@ static struct pci_hostbridge_probe pci_probes[] __initdata = { { 0, 0x18, PCI_VENDOR_ID_AMD, 0x1100 }, { 0, 0x18, PCI_VENDOR_ID_AMD, 0x1200 }, { 0xff, 0, PCI_VENDOR_ID_AMD, 0x1200 }, - { 0, 0x18, PCI_VENDOR_ID_AMD, 0x1300 }, + { 0, 0x18, PCI_VENDOR_ID_AMD, 0x1600 }, };/**
Signature · 1 line
1.7.8.5