Fam10+ extended config space enabling
Robert,
starting with 2.6.27 Linux is doing this, using MSR accesses (these days in CPU hotplug notification callbacks). Is there a reason this cannot be done through PCI config space writes (as the register is aliased), and then only once at boot time (perhaps utilizing some of the code in arch/x86/kernel/k8.c)?
I'm particularly asking from the Xen perspective, where the MSR based mechanism won't reliably work (since there's no guarantee all of Dom0's virtual CPUs would, at initialization time, actually cover all physical CPUs that Dom0 may ever run on).
Thanks, Jan
Quoted 8 lines · Jan Beulich
Robert,
starting with 2.6.27 Linux is doing this, using MSR accesses (these days in CPU hotplug notification callbacks). Is there a reason this cannot be done through PCI config space writes (as the register is aliased), and then only once at boot time (perhaps utilizing some of the code in
arch/x86/kernel/k8.c)?
Quoted 4 lines
I'm particularly asking from the Xen perspective, where the MSR based mechanism won't reliably work (since there's no guarantee all of Dom0's virtual CPUs would, at initialization time, actually cover all physical CPUs that Dom0 may ever run on).
Just thinking, could we simply do this in Xen, either unconditionally, or as soon as we detect dom0 doing it for at least one CPU?
-- Keir
Quoted 7 lines
Thanks, Jan
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 11 lines
Quoted 1 line
Quoted 1 line · Keir Fraser
Quoted 8 lines · Jan Beulich
Robert,
starting with 2.6.27 Linux is doing this, using MSR accesses (these days in CPU hotplug notification callbacks). Is there a reason this cannot be done through PCI config space writes (as the register is aliased), and then only once at boot time (perhaps utilizing some of the code in
arch/x86/kernel/k8.c)?
Quoted 7 lines
Quoted 4 lines
I'm particularly asking from the Xen perspective, where the MSR based mechanism won't reliably work (since there's no guarantee all of Dom0's virtual CPUs would, at initialization time, actually cover all physical CPUs that Dom0 may ever run on).
Just thinking, could we simply do this in Xen, either unconditionally, or as soon as we detect dom0 doing it for at least one CPU?
We certainly could (albeit not unconditional I'd say; this was also my first thought, with the alternative of adding a specific hypercall), but it would mean adding special code for something that can be done in Dom0 (and there even in a way that's suitable for both native and Xen) if using the PCI config space method for this is possible.
Jan
Quoted 9 lines · Keir Fraser
Quoted 5 lines · Jan Beulich
starting with 2.6.27 Linux is doing this, using MSR accesses (these days in CPU hotplug notification callbacks). Is there a reason this cannot be done through PCI config space writes (as the register is aliased), and then only once at boot time (perhaps utilizing some of the code in
arch/x86/kernel/k8.c)?From reading the documentation this should work too. The msr controls
a Northbridge function and thus is per-node. NB function msrs are shared between all cores on the node and there is one instance per node.
IIRC there were several reasons to choose msr access. During early cpu bringup there are only some basic early pci access functions available, requiring a little more complex code to scan NB devices. I think in between there is code available that could be shared. Second, if we implement the setup using pci access, we must adopt the code with every new cpu family with new nb devices to support pci ecs with new cpus. So, new cpus couldn't use pci ecs with an old kernel then.
But the msr 0xC001001F is not architectural. Actually this means the assumption in the code is wrong, that all cpus from fam10h have IO ECS. There might be future cpus not supporting it. But in practice all current cpu families from family 10h support the msr (10h, 11h, 12h, 14h, 15h).
Quoted 7 lines
Quoted 4 lines
I'm particularly asking from the Xen perspective, where the MSR based mechanism won't reliably work (since there's no guarantee all of Dom0's virtual CPUs would, at initialization time, actually cover all physical CPUs that Dom0 may ever run on).
Just thinking, could we simply do this in Xen, either unconditionally, or as soon as we detect dom0 doing it for at least one CPU?
I don't know if xen is capable to emulate per-node msrs, presumable not. I don't have an overview which other per-node msrs might be worth to be supported by xen. If so, it might be better to implement support for per-node msrs.
But I am fine with a setup with pci access for some nb devices, but would rather leave the setup via msrs as a fallback.
-Robert
Signature · 2 lines
Advanced Micro Devices, Inc. Operating System Research Center
Quoted 5 lines
Quoted 1 line
Quoted 1 line · Robert Richter
I don't know if xen is capable to emulate per-node msrs, presumable not. I don't have an overview which other per-node msrs might be worth to be supported by xen. If so, it might be better to implement support for per-node msrs.
The question isn't about emulating, but about letting Dom0 control the whole platform (if e.g. Dom0 is booted with just 2 [virtual] CPUs on a large system, it's clear that it'll be very unlikely that it manages to enabled ECS on all nodes when it issues the MSR writes on just the two physical CPUs underlying two virtual ones happen to run on).
Quoted 2 lines
But I am fine with a setup with pci access for some nb devices, but would rather leave the setup via msrs as a fallback.
Okay, I'll craft something long those lines.
Jan
PATCH 1/1 x86, AMD, PCI: Add AMD northbridge PCI device id for CPU families 12h and 14h
Quoted 6 lines · Jan Beulich
Quoted 4 lines
Quoted 1 line
Quoted 1 line · Robert Richter
But I am fine with a setup with pci access for some nb devices, but would rather leave the setup via msrs as a fallback.
Okay, I'll craft something long those lines.
Jan,
the patch below implements the PCI nb device id for AMD cpu families 12h and 14h.
Ingo,
please apply to tip. There are some future use cases that use this PCI device and we would like to clarify its naming.
Thanks,
-Robert
--
Quoted 1 line
From
f791e16fbbf30d1ab6c66ebb8daa77a6c86bda0bMon Sep 17 00:00:00 2001
From: Robert Richter <[email protected]>
Date: Thu, 6 Jan 2011 15:01:11 +0100
Subject: [PATCH] x86, AMD, PCI: Add AMD northbridge PCI device id for CPU families 12h and 14h
This patch adds the PCI northbridge device id for AMD CPU families 12h and 14h. Both families have implemented the same PCI northbridge device.
--- include/linux/pci_ids.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
include/linux/pci_ids.h +1 -0
@@ -518,6 +518,7 @@#define PCI_DEVICE_ID_AMD_11H_NB_MISC 0x1303#define PCI_DEVICE_ID_AMD_11H_NB_LINK 0x1304#define PCI_DEVICE_ID_AMD_15H_NB_MISC 0x1603 +#define PCI_DEVICE_ID_AMD_CNB17H_F3 0x1703#define PCI_DEVICE_ID_AMD_LANCE 0x2000#define PCI_DEVICE_ID_AMD_LANCE_HOME 0x2001#define PCI_DEVICE_ID_AMD_SCSI 0x2020
Signature · 8 lines
1.7.3.4 -- Advanced Micro Devices, Inc. Operating System Research Center
[tip:x86/urgent] x86, AMD, PCI: Add AMD northbridge PCI device id for CPU families 12h and 14h
Commit-ID: ca86828ccd3128513f6d4e200b437deac95408db Gitweb: http://git.kernel.org/tip/ca86828ccd3128513f6d4e200b437deac95408db Author: Robert Richter <[email protected]> AuthorDate: Thu, 6 Jan 2011 17:51:07 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 7 Jan 2011 14:00:14 +0100
x86, AMD, PCI: Add AMD northbridge PCI device id for CPU families 12h and 14h
This patch adds the PCI northbridge device id for AMD CPU families 12h and 14h. Both families have implemented the same PCI northbridge device.
There are some future use cases that use this PCI device and we would like to clarify its naming.
LKML-Reference: <[email protected]>
--- include/linux/pci_ids.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
include/linux/pci_ids.h +1 -0
@@ -518,6 +518,7 @@#define PCI_DEVICE_ID_AMD_11H_NB_MISC 0x1303#define PCI_DEVICE_ID_AMD_11H_NB_LINK 0x1304#define PCI_DEVICE_ID_AMD_15H_NB_MISC 0x1603 +#define PCI_DEVICE_ID_AMD_CNB17H_F3 0x1703#define PCI_DEVICE_ID_AMD_LANCE 0x2000#define PCI_DEVICE_ID_AMD_LANCE_HOME 0x2001#define PCI_DEVICE_ID_AMD_SCSI 0x2020