platform/x86: ISST: Optimize CPU to PCI device mapping
PATCH 1/2 platform/x86: ISST: Optimize CPU to PCI device mapping
It was observed that some of the high performance benchmarks are spending more time in kernel depending on which CPU package they are executing. The difference is significant and benchmark scores varies more than 10%. These benchmarks adjust class of service to improve thread performance which run in parallel. This class of service change causes access to MMIO region of Intel Speed Select PCI devices depending on the CPU package they are executing.
This mapping from CPU to PCI device instance uses a standard Linux PCI interface "pci_get_domain_bus_and_slot()". This function does a linear search to get to a PCI device. Since these platforms have 100+ PCI devices, this search can be expensive in fast path for benchmarks.
Since the device and function of PCI device is fixed for Intel Speed Select PCI devices, the CPU to PCI device information can be cached at the same time when bus number for the CPU is read. In this way during runtime the cached information can be used. This improves performance of these benchmarks significantly.
--- .../intel_speed_select_if/isst_if_common.c | 29 +++++++++++++++---- 1 file changed, 24 insertions(+), 5 deletions(-)
drivers/platform/x86/intel_speed_select_if/isst_if_common.c +24 -5
@@ -281,11 +281,27 @@ static int isst_if_get_platform_info(void __user *argp)struct isst_if_cpu_info {/* For BUS 0 and BUS 1 only, which we need for PUNIT interface */ int bus_info[2 ]; +struct pci_dev *pci_dev[2 ];int punit_cpu_id; };static struct isst_if_cpu_info *isst_cpu_info; +static struct pci_dev *_isst_if_get_pci_dev (int cpu,int bus_no,int dev,int fn) +{ +int bus_number; + +if (bus_no <0 || bus_no >1 || cpu <0 || cpu >= nr_cpu_ids || + cpu >=num_possible_cpus ()) +return NULL ; + + bus_number = isst_cpu_info[cpu].bus_info[bus_no]; +if (bus_number <0 ) +return NULL ; + +return pci_get_domain_bus_and_slot (0 , bus_number,PCI_DEVFN (dev, fn)); +} +/** * isst_if_get_pci_dev() - Get the PCI device instance for a CPU * @cpu: Logical CPU number.
@@ -300,17 +316,18 @@ static struct isst_if_cpu_info *isst_cpu_info;*/ struct pci_dev *isst_if_get_pci_dev (int cpu,int bus_no,int dev,int fn) { -int bus_number; +struct pci_dev *pci_dev;if (bus_no <0 || bus_no >1 || cpu <0 || cpu >= nr_cpu_ids || cpu >=num_possible_cpus ())return NULL ; - bus_number = isst_cpu_info[cpu].bus_info[bus_no]; -if (bus_number <0 ) -return NULL ; + pci_dev = isst_cpu_info[cpu].pci_dev[bus_no]; -return pci_get_domain_bus_and_slot (0 , bus_number,PCI_DEVFN (dev, fn)); +if (pci_dev->devfn ==PCI_DEVFN (dev, fn)) +return pci_dev; + +return _isst_if_get_pci_dev (cpu, bus_no, dev, fn); }EXPORT_SYMBOL_GPL (isst_if_get_pci_dev);
@@ -327,6 +344,8 @@ static int isst_if_cpu_online(unsigned int cpu) }else { isst_cpu_info[cpu].bus_info[0 ] = data &0xff ; isst_cpu_info[cpu].bus_info[1 ] = (data >>8 ) &0xff ; + isst_cpu_info[cpu].pci_dev[0 ] =_isst_if_get_pci_dev (cpu,0 ,0 ,1 ); + isst_cpu_info[cpu].pci_dev[1 ] =_isst_if_get_pci_dev (cpu,1 ,30 ,1 ); } ret =rdmsrl_safe (MSR_THREAD_ID_INFO , &data);
Signature · 1 line
2.30.2
PATCH 2/2 platform/x86: ISST: Use numa node id for cpu pci dev mapping
There is a problem in mapping CPU to a PCI device instance when the bus numbers are reused in different packages. This was observed on some Sapphire Rapids systems.
The current implementation reads bus number assigned to a CPU package via MSR 0x128. This allows to establish relationship between a CPU and a PCI device. This allows to update power related parameters to a MMIO offset in a PCI device space which is unique to a CPU. But if two packages uses same bus number then this mapping will not be unique.
When bus number is reused, PCI device will use different domain number or segment number. So we need to be aware of this domain information while matching CPU to PCI bus number. This domain information is not available via any MSR. So need to use ACPI numa node information.
There is an interface already available in the Linux to read numa node for a CPU and a PCI device. This change uses this interface to check the numa node of a match PCI device with bus number. If the bus number and numa node matches with the CPU's assigned bus number and numa node, the matched PCI device instance will be returned to the caller.
It is possible that before Sapphire Rapids, the numa node is not defined for the Speed Select PCI device in some OEM systems. In this case to restore old behavior, return the last matched PCI device for domain 0 unlsess there are more than one matches.
--- .../intel_speed_select_if/isst_if_common.c | 48 ++++++++++++++++++- 1 file changed, 46 insertions(+), 2 deletions(-)
drivers/platform/x86/intel_speed_select_if/isst_if_common.c +46 -2
@@ -283,13 +283,18 @@ struct isst_if_cpu_info {int bus_info[2 ];struct pci_dev *pci_dev[2 ];int punit_cpu_id; +int numa_node; };static struct isst_if_cpu_info *isst_cpu_info; +#define ISST_MAX_PCI_DOMAINS 8static struct pci_dev *_isst_if_get_pci_dev (int cpu,int bus_no,int dev,int fn) { -int bus_number; +struct pci_dev *matched_pci_dev =NULL ; +struct pci_dev *pci_dev =NULL ; +int no_matches =0 ; +int i, bus_number;if (bus_no <0 || bus_no >1 || cpu <0 || cpu >= nr_cpu_ids || cpu >=num_possible_cpus ())
@@ -299,7 +304,45 @@ static struct pci_dev *_isst_if_get_pci_dev(int cpu, int bus_no, int dev, int fnif (bus_number <0 )return NULL ; -return pci_get_domain_bus_and_slot (0 , bus_number,PCI_DEVFN (dev, fn)); +for (i =0 ; i <ISST_MAX_PCI_DOMAINS ; ++i) { +struct pci_dev *_pci_dev; +int node; + + _pci_dev =pci_get_domain_bus_and_slot (i, bus_number,PCI_DEVFN (dev, fn)); +if (!_pci_dev) +continue ; + + ++no_matches; +if (!matched_pci_dev) + matched_pci_dev = _pci_dev; + + node =dev_to_node (&_pci_dev->dev); +if (node ==NUMA_NO_NODE ) { +pr_info ("Fail to get numa node for CPU:%d bus:%d dev:%d fn:%d\n" , + cpu, bus_no, dev, fn); +continue ; + } + +if (node == isst_cpu_info[cpu].numa_node) { + pci_dev = _pci_dev; +break ; + } + } + +/* +* If there is no numa matched pci_dev, then there can be following cases: +* 1. CONFIG_NUMA is not defined: In this case if there is only single device +* match, then we don't need numa information. Simply return last match. +* Othewise return NULL. +* 2. NUMA information is not exposed via _SEG method. In this case it is similar +* to case 1. +* 3. Numa information doesn't match with CPU numa node and more than one match +* return NULL. +*/ +if (!pci_dev && no_matches ==1 ) + pci_dev = matched_pci_dev; + +return pci_dev; }/**
@@ -354,6 +397,7 @@ static int isst_if_cpu_online(unsigned int cpu)return ret; } isst_cpu_info[cpu].punit_cpu_id = data; + isst_cpu_info[cpu].numa_node =cpu_to_node (cpu);isst_restore_msr_local (cpu);
Signature · 1 line
2.30.2