PCI: Linux kernel SR-IOV support
Subsystems (5)
v5 cover PCI: Linux kernel SR-IOV support
Greetings,
Following patches are intended to support SR-IOV capability in the Linux kernel. With these patches, people can turn a PCI device with the capability into multiple ones from software perspective, which will benefit KVM and achieve other purposes such as QoS, security, and etc.
Major changes between v4 -> v5:
1, remove interfaces for PF driver to create sysfs entries (Matthew Wilcox) 2, get ride of 'struct kobject' used in 'struct pci_iov' (Greg KH) 3, split big chunk of code into more patches
4, add boot options to reassign resources under a bus
5, add boot option to align MMIO resources of a device
---
[PATCH 1/15 v5] PCI: remove unnecessary arg of pci_update_resource() [PATCH 2/15 v5] PCI: define PCI resource names in an 'enum'
[PATCH 3/15 v5] PCI: export __pci_read_base
[PATCH 4/15 v5] PCI: make pci_alloc_child_bus() be able to handle bridge device [PATCH 5/15 v5] PCI: add a wrapper for resource_alignment()
[PATCH 6/15 v5] PCI: add a new function to map BAR offset
[PATCH 7/15 v5] PCI: cleanup pcibios_allocate_resources()
[PATCH 8/15 v5] PCI: add boot option to reassign resources
[PATCH 9/15 v5] PCI: add boot option to align MMIO resource
[PATCH 10/15 v5] PCI: cleanup pci_bus_add_devices()
[PATCH 11/15 v5] PCI: split a new function from pci_bus_add_devices() [PATCH 12/15 v5] PCI: support the SR-IOV capability
[PATCH 13/15 v5] PCI: reserve bus range for the SR-IOV device [PATCH 14/15 v5] PCI: document the changes
[PATCH 15/15 v5] PCI: document the new PCI boot parameters
---
Single Root I/O Virtualization (SR-IOV) capability defined by PCI-SIG is intended to enable multiple system software to share PCI hardware resources. PCI device that supports this capability can be extended to one Physical Functions plus multiple Virtual Functions. Physical Function, which could be considered as the "real" PCI device, reflects the hardware instance and manages all physical resources. Virtual Functions are associated with a Physical Function and shares physical resources with the Physical Function.Software can control allocation of Virtual Functions via registers encapsulated in the capability structure.
SR-IOV specification can be found at
http://www.pcisig.com/members/downloads/specifications/iov/sr-iov1.0_11Sep07.pdf
Devices that support SR-IOV are available from following vendors:
http://download.intel.com/design/network/ProdBrf/320025.pdf
http://www.netxen.com/products/chipsolutions/NX3031.html
http://www.neterion.com/products/x3100.html
PATCH v5 1/15 PCI: remove unnecessary arg of pci_update_resource()
This cleanup removes unnecessary argument 'struct resource *res' in pci_update_resource(), so it takes same arguments as other companion functions (pci_assign_resource(), etc.).
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 4 ++-- drivers/pci/setup-res.c | 7 ++++--- include/linux/pci.h | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-)
drivers/pci/pci.c +2 -2
@@ -376,8 +376,8 @@ pci_restore_bars(struct pci_dev *dev)return ; } -for (i =0 ; i < numres; i ++) -pci_update_resource (dev, &dev->resource[i], i); +for (i =0 ; i < numres; i++) +pci_update_resource (dev, i); }static struct pci_platform_pm_ops *pci_platform_pm;
drivers/pci/setup-res.c +4 -3
@@ -26,11 +26,12 @@#include "pci.h" -void pci_update_resource (struct pci_dev *dev,struct resource *res,int resno) +void pci_update_resource (struct pci_dev *dev,int resno) {struct pci_bus_region region;u32 new, check, mask;int reg; +struct resource *res = dev->resource + resno;/* * Ignore resources for unimplemented BARs and unused resource slots
@@ -162,7 +163,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno) }else { res->flags &= ~IORESOURCE_STARTALIGN ;if (resno <PCI_BRIDGE_RESOURCES ) -pci_update_resource (dev, res, resno); +pci_update_resource (dev, resno); }return ret;
@@ -197,7 +198,7 @@ int pci_assign_resource_fixed(struct pci_dev *dev, int resno)dev_err (&dev->dev,"BAR %d: can't allocate %s resource %pR\n" , resno, res->flags &IORESOURCE_IO ?"I/O" :"mem" , res); }else if (resno <PCI_BRIDGE_RESOURCES ) { -pci_update_resource (dev, res, resno); +pci_update_resource (dev, resno); }return ret;
include/linux/pci.h +1 -1
@@ -626,7 +626,7 @@ int pcix_get_mmrbc(struct pci_dev *dev);int pcix_set_mmrbc (struct pci_dev *dev,int mmrbc);int pcie_get_readrq (struct pci_dev *dev);int pcie_set_readrq (struct pci_dev *dev,int rq); -void pci_update_resource (struct pci_dev *dev,struct resource *res,int resno); +void pci_update_resource (struct pci_dev *dev,int resno);int __must_check pci_assign_resource (struct pci_dev *dev,int i);int pci_select_bars (struct pci_dev *dev,unsigned long flags);
Signature · 1 line
1.5.6.4
PATCH v5 1/15 PCI: remove unnecessary arg of pci_update_resource()
In reply to Yu ZhaoThis cleanup removes unnecessary argument 'struct resource *res' in pci_update_resource(), so it takes same arguments as other companion functions (pci_assign_resource(), etc.).
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 4 ++-- drivers/pci/setup-res.c | 7 ++++--- include/linux/pci.h | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-)
drivers/pci/pci.c +2 -2
@@ -376,8 +376,8 @@ pci_restore_bars(struct pci_dev *dev)return ; } -for (i =0 ; i < numres; i ++) -pci_update_resource (dev, &dev->resource[i], i); +for (i =0 ; i < numres; i++) +pci_update_resource (dev, i); }static struct pci_platform_pm_ops *pci_platform_pm;
drivers/pci/setup-res.c +4 -3
@@ -26,11 +26,12 @@#include "pci.h" -void pci_update_resource (struct pci_dev *dev,struct resource *res,int resno) +void pci_update_resource (struct pci_dev *dev,int resno) {struct pci_bus_region region;u32 new, check, mask;int reg; +struct resource *res = dev->resource + resno;/* * Ignore resources for unimplemented BARs and unused resource slots
@@ -162,7 +163,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno) }else { res->flags &= ~IORESOURCE_STARTALIGN ;if (resno <PCI_BRIDGE_RESOURCES ) -pci_update_resource (dev, res, resno); +pci_update_resource (dev, resno); }return ret;
@@ -197,7 +198,7 @@ int pci_assign_resource_fixed(struct pci_dev *dev, int resno)dev_err (&dev->dev,"BAR %d: can't allocate %s resource %pR\n" , resno, res->flags &IORESOURCE_IO ?"I/O" :"mem" , res); }else if (resno <PCI_BRIDGE_RESOURCES ) { -pci_update_resource (dev, res, resno); +pci_update_resource (dev, resno); }return ret;
include/linux/pci.h +1 -1
@@ -626,7 +626,7 @@ int pcix_get_mmrbc(struct pci_dev *dev);int pcix_set_mmrbc (struct pci_dev *dev,int mmrbc);int pcie_get_readrq (struct pci_dev *dev);int pcie_set_readrq (struct pci_dev *dev,int rq); -void pci_update_resource (struct pci_dev *dev,struct resource *res,int resno); +void pci_update_resource (struct pci_dev *dev,int resno);int __must_check pci_assign_resource (struct pci_dev *dev,int i);int pci_select_bars (struct pci_dev *dev,unsigned long flags);
Signature · 1 line
1.5.6.4
PATCH v5 2/15 PCI: define PCI resource names in an 'enum'
In reply to Yu ZhaoThis patch moves all definitions of PCI resource names to an 'enum', and also replaces some hard-coded resource variables with symbol names. This change eases the introduction of device specific resources.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci-sysfs.c | 4 +++- drivers/pci/pci.c | 19 ++----------------- drivers/pci/probe.c | 2 +- drivers/pci/proc.c | 7 ++++--- include/linux/pci.h | 37 ++++++++++++++++++++++++------------- 5 files changed, 34 insertions(+), 35 deletions(-)
drivers/pci/pci-sysfs.c +3 -1
@@ -101,11 +101,13 @@ resource_show(struct device * dev, struct device_attribute *attr, char * buf)struct pci_dev * pci_dev =to_pci_dev (dev);char * str = buf;int i; -int max =7 ; +int max;resource_size_t start, end;if (pci_dev->subordinate) max =DEVICE_COUNT_RESOURCE ; +else + max =PCI_BRIDGE_RESOURCES ;for (i =0 ; i < max; i++) {struct resource *res = &pci_dev->resource[i];
drivers/pci/pci.c +2 -17
@@ -359,24 +359,9 @@ pci_find_parent_resource(const struct pci_dev *dev, struct resource *res)static void pci_restore_bars (struct pci_dev *dev) { -int i, numres; - -switch (dev->hdr_type) { -case PCI_HEADER_TYPE_NORMAL : - numres =6 ; -break ; -case PCI_HEADER_TYPE_BRIDGE : - numres =2 ; -break ; -case PCI_HEADER_TYPE_CARDBUS : - numres =1 ; -break ; -default : -/* Should never get here, but just in case... */ -return ; - } +int i; -for (i =0 ; i < numres; i++) +for (i =0 ; i <PCI_BRIDGE_RESOURCES ; i++)pci_update_resource (dev, i); }
drivers/pci/probe.c +1 -1
@@ -426,7 +426,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, child->subordinate =0xff ;/* Set up default resource pointers and names.. */ -for (i =0 ; i <4 ; i++) { +for (i =0 ; i <PCI_BRIDGE_RES_NUM ; i++) { child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES +i]; child->resource[i]->name = child->name; }
drivers/pci/proc.c +4 -3
@@ -352,15 +352,16 @@ static int show_device(struct seq_file *m, void *v) dev->vendor, dev->device, dev->irq); -/* Here should be 7 and not PCI_NUM_RESOURCES as we need to preserve compatibility */ -for (i=0 ; i<7 ; i++) { + +/* only print standard and ROM resources to preserve compatibility */ +for (i =0 ; i <=PCI_ROM_RESOURCE ; i++) {resource_size_t start, end;pci_resource_to_user (dev, i, &dev->resource[i], &start, &end);seq_printf (m,"\t%16llx" , (unsigned long long )(start | (dev->resource[i].flags &PCI_REGION_FLAG_MASK ))); } -for (i=0 ; i<7 ; i++) { +for (i =0 ; i <=PCI_ROM_RESOURCE ; i++) {resource_size_t start, end;pci_resource_to_user (dev, i, &dev->resource[i], &start, &end); seq_printf(m, "\t%16llx",
include/linux/pci.h +24 -13
@@ -76,7 +76,30 @@ enum pci_mmap_state {#define PCI_DMA_FROMDEVICE 2#define PCI_DMA_NONE 3 -#define DEVICE_COUNT_RESOURCE 12 +/* +* For PCI devices, the region numbers are assigned this way: +*/ +enum { +/* #0-5: standard PCI regions */ +PCI_STD_RESOURCES , +PCI_STD_RESOURCES_END =5 , + +/* #6: expansion ROM */ +PCI_ROM_RESOURCE , + +/* address space assigned to buses behind the bridge */ +#ifndef PCI_BRIDGE_RES_NUM +#define PCI_BRIDGE_RES_NUM 4 +#endif +PCI_BRIDGE_RESOURCES , +PCI_BRIDGE_RES_END =PCI_BRIDGE_RESOURCES +PCI_BRIDGE_RES_NUM -1 , + +/* total resources associated with a PCI device */ +PCI_NUM_RESOURCES , + +/* preserve this for compatibility */ +DEVICE_COUNT_RESOURCE +};typedef int __bitwise pci_power_t ;
@@ -262,18 +285,6 @@ static inline void pci_add_saved_cap(struct pci_dev *pci_dev,hlist_add_head (&new_cap->next, &pci_dev->saved_cap_space); } -/* -* For PCI devices, the region numbers are assigned this way: -* -* 0-5 standard PCI regions -* 6 expansion ROM -* 7-10 bridges: address space assigned to buses behind the bridge -*/ - -#define PCI_ROM_RESOURCE 6 -#define PCI_BRIDGE_RESOURCES 7 -#define PCI_NUM_RESOURCES 11 -#ifndef PCI_BUS_NUM_RESOURCES #define PCI_BUS_NUM_RESOURCES 16#endif
Signature · 1 line
1.5.6.4
PATCH v5 2/15 PCI: define PCI resource names in an 'enum'
In reply to Yu ZhaoThis patch moves all definitions of PCI resource names to an 'enum', and also replaces some hard-coded resource variables with symbol names. This change eases the introduction of device specific resources.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci-sysfs.c | 4 +++- drivers/pci/pci.c | 19 ++----------------- drivers/pci/probe.c | 2 +- drivers/pci/proc.c | 7 ++++--- include/linux/pci.h | 37 ++++++++++++++++++++++++------------- 5 files changed, 34 insertions(+), 35 deletions(-)
drivers/pci/pci-sysfs.c +3 -1
@@ -101,11 +101,13 @@ resource_show(struct device * dev, struct device_attribute *attr, char * buf)struct pci_dev * pci_dev =to_pci_dev (dev);char * str = buf;int i; -int max =7 ; +int max;resource_size_t start, end;if (pci_dev->subordinate) max =DEVICE_COUNT_RESOURCE ; +else + max =PCI_BRIDGE_RESOURCES ;for (i =0 ; i < max; i++) {struct resource *res = &pci_dev->resource[i];
drivers/pci/pci.c +2 -17
@@ -359,24 +359,9 @@ pci_find_parent_resource(const struct pci_dev *dev, struct resource *res)static void pci_restore_bars (struct pci_dev *dev) { -int i, numres; - -switch (dev->hdr_type) { -case PCI_HEADER_TYPE_NORMAL : - numres =6 ; -break ; -case PCI_HEADER_TYPE_BRIDGE : - numres =2 ; -break ; -case PCI_HEADER_TYPE_CARDBUS : - numres =1 ; -break ; -default : -/* Should never get here, but just in case... */ -return ; - } +int i; -for (i =0 ; i < numres; i++) +for (i =0 ; i <PCI_BRIDGE_RESOURCES ; i++)pci_update_resource (dev, i); }
drivers/pci/probe.c +1 -1
@@ -426,7 +426,7 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, child->subordinate =0xff ;/* Set up default resource pointers and names.. */ -for (i =0 ; i <4 ; i++) { +for (i =0 ; i <PCI_BRIDGE_RES_NUM ; i++) { child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES +i]; child->resource[i]->name = child->name; }
drivers/pci/proc.c +4 -3
@@ -352,15 +352,16 @@ static int show_device(struct seq_file *m, void *v) dev->vendor, dev->device, dev->irq); -/* Here should be 7 and not PCI_NUM_RESOURCES as we need to preserve compatibility */ -for (i=0 ; i<7 ; i++) { + +/* only print standard and ROM resources to preserve compatibility */ +for (i =0 ; i <=PCI_ROM_RESOURCE ; i++) {resource_size_t start, end;pci_resource_to_user (dev, i, &dev->resource[i], &start, &end);seq_printf (m,"\t%16llx" , (unsigned long long )(start | (dev->resource[i].flags &PCI_REGION_FLAG_MASK ))); } -for (i=0 ; i<7 ; i++) { +for (i =0 ; i <=PCI_ROM_RESOURCE ; i++) {resource_size_t start, end;pci_resource_to_user (dev, i, &dev->resource[i], &start, &end); seq_printf(m, "\t%16llx",
include/linux/pci.h +24 -13
@@ -76,7 +76,30 @@ enum pci_mmap_state {#define PCI_DMA_FROMDEVICE 2#define PCI_DMA_NONE 3 -#define DEVICE_COUNT_RESOURCE 12 +/* +* For PCI devices, the region numbers are assigned this way: +*/ +enum { +/* #0-5: standard PCI regions */ +PCI_STD_RESOURCES , +PCI_STD_RESOURCES_END =5 , + +/* #6: expansion ROM */ +PCI_ROM_RESOURCE , + +/* address space assigned to buses behind the bridge */ +#ifndef PCI_BRIDGE_RES_NUM +#define PCI_BRIDGE_RES_NUM 4 +#endif +PCI_BRIDGE_RESOURCES , +PCI_BRIDGE_RES_END =PCI_BRIDGE_RESOURCES +PCI_BRIDGE_RES_NUM -1 , + +/* total resources associated with a PCI device */ +PCI_NUM_RESOURCES , + +/* preserve this for compatibility */ +DEVICE_COUNT_RESOURCE +};typedef int __bitwise pci_power_t ;
@@ -262,18 +285,6 @@ static inline void pci_add_saved_cap(struct pci_dev *pci_dev,hlist_add_head (&new_cap->next, &pci_dev->saved_cap_space); } -/* -* For PCI devices, the region numbers are assigned this way: -* -* 0-5 standard PCI regions -* 6 expansion ROM -* 7-10 bridges: address space assigned to buses behind the bridge -*/ - -#define PCI_ROM_RESOURCE 6 -#define PCI_BRIDGE_RESOURCES 7 -#define PCI_NUM_RESOURCES 11 -#ifndef PCI_BUS_NUM_RESOURCES #define PCI_BUS_NUM_RESOURCES 16#endif
Signature · 1 line
1.5.6.4
PATCH v5 3/15 PCI: export __pci_read_base
In reply to Yu ZhaoExport __pci_read_base() so it can be used by whole PCI subsystem.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.h | 9 +++++++++ drivers/pci/probe.c | 20 +++++++++----------- 2 files changed, 18 insertions(+), 11 deletions(-)
drivers/pci/pci.h +9 -0
@@ -157,6 +157,15 @@ struct pci_slot_attribute { };#define to_pci_slot_attr (s) container_of(s, struct pci_slot_attribute, attr) +enum pci_bar_type { + pci_bar_unknown,/* Standard PCI BAR probe */ + pci_bar_io,/* An io port BAR */ + pci_bar_mem32,/* A 32-bit memory BAR */ + pci_bar_mem64,/* A 64-bit memory BAR */ +}; + +extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type, +struct resource *res,unsigned int reg);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/probe.c +9 -11
@@ -135,13 +135,6 @@ static u64 pci_size(u64 base, u64 maxbase, u64 mask)return size; } -enum pci_bar_type { - pci_bar_unknown,/* Standard PCI BAR probe */ - pci_bar_io,/* An io port BAR */ - pci_bar_mem32,/* A 32-bit memory BAR */ - pci_bar_mem64,/* A 64-bit memory BAR */ -}; -static inline enum pci_bar_type decode_bar (struct resource *res,u32 bar) {if ((bar &PCI_BASE_ADDRESS_SPACE ) ==PCI_BASE_ADDRESS_SPACE_IO ) {
@@ -156,11 +149,16 @@ static inline enum pci_bar_type decode_bar(struct resource *res, u32 bar)return pci_bar_mem32; } -/* -* If the type is not unknown, we assume that the lowest bit is 'enable'. -* Returns 1 if the BAR was 64-bit and 0 if it was 32-bit. +/** +* pci_read_base - read a PCI BAR +* @dev: the PCI device +* @type: type of the BAR +* @res: resource buffer to be filled in +* @pos: BAR position in the config space +* +* Returns 1 if the BAR is 64-bit, or 0 if 32-bit. */ -static int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type, +int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int pos) {u32 l, sz, mask;
Signature · 1 line
1.5.6.4
PATCH v5 3/15 PCI: export __pci_read_base
In reply to Yu ZhaoExport __pci_read_base() so it can be used by whole PCI subsystem.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.h | 9 +++++++++ drivers/pci/probe.c | 20 +++++++++----------- 2 files changed, 18 insertions(+), 11 deletions(-)
drivers/pci/pci.h +9 -0
@@ -157,6 +157,15 @@ struct pci_slot_attribute { };#define to_pci_slot_attr (s) container_of(s, struct pci_slot_attribute, attr) +enum pci_bar_type { + pci_bar_unknown,/* Standard PCI BAR probe */ + pci_bar_io,/* An io port BAR */ + pci_bar_mem32,/* A 32-bit memory BAR */ + pci_bar_mem64,/* A 64-bit memory BAR */ +}; + +extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type, +struct resource *res,unsigned int reg);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/probe.c +9 -11
@@ -135,13 +135,6 @@ static u64 pci_size(u64 base, u64 maxbase, u64 mask)return size; } -enum pci_bar_type { - pci_bar_unknown,/* Standard PCI BAR probe */ - pci_bar_io,/* An io port BAR */ - pci_bar_mem32,/* A 32-bit memory BAR */ - pci_bar_mem64,/* A 64-bit memory BAR */ -}; -static inline enum pci_bar_type decode_bar (struct resource *res,u32 bar) {if ((bar &PCI_BASE_ADDRESS_SPACE ) ==PCI_BASE_ADDRESS_SPACE_IO ) {
@@ -156,11 +149,16 @@ static inline enum pci_bar_type decode_bar(struct resource *res, u32 bar)return pci_bar_mem32; } -/* -* If the type is not unknown, we assume that the lowest bit is 'enable'. -* Returns 1 if the BAR was 64-bit and 0 if it was 32-bit. +/** +* pci_read_base - read a PCI BAR +* @dev: the PCI device +* @type: type of the BAR +* @res: resource buffer to be filled in +* @pos: BAR position in the config space +* +* Returns 1 if the BAR is 64-bit, or 0 if 32-bit. */ -static int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type, +int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int pos) {u32 l, sz, mask;
Signature · 1 line
1.5.6.4
PATCH v5 4/15 PCI: make pci_alloc_child_bus() be able to handle NULL bridge
In reply to Yu ZhaoMake pci_alloc_child_bus() be able to handle buses without bridge devices. Some devices such as SR-IOV devices use more than one bus number while there is no explicit bridge devices since they have internal routing mechanism.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/probe.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-)
drivers/pci/probe.c +5 -2
@@ -401,12 +401,10 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,if (!child)return NULL ; - child->self = bridge; child->parent = parent; child->ops = parent->ops; child->sysdata = parent->sysdata; child->bus_flags = parent->bus_flags; - child->bridge =get_device (&bridge->dev);/* initialize some portions of the bus device, but don't register it * now as the parent is not properly set up yet. This device will get
@@ -423,6 +421,11 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, child->primary = parent->secondary; child->subordinate =0xff ; +if (!bridge) +return child; + + child->self = bridge; + child->bridge =get_device (&bridge->dev);/* Set up default resource pointers and names.. */ for (i =0 ; i <PCI_BRIDGE_RES_NUM ; i++) { child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES +i];
Signature · 1 line
1.5.6.4
PATCH v5 4/15 PCI: make pci_alloc_child_bus() be able to handle NULL bridge
In reply to Yu ZhaoMake pci_alloc_child_bus() be able to handle buses without bridge devices. Some devices such as SR-IOV devices use more than one bus number while there is no explicit bridge devices since they have internal routing mechanism.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/probe.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-)
drivers/pci/probe.c +5 -2
@@ -401,12 +401,10 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,if (!child)return NULL ; - child->self = bridge; child->parent = parent; child->ops = parent->ops; child->sysdata = parent->sysdata; child->bus_flags = parent->bus_flags; - child->bridge =get_device (&bridge->dev);/* initialize some portions of the bus device, but don't register it * now as the parent is not properly set up yet. This device will get
@@ -423,6 +421,11 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent, child->primary = parent->secondary; child->subordinate =0xff ; +if (!bridge) +return child; + + child->self = bridge; + child->bridge =get_device (&bridge->dev);/* Set up default resource pointers and names.. */ for (i =0 ; i <PCI_BRIDGE_RES_NUM ; i++) { child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES +i];
Signature · 1 line
1.5.6.4
PATCH v5 5/15 PCI: add a wrapper for resource_alignment()
In reply to Yu ZhaoAdd a wrap of resource_alignment() so it can handle device specific resource alignment.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 20 ++++++++++++++++++++ drivers/pci/pci.h | 1 + drivers/pci/setup-bus.c | 4 ++-- drivers/pci/setup-res.c | 7 ++++--- 4 files changed, 27 insertions(+), 5 deletions(-)
drivers/pci/pci.c +20 -0
@@ -1904,6 +1904,26 @@ int pci_select_bars(struct pci_dev *dev, unsigned long flags)return bars; } +/** +* pci_resource_alignment - get a PCI BAR resource alignment +* @dev: the PCI device +* @resno: the resource number +* +* Returns alignment size on success, or 0 on error. +*/ +int pci_resource_alignment (struct pci_dev *dev,int resno) +{ +resource_size_t align; +struct resource *res = dev->resource + resno; + + align =resource_alignment (res); +if (align) +return align; + +dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno); +return 0 ; +} +static void __devinit pci_no_domains (void ) {#ifdef CONFIG_PCI_DOMAINS
drivers/pci/pci.h +1 -0
@@ -166,6 +166,7 @@ enum pci_bar_type {extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int reg); +extern int pci_resource_alignment (struct pci_dev *dev,int resno);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/setup-bus.c +2 -2
@@ -25,6 +25,7 @@#include <linux/ioport.h> #include <linux/cache.h> #include <linux/slab.h> +#include "pci.h" static void pbus_assign_resources_sorted (struct pci_bus *bus)
@@ -351,8 +352,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, unsigned longif (r->parent || (r->flags & mask) != type)continue ; r_size =resource_size (r); -/* For bridges size != alignment */ - align =resource_alignment (r); + align =pci_resource_alignment (dev, i); order =__ffs (align) -20 ;if (order >11 ) { dev_warn(&dev->dev, "BAR %d bad alignment %llx: "
drivers/pci/setup-res.c +4 -3
@@ -133,7 +133,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno) size =resource_size (res); min = (res->flags &IORESOURCE_IO ) ?PCIBIOS_MIN_IO :PCIBIOS_MIN_MEM ; - align =resource_alignment (res); + align =pci_resource_alignment (dev, resno);if (!align) { dev_err(&dev->dev, "BAR %d: can't allocate resource (bogus " "alignment) %pR flags %#lx\n",
@@ -224,7 +224,7 @@ void pdev_sort_resources(struct pci_dev *dev, struct resource_list *head)if (!(r->flags) || r->parent)continue ; - r_align =resource_alignment (r); + r_align =pci_resource_alignment (dev, i);if (!r_align) { dev_warn(&dev->dev, "BAR %d: bogus alignment " "%pR flags %#lx\n",
@@ -236,7 +236,8 @@ void pdev_sort_resources(struct pci_dev *dev, struct resource_list *head)struct resource_list *ln = list->next;if (ln) - align =resource_alignment (ln->res); + align =pci_resource_alignment (ln->dev, + ln->res - ln->dev->resource);if (r_align > align) { tmp =kmalloc (sizeof (*tmp),GFP_KERNEL );
Signature · 1 line
1.5.6.4
PATCH v5 5/15 PCI: add a wrapper for resource_alignment()
In reply to Yu ZhaoAdd a wrap of resource_alignment() so it can handle device specific resource alignment.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 20 ++++++++++++++++++++ drivers/pci/pci.h | 1 + drivers/pci/setup-bus.c | 4 ++-- drivers/pci/setup-res.c | 7 ++++--- 4 files changed, 27 insertions(+), 5 deletions(-)
drivers/pci/pci.c +20 -0
@@ -1904,6 +1904,26 @@ int pci_select_bars(struct pci_dev *dev, unsigned long flags)return bars; } +/** +* pci_resource_alignment - get a PCI BAR resource alignment +* @dev: the PCI device +* @resno: the resource number +* +* Returns alignment size on success, or 0 on error. +*/ +int pci_resource_alignment (struct pci_dev *dev,int resno) +{ +resource_size_t align; +struct resource *res = dev->resource + resno; + + align =resource_alignment (res); +if (align) +return align; + +dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno); +return 0 ; +} +static void __devinit pci_no_domains (void ) {#ifdef CONFIG_PCI_DOMAINS
drivers/pci/pci.h +1 -0
@@ -166,6 +166,7 @@ enum pci_bar_type {extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int reg); +extern int pci_resource_alignment (struct pci_dev *dev,int resno);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/setup-bus.c +2 -2
@@ -25,6 +25,7 @@#include <linux/ioport.h> #include <linux/cache.h> #include <linux/slab.h> +#include "pci.h" static void pbus_assign_resources_sorted (struct pci_bus *bus)
@@ -351,8 +352,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, unsigned longif (r->parent || (r->flags & mask) != type)continue ; r_size =resource_size (r); -/* For bridges size != alignment */ - align =resource_alignment (r); + align =pci_resource_alignment (dev, i); order =__ffs (align) -20 ;if (order >11 ) { dev_warn(&dev->dev, "BAR %d bad alignment %llx: "
drivers/pci/setup-res.c +4 -3
@@ -133,7 +133,7 @@ int pci_assign_resource(struct pci_dev *dev, int resno) size =resource_size (res); min = (res->flags &IORESOURCE_IO ) ?PCIBIOS_MIN_IO :PCIBIOS_MIN_MEM ; - align =resource_alignment (res); + align =pci_resource_alignment (dev, resno);if (!align) { dev_err(&dev->dev, "BAR %d: can't allocate resource (bogus " "alignment) %pR flags %#lx\n",
@@ -224,7 +224,7 @@ void pdev_sort_resources(struct pci_dev *dev, struct resource_list *head)if (!(r->flags) || r->parent)continue ; - r_align =resource_alignment (r); + r_align =pci_resource_alignment (dev, i);if (!r_align) { dev_warn(&dev->dev, "BAR %d: bogus alignment " "%pR flags %#lx\n",
@@ -236,7 +236,8 @@ void pdev_sort_resources(struct pci_dev *dev, struct resource_list *head)struct resource_list *ln = list->next;if (ln) - align =resource_alignment (ln->res); + align =pci_resource_alignment (ln->dev, + ln->res - ln->dev->resource);if (r_align > align) { tmp =kmalloc (sizeof (*tmp),GFP_KERNEL );
Signature · 1 line
1.5.6.4
PATCH v5 6/15 PCI: add a new function to map BAR offset
In reply to Yu ZhaoAdd a new function to map resource number to base register (offset and type).
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 22 ++++++++++++++++++++++ drivers/pci/pci.h | 2 ++ drivers/pci/setup-res.c | 13 +++++-------- 3 files changed, 29 insertions(+), 8 deletions(-)
drivers/pci/pci.c +22 -0
@@ -1924,6 +1924,28 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)return 0 ; } +/** +* pci_resource_bar - get position of the BAR associated with a resource +* @dev: the PCI device +* @resno: the resource number +* @type: the BAR type to be filled in +* +* Returns BAR position in config space, or 0 if the BAR is invalid. +*/ +int pci_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type) +{ +if (resno <PCI_ROM_RESOURCE ) { + *type = pci_bar_unknown; +return PCI_BASE_ADDRESS_0 +4 * resno; + }else if (resno ==PCI_ROM_RESOURCE ) { + *type = pci_bar_mem32; +return dev->rom_base_reg; + } + +dev_err (&dev->dev,"BAR: invalid resource #%d\n" , resno); +return 0 ; +} +static void __devinit pci_no_domains (void ) {#ifdef CONFIG_PCI_DOMAINS
drivers/pci/pci.h +2 -0
@@ -167,6 +167,8 @@ enum pci_bar_type {extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int reg);extern int pci_resource_alignment (struct pci_dev *dev,int resno); +extern int pci_resource_bar (struct pci_dev *dev,int resno, +enum pci_bar_type *type);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/setup-res.c +5 -8
@@ -31,6 +31,7 @@ void pci_update_resource(struct pci_dev *dev, int resno)struct pci_bus_region region;u32 new, check, mask;int reg; +enum pci_bar_type type;struct resource *res = dev->resource + resno;/*
@@ -62,17 +63,13 @@ void pci_update_resource(struct pci_dev *dev, int resno)else mask = (u32 )PCI_BASE_ADDRESS_MEM_MASK ; -if (resno <6 ) { - reg =PCI_BASE_ADDRESS_0 +4 * resno; - }else if (resno ==PCI_ROM_RESOURCE ) { + reg =pci_resource_bar (dev, resno, &type); +if (!reg) +return ; +if (type != pci_bar_unknown) {if (!(res->flags &IORESOURCE_ROM_ENABLE ))return ; new |=PCI_ROM_ADDRESS_ENABLE ; - reg = dev->rom_base_reg; - }else { -/* Hmm, non-standard resource. */ - -return ;/* kill uninitialised var warning */ }pci_write_config_dword (dev, reg, new);
Signature · 1 line
1.5.6.4
PATCH v5 6/15 PCI: add a new function to map BAR offset
In reply to Yu ZhaoAdd a new function to map resource number to base register (offset and type).
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.c | 22 ++++++++++++++++++++++ drivers/pci/pci.h | 2 ++ drivers/pci/setup-res.c | 13 +++++-------- 3 files changed, 29 insertions(+), 8 deletions(-)
drivers/pci/pci.c +22 -0
@@ -1924,6 +1924,28 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)return 0 ; } +/** +* pci_resource_bar - get position of the BAR associated with a resource +* @dev: the PCI device +* @resno: the resource number +* @type: the BAR type to be filled in +* +* Returns BAR position in config space, or 0 if the BAR is invalid. +*/ +int pci_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type) +{ +if (resno <PCI_ROM_RESOURCE ) { + *type = pci_bar_unknown; +return PCI_BASE_ADDRESS_0 +4 * resno; + }else if (resno ==PCI_ROM_RESOURCE ) { + *type = pci_bar_mem32; +return dev->rom_base_reg; + } + +dev_err (&dev->dev,"BAR: invalid resource #%d\n" , resno); +return 0 ; +} +static void __devinit pci_no_domains (void ) {#ifdef CONFIG_PCI_DOMAINS
drivers/pci/pci.h +2 -0
@@ -167,6 +167,8 @@ enum pci_bar_type {extern int __pci_read_base (struct pci_dev *dev,enum pci_bar_type type,struct resource *res,unsigned int reg);extern int pci_resource_alignment (struct pci_dev *dev,int resno); +extern int pci_resource_bar (struct pci_dev *dev,int resno, +enum pci_bar_type *type);extern void pci_enable_ari (struct pci_dev *dev);/** * pci_ari_enabled - query ARI forwarding status
drivers/pci/setup-res.c +5 -8
@@ -31,6 +31,7 @@ void pci_update_resource(struct pci_dev *dev, int resno)struct pci_bus_region region;u32 new, check, mask;int reg; +enum pci_bar_type type;struct resource *res = dev->resource + resno;/*
@@ -62,17 +63,13 @@ void pci_update_resource(struct pci_dev *dev, int resno)else mask = (u32 )PCI_BASE_ADDRESS_MEM_MASK ; -if (resno <6 ) { - reg =PCI_BASE_ADDRESS_0 +4 * resno; - }else if (resno ==PCI_ROM_RESOURCE ) { + reg =pci_resource_bar (dev, resno, &type); +if (!reg) +return ; +if (type != pci_bar_unknown) {if (!(res->flags &IORESOURCE_ROM_ENABLE ))return ; new |=PCI_ROM_ADDRESS_ENABLE ; - reg = dev->rom_base_reg; - }else { -/* Hmm, non-standard resource. */ - -return ;/* kill uninitialised var warning */ }pci_write_config_dword (dev, reg, new);
Signature · 1 line
1.5.6.4
PATCH v5 7/15 PCI: cleanup pcibios_allocate_resources()
In reply to Yu ZhaoThis cleanup makes pcibios_allocate_resources() easier to be read.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/i386.c | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-)
arch/x86/pci/i386.c +14 -14
@@ -147,7 +147,7 @@ static void __init pcibios_allocate_bus_resources(struct list_head *bus_list)static void __init pcibios_allocate_resources (int pass) {struct pci_dev *dev =NULL ; -int idx, disabled; +int idx, enabled;u16 command;struct resource *r, *pr;
@@ -160,22 +160,22 @@ static void __init pcibios_allocate_resources(int pass)if (!r->start)/* Address not assigned at all */ continue ;if (r->flags &IORESOURCE_IO ) - disabled = !(command &PCI_COMMAND_IO ); + enabled = command &PCI_COMMAND_IO ;else - disabled = !(command &PCI_COMMAND_MEMORY ); -if (pass == disabled) { -dev_dbg (&dev->dev,"resource %#08llx-%#08llx (f=%lx, d=%d, p=%d)\n" , + enabled = command &PCI_COMMAND_MEMORY ; +if (pass == enabled) +continue ; +dev_dbg (&dev->dev,"resource %#08llx-%#08llx (f=%lx, d=%d, p=%d)\n" , (unsigned long long ) r->start, (unsigned long long ) r->end, - r->flags, disabled, pass); - pr =pci_find_parent_resource (dev, r); -if (!pr ||request_resource (pr, r) <0 ) { -dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); -/* We'll assign a new address later */ - r->end -= r->start; - r->start =0 ; - } - } + r->flags, enabled, pass); + pr =pci_find_parent_resource (dev, r); +if (pr && !request_resource (pr, r)) +continue ; +dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); +/* We'll assign a new address later */ + r->end -= r->start; + r->start =0 ; }if (!pass) { r = &dev->resource[PCI_ROM_RESOURCE ];
Signature · 1 line
1.5.6.4
PATCH v5 7/15 PCI: cleanup pcibios_allocate_resources()
In reply to Yu ZhaoThis cleanup makes pcibios_allocate_resources() easier to be read.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/i386.c | 28 ++++++++++++++-------------- 1 files changed, 14 insertions(+), 14 deletions(-)
arch/x86/pci/i386.c +14 -14
@@ -147,7 +147,7 @@ static void __init pcibios_allocate_bus_resources(struct list_head *bus_list)static void __init pcibios_allocate_resources (int pass) {struct pci_dev *dev =NULL ; -int idx, disabled; +int idx, enabled;u16 command;struct resource *r, *pr;
@@ -160,22 +160,22 @@ static void __init pcibios_allocate_resources(int pass)if (!r->start)/* Address not assigned at all */ continue ;if (r->flags &IORESOURCE_IO ) - disabled = !(command &PCI_COMMAND_IO ); + enabled = command &PCI_COMMAND_IO ;else - disabled = !(command &PCI_COMMAND_MEMORY ); -if (pass == disabled) { -dev_dbg (&dev->dev,"resource %#08llx-%#08llx (f=%lx, d=%d, p=%d)\n" , + enabled = command &PCI_COMMAND_MEMORY ; +if (pass == enabled) +continue ; +dev_dbg (&dev->dev,"resource %#08llx-%#08llx (f=%lx, d=%d, p=%d)\n" , (unsigned long long ) r->start, (unsigned long long ) r->end, - r->flags, disabled, pass); - pr =pci_find_parent_resource (dev, r); -if (!pr ||request_resource (pr, r) <0 ) { -dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); -/* We'll assign a new address later */ - r->end -= r->start; - r->start =0 ; - } - } + r->flags, enabled, pass); + pr =pci_find_parent_resource (dev, r); +if (pr && !request_resource (pr, r)) +continue ; +dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); +/* We'll assign a new address later */ + r->end -= r->start; + r->start =0 ; }if (!pass) { r = &dev->resource[PCI_ROM_RESOURCE ];
Signature · 1 line
1.5.6.4
PATCH v5 8/15 PCI: add boot options to reassign resources
In reply to Yu ZhaoThis patch adds boot options so user can reassign device resources of all devices under a bus.
The boot options can be used as:
pci=assign-mmio=0000:01;0000:02,assign-pio=0000:03 '0000' and '01/02/03' are domain and bus number.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/common.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/pci/i386.c | 10 ++++--- arch/x86/pci/pci.h | 3 ++ 3 files changed, 80 insertions(+), 4 deletions(-)
arch/x86/pci/common.c +71 -0
@@ -137,6 +137,70 @@ static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) } } +static char *pci_assign_pio; +static char *pci_assign_mmio; + +static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) +{ +int i; +int type =0 ; +int domain, busnr; + +if (!bus->self) +return 0 ; + +for (i =0 ; i <2 ; i++) { +char *str = i ? pci_assign_pio : pci_assign_mmio; +while (str && *str) { +if (sscanf (str,"%04x:%02x" , &domain, &busnr) !=2 ) { +if (sscanf (str,"%02x" , &busnr) !=1 ) +break ; + domain =0 ; + } + +if (pci_domain_nr (bus) == domain && + bus->number == busnr) { + type |= i ?IORESOURCE_IO :IORESOURCE_MEM ; +break ; + } + + str =strchr (str,';' ); +if (str) + str++; + } + } + +return type; +} + +static void __devinit pcibios_fixup_bus_resources (struct pci_bus *bus) +{ +int i; +int type =pcibios_bus_resource_needs_fixup (bus); + +if (!type) +return ; + +for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +struct resource *res = bus->resource[i]; +if (!res) +continue ; +if (res->flags & type) + res->flags =0 ; + } +} + +int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno) +{ +struct pci_bus *bus; + +for (bus = dev->bus; bus && bus != pci_root_bus; bus = bus->parent) +if (pcibios_bus_resource_needs_fixup (bus)) +return 1 ; + +return 0 ; +} +/* * Called after each bus is probed, but before its children * are examined.
@@ -147,6 +211,7 @@ void __devinit pcibios_fixup_bus(struct pci_bus *b)struct pci_dev *dev;pci_read_bridge_bases (b); +pcibios_fixup_bus_resources (b);list_for_each_entry (dev, &b->devices, bus_list)pcibios_fixup_device_resources (dev); }
@@ -519,6 +584,12 @@ char * __devinit pcibios_setup(char *str) }else if (!strcmp (str,"skip_isa_align" )) { pci_probe |=PCI_CAN_SKIP_ISA_ALIGN ;return NULL ; + }else if (!strncmp (str,"assign-pio=" ,11 )) { + pci_assign_pio = str +11 ; +return NULL ; + }else if (!strncmp (str,"assign-mmio=" ,12 )) { + pci_assign_mmio = str +12 ; +return NULL ; }return str; }
arch/x86/pci/i386.c +6 -4
@@ -169,10 +169,12 @@ static void __init pcibios_allocate_resources(int pass) (unsigned long long ) r->start, (unsigned long long ) r->end, r->flags, enabled, pass); - pr =pci_find_parent_resource (dev, r); -if (pr && !request_resource (pr, r)) -continue ; -dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); +if (!pcibios_resource_needs_fixup (dev, idx)) { + pr =pci_find_parent_resource (dev, r); +if (pr && !request_resource (pr, r)) +continue ; +dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); + }/* We'll assign a new address later */ r->end -= r->start; r->start =0 ;
arch/x86/pci/pci.h +3 -0
@@ -117,6 +117,9 @@ extern int __init pcibios_init(void);extern int __init pci_mmcfg_arch_init (void );extern void __init pci_mmcfg_arch_free (void ); +/* pci-common.c */ +extern int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno); +/* * AMD Fam10h CPUs are buggy, and cannot access MMIO config space * on their northbrige except through the * %eax register. As such, you MUST
Signature · 1 line
1.5.6.4
Re: [PATCH 8/15 v5] PCI: add boot options to reassign resources
Quoted 5 lines · Yu Zhao
arch/x86/pci/common.c +5
-0+static char *pci_assign_pio; +static char *pci_assign_mmio; + +static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) +{
arch/x86/pci/common.c static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) · line 154
Quoted 10 lines
arch/x86/pci/common.c +10
-0+int i; +int type =0 ; +int domain, busnr; + +if (!bus->self) +return 0 ; + +for (i =0 ; i <2 ; i++) { +char *str = i ? pci_assign_pio : pci_assign_mmio; +while (str && *str) {
please put a newline after local variable defitions.
arch/x86/pci/common.c static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) · line 186
Quoted 3 lines
arch/x86/pci/common.c +3
-0+for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +struct resource *res = bus->resource[i]; +if (!res)
ditto.
nice debug feature!
Ingo
Re: [PATCH 8/15 v5] PCI: add boot options to reassign resources
In reply to Yu ZhaoQuoted 5 lines · Yu Zhao
arch/x86/pci/common.c +5
-0+static char *pci_assign_pio; +static char *pci_assign_mmio; + +static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) +{
arch/x86/pci/common.c static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) · line 154
Quoted 10 lines
arch/x86/pci/common.c +10
-0+int i; +int type =0 ; +int domain, busnr; + +if (!bus->self) +return 0 ; + +for (i =0 ; i <2 ; i++) { +char *str = i ? pci_assign_pio : pci_assign_mmio; +while (str && *str) {
please put a newline after local variable defitions.
arch/x86/pci/common.c static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) · line 186
Quoted 3 lines
arch/x86/pci/common.c +3
-0+for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +struct resource *res = bus->resource[i]; +if (!res)
ditto.
nice debug feature!
Ingo
PATCH v5 8/15 PCI: add boot options to reassign resources
In reply to Yu ZhaoThis patch adds boot options so user can reassign device resources of all devices under a bus.
The boot options can be used as:
pci=assign-mmio=0000:01;0000:02,assign-pio=0000:03 '0000' and '01/02/03' are domain and bus number.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/common.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/pci/i386.c | 10 ++++--- arch/x86/pci/pci.h | 3 ++ 3 files changed, 80 insertions(+), 4 deletions(-)
arch/x86/pci/common.c +71 -0
@@ -137,6 +137,70 @@ static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev) } } +static char *pci_assign_pio; +static char *pci_assign_mmio; + +static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) +{ +int i; +int type =0 ; +int domain, busnr; + +if (!bus->self) +return 0 ; + +for (i =0 ; i <2 ; i++) { +char *str = i ? pci_assign_pio : pci_assign_mmio; +while (str && *str) { +if (sscanf (str,"%04x:%02x" , &domain, &busnr) !=2 ) { +if (sscanf (str,"%02x" , &busnr) !=1 ) +break ; + domain =0 ; + } + +if (pci_domain_nr (bus) == domain && + bus->number == busnr) { + type |= i ?IORESOURCE_IO :IORESOURCE_MEM ; +break ; + } + + str =strchr (str,';' ); +if (str) + str++; + } + } + +return type; +} + +static void __devinit pcibios_fixup_bus_resources (struct pci_bus *bus) +{ +int i; +int type =pcibios_bus_resource_needs_fixup (bus); + +if (!type) +return ; + +for (i =0 ; i <PCI_BUS_NUM_RESOURCES ; i++) { +struct resource *res = bus->resource[i]; +if (!res) +continue ; +if (res->flags & type) + res->flags =0 ; + } +} + +int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno) +{ +struct pci_bus *bus; + +for (bus = dev->bus; bus && bus != pci_root_bus; bus = bus->parent) +if (pcibios_bus_resource_needs_fixup (bus)) +return 1 ; + +return 0 ; +} +/* * Called after each bus is probed, but before its children * are examined.
@@ -147,6 +211,7 @@ void __devinit pcibios_fixup_bus(struct pci_bus *b)struct pci_dev *dev;pci_read_bridge_bases (b); +pcibios_fixup_bus_resources (b);list_for_each_entry (dev, &b->devices, bus_list)pcibios_fixup_device_resources (dev); }
@@ -519,6 +584,12 @@ char * __devinit pcibios_setup(char *str) }else if (!strcmp (str,"skip_isa_align" )) { pci_probe |=PCI_CAN_SKIP_ISA_ALIGN ;return NULL ; + }else if (!strncmp (str,"assign-pio=" ,11 )) { + pci_assign_pio = str +11 ; +return NULL ; + }else if (!strncmp (str,"assign-mmio=" ,12 )) { + pci_assign_mmio = str +12 ; +return NULL ; }return str; }
arch/x86/pci/i386.c +6 -4
@@ -169,10 +169,12 @@ static void __init pcibios_allocate_resources(int pass) (unsigned long long ) r->start, (unsigned long long ) r->end, r->flags, enabled, pass); - pr =pci_find_parent_resource (dev, r); -if (pr && !request_resource (pr, r)) -continue ; -dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); +if (!pcibios_resource_needs_fixup (dev, idx)) { + pr =pci_find_parent_resource (dev, r); +if (pr && !request_resource (pr, r)) +continue ; +dev_err (&dev->dev,"BAR %d: can't allocate resource\n" , idx); + }/* We'll assign a new address later */ r->end -= r->start; r->start =0 ;
arch/x86/pci/pci.h +3 -0
@@ -117,6 +117,9 @@ extern int __init pcibios_init(void);extern int __init pci_mmcfg_arch_init (void );extern void __init pci_mmcfg_arch_free (void ); +/* pci-common.c */ +extern int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno); +/* * AMD Fam10h CPUs are buggy, and cannot access MMIO config space * on their northbrige except through the * %eax register. As such, you MUST
Signature · 1 line
1.5.6.4
PATCH v5 9/15 PCI: add boot option to align MMIO resource
In reply to Yu ZhaoThis patch adds boot option to align MMIO resource for a device. The alignment is a bigger value between the PAGE_SIZE and the resource size.
The boot option can be used as:
pci=align-mmio=0000:01:02.3;04:05.6
'0000:01:02.3' and '04:05.6' are domain, bus, device and function numbers.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/common.c | 37 +++++++++++++++++++++++++++++++++++++ drivers/pci/pci.c | 20 ++++++++++++++++++-- include/linux/pci.h | 1 + 3 files changed, 56 insertions(+), 2 deletions(-)
arch/x86/pci/common.c +37 -0
@@ -139,6 +139,7 @@ static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev)static char *pci_assign_pio;static char *pci_assign_mmio; +static char *pci_align_mmio;static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) {
@@ -190,6 +191,36 @@ static void __devinit pcibios_fixup_bus_resources(struct pci_bus *bus) } } +int pcibios_resource_alignment (struct pci_dev *dev,int resno) +{ +int domain, busnr, slot, func; +char *str = pci_align_mmio; + +if (dev->resource[resno].flags &IORESOURCE_IO ) +return 0 ; + +while (str && *str) { +if (sscanf (str,"%04x:%02x:%02x.%d" , + &domain, &busnr, &slot, &func) !=4 ) { +if (sscanf (str,"%02x:%02x.%d" , + &busnr, &slot, &func) !=3 ) +break ; + domain =0 ; + } + +if (pci_domain_nr (dev->bus) == domain && + dev->bus->number == busnr && + dev->devfn ==PCI_DEVFN (slot, func)) +return PAGE_SIZE ; + + str =strchr (str,';' ); +if (str) + str++; + } + +return 0 ; +} +int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno) {struct pci_bus *bus;
@@ -198,6 +229,9 @@ int pcibios_resource_needs_fixup(struct pci_dev *dev, int resno)if (pcibios_bus_resource_needs_fixup (bus))return 1 ; +if (pcibios_resource_alignment (dev, resno)) +return 1 ; +return 0 ; }
@@ -590,6 +624,9 @@ char * __devinit pcibios_setup(char *str) }else if (!strncmp (str,"assign-mmio=" ,12 )) { pci_assign_mmio = str +12 ;return NULL ; + }else if (!strncmp (str,"align-mmio=" ,11 )) { + pci_align_mmio = str +11 ; +return NULL ; }return str; }
drivers/pci/pci.c +18 -2
@@ -1015,6 +1015,20 @@ int __attribute__ ((weak)) pcibios_set_pcie_reset_state(struct pci_dev *dev, }/** +* pcibios_resource_alignment - get resource alignment requirement +* @dev: the PCI device +* @resno: resource number +* +* Queries the resource alignment from PCI low level code. Returns positive +* if there is alignment requirement of the resource, or 0 otherwise. +*/ +int __attribute__ ((weak))pcibios_resource_alignment (struct pci_dev *dev, +int resno) +{ +return 0 ; +} + +/** * pci_set_pcie_reset_state - set reset state for device dev * @dev: the PCI-E device reset * @state: Reset state to enter into
@@ -1913,12 +1927,14 @@ int pci_select_bars(struct pci_dev *dev, unsigned long flags)*/ int pci_resource_alignment (struct pci_dev *dev,int resno) { -resource_size_t align; +resource_size_t align, bios_align;struct resource *res = dev->resource + resno; + bios_align =pcibios_resource_alignment (dev, resno); + align =resource_alignment (res);if (align) -return align; +return align > bios_align ? align : bios_align;dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno);return 0 ;
include/linux/pci.h +1 -0
@@ -1121,6 +1121,7 @@ int pcibios_add_platform_entries(struct pci_dev *dev);void pcibios_disable_device (struct pci_dev *dev);int pcibios_set_pcie_reset_state (struct pci_dev *dev,enum pcie_reset_state state); +int pcibios_resource_alignment (struct pci_dev *dev,int resno);#ifdef CONFIG_PCI_MMCONFIG extern void __init pci_mmcfg_early_init (void );
Signature · 1 line
1.5.6.4
PATCH v5 9/15 PCI: add boot option to align MMIO resource
In reply to Yu ZhaoThis patch adds boot option to align MMIO resource for a device. The alignment is a bigger value between the PAGE_SIZE and the resource size.
The boot option can be used as:
pci=align-mmio=0000:01:02.3;04:05.6
'0000:01:02.3' and '04:05.6' are domain, bus, device and function numbers.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- arch/x86/pci/common.c | 37 +++++++++++++++++++++++++++++++++++++ drivers/pci/pci.c | 20 ++++++++++++++++++-- include/linux/pci.h | 1 + 3 files changed, 56 insertions(+), 2 deletions(-)
arch/x86/pci/common.c +37 -0
@@ -139,6 +139,7 @@ static void __devinit pcibios_fixup_device_resources(struct pci_dev *dev)static char *pci_assign_pio;static char *pci_assign_mmio; +static char *pci_align_mmio;static int pcibios_bus_resource_needs_fixup (struct pci_bus *bus) {
@@ -190,6 +191,36 @@ static void __devinit pcibios_fixup_bus_resources(struct pci_bus *bus) } } +int pcibios_resource_alignment (struct pci_dev *dev,int resno) +{ +int domain, busnr, slot, func; +char *str = pci_align_mmio; + +if (dev->resource[resno].flags &IORESOURCE_IO ) +return 0 ; + +while (str && *str) { +if (sscanf (str,"%04x:%02x:%02x.%d" , + &domain, &busnr, &slot, &func) !=4 ) { +if (sscanf (str,"%02x:%02x.%d" , + &busnr, &slot, &func) !=3 ) +break ; + domain =0 ; + } + +if (pci_domain_nr (dev->bus) == domain && + dev->bus->number == busnr && + dev->devfn ==PCI_DEVFN (slot, func)) +return PAGE_SIZE ; + + str =strchr (str,';' ); +if (str) + str++; + } + +return 0 ; +} +int pcibios_resource_needs_fixup (struct pci_dev *dev,int resno) {struct pci_bus *bus;
@@ -198,6 +229,9 @@ int pcibios_resource_needs_fixup(struct pci_dev *dev, int resno)if (pcibios_bus_resource_needs_fixup (bus))return 1 ; +if (pcibios_resource_alignment (dev, resno)) +return 1 ; +return 0 ; }
@@ -590,6 +624,9 @@ char * __devinit pcibios_setup(char *str) }else if (!strncmp (str,"assign-mmio=" ,12 )) { pci_assign_mmio = str +12 ;return NULL ; + }else if (!strncmp (str,"align-mmio=" ,11 )) { + pci_align_mmio = str +11 ; +return NULL ; }return str; }
drivers/pci/pci.c +18 -2
@@ -1015,6 +1015,20 @@ int __attribute__ ((weak)) pcibios_set_pcie_reset_state(struct pci_dev *dev, }/** +* pcibios_resource_alignment - get resource alignment requirement +* @dev: the PCI device +* @resno: resource number +* +* Queries the resource alignment from PCI low level code. Returns positive +* if there is alignment requirement of the resource, or 0 otherwise. +*/ +int __attribute__ ((weak))pcibios_resource_alignment (struct pci_dev *dev, +int resno) +{ +return 0 ; +} + +/** * pci_set_pcie_reset_state - set reset state for device dev * @dev: the PCI-E device reset * @state: Reset state to enter into
@@ -1913,12 +1927,14 @@ int pci_select_bars(struct pci_dev *dev, unsigned long flags)*/ int pci_resource_alignment (struct pci_dev *dev,int resno) { -resource_size_t align; +resource_size_t align, bios_align;struct resource *res = dev->resource + resno; + bios_align =pcibios_resource_alignment (dev, resno); + align =resource_alignment (res);if (align) -return align; +return align > bios_align ? align : bios_align;dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno);return 0 ;
include/linux/pci.h +1 -0
@@ -1121,6 +1121,7 @@ int pcibios_add_platform_entries(struct pci_dev *dev);void pcibios_disable_device (struct pci_dev *dev);int pcibios_set_pcie_reset_state (struct pci_dev *dev,enum pcie_reset_state state); +int pcibios_resource_alignment (struct pci_dev *dev,int resno);#ifdef CONFIG_PCI_MMCONFIG extern void __init pci_mmcfg_early_init (void );
Signature · 1 line
1.5.6.4
PATCH v5 10/15 PCI: cleanup pci_bus_add_devices()
In reply to Yu ZhaoThis cleanup makes pci_bus_add_devices() easier to be read. And it checks if a bus has been added before removing it.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/bus.c | 56 +++++++++++++++++++++++++------------------------ drivers/pci/remove.c | 2 + 2 files changed, 31 insertions(+), 27 deletions(-)
drivers/pci/bus.c +29 -27
@@ -71,7 +71,7 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, }/** -* add a single device +* pci_bus_add_device - add a single device * @dev: device to add * * This adds a single pci device to the global
@@ -105,7 +105,7 @@ int pci_bus_add_device(struct pci_dev *dev)void pci_bus_add_devices (struct pci_bus *bus) {struct pci_dev *dev; -struct pci_bus *child_bus; +struct pci_bus *child;int retval;list_for_each_entry (dev, &bus->devices, bus_list) {
@@ -120,39 +120,41 @@ void pci_bus_add_devices(struct pci_bus *bus)list_for_each_entry (dev, &bus->devices, bus_list) {BUG_ON (!dev->is_added); + child = dev->subordinate;/* * If there is an unattached subordinate bus, attach * it and then scan for unattached PCI devices. */ -if (dev->subordinate) { -if (list_empty (&dev->subordinate->node)) { -down_write (&pci_bus_sem); -list_add_tail (&dev->subordinate->node, - &dev->bus->children); -up_write (&pci_bus_sem); - } -pci_bus_add_devices (dev->subordinate); - -/* register the bus with sysfs as the parent is now -* properly registered. */ - child_bus = dev->subordinate; -if (child_bus->is_added) -continue ; - child_bus->dev.parent = child_bus->bridge; - retval =device_register (&child_bus->dev); -if (retval) -dev_err (&dev->dev,"Error registering pci_bus," -" continuing...\n" ); -else { - child_bus->is_added =1 ; - retval =device_create_file (&child_bus->dev, - &dev_attr_cpuaffinity); - } +if (!child) +continue ; +if (list_empty (&child->node)) { +down_write (&pci_bus_sem); +list_add_tail (&child->node, + &dev->bus->children); +up_write (&pci_bus_sem); + } +pci_bus_add_devices (child); + +/* +* register the bus with sysfs as the parent is now +* properly registered. +*/ +if (child->is_added) +continue ; + child->dev.parent = child->bridge; + retval =device_register (&child->dev); +if (retval) +dev_err (&dev->dev,"Error registering pci_bus," +" continuing...\n" ); +else { + child->is_added =1 ; + retval =device_create_file (&child->dev, + &dev_attr_cpuaffinity);if (retval)dev_err (&dev->dev,"Error creating cpuaffinity" " file, continuing...\n" ); - retval =device_create_file (&child_bus->dev, + retval =device_create_file (&child->dev, &dev_attr_cpulistaffinity);if (retval) dev_err(&dev->dev,
drivers/pci/remove.c +2 -0
@@ -72,6 +72,8 @@ void pci_remove_bus(struct pci_bus *pci_bus)list_del (&pci_bus->node);up_write (&pci_bus_sem);pci_remove_legacy_files (pci_bus); +if (!pci_bus->is_added) +return ;device_remove_file (&pci_bus->dev, &dev_attr_cpuaffinity);device_remove_file (&pci_bus->dev, &dev_attr_cpulistaffinity);device_unregister (&pci_bus->dev);
Signature · 1 line
1.5.6.4
PATCH v5 10/15 PCI: cleanup pci_bus_add_devices()
In reply to Yu ZhaoThis cleanup makes pci_bus_add_devices() easier to be read. And it checks if a bus has been added before removing it.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/bus.c | 56 +++++++++++++++++++++++++------------------------ drivers/pci/remove.c | 2 + 2 files changed, 31 insertions(+), 27 deletions(-)
drivers/pci/bus.c +29 -27
@@ -71,7 +71,7 @@ pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res, }/** -* add a single device +* pci_bus_add_device - add a single device * @dev: device to add * * This adds a single pci device to the global
@@ -105,7 +105,7 @@ int pci_bus_add_device(struct pci_dev *dev)void pci_bus_add_devices (struct pci_bus *bus) {struct pci_dev *dev; -struct pci_bus *child_bus; +struct pci_bus *child;int retval;list_for_each_entry (dev, &bus->devices, bus_list) {
@@ -120,39 +120,41 @@ void pci_bus_add_devices(struct pci_bus *bus)list_for_each_entry (dev, &bus->devices, bus_list) {BUG_ON (!dev->is_added); + child = dev->subordinate;/* * If there is an unattached subordinate bus, attach * it and then scan for unattached PCI devices. */ -if (dev->subordinate) { -if (list_empty (&dev->subordinate->node)) { -down_write (&pci_bus_sem); -list_add_tail (&dev->subordinate->node, - &dev->bus->children); -up_write (&pci_bus_sem); - } -pci_bus_add_devices (dev->subordinate); - -/* register the bus with sysfs as the parent is now -* properly registered. */ - child_bus = dev->subordinate; -if (child_bus->is_added) -continue ; - child_bus->dev.parent = child_bus->bridge; - retval =device_register (&child_bus->dev); -if (retval) -dev_err (&dev->dev,"Error registering pci_bus," -" continuing...\n" ); -else { - child_bus->is_added =1 ; - retval =device_create_file (&child_bus->dev, - &dev_attr_cpuaffinity); - } +if (!child) +continue ; +if (list_empty (&child->node)) { +down_write (&pci_bus_sem); +list_add_tail (&child->node, + &dev->bus->children); +up_write (&pci_bus_sem); + } +pci_bus_add_devices (child); + +/* +* register the bus with sysfs as the parent is now +* properly registered. +*/ +if (child->is_added) +continue ; + child->dev.parent = child->bridge; + retval =device_register (&child->dev); +if (retval) +dev_err (&dev->dev,"Error registering pci_bus," +" continuing...\n" ); +else { + child->is_added =1 ; + retval =device_create_file (&child->dev, + &dev_attr_cpuaffinity);if (retval)dev_err (&dev->dev,"Error creating cpuaffinity" " file, continuing...\n" ); - retval =device_create_file (&child_bus->dev, + retval =device_create_file (&child->dev, &dev_attr_cpulistaffinity);if (retval) dev_err(&dev->dev,
drivers/pci/remove.c +2 -0
@@ -72,6 +72,8 @@ void pci_remove_bus(struct pci_bus *pci_bus)list_del (&pci_bus->node);up_write (&pci_bus_sem);pci_remove_legacy_files (pci_bus); +if (!pci_bus->is_added) +return ;device_remove_file (&pci_bus->dev, &dev_attr_cpuaffinity);device_remove_file (&pci_bus->dev, &dev_attr_cpulistaffinity);device_unregister (&pci_bus->dev);
Signature · 1 line
1.5.6.4
PATCH v5 11/15 PCI: split a new function from pci_bus_add_devices()
In reply to Yu ZhaoThis patch splits a new function from pci_bus_add_devices(). The new function can be used to register a PCI bus to device core and create its sysfs entries.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/bus.c | 48 +++++++++++++++++++++++++++++------------------- include/linux/pci.h | 1 + 2 files changed, 30 insertions(+), 19 deletions(-)
drivers/pci/bus.c +29 -19
@@ -91,6 +91,33 @@ int pci_bus_add_device(struct pci_dev *dev) }/** +* pci_bus_add_child - add a child bus +* @bus: bus to add +* +* This adds sysfs entries for a single bus +*/ +int pci_bus_add_child (struct pci_bus *bus) +{ +int retval; + +if (bus->bridge) + bus->dev.parent = bus->bridge; + + retval =device_register (&bus->dev); +if (retval) +return retval; + + bus->is_added =1 ; + + retval =device_create_file (&bus->dev, &dev_attr_cpuaffinity); +if (retval) +return retval; + + retval =device_create_file (&bus->dev, &dev_attr_cpulistaffinity); +return retval; +} + +/** * pci_bus_add_devices - insert newly discovered PCI devices * @bus: bus to check for new devices *
@@ -141,26 +168,9 @@ void pci_bus_add_devices(struct pci_bus *bus)*/ if (child->is_added)continue ; - child->dev.parent = child->bridge; - retval =device_register (&child->dev); + retval =pci_bus_add_child (child);if (retval) -dev_err (&dev->dev,"Error registering pci_bus," -" continuing...\n" ); -else { - child->is_added =1 ; - retval =device_create_file (&child->dev, - &dev_attr_cpuaffinity); -if (retval) -dev_err (&dev->dev,"Error creating cpuaffinity" -" file, continuing...\n" ); - - retval =device_create_file (&child->dev, - &dev_attr_cpulistaffinity); -if (retval) -dev_err (&dev->dev, -"Error creating cpulistaffinity" -" file, continuing...\n" ); - } +dev_err (&dev->dev,"Error adding bus, continuing\n" ); } }
include/linux/pci.h +1 -0
@@ -528,6 +528,7 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn);void pci_device_add (struct pci_dev *dev,struct pci_bus *bus);unsigned int pci_scan_child_bus (struct pci_bus *bus);int __must_check pci_bus_add_device (struct pci_dev *dev); +int pci_bus_add_child (struct pci_bus *bus);void pci_read_bridge_bases (struct pci_bus *child);struct resource *pci_find_parent_resource (const struct pci_dev *dev,struct resource *res);
Signature · 1 line
1.5.6.4
PATCH v5 11/15 PCI: split a new function from pci_bus_add_devices()
In reply to Yu ZhaoThis patch splits a new function from pci_bus_add_devices(). The new function can be used to register a PCI bus to device core and create its sysfs entries.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/bus.c | 48 +++++++++++++++++++++++++++++------------------- include/linux/pci.h | 1 + 2 files changed, 30 insertions(+), 19 deletions(-)
drivers/pci/bus.c +29 -19
@@ -91,6 +91,33 @@ int pci_bus_add_device(struct pci_dev *dev) }/** +* pci_bus_add_child - add a child bus +* @bus: bus to add +* +* This adds sysfs entries for a single bus +*/ +int pci_bus_add_child (struct pci_bus *bus) +{ +int retval; + +if (bus->bridge) + bus->dev.parent = bus->bridge; + + retval =device_register (&bus->dev); +if (retval) +return retval; + + bus->is_added =1 ; + + retval =device_create_file (&bus->dev, &dev_attr_cpuaffinity); +if (retval) +return retval; + + retval =device_create_file (&bus->dev, &dev_attr_cpulistaffinity); +return retval; +} + +/** * pci_bus_add_devices - insert newly discovered PCI devices * @bus: bus to check for new devices *
@@ -141,26 +168,9 @@ void pci_bus_add_devices(struct pci_bus *bus)*/ if (child->is_added)continue ; - child->dev.parent = child->bridge; - retval =device_register (&child->dev); + retval =pci_bus_add_child (child);if (retval) -dev_err (&dev->dev,"Error registering pci_bus," -" continuing...\n" ); -else { - child->is_added =1 ; - retval =device_create_file (&child->dev, - &dev_attr_cpuaffinity); -if (retval) -dev_err (&dev->dev,"Error creating cpuaffinity" -" file, continuing...\n" ); - - retval =device_create_file (&child->dev, - &dev_attr_cpulistaffinity); -if (retval) -dev_err (&dev->dev, -"Error creating cpulistaffinity" -" file, continuing...\n" ); - } +dev_err (&dev->dev,"Error adding bus, continuing\n" ); } }
include/linux/pci.h +1 -0
@@ -528,6 +528,7 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn);void pci_device_add (struct pci_dev *dev,struct pci_bus *bus);unsigned int pci_scan_child_bus (struct pci_bus *bus);int __must_check pci_bus_add_device (struct pci_dev *dev); +int pci_bus_add_child (struct pci_bus *bus);void pci_read_bridge_bases (struct pci_bus *child);struct resource *pci_find_parent_resource (const struct pci_dev *dev,struct resource *res);
Signature · 1 line
1.5.6.4
PATCH v5 12/15 PCI: support the SR-IOV capability
In reply to Yu ZhaoSupport Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/Kconfig | 12 + drivers/pci/Makefile | 2 + drivers/pci/iov.c | 616 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci-sysfs.c | 4 + drivers/pci/pci.c | 14 + drivers/pci/pci.h | 48 ++++ drivers/pci/probe.c | 4 + include/linux/pci.h | 40 +++ include/linux/pci_regs.h | 21 ++ 9 files changed, 761 insertions(+), 0 deletions(-) create mode 100644 drivers/pci/iov.c
drivers/pci/Kconfig +12 -0
@@ -50,3 +50,15 @@ config HT_IRQ This allows native hypertransport devices to use interrupts. If unsure say Y. + +config PCI_IOV + bool "PCI SR-IOV support" + depends on PCI + select PCI_MSI + default n + help + This option allows device drivers to enable Single Root I/O + Virtualization. Each Virtual Function's PCI configuration + space can be accessed using its own Bus, Device and Function + Number (Routing ID). Each Virtual Function also has PCI Memory + Space, which is used to map its own register set.
drivers/pci/Makefile +2 -0
@@ -55,3 +55,5 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o ifeq ($(CONFIG_PCI_DEBUG),y) EXTRA_CFLAGS += -DDEBUG endif + +obj-$(CONFIG_PCI_IOV) += iov.o
drivers/pci/iov.c new file +616 -0
616 lines not on this page · raw message
drivers/pci/pci-sysfs.c +4 -0
@@ -847,6 +847,9 @@ static int pci_create_capabilities_sysfs(struct pci_dev *dev)/* Active State Power Management */ pcie_aspm_create_sysfs_dev_files (dev); +/* Single Root I/O Virtualization */ +pci_iov_create_sysfs (dev); +return 0 ; }
@@ -932,6 +935,7 @@ static void pci_remove_capabilities_sysfs(struct pci_dev *dev) }pcie_aspm_remove_sysfs_dev_files (dev); +pci_iov_remove_sysfs (dev); }/**
drivers/pci/pci.c +14 -0
@@ -1936,6 +1936,13 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)if (align)return align > bios_align ? align : bios_align; +if (resno >PCI_ROM_RESOURCE && resno <PCI_BRIDGE_RESOURCES ) { +/* device specific resource */ + align =pci_iov_resource_align (dev, resno); +if (align) +return align > bios_align ? align : bios_align; + } +dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno);return 0 ; }
@@ -1950,12 +1957,19 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)*/ int pci_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type) { +int reg; +if (resno <PCI_ROM_RESOURCE ) { *type = pci_bar_unknown;return PCI_BASE_ADDRESS_0 +4 * resno; }else if (resno ==PCI_ROM_RESOURCE ) { *type = pci_bar_mem32;return dev->rom_base_reg; + }else if (resno <PCI_BRIDGE_RESOURCES ) { +/* device specific resource */ + reg =pci_iov_resource_bar (dev, resno, type); +if (reg) +return reg; }dev_err (&dev->dev,"BAR: invalid resource #%d\n" , resno);
drivers/pci/pci.h +48 -0
52 lines not on this page · raw message
drivers/pci/probe.c +4 -0
@@ -779,6 +779,7 @@ static int pci_setup_device(struct pci_dev * dev)static void pci_release_capabilities (struct pci_dev *dev) {pci_vpd_release (dev); +pci_iov_release (dev); }/**
@@ -962,6 +963,9 @@ static void pci_init_capabilities(struct pci_dev *dev)/* Alternative Routing-ID Forwarding */ pci_enable_ari (dev); + +/* Single Root I/O Virtualization */ +pci_iov_init (dev); }void pci_device_add (struct pci_dev *dev,struct pci_bus *bus)
include/linux/pci.h +40 -0
@@ -87,6 +87,12 @@ enum {/* #6: expansion ROM */ PCI_ROM_RESOURCE, +/* device specific resources */ +#ifdef CONFIG_PCI_IOV + PCI_IOV_RESOURCES, + PCI_IOV_RESOURCES_END = PCI_IOV_RESOURCES + PCI_IOV_NUM_BAR - 1, +#endif +/* address space assigned to buses behind the bridge */ #ifndef PCI_BRIDGE_RES_NUM #define PCI_BRIDGE_RES_NUM 4
@@ -165,6 +171,7 @@ struct pci_cap_saved_state {struct pcie_link_state ;struct pci_vpd ; +struct pci_iov ;/* * The pci_dev structure is used to describe PCI devices.
@@ -253,6 +260,7 @@ struct pci_dev {struct list_head msi_list;#endif struct pci_vpd *vpd; +struct pci_iov *iov; };extern struct pci_dev *alloc_pci_dev (void );
@@ -1147,5 +1155,37 @@ static inline void * pci_ioremap_bar(struct pci_dev *pdev, int bar) }#endif +/* SR-IOV events masks */ +#define PCI_IOV_VIRTFN_ID 0x0000FFFFU/* Virtual Function Number */ +#define PCI_IOV_NUM_VIRTFN 0x0000FFFFU/* NumVFs to be set */ +/* SR-IOV events values */ +#define PCI_IOV_ENABLE 0x00010000U/* SR-IOV enable request */ +#define PCI_IOV_DISABLE 0x00020000U/* SR-IOV disable request */ +#define PCI_IOV_NUMVFS 0x00040000U/* SR-IOV disable request */ + +#ifdef CONFIG_PCI_IOV +extern int pci_iov_enable (struct pci_dev *dev,int numvfs); +extern void pci_iov_disable (struct pci_dev *dev); +extern int pci_iov_register (struct pci_dev *dev, +int (*callback)(struct pci_dev *dev,u32 event)); +extern void pci_iov_unregister (struct pci_dev *dev); +#else +static inline int pci_iov_enable (struct pci_dev *dev,int numvfs) +{ +return -EIO ; +} +static inline void pci_iov_disable (struct pci_dev *dev) +{ +} +static inline int pci_iov_register (struct pci_dev *dev, +int (*callback)(struct pci_dev *dev,u32 event)) +{ +return -EIO ; +} +static inline void pci_iov_unregister (struct pci_dev *dev) +{ +} +#endif /* CONFIG_PCI_IOV */ +#endif /* __KERNEL__ */ #endif /* LINUX_PCI_H */
include/linux/pci_regs.h +21 -0
@@ -363,6 +363,7 @@#define PCI_EXP_TYPE_UPSTREAM 0x5/* Upstream Port */ #define PCI_EXP_TYPE_DOWNSTREAM 0x6/* Downstream Port */ #define PCI_EXP_TYPE_PCI_BRIDGE 0x7/* PCI/PCI-X Bridge */ +#define PCI_EXP_TYPE_RC_END 0x9/* Root Complex Integrated Endpoint */ #define PCI_EXP_FLAGS_SLOT 0x0100/* Slot implemented */ #define PCI_EXP_FLAGS_IRQ 0x3e00/* Interrupt message number */ #define PCI_EXP_DEVCAP 4/* Device capabilities */
@@ -434,6 +435,7 @@#define PCI_EXT_CAP_ID_DSN 3#define PCI_EXT_CAP_ID_PWR 4#define PCI_EXT_CAP_ID_ARI 14 +#define PCI_EXT_CAP_ID_IOV 16/* Advanced Error Reporting */ #define PCI_ERR_UNCOR_STATUS 4/* Uncorrectable Error Status */
@@ -551,4 +553,23 @@#define PCI_ARI_CTRL_ACS 0x0002/* ACS Function Groups Enable */ #define PCI_ARI_CTRL_FG (x) (((x) >> 4) & 7)/* Function Group */ +/* Single Root I/O Virtualization */ +#define PCI_IOV_CAP 0x04/* SR-IOV Capabilities */ +#define PCI_IOV_CTRL 0x08/* SR-IOV Control */ +#define PCI_IOV_CTRL_VFE 0x01/* VF Enable */ +#define PCI_IOV_CTRL_MSE 0x08/* VF Memory Space Enable */ +#define PCI_IOV_CTRL_ARI 0x10/* ARI Capable Hierarchy */ +#define PCI_IOV_STATUS 0x0a/* SR-IOV Status */ +#define PCI_IOV_INITIAL_VF 0x0c/* Initial VFs */ +#define PCI_IOV_TOTAL_VF 0x0e/* Total VFs */ +#define PCI_IOV_NUM_VF 0x10/* Number of VFs */ +#define PCI_IOV_FUNC_LINK 0x12/* Function Dependency Link */ +#define PCI_IOV_VF_OFFSET 0x14/* First VF Offset */ +#define PCI_IOV_VF_STRIDE 0x16/* Following VF Stride */ +#define PCI_IOV_VF_DID 0x1a/* VF Device ID */ +#define PCI_IOV_SUP_PGSIZE 0x1c/* Supported Page Sizes */ +#define PCI_IOV_SYS_PGSIZE 0x20/* System Page Size */ +#define PCI_IOV_BAR_0 0x24/* VF BAR0 */ +#define PCI_IOV_NUM_BAR 6/* Number of VF BARs */ +#endif /* LINUX_PCI_REGS_H */
Signature · 1 line
1.5.6.4
Re: [PATCH 12/15 v5] PCI: support the SR-IOV capability
Quoted 78 lines · Yu Zhao
Support Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>Cc Randy Dunlap <[email protected]>Cc Grant Grundler <[email protected]>Cc Alex Chiang <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Roland Dreier <[email protected]>Cc Greg KH <[email protected]>Signed-off-by Yu Zhao <[email protected]>--- drivers/pci/Kconfig | 12 + drivers/pci/Makefile | 2 + drivers/pci/iov.c | 616 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci-sysfs.c | 4 + drivers/pci/pci.c | 14 + drivers/pci/pci.h | 48 ++++ drivers/pci/probe.c | 4 + include/linux/pci.h | 40 +++ include/linux/pci_regs.h | 21 ++ 9 files changed, 761 insertions(+), 0 deletions(-) create mode 100644 drivers/pci/iov.cdrivers/pci/Kconfig +12
-0@@ -50,3 +50,15 @@ config HT_IRQ This allows native hypertransport devices to use interrupts. If unsure say Y. + +config PCI_IOV + bool "PCI SR-IOV support" + depends on PCI + select PCI_MSI + default n + help + This option allows device drivers to enable Single Root I/O + Virtualization. Each Virtual Function's PCI configuration + space can be accessed using its own Bus, Device and Function + Number (Routing ID). Each Virtual Function also has PCI Memory + Space, which is used to map its own register set.drivers/pci/Makefile +2
-0@@ -55,3 +55,5 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o ifeq ($(CONFIG_PCI_DEBUG),y) EXTRA_CFLAGS += -DDEBUG endif + +obj-$(CONFIG_PCI_IOV) += iov.odrivers/pci/iov.c new file +18
-0@@ -0,0 +1,616 @@ +/* +* drivers/pci/iov.c +* +* Copyright (C) 2008 Intel Corporation +* +* PCI Express Single Root I/O Virtualization capability support. +*/ + +#include <linux/ctype.h> +#include <linux/string.h> +#include <linux/pci.h> +#include <linux/delay.h> +#include <asm/page.h> +#include "pci.h" + + +#define iov_config_attr (field) \ +static ssize_t field##_show(struct device *dev, \
drivers/pci/iov.c line 28
Quoted 10 lines
drivers/pci/iov.c new file +10
-0+ struct device_attribute *attr, char *buf) \ +{ \ + struct pci_dev *pdev = to_pci_dev(dev); \ +return sprintf (buf,"%d\n" , pdev->iov->field); \ +} + +iov_config_attr (status); +iov_config_attr (totalvfs); +iov_config_attr (initialvfs); +iov_config_attr (numvfs);
As you are adding new sysfs entries, can you also create the proper documentation in Documentation/ABI/ so that people can understand how to use them? Yes, I see you added a stand-alone document, but putting it in the "standard" format is also necessary.
thanks,
greg k-h
Re: [PATCH 12/15 v5] PCI: support the SR-IOV capability
In reply to Yu ZhaoQuoted 78 lines · Yu Zhao
Support Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>Cc Randy Dunlap <[email protected]>Cc Grant Grundler <[email protected]>Cc Alex Chiang <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Roland Dreier <[email protected]>Cc Greg KH <[email protected]>Signed-off-by Yu Zhao <[email protected]>--- drivers/pci/Kconfig | 12 + drivers/pci/Makefile | 2 + drivers/pci/iov.c | 616 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci-sysfs.c | 4 + drivers/pci/pci.c | 14 + drivers/pci/pci.h | 48 ++++ drivers/pci/probe.c | 4 + include/linux/pci.h | 40 +++ include/linux/pci_regs.h | 21 ++ 9 files changed, 761 insertions(+), 0 deletions(-) create mode 100644 drivers/pci/iov.cdrivers/pci/Kconfig +12
-0@@ -50,3 +50,15 @@ config HT_IRQ This allows native hypertransport devices to use interrupts. If unsure say Y. + +config PCI_IOV + bool "PCI SR-IOV support" + depends on PCI + select PCI_MSI + default n + help + This option allows device drivers to enable Single Root I/O + Virtualization. Each Virtual Function's PCI configuration + space can be accessed using its own Bus, Device and Function + Number (Routing ID). Each Virtual Function also has PCI Memory + Space, which is used to map its own register set.drivers/pci/Makefile +2
-0@@ -55,3 +55,5 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o ifeq ($(CONFIG_PCI_DEBUG),y) EXTRA_CFLAGS += -DDEBUG endif + +obj-$(CONFIG_PCI_IOV) += iov.odrivers/pci/iov.c new file +18
-0@@ -0,0 +1,616 @@ +/* +* drivers/pci/iov.c +* +* Copyright (C) 2008 Intel Corporation +* +* PCI Express Single Root I/O Virtualization capability support. +*/ + +#include <linux/ctype.h> +#include <linux/string.h> +#include <linux/pci.h> +#include <linux/delay.h> +#include <asm/page.h> +#include "pci.h" + + +#define iov_config_attr (field) \ +static ssize_t field##_show(struct device *dev, \
drivers/pci/iov.c line 28
Quoted 10 lines
drivers/pci/iov.c new file +10
-0+ struct device_attribute *attr, char *buf) \ +{ \ + struct pci_dev *pdev = to_pci_dev(dev); \ +return sprintf (buf,"%d\n" , pdev->iov->field); \ +} + +iov_config_attr (status); +iov_config_attr (totalvfs); +iov_config_attr (initialvfs); +iov_config_attr (numvfs);
As you are adding new sysfs entries, can you also create the proper documentation in Documentation/ABI/ so that people can understand how to use them? Yes, I see you added a stand-alone document, but putting it in the "standard" format is also necessary.
thanks,
greg k-h
Re: [PATCH 12/15 v5] PCI: support the SR-IOV capability
Quoted 22 lines · Greg KH
Quoted 21 lines · Yu Zhao
Support Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>Cc Randy Dunlap <[email protected]>Cc Grant Grundler <[email protected]>Cc Alex Chiang <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Roland Dreier <[email protected]>Cc Greg KH <[email protected]>Signed-off-by Yu Zhao <[email protected]>drivers/pci/iov.c new file +9
-0+#define iov_config_attr (field) \ +static ssize_t field##_show(struct device *dev, \ + struct device_attribute *attr, char *buf) \ +{ \ + struct pci_dev *pdev = to_pci_dev(dev); \ + return sprintf(buf, "%d\n", pdev->iov->field); \ +} + +iov_config_attr (status);
Quoted 8 lines
drivers/pci/iov.c line 28
Quoted 3 lines
drivers/pci/iov.c new file +3
-0+iov_config_attr (totalvfs); +iov_config_attr (initialvfs); +iov_config_attr (numvfs);As you are adding new sysfs entries, can you also create the proper documentation in
Documentation/ABI/so that people can understand how to use them? Yes, I see you added a stand-alone document, but putting it in the "standard" format is also necessary.
Thanks for reminding me about this. I used to update ABI doc in earlier versions, but somehow forgot to do this after several carry-forwards...
Will complete it in next version.
Regards,
Yu
Re: [PATCH 12/15 v5] PCI: support the SR-IOV capability
In reply to Greg KHQuoted 22 lines · Greg KH
Quoted 21 lines · Yu Zhao
Support Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>Cc Randy Dunlap <[email protected]>Cc Grant Grundler <[email protected]>Cc Alex Chiang <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Roland Dreier <[email protected]>Cc Greg KH <[email protected]>Signed-off-by Yu Zhao <[email protected]>drivers/pci/iov.c new file +9
-0+#define iov_config_attr (field) \ +static ssize_t field##_show(struct device *dev, \ + struct device_attribute *attr, char *buf) \ +{ \ + struct pci_dev *pdev = to_pci_dev(dev); \ + return sprintf(buf, "%d\n", pdev->iov->field); \ +} + +iov_config_attr (status);
Quoted 8 lines
drivers/pci/iov.c line 28
Quoted 3 lines
drivers/pci/iov.c new file +3
-0+iov_config_attr (totalvfs); +iov_config_attr (initialvfs); +iov_config_attr (numvfs);As you are adding new sysfs entries, can you also create the proper documentation in
Documentation/ABI/so that people can understand how to use them? Yes, I see you added a stand-alone document, but putting it in the "standard" format is also necessary.
Thanks for reminding me about this. I used to update ABI doc in earlier versions, but somehow forgot to do this after several carry-forwards...
Will complete it in next version.
Regards,
Yu
PATCH v5 12/15 PCI: support the SR-IOV capability
In reply to Yu ZhaoSupport Single Root I/O Virtualization (SR-IOV) capability.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/Kconfig | 12 + drivers/pci/Makefile | 2 + drivers/pci/iov.c | 616 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/pci-sysfs.c | 4 + drivers/pci/pci.c | 14 + drivers/pci/pci.h | 48 ++++ drivers/pci/probe.c | 4 + include/linux/pci.h | 40 +++ include/linux/pci_regs.h | 21 ++ 9 files changed, 761 insertions(+), 0 deletions(-) create mode 100644 drivers/pci/iov.c
drivers/pci/Kconfig +12 -0
@@ -50,3 +50,15 @@ config HT_IRQ This allows native hypertransport devices to use interrupts. If unsure say Y. + +config PCI_IOV + bool "PCI SR-IOV support" + depends on PCI + select PCI_MSI + default n + help + This option allows device drivers to enable Single Root I/O + Virtualization. Each Virtual Function's PCI configuration + space can be accessed using its own Bus, Device and Function + Number (Routing ID). Each Virtual Function also has PCI Memory + Space, which is used to map its own register set.
drivers/pci/Makefile +2 -0
@@ -55,3 +55,5 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o ifeq ($(CONFIG_PCI_DEBUG),y) EXTRA_CFLAGS += -DDEBUG endif + +obj-$(CONFIG_PCI_IOV) += iov.o
drivers/pci/iov.c new file +616 -0
616 lines not on this page · raw message
drivers/pci/pci-sysfs.c +4 -0
@@ -847,6 +847,9 @@ static int pci_create_capabilities_sysfs(struct pci_dev *dev)/* Active State Power Management */ pcie_aspm_create_sysfs_dev_files (dev); +/* Single Root I/O Virtualization */ +pci_iov_create_sysfs (dev); +return 0 ; }
@@ -932,6 +935,7 @@ static void pci_remove_capabilities_sysfs(struct pci_dev *dev) }pcie_aspm_remove_sysfs_dev_files (dev); +pci_iov_remove_sysfs (dev); }/**
drivers/pci/pci.c +14 -0
@@ -1936,6 +1936,13 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)if (align)return align > bios_align ? align : bios_align; +if (resno >PCI_ROM_RESOURCE && resno <PCI_BRIDGE_RESOURCES ) { +/* device specific resource */ + align =pci_iov_resource_align (dev, resno); +if (align) +return align > bios_align ? align : bios_align; + } +dev_err (&dev->dev,"alignment: invalid resource #%d\n" , resno);return 0 ; }
@@ -1950,12 +1957,19 @@ int pci_resource_alignment(struct pci_dev *dev, int resno)*/ int pci_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type) { +int reg; +if (resno <PCI_ROM_RESOURCE ) { *type = pci_bar_unknown;return PCI_BASE_ADDRESS_0 +4 * resno; }else if (resno ==PCI_ROM_RESOURCE ) { *type = pci_bar_mem32;return dev->rom_base_reg; + }else if (resno <PCI_BRIDGE_RESOURCES ) { +/* device specific resource */ + reg =pci_iov_resource_bar (dev, resno, type); +if (reg) +return reg; }dev_err (&dev->dev,"BAR: invalid resource #%d\n" , resno);
drivers/pci/pci.h +48 -0
52 lines not on this page · raw message
drivers/pci/probe.c +4 -0
@@ -779,6 +779,7 @@ static int pci_setup_device(struct pci_dev * dev)static void pci_release_capabilities (struct pci_dev *dev) {pci_vpd_release (dev); +pci_iov_release (dev); }/**
@@ -962,6 +963,9 @@ static void pci_init_capabilities(struct pci_dev *dev)/* Alternative Routing-ID Forwarding */ pci_enable_ari (dev); + +/* Single Root I/O Virtualization */ +pci_iov_init (dev); }void pci_device_add (struct pci_dev *dev,struct pci_bus *bus)
include/linux/pci.h +40 -0
@@ -87,6 +87,12 @@ enum {/* #6: expansion ROM */ PCI_ROM_RESOURCE, +/* device specific resources */ +#ifdef CONFIG_PCI_IOV + PCI_IOV_RESOURCES, + PCI_IOV_RESOURCES_END = PCI_IOV_RESOURCES + PCI_IOV_NUM_BAR - 1, +#endif +/* address space assigned to buses behind the bridge */ #ifndef PCI_BRIDGE_RES_NUM #define PCI_BRIDGE_RES_NUM 4
@@ -165,6 +171,7 @@ struct pci_cap_saved_state {struct pcie_link_state ;struct pci_vpd ; +struct pci_iov ;/* * The pci_dev structure is used to describe PCI devices.
@@ -253,6 +260,7 @@ struct pci_dev {struct list_head msi_list;#endif struct pci_vpd *vpd; +struct pci_iov *iov; };extern struct pci_dev *alloc_pci_dev (void );
@@ -1147,5 +1155,37 @@ static inline void * pci_ioremap_bar(struct pci_dev *pdev, int bar) }#endif +/* SR-IOV events masks */ +#define PCI_IOV_VIRTFN_ID 0x0000FFFFU/* Virtual Function Number */ +#define PCI_IOV_NUM_VIRTFN 0x0000FFFFU/* NumVFs to be set */ +/* SR-IOV events values */ +#define PCI_IOV_ENABLE 0x00010000U/* SR-IOV enable request */ +#define PCI_IOV_DISABLE 0x00020000U/* SR-IOV disable request */ +#define PCI_IOV_NUMVFS 0x00040000U/* SR-IOV disable request */ + +#ifdef CONFIG_PCI_IOV +extern int pci_iov_enable (struct pci_dev *dev,int numvfs); +extern void pci_iov_disable (struct pci_dev *dev); +extern int pci_iov_register (struct pci_dev *dev, +int (*callback)(struct pci_dev *dev,u32 event)); +extern void pci_iov_unregister (struct pci_dev *dev); +#else +static inline int pci_iov_enable (struct pci_dev *dev,int numvfs) +{ +return -EIO ; +} +static inline void pci_iov_disable (struct pci_dev *dev) +{ +} +static inline int pci_iov_register (struct pci_dev *dev, +int (*callback)(struct pci_dev *dev,u32 event)) +{ +return -EIO ; +} +static inline void pci_iov_unregister (struct pci_dev *dev) +{ +} +#endif /* CONFIG_PCI_IOV */ +#endif /* __KERNEL__ */ #endif /* LINUX_PCI_H */
include/linux/pci_regs.h +21 -0
@@ -363,6 +363,7 @@#define PCI_EXP_TYPE_UPSTREAM 0x5/* Upstream Port */ #define PCI_EXP_TYPE_DOWNSTREAM 0x6/* Downstream Port */ #define PCI_EXP_TYPE_PCI_BRIDGE 0x7/* PCI/PCI-X Bridge */ +#define PCI_EXP_TYPE_RC_END 0x9/* Root Complex Integrated Endpoint */ #define PCI_EXP_FLAGS_SLOT 0x0100/* Slot implemented */ #define PCI_EXP_FLAGS_IRQ 0x3e00/* Interrupt message number */ #define PCI_EXP_DEVCAP 4/* Device capabilities */
@@ -434,6 +435,7 @@#define PCI_EXT_CAP_ID_DSN 3#define PCI_EXT_CAP_ID_PWR 4#define PCI_EXT_CAP_ID_ARI 14 +#define PCI_EXT_CAP_ID_IOV 16/* Advanced Error Reporting */ #define PCI_ERR_UNCOR_STATUS 4/* Uncorrectable Error Status */
@@ -551,4 +553,23 @@#define PCI_ARI_CTRL_ACS 0x0002/* ACS Function Groups Enable */ #define PCI_ARI_CTRL_FG (x) (((x) >> 4) & 7)/* Function Group */ +/* Single Root I/O Virtualization */ +#define PCI_IOV_CAP 0x04/* SR-IOV Capabilities */ +#define PCI_IOV_CTRL 0x08/* SR-IOV Control */ +#define PCI_IOV_CTRL_VFE 0x01/* VF Enable */ +#define PCI_IOV_CTRL_MSE 0x08/* VF Memory Space Enable */ +#define PCI_IOV_CTRL_ARI 0x10/* ARI Capable Hierarchy */ +#define PCI_IOV_STATUS 0x0a/* SR-IOV Status */ +#define PCI_IOV_INITIAL_VF 0x0c/* Initial VFs */ +#define PCI_IOV_TOTAL_VF 0x0e/* Total VFs */ +#define PCI_IOV_NUM_VF 0x10/* Number of VFs */ +#define PCI_IOV_FUNC_LINK 0x12/* Function Dependency Link */ +#define PCI_IOV_VF_OFFSET 0x14/* First VF Offset */ +#define PCI_IOV_VF_STRIDE 0x16/* Following VF Stride */ +#define PCI_IOV_VF_DID 0x1a/* VF Device ID */ +#define PCI_IOV_SUP_PGSIZE 0x1c/* Supported Page Sizes */ +#define PCI_IOV_SYS_PGSIZE 0x20/* System Page Size */ +#define PCI_IOV_BAR_0 0x24/* VF BAR0 */ +#define PCI_IOV_NUM_BAR 6/* Number of VF BARs */ +#endif /* LINUX_PCI_REGS_H */
Signature · 1 line
1.5.6.4
PATCH v5 13/15 PCI: reserve bus range for the SR-IOV device
In reply to Yu ZhaoReserve bus range for SR-IOV at device scanning stage.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.h | 5 +++++ drivers/pci/probe.c | 3 +++ 2 files changed, 8 insertions(+), 0 deletions(-)
drivers/pci/pci.h +5 -0
@@ -204,6 +204,7 @@ void pci_iov_remove_sysfs(struct pci_dev *dev);extern int pci_iov_resource_align (struct pci_dev *dev,int resno);extern int pci_iov_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type); +extern int pci_iov_bus_range (struct pci_bus *bus);#else static inline int pci_iov_init (struct pci_dev *dev) {
@@ -227,6 +228,10 @@ static inline int pci_iov_resource_bar(struct pci_dev *dev, int resno, {return 0 ; } +extern inline int pci_iov_bus_range (struct pci_bus *bus) +{ +return 0 ; +}#endif /* CONFIG_PCI_IOV */ #endif /* DRIVERS_PCI_H */
drivers/pci/probe.c +3 -0
@@ -1068,6 +1068,9 @@ unsigned int __devinit pci_scan_child_bus(struct pci_bus *bus)for (devfn =0 ; devfn <0x100 ; devfn +=8 )pci_scan_slot (bus, devfn); +/* Reserve buses for SR-IOV capability. */ + max +=pci_iov_bus_range (bus); +/* * After performing arch-dependent fixup of the bus, look behind * all PCI-to-PCI bridges on this bus.
Signature · 1 line
1.5.6.4
PATCH v5 13/15 PCI: reserve bus range for the SR-IOV device
In reply to Yu ZhaoReserve bus range for SR-IOV at device scanning stage.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- drivers/pci/pci.h | 5 +++++ drivers/pci/probe.c | 3 +++ 2 files changed, 8 insertions(+), 0 deletions(-)
drivers/pci/pci.h +5 -0
@@ -204,6 +204,7 @@ void pci_iov_remove_sysfs(struct pci_dev *dev);extern int pci_iov_resource_align (struct pci_dev *dev,int resno);extern int pci_iov_resource_bar (struct pci_dev *dev,int resno,enum pci_bar_type *type); +extern int pci_iov_bus_range (struct pci_bus *bus);#else static inline int pci_iov_init (struct pci_dev *dev) {
@@ -227,6 +228,10 @@ static inline int pci_iov_resource_bar(struct pci_dev *dev, int resno, {return 0 ; } +extern inline int pci_iov_bus_range (struct pci_bus *bus) +{ +return 0 ; +}#endif /* CONFIG_PCI_IOV */ #endif /* DRIVERS_PCI_H */
drivers/pci/probe.c +3 -0
@@ -1068,6 +1068,9 @@ unsigned int __devinit pci_scan_child_bus(struct pci_bus *bus)for (devfn =0 ; devfn <0x100 ; devfn +=8 )pci_scan_slot (bus, devfn); +/* Reserve buses for SR-IOV capability. */ + max +=pci_iov_bus_range (bus); +/* * After performing arch-dependent fixup of the bus, look behind * all PCI-to-PCI bridges on this bus.
Signature · 1 line
1.5.6.4
PATCH v5 14/15 PCI: document the SR-IOV
In reply to Yu ZhaoCreate how-to for SR-IOV user and device driver developer.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- Documentation/PCI/pci-iov-howto.txt | 181 +++++++++++++++++++++++++++++++++++ 1 files changed, 181 insertions(+), 0 deletions(-) create mode 100644 Documentation/PCI/pci-iov-howto.txt
Documentation/PCI/pci-iov-howto.txt new file +181 -0
@@ -0,0 +1,181 @@ + PCI Express Single Root I/O Virtualization HOWTO + Copyright (C) 2008 Intel Corporation + + +1. Overview + +1.1 What is SR-IOV + +Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended +capability which makes one physical device appear as multiple virtual +devices. The physical device is referred to as Physical Function while +the virtual devices are referred to as Virtual Functions. Allocation +of Virtual Functions can be dynamically controlled by Physical Function +via registers encapsulated in the capability. By default, this feature +is not enabled and the Physical Function behaves as traditional PCIe +device. Once it's turned on, each Virtual Function's PCI configuration +space can be accessed by its own Bus, Device and Function Number (Routing +ID). And each Virtual Function also has PCI Memory Space, which is used +to map its register set. Virtual Function device driver operates on the +register set so it can be functional and appear as a real existing PCI +device. + +2. User Guide + +2.1 How can I manage SR-IOV + +If a device supports SR-IOV, then there should be some entries under +Physical Function's PCI device directory. These entries are in directory: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/ + (XXXX:BB:DD:F is the domain, bus, device and function number) + +To enable or disable SR-IOV: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/enable + (writing 1/0 means enable/disable VFs, state change will + notify PF driver) + +To change number of Virtual Functions: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/numvfs + (writing positive integer to this file will change NumVFs) + +The total and initial number of VFs can get from: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/totalvfs + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/initialvfs + +2.2 How can I use Virtual Functions + +Virtual Functions are treated as hot-plugged PCI devices in the kernel, +so they should be able to work in the same way as real PCI devices. +NOTE: Virtual Function device driver must be loaded to make it work. + + +3. Developer Guide + +3.1 SR-IOV APIs + +To register SR-IOV service, Physical Function device driver needs to call: + int pci_iov_register(struct pci_dev *dev, + int (*callback)(struct pci_dev *, u32)) + The 'callback' is a callback function that the SR-IOV code will invoke + it when events related to VFs happen (e.g. user enable/disable VFs). + The first argument is PF itself, the second argument is event type and + value. For now, following events type are supported: + - PCI_IOV_ENABLE: SR-IOV enable request + - PCI_IOV_DISABLE: SR-IOV disable request + - PCI_IOV_NUMVFS: changing Number of VFs request + And event values can be extract using following masks: + - PCI_IOV_NUM_VIRTFN: Number of Virtual Functions + +To unregister SR-IOV service, Physical Function device driver needs to call: + void pci_iov_unregister(struct pci_dev *dev) + +To enable SR-IOV, Physical Function device driver needs to call: + int pci_iov_enable(struct pci_dev *dev, int numvfs) + 'numvfs' is the number of VFs that PF wants to enable. + +To disable SR-IOV, Physical Function device driver needs to call: + void pci_iov_disable(struct pci_dev *dev) + +Note: above two functions sleeps 1 second waiting on hardware transaction +completion according to SR-IOV specification. + +3.2 Usage example + +Following piece of code illustrates the usage of APIs above. + +static int callback(struct pci_dev *dev, u32 event) +{ + int numvfs; + + if (event & PCI_IOV_ENABLE) { + /* + * request to enable SR-IOV. + * Note: if the PF driver want to support PM, it has + * to check the device power state here to see if this + * request is allowed or not. + */ + ... + + } else if (event & PCI_IOV_DISABLE) { + /* + * request to disable SR-IOV. + */ + ... + + } else if (event & PCI_IOV_NUMVFS) { + /* + * request to change NumVFs. + */ + numvfs = event & PCI_IOV_NUM_VIRTFN; + ... + + } else + return -EINVAL; + + return 0; +} + +static int __devinit dev_probe(struct pci_dev *dev, + const struct pci_device_id *id) +{ + int err; + int numvfs; + + ... + err = pci_iov_register(dev, callback); + ... + err = pci_iov_enable(dev, numvfs); + ... + + return err; +} + +static void __devexit dev_remove(struct pci_dev *dev) +{ + ... + pci_iov_disable(dev); + ... + pci_iov_unregister(dev); + ... +} + +#ifdef CONFIG_PM +/* + * If Physical Function supports the power management, then the + * SR-IOV needs to be disabled before the adapter goes to sleep, + * because Virtual Functions will not work when the adapter is in + * the power-saving mode. + * The SR-IOV can be enabled again after the adapter wakes up. + */ +static int dev_suspend(struct pci_dev *dev, pm_message_t state) +{ + ... + pci_iov_disable(dev); + ... + + return 0; +} + +static int dev_resume(struct pci_dev *dev) +{ + int err; + int numvfs; + + ... + rc = pci_iov_enable(dev, numvfs); + ... + + return 0; +} +#endif + +static struct pci_driver dev_driver = { + .name = "SR-IOV Physical Function driver", + .id_table = dev_id_table, + .probe = dev_probe, + .remove = __devexit_p(dev_remove), +#ifdef CONFIG_PM + .suspend = dev_suspend, + .resume = dev_resume, +#endif +};
Signature · 1 line
1.5.6.4
PATCH v5 14/15 PCI: document the SR-IOV
In reply to Yu ZhaoCreate how-to for SR-IOV user and device driver developer.
Cc Jesse Barnes <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Grant Grundler <[email protected]>
Cc Alex Chiang <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Roland Dreier <[email protected]>
Cc Greg KH <[email protected]>
Signed-off-by Yu Zhao <[email protected]>
--- Documentation/PCI/pci-iov-howto.txt | 181 +++++++++++++++++++++++++++++++++++ 1 files changed, 181 insertions(+), 0 deletions(-) create mode 100644 Documentation/PCI/pci-iov-howto.txt
Documentation/PCI/pci-iov-howto.txt new file +181 -0
@@ -0,0 +1,181 @@ + PCI Express Single Root I/O Virtualization HOWTO + Copyright (C) 2008 Intel Corporation + + +1. Overview + +1.1 What is SR-IOV + +Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended +capability which makes one physical device appear as multiple virtual +devices. The physical device is referred to as Physical Function while +the virtual devices are referred to as Virtual Functions. Allocation +of Virtual Functions can be dynamically controlled by Physical Function +via registers encapsulated in the capability. By default, this feature +is not enabled and the Physical Function behaves as traditional PCIe +device. Once it's turned on, each Virtual Function's PCI configuration +space can be accessed by its own Bus, Device and Function Number (Routing +ID). And each Virtual Function also has PCI Memory Space, which is used +to map its register set. Virtual Function device driver operates on the +register set so it can be functional and appear as a real existing PCI +device. + +2. User Guide + +2.1 How can I manage SR-IOV + +If a device supports SR-IOV, then there should be some entries under +Physical Function's PCI device directory. These entries are in directory: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/ + (XXXX:BB:DD:F is the domain, bus, device and function number) + +To enable or disable SR-IOV: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/enable + (writing 1/0 means enable/disable VFs, state change will + notify PF driver) + +To change number of Virtual Functions: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/numvfs + (writing positive integer to this file will change NumVFs) + +The total and initial number of VFs can get from: + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/totalvfs + - /sys/bus/pci/devices/XXXX:BB:DD.F/iov/initialvfs + +2.2 How can I use Virtual Functions + +Virtual Functions are treated as hot-plugged PCI devices in the kernel, +so they should be able to work in the same way as real PCI devices. +NOTE: Virtual Function device driver must be loaded to make it work. + + +3. Developer Guide + +3.1 SR-IOV APIs + +To register SR-IOV service, Physical Function device driver needs to call: + int pci_iov_register(struct pci_dev *dev, + int (*callback)(struct pci_dev *, u32)) + The 'callback' is a callback function that the SR-IOV code will invoke + it when events related to VFs happen (e.g. user enable/disable VFs). + The first argument is PF itself, the second argument is event type and + value. For now, following events type are supported: + - PCI_IOV_ENABLE: SR-IOV enable request + - PCI_IOV_DISABLE: SR-IOV disable request + - PCI_IOV_NUMVFS: changing Number of VFs request + And event values can be extract using following masks: + - PCI_IOV_NUM_VIRTFN: Number of Virtual Functions + +To unregister SR-IOV service, Physical Function device driver needs to call: + void pci_iov_unregister(struct pci_dev *dev) + +To enable SR-IOV, Physical Function device driver needs to call: + int pci_iov_enable(struct pci_dev *dev, int numvfs) + 'numvfs' is the number of VFs that PF wants to enable. + +To disable SR-IOV, Physical Function device driver needs to call: + void pci_iov_disable(struct pci_dev *dev) + +Note: above two functions sleeps 1 second waiting on hardware transaction +completion according to SR-IOV specification. + +3.2 Usage example + +Following piece of code illustrates the usage of APIs above. + +static int callback(struct pci_dev *dev, u32 event) +{ + int numvfs; + + if (event & PCI_IOV_ENABLE) { + /* + * request to enable SR-IOV. + * Note: if the PF driver want to support PM, it has + * to check the device power state here to see if this + * request is allowed or not. + */ + ... + + } else if (event & PCI_IOV_DISABLE) { + /* + * request to disable SR-IOV. + */ + ... + + } else if (event & PCI_IOV_NUMVFS) { + /* + * request to change NumVFs. + */ + numvfs = event & PCI_IOV_NUM_VIRTFN; + ... + + } else + return -EINVAL; + + return 0; +} + +static int __devinit dev_probe(struct pci_dev *dev, + const struct pci_device_id *id) +{ + int err; + int numvfs; + + ... + err = pci_iov_register(dev, callback); + ... + err = pci_iov_enable(dev, numvfs); + ... + + return err; +} + +static void __devexit dev_remove(struct pci_dev *dev) +{ + ... + pci_iov_disable(dev); + ... + pci_iov_unregister(dev); + ... +} + +#ifdef CONFIG_PM +/* + * If Physical Function supports the power management, then the + * SR-IOV needs to be disabled before the adapter goes to sleep, + * because Virtual Functions will not work when the adapter is in + * the power-saving mode. + * The SR-IOV can be enabled again after the adapter wakes up. + */ +static int dev_suspend(struct pci_dev *dev, pm_message_t state) +{ + ... + pci_iov_disable(dev); + ... + + return 0; +} + +static int dev_resume(struct pci_dev *dev) +{ + int err; + int numvfs; + + ... + rc = pci_iov_enable(dev, numvfs); + ... + + return 0; +} +#endif + +static struct pci_driver dev_driver = { + .name = "SR-IOV Physical Function driver", + .id_table = dev_id_table, + .probe = dev_probe, + .remove = __devexit_p(dev_remove), +#ifdef CONFIG_PM + .suspend = dev_suspend, + .resume = dev_resume, +#endif +};
Signature · 1 line
1.5.6.4
PATCH v5 15/15 PCI: document the new PCI boot parameters
In reply to Yu ZhaoComplete document for the new PCI boot parameters.
--- Documentation/kernel-parameters.txt | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
Documentation/kernel-parameters.txt +10 -0
@@ -1677,6 +1677,16 @@ and is between 256 and 4096 characters. It is defined in the file cbmemsize=nn[KMG] The fixed amount of bus space which is reserved for the CardBus bridge's memory window. The default value is 64 megabytes. + assign-mmio=[dddd:]bb [X86] reassign memory resources of all + devices under bus [dddd:]bb (dddd is the domain + number and bb is the bus number). + assign-pio=[dddd:]bb [X86] reassign io port resources of all + devices under bus [dddd:]bb (dddd is the domain + number and bb is the bus number). + align-mmio=[dddd:]bb:dd.f [X86] relocate memory resources of a + device with minimum PAGE_SIZE alignment (dddd + is the domain number and bb, dd and f is the + bus, device and function number). pcie_aspm= [PCIE] Forcibly enable or disable PCIe Active State Power Management.
Signature · 1 line
1.5.6.4
PATCH v5 15/15 PCI: document the new PCI boot parameters
In reply to Yu ZhaoComplete document for the new PCI boot parameters.
--- Documentation/kernel-parameters.txt | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
Documentation/kernel-parameters.txt +10 -0
@@ -1677,6 +1677,16 @@ and is between 256 and 4096 characters. It is defined in the file cbmemsize=nn[KMG] The fixed amount of bus space which is reserved for the CardBus bridge's memory window. The default value is 64 megabytes. + assign-mmio=[dddd:]bb [X86] reassign memory resources of all + devices under bus [dddd:]bb (dddd is the domain + number and bb is the bus number). + assign-pio=[dddd:]bb [X86] reassign io port resources of all + devices under bus [dddd:]bb (dddd is the domain + number and bb is the bus number). + align-mmio=[dddd:]bb:dd.f [X86] relocate memory resources of a + device with minimum PAGE_SIZE alignment (dddd + is the domain number and bb, dd and f is the + bus, device and function number). pcie_aspm= [PCIE] Forcibly enable or disable PCIe Active State Power Management.
Signature · 1 line
1.5.6.4