Skip to contentSkip to conversation

take 2: PCIe IO space multiplexing: Linux part: pci io space sharing and its driver

PATCH New Isaku Yamahata 1 participant 1 message Apr 17, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 2/5 take 2: PCIe IO space multiplexing: Linux part: pci io space sharing and its driver

PCI pass through: PCIe IO space multiplexing

This patch is for PCIe IO space multiplexing. This is required for more than 16 HVM domain to boot from PCIe pass through device.

Linux as dom0 exclusively assigns IO space to downstream PCI bridges and the assignment unit of PCI bridge IO space is 4K. So the only up to 16 PCIe device can be accessed via IO space within 64K IO ports. PCI expansion ROM BIOS often uses IO port access to boot from the device, so on virtualized environment, it means only up to 16 guest domain can boot from pass-through device.

This patch allows PCIe IO space sharing of pass-through device.
- reassign IO space of PCIe devices specified by "guestiomuldev=[<segment>:]<bus>:<dev>[,[<segment:><bus>:dev]][,...]" to be shared.
This is implemented as Linux PCI quirk fixup.

The sharing unit is PCIe switch. Ie IO space of the end point devices under the same switch will be shared.
If there are more than one switches, two areas of IO space will be used.

- And the driver which arbitrates the accesses to the multiplexed PCIe IO space. Later qemu-dm will use this.

Limitation:
IO port of IO shared devices can't be accessed from dom0 Linux device driver. But this wouldn't be a big issue because PCIe specification discourages the use of IO space and recommends that IO space should be used only for bootable device with ROM code. OS device driver should work without IO space access.

Changes
* Fri Apr 17 16:13:33 JST 2009
Add capability check.

* Tue Apr 14 16:48:21 JST 2009
Update kernel-parameters.txt

* Mon Apr 13 17:57:01 JST 2009
Supported PCI bridge hotplugging
many clean ups.

* Fri Apr 10 12:44:43 JST 2009
PCI Hotplug support and various clean ups.

Documentation/kernel-parameters.txt +6 -0
@@ -583,6 +583,12 @@ running once the system is up.
 			If PCI-PCI bridge is specified, all PCI devices
 			behind PCI-PCI bridge are reserved.
 
+	guestiomuldev=	[PCI]
+			Format: [sbd][,<sbd>][,...]
+			Format of sbdf: [<segment>:]<bus>:<dev>
+			Note: function shouldn't be specified.
+			Specifies PCI device for IO port multiplexing driver.
+
 	gus=		[HW,OSS]
 			Format: <io>,<irq>,<dma>,<dma16>
 
drivers/pci/Kconfig +6 -0
@@ -37,6 +37,12 @@ config PCI_GUESTDEV
 	help
 	  Say Y here if you want to reserve PCI device for passthrough.
 
+config PCI_IOMULTI
+	bool "PCI Device IO Multiplex for Passthrough"
+	depends on PCI && ACPI
+	default y
+	help
+	  Say Y here if you need io multiplexing.
 config PCI_IOV
 	bool "PCI IOV support"
 	depends on PCI
drivers/pci/Makefile +1 -0
@@ -6,6 +6,7 @@ obj-y		+= access.o bus.o probe.o remove.
 			pci-driver.o search.o pci-sysfs.o rom.o setup-res.o
 obj-$(CONFIG_PROC_FS) += proc.o
 obj-$(CONFIG_PCI_GUESTDEV) += guestdev.o
+obj-$(CONFIG_PCI_IOMULTI) += iomulti.o
 
 # Build PCI Express stuff if needed
 obj-$(CONFIG_PCIEPORTBUS) += pcie/
drivers/pci/iomulti.c new file +1412 -0

1,412 lines not on this page · raw message

drivers/pci/iomulti.h new file +51 -0

51 lines not on this page · raw message

PCI pass through: PCIe IO space multiplexing

This patch is for PCIe IO space multiplexing. This is required for more than 16 HVM domain to boot from PCIe pass through device.

Linux as dom0 exclusively assigns IO space to downstream PCI bridges and the assignment unit of PCI bridge IO space is 4K. So the only up to 16 PCIe device can be accessed via IO space within 64K IO ports. PCI expansion ROM BIOS often uses IO port access to boot from the device, so on virtualized environment, it means only up to 16 guest domain can boot from pass-through device.

This patch allows PCIe IO space sharing of pass-through device.
- reassign IO space of PCIe devices specified by "guestiomuldev=[<segment>:]<bus>:<dev>[,[<segment:><bus>:dev]][,...]" to be shared.
This is implemented as Linux PCI quirk fixup.

The sharing unit is PCIe switch. Ie IO space of the end point devices under the same switch will be shared.
If there are more than one switches, two areas of IO space will be used.

- And the driver which arbitrates the accesses to the multiplexed PCIe IO space. Later qemu-dm will use this.

Limitation:
IO port of IO shared devices can't be accessed from dom0 Linux device driver. But this wouldn't be a big issue because PCIe specification discourages the use of IO space and recommends that IO space should be used only for bootable device with ROM code. OS device driver should work without IO space access.

Changes
* Fri Apr 17 16:13:33 JST 2009
Add capability check.

* Tue Apr 14 16:48:21 JST 2009
Update kernel-parameters.txt

* Mon Apr 13 17:57:01 JST 2009
Supported PCI bridge hotplugging
many clean ups.

* Fri Apr 10 12:44:43 JST 2009
PCI Hotplug support and various clean ups.

Documentation/kernel-parameters.txt +6 -0
@@ -583,6 +583,12 @@ running once the system is up.
 			If PCI-PCI bridge is specified, all PCI devices
 			behind PCI-PCI bridge are reserved.
 
+	guestiomuldev=	[PCI]
+			Format: [sbd][,<sbd>][,...]
+			Format of sbdf: [<segment>:]<bus>:<dev>
+			Note: function shouldn't be specified.
+			Specifies PCI device for IO port multiplexing driver.
+
 	gus=		[HW,OSS]
 			Format: <io>,<irq>,<dma>,<dma16>
 
drivers/pci/Kconfig +6 -0
@@ -37,6 +37,12 @@ config PCI_GUESTDEV
 	help
 	  Say Y here if you want to reserve PCI device for passthrough.
 
+config PCI_IOMULTI
+	bool "PCI Device IO Multiplex for Passthrough"
+	depends on PCI && ACPI
+	default y
+	help
+	  Say Y here if you need io multiplexing.
 config PCI_IOV
 	bool "PCI IOV support"
 	depends on PCI
drivers/pci/Makefile +1 -0
@@ -6,6 +6,7 @@ obj-y		+= access.o bus.o probe.o remove.
 			pci-driver.o search.o pci-sysfs.o rom.o setup-res.o
 obj-$(CONFIG_PROC_FS) += proc.o
 obj-$(CONFIG_PCI_GUESTDEV) += guestdev.o
+obj-$(CONFIG_PCI_IOMULTI) += iomulti.o
 
 # Build PCI Express stuff if needed
 obj-$(CONFIG_PCIEPORTBUS) += pcie/
drivers/pci/iomulti.c new file +1412 -0

1,412 lines not on this page · raw message

drivers/pci/iomulti.h new file +51 -0

51 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel