Skip to contentSkip to conversation

pciback support

PATCH In review Konrad Rzeszutek Wilk 5 participants 25 messages Oct 13, 2009 – Oct 20, 2009

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 pciback support

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

The usage of this is, as said, is similar to pci-stub:
lspci | grep SCSI
01:14.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/unbind echo "0000:01:14.0" > /sys/bus/pci/drivers/pciback/new-slot echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/bind

and add this entry:

pci = [ '0000:01.14.0' ]

in your .xm file.

The PV guest, if it has the PCI frontend, should now see the PCI device. I've tested this succesfully with a SLES10 PV guest with a couple of devices.

But please be beware of this warning if it shows up:
(XEN) irq.c:1113:d1 Cannot bind IRQ 17 to guest. Others do not share.

On my machine it lead to Dom0 deciding that a spurrious interrupt kicked off and it disabled the IRQ. The end result was that other devices on the same interrupt line stopped working. I am not yet certain how to make this work properly (whether to check if the PCI device in question interrupt line is being shared beforehand by xm?, or do something in Xen?).

#1 lore raw

PATCH 1/13 Initial copy from linux-2.6.18.hg off pciback driver.

---
 drivers/xen/pciback/Makefile                    |   17 +
 drivers/xen/pciback/conf_space.c                |  435 ++++++++
 drivers/xen/pciback/conf_space.h                |  126 +++
 drivers/xen/pciback/conf_space_capability.c     |   69 ++
 drivers/xen/pciback/conf_space_capability.h     |   23 +
 drivers/xen/pciback/conf_space_capability_msi.c |   79 ++
 drivers/xen/pciback/conf_space_capability_pm.c  |  126 +++
 drivers/xen/pciback/conf_space_capability_vpd.c |   40 +
 drivers/xen/pciback/conf_space_header.c         |  317 ++++++
 drivers/xen/pciback/conf_space_quirks.c         |  138 +++
 drivers/xen/pciback/conf_space_quirks.h         |   35 +
 drivers/xen/pciback/controller.c                |  443 ++++++++
 drivers/xen/pciback/passthrough.c               |  176 +++
 drivers/xen/pciback/pci_stub.c                  | 1316 +++++++++++++++++++++++
 drivers/xen/pciback/pciback.h                   |  126 +++
 drivers/xen/pciback/pciback_ops.c               |  134 +++
 drivers/xen/pciback/slot.c                      |  187 ++++
 drivers/xen/pciback/vpci.c                      |  242 +++++
 drivers/xen/pciback/xenbus.c                    |  710 ++++++++++++
 19 files changed, 4739 insertions(+), 0 deletions(-)
 create mode 100644 drivers/xen/pciback/Makefile
 create mode 100644 drivers/xen/pciback/conf_space.c
 create mode 100644 drivers/xen/pciback/conf_space.h
 create mode 100644 drivers/xen/pciback/conf_space_capability.c
 create mode 100644 drivers/xen/pciback/conf_space_capability.h
 create mode 100644 drivers/xen/pciback/conf_space_capability_msi.c
 create mode 100644 drivers/xen/pciback/conf_space_capability_pm.c
 create mode 100644 drivers/xen/pciback/conf_space_capability_vpd.c
 create mode 100644 drivers/xen/pciback/conf_space_header.c
 create mode 100644 drivers/xen/pciback/conf_space_quirks.c
 create mode 100644 drivers/xen/pciback/conf_space_quirks.h
 create mode 100644 drivers/xen/pciback/controller.c
 create mode 100644 drivers/xen/pciback/passthrough.c
 create mode 100644 drivers/xen/pciback/pci_stub.c
 create mode 100644 drivers/xen/pciback/pciback.h
 create mode 100644 drivers/xen/pciback/pciback_ops.c
 create mode 100644 drivers/xen/pciback/slot.c
 create mode 100644 drivers/xen/pciback/vpci.c
 create mode 100644 drivers/xen/pciback/xenbus.c
drivers/xen/pciback/Makefile new file +17 -0
new file mode 100644
index 0000000..106dae7
@@ -0,0 +1,17 @@
+obj-$(CONFIG_XEN_PCIDEV_BACKEND) += pciback.o
+
+pciback-y := pci_stub.o pciback_ops.o xenbus.o
+pciback-y += conf_space.o conf_space_header.o \
+	     conf_space_capability.o \
+	     conf_space_capability_vpd.o \
+	     conf_space_capability_pm.o \
+             conf_space_quirks.o
+pciback-$(CONFIG_PCI_MSI) += conf_space_capability_msi.o
+pciback-$(CONFIG_XEN_PCIDEV_BACKEND_VPCI) += vpci.o
+pciback-$(CONFIG_XEN_PCIDEV_BACKEND_SLOT) += slot.o
+pciback-$(CONFIG_XEN_PCIDEV_BACKEND_PASS) += passthrough.o
+pciback-$(CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER) += controller.o
+
+ifeq ($(CONFIG_XEN_PCIDEV_BE_DEBUG),y)
+EXTRA_CFLAGS += -DDEBUG
+endif
drivers/xen/pciback/conf_space.c new file +435 -0

435 lines not on this page · raw message

drivers/xen/pciback/conf_space.h new file +126 -0

126 lines not on this page · raw message

drivers/xen/pciback/conf_space_capability.c new file +69 -0

69 lines not on this page · raw message

drivers/xen/pciback/conf_space_capability.h new file +23 -0
new file mode 100644
index 0000000..823392e
@@ -0,0 +1,23 @@
+/*
+ * PCI Backend - Data structures for special overlays for structures on
+ *               the capability list.
+ *
+ * Author: Ryan Wilson <[email protected]>
+ */
+
+#ifndef __PCIBACK_CONFIG_CAPABILITY_H__
+#define __PCIBACK_CONFIG_CAPABILITY_H__
+
+#include <linux/pci.h>
+#include <linux/list.h>
+
+struct pciback_config_capability {
+	struct list_head cap_list;
+
+	int capability;
+
+	/* If the device has the capability found above, add these fields */
+	const struct config_field *fields;
+};
+
+#endif
drivers/xen/pciback/conf_space_capability_msi.c new file +79 -0

79 lines not on this page · raw message

drivers/xen/pciback/conf_space_capability_pm.c new file +126 -0

126 lines not on this page · raw message

drivers/xen/pciback/conf_space_capability_vpd.c new file +40 -0
new file mode 100644
index 0000000..920cb4a
@@ -0,0 +1,40 @@
+/*
+ * PCI Backend - Configuration space overlay for Vital Product Data
+ *
+ * Author: Ryan Wilson <[email protected]>
+ */
+
+#include <linux/pci.h>
+#include "conf_space.h"
+#include "conf_space_capability.h"
+
+static int vpd_address_write(struct pci_dev *dev, int offset, u16 value,
+			     void *data)
+{
+	/* Disallow writes to the vital product data */
+	if (value & PCI_VPD_ADDR_F)
+		return PCIBIOS_SET_FAILED;
+	else
+		return pci_write_config_word(dev, offset, value);
+}
+
+static const struct config_field caplist_vpd[] = {
+	{
+	 .offset    = PCI_VPD_ADDR,
+	 .size      = 2,
+	 .u.w.read  = pciback_read_config_word,
+	 .u.w.write = vpd_address_write,
+	 },
+	{
+	 .offset     = PCI_VPD_DATA,
+	 .size       = 4,
+	 .u.dw.read  = pciback_read_config_dword,
+	 .u.dw.write = NULL,
+	 },
+	{}
+};
+ 
+struct pciback_config_capability pciback_config_capability_vpd = {
+	.capability = PCI_CAP_ID_VPD,
+	.fields = caplist_vpd,
+};
drivers/xen/pciback/conf_space_header.c new file +317 -0

317 lines not on this page · raw message

drivers/xen/pciback/conf_space_quirks.c new file +138 -0

138 lines not on this page · raw message

drivers/xen/pciback/conf_space_quirks.h new file +35 -0
new file mode 100644
index 0000000..acd0e1a
@@ -0,0 +1,35 @@
+/*
+ * PCI Backend - Data structures for special overlays for broken devices.
+ *
+ * Ryan Wilson <[email protected]>
+ * Chris Bookholt <[email protected]>
+ */
+
+#ifndef __XEN_PCIBACK_CONF_SPACE_QUIRKS_H__
+#define __XEN_PCIBACK_CONF_SPACE_QUIRKS_H__
+
+#include <linux/pci.h>
+#include <linux/list.h>
+
+struct pciback_config_quirk {
+	struct list_head quirks_list;
+	struct pci_device_id devid;
+	struct pci_dev *pdev;
+};
+
+struct pciback_config_quirk *pciback_find_quirk(struct pci_dev *dev);
+
+int pciback_config_quirks_add_field(struct pci_dev *dev, struct config_field
+				    *field);
+
+int pciback_config_quirks_remove_field(struct pci_dev *dev, int reg);
+
+int pciback_config_quirks_init(struct pci_dev *dev);
+
+void pciback_config_field_free(struct config_field *field);
+
+int pciback_config_quirk_release(struct pci_dev *dev);
+
+int pciback_field_is_dup(struct pci_dev *dev, unsigned int reg);
+
+#endif
drivers/xen/pciback/controller.c new file +443 -0

443 lines not on this page · raw message

drivers/xen/pciback/passthrough.c new file +176 -0

176 lines not on this page · raw message

drivers/xen/pciback/pci_stub.c new file +1316 -0

976 lines not on this page · raw message

drivers/xen/pciback/pciback.h new file +126 -0

126 lines not on this page · raw message

drivers/xen/pciback/pciback_ops.c new file +134 -0

134 lines not on this page · raw message

drivers/xen/pciback/slot.c new file +187 -0

187 lines not on this page · raw message

drivers/xen/pciback/vpci.c new file +242 -0

242 lines not on this page · raw message

drivers/xen/pciback/xenbus.c new file +710 -0

710 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.6.2.5

Part of the diff is not shown here. View the raw message.

#2 lore raw

PATCH 2/13 Including the pciif.h header file.

---
 include/xen/interface/io/pciif.h |  124 ++++++++++++++++++++++++++++++++++++++
 1 files changed, 124 insertions(+), 0 deletions(-)
 create mode 100644 include/xen/interface/io/pciif.h
include/xen/interface/io/pciif.h new file +124 -0
new file mode 100644
index 0000000..c4177f3
@@ -0,0 +1,124 @@
+/*
+ * PCI Backend/Frontend Common Data Structures & Macros
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ *   Author: Ryan Wilson <[email protected]>
+ */
+#ifndef __XEN_PCI_COMMON_H__
+#define __XEN_PCI_COMMON_H__
+
+/* Be sure to bump this number if you change this file */
+#define XEN_PCI_MAGIC "7"
+
+/* xen_pci_sharedinfo flags */
+#define _XEN_PCIF_active     (0)
+#define XEN_PCIF_active      (1<<_XEN_PCIF_active)
+#define _XEN_PCIB_AERHANDLER (1)
+#define XEN_PCIB_AERHANDLER  (1<<_XEN_PCIB_AERHANDLER)
+#define _XEN_PCIB_active     (2)
+#define XEN_PCIB_active      (1<<_XEN_PCIB_active)
+
+/* xen_pci_op commands */
+#define XEN_PCI_OP_conf_read    	(0)
+#define XEN_PCI_OP_conf_write   	(1)
+#define XEN_PCI_OP_enable_msi   	(2)
+#define XEN_PCI_OP_disable_msi  	(3)
+#define XEN_PCI_OP_enable_msix  	(4)
+#define XEN_PCI_OP_disable_msix 	(5)
+#define XEN_PCI_OP_aer_detected 	(6)
+#define XEN_PCI_OP_aer_resume		(7)
+#define XEN_PCI_OP_aer_mmio		(8)
+#define XEN_PCI_OP_aer_slotreset	(9)
+
+/* xen_pci_op error numbers */
+#define XEN_PCI_ERR_success          (0)
+#define XEN_PCI_ERR_dev_not_found   (-1)
+#define XEN_PCI_ERR_invalid_offset  (-2)
+#define XEN_PCI_ERR_access_denied   (-3)
+#define XEN_PCI_ERR_not_implemented (-4)
+/* XEN_PCI_ERR_op_failed - backend failed to complete the operation */
+#define XEN_PCI_ERR_op_failed       (-5)
+
+/*
+ * it should be PAGE_SIZE-sizeof(struct xen_pci_op))/sizeof(struct msix_entry))
+ * Should not exceed 128
+ */
+#define SH_INFO_MAX_VEC     128
+
+struct xen_msix_entry {
+    uint16_t vector;
+    uint16_t entry;
+};
+struct xen_pci_op {
+    /* IN: what action to perform: XEN_PCI_OP_* */
+    uint32_t cmd;
+
+    /* OUT: will contain an error number (if any) from errno.h */
+    int32_t err;
+
+    /* IN: which device to touch */
+    uint32_t domain; /* PCI Domain/Segment */
+    uint32_t bus;
+    uint32_t devfn;
+
+    /* IN: which configuration registers to touch */
+    int32_t offset;
+    int32_t size;
+
+    /* IN/OUT: Contains the result after a READ or the value to WRITE */
+    uint32_t value;
+    /* IN: Contains extra infor for this operation */
+    uint32_t info;
+    /*IN:  param for msi-x */
+    struct xen_msix_entry msix_entries[SH_INFO_MAX_VEC];
+};
+
+/*used for pcie aer handling*/
+struct xen_pcie_aer_op
+{
+
+    /* IN: what action to perform: XEN_PCI_OP_* */
+    uint32_t cmd;
+    /*IN/OUT: return aer_op result or carry error_detected state as input*/
+    int32_t err;
+
+    /* IN: which device to touch */
+    uint32_t domain; /* PCI Domain/Segment*/
+    uint32_t bus;
+    uint32_t devfn;
+};
+struct xen_pci_sharedinfo {
+    /* flags - XEN_PCIF_* */
+    uint32_t flags;
+    struct xen_pci_op op;
+    struct xen_pcie_aer_op aer_op;
+};
+
+#endif /* __XEN_PCI_COMMON_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-set-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
Signature · 1 line
1.6.2.5
#3 lore raw

PATCH 3/13 Fix include header name change (evtchn.h is now events.h)

---
 drivers/xen/pciback/pci_stub.c    |    2 +-
 drivers/xen/pciback/pciback_ops.c |    2 +-
 drivers/xen/pciback/xenbus.c      |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/xen/pciback/pci_stub.c +1 -1
index c481a73..c02f21f 100644
@@ -13,7 +13,7 @@
 #include <linux/pci.h>
 #include <linux/wait.h>
 #include <asm/atomic.h>
-#include <xen/evtchn.h>
+#include <xen/events.h>
 #include "pciback.h"
 #include "conf_space.h"
 #include "conf_space_quirks.h"
drivers/xen/pciback/pciback_ops.c +1 -1
index b85b2db..58d09eb 100644
@@ -6,7 +6,7 @@
 #include <linux/module.h>
 #include <linux/wait.h>
 #include <asm/bitops.h>
-#include <xen/evtchn.h>
+#include <xen/events.h>
 #include "pciback.h"
 
 int verbose_request = 0;
drivers/xen/pciback/xenbus.c +1 -1
index 4d56c45..bbca3fe 100644
@@ -8,7 +8,7 @@
 #include <linux/list.h>
 #include <linux/vmalloc.h>
 #include <xen/xenbus.h>
-#include <xen/evtchn.h>
+#include <xen/events.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
Signature · 1 line
1.6.2.5
#4 lore raw

PATCH 4/13 Removed MSI capability in the pci-back driver.

Currently the tree does not have the Xen MSI capability. Will re-introduce when the capability exists.

---
 drivers/xen/pciback/Makefile                    |    1 -
 drivers/xen/pciback/conf_space_capability_msi.c |   79 -----------------------
 drivers/xen/pciback/pci_stub.c                  |   18 -----
 drivers/xen/pciback/pciback.h                   |   14 ----
 drivers/xen/pciback/pciback_ops.c               |   14 ----
 5 files changed, 0 insertions(+), 126 deletions(-)
 delete mode 100644 drivers/xen/pciback/conf_space_capability_msi.c
drivers/xen/pciback/Makefile +0 -1
index 106dae7..a99bdaa 100644
@@ -6,7 +6,6 @@ pciback-y += conf_space.o conf_space_header.o \
 	     conf_space_capability_vpd.o \
 	     conf_space_capability_pm.o \
              conf_space_quirks.o
-pciback-$(CONFIG_PCI_MSI) += conf_space_capability_msi.o
 pciback-$(CONFIG_XEN_PCIDEV_BACKEND_VPCI) += vpci.o
 pciback-$(CONFIG_XEN_PCIDEV_BACKEND_SLOT) += slot.o
 pciback-$(CONFIG_XEN_PCIDEV_BACKEND_PASS) += passthrough.o
drivers/xen/pciback/conf_space_capability_msi.c deleted +0 -79

79 lines not on this page · raw message

drivers/xen/pciback/pci_stub.c +0 -18
index c02f21f..036d7bb 100644
@@ -1173,21 +1173,6 @@ static ssize_t permissive_show(struct device_driver *drv, char *buf)
 
 DRIVER_ATTR(permissive, S_IRUSR | S_IWUSR, permissive_show, permissive_add);
 
-#ifdef CONFIG_PCI_MSI
-
-int pciback_get_owner(struct pci_dev *dev)
-{
-	struct pcistub_device *psdev;
-
-	psdev = pcistub_device_find(pci_domain_nr(dev->bus), dev->bus->number,
-			PCI_SLOT(dev->devfn), PCI_FUNC(dev->devfn));
-
-	if (!psdev || !psdev->pdev)
-		return -1;
-
-	return psdev->pdev->xdev->otherend_id;
-}
-#endif
 
 static void pcistub_exit(void)
 {
@@ -1199,7 +1184,6 @@ static void pcistub_exit(void)
 	driver_remove_file(&pciback_pci_driver.driver, &driver_attr_permissive);
 
 	pci_unregister_driver(&pciback_pci_driver);
-	WARN_ON(unregister_msi_get_owner(pciback_get_owner));
 }
 
 static int __init pcistub_init(void)
@@ -1257,8 +1241,6 @@ static int __init pcistub_init(void)
 		err = driver_create_file(&pciback_pci_driver.driver,
 					 &driver_attr_permissive);
 
-	if (!err)
-		err = register_msi_get_owner(pciback_get_owner);
 	if (err)
 		pcistub_exit();
 
drivers/xen/pciback/pciback.h +0 -14
index 6744f45..27d0b2f 100644
@@ -105,20 +105,6 @@ void pciback_do_op(void *data);
 int pciback_xenbus_register(void);
 void pciback_xenbus_unregister(void);
 
-#ifdef CONFIG_PCI_MSI
-int pciback_enable_msi(struct pciback_device *pdev,
-                       struct pci_dev *dev, struct xen_pci_op *op);
-
-int pciback_disable_msi(struct pciback_device *pdev,
-                         struct pci_dev *dev, struct xen_pci_op *op);
-
-
-int pciback_enable_msix(struct pciback_device *pdev,
-                        struct pci_dev *dev, struct xen_pci_op *op);
-
-int pciback_disable_msix(struct pciback_device *pdev,
-                        struct pci_dev *dev, struct xen_pci_op *op);
-#endif
 extern int verbose_request;
 
 void test_and_schedule_op(struct pciback_device *pdev);
drivers/xen/pciback/pciback_ops.c +0 -14
index 58d09eb..a937564 100644
@@ -89,20 +89,6 @@ void pciback_do_op(void *data)
 				op->err = pciback_config_write(dev,
 					  op->offset, op->size,	op->value);
 				break;
-#ifdef CONFIG_PCI_MSI
-			case XEN_PCI_OP_enable_msi:
-				op->err = pciback_enable_msi(pdev, dev, op);
-				break;
-			case XEN_PCI_OP_disable_msi:
-				op->err = pciback_disable_msi(pdev, dev, op);
-				break;
-			case XEN_PCI_OP_enable_msix:
-				op->err = pciback_enable_msix(pdev, dev, op);
-				break;
-			case XEN_PCI_OP_disable_msix:
-				op->err = pciback_disable_msix(pdev, dev, op);
-				break;
-#endif
 			default:
 				op->err = XEN_PCI_ERR_not_implemented;
 				break;
Signature · 1 line
1.6.2.5
#5 lore raw

PATCH 5/13 Use pci_is_enabled() instead of is_enabled.

---
 drivers/xen/pciback/conf_space_header.c |    4 ++--
 drivers/xen/pciback/pciback_ops.c       |    1 -
 2 files changed, 2 insertions(+), 3 deletions(-)
drivers/xen/pciback/conf_space_header.c +2 -2
index f794e12..5a9e028 100644
@@ -22,14 +22,14 @@ static int command_write(struct pci_dev *dev, int offset, u16 value, void *data)
 {
 	int err;
 
-	if (!dev->is_enabled && is_enable_cmd(value)) {
+	if (!pci_is_enabled(dev) && is_enable_cmd(value)) {
 		if (unlikely(verbose_request))
 			printk(KERN_DEBUG "pciback: %s: enable\n",
 			       pci_name(dev));
 		err = pci_enable_device(dev);
 		if (err)
 			return err;
-	} else if (dev->is_enabled && !is_enable_cmd(value)) {
+	} else if (pci_is_enabled(dev) && !is_enable_cmd(value)) {
 		if (unlikely(verbose_request))
 			printk(KERN_DEBUG "pciback: %s: disable\n",
 			       pci_name(dev));
drivers/xen/pciback/pciback_ops.c +0 -1
index a937564..5bc68b2 100644
@@ -26,7 +26,6 @@ void pciback_reset_device(struct pci_dev *dev)
 
 		pci_write_config_word(dev, PCI_COMMAND, 0);
 
-		dev->is_enabled = 0;
 		dev->is_busmaster = 0;
 	} else {
 		pci_read_config_word(dev, PCI_COMMAND, &cmd);
Signature · 1 line
1.6.2.5
#6 lore raw

PATCH 6/13 Fix usage of INIT_WORK.

---
 drivers/xen/pciback/pciback.h     |    4 ++--
 drivers/xen/pciback/pciback_ops.c |    7 ++++---
 drivers/xen/pciback/xenbus.c      |    3 ++-
 3 files changed, 8 insertions(+), 6 deletions(-)
drivers/xen/pciback/pciback.h +2 -2
index 27d0b2f..e8cfe74 100644
@@ -99,8 +99,8 @@ int pciback_publish_pci_roots(struct pciback_device *pdev,
 void pciback_release_devices(struct pciback_device *pdev);
 
 /* Handles events from front-end */
-irqreturn_t pciback_handle_event(int irq, void *dev_id, struct pt_regs *regs);
-void pciback_do_op(void *data);
+irqreturn_t pciback_handle_event(int irq, void *dev_id);
+void pciback_do_op(struct work_struct *data);
 
 int pciback_xenbus_register(void);
 void pciback_xenbus_unregister(void);
drivers/xen/pciback/pciback_ops.c +4 -3
index 5bc68b2..834663d 100644
@@ -66,9 +66,10 @@ void test_and_schedule_op(struct pciback_device *pdev)
  * context because some of the pci_* functions can sleep (mostly due to ACPI
  * use of semaphores). This function is intended to be called from a work
  * queue in process context taking a struct pciback_device as a parameter */
-void pciback_do_op(void *data)
+
+void pciback_do_op(struct work_struct *data)
 {
-	struct pciback_device *pdev = data;
+	struct pciback_device *pdev = container_of(data, struct pciback_device, op_work);
 	struct pci_dev *dev;
 	struct xen_pci_op *op = &pdev->sh_info->op;
 
@@ -109,7 +110,7 @@ void pciback_do_op(void *data)
 	test_and_schedule_op(pdev);
 }
 
-irqreturn_t pciback_handle_event(int irq, void *dev_id, struct pt_regs *regs)
+irqreturn_t pciback_handle_event(int irq, void *dev_id)
 {
 	struct pciback_device *pdev = dev_id;
 
drivers/xen/pciback/xenbus.c +2 -1
index bbca3fe..bd52289 100644
@@ -9,6 +9,7 @@
 #include <linux/vmalloc.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
+#include <linux/workqueue.h>
 #include "pciback.h"
 
 #define INVALID_EVTCHN_IRQ  (-1)
@@ -33,7 +34,7 @@ static struct pciback_device *alloc_pdev(struct xenbus_device *xdev)
 	pdev->evtchn_irq = INVALID_EVTCHN_IRQ;
 	pdev->be_watching = 0;
 
-	INIT_WORK(&pdev->op_work, pciback_do_op, pdev);
+	INIT_WORK(&pdev->op_work, pciback_do_op);
 
 	if (pciback_init_devices(pdev)) {
 		kfree(pdev);
Signature · 1 line
1.6.2.5
#7 lore raw

PATCH 7/13 Update the calling mechanism for xenbus_[map|unmap]_ring_valloc functions.

---
 drivers/xen/pciback/pciback.h |    1 -
 drivers/xen/pciback/xenbus.c  |   18 +++++++++---------
 2 files changed, 9 insertions(+), 10 deletions(-)
drivers/xen/pciback/pciback.h +0 -1
index e8cfe74..421b81e 100644
@@ -36,7 +36,6 @@ struct pciback_device {
 
 	int evtchn_irq;
 
-	struct vm_struct *sh_area;
 	struct xen_pci_sharedinfo *sh_info;
 
 	unsigned long flags;
drivers/xen/pciback/xenbus.c +9 -9
index bd52289..5be1350 100644
@@ -7,6 +7,7 @@
 #include <linux/init.h>
 #include <linux/list.h>
 #include <linux/vmalloc.h>
+#include <linux/workqueue.h>
 #include <xen/xenbus.h>
 #include <xen/events.h>
 #include <linux/workqueue.h>
@@ -29,7 +30,6 @@ static struct pciback_device *alloc_pdev(struct xenbus_device *xdev)
 
 	spin_lock_init(&pdev->dev_lock);
 
-	pdev->sh_area = NULL;
 	pdev->sh_info = NULL;
 	pdev->evtchn_irq = INVALID_EVTCHN_IRQ;
 	pdev->be_watching = 0;
@@ -59,7 +59,7 @@ static void pciback_disconnect(struct pciback_device *pdev)
 	flush_workqueue(pciback_wq);
 
 	if (pdev->sh_info != NULL) {
-		xenbus_unmap_ring_vfree(pdev->xdev, pdev->sh_area);
+		xenbus_unmap_ring_vfree(pdev->xdev, pdev->sh_info);
 		pdev->sh_info = NULL;
 	}
 
@@ -85,23 +85,23 @@ static int pciback_do_attach(struct pciback_device *pdev, int gnt_ref,
 			     int remote_evtchn)
 {
 	int err = 0;
-	struct vm_struct *area;
+	void *vaddr;
 
 	dev_dbg(&pdev->xdev->dev,
 		"Attaching to frontend resources - gnt_ref=%d evtchn=%d\n",
 		gnt_ref, remote_evtchn);
 
-	area = xenbus_map_ring_valloc(pdev->xdev, gnt_ref);
-	if (IS_ERR(area)) {
-		err = PTR_ERR(area);
+	err = xenbus_map_ring_valloc(pdev->xdev, gnt_ref, &vaddr);
+	if (err < 0) {
+		xenbus_dev_fatal(pdev->xdev, err,
+				"Error mapping other domain page in ours.");
 		goto out;
 	}
-	pdev->sh_area = area;
-	pdev->sh_info = area->addr;
+	pdev->sh_info = vaddr;
 
 	err = bind_interdomain_evtchn_to_irqhandler(
 		pdev->xdev->otherend_id, remote_evtchn, pciback_handle_event,
-		SA_SAMPLE_RANDOM, "pciback", pdev);
+		0, "pciback", pdev);
 	if (err < 0) {
 		xenbus_dev_fatal(pdev->xdev, err,
 				 "Error binding event channel to IRQ");
Signature · 1 line
1.6.2.5
#8 lore raw

PATCH 8/13 Xen paravirtualised PCI hotplug.

The pciback driver adds two new states that are shared by the pcifront and pciback. This is a patch pulled from the linux-2.6-xen-sparse tree.

---
 include/xen/interface/io/xenbus.h |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
include/xen/interface/io/xenbus.h +7 -1
index 46508c7..9fda532 100644
@@ -27,8 +27,14 @@ enum xenbus_state
 	XenbusStateClosing      = 5,  /* The device is being closed
 					 due to an error or an unplug
 					 event. */
-	XenbusStateClosed       = 6
+	XenbusStateClosed       = 6,
 
+	/*
+	* Reconfiguring: The device is being reconfigured.
+	*/
+	XenbusStateReconfiguring = 7,
+
+	XenbusStateReconfigured  = 8
 };
 
 #endif /* _XEN_PUBLIC_IO_XENBUS_H */
Signature · 1 line
1.6.2.5
#9 lore raw

PATCH 9/13 xenbus: Add new states to xenbus_strstate()

This patch adds the following states to the xenbus_strstate().
- XenbusStateReconfiguring
- XenbusStateReconfigured

Pulled from the linux-2.6-xen-sparse tree.

---
 drivers/xen/xenbus/xenbus_client.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
drivers/xen/xenbus/xenbus_client.c +2 -0
index 92a1ef8..da3ca9e 100644
@@ -49,6 +49,8 @@ const char *xenbus_strstate(enum xenbus_state state)
 		[ XenbusStateConnected    ] = "Connected",
 		[ XenbusStateClosing      ] = "Closing",
 		[ XenbusStateClosed	  ] = "Closed",
+		[ XenbusStateReconfiguring ] = "Reconfiguring",
+		[ XenbusStateReconfigured ] = "Reconfigured",
 	};
 	return (state < ARRAY_SIZE(name)) ? name[state] : "INVALID";
 }
Signature · 1 line
1.6.2.5
#10 lore raw

PATCH 10/13 xenbus: prevent warnings on unhandled enumeration values

XenbusStateReconfiguring/XenbusStateReconfigured were introduced by c/s 437, but aren't handled in many switch statements.

.. also pulled from the linux-2.6-sparse-tree tree.

---
 drivers/block/xen-blkfront.c |    2 ++
 drivers/input/xen-kbdfront.c |    2 ++
 drivers/net/xen-netfront.c   |    2 ++
 drivers/video/xen-fbfront.c  |    2 ++
 4 files changed, 8 insertions(+), 0 deletions(-)
drivers/block/xen-blkfront.c +2 -0
index e532847..9aff0f5 100644
@@ -966,6 +966,8 @@ static void backend_changed(struct xenbus_device *dev,
 	case XenbusStateInitialising:
 	case XenbusStateInitWait:
 	case XenbusStateInitialised:
+	case XenbusStateReconfiguring:
+	case XenbusStateReconfigured:
 	case XenbusStateUnknown:
 	case XenbusStateClosed:
 		break;
drivers/input/xen-kbdfront.c +2 -0
index b115726..97cad30 100644
@@ -272,6 +272,8 @@ static void xenkbd_backend_changed(struct xenbus_device *dev,
 	switch (backend_state) {
 	case XenbusStateInitialising:
 	case XenbusStateInitialised:
+	case XenbusStateReconfiguring:
+	case XenbusStateReconfigured:
 	case XenbusStateUnknown:
 	case XenbusStateClosed:
 		break;
drivers/net/xen-netfront.c +2 -0
index 8d88dae..9b1cd57 100644
@@ -1608,6 +1608,8 @@ static void backend_changed(struct xenbus_device *dev,
 	switch (backend_state) {
 	case XenbusStateInitialising:
 	case XenbusStateInitialised:
+	case XenbusStateReconfiguring:
+	case XenbusStateReconfigured:
 	case XenbusStateConnected:
 	case XenbusStateUnknown:
 	case XenbusStateClosed:
drivers/video/xen-fbfront.c +2 -0
index 54cd916..0c6b1c6 100644
@@ -627,6 +627,8 @@ static void xenfb_backend_changed(struct xenbus_device *dev,
 	switch (backend_state) {
 	case XenbusStateInitialising:
 	case XenbusStateInitialised:
+	case XenbusStateReconfiguring:
+	case XenbusStateReconfigured:
 	case XenbusStateUnknown:
 	case XenbusStateClosed:
 		break;
Signature · 1 line
1.6.2.5
#11 lore raw

PATCH 11/13 pciback: Add check to load only under priviliged domain.

---
 drivers/xen/pciback/pci_stub.c |    3 +++
 drivers/xen/pciback/xenbus.c   |    2 --
 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/xen/pciback/pci_stub.c +3 -0
index 036d7bb..5ea594b 100644
@@ -1268,6 +1268,9 @@ static int __init pciback_init(void)
 {
 	int err;
 
+	if (!xen_initial_domain())
+		return -ENODEV;
+
 	err = pciback_config_init();
 	if (err)
 		return err;
drivers/xen/pciback/xenbus.c +0 -2
index 5be1350..a85c413 100644
@@ -693,8 +693,6 @@ static struct xenbus_driver xenbus_pciback_driver = {
 
 int __init pciback_xenbus_register(void)
 {
-	if (!is_running_on_xen())
-		return -ENODEV;
 	pciback_wq = create_workqueue("pciback_workqueue");
 	if (!pciback_wq) {
 		printk(KERN_ERR "pciback_xenbus_register: create"
Signature · 1 line
1.6.2.5
#12 lore raw

PATCH 12/13 Remove usage of pci_restore_bars() as Linux handles the power-up states correctly now.

Originally this code was pulled from the upstream kernel, and stuck in the linux-2.6-sparse tree. At that point of time, the Linux tree (2.6.16?) did not know how to handle this. Nowadays the pci_set_power_state routine handles this case so we do not need this anymore.

---
 drivers/xen/pciback/conf_space_capability_pm.c |   13 -------------
 1 files changed, 0 insertions(+), 13 deletions(-)
drivers/xen/pciback/conf_space_capability_pm.c +0 -13
index e2f99c7..e1d3af4 100644
@@ -58,19 +58,6 @@ static int pm_ctrl_write(struct pci_dev *dev, int offset, u16 new_value,
 		goto out;
 	}
 
-	/*
-	 * Device may lose PCI config info on D3->D0 transition. This
-	 * is a problem for some guests which will not reset BARs. Even
-	 * those that have a go will be foiled by our BAR-write handler
-	 * which will discard the write! Since Linux won't re-init
-	 * the config space automatically in all cases, we do it here.
-	 * Future: Should we re-initialise all first 64 bytes of config space?
-	 */
-	if (new_state == PCI_D0 &&
-	    (old_state == PCI_D3hot || old_state == PCI_D3cold) &&
-	    !(old_value & PCI_PM_CTRL_NO_SOFT_RESET))
-		pci_restore_bars(dev);
-
  out:
 	return err;
 }
Signature · 1 line
1.6.2.5
#13 lore raw

PATCH 13/13 pciback: Enable Xen-PCI-back to be compiled.

---
 drivers/xen/Kconfig  |   65 ++++++++++++++++++++++++++++++++++++++++++++++++++
 drivers/xen/Makefile |    1 +
 2 files changed, 66 insertions(+), 0 deletions(-)
drivers/xen/Kconfig +65 -0
index c1b560d..1106607 100644
@@ -51,6 +51,71 @@ config XEN_NETDEV_BACKEND
          Implement the network backend driver, which passes packets
          from the guest domain's frontend drivers to the network.
 
+config XEN_PCIDEV_BACKEND
+	tristate "PCI-device backend driver"
+	depends on PCI && XEN_BACKEND
+	default XEN_BACKEND
+	help
+	  The PCI device backend driver allows the kernel to export arbitrary
+	  PCI devices to other guests. If you select this to be a module, you
+	  will need to make sure no other driver has bound to the device(s)
+	  you want to make visible to other guests.
+
+choice
+	prompt "PCI Backend Mode"
+	depends on XEN_PCIDEV_BACKEND
+	default XEN_PCIDEV_BACKEND_VPCI if !IA64
+	default XEN_PCIDEV_BACKEND_CONTROLLER if IA64
+
+config XEN_PCIDEV_BACKEND_VPCI
+	bool "Virtual PCI"
+	---help---
+	  This PCI Backend hides the true PCI topology and makes the frontend
+	  think there is a single PCI bus with only the exported devices on it.
+	  For example, a device at 03:05.0 will be re-assigned to 00:00.0. A
+	  second device at 02:1a.1 will be re-assigned to 00:01.1.
+
+config XEN_PCIDEV_BACKEND_PASS
+	bool "Passthrough"
+	---help---
+	  This PCI Backend provides a real view of the PCI topology to the
+	  frontend (for example, a device at 06:01.b will still appear at
+	  06:01.b to the frontend). This is similar to how Xen 2.0.x exposed
+	  PCI devices to its driver domains. This may be required for drivers
+	  which depend on finding their hardward in certain bus/slot
+	  locations.
+
+config XEN_PCIDEV_BACKEND_SLOT
+	bool "Slot"
+	---help---
+	  This PCI Backend hides the true PCI topology and makes the frontend
+	  think there is a single PCI bus with only the exported devices on it.
+	  Contrary to the virtual PCI backend, a function becomes a new slot.
+	  For example, a device at 03:05.2 will be re-assigned to 00:00.0. A
+	  second device at 02:1a.1 will be re-assigned to 00:01.0.
+
+config XEN_PCIDEV_BACKEND_CONTROLLER
+	bool "Controller"
+	depends on IA64
+	---help---
+	  This PCI backend virtualizes the PCI bus topology by providing a
+	  virtual bus per PCI root device.  Devices which are physically under
+	  the same root bus will appear on the same virtual bus.  For systems
+	  with complex I/O addressing, this is the only backend which supports
+	  extended I/O port spaces and MMIO translation offsets.  This backend
+	  also supports slot virtualization.  For example, a device at
+	  0000:01:02.1 will be re-assigned to 0000:00:00.0.  A second device
+	  at 0000:02:05.0 (behind a P2P bridge on bus 0000:01) will be
+	  re-assigned to 0000:00:01.0.  A third device at 0000:16:05.0 (under
+	  a different PCI root bus) will be re-assigned to 0000:01:00.0.
+
+endchoice
+
+config XEN_PCIDEV_BE_DEBUG
+	bool "PCI Backend Debugging"
+	depends on XEN_PCIDEV_BACKEND
+
+
 config XENFS
 	tristate "Xen filesystem"
 	depends on XEN
drivers/xen/Makefile +1 -0
index 116030e..64bafb4 100644
@@ -12,6 +12,7 @@ obj-$(CONFIG_XEN_DEV_EVTCHN)		+= evtchn.o
 obj-$(CONFIG_XEN_GNTDEV)		+= gntdev.o
 obj-$(CONFIG_XEN_BLKDEV_BACKEND)	+= blkback/
 obj-$(CONFIG_XEN_NETDEV_BACKEND)	+= netback/
+obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= pciback/
 obj-$(CONFIG_XENFS)			+= xenfs/
 obj-$(CONFIG_XEN_SYS_HYPERVISOR)	+= sys-hypervisor.o
 obj-$(CONFIG_XEN_S3)			+= acpi.o
Signature · 1 line
1.6.2.5
#14 lore raw

Re: [PATCH 08/13] Xen paravirtualised PCI hotplug.

In reply to Konrad Rzeszutek Wilk
Quoted 18 lines · Konrad Rzeszutek Wilk

The pciback driver adds two new states that are shared by the pcifront and pciback. This is a patch pulled from the linux-2.6-xen-sparse tree.

---
 include/xen/interface/io/xenbus.h |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
include/xen/interface/io/xenbus.h +0 -1
index 46508c7..9fda532 100644
@@ -27,8 +27,14 @@ enum xenbus_state
 	XenbusStateClosing      = 5,  /* The device is being closed
 					 due to an error or an unplug
 					 event. */
-	XenbusStateClosed       = 6

include/xen/interface/io/xenbus.h enum xenbus_state · line 39

Quoted 10 lines
include/xen/interface/io/xenbus.h +7 -0
+	XenbusStateClosed       = 6,
 
+	/*
+	* Reconfiguring: The device is being reconfigured.
+	*/
+	XenbusStateReconfiguring = 7,
+
+	XenbusStateReconfigured  = 8
 };
 

ouch.

what's the usefulness of those 2 new states here ?
the xenbus state is just here to represent the connection status of the xenbus device, doesn't reconfiguring/reconfigured state is something about the underlying device more than the xenbus device ?

And regarding the xenbus-state state machine, what happen when you reach 8 ? does every userspace handler need to handle it as a 4 ? if yes why is it not 4 ?

Signature · 1 line
Vincent
#15 lore raw

Re: [PATCH 08/13] Xen paravirtualised PCI hotplug.

Quoted 8 lines

include/xen/interface/io/xenbus.h enum xenbus_state · line 38

Quoted 5 lines
include/xen/interface/io/xenbus.h +3 -0
+	XenbusStateReconfiguring = 7,
+
+	XenbusStateReconfigured  = 8
 };

ouch.

what's the usefulness of those 2 new states here ?

The changes came originally from the c/s 17223, which was titled:
Add PV passthrough PCI device hotplug support.

The short summary of that is that you can dynamically attach
PCI devices by doing: 'xm pci-attach'. And also detach by doing 'xm pci-dettach'.

Quoted 3 lines

the xenbus state is just here to represent the connection status of the xenbus device, doesn't reconfiguring/reconfigured state is something about the underlying device more than the xenbus device ?

I suppose it is. Thought it is more of a state of evolution - the blkback device could support changing of size dynamically (for example if it was mounted on raw LUN which changed size as well). Or if the blkback was binded to a CD-ROM device - support reconfiguring it (new media perhaps?).

Quoted 4 lines

And regarding the xenbus-state state machine, what happen when you reach 8 ? does every userspace handler need to handle it as a 4 ? if yes why is it not 4 ?

Good question. Lets ask Yosuke Iwamatsu who posted the initial patch for this.

#16 lore raw

PATCH 8/13 Xen paravirtualised PCI hotplug.

Quoted 15 lines · Konrad Rzeszutek Wilk
Quoted 8 lines

include/xen/interface/io/xenbus.h enum xenbus_state · line 38

Quoted 5 lines
include/xen/interface/io/xenbus.h +3 -0
+	XenbusStateReconfiguring = 7,
+
+	XenbusStateReconfigured  = 8
 };

ouch.

what's the usefulness of those 2 new states here ?

The changes came originally from the c/s 17223, which was titled:
Add PV passthrough PCI device hotplug support.

The short summary of that is that you can dynamically attach
PCI devices by doing: 'xm pci-attach'. And also detach by doing 'xm pci-dettach'.

Quoted 6 lines
Quoted 1 line

the xenbus state is just here to represent the connection status of the xenbus device, doesn't reconfiguring/reconfigured state is something about the underlying device more than the xenbus device ?

I suppose it is. Thought it is more of a state of evolution - the blkback device could support changing of size dynamically (for example if it was mounted on raw LUN which changed size as well). Or if the blkback was binded to a CD-ROM device - support reconfiguring it (new media perhaps?).

Please also note that those two states have been already used by other devices than pvpci, such as pvscsi and pvusb.

Quoted 6 lines
Quoted 3 lines

And regarding the xenbus-state state machine, what happen when you reach 8 ? does every userspace handler need to handle it as a 4 ? if yes why is it not 4 ?

Good question. Lets ask Yosuke Iwamatsu who posted the initial patch for this.

The state of pciback will reach 8 during the interaction between pcifront and
pciback, but in the end the state of both ends will become 4, so userspace handlers do not need to care about it indeed.

Regards,
-- Yosuke Iwamatsu

Quoted 5 lines

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#17 lore raw

Re: [PATCH 04/13] Removed MSI capability in the pci-back driver.

In reply to Konrad Rzeszutek Wilk
Quoted 3 lines · Konrad Rzeszutek Wilk

Currently the tree does not have the Xen MSI capability. Will re-introduce when the capability exists.

Devices with MSI interrupts work fine in dom0. Are you referring to something else?

J

#18 lore raw

Re: [PATCH 04/13] Removed MSI capability in the pci-back driver.

Quoted 7 lines · Jeremy Fitzhardinge
Quoted 3 lines · Konrad Rzeszutek Wilk

Currently the tree does not have the Xen MSI capability. Will re-introduce when the capability exists.

Devices with MSI interrupts work fine in dom0. Are you referring to something else?

The issue I was referring to was the ownership of MSI interrupts for PV guests. Mainly the mechanism for binding an MSI interrupt to a guest is not part of the Linux pv-ops kernel. If you look in the xen_setup_msi_irqs function,

 624         map_irq.domid = domid;
 625         map_irq.type = MAP_PIRQ_TYPE_MSI;
 626         map_irq.index = -1;
 627         map_irq.pirq = -1;
 628         map_irq.bus = dev->bus->number;
 629         map_irq.devfn = dev->devfn;

630
.. snip ..
649 rc = HYPERVISOR_physdev_op(PHYSDEVOP_map_pirq, &map_irq);

The domain that the vector is assigned to is the DOMID_SELF (domid earlier on is set to that). For pciback purpose, this needs to be the guest domain id. HVM guests accomplish this in the hw/pt-msi.c code, so we need only to worry about the PV case.

In the 2.6.18 code there was hook that would pass this domaid ID around for the device (register_msi_get_owner and its friend). I am trying to figure out if there is a more upstream-applicable way of doing this.

Quoted 3 lines

J

#19 lore raw
In reply to Konrad Rzeszutek Wilk
Quoted 6 lines · Konrad Rzeszutek Wilk

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

OK, I've stuck this in xen/dom0/pciback for now. I haven't merged it yet or even looked at it very closely yet.

J

#20 lore raw
Quoted 10 lines · Jeremy Fitzhardinge
Quoted 6 lines · Konrad Rzeszutek Wilk

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

OK, I've stuck this in xen/dom0/pciback for now. I haven't merged it yet or even looked at it very closely yet.

Sure. Let me look more closely at the MSI part. I thought that the kernel did not have the support for this yet.

#21 lore raw
Quoted 3 lines · Konrad Rzeszutek Wilk

Sure. Let me look more closely at the MSI part. I thought that the kernel did not have the support for this yet.

The code lives in xen/dom0/msi.

J

#22 lore raw
In reply to Konrad Rzeszutek Wilk

Hello Konrad,

With the 2.6.18.8 kernel and 2.6.29.6 port of the xen kernel (on xen 3.4.1 hypervisor) I had to use an additional guestdev= and reassign_resources to make things work.

Are these also ported/available on the 2.6.31.1/pv_ops kernel or not needed anymore ?

Complete lines in grub dom0:

kernel          /xen-3.4.1.gz dom0_mem=768M xencons=hvc
module          /vmlinuz-2.6.29.6 root=/dev/mapper/serveerstertje-root ro pci=nomsi pciback.hide=(00:07.0)(06:01.0)(06:01.1)(06:01.2)(01:08.0)(01:08.1)(01:08.2)(01:0a.0)  guestdev=00:07.0,06:01.0,06:01.1,06:01.2,01:08.0,01:08.1,01:08.2,01:0a.0 reassign_resources swiotlb=256,force console=tty0
module          /initrd.img-2.6.29.6

making this work for me by passing through 2 USB cards (one pci one pci-e) and a integrated intel hda sound card) to domU's

Regards,

Sander Eikelenboom

Quoted 30 lines · you

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

The usage of this is, as said, is similar to pci-stub:
lspci | grep SCSI
01:14.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/unbind echo "0000:01:14.0" > /sys/bus/pci/drivers/pciback/new-slot echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/bind

and add this entry:

pci = [ '0000:01.14.0' ]

in your .xm file.

The PV guest, if it has the PCI frontend, should now see the PCI device. I've tested this succesfully with a SLES10 PV guest with a couple of devices.

But please be beware of this warning if it shows up:
(XEN) irq.c:1113:d1 Cannot bind IRQ 17 to guest. Others do not share.

On my machine it lead to Dom0 deciding that a spurrious interrupt kicked off and it disabled the IRQ. The end result was that other devices on the same interrupt line stopped working. I am not yet certain how to make this work properly (whether to check if the PCI device in question interrupt line is being shared beforehand by xm?, or do something in Xen?).

Signature · 2 lines
Best regards,
 Sander                            mailto:[email protected]
#23 lore raw
Quoted 2 lines · Sander Eikelenboom

Hello Konrad,

Hey Sander,

Quoted 2 lines

With the 2.6.18.8 kernel and 2.6.29.6 port of the xen kernel (on xen 3.4.1 hypervisor) I had to use an additional guestdev= and reassign_resources to make things work.

I digged a bit more in this. What was the result of not passing those
arguments? Did the kernel notice a spurious interrupt? Or would the device not work? Can you attach the output of 'lspci -vvv' with and without the "guestdev=.. reassign_resources" parameters with your 2.6.18.8 (or 2.6.29.6 port) kernel.

Yuji, Yu, Jesse, and Barak:

The 'guestdev.c' and its parameter 'guestdev' look to be doing the same thing as the the 'pciback.hide=' argument? The extra functionality is with the 'reassign_resources' which does two major things:
1) in quirk_release_resources disables the PCI card from latching on the bus addresses that fall within its BARs - which is done during the chain of invocations when 'pci_enable_device' was called.
Thought I am curious - who then re-enables the card to latch on the bus addresses? Presumarily the guest OS?
2) in pci_assign_resources, pbus_size_mem, and in pdev_sort_resources aligns the BARs to page size. Past checkins in the 2.6.18.hg suggest this is b/c in the past mmio resources were translated from PFNs->MFNs and there were no checks whether it was page-aligned.

So, I was wondering whether both of these things were neccessary in the Xen 3.5 and PV-OPS kernel?

Is the mapping of the MMIO resources done b/c xm/xc/qemu can't deal with resources which are not page-aligned? Would it not be easier to fix xm/xc/qemu to map page-aligned addresses to the MMIO for the guest? I thought I saw a patch couple of days that just did that...

Any response would be appreciated - I was thinking to see how Sander fares and if it fails for him, look at xm/xc and qemu to see if something can be done there.

Quoted 67 lines

Are these also ported/available on the 2.6.31.1/pv_ops kernel or not needed anymore ?

Complete lines in grub dom0:

kernel          /xen-3.4.1.gz dom0_mem=768M xencons=hvc
module          /vmlinuz-2.6.29.6 root=/dev/mapper/serveerstertje-root ro pci=nomsi pciback.hide=(00:07.0)(06:01.0)(06:01.1)(06:01.2)(01:08.0)(01:08.1)(01:08.2)(01:0a.0)  guestdev=00:07.0,06:01.0,06:01.1,06:01.2,01:08.0,01:08.1,01:08.2,01:0a.0 reassign_resources swiotlb=256,force console=tty0
module          /initrd.img-2.6.29.6

making this work for me by passing through 2 USB cards (one pci one pci-e) and a integrated intel hda sound card) to domU's

Regards,

Sander Eikelenboom

Quoted 30 lines · you

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

The usage of this is, as said, is similar to pci-stub:
lspci | grep SCSI
01:14.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/unbind echo "0000:01:14.0" > /sys/bus/pci/drivers/pciback/new-slot echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/bind

and add this entry:

pci = [ '0000:01.14.0' ]

in your .xm file.

The PV guest, if it has the PCI frontend, should now see the PCI device. I've tested this succesfully with a SLES10 PV guest with a couple of devices.

But please be beware of this warning if it shows up:
(XEN) irq.c:1113:d1 Cannot bind IRQ 17 to guest. Others do not share.

On my machine it lead to Dom0 deciding that a spurrious interrupt kicked off and it disabled the IRQ. The end result was that other devices on the same interrupt line stopped working. I am not yet certain how to make this work properly (whether to check if the PCI device in question interrupt line is being shared beforehand by xm?, or do something in Xen?).

Signature · 8 lines
Best regards,
 Sander                            mailto:[email protected]


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
#24 lore raw

Hello Konrad,

hmmm that was some time ago when i figured this one out :-)
But i think i will have some time tomorrow to boot without on the 2.6.29.6 kernel and see what is does ... For what i remember it was especially necessary for the USB cards to work, the internal sound card worked ok without.

Regards,

Sander

Quoted 105 lines · you
Quoted 2 lines · Sander Eikelenboom

Hello Konrad,

Hey Sander,

Quoted 2 lines

With the 2.6.18.8 kernel and 2.6.29.6 port of the xen kernel (on xen 3.4.1 hypervisor) I had to use an additional guestdev= and reassign_resources to make things work.

I digged a bit more in this. What was the result of not passing those
arguments? Did the kernel notice a spurious interrupt? Or would the device not work? Can you attach the output of 'lspci -vvv' with and without the "guestdev=.. reassign_resources" parameters with your 2.6.18.8 (or 2.6.29.6 port) kernel.

Yuji, Yu, Jesse, and Barak:

The 'guestdev.c' and its parameter 'guestdev' look to be doing the same thing as the the 'pciback.hide=' argument? The extra functionality is with the 'reassign_resources' which does two major things:
1) in quirk_release_resources disables the PCI card from latching on the bus addresses that fall within its BARs - which is done during the chain of invocations when 'pci_enable_device' was called.
Thought I am curious - who then re-enables the card to latch on the bus addresses? Presumarily the guest OS?
2) in pci_assign_resources, pbus_size_mem, and in pdev_sort_resources aligns the BARs to page size. Past checkins in the 2.6.18.hg suggest this is b/c in the past mmio resources were translated from PFNs->MFNs and there were no checks whether it was page-aligned.

So, I was wondering whether both of these things were neccessary in the Xen 3.5 and PV-OPS kernel?

Is the mapping of the MMIO resources done b/c xm/xc/qemu can't deal with resources which are not page-aligned? Would it not be easier to fix xm/xc/qemu to map page-aligned addresses to the MMIO for the guest? I thought I saw a patch couple of days that just did that...

Any response would be appreciated - I was thinking to see how Sander fares and if it fails for him, look at xm/xc and qemu to see if something can be done there.

Quoted 67 lines

Are these also ported/available on the 2.6.31.1/pv_ops kernel or not needed anymore ?

Complete lines in grub dom0:

kernel          /xen-3.4.1.gz dom0_mem=768M xencons=hvc
module          /vmlinuz-2.6.29.6 root=/dev/mapper/serveerstertje-root ro pci=nomsi pciback.hide=(00:07.0)(06:01.0)(06:01.1)(06:01.2)(01:08.0)(01:08.1)(01:08.2)(01:0a.0)  guestdev=00:07.0,06:01.0,06:01.1,06:01.2,01:08.0,01:08.1,01:08.2,01:0a.0 reassign_resources swiotlb=256,force console=tty0
module          /initrd.img-2.6.29.6

making this work for me by passing through 2 USB cards (one pci one pci-e) and a integrated intel hda sound card) to domU's

Regards,

Sander Eikelenboom

Quoted 30 lines · you

This is back-port (up-port?) of the pci-back driver from the 2.6.18.hg tree. The driver is quite similar to the pci-stub, excep that is intended for paravirtualized guests. This driver works in conjunction with the pci-front (frontend driver) to exchange PCI write/read to the configuration space and to have the BARs mapped properly for the guest.

The usage of this is, as said, is similar to pci-stub:
lspci | grep SCSI
01:14.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/unbind echo "0000:01:14.0" > /sys/bus/pci/drivers/pciback/new-slot echo "0000:01:14.0" > /sys/bus/pci/drivers/aic94xx/bind

and add this entry:

pci = [ '0000:01.14.0' ]

in your .xm file.

The PV guest, if it has the PCI frontend, should now see the PCI device. I've tested this succesfully with a SLES10 PV guest with a couple of devices.

But please be beware of this warning if it shows up:
(XEN) irq.c:1113:d1 Cannot bind IRQ 17 to guest. Others do not share.

On my machine it lead to Dom0 deciding that a spurrious interrupt kicked off and it disabled the IRQ. The end result was that other devices on the same interrupt line stopped working. I am not yet certain how to make this work properly (whether to check if the PCI device in question interrupt line is being shared beforehand by xm?, or do something in Xen?).

Signature · 8 lines
Best regards,
 Sander                            mailto:[email protected]


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Signature · 2 lines
Best regards,
 Sander                            mailto:[email protected]