Skip to contentSkip to conversation

PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

PATCH Superseded [email protected] 4 participants 16 messages Dec 9, 2015 – Jan 8, 2016

Subsystems (5)

v9 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v9 cover PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

From: Jake Oshins <[email protected]>

This version of the patch series removes warning when compiling x86 32-bit.

First, export functions that allow correlating Hyper-V virtual processors and Linux cpus, along with the means for invoking a hypercall that targets interrupts at chosen vectors on specific cpus.

Second, mark various parts of IRQ domain related code as exported, so that this PCI front-end can implement an IRQ domain as part of a module. (The alternative would be to pull all tyhis into the kernel, which would pull in a lot of other Hyper-V related code, as this IRQ domain depends on vmbus.ko.)

Third, modify PCI so that new root PCI buses can be marked with an associated fwnode_handle, and so that root PCI buses can look up their associated IRQ domain by that handle.

Fourth, introduce a new driver, hv_pcifront, which exposes root PCI buses in a Hyper-V VM. These root PCI buses expose real PCIe device, or PCI Virtual Functions.

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.
PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

 MAINTAINERS                    |    1 +
 arch/x86/include/asm/msi.h     |    6 +
 arch/x86/include/asm/pci.h     |   15 +
 arch/x86/kernel/apic/msi.c     |    8 +-
 arch/x86/kernel/apic/vector.c  |    2 +
 drivers/hv/hv.c                |   20 +-
 drivers/hv/hyperv_vmbus.h      |    2 +-
 drivers/hv/vmbus_drv.c         |   17 +
 drivers/pci/Kconfig            |    7 +
 drivers/pci/host/Makefile      |    1 +
 drivers/pci/host/hv_pcifront.c | 2250 ++++++++++++++++++++++++++++++++++++++++
 drivers/pci/msi.c              |    4 +
 drivers/pci/probe.c            |   15 +
 include/linux/hyperv.h         |   14 +
 include/linux/pci.h            |    4 +
 kernel/irq/chip.c              |    1 +
 kernel/irq/irqdomain.c         |    4 +
 17 files changed, 2357 insertions(+), 14 deletions(-)
 create mode 100644 drivers/pci/host/hv_pcifront.c
Signature · 1 line
1.9.1
#1 lore raw

PATCH v9 1/7 drivers:hv: Export a function that maps Linux CPU num onto Hyper-V proc num

From: Jake Oshins <[email protected]>

This patch exposes the mapping between Linux CPU number and Hyper-V virtual processor number. This is necessary because the hypervisor needs to know which virtual processor to target when making a mapping in the Interrupt Redirection Table in the I/O MMU.

---
 drivers/hv/vmbus_drv.c | 17 +++++++++++++++++
 include/linux/hyperv.h |  2 ++
 2 files changed, 19 insertions(+)
drivers/hv/vmbus_drv.c +17 -0
index f19b6f7..c89d0e5 100644
@@ -1191,6 +1191,23 @@ int vmbus_allocate_mmio(struct resource **new, struct hv_device *device_obj,
 }
 EXPORT_SYMBOL_GPL(vmbus_allocate_mmio);
 
+/**
+ * vmbus_cpu_number_to_vp_number() - Map CPU to VP.
+ * @cpu_number: CPU number in Linux terms
+ *
+ * This function returns the mapping between the Linux processor
+ * number and the hypervisor's virtual processor number, useful
+ * in making hypercalls and such that talk about specific
+ * processors.
+ *
+ * Return: Virtual processor number in Hyper-V terms
+ */
+int vmbus_cpu_number_to_vp_number(int cpu_number)
+{
+	return hv_context.vp_index[cpu_number];
+}
+EXPORT_SYMBOL_GPL(vmbus_cpu_number_to_vp_number);
+
 static int vmbus_acpi_add(struct acpi_device *device)
 {
 	acpi_status result;
include/linux/hyperv.h +2 -0
index 8fdc17b..fddb3e0 100644
@@ -983,6 +983,8 @@ int vmbus_allocate_mmio(struct resource **new, struct hv_device *device_obj,
 			resource_size_t size, resource_size_t align,
 			bool fb_overlap_ok);
 
+int vmbus_cpu_number_to_vp_number(int cpu_number);
+
 /**
  * VMBUS_DEVICE - macro used to describe a specific hyperv vmbus device
  *
Signature · 1 line
1.9.1
#2 lore raw

PATCH v9 2/7 drivers:hv: Export hv_do_hypercall()

In reply to [email protected]

From: Jake Oshins <[email protected]>

This patch exposes the function that hv_vmbus.ko uses to make hypercalls. This is necessary for retargeting an interrupt when it is given a new affinity and vector.

---
 drivers/hv/hv.c           | 20 ++++++++++----------
 drivers/hv/hyperv_vmbus.h |  2 +-
 include/linux/hyperv.h    |  1 +
 3 files changed, 12 insertions(+), 11 deletions(-)
drivers/hv/hv.c +10 -10
index 6341be8..7a06933 100644
@@ -89,9 +89,9 @@ static int query_hypervisor_info(void)
 }
 
 /*
- * do_hypercall- Invoke the specified hypercall
+ * hv_do_hypercall- Invoke the specified hypercall
  */
-static u64 do_hypercall(u64 control, void *input, void *output)
+u64 hv_do_hypercall(u64 control, void *input, void *output)
 {
 	u64 input_address = (input) ? virt_to_phys(input) : 0;
 	u64 output_address = (output) ? virt_to_phys(output) : 0;
@@ -132,6 +132,7 @@ static u64 do_hypercall(u64 control, void *input, void *output)
 	return hv_status_lo | ((u64)hv_status_hi << 32);
 #endif /* !x86_64 */
 }
+EXPORT_SYMBOL_GPL(hv_do_hypercall);
 
 #ifdef CONFIG_X86_64
 static cycle_t read_hv_clock_tsc(struct clocksource *arg)
@@ -315,7 +316,7 @@ int hv_post_message(union hv_connection_id connection_id,
 {
 
 	struct hv_input_post_message *aligned_msg;
-	u16 status;
+	u64 status;
 
 	if (payload_size > HV_MESSAGE_PAYLOAD_BYTE_COUNT)
 		return -EMSGSIZE;
@@ -329,11 +330,10 @@ int hv_post_message(union hv_connection_id connection_id,
 	aligned_msg->payload_size = payload_size;
 	memcpy((void *)aligned_msg->payload, payload, payload_size);
 
-	status = do_hypercall(HVCALL_POST_MESSAGE, aligned_msg, NULL)
-		& 0xFFFF;
+	status = hv_do_hypercall(HVCALL_POST_MESSAGE, aligned_msg, NULL);
 
 	put_cpu();
-	return status;
+	return status & 0xFFFF;
 }
 
 
@@ -343,13 +343,13 @@ int hv_post_message(union hv_connection_id connection_id,
  *
  * This involves a hypercall.
  */
-u16 hv_signal_event(void *con_id)
+int hv_signal_event(void *con_id)
 {
-	u16 status;
+	u64 status;
 
-	status = (do_hypercall(HVCALL_SIGNAL_EVENT, con_id, NULL) & 0xFFFF);
+	status = hv_do_hypercall(HVCALL_SIGNAL_EVENT, con_id, NULL);
 
-	return status;
+	return status & 0xFFFF;
 }
 
 static int hv_ce_set_next_event(unsigned long delta,
drivers/hv/hyperv_vmbus.h +1 -1
index 3782636..cc2fb53 100644
@@ -582,7 +582,7 @@ extern int hv_post_message(union hv_connection_id connection_id,
 			 enum hv_message_type message_type,
 			 void *payload, size_t payload_size);
 
-extern u16 hv_signal_event(void *con_id);
+extern int hv_signal_event(void *con_id);
 
 extern int hv_synic_alloc(void);
 
include/linux/hyperv.h +1 -0
index fddb3e0..24d0b65 100644
@@ -984,6 +984,7 @@ int vmbus_allocate_mmio(struct resource **new, struct hv_device *device_obj,
 			bool fb_overlap_ok);
 
 int vmbus_cpu_number_to_vp_number(int cpu_number);
+u64 hv_do_hypercall(u64 control, void *input, void *output);
 
 /**
  * VMBUS_DEVICE - macro used to describe a specific hyperv vmbus device
Signature · 1 line
1.9.1
#3 lore raw

PATCH v9 3/7 PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.

In reply to [email protected]

From: Jake Oshins <[email protected]>

The Linux kernel already has the concpet of IRQ domain, wherein a component can expose a set of IRQs which are managed by a particular interrupt controller chip or other subsystem. The PCI driver exposes the notion of an IRQ domain for Message-Signaled Interrupts (MSI) from PCI Express devices. This patch exposes the functions which are necessary for making an MSI IRQ domain within a module.

---
 arch/x86/include/asm/msi.h    | 6 ++++++
 arch/x86/kernel/apic/msi.c    | 8 +++++---
 arch/x86/kernel/apic/vector.c | 2 ++
 drivers/pci/msi.c             | 4 ++++
 kernel/irq/chip.c             | 1 +
 kernel/irq/irqdomain.c        | 4 ++++
 6 files changed, 22 insertions(+), 3 deletions(-)
arch/x86/include/asm/msi.h +6 -0
index 93724cc..eb4b09b 100644
@@ -1,7 +1,13 @@
 #ifndef _ASM_X86_MSI_H
 #define _ASM_X86_MSI_H
 #include <asm/hw_irq.h>
+#include <asm/irqdomain.h>
 
 typedef struct irq_alloc_info msi_alloc_info_t;
 
+int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec,
+		    msi_alloc_info_t *arg);
+
+void pci_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc);
+
 #endif /* _ASM_X86_MSI_H */
arch/x86/kernel/apic/msi.c +5 -3
index 5f1feb6..ade2532 100644
@@ -96,8 +96,8 @@ static irq_hw_number_t pci_msi_get_hwirq(struct msi_domain_info *info,
 	return arg->msi_hwirq;
 }
 
-static int pci_msi_prepare(struct irq_domain *domain, struct device *dev,
-			   int nvec, msi_alloc_info_t *arg)
+int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec,
+		    msi_alloc_info_t *arg)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
 	struct msi_desc *desc = first_pci_msi_entry(pdev);
@@ -113,11 +113,13 @@ static int pci_msi_prepare(struct irq_domain *domain, struct device *dev,
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(pci_msi_prepare);
 
-static void pci_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+void pci_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->msi_hwirq = pci_msi_domain_calc_hwirq(arg->msi_dev, desc);
 }
+EXPORT_SYMBOL_GPL(pci_msi_set_desc);
 
 static struct msi_domain_ops pci_msi_domain_ops = {
 	.get_hwirq	= pci_msi_get_hwirq,
arch/x86/kernel/apic/vector.c +2 -0
index 861bc59..908cb37 100644
@@ -29,6 +29,7 @@ struct apic_chip_data {
 };
 
 struct irq_domain *x86_vector_domain;
+EXPORT_SYMBOL_GPL(x86_vector_domain);
 static DEFINE_RAW_SPINLOCK(vector_lock);
 static cpumask_var_t vector_cpumask;
 static struct irq_chip lapic_controller;
@@ -66,6 +67,7 @@ struct irq_cfg *irqd_cfg(struct irq_data *irq_data)
 
 	return data ? &data->cfg : NULL;
 }
+EXPORT_SYMBOL_GPL(irqd_cfg);
 
 struct irq_cfg *irq_cfg(unsigned int irq)
 {
drivers/pci/msi.c +4 -0
index 7eaa4c8..7a0df3f 100644
@@ -257,6 +257,7 @@ void pci_msi_mask_irq(struct irq_data *data)
 {
 	msi_set_mask_bit(data, 1);
 }
+EXPORT_SYMBOL_GPL(pci_msi_mask_irq);
 
 /**
  * pci_msi_unmask_irq - Generic irq chip callback to unmask PCI/MSI interrupts
@@ -266,6 +267,7 @@ void pci_msi_unmask_irq(struct irq_data *data)
 {
 	msi_set_mask_bit(data, 0);
 }
+EXPORT_SYMBOL_GPL(pci_msi_unmask_irq);
 
 void default_restore_msi_irqs(struct pci_dev *dev)
 {
@@ -1126,6 +1128,7 @@ struct pci_dev *msi_desc_to_pci_dev(struct msi_desc *desc)
 {
 	return to_pci_dev(desc->dev);
 }
+EXPORT_SYMBOL(msi_desc_to_pci_dev);
 
 void *msi_desc_to_pci_sysdata(struct msi_desc *desc)
 {
@@ -1285,6 +1288,7 @@ struct irq_domain *pci_msi_create_irq_domain(struct fwnode_handle *fwnode,
 	domain->bus_token = DOMAIN_BUS_PCI_MSI;
 	return domain;
 }
+EXPORT_SYMBOL_GPL(pci_msi_create_irq_domain);
 
 /**
  * pci_msi_domain_alloc_irqs - Allocate interrupts for @dev in @domain
kernel/irq/chip.c +1 -0
index 05e29de..5797909 100644
@@ -950,6 +950,7 @@ void irq_chip_ack_parent(struct irq_data *data)
 	data = data->parent_data;
 	data->chip->irq_ack(data);
 }
+EXPORT_SYMBOL_GPL(irq_chip_ack_parent);
 
 /**
  * irq_chip_mask_parent - Mask the parent interrupt
kernel/irq/irqdomain.c +4 -0
index 22aa961..aa6ac44 100644
@@ -60,6 +60,7 @@ struct fwnode_handle *irq_domain_alloc_fwnode(void *data)
 	fwid->fwnode.type = FWNODE_IRQCHIP;
 	return &fwid->fwnode;
 }
+EXPORT_SYMBOL_GPL(irq_domain_alloc_fwnode);
 
 /**
  * irq_domain_free_fwnode - Free a non-OF-backed fwnode_handle
@@ -77,6 +78,7 @@ void irq_domain_free_fwnode(struct fwnode_handle *fwnode)
 	kfree(fwid->name);
 	kfree(fwid);
 }
+EXPORT_SYMBOL_GPL(irq_domain_free_fwnode);
 
 /**
  * __irq_domain_add() - Allocate a new irq_domain data structure
@@ -1013,6 +1015,7 @@ struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
 
 	return NULL;
 }
+EXPORT_SYMBOL_GPL(irq_domain_get_irq_data);
 
 /**
  * irq_domain_set_hwirq_and_chip - Set hwirq and irqchip of @virq at @domain
@@ -1343,6 +1346,7 @@ struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
 
 	return (irq_data && irq_data->domain == domain) ? irq_data : NULL;
 }
+EXPORT_SYMBOL_GPL(irq_domain_get_irq_data);
 
 /**
  * irq_domain_set_info - Set the complete data for a @virq in @domain
Signature · 1 line
1.9.1
#4 lore raw

PATCH v9 4/7 PCI: Add fwnode_handle to pci_sysdata

In reply to [email protected]

From: Jake Oshins <[email protected]>

This patch adds an fwnode_handle to struct pci_sysdata, which is used by the next patch in the series when trying to locate an IRQ domain associated with a root PCI bus.

---
 arch/x86/include/asm/pci.h | 15 +++++++++++++++
 drivers/pci/probe.c        |  1 +
 include/linux/pci.h        |  4 ++++
 3 files changed, 20 insertions(+)
arch/x86/include/asm/pci.h +15 -0
index 4625943..6fc3c7c 100644
@@ -20,6 +20,9 @@ struct pci_sysdata {
 #ifdef CONFIG_X86_64
 	void		*iommu;		/* IOMMU private data */
 #endif
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+	void		*fwnode;	/* IRQ domain for MSI assignment */
+#endif
 };
 
 extern int pci_routeirq;
@@ -32,6 +35,7 @@ extern int noioapicreroute;
 static inline int pci_domain_nr(struct pci_bus *bus)
 {
 	struct pci_sysdata *sd = bus->sysdata;
+
 	return sd->domain;
 }
 
@@ -41,6 +45,17 @@ static inline int pci_proc_domain(struct pci_bus *bus)
 }
 #endif
 
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+static inline void *_pci_root_bus_fwnode(struct pci_bus *bus)
+{
+	struct pci_sysdata *sd = bus->sysdata;
+
+	return sd->fwnode;
+}
+
+#define pci_root_bus_fwnode	_pci_root_bus_fwnode
+#endif
+
 /* Can be used to override the logic in pci_scan_bus for skipping
    already-configured bus numbers - to be used for buggy BIOSes
    or architectures with incomplete PCI setup by the loader */
drivers/pci/probe.c +1 -0
index edb1984..750f907 100644
@@ -15,6 +15,7 @@
 #include <linux/pci-aspm.h>
 #include <linux/aer.h>
 #include <linux/acpi.h>
+#include <linux/irqdomain.h>
 #include <asm-generic/pci-bridge.h>
 #include "pci.h"
 
include/linux/pci.h +4 -0
index 6ae25aa..b414422 100644
@@ -1517,6 +1517,10 @@ static inline int pci_get_new_domain_nr(void) { return -ENOSYS; }
 
 #include <asm/pci.h>
 
+#ifndef pci_root_bus_fwnode
+#define pci_root_bus_fwnode(bus)	((void)(bus), NULL)
+#endif
+
 /* these helpers provide future and backwards compatibility
  * for accessing popular PCI BAR info */
 #define pci_resource_start(dev, bar)	((dev)->resource[(bar)].start)
Signature · 1 line
1.9.1
#5 lore raw

RE: [PATCH v9 4/7] PCI: Add fwnode_handle to pci_sysdata

Quoted 91 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, December 9, 2015 2:55 PM
To: [email protected]; KY Srinivasan <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Haiyang Zhang <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Jake Oshins <[email protected]>
Subject: [PATCH v9 4/7] PCI: Add fwnode_handle to pci_sysdata

From: Jake Oshins <[email protected]>

This patch adds an fwnode_handle to struct pci_sysdata, which is used by the next patch in the series when trying to locate an IRQ domain associated with a root PCI bus.

---
 arch/x86/include/asm/pci.h | 15 +++++++++++++++
 drivers/pci/probe.c        |  1 +
 include/linux/pci.h        |  4 ++++
 3 files changed, 20 insertions(+)
arch/x86/include/asm/pci.h +15 -0
index 4625943..6fc3c7c 100644
@@ -20,6 +20,9 @@ struct pci_sysdata {
 #ifdef CONFIG_X86_64
 	void		*iommu;		/* IOMMU private data */
 #endif
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+	void		*fwnode;	/* IRQ domain for MSI assignment */
+#endif
 };
 
 extern int pci_routeirq;
@@ -32,6 +35,7 @@ extern int noioapicreroute;
 static inline int pci_domain_nr(struct pci_bus *bus)
 {
 	struct pci_sysdata *sd = bus->sysdata;
+
 	return sd->domain;
 }
 
@@ -41,6 +45,17 @@ static inline int pci_proc_domain(struct pci_bus *bus)
 }
 #endif
 
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+static inline void *_pci_root_bus_fwnode(struct pci_bus *bus)
+{
+	struct pci_sysdata *sd = bus->sysdata;
+
+	return sd->fwnode;
+}
+
+#define pci_root_bus_fwnode	_pci_root_bus_fwnode
+#endif
+
 /* Can be used to override the logic in pci_scan_bus for skipping
    already-configured bus numbers - to be used for buggy BIOSes
    or architectures with incomplete PCI setup by the loader */
drivers/pci/probe.c +1 -0
index edb1984..750f907 100644
@@ -15,6 +15,7 @@
 #include <linux/pci-aspm.h>
 #include <linux/aer.h>
 #include <linux/acpi.h>
+#include <linux/irqdomain.h>
 #include <asm-generic/pci-bridge.h>
 #include "pci.h"
 
include/linux/pci.h +4 -0
index 6ae25aa..b414422 100644
@@ -1517,6 +1517,10 @@ static inline int pci_get_new_domain_nr(void) {
 return -ENOSYS; }
 
 #include <asm/pci.h>
 
+#ifndef pci_root_bus_fwnode
+#define pci_root_bus_fwnode(bus)	((void)(bus), NULL)
+#endif
+
 /* these helpers provide future and backwards compatibility
  * for accessing popular PCI BAR info */
Quoted 3 lines

#define pci_resource_start(dev, bar) ((dev)->resource[(bar)].start)
--
1.9.1

Bjorn, are you willing to take this patch and the next one in the series through your tree? The other patches in this series have already been taken into linux-next by either Greg K-H or Thomas Gleixner, or they're dependent on these two. Thomas has suggested that the PCI tree would be the right one to incorporate them.

Thanks,
Jake Oshins

#6 lore raw

PATCH v9 5/7 PCI: irqdomain: Look up IRQ domain by fwnode_handle

In reply to [email protected]

From: Jake Oshins <[email protected]>

This patch adds a second way of finding an IRQ domain associated with a root PCI bus. After looking to see if one can be found through the OF tree, it attempts to look up the IRQ domain through an fwnode_handle stored in the pci_sysdata struct.

---
 drivers/pci/probe.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/pci/probe.c +14 -0
index 750f907..c6369dd 100644
@@ -674,6 +674,20 @@ static struct irq_domain *pci_host_bridge_msi_domain(struct pci_bus *bus)
 	 */
 	d = pci_host_bridge_of_msi_domain(bus);
 
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+	/*
+	 * If no IRQ domain was found via the OF tree, try looking it up
+	 * directly through the fwnode_handle.
+	 */
+	if (!d) {
+		struct fwnode_handle *fwnode = pci_root_bus_fwnode(bus);
+
+		if (fwnode)
+			d = irq_find_matching_fwnode(fwnode,
+						     DOMAIN_BUS_PCI_MSI);
+	}
+#endif
+
 	return d;
 }
 
Signature · 1 line
1.9.1
#7 lore raw

PATCH v9 6/7 drivers:hv: Define the channel type of Hyper-V PCI Express pass-through

In reply to [email protected]

From: Jake Oshins <[email protected]>

This defines the channel type for PCI front-ends in Hyper-V VMs.

---
 include/linux/hyperv.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
include/linux/hyperv.h +11 -0
index 24d0b65..c9a9eed 100644
@@ -1141,6 +1141,17 @@ u64 hv_do_hypercall(u64 control, void *input, void *output);
 		}
 
 /*
+ * PCI Express Pass Through
+ * {44C4F61D-4444-4400-9D52-802E27EDE19F}
+ */
+
+#define HV_PCIE_GUID \
+	.guid = { \
+			0x1D, 0xF6, 0xC4, 0x44, 0x44, 0x44, 0x00, 0x44, \
+			0x9D, 0x52, 0x80, 0x2E, 0x27, 0xED, 0xE1, 0x9F \
+		}
+
+/*
  * Common header for Hyper-V ICs
  */
 
Signature · 1 line
1.9.1
#8 lore raw

PATCH v9 7/7 PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

In reply to [email protected]

From: Jake Oshins <[email protected]>

This patch introduces a new driver which exposes a root PCI bus whenever a PCI Express device is passed through to a guest VM under Hyper-V. The device can be single- or multi-function. The interrupts for the devices are managed by an IRQ domain, implemented within the driver.

---
 MAINTAINERS                    |    1 +
 drivers/pci/Kconfig            |    7 +
 drivers/pci/host/Makefile      |    1 +
 drivers/pci/host/hv_pcifront.c | 2250 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 2259 insertions(+)
 create mode 100644 drivers/pci/host/hv_pcifront.c
MAINTAINERS +1 -0
index e6ec986..40747e8 100644
@@ -5115,6 +5115,7 @@ F:	arch/x86/kernel/cpu/mshyperv.c
 F:	drivers/hid/hid-hyperv.c
 F:	drivers/hv/
 F:	drivers/input/serio/hyperv-keyboard.c
+F:	drivers/pci/host/hv_pcifront.c
 F:	drivers/net/hyperv/
 F:	drivers/scsi/storvsc_drv.c
 F:	drivers/video/fbdev/hyperv_fb.c
drivers/pci/Kconfig +7 -0
index 73de4ef..573b8d6 100644
@@ -118,4 +118,11 @@ config PCI_LABEL
 	def_bool y if (DMI || ACPI)
 	select NLS
 
+config HYPERV_VPCI
+        tristate "Hyper-V PCI Frontend"
+        depends on PCI && X86 && HYPERV && PCI_MSI && PCI_MSI_IRQ_DOMAIN && X86_64
+        help
+          The PCI device frontend driver allows the kernel to import arbitrary
+          PCI devices from a PCI backend to support PCI driver domains.
+
 source "drivers/pci/host/Kconfig"
drivers/pci/host/Makefile +1 -0
index 9d4d3c6..d9abd2a 100644
@@ -2,6 +2,7 @@ obj-$(CONFIG_PCIE_DW) += pcie-designware.o
 obj-$(CONFIG_PCI_DRA7XX) += pci-dra7xx.o
 obj-$(CONFIG_PCI_EXYNOS) += pci-exynos.o
 obj-$(CONFIG_PCI_IMX6) += pci-imx6.o
+obj-$(CONFIG_HYPERV_VPCI) += hv_pcifront.o
 obj-$(CONFIG_PCI_MVEBU) += pci-mvebu.o
 obj-$(CONFIG_PCI_TEGRA) += pci-tegra.o
 obj-$(CONFIG_PCI_RCAR_GEN2) += pci-rcar-gen2.o
drivers/pci/host/hv_pcifront.c new file +2250 -0

2,250 lines not on this page · raw message

Signature · 1 line
1.9.1
#9 lore raw
reproduce:

Hi Jake,

[auto build test WARNING on pci/next]
[also build test WARNING on v4.4-rc4 next-20151209]

url:    https://github.com/0day-ci/linux/commits/jakeo-microsoft-com/PCI-hv-New-paravirtual-PCI-front-end-for-Hyper-V-VMs/20151210-070027
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: x86_64-allmodconfig (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All warnings (new ones prefixed by >>):

include/linux/compiler.h:228:8: sparse: attribute 'no_sanitize_address': unknown attribute drivers/pci/host/hv_pcifront.c: In function 'hv_pci_allocate_bridge_windows':

>> drivers/pci/host/hv_pcifront.c:1776:18: warning: unused variable 'length' [-Wunused-variable]
     resource_size_t length;
                     ^

vim +/length +1776 drivers/pci/host/hv_pcifront.c

1760 hbus->high_mmio_res->flags |= IORESOURCE_BUSY;
1761 release_mem_region(hbus->high_mmio_res->start,
1762 resource_size(hbus->high_mmio_res));
1763 }
1764 }
1765
1766 /**
1767 * hv_pci_allocate_bridge_windows() - Allocate memory regions
1768 * for the bus
1769 * @hbus: Root PCI bus, as understood by this driver
1770 *
1771 * Return: 0 on success, -errno on failure
1772 */
1773 static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)
1774 {
1775 resource_size_t align;

> 1776		resource_size_t length;

1777 int ret;
1778
1779 if (hbus->low_mmio_space) {
1780 align = 1ULL << (63 - __builtin_clzll(hbus->low_mmio_space));
1781 ret = vmbus_allocate_mmio(&hbus->low_mmio_res, hbus->hdev, 0,
1782 0xffffffffULL,
1783 (resource_size_t)hbus->low_mmio_space,
1784 align, false);

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#10 lore raw
In reply to [email protected]
reproduce:

Hi Jake,

[auto build test ERROR on pci/next]
[also build test ERROR on v4.4-rc4 next-20151209]

url:    https://github.com/0day-ci/linux/commits/jakeo-microsoft-com/PCI-hv-New-paravirtual-PCI-front-end-for-Hyper-V-VMs/20151210-070027
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: x86_64-randconfig-x006-12100835 (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

drivers/pci/host/hv_pcifront.c: In function 'hv_compose_msi_msg':

>> drivers/pci/host/hv_pcifront.c:830:4: error: 'apic' undeclared (first use in this function)
      (apic->irq_delivery_mode == dest_LowestPrio) ? 1 : 0;
       ^
   drivers/pci/host/hv_pcifront.c:830:4: note: each undeclared identifier is reported only once for each function it appears in
   drivers/pci/host/hv_pcifront.c: In function 'hv_pci_allocate_bridge_windows':
   drivers/pci/host/hv_pcifront.c:1776:18: warning: unused variable 'length' [-Wunused-variable]
     resource_size_t length;
                     ^

vim +/apic +830 drivers/pci/host/hv_pcifront.c

824 int_pkt = (struct pci_create_interrupt *)&ctxt.pkt.message;
825 int_pkt->message_type.message_type = PCI_CREATE_INTERRUPT_MESSAGE;
826 int_pkt->wslot.slot = hpdev->desc.win_slot.slot;
827 int_pkt->int_desc.vector = cfg->vector;
828 int_pkt->int_desc.vector_count = 1;
829 int_pkt->int_desc.delivery_mode =

 > 830			(apic->irq_delivery_mode == dest_LowestPrio) ? 1 : 0;

831
832 /*
833 * This bit doesn't have to work on machines with more than 64

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#11 lore raw
In reply to [email protected]
Quoted 71 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, December 9, 2015 2:55 PM
To: [email protected]; KY Srinivasan <[email protected]>; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Haiyang Zhang <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Jake Oshins <[email protected]>
Subject: [PATCH v9 0/7] PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

From: Jake Oshins <[email protected]>

This version of the patch series removes warning when compiling x86 32-bit.

First, export functions that allow correlating Hyper-V virtual processors and Linux cpus, along with the means for invoking a hypercall that targets interrupts at chosen vectors on specific cpus.

Second, mark various parts of IRQ domain related code as exported, so that this PCI front-end can implement an IRQ domain as part of a module. (The alternative would be to pull all tyhis into the kernel, which would pull in a lot of other Hyper-V related code, as this IRQ domain depends on vmbus.ko.)

Third, modify PCI so that new root PCI buses can be marked with an associated
fwnode_handle, and so that root PCI buses can look up their associated IRQ domain by that handle.

Fourth, introduce a new driver, hv_pcifront, which exposes root PCI buses in a Hyper-V VM. These root PCI buses expose real PCIe device, or PCI Virtual Functions.

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.
PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

 MAINTAINERS                    |    1 +
 arch/x86/include/asm/msi.h     |    6 +
 arch/x86/include/asm/pci.h     |   15 +
 arch/x86/kernel/apic/msi.c     |    8 +-
 arch/x86/kernel/apic/vector.c  |    2 +
 drivers/hv/hv.c                |   20 +-
 drivers/hv/hyperv_vmbus.h      |    2 +-
 drivers/hv/vmbus_drv.c         |   17 +
 drivers/pci/Kconfig            |    7 +
 drivers/pci/host/Makefile      |    1 +
 drivers/pci/host/hv_pcifront.c | 2250
++++++++++++++++++++++++++++++++++++++++
 drivers/pci/msi.c              |    4 +
 drivers/pci/probe.c            |   15 +
 include/linux/hyperv.h         |   14 +
 include/linux/pci.h            |    4 +
 kernel/irq/chip.c              |    1 +
 kernel/irq/irqdomain.c         |    4 +
 17 files changed, 2357 insertions(+), 14 deletions(-)
 create mode 100644 drivers/pci/host/hv_pcifront.c
Quoted 2 lines

--
1.9.1

Of these 7 patches, Greg has committed all of the VMBUS related supporting patches (3 patches). Thomas, can you take the IRQ related patches through your tree.

Regards,

K. Y

#12 lore raw
Quoted 4 lines · KY Srinivasan

Of these 7 patches, Greg has committed all of the VMBUS related supporting patches (3 patches). Thomas, can you take the IRQ related patches through your tree.

That does not make any sense.

Quoted 10 lines
Quoted 10 lines

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.
PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

That series cannot be ripped apart. The PCI part does not even compile w/o the drivers/hv part in place.

Thanks,

tglx

#13 lore raw
Quoted 33 lines

-----Original Message-----
From: Thomas Gleixner [mailto:[email protected]]
Sent: Saturday, December 19, 2015 3:27 AM
To: KY Srinivasan <[email protected]>
Cc: Jake Oshins <[email protected]>; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Haiyang Zhang <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v9 0/7] PCI: hv: New paravirtual PCI front-end for Hyper- V VMs

Quoted 4 lines · KY Srinivasan

Of these 7 patches, Greg has committed all of the VMBUS related supporting patches (3 patches). Thomas, can you take the IRQ related patches through your tree.

That does not make any sense.

Quoted 10 lines
Quoted 10 lines

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.
PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

Quoted 2 lines

That series cannot be ripped apart. The PCI part does not even compile w/o the drivers/hv part in place.

What I was proposing was that after you pick up the hv related patches in Greg's tree in the next cycle, would you be willing to take the IRQ related patches through your tree. Sorry for the confusion.

Regards,

K. Y

#14 lore raw
Quoted 20 lines · KY Srinivasan
Quoted 20 lines

From: Thomas Gleixner [mailto:[email protected]]

Quoted 4 lines · KY Srinivasan

Of these 7 patches, Greg has committed all of the VMBUS related supporting patches (3 patches). Thomas, can you take the IRQ related patches through your tree.

That does not make any sense.

Quoted 10 lines
Quoted 10 lines

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.
PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

Quoted 6 lines
Quoted 2 lines

That series cannot be ripped apart. The PCI part does not even compile w/o the drivers/hv part in place.

What I was proposing was that after you pick up the hv related patches in Greg's tree in the next cycle, would you be willing to take the IRQ related patches through your tree. Sorry for the confusion.

You mean: PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.

Right? Aside from the misleading subject line, I can pick that one up. The rest wants to go through the PCI tree.

Thanks,

tglx

#15 lore raw
Quoted 43 lines

-----Original Message-----
From: Thomas Gleixner [mailto:[email protected]]
Sent: Saturday, December 19, 2015 12:21 PM
To: KY Srinivasan <[email protected]>
Cc: Jake Oshins <[email protected]>; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Haiyang Zhang <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v9 0/7] PCI: hv: New paravirtual PCI front-end for Hyper- V VMs

Quoted 7 lines · KY Srinivasan
Quoted 7 lines

From: Thomas Gleixner [mailto:[email protected]]

Quoted 4 lines · KY Srinivasan

Of these 7 patches, Greg has committed all of the VMBUS related supporting patches (3 patches). Thomas, can you take the IRQ related patches through your tree.

Quoted 7 lines
Quoted 7 lines

That does not make any sense.

Quoted 5 lines
Quoted 5 lines

Jake Oshins (7):
drivers:hv: Export a function that maps Linux CPU num onto Hyper-V
proc num
drivers:hv: Export hv_do_hypercall()
PCI: Make it possible to implement a PCI MSI IRQ Domain in a

module.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

PCI: Add fwnode_handle to pci_sysdata
PCI: irqdomain: Look up IRQ domain by fwnode_handle drivers:hv: Define the channel type of Hyper-V PCI Express
pass-through
PCI: hv: New paravirtual PCI front-end for Hyper-V VMs

Quoted 6 lines
Quoted 2 lines

That series cannot be ripped apart. The PCI part does not even compile w/o the drivers/hv part in place.

What I was proposing was that after you pick up the hv related patches in Greg's tree in the next cycle, would you be willing to take the IRQ related patches through your tree. Sorry for the confusion.

Quoted 5 lines

You mean: PCI: Make it possible to implement a PCI MSI IRQ Domain in a module.

Right? Aside from the misleading subject line, I can pick that one up. The rest wants to go through the PCI tree.

That is what I meant. Thanks Thomas.

K. Y