Skip to contentSkip to conversation

Hyper-V vPCI: use vPCI protocol version 1.2

PATCH In review Jork Loeser 3 participants 9 messages May 24, 2017 – May 30, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Hyper-V vPCI: use vPCI protocol version 1.2

From: Jork Loeser <[email protected]>

Update the Hyper-V vPCI driver to use the Server-2016 version of the vPCI protocol, fixing MSI creation and retargeting issues.

Changes since v1:
- reduced spew in protocol negotiation (Dan Carpenter)
- work-around work racing Hyper-V patch (Stephen Hemminger)
- formatting (Dan)

Jork Loeser (5):
Hyper-V vPCI: Minor format and semantic fix
Hyper-V vPCI: Use page allocation for hbus structure PCI-HyperV vPCI: Temporary own CPU-number-to-vCPU-number infra Hyper-V vPCI: Add vPCI version protocol negotiation Hyper-V vPCI: use vPCI protocol version 1.2

 arch/x86/include/uapi/asm/hyperv.h |   6 +
 drivers/pci/host/pci-hyperv.c      | 445 ++++++++++++++++++++++++++++++-------
 2 files changed, 370 insertions(+), 81 deletions(-)
Signature · 1 line
2.7.4
#1 lore raw

PATCH v2 1/5 Hyper-V vPCI: Minor format and semantic fix

From: Jork Loeser <[email protected]>

Fix comment formatting and use proper integer fields.

---
 drivers/pci/host/pci-hyperv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/pci/host/pci-hyperv.c +5 -5
index 8493638..7bebdc6 100644
@@ -245,7 +245,7 @@ struct pci_packet {
 
 struct pci_version_request {
 	struct pci_message message_type;
-	enum pci_message_type protocol_version;
+	u32 protocol_version;
 } __packed;
 
 /*
@@ -1513,12 +1513,12 @@ static void pci_devices_present_work(struct work_struct *work)
 		put_pcichild(hpdev, hv_pcidev_ref_initial);
 	}
 
-	switch(hbus->state) {
+	switch (hbus->state) {
 	case hv_pcibus_installed:
 		/*
-		* Tell the core to rescan bus
-		* because there may have been changes.
-		*/
+		 * Tell the core to rescan bus
+		 * because there may have been changes.
+		 */
 		pci_lock_rescan_remove();
 		pci_scan_child_bus(hbus->pci_bus);
 		pci_unlock_rescan_remove();
Signature · 1 line
2.7.4
#2 lore raw

PATCH v2 2/5 Hyper-V vPCI: Use page allocation for hbus structure

In reply to Jork Loeser

From: Jork Loeser <[email protected]>

The hv_pcibus_device structure contains an in-memory hypercall argument that must not cross a page boundary. Allocate the structure as a page to ensure that.

---
 drivers/pci/host/pci-hyperv.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
drivers/pci/host/pci-hyperv.c +12 -3
index 7bebdc6..6e4b026 100644
@@ -382,7 +382,10 @@ struct hv_pcibus_device {
 	struct msi_domain_info msi_info;
 	struct msi_controller msi_chip;
 	struct irq_domain *irq_domain;
+
+	/* hypercall arg, must not cross page boundary */
 	struct retarget_msi_interrupt retarget_msi_interrupt_params;
+
 	spinlock_t retarget_msi_interrupt_lock;
 };
 
@@ -2204,7 +2207,13 @@ static int hv_pci_probe(struct hv_device *hdev,
 	struct hv_pcibus_device *hbus;
 	int ret;
 
-	hbus = kzalloc(sizeof(*hbus), GFP_KERNEL);
+	/*
+	 * hv_pcibus_device contains the hypercall arguments for retargeting in
+	 * hv_irq_unmask(). Those must not cross a page boundary.
+	 */
+	BUILD_BUG_ON(sizeof(*hbus) > PAGE_SIZE);
+
+	hbus = (struct hv_pcibus_device *)get_zeroed_page(GFP_KERNEL);
 	if (!hbus)
 		return -ENOMEM;
 	hbus->state = hv_pcibus_init;
@@ -2308,7 +2317,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 close:
 	vmbus_close(hdev->channel);
 free_bus:
-	kfree(hbus);
+	free_page((unsigned long)hbus);
 	return ret;
 }
 
@@ -2386,7 +2395,7 @@ static int hv_pci_remove(struct hv_device *hdev)
 	irq_domain_free_fwnode(hbus->sysdata.fwnode);
 	put_hvpcibus(hbus);
 	wait_for_completion(&hbus->remove_event);
-	kfree(hbus);
+	free_page((unsigned long)hbus);
 	return 0;
 }
 
Signature · 1 line
2.7.4
#3 lore raw

PATCH v2 3/5 PCI-HyperV vPCI: Temporary own CPU-number-to-vCPU-number infra

In reply to Jork Loeser

From: Jork Loeser <[email protected]>

To ease parallel effort to centralize CPU-number-to-vCPU-number conversion, temporarily stand up own version, file-local hv_tmp_cpu_nr_to_vp_nr(). Once the changes have merged, this work-around can be removed, and the calls replaced with hv_cpu_number_to_vp_number().

---
 drivers/pci/host/pci-hyperv.c | 52 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 50 insertions(+), 2 deletions(-)
drivers/pci/host/pci-hyperv.c +50 -2
index 6e4b026..ee70b1f 100644
@@ -479,6 +479,52 @@ static void put_pcichild(struct hv_pci_dev *hv_pcidev,
 static void get_hvpcibus(struct hv_pcibus_device *hv_pcibus);
 static void put_hvpcibus(struct hv_pcibus_device *hv_pcibus);
 
+
+/*
+ * Temporary CPU to vCPU mapping to address transitioning
+ * vmbus_cpu_number_to_vp_number() being migrated to
+ * hv_cpu_number_to_vp_number() in a separate patch. Once that patch
+ * has been picked up in the main line, remove this code here and use
+ * the offical code.
+ */
+static struct hv_tmpcpumap
+{
+	bool initialized;
+	u32 vp_index[NR_CPUS];
+} hv_tmpcpumap;
+
+static void hv_tmpcpumap_init_cpu(void *_unused)
+{
+	int cpu = smp_processor_id();
+	u64 vp_index;
+
+	hv_get_vp_index(vp_index);
+
+	hv_tmpcpumap.vp_index[cpu] = vp_index;
+}
+
+static void hv_tmpcpumap_init(void)
+{
+	if (hv_tmpcpumap.initialized)
+		return;
+
+	memset(hv_tmpcpumap.vp_index, -1, sizeof(hv_tmpcpumap.vp_index));
+	on_each_cpu(hv_tmpcpumap_init_cpu, NULL, true);
+	hv_tmpcpumap.initialized = true;
+}
+
+/**
+ * hv_tmp_cpu_nr_to_vp_nr() - Convert Linux CPU nr to Hyper-V vCPU nr
+ *
+ * Remove once vmbus_cpu_number_to_vp_number() has been converted to
+ * hv_cpu_number_to_vp_number() and replace callers appropriately.
+ */
+static u32 hv_tmp_cpu_nr_to_vp_nr(int cpu)
+{
+	return hv_tmpcpumap.vp_index[cpu];
+}
+
+
 /**
  * devfn_to_wslot() - Convert from Linux PCI slot to Windows
  * @devfn:	The Linux representation of PCI slot
@@ -813,7 +859,7 @@ static void hv_irq_unmask(struct irq_data *data)
 	params->vector = cfg->vector;
 
 	for_each_cpu_and(cpu, dest, cpu_online_mask)
-		params->vp_mask |= (1ULL << vmbus_cpu_number_to_vp_number(cpu));
+		params->vp_mask |= (1ULL << hv_tmp_cpu_nr_to_vp_nr(cpu));
 
 	hv_do_hypercall(HVCALL_RETARGET_INTERRUPT, params, NULL);
 
@@ -908,7 +954,7 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	} else {
 		for_each_cpu_and(cpu, affinity, cpu_online_mask) {
 			int_pkt->int_desc.cpu_mask |=
-				(1ULL << vmbus_cpu_number_to_vp_number(cpu));
+				(1ULL << hv_tmp_cpu_nr_to_vp_nr(cpu));
 		}
 	}
 
@@ -2218,6 +2264,8 @@ static int hv_pci_probe(struct hv_device *hdev,
 		return -ENOMEM;
 	hbus->state = hv_pcibus_init;
 
+	hv_tmpcpumap_init();
+
 	/*
 	 * The PCI bus "domain" is what is called "segment" in ACPI and
 	 * other specs.  Pull it from the instance ID, to get something
Signature · 1 line
2.7.4
#4 lore raw

PATCH v2 4/5 Hyper-V vPCI: Add vPCI version protocol negotiation

In reply to Jork Loeser

From: Jork Loeser <[email protected]>

Hyper-V vPCI offers different protocol versions. This patch creates the the infra for negotiating the one to use.

---
 drivers/pci/host/pci-hyperv.c | 72 +++++++++++++++++++++++++++++++------------
 1 file changed, 53 insertions(+), 19 deletions(-)
drivers/pci/host/pci-hyperv.c +53 -19
index ee70b1f..2d69b00 100644
@@ -64,22 +64,37 @@
  * major version.
  */
 
-#define PCI_MAKE_VERSION(major, minor) ((u32)(((major) << 16) | (major)))
+#define PCI_MAKE_VERSION(major, minor) ((u32)(((major) << 16) | (minor)))
 #define PCI_MAJOR_VERSION(version) ((u32)(version) >> 16)
 #define PCI_MINOR_VERSION(version) ((u32)(version) & 0xff)
 
-enum {
-	PCI_PROTOCOL_VERSION_1_1 = PCI_MAKE_VERSION(1, 1),
-	PCI_PROTOCOL_VERSION_CURRENT = PCI_PROTOCOL_VERSION_1_1
+enum pci_protocol_version_t {
+	PCI_PROTOCOL_VERSION_1_1 = PCI_MAKE_VERSION(1, 1),	/* Win10 */
 };
 
 #define CPU_AFFINITY_ALL	-1ULL
+
+/*
+ * Supported protocol versions in the order of probing - highest go
+ * first.
+ */
+static enum pci_protocol_version_t pci_protocol_versions[] = {
+	PCI_PROTOCOL_VERSION_1_1,
+};
+
+/*
+ * Protocol version negotiated by hv_pci_protocol_negotiation().
+ */
+static enum pci_protocol_version_t pci_protocol_version;
+
 #define PCI_CONFIG_MMIO_LENGTH	0x2000
 #define CFG_PAGE_OFFSET 0x1000
 #define CFG_PAGE_SIZE (PCI_CONFIG_MMIO_LENGTH - CFG_PAGE_OFFSET)
 
 #define MAX_SUPPORTED_MSI_MESSAGES 0x400
 
+#define STATUS_REVISION_MISMATCH 0xC0000059
+
 /*
  * Message Types
  */
@@ -1849,6 +1864,7 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev)
 	struct hv_pci_compl comp_pkt;
 	struct pci_packet *pkt;
 	int ret;
+	int i;
 
 	/*
 	 * Initiate the handshake with the host and negotiate
@@ -1865,26 +1881,44 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev)
 	pkt->compl_ctxt = &comp_pkt;
 	version_req = (struct pci_version_request *)&pkt->message;
 	version_req->message_type.type = PCI_QUERY_PROTOCOL_VERSION;
-	version_req->protocol_version = PCI_PROTOCOL_VERSION_CURRENT;
 
-	ret = vmbus_sendpacket(hdev->channel, version_req,
-			       sizeof(struct pci_version_request),
-			       (unsigned long)pkt, VM_PKT_DATA_INBAND,
-			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
-	if (ret)
-		goto exit;
+	for (i = 0; i < ARRAY_SIZE(pci_protocol_versions); i++) {
+		version_req->protocol_version = pci_protocol_versions[i];
+		ret = vmbus_sendpacket(hdev->channel, version_req,
+				sizeof(struct pci_version_request),
+				(unsigned long)pkt, VM_PKT_DATA_INBAND,
+				VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
+		if (ret) {
+			dev_err(&hdev->device,
+				"PCI Pass-through VSP failed sending version reqquest: %#x",
+				ret);
+			goto exit;
+		}
 
-	wait_for_completion(&comp_pkt.host_event);
+		wait_for_completion(&comp_pkt.host_event);
 
-	if (comp_pkt.completion_status < 0) {
-		dev_err(&hdev->device,
-			"PCI Pass-through VSP failed version request %x\n",
-			comp_pkt.completion_status);
-		ret = -EPROTO;
-		goto exit;
+		if (comp_pkt.completion_status >= 0) {
+			pci_protocol_version = pci_protocol_versions[i];
+			dev_info(&hdev->device,
+				"PCI VMBus probing: Using version %#x\n",
+				pci_protocol_version);
+			goto exit;
+		}
+
+		if (comp_pkt.completion_status != STATUS_REVISION_MISMATCH) {
+			dev_err(&hdev->device,
+				"PCI Pass-through VSP failed version request: %#x",
+				comp_pkt.completion_status);
+			ret = -EPROTO;
+			goto exit;
+		}
+
+		reinit_completion(&comp_pkt.host_event);
 	}
 
-	ret = 0;
+	dev_err(&hdev->device,
+		"PCI pass-through VSP failed to find supported version");
+	ret = -EPROTO;
 
 exit:
 	kfree(pkt);
Signature · 1 line
2.7.4
#5 lore raw

PATCH v2 5/5 Hyper-V vPCI: use vPCI protocol version 1.2

In reply to Jork Loeser

From: Jork Loeser <[email protected]>

Update the Hyper-V vPCI driver to use the Server-2016 version of the vPCI protocol, fixing MSI creation and retargeting issues.

---
 arch/x86/include/uapi/asm/hyperv.h |   6 +
 drivers/pci/host/pci-hyperv.c      | 300 ++++++++++++++++++++++++++++++-------
 2 files changed, 252 insertions(+), 54 deletions(-)
arch/x86/include/uapi/asm/hyperv.h +6 -0
index 432df4b..237ec6c 100644
@@ -153,6 +153,12 @@
 #define HV_X64_DEPRECATING_AEOI_RECOMMENDED	(1 << 9)
 
 /*
+ * HV_VP_SET available
+ */
+#define HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED	(1 << 11)
+
+
+/*
  * Crash notification flag.
  */
 #define HV_CRASH_CTL_CRASH_NOTIFY (1ULL << 63)
drivers/pci/host/pci-hyperv.c +246 -54

422 lines not on this page · raw message

Signature · 1 line
2.7.4
#6 lore raw
In reply to Jork Loeser
Quoted 17 lines · Jork Loeser

From: Jork Loeser <[email protected]>

Update the Hyper-V vPCI driver to use the Server-2016 version of the vPCI protocol, fixing MSI creation and retargeting issues.

Changes since v1:
- reduced spew in protocol negotiation (Dan Carpenter)
- work-around work racing Hyper-V patch (Stephen Hemminger)
- formatting (Dan)

Jork Loeser (5):
Hyper-V vPCI: Minor format and semantic fix
Hyper-V vPCI: Use page allocation for hbus structure PCI-HyperV vPCI: Temporary own CPU-number-to-vCPU-number infra Hyper-V vPCI: Add vPCI version protocol negotiation Hyper-V vPCI: use vPCI protocol version 1.2

Quoted 3 lines
 arch/x86/include/uapi/asm/hyperv.h |   6 +
 drivers/pci/host/pci-hyperv.c      | 445 ++++++++++++++++++++++++++++++-------
 2 files changed, 370 insertions(+), 81 deletions(-)

Waiting for a maintainer ack:

Hyper-V CORE AND DRIVERS

  M:      "K. Y. Srinivasan" <[email protected]>
  M:      Haiyang Zhang <[email protected]>
  M:      Stephen Hemminger <[email protected]>

...
F: drivers/pci/host/pci-hyperv.c

Reviewed-by Acked-by
#7 lore raw
Quoted 18 lines

-----Original Message-----
From: devel [mailto:[email protected]] On Behalf Of Bjorn Helgaas
Sent: Tuesday, May 30, 2017 12:45 PM
To: Jork Loeser <[email protected]>
Cc: [email protected]; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH-v2 0/5] Hyper-V vPCI: use vPCI protocol version 1.2 =20

Quoted 3 lines · Jork Loeser

From: Jork Loeser <[email protected]>

Update the Hyper-V vPCI driver to use the Server-2016 version of the vP=

CI

Quoted 14 lines
Quoted 14 lines

protocol, fixing MSI creation and retargeting issues.

Changes since v1:
- reduced spew in protocol negotiation (Dan Carpenter)
- work-around work racing Hyper-V patch (Stephen Hemminger)
- formatting (Dan)

Jork Loeser (5):
Hyper-V vPCI: Minor format and semantic fix
Hyper-V vPCI: Use page allocation for hbus structure PCI-HyperV vPCI: Temporary own CPU-number-to-vCPU-number infra Hyper-V vPCI: Add vPCI version protocol negotiation Hyper-V vPCI: use vPCI protocol version 1.2

Quoted 6 lines
Quoted 2 lines
 arch/x86/include/uapi/asm/hyperv.h |   6 +
 drivers/pci/host/pci-hyperv.c      | 445

++++++++++++++++++++++++++++++-------

Quoted 1 line
 2 files changed, 370 insertions(+), 81 deletions(-)

=20
Waiting for a maintainer ack:

Quoted 16 lines

=20
Hyper-V CORE AND DRIVERS

  M:      "K. Y. Srinivasan" <[email protected]>
  M:      Haiyang Zhang <[email protected]>
  M:      Stephen Hemminger <[email protected]>

...
F: drivers/pci/host/pci-hyperv.c
_______________________________________________
devel mailing list
[email protected]
https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fdriverd
ev.linuxdriverproject.org%2Fmailman%2Flistinfo%2Fdriverdev- devel&data=3D02%7C01%7Ckys%40microsoft.com%7C08ff2147e87e4b9957090 8d4a794658f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6363177 03190907906&sdata=3DET7UEDnBub1iNEOAb509JrjEZxKZjyhmLSubVzYZtic%3D &reserved=3D0

#8 lore raw
In reply to Jork Loeser
Quoted 17 lines · Jork Loeser

From: Jork Loeser <[email protected]>

Update the Hyper-V vPCI driver to use the Server-2016 version of the vPCI protocol, fixing MSI creation and retargeting issues.

Changes since v1:
- reduced spew in protocol negotiation (Dan Carpenter)
- work-around work racing Hyper-V patch (Stephen Hemminger)
- formatting (Dan)

Jork Loeser (5):
Hyper-V vPCI: Minor format and semantic fix
Hyper-V vPCI: Use page allocation for hbus structure PCI-HyperV vPCI: Temporary own CPU-number-to-vCPU-number infra Hyper-V vPCI: Add vPCI version protocol negotiation Hyper-V vPCI: use vPCI protocol version 1.2

Quoted 3 lines
 arch/x86/include/uapi/asm/hyperv.h |   6 +
 drivers/pci/host/pci-hyperv.c      | 445 ++++++++++++++++++++++++++++++-------
 2 files changed, 370 insertions(+), 81 deletions(-)

Applied with K.Y.'s ack to pci/host-hv for v4.13, thanks!