Skip to contentSkip to conversation

Drivers: hv: Update the Vmbus protocol

PATCH In review K. Y. Srinivasan 2 participants 32 messages Dec 1, 2012 – Jan 17, 2013 Hyper-V/Azure core and drivers

lore
Share & tools
lore
#0 lore raw

cover Drivers: hv: Update the Vmbus protocol

The current vmbus protocol supported in the vmbus driver for Linux is a version of the protocol that shipped with Windows Server 2008. Since then a lot of enhancements have been made in the protocol to specifically address performance and scalability of the communication infrastructure between the host and the guest.
This patch-set brings our driver to the Win8 (Windows Server 2012) level. Furthermore, this code will dynamically negotiate the most efficient protocol that the host can support - the same code can be deployed on all supported hosts (WS2008, WS2008R2 and WS2012).
Following are some of the key features implemented in this patch-set:

1. More efficient signaling protocol between the host and the guest
2. Distribution of interrupt load across available CPUs in the guest; currently all vmbus interrupts are delivered to the boot CPU (CPU 0) in the guest.
3. Per- channel interrupt binding (as part of item 2)
4. More efficient demultiplexing of incoming interrupts
5. Per-channel signaling mechanism for host to guest communication

K. Y. Srinivasan (28):
Drivers: hv: Implement routines for read side signaling optimization
Drivers: hv: Add state to manage batched reading
Drivers: hv: Turn off batched reading for util drivers
Drivers: hv: Optimize signaling in the read path
Drivers: hv: Optimize the signaling on the write path
Drivers: hv: Get rid of hv_get_ringbuffer_interrupt_mask()
Drivers: hv: Support handling multiple VMBUS versions
Drivers: hv: Update the ring buffer structure to match win8
functionality
Drivers: hv: Extend/modify vmbus_channel_offer_channel for win7 and
beyond
Drivers: hv: Save and export negotiated vmbus version
Drivers: hv: Change the signature for hv_signal_event()
Drivers: hv: Change the signature of vmbus_set_event()
Drivers: hv: Move vmbus version definitions to hyperv.h
Drivers: hv: Manage signaling state on a per-connection basis
Drivers: hv: Cleanup vmbus_set_event() to support win7 and beyond
Drivers: hv: Setup a mapping for Hyper-V's notion cpu ID
Drivers: hv: Add state to manage incoming channel interrupt load
Drivers: hv: Modify the interrupt handling code to support win8 and
beyond
Drivers: hv: Add code to distribute channel interrupt load
Drivers: hv: Get rid of the unused global signaling state
Drivers: hv: Get rid of unnecessary request for offers
Drivers: hv: Manage event tasklets on per-cpu basis
Drivers: hv: Handle vmbus interrupts concurrently on all cpus
Drivers: hv: Add a check to deal with spurious interrupts
Drivers: hv: Enable protocol negotiation with win8 hosts
Drivers: hv: Implement flow management on the send side
Drivers: hv: Capture the host build information
Drivers: hv: Cleanup and consolidate reporting of build/version info

 drivers/hv/channel.c      |   31 ++++--
 drivers/hv/channel_mgmt.c |  112 ++++++++++++++++++++++
 drivers/hv/connection.c   |  232 ++++++++++++++++++++++++++++++++-------------
 drivers/hv/hv.c           |   72 +++++++-------
 drivers/hv/hv_util.c      |   10 ++
 drivers/hv/hyperv_vmbus.h |   65 ++++++-------
 drivers/hv/ring_buffer.c  |  130 ++++++++++++++++++++++----
 drivers/hv/vmbus_drv.c    |   54 +++++++++--
 include/linux/hyperv.h    |  154 ++++++++++++++++++++++++++----
 9 files changed, 671 insertions(+), 189 deletions(-)
Signature · 1 line
1.7.4.1
#1 lore raw

PATCH 1/28 Drivers: hv: Implement routines for read side signaling optimization

Implement functions that will support read-side signaling optimization. By having the reader indicate the start of the "read" operation and the "end" of the read operation we can more efficiently handle the signaling
protocol: while the read is in progress, there is no need for the "writer" to signal the "reader" as new items are put on the read queue.

---
 drivers/hv/hyperv_vmbus.h |    4 ++++
 drivers/hv/ring_buffer.c  |   24 ++++++++++++++++++++++++
 2 files changed, 28 insertions(+), 0 deletions(-)
drivers/hv/hyperv_vmbus.h +4 -0
index d8d1fad..3184f6f 100644
@@ -570,6 +570,10 @@ u32 hv_get_ringbuffer_interrupt_mask(struct hv_ring_buffer_info *ring_info);
 void hv_ringbuffer_get_debuginfo(struct hv_ring_buffer_info *ring_info,
 			    struct hv_ring_buffer_debug_info *debug_info);
 
+void hv_begin_read(struct hv_ring_buffer_info *rbi);
+
+u32 hv_end_read(struct hv_ring_buffer_info *rbi);
+
 /*
  * Maximum channels is determined by the size of the interrupt page
  * which is PAGE_SIZE. 1/2 of PAGE_SIZE is for send endpoint interrupt
drivers/hv/ring_buffer.c +24 -0
index 7233c88..0010792 100644
@@ -29,6 +29,30 @@
 
 #include "hyperv_vmbus.h"
 
+void hv_begin_read(struct hv_ring_buffer_info *rbi)
+{
+	rbi->ring_buffer->interrupt_mask = 1;
+	smp_mb();
+}
+
+u32 hv_end_read(struct hv_ring_buffer_info *rbi)
+{
+	u32 read;
+	u32 write;
+
+	rbi->ring_buffer->interrupt_mask = 0;
+	smp_mb();
+
+	/*
+	 * Now check to see if the ring buffer is still empty.
+	 * If it is not, we raced and we need to process new
+	 * incoming messages.
+	 */
+	hv_get_ringbuffer_availbytes(rbi, &read, &write);
+
+	return read;
+}
+
 
 /*
  * hv_get_next_write_location()
Signature · 1 line
1.7.4.1
#2 lore raw

PATCH 2/28 Drivers: hv: Add state to manage batched reading

For the "read" side signaling optimization, the reader has to completely drain the queue before exiting. Add state to manage this "batched" reading.

---
 drivers/hv/channel_mgmt.c |    7 +++++++
 include/linux/hyperv.h    |   20 ++++++++++++++++++++
 2 files changed, 27 insertions(+), 0 deletions(-)
drivers/hv/channel_mgmt.c +7 -0
index 2f84c5c..7bf5917 100644
@@ -275,6 +275,13 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
 		return;
 	}
 
+	/*
+	 * By default we setup state to enable batched
+	 * reading. A specific service can choose to
+	 * disable this prior to opening the channel.
+	 */
+	newchannel->batched_reading = true;
+
 	memcpy(&newchannel->offermsg, offer,
 	       sizeof(struct vmbus_channel_offer_channel));
 	newchannel->monitor_grp = (u8)offer->monitorid / 32;
include/linux/hyperv.h +20 -0
index e73b852..1ffe84d 100644
@@ -882,8 +882,28 @@ struct vmbus_channel {
 
 	void (*onchannel_callback)(void *context);
 	void *channel_callback_context;
+
+	/*
+	 * A channel can be marked for efficient (batched)
+	 * reading:
+	 * If batched_reading is set to "true", we read until the
+	 * channel is empty and hold off interrupts from the host
+	 * during the entire read process.
+	 * If batched_reading is set to "false", the client is not
+	 * going to perform batched reading.
+	 *
+	 * By default we will enable batched reading; specific
+	 * drivers that don't want this behavior can turn it off.
+	 */
+
+	bool batched_reading;
 };
 
+static inline void set_channel_read_state(struct vmbus_channel *c, bool state)
+{
+	c->batched_reading = state;
+}
+
 void vmbus_onmessage(void *context);
 
 int vmbus_request_offers(void);
Signature · 1 line
1.7.4.1
#3 lore raw

PATCH 3/28 Drivers: hv: Turn off batched reading for util drivers

In reply to K. Y. Srinivasan

Util driver is not a performance critical driver and furthermore some util services such as KVP can only handle one outstanding message from the host. Turn off batched reading for util drivers.

---
 drivers/hv/hv_util.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
drivers/hv/hv_util.c +10 -0
index a0667de..a62a760 100644
@@ -274,6 +274,16 @@ static int util_probe(struct hv_device *dev,
 		}
 	}
 
+	/*
+	 * The set of services managed by the util driver are not performance
+	 * critical and do not need batched reading. Furthermore, some services
+	 * such as KVP can only handle one message from the host at a time.
+	 * Turn off batched reading for all util drivers before we open the
+	 * channel.
+	 */
+
+	set_channel_read_state(dev->channel, false);
+
 	ret = vmbus_open(dev->channel, 4 * PAGE_SIZE, 4 * PAGE_SIZE, NULL, 0,
 			srv->util_cb, dev->channel);
 	if (ret)
Signature · 1 line
1.7.4.1
#4 lore raw

PATCH 4/28 Drivers: hv: Optimize signaling in the read path

In reply to K. Y. Srinivasan

Now that we have the infratructure for correctly determining when we should signal the host; optimize the signaling on the read side - signaling the guest from the host.

---
 drivers/hv/connection.c |   28 +++++++++++++++++++++++++---
 1 files changed, 25 insertions(+), 3 deletions(-)
drivers/hv/connection.c +25 -3
index 650c9f0..d1019a7 100644
@@ -212,6 +212,9 @@ static void process_chn_event(u32 relid)
 {
 	struct vmbus_channel *channel;
 	unsigned long flags;
+	void *arg;
+	bool read_state;
+	u32 bytes_to_read;
 
 	/*
 	 * Find the channel based on this relid and invokes the
@@ -234,10 +237,29 @@ static void process_chn_event(u32 relid)
 	 */
 
 	spin_lock_irqsave(&channel->inbound_lock, flags);
-	if (channel->onchannel_callback != NULL)
-		channel->onchannel_callback(channel->channel_callback_context);
-	else
+	if (channel->onchannel_callback != NULL) {
+		arg = channel->channel_callback_context;
+		read_state = channel->batched_reading;
+		/*
+		 * This callback reads the messages sent by the host.
+		 * We can optimize host to guest signaling by ensuring:
+		 * 1. While reading the channel, we disable interrupts from
+		 *    host.
+		 * 2. Ensure that we process all posted messages from the host
+		 *    before returning from this callback.
+		 * 3. Once we return, enable signaling from the host. Once this
+		 *    state is set we check to see if additional packets are
+		 *    available to read. In this case we repeat the process.
+		 */
+
+		do {
+			hv_begin_read(&channel->inbound);
+			channel->onchannel_callback(arg);
+			bytes_to_read = hv_end_read(&channel->inbound);
+		} while (read_state && (bytes_to_read != 0));
+	} else {
 		pr_err("no channel callback for relid - %u\n", relid);
+	}
 
 	spin_unlock_irqrestore(&channel->inbound_lock, flags);
 }
Signature · 1 line
1.7.4.1
#5 lore raw

PATCH 5/28 Drivers: hv: Optimize the signaling on the write path

In reply to K. Y. Srinivasan

The host has already implemented the "read" side optimizations. Leverage that to optimize "write" side signaling.

---
 drivers/hv/channel.c      |   15 +++++++++------
 drivers/hv/hyperv_vmbus.h |    2 +-
 drivers/hv/ring_buffer.c  |   42 +++++++++++++++++++++++++++++++++++++++---
 3 files changed, 49 insertions(+), 10 deletions(-)
drivers/hv/channel.c +9 -6
index 773a2f2..727c5f1 100644
@@ -564,6 +564,7 @@ int vmbus_sendpacket(struct vmbus_channel *channel, const void *buffer,
 	struct scatterlist bufferlist[3];
 	u64 aligned_data = 0;
 	int ret;
+	bool signal = false;
 
 
 	/* Setup the descriptor */
@@ -580,9 +581,9 @@ int vmbus_sendpacket(struct vmbus_channel *channel, const void *buffer,
 	sg_set_buf(&bufferlist[2], &aligned_data,
 		   packetlen_aligned - packetlen);
 
-	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3);
+	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3, &signal);
 
-	if (ret == 0 && !hv_get_ringbuffer_interrupt_mask(&channel->outbound))
+	if (ret == 0 && signal)
 		vmbus_setevent(channel);
 
 	return ret;
@@ -606,6 +607,7 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel,
 	u32 packetlen_aligned;
 	struct scatterlist bufferlist[3];
 	u64 aligned_data = 0;
+	bool signal = false;
 
 	if (pagecount > MAX_PAGE_BUFFER_COUNT)
 		return -EINVAL;
@@ -641,9 +643,9 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel,
 	sg_set_buf(&bufferlist[2], &aligned_data,
 		packetlen_aligned - packetlen);
 
-	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3);
+	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3, &signal);
 
-	if (ret == 0 && !hv_get_ringbuffer_interrupt_mask(&channel->outbound))
+	if (ret == 0 && signal)
 		vmbus_setevent(channel);
 
 	return ret;
@@ -665,6 +667,7 @@ int vmbus_sendpacket_multipagebuffer(struct vmbus_channel *channel,
 	u32 packetlen_aligned;
 	struct scatterlist bufferlist[3];
 	u64 aligned_data = 0;
+	bool signal = false;
 	u32 pfncount = NUM_PAGES_SPANNED(multi_pagebuffer->offset,
 					 multi_pagebuffer->len);
 
@@ -703,9 +706,9 @@ int vmbus_sendpacket_multipagebuffer(struct vmbus_channel *channel,
 	sg_set_buf(&bufferlist[2], &aligned_data,
 		packetlen_aligned - packetlen);
 
-	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3);
+	ret = hv_ringbuffer_write(&channel->outbound, bufferlist, 3, &signal);
 
-	if (ret == 0 && !hv_get_ringbuffer_interrupt_mask(&channel->outbound))
+	if (ret == 0 && signal)
 		vmbus_setevent(channel);
 
 	return ret;
drivers/hv/hyperv_vmbus.h +1 -1
index 3184f6f..895c898 100644
@@ -555,7 +555,7 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info);
 
 int hv_ringbuffer_write(struct hv_ring_buffer_info *ring_info,
 		    struct scatterlist *sglist,
-		    u32 sgcount);
+		    u32 sgcount, bool *signal);
 
 int hv_ringbuffer_peek(struct hv_ring_buffer_info *ring_info, void *buffer,
 		   u32 buflen);
drivers/hv/ring_buffer.c +39 -3

78 lines not on this page · raw message

Signature · 1 line
1.7.4.1
#6 lore raw

PATCH 6/28 Drivers: hv: Get rid of hv_get_ringbuffer_interrupt_mask()

In reply to K. Y. Srinivasan

This function is no longer used; get rid of it.

---
 drivers/hv/hyperv_vmbus.h |    1 -
 drivers/hv/ring_buffer.c  |   13 -------------
 2 files changed, 0 insertions(+), 14 deletions(-)
drivers/hv/hyperv_vmbus.h +0 -1
index 895c898..ac1e419 100644
@@ -565,7 +565,6 @@ int hv_ringbuffer_read(struct hv_ring_buffer_info *ring_info,
 		   u32 buflen,
 		   u32 offset);
 
-u32 hv_get_ringbuffer_interrupt_mask(struct hv_ring_buffer_info *ring_info);
 
 void hv_ringbuffer_get_debuginfo(struct hv_ring_buffer_info *ring_info,
 			    struct hv_ring_buffer_debug_info *debug_info);
drivers/hv/ring_buffer.c +0 -13
index 9bc55f0..2a0babc 100644
@@ -294,19 +294,6 @@ void hv_ringbuffer_get_debuginfo(struct hv_ring_buffer_info *ring_info,
 	}
 }
 
-
-/*
- *
- * hv_get_ringbuffer_interrupt_mask()
- *
- * Get the interrupt mask for the specified ring buffer
- *
- */
-u32 hv_get_ringbuffer_interrupt_mask(struct hv_ring_buffer_info *rbi)
-{
-	return rbi->ring_buffer->interrupt_mask;
-}
-
 /*
  *
  * hv_ringbuffer_init()
Signature · 1 line
1.7.4.1
#7 lore raw

PATCH 7/28 Drivers: hv: Support handling multiple VMBUS versions

In reply to K. Y. Srinivasan

The current code hard coded the vmbus version independent of the host it was running on. Add code to dynamically negotiate the most appropriate version.

---
 drivers/hv/connection.c |  165 +++++++++++++++++++++++++++++++---------------
 include/linux/hyperv.h  |    6 --
 2 files changed, 111 insertions(+), 60 deletions(-)
drivers/hv/connection.c +111 -54
index d1019a7..2b56a3f 100644
@@ -40,15 +40,111 @@ struct vmbus_connection vmbus_connection = {
 };
 
 /*
+ * VMBUS version is 32 bit entity broken up into
+ * two 16 bit quantities: major_number. minor_number.
+ *
+ * 0 . 13 (Windows Server 2008)
+ * 1 . 1  (Windows 7)
+ * 2 . 4  (Windows 8)
+ */
+
+#define VERSION_WS2008	((0 << 16) | (13))
+#define VERSION_WIN7	((1 << 16) | (1))
+#define VERSION_WIN8	((2 << 16) | (4))
+
+#define VERSION_INVAL -1
+
+static __u32 vmbus_get_next_version(__u32 current_version)
+{
+	switch (current_version) {
+	case (VERSION_WIN7):
+		return VERSION_WS2008;
+
+	case (VERSION_WIN8):
+		return VERSION_WIN7;
+
+	case (VERSION_WS2008):
+	default:
+		return VERSION_INVAL;
+	}
+}
+
+static int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo,
+					__u32 version)
+{
+	int ret = 0;
+	struct vmbus_channel_initiate_contact *msg;
+	unsigned long flags;
+	int t;
+
+	init_completion(&msginfo->waitevent);
+
+	msg = (struct vmbus_channel_initiate_contact *)msginfo->msg;
+
+	msg->header.msgtype = CHANNELMSG_INITIATE_CONTACT;
+	msg->vmbus_version_requested = version;
+	msg->interrupt_page = virt_to_phys(vmbus_connection.int_page);
+	msg->monitor_page1 = virt_to_phys(vmbus_connection.monitor_pages);
+	msg->monitor_page2 = virt_to_phys(
+			(void *)((unsigned long)vmbus_connection.monitor_pages +
+				 PAGE_SIZE));
+
+	/*
+	 * Add to list before we send the request since we may
+	 * receive the response before returning from this routine
+	 */
+	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
+	list_add_tail(&msginfo->msglistentry,
+		      &vmbus_connection.chn_msg_list);
+
+	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
+
+	ret = vmbus_post_msg(msg,
+			       sizeof(struct vmbus_channel_initiate_contact));
+	if (ret != 0) {
+		spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
+		list_del(&msginfo->msglistentry);
+		spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock,
+					flags);
+		return ret;
+	}
+
+	/* Wait for the connection response */
+	t =  wait_for_completion_timeout(&msginfo->waitevent, 5*HZ);
+	if (t == 0) {
+		spin_lock_irqsave(&vmbus_connection.channelmsg_lock,
+				flags);
+		list_del(&msginfo->msglistentry);
+		spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock,
+					flags);
+		return -ETIMEDOUT;
+	}
+
+	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
+	list_del(&msginfo->msglistentry);
+	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
+
+	/* Check if successful */
+	if (msginfo->response.version_response.version_supported) {
+		vmbus_connection.conn_state = CONNECTED;
+	} else {
+		pr_err("Unable to connect, "
+			"Version %d not supported by Hyper-V\n",
+			version);
+		return -ECONNREFUSED;
+	}
+
+	return ret;
+}
+
+/*
  * vmbus_connect - Sends a connect request on the partition service connection
  */
 int vmbus_connect(void)
 {
 	int ret = 0;
-	int t;
 	struct vmbus_channel_msginfo *msginfo = NULL;
-	struct vmbus_channel_initiate_contact *msg;
-	unsigned long flags;
+	__u32 version;
 
 	/* Initialize the vmbus connection */
 	vmbus_connection.conn_state = CONNECTING;
@@ -99,64 +195,25 @@ int vmbus_connect(void)
 		goto cleanup;
 	}
 
-	init_completion(&msginfo->waitevent);
-
-	msg = (struct vmbus_channel_initiate_contact *)msginfo->msg;
-
-	msg->header.msgtype = CHANNELMSG_INITIATE_CONTACT;
-	msg->vmbus_version_requested = VMBUS_REVISION_NUMBER;
-	msg->interrupt_page = virt_to_phys(vmbus_connection.int_page);
-	msg->monitor_page1 = virt_to_phys(vmbus_connection.monitor_pages);
-	msg->monitor_page2 = virt_to_phys(
-			(void *)((unsigned long)vmbus_connection.monitor_pages +
-				 PAGE_SIZE));
-
 	/*
-	 * Add to list before we send the request since we may
-	 * receive the response before returning from this routine
+	 * Negotiate a compatible VMBUS version number with the
+	 * host. We start with the highest number we can support
+	 * and work our way down until we negotiate a compatible
+	 * version.
 	 */
-	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
-	list_add_tail(&msginfo->msglistentry,
-		      &vmbus_connection.chn_msg_list);
-
-	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
 
-	ret = vmbus_post_msg(msg,
-			       sizeof(struct vmbus_channel_initiate_contact));
-	if (ret != 0) {
-		spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
-		list_del(&msginfo->msglistentry);
-		spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock,
-					flags);
-		goto cleanup;
-	}
+	version = VERSION_WS2008;
 
-	/* Wait for the connection response */
-	t =  wait_for_completion_timeout(&msginfo->waitevent, 5*HZ);
-	if (t == 0) {
-		spin_lock_irqsave(&vmbus_connection.channelmsg_lock,
-				flags);
-		list_del(&msginfo->msglistentry);
-		spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock,
-					flags);
-		ret = -ETIMEDOUT;
-		goto cleanup;
-	}
+	do {
+		ret = vmbus_negotiate_version(msginfo, version);
+		if (ret == 0)
+			break;
 
-	spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags);
-	list_del(&msginfo->msglistentry);
-	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
+		version = vmbus_get_next_version(version);
+	} while (version != VERSION_INVAL);
 
-	/* Check if successful */
-	if (msginfo->response.version_response.version_supported) {
-		vmbus_connection.conn_state = CONNECTED;
-	} else {
-		pr_err("Unable to connect, "
-			"Version %d not supported by Hyper-V\n",
-			VMBUS_REVISION_NUMBER);
-		ret = -ECONNREFUSED;
+	if (version == VERSION_INVAL)
 		goto cleanup;
-	}
 
 	kfree(msginfo);
 	return 0;
include/linux/hyperv.h +0 -6
index 1ffe84d..b097bf9 100644
@@ -406,12 +406,6 @@ hv_get_ringbuffer_availbytes(struct hv_ring_buffer_info *rbi,
 #define HV_DRV_VERSION           "3.1"
 
 
-/*
- * A revision number of vmbus that is used for ensuring both ends on a
- * partition are using compatible versions.
- */
-#define VMBUS_REVISION_NUMBER		13
-
 /* Make maximum size of pipe payload of 16K */
 #define MAX_PIPE_DATA_PAYLOAD		(sizeof(u8) * 16384)
 
Signature · 1 line
1.7.4.1
#8 lore raw

PATCH 8/28 Drivers: hv: Update the ring buffer structure to match win8 functionality

In reply to K. Y. Srinivasan

Update the ringbuffer structure to support win8 functionality.

---
 include/linux/hyperv.h |   28 +++++++++++++++++++++-------
 1 files changed, 21 insertions(+), 7 deletions(-)
include/linux/hyperv.h +21 -7
index b097bf9..2b54801 100644
@@ -325,14 +325,28 @@ struct hv_ring_buffer {
 
 	u32 interrupt_mask;
 
-	/* Pad it to PAGE_SIZE so that data starts on page boundary */
-	u8	reserved[4084];
-
-	/* NOTE:
-	 * The interrupt_mask field is used only for channels but since our
-	 * vmbus connection also uses this data structure and its data starts
-	 * here, we commented out this field.
+	/*
+	 * Win8 uses some of the reserved bits to implement
+	 * interrupt driven flow management. On the send side
+	 * we can request that the receiver interrupt the sender
+	 * when the ring transitions from being full to being able
+	 * to handle a message of size "pending_send_sz".
+	 *
+	 * Add necessary state for this enhancement.
 	 */
+	u32 pending_send_sz;
+
+	u32 reserved1[12];
+
+	union {
+		struct {
+			u32 feat_pending_send_sz:1;
+		};
+		u32 value;
+	} feature_bits;
+
+	/* Pad it to PAGE_SIZE so that data starts on page boundary */
+	u8	reserved2[4028];
 
 	/*
 	 * Ring data starts here + RingDataStartOffset
Signature · 1 line
1.7.4.1
#9 lore raw

PATCH 9/28 Drivers: hv: Extend/modify vmbus_channel_offer_channel for win7 and beyond

In reply to K. Y. Srinivasan

The "offfer" message sent by the host has been extended in win7 (ws2008 R2). Add/modify state to reflect this extension. All these changes are backward compatible.

---
 include/linux/hyperv.h |   36 +++++++++++++++++++++++++++++++-----
 1 files changed, 31 insertions(+), 5 deletions(-)
include/linux/hyperv.h +31 -5
index 2b54801..bee559a 100644
@@ -440,9 +440,13 @@ hv_get_ringbuffer_availbytes(struct hv_ring_buffer_info *rbi,
 struct vmbus_channel_offer {
 	uuid_le if_type;
 	uuid_le if_instance;
-	u64 int_latency; /* in 100ns units */
-	u32 if_revision;
-	u32 server_ctx_size;	/* in bytes */
+
+	/*
+	 * These two fields are not currently used.
+	 */
+	u64 reserved1;
+	u64 reserved2;
+
 	u16 chn_flags;
 	u16 mmio_megabytes;		/* in bytes * 1024 * 1024 */
 
@@ -464,7 +468,11 @@ struct vmbus_channel_offer {
 			unsigned char user_def[MAX_PIPE_USER_DEFINED_BYTES];
 		} pipe;
 	} u;
-	u32 padding;
+	/*
+	 * The sub_channel_index is defined in win8.
+	 */
+	u16 sub_channel_index;
+	u16 reserved3;
 } __packed;
 
 /* Server Flags */
@@ -660,7 +668,25 @@ struct vmbus_channel_offer_channel {
 	struct vmbus_channel_offer offer;
 	u32 child_relid;
 	u8 monitorid;
-	u8 monitor_allocated;
+	/*
+	 * win7 and beyond splits this field into a bit field.
+	 */
+	u8 monitor_allocated:1;
+	u8 reserved:7;
+	/*
+	 * These are new fields added in win7 and later.
+	 * Do not access these fields without checking the
+	 * negotiated protocol.
+	 *
+	 * If "is_dedicated_interrupt" is set, we must not set the
+	 * associated bit in the channel bitmap while sending the
+	 * interrupt to the host.
+	 *
+	 * connection_id is to be used in signaling the host.
+	 */
+	u16 is_dedicated_interrupt:1;
+	u16 reserved1:15;
+	u32 connection_id;
 } __packed;
 
 /* Rescind Offer parameters */
Signature · 1 line
1.7.4.1
#10 lore raw

PATCH 10/28 Drivers: hv: Save and export negotiated vmbus version

In reply to K. Y. Srinivasan

Export the negotiated vmbus version as this may be useful for individual drivers.

---
 drivers/hv/connection.c |    9 +++++++++
 include/linux/hyperv.h  |    6 ++++++
 2 files changed, 15 insertions(+), 0 deletions(-)
drivers/hv/connection.c +9 -0
index 2b56a3f..70ea5d1 100644
@@ -30,6 +30,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/hyperv.h>
+#include <linux/export.h>
 #include <asm/hyperv.h>
 #include "hyperv_vmbus.h"
 
@@ -54,6 +55,12 @@ struct vmbus_connection vmbus_connection = {
 
 #define VERSION_INVAL -1
 
+/*
+ * Negotiated protocol version with the host.
+ */
+__u32 vmbus_proto_version;
+EXPORT_SYMBOL_GPL(vmbus_proto_version);
+
 static __u32 vmbus_get_next_version(__u32 current_version)
 {
 	switch (current_version) {
@@ -215,6 +222,8 @@ int vmbus_connect(void)
 	if (version == VERSION_INVAL)
 		goto cleanup;
 
+	vmbus_proto_version = version;
+	pr_info("Negotiated host information %d\n", version);
 	kfree(msginfo);
 	return 0;
 
include/linux/hyperv.h +6 -0
index bee559a..134a202 100644
@@ -1204,5 +1204,11 @@ int hv_kvp_init(struct hv_util_service *);
 void hv_kvp_deinit(void);
 void hv_kvp_onchannelcallback(void *);
 
+/*
+ * Negotiated version with the Host.
+ */
+
+extern __u32 vmbus_proto_version;
+
 #endif /* __KERNEL__ */
 #endif /* _HYPERV_H */
Signature · 1 line
1.7.4.1
#11 lore raw

PATCH 11/28 Drivers: hv: Change the signature for hv_signal_event()

In reply to K. Y. Srinivasan

In preparation for implementing a per-connection signaling framework, change the signature of the function hv_signal_event(). The current code uses a global handle for signaling the host.

---
 drivers/hv/connection.c   |    2 +-
 drivers/hv/hv.c           |    7 +++----
 drivers/hv/hyperv_vmbus.h |    2 +-
 3 files changed, 5 insertions(+), 6 deletions(-)
drivers/hv/connection.c +1 -1
index 70ea5d1..3f8ce7b 100644
@@ -402,5 +402,5 @@ int vmbus_set_event(u32 child_relid)
 		(unsigned long *)vmbus_connection.send_int_page +
 		(child_relid >> 5));
 
-	return hv_signal_event();
+	return hv_signal_event(hv_context.signal_event_param);
 }
drivers/hv/hv.c +3 -4
index 3648f8f..dd0af89 100644
@@ -273,13 +273,12 @@ int hv_post_message(union hv_connection_id connection_id,
  *
  * This involves a hypercall.
  */
-u16 hv_signal_event(void)
+u16 hv_signal_event(void *con_id)
 {
 	u16 status;
 
-	status = do_hypercall(HVCALL_SIGNAL_EVENT,
-			       hv_context.signal_event_param,
-			       NULL) & 0xFFFF;
+	status = (do_hypercall(HVCALL_SIGNAL_EVENT, con_id, NULL) & 0xFFFF);
+
 	return status;
 }
 
drivers/hv/hyperv_vmbus.h +1 -1
index ac1e419..61d2c4f 100644
@@ -538,7 +538,7 @@ extern int hv_post_message(union hv_connection_id connection_id,
 			 enum hv_message_type message_type,
 			 void *payload, size_t payload_size);
 
-extern u16 hv_signal_event(void);
+extern u16 hv_signal_event(void *con_id);
 
 extern void hv_synic_init(void *irqarg);
 
Signature · 1 line
1.7.4.1
#12 lore raw

PATCH 12/28 Drivers: hv: Change the signature of vmbus_set_event()

In reply to K. Y. Srinivasan

In preparation for supporting a per-connection signaling mechanism, change the signature of vmbus_set_event(). This change is also needed to implement other aspects of the signaling optimization.

---
 drivers/hv/channel.c      |    2 +-
 drivers/hv/connection.c   |    3 ++-
 drivers/hv/hyperv_vmbus.h |    2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)
drivers/hv/channel.c +1 -1
index 727c5f1..70a34da 100644
@@ -55,7 +55,7 @@ static void vmbus_setevent(struct vmbus_channel *channel)
 					[channel->monitor_grp].pending);
 
 	} else {
-		vmbus_set_event(channel->offermsg.child_relid);
+		vmbus_set_event(channel);
 	}
 }
 
drivers/hv/connection.c +2 -1
index 3f8ce7b..0d8b132 100644
@@ -395,8 +395,9 @@ int vmbus_post_msg(void *buffer, size_t buflen)
 /*
  * vmbus_set_event - Send an event notification to the parent
  */
-int vmbus_set_event(u32 child_relid)
+int vmbus_set_event(struct vmbus_channel *channel)
 {
+	u32 child_relid = channel->offermsg.child_relid;
 	/* Each u32 represents 32 channels */
 	sync_set_bit(child_relid & 31,
 		(unsigned long *)vmbus_connection.send_int_page +
drivers/hv/hyperv_vmbus.h +1 -1
index 61d2c4f..cd48ac3 100644
@@ -660,7 +660,7 @@ int vmbus_connect(void);
 
 int vmbus_post_msg(void *buffer, size_t buflen);
 
-int vmbus_set_event(u32 child_relid);
+int vmbus_set_event(struct vmbus_channel *channel);
 
 void vmbus_on_event(unsigned long data);
 
Signature · 1 line
1.7.4.1
#13 lore raw

PATCH 13/28 Drivers: hv: Move vmbus version definitions to hyperv.h

In reply to K. Y. Srinivasan

To support version specific optimization in various vmbus drivers, move the vmbus definitions to the public header file.

---
 drivers/hv/connection.c |   15 ---------------
 include/linux/hyperv.h  |   15 +++++++++++++++
 2 files changed, 15 insertions(+), 15 deletions(-)
drivers/hv/connection.c +0 -15
index 0d8b132..56b14e5 100644
@@ -41,21 +41,6 @@ struct vmbus_connection vmbus_connection = {
 };
 
 /*
- * VMBUS version is 32 bit entity broken up into
- * two 16 bit quantities: major_number. minor_number.
- *
- * 0 . 13 (Windows Server 2008)
- * 1 . 1  (Windows 7)
- * 2 . 4  (Windows 8)
- */
-
-#define VERSION_WS2008	((0 << 16) | (13))
-#define VERSION_WIN7	((1 << 16) | (1))
-#define VERSION_WIN8	((2 << 16) | (4))
-
-#define VERSION_INVAL -1
-
-/*
  * Negotiated protocol version with the host.
  */
 __u32 vmbus_proto_version;
include/linux/hyperv.h +15 -0
index 134a202..e725026 100644
@@ -419,6 +419,21 @@ hv_get_ringbuffer_availbytes(struct hv_ring_buffer_info *rbi,
  */
 #define HV_DRV_VERSION           "3.1"
 
+/*
+ * VMBUS version is 32 bit entity broken up into
+ * two 16 bit quantities: major_number. minor_number.
+ *
+ * 0 . 13 (Windows Server 2008)
+ * 1 . 1  (Windows 7)
+ * 2 . 4  (Windows 8)
+ */
+
+#define VERSION_WS2008  ((0 << 16) | (13))
+#define VERSION_WIN7    ((1 << 16) | (1))
+#define VERSION_WIN8    ((2 << 16) | (4))
+
+#define VERSION_INVAL -1
+
 
 /* Make maximum size of pipe payload of 16K */
 #define MAX_PIPE_DATA_PAYLOAD		(sizeof(u8) * 16384)
Signature · 1 line
1.7.4.1
#14 lore raw

PATCH 14/28 Drivers: hv: Manage signaling state on a per-connection basis

In reply to K. Y. Srinivasan

The current code has a global handle for supporting signaling of the host from guest. Make this a per-channel attribute as on some versions of the host we can signal on per-channel handle.

---
 drivers/hv/channel_mgmt.c |   20 ++++++++++++++++++++
 drivers/hv/hyperv_vmbus.h |   21 ---------------------
 include/linux/hyperv.h    |   25 +++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 21 deletions(-)
drivers/hv/channel_mgmt.c +20 -0
index 7bf5917..f4d9902 100644
@@ -282,6 +282,26 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
 	 */
 	newchannel->batched_reading = true;
 
+	/*
+	 * Setup state for signalling the host.
+	 */
+	newchannel->sig_event = (struct hv_input_signal_event *)
+				(ALIGN((unsigned long)
+				&newchannel->sig_buf,
+				HV_HYPERCALL_PARAM_ALIGN));
+
+	newchannel->sig_event->connectionid.asu32 = 0;
+	newchannel->sig_event->connectionid.u.id = VMBUS_EVENT_CONNECTION_ID;
+	newchannel->sig_event->flag_number = 0;
+	newchannel->sig_event->rsvdz = 0;
+
+	if (vmbus_proto_version != VERSION_WS2008) {
+		newchannel->is_dedicated_interrupt =
+				(offer->is_dedicated_interrupt != 0);
+		newchannel->sig_event->connectionid.u.id =
+				offer->connection_id;
+	}
+
 	memcpy(&newchannel->offermsg, offer,
 	       sizeof(struct vmbus_channel_offer_channel));
 	newchannel->monitor_grp = (u8)offer->monitorid / 32;
drivers/hv/hyperv_vmbus.h +0 -21
index cd48ac3..1bc7500 100644
@@ -101,15 +101,6 @@ enum hv_message_type {
 /* Define invalid partition identifier. */
 #define HV_PARTITION_ID_INVALID		((u64)0x0)
 
-/* Define connection identifier type. */
-union hv_connection_id {
-	u32 asu32;
-	struct {
-		u32 id:24;
-		u32 reserved:8;
-	} u;
-};
-
 /* Define port identifier type. */
 union hv_port_id {
 	u32 asu32;
@@ -338,13 +329,6 @@ struct hv_input_post_message {
 	u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT];
 };
 
-/* Definition of the hv_signal_event hypercall input structure. */
-struct hv_input_signal_event {
-	union hv_connection_id connectionid;
-	u16 flag_number;
-	u16 rsvdz;
-};
-
 /*
  * Versioning definitions used for guests reporting themselves to the
  * hypervisor, and visa versa.
@@ -498,11 +482,6 @@ static const uuid_le VMBUS_SERVICE_ID = {
 
 
 
-struct hv_input_signal_event_buffer {
-	u64 align8;
-	struct hv_input_signal_event event;
-};
-
 struct hv_context {
 	/* We only support running on top of Hyper-V
 	* So at this point this really can only contain the Hyper-V ID
include/linux/hyperv.h +25 -0
index e725026..c6e2c44 100644
@@ -897,6 +897,27 @@ struct vmbus_close_msg {
 	struct vmbus_channel_close_channel msg;
 };
 
+/* Define connection identifier type. */
+union hv_connection_id {
+	u32 asu32;
+	struct {
+		u32 id:24;
+		u32 reserved:8;
+	} u;
+};
+
+/* Definition of the hv_signal_event hypercall input structure. */
+struct hv_input_signal_event {
+	union hv_connection_id connectionid;
+	u16 flag_number;
+	u16 rsvdz;
+};
+
+struct hv_input_signal_event_buffer {
+	u64 align8;
+	struct hv_input_signal_event event;
+};
+
 struct vmbus_channel {
 	struct list_head listentry;
 
@@ -946,6 +967,10 @@ struct vmbus_channel {
 	 */
 
 	bool batched_reading;
+
+	bool is_dedicated_interrupt;
+	struct hv_input_signal_event_buffer sig_buf;
+	struct hv_input_signal_event *sig_event;
 };
 
 static inline void set_channel_read_state(struct vmbus_channel *c, bool state)
Signature · 1 line
1.7.4.1
#15 lore raw

PATCH 15/28 Drivers: hv: Cleanup vmbus_set_event() to support win7 and beyond

In reply to K. Y. Srinivasan

On win7 (ws2008 R2) and beyond, we have the notion of having dedicated interrupts on a per-channel basis. When a channel has a dedicated interrupt assigned, there is no need to set the interrupt bit in the shared page. Implement this optimization.

---
 drivers/hv/connection.c |   13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)
drivers/hv/connection.c +8 -5
index 56b14e5..114050d 100644
@@ -383,10 +383,13 @@ int vmbus_post_msg(void *buffer, size_t buflen)
 int vmbus_set_event(struct vmbus_channel *channel)
 {
 	u32 child_relid = channel->offermsg.child_relid;
-	/* Each u32 represents 32 channels */
-	sync_set_bit(child_relid & 31,
-		(unsigned long *)vmbus_connection.send_int_page +
-		(child_relid >> 5));
 
-	return hv_signal_event(hv_context.signal_event_param);
+	if (!channel->is_dedicated_interrupt) {
+		/* Each u32 represents 32 channels */
+		sync_set_bit(child_relid & 31,
+			(unsigned long *)vmbus_connection.send_int_page +
+			(child_relid >> 5));
+	}
+
+	return hv_signal_event(channel->sig_event);
 }
Signature · 1 line
1.7.4.1
#16 lore raw

PATCH 16/28 Drivers: hv: Setup a mapping for Hyper-V's notion cpu ID

In reply to K. Y. Srinivasan

On win8 (ws2012), incoming vmbus interrupt load can be spread across all available VCPUs in the guest. On a per-channel basis, the interrupts can be bound to specific CPUs. The Linux notion of cpu ID may be different from that of the hypervisor's. Setup a mapping structure.

---
 drivers/hv/hv.c           |   11 +++++++++++
 drivers/hv/hyperv_vmbus.h |   10 ++++++++++
 2 files changed, 21 insertions(+), 0 deletions(-)
drivers/hv/hv.c +11 -0
index dd0af89..76304a6 100644
@@ -137,6 +137,8 @@ int hv_init(void)
 	memset(hv_context.synic_event_page, 0, sizeof(void *) * NR_CPUS);
 	memset(hv_context.synic_message_page, 0,
 	       sizeof(void *) * NR_CPUS);
+	memset(hv_context.vp_index, 0,
+	       sizeof(int) * NR_CPUS);
 
 	max_leaf = query_hypervisor_info();
 
@@ -296,6 +298,7 @@ void hv_synic_init(void *irqarg)
 	union hv_synic_siefp siefp;
 	union hv_synic_sint shared_sint;
 	union hv_synic_scontrol sctrl;
+	u64 vp_index;
 
 	u32 irq_vector = *((u32 *)(irqarg));
 	int cpu = smp_processor_id();
@@ -355,6 +358,14 @@ void hv_synic_init(void *irqarg)
 	wrmsrl(HV_X64_MSR_SCONTROL, sctrl.as_uint64);
 
 	hv_context.synic_initialized = true;
+
+	/*
+	 * Setup the mapping between Hyper-V's notion
+	 * of cpuid and Linux' notion of cpuid.
+	 * This array will be indexed using Linux cpuid.
+	 */
+	rdmsrl(HV_X64_MSR_VP_INDEX, vp_index);
+	hv_context.vp_index[cpu] = (u32)vp_index;
 	return;
 
 cleanup:
drivers/hv/hyperv_vmbus.h +10 -0
index 1bc7500..6bbc197 100644
@@ -502,6 +502,16 @@ struct hv_context {
 
 	void *synic_message_page[NR_CPUS];
 	void *synic_event_page[NR_CPUS];
+	/*
+	 * Hypervisor's notion of virtual processor ID is different from
+	 * Linux' notion of CPU ID. This information can only be retrieved
+	 * in the context of the calling CPU. Setup a map for easy access
+	 * to this information:
+	 *
+	 * vp_index[a] is the Hyper-V's processor ID corresponding to
+	 * Linux cpuid 'a'.
+	 */
+	u32 vp_index[NR_CPUS];
 };
 
 extern struct hv_context hv_context;
Signature · 1 line
1.7.4.1
#17 lore raw

PATCH 17/28 Drivers: hv: Add state to manage incoming channel interrupt load

In reply to K. Y. Srinivasan

Add state to bind a channel to a specific VCPU. This will help us better distribute incoming interrupt load.

---
 drivers/hv/channel.c      |    2 +-
 drivers/hv/channel_mgmt.c |    2 ++
 include/linux/hyperv.h    |   21 +++++++++++++++++++--
 3 files changed, 22 insertions(+), 3 deletions(-)
drivers/hv/channel.c +1 -1
index 70a34da..9303252 100644
@@ -181,7 +181,7 @@ int vmbus_open(struct vmbus_channel *newchannel, u32 send_ringbuffer_size,
 	open_msg->ringbuffer_gpadlhandle = newchannel->ringbuffer_gpadlhandle;
 	open_msg->downstream_ringbuffer_pageoffset = send_ringbuffer_size >>
 						  PAGE_SHIFT;
-	open_msg->server_contextarea_gpadlhandle = 0;
+	open_msg->target_vp = newchannel->target_vp;
 
 	if (userdatalen > MAX_USER_DEFINED_BYTES) {
 		err = -EINVAL;
drivers/hv/channel_mgmt.c +2 -0
index f4d9902..56ed45c 100644
@@ -302,6 +302,8 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
 				offer->connection_id;
 	}
 
+	newchannel->target_vp = 0;
+
 	memcpy(&newchannel->offermsg, offer,
 	       sizeof(struct vmbus_channel_offer_channel));
 	newchannel->monitor_grp = (u8)offer->monitorid / 32;
include/linux/hyperv.h +19 -2
index c6e2c44..8c3cb1f 100644
@@ -732,8 +732,15 @@ struct vmbus_channel_open_channel {
 	/* GPADL for the channel's ring buffer. */
 	u32 ringbuffer_gpadlhandle;
 
-	/* GPADL for the channel's server context save area. */
-	u32 server_contextarea_gpadlhandle;
+	/*
+	 * Starting with win8, this field will be used to specify
+	 * the target virtual processor on which to deliver the interrupt for
+	 * the host to guest communication.
+	 * Prior to win8, incoming channel interrupts would only
+	 * be delivered on cpu 0. Setting this value to 0 would
+	 * preserve the earlier behavior.
+	 */
+	u32 target_vp;
 
 	/*
 	* The upstream ring buffer begins at offset zero in the memory
@@ -971,6 +978,16 @@ struct vmbus_channel {
 	bool is_dedicated_interrupt;
 	struct hv_input_signal_event_buffer sig_buf;
 	struct hv_input_signal_event *sig_event;
+
+	/*
+	 * Starting with win8, this field will be used to specify
+	 * the target virtual processor on which to deliver the interrupt for
+	 * the host to guest communication.
+	 * Prior to win8, incoming channel interrupts would only
+	 * be delivered on cpu 0. Setting this value to 0 would
+	 * preserve the earlier behavior.
+	 */
+	u32 target_vp;
 };
 
 static inline void set_channel_read_state(struct vmbus_channel *c, bool state)
Signature · 1 line
1.7.4.1
#18 lore raw

PATCH 18/28 Drivers: hv: Modify the interrupt handling code to support win8 and beyond

In reply to K. Y. Srinivasan

Starting with Win8 (WS2012), the event page can be used to directly get the channel ID that needs servicing. Modify the channel event handling code to take advantage of this feature.

---
 drivers/hv/connection.c |   26 ++++++++++++++++++++++++--
 drivers/hv/vmbus_drv.c  |   27 ++++++++++++++++++++++-----
 2 files changed, 46 insertions(+), 7 deletions(-)
drivers/hv/connection.c +24 -2
index 114050d..ac71653 100644
@@ -321,10 +321,32 @@ static void process_chn_event(u32 relid)
 void vmbus_on_event(unsigned long data)
 {
 	u32 dword;
-	u32 maxdword = MAX_NUM_CHANNELS_SUPPORTED >> 5;
+	u32 maxdword;
 	int bit;
 	u32 relid;
-	u32 *recv_int_page = vmbus_connection.recv_int_page;
+	u32 *recv_int_page = NULL;
+	void *page_addr;
+	int cpu = smp_processor_id();
+	union hv_synic_event_flags *event;
+
+	if ((vmbus_proto_version == VERSION_WS2008) ||
+		(vmbus_proto_version == VERSION_WIN7)) {
+		maxdword = MAX_NUM_CHANNELS_SUPPORTED >> 5;
+		recv_int_page = vmbus_connection.recv_int_page;
+	} else {
+		/*
+		 * When the host is win8 and beyond, the event page
+		 * can be directly checked to get the id of the channel
+		 * that has the interrupt pending.
+		 */
+		maxdword = HV_EVENT_FLAGS_DWORD_COUNT;
+		page_addr = hv_context.synic_event_page[cpu];
+		event = (union hv_synic_event_flags *)page_addr +
+						 VMBUS_MESSAGE_SINT;
+		recv_int_page = event->flags32;
+	}
+
+
 
 	/* Check events */
 	if (!recv_int_page)
drivers/hv/vmbus_drv.c +22 -5
index 8e1a9ec..c583a04 100644
@@ -460,15 +460,32 @@ static irqreturn_t vmbus_isr(int irq, void *dev_id)
 	 * Hyper-V, and the Windows team suggested we do the same.
 	 */
 
-	page_addr = hv_context.synic_event_page[cpu];
-	event = (union hv_synic_event_flags *)page_addr + VMBUS_MESSAGE_SINT;
+	if ((vmbus_proto_version == VERSION_WS2008) ||
+		(vmbus_proto_version == VERSION_WIN7)) {
 
-	/* Since we are a child, we only need to check bit 0 */
-	if (sync_test_and_clear_bit(0, (unsigned long *) &event->flags32[0])) {
+		page_addr = hv_context.synic_event_page[cpu];
+		event = (union hv_synic_event_flags *)page_addr +
+					 VMBUS_MESSAGE_SINT;
+
+		/* Since we are a child, we only need to check bit 0 */
+		if (sync_test_and_clear_bit(0,
+			(unsigned long *) &event->flags32[0])) {
+			handled = true;
+		}
+	} else {
+		/*
+		 * Our host is win8 or above. The signaling mechanism
+		 * has changed and we can directly look at the event page.
+		 * If bit n is set then we have an interrup on the channel
+		 * whose id is n.
+		 */
 		handled = true;
-		tasklet_schedule(&event_dpc);
 	}
 
+	if (handled)
+		tasklet_schedule(&event_dpc);
+
+
 	page_addr = hv_context.synic_message_page[cpu];
 	msg = (struct hv_message *)page_addr + VMBUS_MESSAGE_SINT;
 
Signature · 1 line
1.7.4.1
#19 lore raw

PATCH 19/28 Drivers: hv: Add code to distribute channel interrupt load

In reply to K. Y. Srinivasan

Implement a simple policy for distributing incoming interrupt load. We classify channels as (a) performance critical and (b) not performance critical. All non-performance critical channels will be bound to the boot cpu. Performance critical channels will be bound to the remaining available CPUs on a round-robin basis.

---
 drivers/hv/channel_mgmt.c |   85 ++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 84 insertions(+), 1 deletions(-)
drivers/hv/channel_mgmt.c +84 -1
index 56ed45c..c80fe62 100644
@@ -257,6 +257,89 @@ static void vmbus_process_offer(struct work_struct *work)
 	}
 }
 
+enum {
+	IDE = 0,
+	SCSI,
+	NIC,
+	MAX_PERF_CHN,
+};
+
+/*
+ * This is an array of channels (devices) that are performance critical.
+ * We attempt to distribute the interrupt load for these devices across
+ * all available CPUs.
+ */
+static const uuid_le hp_devs[] = {
+	/* {32412632-86cb-44a2-9b5c-50d1417354f5} */
+	/* IDE */
+	{
+		.b = {
+			0x32, 0x26, 0x41, 0x32, 0xcb, 0x86, 0xa2, 0x44,
+			0x9b, 0x5c, 0x50, 0xd1, 0x41, 0x73, 0x54, 0xf5
+		}
+	},
+	/* {ba6163d9-04a1-4d29-b605-72e2ffb1dc7f} */
+	/* Storage - SCSI */
+	{
+		.b  = {
+			0xd9, 0x63, 0x61, 0xba, 0xa1, 0x04, 0x29, 0x4d,
+			0xb6, 0x05, 0x72, 0xe2, 0xff, 0xb1, 0xdc, 0x7f
+		}
+	},
+	/* {F8615163-DF3E-46c5-913F-F2D2F965ED0E} */
+	/* Network */
+	{
+		.b = {
+			0x63, 0x51, 0x61, 0xF8, 0x3E, 0xDF, 0xc5, 0x46,
+			0x91, 0x3F, 0xF2, 0xD2, 0xF9, 0x65, 0xED, 0x0E
+		}
+	},
+
+};
+
+
+/*
+ * We use this state to statically distribute the channel interrupt load.
+ */
+static u32  next_vp;
+
+/*
+ * Starting with Win8, we can statically distribute the incoming
+ * channel interrupt load by binding a channel to VCPU. We
+ * implement here a simple round robin scheme for distributing
+ * the interrupt load.
+ * We will bind channels that are not performance critical to cpu 0 and
+ * performance critical channels (IDE, SCSI and Network) will be uniformly
+ * distributed across all available CPUs.
+ */
+static u32 get_vp_index(uuid_le *type_guid)
+{
+	u32 cur_cpu;
+	int i;
+	bool perf_chn = false;
+	u32 max_cpus = num_online_cpus();
+
+	for (i = IDE; i < MAX_PERF_CHN; i++) {
+		if (!memcmp(type_guid->b, hp_devs[i].b,
+				 sizeof(uuid_le))) {
+			perf_chn = true;
+			break;
+		}
+	}
+	if ((vmbus_proto_version == VERSION_WS2008) ||
+	    (vmbus_proto_version == VERSION_WIN7) || (!perf_chn)) {
+		/*
+		 * Prior to win8, all channel interrupts are
+		 * delivered on cpu 0.
+		 * Also if the channel is not a performance critical
+		 * channel, bind it to cpu 0.
+		 */
+		return 0;
+	}
+	cur_cpu = (++next_vp % max_cpus);
+	return hv_context.vp_index[cur_cpu];
+}
+
 /*
  * vmbus_onoffer - Handler for channel offers from vmbus in parent partition.
  *
@@ -302,7 +385,7 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr)
 				offer->connection_id;
 	}
 
-	newchannel->target_vp = 0;
+	newchannel->target_vp = get_vp_index(&offer->offer.if_type);
 
 	memcpy(&newchannel->offermsg, offer,
 	       sizeof(struct vmbus_channel_offer_channel));
Signature · 1 line
1.7.4.1
#20 lore raw

Re: [PATCH 19/28] Drivers: hv: Add code to distribute channel interrupt load

Quoted 22 lines · K. Y. Srinivasan
drivers/hv/channel_mgmt.c +22 -0
+/*
+ * This is an array of channels (devices) that are performance critical.
+ * We attempt to distribute the interrupt load for these devices across
+ * all available CPUs.
+ */
+static const uuid_le hp_devs[] = {
+	/* {32412632-86cb-44a2-9b5c-50d1417354f5} */
+	/* IDE */
+	{
+		.b = {
+			0x32, 0x26, 0x41, 0x32, 0xcb, 0x86, 0xa2, 0x44,
+			0x9b, 0x5c, 0x50, 0xd1, 0x41, 0x73, 0x54, 0xf5
+		}
+	},
+	/* {ba6163d9-04a1-4d29-b605-72e2ffb1dc7f} */
+	/* Storage - SCSI */
+	{
+		.b  = {
+			0xd9, 0x63, 0x61, 0xba, 0xa1, 0x04, 0x29, 0x4d,
+			0xb6, 0x05, 0x72, 0xe2, 0xff, 0xb1, 0xdc, 0x7f
+		}
+	},

drivers/hv/channel_mgmt.c static void vmbus_process_offer(struct work_struct *work) · line 298

Quoted 10 lines
drivers/hv/channel_mgmt.c +10 -0
+	/* {F8615163-DF3E-46c5-913F-F2D2F965ED0E} */
+	/* Network */
+	{
+		.b = {
+			0x63, 0x51, 0x61, 0xF8, 0x3E, 0xDF, 0xc5, 0x46,
+			0x91, 0x3F, 0xF2, 0xD2, 0xF9, 0x65, 0xED, 0x0E
+		}
+	},
+
+};

You are now duplicating these "ids" in different places in the kernel, perhaps you should put them in one place (hyperv.h?) and then reference them that way? It's like a device id for pci, once you use it in two places, it goes into pci_ids.h.

I'm not going to reject this patch, but can you please do this in a follow-on patch?

thanks,

greg k-h

#21 lore raw

RE: [PATCH 19/28] Drivers: hv: Add code to distribute channel interrupt load

Quoted 50 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Thursday, January 17, 2013 2:39 PM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 19/28] Drivers: hv: Add code to distribute channel interrupt load

Quoted 22 lines · K. Y. Srinivasan
drivers/hv/channel_mgmt.c +22 -0
+/*
+ * This is an array of channels (devices) that are performance critical.
+ * We attempt to distribute the interrupt load for these devices across
+ * all available CPUs.
+ */
+static const uuid_le hp_devs[] = {
+	/* {32412632-86cb-44a2-9b5c-50d1417354f5} */
+	/* IDE */
+	{
+		.b = {
+			0x32, 0x26, 0x41, 0x32, 0xcb, 0x86, 0xa2, 0x44,
+			0x9b, 0x5c, 0x50, 0xd1, 0x41, 0x73, 0x54, 0xf5
+		}
+	},
+	/* {ba6163d9-04a1-4d29-b605-72e2ffb1dc7f} */
+	/* Storage - SCSI */
+	{
+		.b  = {
+			0xd9, 0x63, 0x61, 0xba, 0xa1, 0x04, 0x29, 0x4d,
+			0xb6, 0x05, 0x72, 0xe2, 0xff, 0xb1, 0xdc, 0x7f
+		}
+	},

drivers/hv/channel_mgmt.c static void vmbus_process_offer(struct work_struct *work) · line 298

Quoted 10 lines
drivers/hv/channel_mgmt.c +10 -0
+	/* {F8615163-DF3E-46c5-913F-F2D2F965ED0E} */
+	/* Network */
+	{
+		.b = {
+			0x63, 0x51, 0x61, 0xF8, 0x3E, 0xDF, 0xc5, 0x46,
+			0x91, 0x3F, 0xF2, 0xD2, 0xF9, 0x65, 0xED, 0x0E
+		}
+	},
+
+};

You are now duplicating these "ids" in different places in the kernel, perhaps you should put them in one place (hyperv.h?) and then reference them that way? It's like a device id for pci, once you use it in two places, it goes into pci_ids.h.

Quoted 2 lines

I'm not going to reject this patch, but can you please do this in a follow-on patch?

Thanks Greg. Will do.

Regards,

K. Y

Quoted 5 lines

thanks,

greg k-h

#22 lore raw

PATCH 20/28 Drivers: hv: Get rid of the unused global signaling state

In reply to K. Y. Srinivasan

Now that we have implemented a per-connection signaling mechanism, get rid of the global signaling state. For hosts that don't support per-connection signaling handle, we have moved the global state to be a per-channel state.

---
 drivers/hv/hv.c           |   24 ------------------------
 drivers/hv/hyperv_vmbus.h |    8 --------
 2 files changed, 0 insertions(+), 32 deletions(-)
drivers/hv/hv.c +0 -24
index 76304a6..e989c6fd 100644
@@ -34,8 +34,6 @@
 struct hv_context hv_context = {
 	.synic_initialized	= false,
 	.hypercall_page		= NULL,
-	.signal_event_param	= NULL,
-	.signal_event_buffer	= NULL,
 };
 
 /*
@@ -170,24 +168,6 @@ int hv_init(void)
 
 	hv_context.hypercall_page = virtaddr;
 
-	/* Setup the global signal event param for the signal event hypercall */
-	hv_context.signal_event_buffer =
-			kmalloc(sizeof(struct hv_input_signal_event_buffer),
-				GFP_KERNEL);
-	if (!hv_context.signal_event_buffer)
-		goto cleanup;
-
-	hv_context.signal_event_param =
-		(struct hv_input_signal_event *)
-			(ALIGN((unsigned long)
-				  hv_context.signal_event_buffer,
-				  HV_HYPERCALL_PARAM_ALIGN));
-	hv_context.signal_event_param->connectionid.asu32 = 0;
-	hv_context.signal_event_param->connectionid.u.id =
-						VMBUS_EVENT_CONNECTION_ID;
-	hv_context.signal_event_param->flag_number = 0;
-	hv_context.signal_event_param->rsvdz = 0;
-
 	return 0;
 
 cleanup:
@@ -215,10 +195,6 @@ void hv_cleanup(void)
 	/* Reset our OS id */
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
 
-	kfree(hv_context.signal_event_buffer);
-	hv_context.signal_event_buffer = NULL;
-	hv_context.signal_event_param = NULL;
-
 	if (hv_context.hypercall_page) {
 		hypercall_msr.as_uint64 = 0;
 		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
drivers/hv/hyperv_vmbus.h +0 -8
index 6bbc197..9135a6f 100644
@@ -492,14 +492,6 @@ struct hv_context {
 
 	bool synic_initialized;
 
-	/*
-	 * This is used as an input param to HvCallSignalEvent hypercall. The
-	 * input param is immutable in our usage and must be dynamic mem (vs
-	 * stack or global). */
-	struct hv_input_signal_event_buffer *signal_event_buffer;
-	/* 8-bytes aligned of the buffer above */
-	struct hv_input_signal_event *signal_event_param;
-
 	void *synic_message_page[NR_CPUS];
 	void *synic_event_page[NR_CPUS];
 	/*
Signature · 1 line
1.7.4.1
#23 lore raw

PATCH 21/28 Drivers: hv: Get rid of unnecessary request for offers

In reply to K. Y. Srinivasan

This call to seek offers is not necessary and just adds unnecessary delay. Get rid of it.

---
 drivers/hv/vmbus_drv.c |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)
drivers/hv/vmbus_drv.c +0 -2
index c583a04..4c92337 100644
@@ -592,8 +592,6 @@ int __vmbus_driver_register(struct hv_driver *hv_driver, struct module *owner, c
 
 	ret = driver_register(&hv_driver->driver);
 
-	vmbus_request_offers();
-
 	return ret;
 }
 EXPORT_SYMBOL_GPL(__vmbus_driver_register);
Signature · 1 line
1.7.4.1
#24 lore raw

PATCH 22/28 Drivers: hv: Manage event tasklets on per-cpu basis

In reply to K. Y. Srinivasan

Now that we can potentially take vmbus interrupts on any CPU, make the tasklets per-CPU.

---
 drivers/hv/hv.c           |   12 ++++++++++++
 drivers/hv/hyperv_vmbus.h |    6 ++++++
 drivers/hv/vmbus_drv.c    |    4 +---
 3 files changed, 19 insertions(+), 3 deletions(-)
drivers/hv/hv.c +12 -0
index e989c6fd..363532a 100644
@@ -27,6 +27,7 @@
 #include <linux/vmalloc.h>
 #include <linux/hyperv.h>
 #include <linux/version.h>
+#include <linux/interrupt.h>
 #include <asm/hyperv.h>
 #include "hyperv_vmbus.h"
 
@@ -137,6 +138,8 @@ int hv_init(void)
 	       sizeof(void *) * NR_CPUS);
 	memset(hv_context.vp_index, 0,
 	       sizeof(int) * NR_CPUS);
+	memset(hv_context.event_dpc, 0,
+	       sizeof(void *) * NR_CPUS);
 
 	max_leaf = query_hypervisor_info();
 
@@ -285,6 +288,15 @@ void hv_synic_init(void *irqarg)
 	/* Check the version */
 	rdmsrl(HV_X64_MSR_SVERSION, version);
 
+	hv_context.event_dpc[cpu] = (struct tasklet_struct *)
+					kmalloc(sizeof(struct tasklet_struct),
+						GFP_ATOMIC);
+	if (hv_context.event_dpc[cpu] == NULL) {
+		pr_err("Unable to allocate event dpc\n");
+		goto cleanup;
+	}
+	tasklet_init(hv_context.event_dpc[cpu], vmbus_on_event, cpu);
+
 	hv_context.synic_message_page[cpu] =
 		(void *)get_zeroed_page(GFP_ATOMIC);
 
drivers/hv/hyperv_vmbus.h +6 -0
index 9135a6f..becb106 100644
@@ -504,6 +504,12 @@ struct hv_context {
 	 * Linux cpuid 'a'.
 	 */
 	u32 vp_index[NR_CPUS];
+	/*
+	 * Starting with win8, we can take channel interrupts on any CPU;
+	 * we will manage the tasklet that handles events on a per CPU
+	 * basis.
+	 */
+	struct tasklet_struct *event_dpc[NR_CPUS];
 };
 
 extern struct hv_context hv_context;
drivers/hv/vmbus_drv.c +1 -3
index 4c92337..6e4f857 100644
@@ -41,7 +41,6 @@
 static struct acpi_device  *hv_acpi_dev;
 
 static struct tasklet_struct msg_dpc;
-static struct tasklet_struct event_dpc;
 static struct completion probe_event;
 static int irq;
 
@@ -483,7 +482,7 @@ static irqreturn_t vmbus_isr(int irq, void *dev_id)
 	}
 
 	if (handled)
-		tasklet_schedule(&event_dpc);
+		tasklet_schedule(hv_context.event_dpc[cpu]);
 
 
 	page_addr = hv_context.synic_message_page[cpu];
@@ -523,7 +522,6 @@ static int vmbus_bus_init(int irq)
 	}
 
 	tasklet_init(&msg_dpc, vmbus_on_msg_dpc, 0);
-	tasklet_init(&event_dpc, vmbus_on_event, 0);
 
 	ret = bus_register(&hv_bus);
 	if (ret)
Signature · 1 line
1.7.4.1
#25 lore raw

PATCH 23/28 Drivers: hv: Handle vmbus interrupts concurrently on all cpus

In reply to K. Y. Srinivasan

Vmbus interrupts are unique in that while the interrupt is delivered on a given vector, these can be handled concurrently on different CPUs. Handle the vmbus interrupts concurrently on all the CPUs.

---
 drivers/hv/hv.c        |    2 +-
 drivers/hv/vmbus_drv.c |   21 +++++++++++++++++++++
 2 files changed, 22 insertions(+), 1 deletions(-)
drivers/hv/hv.c +1 -1
index 363532a..03e6a1e 100644
@@ -335,7 +335,7 @@ void hv_synic_init(void *irqarg)
 	shared_sint.as_uint64 = 0;
 	shared_sint.vector = irq_vector; /* HV_SHARED_SINT_IDT_VECTOR + 0x20; */
 	shared_sint.masked = false;
-	shared_sint.auto_eoi = false;
+	shared_sint.auto_eoi = true;
 
 	wrmsrl(HV_X64_MSR_SINT0 + VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
 
drivers/hv/vmbus_drv.c +21 -0
index 6e4f857..e066d41 100644
@@ -33,6 +33,7 @@
 #include <acpi/acpi_bus.h>
 #include <linux/completion.h>
 #include <linux/hyperv.h>
+#include <linux/kernel_stat.h>
 #include <asm/hyperv.h>
 #include <asm/hypervisor.h>
 #include "hyperv_vmbus.h"
@@ -501,6 +502,19 @@ static irqreturn_t vmbus_isr(int irq, void *dev_id)
 }
 
 /*
+ * vmbus interrupt flow handler:
+ * vmbus interrupts can concurrently occur on multiple CPUs and
+ * can be handled concurrently.
+ */
+
+void vmbus_flow_handler(unsigned int irq, struct irq_desc *desc)
+{
+	kstat_incr_irqs_this_cpu(irq, desc);
+
+	desc->action->handler(irq, desc->action->dev_id);
+}
+
+/*
  * vmbus_bus_init -Main vmbus driver initialization routine.
  *
  * Here, we
@@ -535,6 +549,13 @@ static int vmbus_bus_init(int irq)
 		goto err_unregister;
 	}
 
+	/*
+	 * Vmbus interrupts can be handled concurrently on
+	 * different CPUs. Establish an appropriate interrupt flow
+	 * handler that can support this model.
+	 */
+	irq_set_handler(irq, vmbus_flow_handler);
+
 	vector = IRQ0_VECTOR + irq;
 
 	/*
Signature · 1 line
1.7.4.1
#26 lore raw

PATCH 24/28 Drivers: hv: Add a check to deal with spurious interrupts

In reply to K. Y. Srinivasan

We establish the handler before we have fully initialized the VMBUS state. Deal with spurious interrupts.

---
 drivers/hv/vmbus_drv.c |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)
drivers/hv/vmbus_drv.c +6 -4
index e066d41..797142c 100644
@@ -454,6 +454,12 @@ static irqreturn_t vmbus_isr(int irq, void *dev_id)
 	union hv_synic_event_flags *event;
 	bool handled = false;
 
+	page_addr = hv_context.synic_event_page[cpu];
+	if (page_addr == NULL)
+		return IRQ_NONE;
+
+	event = (union hv_synic_event_flags *)page_addr +
+					 VMBUS_MESSAGE_SINT;
 	/*
 	 * Check for events before checking for messages. This is the order
 	 * in which events and messages are checked in Windows guests on
@@ -463,10 +469,6 @@ static irqreturn_t vmbus_isr(int irq, void *dev_id)
 	if ((vmbus_proto_version == VERSION_WS2008) ||
 		(vmbus_proto_version == VERSION_WIN7)) {
 
-		page_addr = hv_context.synic_event_page[cpu];
-		event = (union hv_synic_event_flags *)page_addr +
-					 VMBUS_MESSAGE_SINT;
-
 		/* Since we are a child, we only need to check bit 0 */
 		if (sync_test_and_clear_bit(0,
 			(unsigned long *) &event->flags32[0])) {
Signature · 1 line
1.7.4.1
#27 lore raw

PATCH 25/28 Drivers: hv: Enable protocol negotiation with win8 hosts

In reply to K. Y. Srinivasan

Now that we have implemented all of the Win8 (WS2012) functionality, negotiate Win8 protocol with the host.

---
 drivers/hv/connection.c |    2 +-
 include/linux/hyperv.h  |    1 +
 2 files changed, 2 insertions(+), 1 deletions(-)
drivers/hv/connection.c +1 -1
index ac71653..3965537 100644
@@ -194,7 +194,7 @@ int vmbus_connect(void)
 	 * version.
 	 */
 
-	version = VERSION_WS2008;
+	version = VERSION_CURRENT;
 
 	do {
 		ret = vmbus_negotiate_version(msginfo, version);
include/linux/hyperv.h +1 -0
index 8c3cb1f..5095b06 100644
@@ -434,6 +434,7 @@ hv_get_ringbuffer_availbytes(struct hv_ring_buffer_info *rbi,
 
 #define VERSION_INVAL -1
 
+#define VERSION_CURRENT VERSION_WIN8
 
 /* Make maximum size of pipe payload of 16K */
 #define MAX_PIPE_DATA_PAYLOAD		(sizeof(u8) * 16384)
Signature · 1 line
1.7.4.1
#28 lore raw

PATCH 26/28 Drivers: hv: Implement flow management on the send side

In reply to K. Y. Srinivasan

Implement flow management on the send side. When the sender is blocked, the reader can potentially signal the sender to indicate there is now room to send.

---
 drivers/hv/channel.c      |   12 ++++++++-
 drivers/hv/hyperv_vmbus.h |    2 +-
 drivers/hv/ring_buffer.c  |   51 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 61 insertions(+), 4 deletions(-)
drivers/hv/channel.c +10 -2
index 9303252..064257e 100644
@@ -735,6 +735,7 @@ int vmbus_recvpacket(struct vmbus_channel *channel, void *buffer,
 	u32 packetlen;
 	u32 userlen;
 	int ret;
+	bool signal = false;
 
 	*buffer_actual_len = 0;
 	*requestid = 0;
@@ -761,8 +762,10 @@ int vmbus_recvpacket(struct vmbus_channel *channel, void *buffer,
 
 	/* Copy over the packet to the user buffer */
 	ret = hv_ringbuffer_read(&channel->inbound, buffer, userlen,
-			     (desc.offset8 << 3));
+			     (desc.offset8 << 3), &signal);
 
+	if (signal)
+		vmbus_setevent(channel);
 
 	return 0;
 }
@@ -779,6 +782,7 @@ int vmbus_recvpacket_raw(struct vmbus_channel *channel, void *buffer,
 	u32 packetlen;
 	u32 userlen;
 	int ret;
+	bool signal = false;
 
 	*buffer_actual_len = 0;
 	*requestid = 0;
@@ -805,7 +809,11 @@ int vmbus_recvpacket_raw(struct vmbus_channel *channel, void *buffer,
 	*requestid = desc.trans_id;
 
 	/* Copy over the entire packet to the user buffer */
-	ret = hv_ringbuffer_read(&channel->inbound, buffer, packetlen, 0);
+	ret = hv_ringbuffer_read(&channel->inbound, buffer, packetlen, 0,
+				 &signal);
+
+	if (signal)
+		vmbus_setevent(channel);
 
 	return 0;
 }
drivers/hv/hyperv_vmbus.h +1 -1
index becb106..ac111f2 100644
@@ -550,7 +550,7 @@ int hv_ringbuffer_peek(struct hv_ring_buffer_info *ring_info, void *buffer,
 int hv_ringbuffer_read(struct hv_ring_buffer_info *ring_info,
 		   void *buffer,
 		   u32 buflen,
-		   u32 offset);
+		   u32 offset, bool *signal);
 
 
 void hv_ringbuffer_get_debuginfo(struct hv_ring_buffer_info *ring_info,
drivers/hv/ring_buffer.c +50 -1

80 lines not on this page · raw message

Signature · 1 line
1.7.4.1
#29 lore raw

PATCH 27/28 Drivers: hv: Capture the host build information

In reply to K. Y. Srinivasan

Capture the host build information so it can be presented along with the negotiated vmbus version information.

---
 drivers/hv/hv.c           |    9 +++++++++
 drivers/hv/hyperv_vmbus.h |    7 +++++++
 2 files changed, 16 insertions(+), 0 deletions(-)
drivers/hv/hv.c +9 -0
index 03e6a1e..0cd2da3 100644
@@ -40,6 +40,11 @@ struct hv_context hv_context = {
 /*
  * query_hypervisor_info - Get version info of the windows hypervisor
  */
+unsigned int host_info_eax;
+unsigned int host_info_ebx;
+unsigned int host_info_ecx;
+unsigned int host_info_edx;
+
 static int query_hypervisor_info(void)
 {
 	unsigned int eax;
@@ -76,6 +81,10 @@ static int query_hypervisor_info(void)
 			    ecx,
 			    edx >> 24,
 			    edx & 0xFFFFFF);
+		host_info_eax = eax;
+		host_info_ebx = ebx;
+		host_info_ecx = ecx;
+		host_info_edx = edx;
 	}
 	return max_leaf;
 }
drivers/hv/hyperv_vmbus.h +7 -0
index ac111f2..12f2f9e 100644
@@ -531,6 +531,13 @@ extern void hv_synic_init(void *irqarg);
 
 extern void hv_synic_cleanup(void *arg);
 
+/*
+ * Host version information.
+ */
+extern unsigned int host_info_eax;
+extern unsigned int host_info_ebx;
+extern unsigned int host_info_ecx;
+extern unsigned int host_info_edx;
 
 /* Interface */
 
Signature · 1 line
1.7.4.1
#30 lore raw

PATCH 28/28 Drivers: hv: Cleanup and consolidate reporting of build/version info

In reply to K. Y. Srinivasan

Now, cleanup and consolidate reporting of host and vmbus version numbers.

---
 drivers/hv/connection.c |   11 +++++++----
 drivers/hv/hv.c         |    7 -------
 2 files changed, 7 insertions(+), 11 deletions(-)
drivers/hv/connection.c +7 -4
index 3965537..253a74b 100644
@@ -120,9 +120,6 @@ static int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo,
 	if (msginfo->response.version_response.version_supported) {
 		vmbus_connection.conn_state = CONNECTED;
 	} else {
-		pr_err("Unable to connect, "
-			"Version %d not supported by Hyper-V\n",
-			version);
 		return -ECONNREFUSED;
 	}
 
@@ -208,11 +205,17 @@ int vmbus_connect(void)
 		goto cleanup;
 
 	vmbus_proto_version = version;
-	pr_info("Negotiated host information %d\n", version);
+	pr_info("Hyper-V Host Build:%d-%d.%d-%d-%d.%d; Vmbus version:%d.%d\n",
+		    host_info_eax, host_info_ebx >> 16,
+		    host_info_ebx & 0xFFFF, host_info_ecx,
+		    host_info_edx >> 24, host_info_edx & 0xFFFFFF,
+		    version >> 16, version & 0xFFFF);
+
 	kfree(msginfo);
 	return 0;
 
 cleanup:
+	pr_err("Unable to connect to host\n");
 	vmbus_connection.conn_state = DISCONNECTED;
 
 	if (vmbus_connection.work_queue)
drivers/hv/hv.c +0 -7
index 0cd2da3..1c5481d 100644
@@ -74,13 +74,6 @@ static int query_hypervisor_info(void)
 		edx = 0;
 		op = HVCPUID_VERSION;
 		cpuid(op, &eax, &ebx, &ecx, &edx);
-		pr_info("Hyper-V Host OS Build:%d-%d.%d-%d-%d.%d\n",
-			    eax,
-			    ebx >> 16,
-			    ebx & 0xFFFF,
-			    ecx,
-			    edx >> 24,
-			    edx & 0xFFFFFF);
 		host_info_eax = eax;
 		host_info_ebx = ebx;
 		host_info_ecx = ecx;
Signature · 1 line
1.7.4.1
#31 lore raw
In reply to K. Y. Srinivasan
Quoted 11 lines · K. Y. Srinivasan

The current vmbus protocol supported in the vmbus driver for Linux is a version of the protocol that shipped with Windows Server 2008. Since then a lot of enhancements have been made in the protocol to specifically address performance and scalability of the communication infrastructure between the host and the guest.
This patch-set brings our driver to the Win8 (Windows Server 2012) level. Furthermore, this code will dynamically negotiate the most efficient protocol that the host can support - the same code can be deployed on all supported hosts (WS2008, WS2008R2 and WS2012).
Following are some of the key features implemented in this patch-set:

Quoted 7 lines

1. More efficient signaling protocol between the host and the guest
2. Distribution of interrupt load across available CPUs in the guest; currently all vmbus interrupts are delivered to the boot CPU (CPU 0) in the guest.
3. Per- channel interrupt binding (as part of item 2)
4. More efficient demultiplexing of incoming interrupts
5. Per-channel signaling mechanism for host to guest communication

Nice work, all now applied.

greg k-h