Skip to contentSkip to conversation

Drivers: hv/scsi: Implement multi-channel support

PATCH Superseded K. Y. Srinivasan 3 participants 16 messages May 16, 2013 – May 23, 2013

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Drivers: hv/scsi: Implement multi-channel support

This patch-set implements multi-channel support for Hyper-V devices. Also support for synthetic Fiber Channel device is included. The first two patches in the series are the foundational pieces for many of the remaining patches.

James, once Greg accepts the first two patches, you can consider the scsi patches.

In this version, based on Greg's comments, I have fixed the names of the exported symbols from the vmbus driver.

K. Y. Srinivasan (7):
Drivers: hv: vmbus: Implement multi-channel support
Drivers: hv: Add the GUID fot synthetic fiber channel device
Drivers: scsi: storvsc: Make the scsi timeout a module parameter
Drivers: scsi: storvsc: Update the storage protocol to win8 level
Drivers: scsi: storvsc: Implement multi-channel support
Drivers: scsi: storvsc: Support FC devices
Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

 drivers/hv/channel.c       |   41 +++++-
 drivers/hv/channel_mgmt.c  |  116 ++++++++++++++-
 drivers/scsi/storvsc_drv.c |  342 +++++++++++++++++++++++++++++++++++++++-----
 include/linux/hyperv.h     |   70 +++++++++
 4 files changed, 522 insertions(+), 47 deletions(-)
Signature · 1 line
1.7.4.1
#1 lore raw

PATCH 1/7 Drivers: hv: vmbus: Implement multi-channel support

Starting with Win8, the host supports multiple sub-channels for a given device. As in the past, the initial channel offer specifies the device and is associated with both the type and the instance GUIDs. For performance critical devices, the host may support multiple sub-channels. The sub-channels share the same type and instance GUID as the primary channel. The number of sub-channels offerrred to the guest depends on the number of virtual CPUs assigned to the guest. The guest can request the creation of these sub-channels and once created and opened, the guest can distribute the traffic across all the channels (the primary and the sub-channels). A request sent on a sub-channel will have the response delivered on the same sub-channel.

At channel (sub-channel) creation we bind the channel interrupt to a CPU and with this sub-channel support we will be able to spread the interrupt load of a given device across all available CPUs.

---
 drivers/hv/channel.c      |   41 ++++++++++++++--
 drivers/hv/channel_mgmt.c |  116 +++++++++++++++++++++++++++++++++++++++++++--
 include/linux/hyperv.h    |   62 +++++++++++++++++++++++-
 3 files changed, 210 insertions(+), 9 deletions(-)
drivers/hv/channel.c +37 -4
index 0b122f8..c219e9f 100644
@@ -216,6 +216,9 @@ int vmbus_open(struct vmbus_channel *newchannel, u32 send_ringbuffer_size,
 	list_del(&open_info->msglistentry);
 	spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags);
 
+	if (err == 0)
+		newchannel->state = CHANNEL_OPENED_STATE;
+
 	kfree(open_info);
 	return err;
 
@@ -500,15 +503,14 @@ int vmbus_teardown_gpadl(struct vmbus_channel *channel, u32 gpadl_handle)
 }
 EXPORT_SYMBOL_GPL(vmbus_teardown_gpadl);
 
-/*
- * vmbus_close - Close the specified channel
- */
-void vmbus_close(struct vmbus_channel *channel)
+static void vmbus_close_internal(struct vmbus_channel *channel)
 {
 	struct vmbus_channel_close_channel *msg;
 	int ret;
 	unsigned long flags;
 
+	channel->state = CHANNEL_OPEN_STATE;
+	channel->sc_creation_callback = NULL;
 	/* Stop callback and cancel the timer asap */
 	spin_lock_irqsave(&channel->inbound_lock, flags);
 	channel->onchannel_callback = NULL;
@@ -538,6 +540,37 @@ void vmbus_close(struct vmbus_channel *channel)
 
 
 }
+
+/*
+ * vmbus_close - Close the specified channel
+ */
+void vmbus_close(struct vmbus_channel *channel)
+{
+	struct list_head *cur, *tmp;
+	struct vmbus_channel *cur_channel;
+
+	if (channel->primary_channel != NULL) {
+		/*
+		 * We will only close sub-channels when
+		 * the primary is closed.
+		 */
+		return;
+	}
+	/*
+	 * Close all the sub-channels first and then close the
+	 * primary channel.
+	 */
+	list_for_each_safe(cur, tmp, &channel->sc_list) {
+		cur_channel = list_entry(cur, struct vmbus_channel, sc_list);
+		if (cur_channel->state != CHANNEL_OPENED_STATE)
+			continue;
+		vmbus_close_internal(cur_channel);
+	}
+	/*
+	 * Now close the primary.
+	 */
+	vmbus_close_internal(channel);
+}
 EXPORT_SYMBOL_GPL(vmbus_close);
 
 /**
drivers/hv/channel_mgmt.c +112 -4

143 lines not on this page · raw message

include/linux/hyperv.h +61 -1

86 lines not on this page · raw message

Signature · 1 line
1.7.4.1
#2 lore raw

PATCH 2/7 Drivers: hv: Add the GUID fot synthetic fiber channel device

In preparation for supporting synthetic Fiber Channel device, add the GUID for this service.

---
 include/linux/hyperv.h |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
include/linux/hyperv.h +10 -0
index 405e05a..fae8bac 100644
@@ -1389,6 +1389,16 @@ void vmbus_driver_unregister(struct hv_driver *hv_driver);
 		}
 
 /*
+ * Synthetic FC GUID
+ * {2f9bcc4a-0069-4af3-b76b-6fd0be528cda}
+ */
+#define HV_SYNTHFC_GUID \
+	.guid = { \
+			0x4A, 0xCC, 0x9B, 0x2F, 0x69, 0x00, 0xF3, 0x4A, \
+			0xB7, 0x6B, 0x6F, 0xD0, 0xBE, 0x52, 0x8C, 0xDA \
+		}
+
+/*
  * Common header for Hyper-V ICs
  */
 
Signature · 1 line
1.7.4.1
#4 lore raw

PATCH 3/7 Drivers: scsi: storvsc: Make the scsi timeout a module parameter

In reply to K. Y. Srinivasan

The standard scsi timeout is not appropriate in some of the environments where Hyper-V is deployed. Set this timeout appropriately for all devices managed by this driver. Further make this a module parameter.

---
 drivers/scsi/storvsc_drv.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
drivers/scsi/storvsc_drv.c +9 -0
index 16a3a0c..8d29a95 100644
@@ -221,6 +221,13 @@ static int storvsc_ringbuffer_size = (20 * PAGE_SIZE);
 module_param(storvsc_ringbuffer_size, int, S_IRUGO);
 MODULE_PARM_DESC(storvsc_ringbuffer_size, "Ring buffer size (bytes)");
 
+/*
+ * Timeout in seconds for all devices managed by this driver.
+ */
+static int storvsc_timeout = 180;
+module_param(storvsc_timeout, uint, (S_IRUGO | S_IWUSR));
+MODULE_PARM_DESC(storvsc_timeout, "Device timeout (seconds)");
+
 #define STORVSC_MAX_IO_REQUESTS				128
 
 /*
@@ -1204,6 +1211,8 @@ static int storvsc_device_configure(struct scsi_device *sdevice)
 
 	blk_queue_bounce_limit(sdevice->request_queue, BLK_BOUNCE_ANY);
 
+	blk_queue_rq_timeout(sdevice->request_queue, (storvsc_timeout * HZ));
+
 	sdevice->no_write_same = 1;
 
 	return 0;
Signature · 1 line
1.7.4.1
#5 lore raw

PATCH 4/7 Drivers: scsi: storvsc: Update the storage protocol to win8 level

In reply to K. Y. Srinivasan

Update the storage protocol to the win8 level.

---
 drivers/scsi/storvsc_drv.c |  203 ++++++++++++++++++++++++++++++++++++--------
 1 files changed, 168 insertions(+), 35 deletions(-)
drivers/scsi/storvsc_drv.c +168 -35
index 8d29a95..a3cf85c 100644
@@ -55,10 +55,15 @@
  * V1 RC < 2008/1/31: 1.0
  * V1 RC > 2008/1/31:  2.0
  * Win7: 4.2
+ * Win8: 5.1
  */
 
-#define VMSTOR_CURRENT_MAJOR  4
-#define VMSTOR_CURRENT_MINOR  2
+
+#define VMSTOR_WIN7_MAJOR 4
+#define VMSTOR_WIN7_MINOR 2
+
+#define VMSTOR_WIN8_MAJOR 5
+#define VMSTOR_WIN8_MINOR 1
 
 
 /*  Packet structure describing virtual storage requests. */
@@ -74,18 +79,103 @@ enum vstor_packet_operation {
 	VSTOR_OPERATION_QUERY_PROTOCOL_VERSION	= 9,
 	VSTOR_OPERATION_QUERY_PROPERTIES	= 10,
 	VSTOR_OPERATION_ENUMERATE_BUS		= 11,
-	VSTOR_OPERATION_MAXIMUM			= 11
+	VSTOR_OPERATION_FCHBA_DATA              = 12,
+	VSTOR_OPERATION_CREATE_SUB_CHANNELS     = 13,
+	VSTOR_OPERATION_MAXIMUM                 = 13
+};
+
+/*
+ * WWN packet for Fiber Channel HBA
+ */
+
+struct hv_fc_wwn_packet {
+	bool	primary_active;
+	u8	reserved1;
+	u8	reserved2;
+	u8	primary_port_wwn[8];
+	u8	primary_node_wwn[8];
+	u8	secondary_port_wwn[8];
+	u8	secondary_node_wwn[8];
 };
 
+
+
+/*
+ * SRB Flag Bits
+ */
+
+#define SRB_FLAGS_QUEUE_ACTION_ENABLE		0x00000002
+#define SRB_FLAGS_DISABLE_DISCONNECT		0x00000004
+#define SRB_FLAGS_DISABLE_SYNCH_TRANSFER	0x00000008
+#define SRB_FLAGS_BYPASS_FROZEN_QUEUE		0x00000010
+#define SRB_FLAGS_DISABLE_AUTOSENSE		0x00000020
+#define SRB_FLAGS_DATA_IN			0x00000040
+#define SRB_FLAGS_DATA_OUT			0x00000080
+#define SRB_FLAGS_NO_DATA_TRANSFER		0x00000000
+#define SRB_FLAGS_UNSPECIFIED_DIRECTION	(SRB_FLAGS_DATA_IN | SRB_FLAGS_DATA_OUT)
+#define SRB_FLAGS_NO_QUEUE_FREEZE		0x00000100
+#define SRB_FLAGS_ADAPTER_CACHE_ENABLE		0x00000200
+#define SRB_FLAGS_FREE_SENSE_BUFFER		0x00000400
+
+/*
+ * This flag indicates the request is part of the workflow for processing a D3.
+ */
+#define SRB_FLAGS_D3_PROCESSING			0x00000800
+#define SRB_FLAGS_IS_ACTIVE			0x00010000
+#define SRB_FLAGS_ALLOCATED_FROM_ZONE		0x00020000
+#define SRB_FLAGS_SGLIST_FROM_POOL		0x00040000
+#define SRB_FLAGS_BYPASS_LOCKED_QUEUE		0x00080000
+#define SRB_FLAGS_NO_KEEP_AWAKE			0x00100000
+#define SRB_FLAGS_PORT_DRIVER_ALLOCSENSE	0x00200000
+#define SRB_FLAGS_PORT_DRIVER_SENSEHASPORT	0x00400000
+#define SRB_FLAGS_DONT_START_NEXT_PACKET	0x00800000
+#define SRB_FLAGS_PORT_DRIVER_RESERVED		0x0F000000
+#define SRB_FLAGS_CLASS_DRIVER_RESERVED		0xF0000000
+
+
 /*
  * Platform neutral description of a scsi request -
  * this remains the same across the write regardless of 32/64 bit
  * note: it's patterned off the SCSI_PASS_THROUGH structure
  */
 #define STORVSC_MAX_CMD_LEN			0x10
-#define STORVSC_SENSE_BUFFER_SIZE		0x12
+
+#define POST_WIN7_STORVSC_SENSE_BUFFER_SIZE	0x14
+#define PRE_WIN8_STORVSC_SENSE_BUFFER_SIZE	0x12
+
+#define STORVSC_SENSE_BUFFER_SIZE		0x14
 #define STORVSC_MAX_BUF_LEN_WITH_PADDING	0x14
 
+/*
+ * Sense buffer size changed in win8; have a run-time
+ * variable to track the size we should use.
+ */
+static int sense_buffer_size;
+
+/*
+ * The size of the vmscsi_request has changed in win8. The
+ * additional size is because of new elements added to the
+ * structure. These elements are valid only when we are talking
+ * to a win8 host.
+ * Track the correction to size we need to apply.
+ */
+
+static int vmscsi_size_delta;
+static int vmstor_current_major;
+static int vmstor_current_minor;
+
+struct vmscsi_win8_extension {
+	/*
+	 * The following were added in Windows 8
+	 */
+	u16 reserve;
+	u8  queue_tag;
+	u8  queue_action;
+	u32 srb_flags;
+	u32 time_out_value;
+	u32 queue_sort_ey;
+} __packed;
+
 struct vmscsi_request {
 	u16 length;
 	u8 srb_status;
@@ -108,6 +198,11 @@ struct vmscsi_request {
 		u8 sense_data[STORVSC_SENSE_BUFFER_SIZE];
 		u8 reserved_array[STORVSC_MAX_BUF_LEN_WITH_PADDING];
 	};
+	/*
+	 * The following was added in win8.
+	 */
+	struct vmscsi_win8_extension win8_extension;
+
 } __attribute((packed));
 
 
@@ -115,22 +210,18 @@ struct vmscsi_request {
  * This structure is sent during the intialization phase to get the different
  * properties of the channel.
  */
+
+#define STORAGE_CHANNEL_SUPPORTS_MULTI_CHANNEL		0x1
+
 struct vmstorage_channel_properties {
-	u16 protocol_version;
-	u8  path_id;
-	u8 target_id;
+	u32 reserved;
+	u16 max_channel_cnt;
+	u16 reserved1;
 
-	/* Note: port number is only really known on the client side */
-	u32  port_number;
-	u32  flags;
+	u32 flags;
 	u32   max_transfer_bytes;
 
-	/*
-	 * This id is unique for each channel and will correspond with
-	 * vendor specific data in the inquiry data.
-	 */
-
-	u64  unique_id;
+	u64  reserved2;
 } __packed;
 
 /*  This structure is sent during the storage protocol negotiations. */
@@ -175,6 +266,15 @@ struct vstor_packet {
 
 		/* Used during version negotiations. */
 		struct vmstorage_protocol_version version;
+
+		/* Fiber channel address packet */
+		struct hv_fc_wwn_packet wwn_packet;
+
+		/* Number of sub-channels to create */
+		u16 sub_channel_count;
+
+		/* This will be the maximum of the union members */
+		u8  buffer[0x34];
 	};
 } __packed;
 
@@ -681,7 +781,8 @@ static int storvsc_channel_init(struct hv_device *device)
 	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
 
 	ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+			       vmscsi_size_delta),
 			       (unsigned long)request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -705,7 +806,7 @@ static int storvsc_channel_init(struct hv_device *device)
 	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
 
 	vstor_packet->version.major_minor =
-		storvsc_get_version(VMSTOR_CURRENT_MAJOR, VMSTOR_CURRENT_MINOR);
+		storvsc_get_version(vmstor_current_major, vmstor_current_minor);
 
 	/*
 	 * The revision number is only used in Windows; set it to 0.
@@ -713,7 +814,8 @@ static int storvsc_channel_init(struct hv_device *device)
 	vstor_packet->version.revision = 0;
 
 	ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 			       (unsigned long)request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -734,11 +836,10 @@ static int storvsc_channel_init(struct hv_device *device)
 	memset(vstor_packet, 0, sizeof(struct vstor_packet));
 	vstor_packet->operation = VSTOR_OPERATION_QUERY_PROPERTIES;
 	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
-	vstor_packet->storage_channel_properties.port_number =
-					stor_device->port_number;
 
 	ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 			       (unsigned long)request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -756,16 +857,13 @@ static int storvsc_channel_init(struct hv_device *device)
 	    vstor_packet->status != 0)
 		goto cleanup;
 
-	stor_device->path_id = vstor_packet->storage_channel_properties.path_id;
-	stor_device->target_id
-		= vstor_packet->storage_channel_properties.target_id;
-
 	memset(vstor_packet, 0, sizeof(struct vstor_packet));
 	vstor_packet->operation = VSTOR_OPERATION_END_INITIALIZATION;
 	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
 
 	ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 			       (unsigned long)request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -1019,7 +1117,8 @@ static void storvsc_on_channel_callback(void *context)
 
 	do {
 		ret = vmbus_recvpacket(device->channel, packet,
-				       ALIGN(sizeof(struct vstor_packet), 8),
+				       ALIGN((sizeof(struct vstor_packet) -
+					     vmscsi_size_delta), 8),
 				       &bytes_recvd, &request_id);
 		if (ret == 0 && bytes_recvd > 0) {
 
@@ -1030,7 +1129,8 @@ static void storvsc_on_channel_callback(void *context)
 			    (request == &stor_device->reset_request)) {
 
 				memcpy(&request->vstor_packet, packet,
-				       sizeof(struct vstor_packet));
+				       (sizeof(struct vstor_packet) -
+					vmscsi_size_delta));
 				complete(&request->wait_event);
 			} else {
 				storvsc_on_receive(device,
@@ -1123,10 +1223,11 @@ static int storvsc_do_io(struct hv_device *device,
 
 	vstor_packet->flags |= REQUEST_COMPLETION_FLAG;
 
-	vstor_packet->vm_srb.length = sizeof(struct vmscsi_request);
+	vstor_packet->vm_srb.length = (sizeof(struct vmscsi_request) -
+					vmscsi_size_delta);
 
 
-	vstor_packet->vm_srb.sense_info_length = STORVSC_SENSE_BUFFER_SIZE;
+	vstor_packet->vm_srb.sense_info_length = sense_buffer_size;
 
 
 	vstor_packet->vm_srb.data_transfer_length =
@@ -1138,11 +1239,13 @@ static int storvsc_do_io(struct hv_device *device,
 		ret = vmbus_sendpacket_multipagebuffer(device->channel,
 				&request->data_buffer,
 				vstor_packet,
-				sizeof(struct vstor_packet),
+				(sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 				(unsigned long)request);
 	} else {
 		ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 			       (unsigned long)request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -1266,7 +1369,8 @@ static int storvsc_host_reset_handler(struct scsi_cmnd *scmnd)
 	vstor_packet->vm_srb.path_id = stor_device->path_id;
 
 	ret = vmbus_sendpacket(device->channel, vstor_packet,
-			       sizeof(struct vstor_packet),
+			       (sizeof(struct vstor_packet) -
+				vmscsi_size_delta),
 			       (unsigned long)&stor_device->reset_request,
 			       VM_PKT_DATA_INBAND,
 			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
@@ -1351,18 +1455,28 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scmnd)
 	scmnd->host_scribble = (unsigned char *)cmd_request;
 
 	vm_srb = &cmd_request->vstor_packet.vm_srb;
+	vm_srb->win8_extension.time_out_value = 60;
 
 
 	/* Build the SRB */
 	switch (scmnd->sc_data_direction) {
 	case DMA_TO_DEVICE:
 		vm_srb->data_in = WRITE_TYPE;
+		vm_srb->win8_extension.srb_flags |= SRB_FLAGS_DATA_OUT;
+		vm_srb->win8_extension.srb_flags |=
+			(SRB_FLAGS_QUEUE_ACTION_ENABLE |
+			SRB_FLAGS_DISABLE_SYNCH_TRANSFER);
 		break;
 	case DMA_FROM_DEVICE:
 		vm_srb->data_in = READ_TYPE;
+		vm_srb->win8_extension.srb_flags |= SRB_FLAGS_DATA_IN;
+		vm_srb->win8_extension.srb_flags |=
+			(SRB_FLAGS_QUEUE_ACTION_ENABLE |
+			SRB_FLAGS_DISABLE_SYNCH_TRANSFER);
 		break;
 	default:
 		vm_srb->data_in = UNKNOWN_TYPE;
+		vm_srb->win8_extension.srb_flags = 0;
 		break;
 	}
 
@@ -1494,6 +1608,24 @@ static int storvsc_probe(struct hv_device *device,
 	int target = 0;
 	struct storvsc_device *stor_device;
 
+	/*
+	 * Based on the windows host we are running on,
+	 * set state to properly communicate with the host.
+	 */
+
+	if (vmbus_proto_version == VERSION_WIN8) {
+		sense_buffer_size = POST_WIN7_STORVSC_SENSE_BUFFER_SIZE;
+		vmscsi_size_delta = 0;
+		vmstor_current_major = VMSTOR_WIN8_MAJOR;
+		vmstor_current_minor = VMSTOR_WIN8_MINOR;
+	} else {
+		sense_buffer_size = PRE_WIN8_STORVSC_SENSE_BUFFER_SIZE;
+		vmscsi_size_delta = sizeof(struct vmscsi_win8_extension);
+		vmstor_current_major = VMSTOR_WIN7_MAJOR;
+		vmstor_current_minor = VMSTOR_WIN7_MINOR;
+	}
+
+
 	host = scsi_host_alloc(&scsi_driver,
 			       sizeof(struct hv_host_device));
 	if (!host)
@@ -1603,7 +1735,8 @@ static int __init storvsc_drv_init(void)
 	max_outstanding_req_per_channel =
 		((storvsc_ringbuffer_size - PAGE_SIZE) /
 		ALIGN(MAX_MULTIPAGE_BUFFER_PACKET +
-		sizeof(struct vstor_packet) + sizeof(u64),
+		sizeof(struct vstor_packet) + sizeof(u64) -
+		vmscsi_size_delta,
 		sizeof(u64)));
 
 	if (max_outstanding_req_per_channel <
Signature · 1 line
1.7.4.1
#6 lore raw

PATCH 5/7 Drivers: scsi: storvsc: Implement multi-channel support

In reply to K. Y. Srinivasan

Implement multi-channel support for the storage devices.

---
 drivers/scsi/storvsc_drv.c |  124 ++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 120 insertions(+), 4 deletions(-)
drivers/scsi/storvsc_drv.c +120 -4
index a3cf85c..6bc4e3b 100644
@@ -330,6 +330,8 @@ MODULE_PARM_DESC(storvsc_timeout, "Device timeout (seconds)");
 
 #define STORVSC_MAX_IO_REQUESTS				128
 
+static void storvsc_on_channel_callback(void *context);
+
 /*
  * In Hyper-V, each port/path/target maps to 1 scsi host adapter.  In
  * reality, the path/target is not used (ie always set to 0) so our
@@ -757,12 +759,99 @@ static unsigned int copy_to_bounce_buffer(struct scatterlist *orig_sgl,
 	return total_copied;
 }
 
+static void handle_sc_creation(struct vmbus_channel *new_sc)
+{
+	struct hv_device *device = new_sc->primary_channel->device_obj;
+	struct storvsc_device *stor_device;
+	struct vmstorage_channel_properties props;
+
+	stor_device = get_out_stor_device(device);
+	if (!stor_device)
+		return;
+
+
+	memset(&props, 0, sizeof(struct vmstorage_channel_properties));
+
+	vmbus_open(new_sc,
+		   storvsc_ringbuffer_size,
+		   storvsc_ringbuffer_size,
+		   (void *)&props,
+		   sizeof(struct vmstorage_channel_properties),
+		   storvsc_on_channel_callback, new_sc);
+}
+
+static void  handle_multichannel_storage(struct hv_device *device, int max_chns)
+{
+	struct storvsc_device *stor_device;
+	int num_cpus = num_online_cpus();
+	int num_sc;
+	struct storvsc_cmd_request *request;
+	struct vstor_packet *vstor_packet;
+	int ret, t;
+
+	num_sc = ((max_chns > num_cpus) ? num_cpus : max_chns);
+	stor_device = get_out_stor_device(device);
+	if (!stor_device)
+		return;
+
+	request = &stor_device->init_request;
+	vstor_packet = &request->vstor_packet;
+
+	/*
+	 * Establish a handler for dealing with subchannels.
+	 */
+	vmbus_set_sc_create_callback(device->channel, handle_sc_creation);
+
+	/*
+	 * Check to see if sub-channels have already been created. This
+	 * can happen when this driver is re-loaded after unloading.
+	 */
+
+	if (vmbus_are_subchannels_present(device->channel))
+		return;
+
+	/*
+	 * Request the host to create sub-channels.
+	 */
+	memset(request, 0, sizeof(struct storvsc_cmd_request));
+	init_completion(&request->wait_event);
+	vstor_packet->operation = VSTOR_OPERATION_CREATE_SUB_CHANNELS;
+	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
+	vstor_packet->sub_channel_count = num_sc;
+
+	ret = vmbus_sendpacket(device->channel, vstor_packet,
+			       (sizeof(struct vstor_packet) -
+			       vmscsi_size_delta),
+			       (unsigned long)request,
+			       VM_PKT_DATA_INBAND,
+			       VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
+
+	if (ret != 0)
+		return;
+
+	t = wait_for_completion_timeout(&request->wait_event, 10*HZ);
+	if (t == 0)
+		return;
+
+	if (vstor_packet->operation != VSTOR_OPERATION_COMPLETE_IO ||
+	    vstor_packet->status != 0)
+		return;
+
+	/*
+	 * Now that we created the sub-channels, invoke the check; this
+	 * will trigger the callback.
+	 */
+	vmbus_are_subchannels_present(device->channel);
+}
+
 static int storvsc_channel_init(struct hv_device *device)
 {
 	struct storvsc_device *stor_device;
 	struct storvsc_cmd_request *request;
 	struct vstor_packet *vstor_packet;
 	int ret, t;
+	int max_chns;
+	bool process_sub_channels = false;
 
 	stor_device = get_out_stor_device(device);
 	if (!stor_device)
@@ -857,6 +946,19 @@ static int storvsc_channel_init(struct hv_device *device)
 	    vstor_packet->status != 0)
 		goto cleanup;
 
+	/*
+	 * Check to see if multi-channel support is there.
+	 * Hosts that implement protocol version of 5.1 and above
+	 * support multi-channel.
+	 */
+	max_chns = vstor_packet->storage_channel_properties.max_channel_cnt;
+	if ((vmbus_proto_version != VERSION_WIN7) &&
+	   (vmbus_proto_version != VERSION_WS2008))  {
+		if (vstor_packet->storage_channel_properties.flags &
+		    STORAGE_CHANNEL_SUPPORTS_MULTI_CHANNEL)
+			process_sub_channels = true;
+	}
+
 	memset(vstor_packet, 0, sizeof(struct vstor_packet));
 	vstor_packet->operation = VSTOR_OPERATION_END_INITIALIZATION;
 	vstor_packet->flags = REQUEST_COMPLETION_FLAG;
@@ -881,6 +983,9 @@ static int storvsc_channel_init(struct hv_device *device)
 	    vstor_packet->status != 0)
 		goto cleanup;
 
+	if (process_sub_channels)
+		handle_multichannel_storage(device, max_chns);
+
 
 cleanup:
 	return ret;
@@ -1102,7 +1207,8 @@ static void storvsc_on_receive(struct hv_device *device,
 
 static void storvsc_on_channel_callback(void *context)
 {
-	struct hv_device *device = (struct hv_device *)context;
+	struct vmbus_channel *channel = (struct vmbus_channel *)context;
+	struct hv_device *device;
 	struct storvsc_device *stor_device;
 	u32 bytes_recvd;
 	u64 request_id;
@@ -1110,13 +1216,17 @@ static void storvsc_on_channel_callback(void *context)
 	struct storvsc_cmd_request *request;
 	int ret;
 
+	if (channel->primary_channel != NULL)
+		device = channel->primary_channel->device_obj;
+	else
+		device = channel->device_obj;
 
 	stor_device = get_in_stor_device(device);
 	if (!stor_device)
 		return;
 
 	do {
-		ret = vmbus_recvpacket(device->channel, packet,
+		ret = vmbus_recvpacket(channel, packet,
 				       ALIGN((sizeof(struct vstor_packet) -
 					     vmscsi_size_delta), 8),
 				       &bytes_recvd, &request_id);
@@ -1157,7 +1267,7 @@ static int storvsc_connect_to_vsp(struct hv_device *device, u32 ring_size)
 			 ring_size,
 			 (void *)&props,
 			 sizeof(struct vmstorage_channel_properties),
-			 storvsc_on_channel_callback, device);
+			 storvsc_on_channel_callback, device->channel);
 
 	if (ret != 0)
 		return ret;
@@ -1209,6 +1319,7 @@ static int storvsc_do_io(struct hv_device *device,
 {
 	struct storvsc_device *stor_device;
 	struct vstor_packet *vstor_packet;
+	struct vmbus_channel *outgoing_channel;
 	int ret = 0;
 
 	vstor_packet = &request->vstor_packet;
@@ -1219,6 +1330,11 @@ static int storvsc_do_io(struct hv_device *device,
 
 
 	request->device  = device;
+	/*
+	 * Select an an appropriate channel to send the request out.
+	 */
+
+	outgoing_channel = vmbus_get_outgoing_channel(device->channel);
 
 
 	vstor_packet->flags |= REQUEST_COMPLETION_FLAG;
@@ -1236,7 +1352,7 @@ static int storvsc_do_io(struct hv_device *device,
 	vstor_packet->operation = VSTOR_OPERATION_EXECUTE_SRB;
 
 	if (request->data_buffer.len) {
-		ret = vmbus_sendpacket_multipagebuffer(device->channel,
+		ret = vmbus_sendpacket_multipagebuffer(outgoing_channel,
 				&request->data_buffer,
 				vstor_packet,
 				(sizeof(struct vstor_packet) -
Signature · 1 line
1.7.4.1
#7 lore raw

PATCH 6/7 Drivers: scsi: storvsc: Support FC devices

In reply to K. Y. Srinivasan

Add support for synthetic Fiber Channel devices.

---
 drivers/scsi/storvsc_drv.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
drivers/scsi/storvsc_drv.c +6 -0
index 6bc4e3b..124cf0b 100644
@@ -1698,6 +1698,7 @@ static struct scsi_host_template scsi_driver = {
 enum {
 	SCSI_GUID,
 	IDE_GUID,
+	SFC_GUID,
 };
 
 static const struct hv_vmbus_device_id id_table[] = {
@@ -1709,6 +1710,11 @@ static const struct hv_vmbus_device_id id_table[] = {
 	{ HV_IDE_GUID,
 	  .driver_data = IDE_GUID
 	},
+	/* Fiber Channel GUID */
+	{
+	  HV_SYNTHFC_GUID,
+	  .driver_data = SFC_GUID
+	},
 	{ },
 };
 
Signature · 1 line
1.7.4.1
#8 lore raw

PATCH 7/7 Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

In reply to K. Y. Srinivasan

Increase the value of STORVSC_MAX_IO_REQUESTS to 200 requests. The current ringbuffer size can support this higher value.

---
 drivers/scsi/storvsc_drv.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/scsi/storvsc_drv.c +1 -1
index 124cf0b..e2ab7cd 100644
@@ -328,7 +328,7 @@ static int storvsc_timeout = 180;
 module_param(storvsc_timeout, uint, (S_IRUGO | S_IWUSR));
 MODULE_PARM_DESC(storvsc_timeout, "Device timeout (seconds)");
 
-#define STORVSC_MAX_IO_REQUESTS				128
+#define STORVSC_MAX_IO_REQUESTS				200
 
 static void storvsc_on_channel_callback(void *context);
 
Signature · 1 line
1.7.4.1
#9 lore raw

Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 3 lines · K. Y. Srinivasan

Increase the value of STORVSC_MAX_IO_REQUESTS to 200 requests. The current ringbuffer size can support this higher value.

The ringbuffer size is a module parameter so it's odd to talk about the "current" size.

regards,
dan carpenter

#10 lore raw

RE: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 13 lines

-----Original Message-----
From: Dan Carpenter [mailto:[email protected]]
Sent: Thursday, May 16, 2013 8:02 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 8 lines
Quoted 1 line · K. Y. Srinivasan

Increase the value of STORVSC_MAX_IO_REQUESTS to 200 requests. The

current

Quoted 2 lines

ringbuffer size can support this higher value.

The ringbuffer size is a module parameter so it's odd to talk about the "current" size.

While the ringbuffer size is a module parameter; there is a default value. The current size refers to the default. Your comment applies to the current value (of 128) as well in that it is possible for somebody to load this driver with a ringbuffer size that could not support the value of 128. If this is the case, we fail the load. This safety check continues to exist.

Regards,

K. Y

Quoted 5 lines

regards,
dan carpenter

#11 lore raw

Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 19 lines · KY Srinivasan
Quoted 19 lines

-----Original Message-----
From: Dan Carpenter [mailto:[email protected]]
Sent: Thursday, May 16, 2013 8:02 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 1 line · K. Y. Srinivasan

Increase the value of STORVSC_MAX_IO_REQUESTS to 200 requests. The

current

Quoted 2 lines

ringbuffer size can support this higher value.

Quoted 7 lines
Quoted 2 lines

The ringbuffer size is a module parameter so it's odd to talk about the "current" size.

While the ringbuffer size is a module parameter; there is a default value. The current size refers to the default. Your comment applies to the current value (of 128) as well in that it is possible for somebody to load this driver with a ringbuffer size that could not support the value of 128. If this is the case, we fail the load. This safety check continues to exist.

The issue is there in the original code, true.

Would the right fix be to add some sanity checks in module_init()?

regards,
dan carpenter

#12 lore raw

RE: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 40 lines

-----Original Message-----
From: Dan Carpenter [mailto:[email protected]]
Sent: Thursday, May 16, 2013 9:55 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 8 lines · KY Srinivasan
Quoted 8 lines

-----Original Message-----
From: Dan Carpenter [mailto:[email protected]]
Sent: Thursday, May 16, 2013 8:02 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected];

[email protected];

Quoted 7 lines
Quoted 7 lines

[email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 1 line · K. Y. Srinivasan

Increase the value of STORVSC_MAX_IO_REQUESTS to 200 requests. The

Quoted 8 lines
Quoted 6 lines

current

Quoted 2 lines

ringbuffer size can support this higher value.

The ringbuffer size is a module parameter so it's odd to talk about the "current" size.

While the ringbuffer size is a module parameter; there is a default value. The

current size refers to the default.

Quoted 1 line

Your comment applies to the current value (of 128) as well in that it is possible

Quoted 8 lines

for somebody to load this

Quoted 1 line

driver with a ringbuffer size that could not support the value of 128. If this is

the case, we fail the load.

Quoted 1 line

This safety check continues to exist.

The issue is there in the original code, true.

Would the right fix be to add some sanity checks in module_init()?

The check is already there (as I noted above). Look at the function:
storvsc_drv_init(). If the ring size is picked incorrectly, the load is failed.

Regards,

K. Y

Quoted 5 lines

regards,
dan carpenter

#13 lore raw

Re: [PATCH V1 7/7] Drivers: scsi: storvsc: Increase the value of STORVSC_MAX_IO_REQUESTS

Quoted 5 lines · KY Srinivasan
Quoted 1 line

Would the right fix be to add some sanity checks in module_init()?

The check is already there (as I noted above). Look at the function:
storvsc_drv_init(). If the ring size is picked incorrectly, the load is failed.

Ah. I see that now. My mistake.

regards,
dan carpenter

Acked-by on v1 1/7
#14 lore raw

Re: [PATCH V1 1/7] Drivers: hv: vmbus: Implement multi-channel support

In reply to K. Y. Srinivasan
Quoted 11 lines · K. Y. Srinivasan

Starting with Win8, the host supports multiple sub-channels for a given device. As in the past, the initial channel offer specifies the device and is associated with both the type and the instance GUIDs. For performance critical devices, the host may support multiple sub-channels. The sub-channels share the same type and instance GUID as the primary channel. The number of sub-channels offerrred to the guest depends on the number of virtual CPUs assigned to the guest. The guest can request the creation of these sub-channels and once created and opened, the guest can distribute the traffic across all the channels (the primary and the sub-channels). A request sent on a sub-channel will have the response delivered on the same sub-channel.

Quoted 6 lines

At channel (sub-channel) creation we bind the channel interrupt to a CPU and with this sub-channel support we will be able to spread the interrupt load of a given device across all available CPUs.

#15 lore raw

RE: [PATCH V1 1/7] Drivers: hv: vmbus: Implement multi-channel support

Quoted 25 lines

-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Thursday, May 16, 2013 10:01 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH V1 1/7] Drivers: hv: vmbus: Implement multi-channel support

Quoted 7 lines · K. Y. Srinivasan

Starting with Win8, the host supports multiple sub-channels for a given device. As in the past, the initial channel offer specifies the device and is associated with both the type and the instance GUIDs. For performance critical devices, the host may support multiple sub-channels. The sub-channels share the same type and instance GUID as the primary channel. The number of sub-channels offerrred to the guest depends on the number of virtual CPUs assigned to the guest. The guest can request the creation of these sub-

channels

Quoted 2 lines

and once created and opened, the guest can distribute the traffic across all the channels (the primary and the sub-channels). A request sent on a sub-

channel

Quoted 2 lines

will have the response delivered on the same sub-channel.

Quoted 9 lines
Quoted 6 lines

At channel (sub-channel) creation we bind the channel interrupt to a CPU and with this sub-channel support we will be able to spread the interrupt load of a given device across all available CPUs.

James,

Please drop this patch-set. I fixed a couple of bugs in the vmbus support for multi-channel and I would need to resubmit this. Greg, I think it may make sense to first get the vmbus support checked in before I submit the storage related patches. There are networking related patches for multi-channel support that we will be submitting shortly. Getting the vmbus support in first would allow us to decouple network and storage patches. Greg, I will shortly submit the next version of the vmbus related patches for your consideration.

Regards,

K. Y