Drivers: hv: Miscellaneous fixes
cover Drivers: hv: Miscellaneous fixes
From: "K. Y. Srinivasan" <[email protected]>
Miscellaneous fixes.
Dexuan Cui (2):
vmbus: don't acquire the mutex in vmbus_hvsock_device_unregister()
vmbus: suppress uevents for hv_sock devices
Olaf Hering (1):
Drivers: hv: fcopy: restore correct transfer length
Stephen Hemminger (2):
vmbus: add per-channel sysfs info
Drivers: hv: vmbus: Expose per-channel interrupts and events counters
Documentation/ABI/stable/sysfs-bus-vmbus | 70 +++++++++++ drivers/hv/channel_mgmt.c | 14 ++- drivers/hv/connection.c | 2 + drivers/hv/hv_fcopy.c | 4 + drivers/hv/hyperv_vmbus.h | 2 + drivers/hv/vmbus_drv.c | 205 +++++++++++++++++++++++++++++-- include/linux/hyperv.h | 10 ++ 7 files changed, 290 insertions(+), 17 deletions(-)
Signature · 1 line
2.14.1
PATCH 1/5 vmbus: don't acquire the mutex in vmbus_hvsock_device_unregister()
From: Dexuan Cui <[email protected]>
Due to commit 54a66265d675 ("Drivers: hv: vmbus: Fix rescind handling"), we need this patch to resolve the below deadlock:
after we get the mutex in vmbus_hvsock_device_unregister() and call vmbus_device_unregister() -> device_unregister() -> ... -> device_release() -> vmbus_device_release(), we'll get a deadlock, because vmbus_device_release() tries to get the same mutex.
--- drivers/hv/channel_mgmt.c | 4 ---- 1 file changed, 4 deletions(-)
drivers/hv/channel_mgmt.c +0 -4
@@ -945,14 +945,10 @@ static void vmbus_onoffer_rescind(struct vmbus_channel_message_header *hdr)void vmbus_hvsock_device_unregister (struct vmbus_channel *channel) { -mutex_lock (&vmbus_connection.channel_mutex); -BUG_ON (!is_hvsock_channel (channel)); channel->rescind = true;vmbus_device_unregister (channel->device_obj); - -mutex_unlock (&vmbus_connection.channel_mutex); }EXPORT_SYMBOL_GPL (vmbus_hvsock_device_unregister);
Signature · 1 line
2.14.1
PATCH 2/5 vmbus: suppress uevents for hv_sock devices
From: Dexuan Cui <[email protected]>
hv_sock driver is automatically loaded when an application creates an AF_VSOCK socket, so we don't really need to trigger uevents to the user space udevd.
And hv_sock devices can appear and disappear frequency, e.g. 100 per second, so triggering the udevents can cause a high cpu utilization of udevd, e.g. 30% on a 2-cpu virtual machine. So let's suppress the uevents to avoid this.
--- drivers/hv/vmbus_drv.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/hv/vmbus_drv.c +2 -0
@@ -1174,6 +1174,8 @@ int vmbus_device_register(struct hv_device *child_device_obj) child_device_obj->device.parent = &hv_acpi_dev->dev; child_device_obj->device.release = vmbus_device_release; +if (is_hvsock_channel (child_device_obj->channel)) +dev_set_uevent_suppress (&child_device_obj->device,1 );/* * Register with the LDM. This will kick off the driver/device * binding...which will eventually call vmbus_match() and vmbus_probe()
Signature · 1 line
2.14.1
Re: [PATCH 2/5] vmbus: suppress uevents for hv_sock devices
Quoted 10 lines · On Sat, Sep 09, 2017 at 11:08:46PM -0700, [email protected] wrote:
From: Dexuan Cui <[email protected]>
hv_sockdriver is automatically loaded when an application creates anAF_VSOCKsocket, so we don't really need to trigger uevents to the user space udevd.And
hv_sockdevices can appear and disappear frequency, e.g. 100 per second, so triggering the udevents can cause a high cpu utilization of udevd, e.g. 30% on a 2-cpu virtual machine. So let's suppress the uevents to avoid this.
100 per second for a struct device? That's crazy, and the uevent is the least of your worries. Please fix that, as it's not the correct way to use the driver model at all.
And really, why is uevent taking all that much cpu time anyway? It _should_ be pretty fast, unless your distro is doing crazy things with it...
sorry, am not going to take this patch.
greg k-h
RE: [PATCH 2/5] vmbus: suppress uevents for hv_sock devices
Quoted 31 lines
-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Sunday, September 10, 2017 1:10 AM
To: KY Srinivasan <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Stephen Hemminger <[email protected]>; Dexuan Cui <[email protected]>; Haiyang Zhang <[email protected]>
Subject: Re: [PATCH 2/5] vmbus: suppress uevents forhv_sockdevicesQuoted 10 lines · On Sat, Sep 09, 2017 at 11:08:46PM -0700, [email protected] wrote:
From: Dexuan Cui <[email protected]>
hv_sockdriver is automatically loaded when an application creates anAF_VSOCKsocket, so we don't really need to trigger uevents to the user space udevd.And
hv_sockdevices can appear and disappear frequency, e.g. 100 per second, so triggering the udevents can cause a high cpu utilization of udevd, e.g. 30% on a 2-cpu virtual machine. So let's suppress the uevents to avoid this.100 per second for a struct device? That's crazy, and the uevent is the least of your worries. Please fix that, as it's not the correct way to use the driver model at all.
Quoted 5 lines
And really, why is uevent taking all that much cpu time anyway? It _should_ be pretty fast, unless your distro is doing crazy things with it...
sorry, am not going to take this patch.
Greg,
This is not a real problem that needs fixing. The test automation triggered this condition. I will drop this patch and send the rest.
Regards,
K. Y
Quoted 2 lines
greg k-h
PATCH 3/5 Drivers: hv: fcopy: restore correct transfer length
In reply to [email protected]From: Olaf Hering <[email protected]>
Till recently the expected length of bytes read by the daemon did depend on the context. It was either hv_start_fcopy or hv_do_fcopy. The daemon had a buffer size of two pages, which was much larger than needed.
Now the expected length of bytes read by the
daemon changed slightly. For START_FILE_COPY it is still the size of hv_start_fcopy. But for WRITE_TO_FILE and the other operations it is as large as the buffer that arrived via vmbus. In case of WRITE_TO_FILE that is slightly larger than a struct hv_do_fcopy. Since the buffer in the daemon was still larger everything was fine.
Currently, the daemon reads only what is actually needed. The new buffer layout is as large as a struct hv_do_fcopy, for the WRITE_TO_FILE operation. Since the kernel expects a slightly larger size, hvt_op_read will return -EINVAL because the daemon will read slightly less than expected. Address this by restoring the expected buffer size in case of WRITE_TO_FILE.
--- drivers/hv/hv_fcopy.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/hv/hv_fcopy.c +4 -0
@@ -170,6 +170,10 @@ static void fcopy_send_data(struct work_struct *dummy) out_src = smsg_out;break ; +case WRITE_TO_FILE : + out_src = fcopy_transaction.fcopy_msg; + out_len =sizeof (struct hv_do_fcopy ); +break ;default : out_src = fcopy_transaction.fcopy_msg; out_len = fcopy_transaction.recv_len;
Signature · 1 line
2.14.1
PATCH 4/5 vmbus: add per-channel sysfs info
In reply to [email protected]From: Stephen Hemminger <[email protected]>
This extends existing vmbus related sysfs structure to provide per-channel state information. This is useful when diagnosing issues with multiple queues in networking and storage.
The existing sysfs only displayed information about the primary channel. The one place it reported multiple channels was the channel_vp_mapping file which violated the sysfs convention of one value per file.
--- Documentation/ABI/stable/sysfs-bus-vmbus | 56 ++++++++++ drivers/hv/channel_mgmt.c | 10 +- drivers/hv/hyperv_vmbus.h | 2 + drivers/hv/vmbus_drv.c | 185 +++++++++++++++++++++++++++++-- include/linux/hyperv.h | 6 + 5 files changed, 246 insertions(+), 13 deletions(-)
Documentation/ABI/stable/sysfs-bus-vmbus +56 -0
@@ -41,3 +41,59 @@ KernelVersion: 4.5 Contact: K. Y. Srinivasan <[email protected]> Description: The 16 bit vendor ID of the device Users: tools/hv/lsvmbus and user level RDMA libraries + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/cpu +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: VCPU (sub)channel is affinitized to +Users: tools/hv/lsvmbus and other debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/cpu +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: VCPU (sub)channel is affinitized to +Users: tools/hv/lsvmbus and other debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/in_mask +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Inbound channel signaling state +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/latency +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Channel signaling latency +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/out_mask +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Outbound channel signaling state +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/pending +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Channel interrupt pending state +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/read_avail +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Bytes availabble to read +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/write_avail +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Bytes availabble to write +Users: Debuggig tools
drivers/hv/channel_mgmt.c +9 -1
@@ -350,7 +350,7 @@ static void free_channel(struct vmbus_channel *channel) {tasklet_kill (&channel->callback_event); -kfree_rcu (channel, rcu); +kobject_put (&channel->kobj); }static void percpu_channel_enq (void *arg)
@@ -513,6 +513,14 @@ static void vmbus_process_offer(struct vmbus_channel *newchannel) newchannel->state =CHANNEL_OPEN_STATE ;if (!fnew) { +struct hv_device *dev + = newchannel->primary_channel->device_obj; + +if (vmbus_add_channel_kobj (dev, newchannel)) { +atomic_dec (&vmbus_connection.offer_in_progress); + goto err_free_chan; + } +if (channel->sc_creation_callback !=NULL ) channel->sc_creation_callback (newchannel);return ;
drivers/hv/hyperv_vmbus.h +2 -0
@@ -384,6 +384,8 @@ struct hv_device *vmbus_device_create(const uuid_le *type,int vmbus_device_register (struct hv_device *child_device_obj);void vmbus_device_unregister (struct hv_device *device_obj); +int vmbus_add_channel_kobj (struct hv_device *device_obj, +struct vmbus_channel *channel);struct vmbus_channel *relid2channel (u32 relid);
drivers/hv/vmbus_drv.c +173 -12
227 lines not on this page · raw message
include/linux/hyperv.h +6 -0
@@ -841,6 +841,11 @@ struct vmbus_channel {*/ struct rcu_head rcu; +/* +* For sysfs per-channel properties. +*/ +struct kobject kobj; +/* * For performance critical channels (storage, networking * etc,), Hyper-V has a mechanism to enhance the throughput
@@ -1123,6 +1128,7 @@ struct hv_device {struct device device;struct vmbus_channel *channel; +struct kset *channels_kset; };
Signature · 1 line
2.14.1
PATCH 5/5 Drivers: hv: vmbus: Expose per-channel event counters
In reply to [email protected]From: Stephen Hemminger <[email protected]>
When investigating performance, it is useful to be able to look at the number of host and guest events per-channel. This is equivalent to per-device interrupt statistics.
--- Documentation/ABI/stable/sysfs-bus-vmbus | 14 ++++++++++++++ drivers/hv/connection.c | 2 ++ drivers/hv/vmbus_drv.c | 18 ++++++++++++++++++ include/linux/hyperv.h | 4 ++++ 4 files changed, 38 insertions(+)
Documentation/ABI/stable/sysfs-bus-vmbus +14 -0
@@ -97,3 +97,17 @@ KernelVersion: 4.14 Contact: Stephen Hemminger <[email protected]> Description: Bytes availabble to write Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/events +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have signaled the host +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/interrupts +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have taken an interrupt (incoming) +Users: Debuggig tools
drivers/hv/connection.c +2 -0
@@ -406,6 +406,8 @@ void vmbus_set_event(struct vmbus_channel *channel)if (!channel->is_dedicated_interrupt)vmbus_send_interrupt (child_relid); + ++channel->sig_events; +hv_do_hypercall (HVCALL_SIGNAL_EVENT , channel->sig_event,NULL ); }EXPORT_SYMBOL_GPL (vmbus_set_event);
drivers/hv/vmbus_drv.c +18 -0
@@ -945,6 +945,8 @@ static void vmbus_chan_sched(struct hv_per_cpu_context *hv_cpu)if (channel->rescind)continue ; + ++channel->interrupts; +switch (channel->callback_mode) {case HV_CALL_ISR :vmbus_channel_isr (channel);
@@ -1238,6 +1240,20 @@ static ssize_t channel_latency_show(const struct vmbus_channel *channel, }VMBUS_CHAN_ATTR (latency,S_IRUGO , channel_latency_show,NULL ); +static ssize_t channel_interrupts_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->interrupts); +} +VMBUS_CHAN_ATTR (interrupts,S_IRUGO , channel_interrupts_show,NULL ); + +static ssize_t channel_events_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->sig_events); +} +VMBUS_CHAN_ATTR (events,S_IRUGO , channel_events_show,NULL ); + + +static struct attribute *vmbus_chan_attrs[] = { &chan_attr_out_mask.attr, &chan_attr_in_mask.attr,
@@ -1246,6 +1262,8 @@ static struct attribute *vmbus_chan_attrs[] = { &chan_attr_cpu.attr, &chan_attr_pending.attr, &chan_attr_latency.attr, + &chan_attr_interrupts.attr, + &chan_attr_events.attr, NULL };
include/linux/hyperv.h +4 -0
@@ -731,6 +731,10 @@ struct vmbus_channel {struct vmbus_close_msg close_msg; +/* Statistics */ +unsigned int interrupts;/* Host to Guest interrupts */ +unsigned int sig_events;/* Guest to Host events */ +/* Channel callback's invoked in softirq context */ struct tasklet_struct callback_event;void (*onchannel_callback)(void *context);
Signature · 1 line
2.14.1