Drivers: hv: vmbus: Miscellaneous fixes
Subsystems (5)
cover Drivers: hv: vmbus: Miscellaneous fixes
Miscellaneous fixes/improvements. The patches that were sent over the last couple of months have been consolidated in this submission. All review comments have been addressed.
Christopher Oo (1):
Drivers: hv_vmbus: Fix signal to host condition
Dexuan Cui (4):
Drivers: hv: vmbus: Further improve CPU affiliation logic
Drivers: hv: vmbus: add a sysfs attr to show the binding of
channel/VP
tools: hv: add a python script lsvmbus to list VMBus devices
Drivers: hv: vmbus: document the VMBus sysfs files
Jake Oshins (2):
drivers:hv: Modify hv_vmbus to search for all MMIO ranges available. drivers:hv: Move MMIO range picking from hyper_fb to hv_vmbus
K. Y. Srinivasan (2):
Drivers: hv: vmbus: Improve the CPU affiliation for channels
Drivers: hv: vmbus: Implement a clocksource based on the TSC page
Viresh Kumar (1):
drivers/hv: Migrate to new 'set-state' interface
Vitaly Kuznetsov (3):
cpu-hotplug: convert cpu_hotplug_disabled to a counter
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disable
Documentation/ABI/stable/sysfs-bus-vmbus | 29 +++ Documentation/power/suspend-and-cpuhotplug.txt | 6 +- MAINTAINERS | 1 + arch/x86/include/uapi/asm/hyperv.h | 2 + drivers/hv/channel_mgmt.c | 29 ++- drivers/hv/hv.c | 137 ++++++++++--- drivers/hv/hyperv_vmbus.h | 14 ++ drivers/hv/ring_buffer.c | 14 +- drivers/hv/vmbus_drv.c | 280 ++++++++++++++++++------ drivers/video/fbdev/hyperv_fb.c | 46 ++-- include/linux/hyperv.h | 7 +- kernel/cpu.c | 24 ++- tools/hv/lsvmbus | 101 +++++++++ 13 files changed, 544 insertions(+), 146 deletions(-) create mode 100644 Documentation/ABI/stable/sysfs-bus-vmbus create mode 100644 tools/hv/lsvmbus
Signature · 1 line
1.7.4.1
PATCH 1/13 drivers:hv: Modify hv_vmbus to search for all MMIO ranges available.
From: Jake Oshins <[email protected]>
This patch changes the logic in hv_vmbus to record all of the ranges in the VM's firmware (BIOS or UEFI) that offer regions of memory-mapped I/O space for use by paravirtual front-end drivers. The old logic just found one range above 4GB and called it good. This logic will find any ranges above 1MB.
It would have been possible with this patch to just use existing resource allocation functions, rather than keep track of the entire set of Hyper-V related MMIO regions in VMBus. This strategy, however, is not sufficient when the resource allocator needs to be aware of the constraints of a Hyper-V virtual machine, which is what happens in the next patch in the series. So this first patch exists to show the first steps in reworking the MMIO allocation paths for Hyper-V front-end drivers.
--- drivers/hv/vmbus_drv.c | 116 ++++++++++++++++++++++++++++++--------- drivers/video/fbdev/hyperv_fb.c | 2 +- include/linux/hyperv.h | 2 +- 3 files changed, 92 insertions(+), 28 deletions(-)
drivers/hv/vmbus_drv.c +90 -26
@@ -102,10 +102,7 @@ static struct notifier_block hyperv_panic_block = { .notifier_call = hyperv_panic_event, }; -struct resource hyperv_mmio = { - .name ="hyperv mmio" , - .flags =IORESOURCE_MEM , -}; +struct resource *hyperv_mmio;EXPORT_SYMBOL_GPL (hyperv_mmio);static int vmbus_exists (void )
@@ -1013,30 +1010,105 @@ void vmbus_device_unregister(struct hv_device *device_obj)/* -* VMBUS is an acpi enumerated device. Get the the information we +* VMBUS is an acpi enumerated device. Get the information we * need from DSDT. */ - +#define VTPM_BASE_ADDRESS 0xfed40000static acpi_status vmbus_walk_resources (struct acpi_resource *res,void *ctx) { +resource_size_t start =0 ; +resource_size_t end =0 ; +struct resource *new_res; +struct resource **old_res = &hyperv_mmio; +struct resource **prev_res =NULL ; +switch (res->type) {case ACPI_RESOURCE_TYPE_IRQ : irq = res->data.irq.interrupts[0 ]; +return AE_OK ; + +/* +* "Address" descriptors are for bus windows. Ignore +* "memory" descriptors, which are for registers on +* devices. +*/ +case ACPI_RESOURCE_TYPE_ADDRESS32 : + start = res->data.address32.address.minimum; + end = res->data.address32.address.maximum;break ;case ACPI_RESOURCE_TYPE_ADDRESS64 : - hyperv_mmio.start = res->data.address64.address.minimum; - hyperv_mmio.end = res->data.address64.address.maximum; + start = res->data.address64.address.minimum; + end = res->data.address64.address.maximum;break ; + +default : +/* Unused resource type */ +return AE_OK ; + } +/* +* Ignore ranges that are below 1MB, as they're not +* necessary or useful here. +*/ +if (end <0x100000 ) +return AE_OK ; + + new_res =kzalloc (sizeof (*new_res),GFP_ATOMIC ); +if (!new_res) +return AE_NO_MEMORY ; + +/* If this range overlaps the virtual TPM, truncate it. */ +if (end >VTPM_BASE_ADDRESS && start <VTPM_BASE_ADDRESS ) + end =VTPM_BASE_ADDRESS ; + + new_res->name ="hyperv mmio" ; + new_res->flags =IORESOURCE_MEM ; + new_res->start = start; + new_res->end = end; + +do { +if (!*old_res) { + *old_res = new_res; +break ; + } + +if ((*old_res)->end < new_res->start) { + new_res->sibling = *old_res; +if (prev_res) + (*prev_res)->sibling = new_res; + *old_res = new_res; +break ; + } + + prev_res = old_res; + old_res = &(*old_res)->sibling; + + }while (1 );return AE_OK ; } +static int vmbus_acpi_remove (struct acpi_device *device) +{ +struct resource *cur_res; +struct resource *next_res; + +if (hyperv_mmio) { +for (cur_res = hyperv_mmio; cur_res; cur_res = next_res) { + next_res = cur_res->sibling; +kfree (cur_res); + } + } + +return 0 ; +} +static int vmbus_acpi_add (struct acpi_device *device) {acpi_status result;int ret_val = -ENODEV ; +struct acpi_device *ancestor; hv_acpi_dev = device;
@@ -1046,35 +1118,27 @@ static int vmbus_acpi_add(struct acpi_device *device)if (ACPI_FAILURE (result)) goto acpi_walk_err;/* -* The parent of the vmbus acpi device (Gen2 firmware) is the VMOD that -* has the mmio ranges. Get that. +* Some ancestor of the vmbus acpi device (Gen1 or Gen2 +* firmware) is the VMOD that has the mmio ranges. Get that. */ -if (device->parent) { - result =acpi_walk_resources (device->parent->handle, -METHOD_NAME__CRS , - vmbus_walk_resources,NULL ); +for (ancestor = device->parent; ancestor; ancestor = ancestor->parent) { + result =acpi_walk_resources (ancestor->handle,METHOD_NAME__CRS , + vmbus_walk_resources,NULL );if (ACPI_FAILURE (result)) - goto acpi_walk_err; -if (hyperv_mmio.start && hyperv_mmio.end) -request_resource (&iomem_resource, &hyperv_mmio); +continue ; +if (hyperv_mmio) +break ; } ret_val =0 ; acpi_walk_err:complete (&probe_event); +if (ret_val) +vmbus_acpi_remove (device);return ret_val; } -static int vmbus_acpi_remove (struct acpi_device *device) -{ -int ret =0 ; - -if (hyperv_mmio.start && hyperv_mmio.end) - ret =release_resource (&hyperv_mmio); -return ret; -} -static const struct acpi_device_id vmbus_acpi_device_ids[] = { {"VMBUS" ,0 }, {"VMBus" ,0 },
drivers/video/fbdev/hyperv_fb.c +1 -1
@@ -688,7 +688,7 @@ static int hvfb_getmem(struct fb_info *info) par->mem.name =KBUILD_MODNAME ; par->mem.flags =IORESOURCE_MEM |IORESOURCE_BUSY ;if (gen2vm) { - ret = allocate_resource(&hyperv_mmio, &par->mem, + ret = allocate_resource(hyperv_mmio, &par->mem, screen_fb_size, 0, -1, screen_fb_size,
include/linux/hyperv.h +1 -1
@@ -1233,7 +1233,7 @@ extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *,void hv_process_channel_removal (struct vmbus_channel *channel,u32 relid); -extern struct resource hyperv_mmio; +extern struct resource *hyperv_mmio;/* * Negotiated version with the Host.
Signature · 1 line
1.7.4.1
PATCH 2/13 drivers:hv: Move MMIO range picking from hyper_fb to hv_vmbus
From: Jake Oshins <[email protected]>
This patch deletes the logic from hyperv_fb which picked a range of MMIO space for the frame buffer and adds new logic to hv_vmbus which picks ranges for child drivers. The new logic isn't quite the same as the old, as it considers more possible ranges.
--- drivers/hv/vmbus_drv.c | 88 +++++++++++++++++++++++++++++++++++++-- drivers/video/fbdev/hyperv_fb.c | 46 +++++++++----------- include/linux/hyperv.h | 7 ++- 3 files changed, 110 insertions(+), 31 deletions(-)
drivers/hv/vmbus_drv.c +84 -4
@@ -39,6 +39,7 @@#include <asm/mshyperv.h> #include <linux/notifier.h> #include <linux/ptrace.h> +#include <linux/screen_info.h> #include <linux/kdebug.h> #include "hyperv_vmbus.h"
@@ -103,7 +104,6 @@ static struct notifier_block hyperv_panic_block = { };struct resource *hyperv_mmio; -EXPORT_SYMBOL_GPL (hyperv_mmio);static int vmbus_exists (void ) {
@@ -891,8 +891,8 @@ err_cleanup: }/** -* __vmbus_child_driver_register - Register a vmbus's driver -* @drv: Pointer to driver structure you want to register +* __vmbus_child_driver_register() - Register a vmbus's driver +* @hv_driver: Pointer to driver structure you want to register * @owner: owner module of the drv * @mod_name: module name string *
@@ -924,7 +924,8 @@ EXPORT_SYMBOL_GPL(__vmbus_driver_register);/** * vmbus_driver_unregister() - Unregister a vmbus's driver -* @drv: Pointer to driver structure you want to un-register +* @hv_driver: Pointer to driver structure you want to +* un-register * * Un-register the given driver that was previous registered with a call to * vmbus_driver_register()
@@ -1104,6 +1105,85 @@ static int vmbus_acpi_remove(struct acpi_device *device)return 0 ; } +/** +* vmbus_allocate_mmio() - Pick a memory-mapped I/O range. +* @new: If successful, supplied a pointer to the +* allocated MMIO space. +* @device_obj: Identifies the caller +* @min: Minimum guest physical address of the +* allocation +* @max: Maximum guest physical address +* @size: Size of the range to be allocated +* @align: Alignment of the range to be allocated +* @fb_overlap_ok: Whether this allocation can be allowed +* to overlap the video frame buffer. +* +* This function walks the resources granted to VMBus by the +* _CRS object in the ACPI namespace underneath the parent +* "bridge" whether that's a root PCI bus in the Generation 1 +* case or a Module Device in the Generation 2 case. It then +* attempts to allocate from the global MMIO pool in a way that +* matches the constraints supplied in these parameters and by +* that _CRS. +* +* Return: 0 on success, -errno on failure +*/ +int vmbus_allocate_mmio (struct resource **new,struct hv_device *device_obj, +resource_size_t min,resource_size_t max, +resource_size_t size,resource_size_t align, +bool fb_overlap_ok) +{ +struct resource *iter; +resource_size_t range_min, range_max, start, local_min, local_max; +const char *dev_n =dev_name (&device_obj->device); +u32 fb_end = screen_info.lfb_base + (screen_info.lfb_size <<1 ); +int i; + +for (iter = hyperv_mmio; iter; iter = iter->sibling) { +if ((iter->start >= max) || (iter->end <= min)) +continue ; + + range_min = iter->start; + range_max = iter->end; + +/* If this range overlaps the frame buffer, split it into +two tries. */ +for (i =0 ; i <2 ; i++) { + local_min = range_min; + local_max = range_max; +if (fb_overlap_ok || (range_min >= fb_end) || + (range_max <= screen_info.lfb_base)) { + i++; + }else { +if ((range_min <= screen_info.lfb_base) && + (range_max >= screen_info.lfb_base)) { +/* +* The frame buffer is in this window, +* so trim this into the part that +* preceeds the frame buffer. +*/ + local_max = screen_info.lfb_base -1 ; + range_min = fb_end; + }else { + range_min = fb_end; +continue ; + } + } + + start = (local_min + align -1 ) & ~(align -1 ); +for (; start + size -1 <= local_max; start += align) { + *new =request_mem_region_exclusive (start, size, + dev_n); +if (*new) +return 0 ; + } + } + } + +return -ENXIO ; +} +EXPORT_SYMBOL_GPL (vmbus_allocate_mmio); +static int vmbus_acpi_add (struct acpi_device *device) {acpi_status result;
drivers/video/fbdev/hyperv_fb.c +21 -25
98 lines not on this page · raw message
include/linux/hyperv.h +5 -2
@@ -977,6 +977,11 @@ int __must_check __vmbus_driver_register(struct hv_driver *hv_driver, const char *mod_name);void vmbus_driver_unregister (struct hv_driver *hv_driver); +int vmbus_allocate_mmio (struct resource **new,struct hv_device *device_obj, +resource_size_t min,resource_size_t max, +resource_size_t size,resource_size_t align, +bool fb_overlap_ok); +/** * VMBUS_DEVICE - macro used to describe a specific hyperv vmbus device *
@@ -1233,8 +1238,6 @@ extern bool vmbus_prep_negotiate_resp(struct icmsg_hdr *,void hv_process_channel_removal (struct vmbus_channel *channel,u32 relid); -extern struct resource *hyperv_mmio; -/* * Negotiated version with the Host. */
Signature · 1 line
1.7.4.1
PATCH 3/13 Drivers: hv: vmbus: Improve the CPU affiliation for channels
In reply to K. Y. SrinivasanThe current code tracks the assigned CPUs within a NUMA node in the context of the primary channel. So, if we have a VM with a single NUMA node with 8 VCPUs, we may end up unevenly distributing the channel load. Fix the issue by tracking affiliations globally.
--- drivers/hv/channel_mgmt.c | 11 ++++++----- drivers/hv/hv.c | 9 +++++++++ drivers/hv/hyperv_vmbus.h | 5 +++++ include/linux/hyperv.h | 1 - 4 files changed, 20 insertions(+), 6 deletions(-)
drivers/hv/channel_mgmt.c +6 -5
@@ -391,6 +391,7 @@ static void init_vp_index(struct vmbus_channel *channel, const uuid_le *type_guistruct vmbus_channel *primary = channel->primary_channel;int next_node;struct cpumask available_mask; +struct cpumask *alloced_mask; for (i = IDE; i < MAX_PERF_CHN; i++) { if (!memcmp(type_guid->b, hp_devs[i].guid,
@@ -408,7 +409,6 @@ static void init_vp_index(struct vmbus_channel *channel, const uuid_le *type_gui* channel, bind it to cpu 0. */ channel->numa_node =0 ; -cpumask_set_cpu (0 , &channel->alloced_cpus_in_node); channel->target_cpu =0 ; channel->target_vp = hv_context.vp_index[0 ];return ;
@@ -433,21 +433,22 @@ static void init_vp_index(struct vmbus_channel *channel, const uuid_le *type_gui channel->numa_node = next_node; primary = channel; } + alloced_mask = &hv_context.hv_numa_map[primary->numa_node]; -if (cpumask_weight (&primary->alloced_cpus_in_node) == +if (cpumask_weight (alloced_mask) ==cpumask_weight (cpumask_of_node (primary->numa_node))) {/* * We have cycled through all the CPUs in the node; * reset the alloced map. */ -cpumask_clear (&primary->alloced_cpus_in_node); +cpumask_clear (alloced_mask); } -cpumask_xor (&available_mask, &primary->alloced_cpus_in_node, +cpumask_xor (&available_mask, alloced_mask,cpumask_of_node (primary->numa_node)); cur_cpu =cpumask_next (-1 , &available_mask); -cpumask_set_cpu (cur_cpu, &primary->alloced_cpus_in_node); +cpumask_set_cpu (cur_cpu, alloced_mask); channel->target_cpu = cur_cpu; channel->target_vp = hv_context.vp_index[cur_cpu];
drivers/hv/hv.c +9 -0
@@ -332,6 +332,13 @@ int hv_synic_alloc(void)size_t ced_size =sizeof (struct clock_event_device );int cpu; + hv_context.hv_numa_map =kzalloc (sizeof (struct cpumask ) * nr_node_ids, +GFP_ATOMIC ); +if (hv_context.hv_numa_map ==NULL ) { +pr_err ("Unable to allocate NUMA map\n" ); + goto err; + } +for_each_online_cpu (cpu) { hv_context.event_dpc[cpu] =kmalloc (size,GFP_ATOMIC );if (hv_context.event_dpc[cpu] ==NULL ) {
@@ -345,6 +352,7 @@ int hv_synic_alloc(void)pr_err ("Unable to allocate clock event device\n" ); goto err; } +hv_init_clockevent_device (hv_context.clk_evt[cpu], cpu); hv_context.synic_message_page[cpu] =
@@ -393,6 +401,7 @@ void hv_synic_free(void) {int cpu; +kfree (hv_context.hv_numa_map);for_each_online_cpu (cpu)hv_synic_free_cpu (cpu); }
drivers/hv/hyperv_vmbus.h +5 -0
@@ -551,6 +551,11 @@ struct hv_context {* Support PV clockevent device. */ struct clock_event_device *clk_evt[NR_CPUS ]; +/* +* To manage allocations in a NUMA node. +* Array indexed by numa node ID. +*/ +struct cpumask *hv_numa_map; };extern struct hv_context hv_context;
include/linux/hyperv.h +0 -1
@@ -699,7 +699,6 @@ struct vmbus_channel {/* * State to manage the CPU affiliation of channels. */ -struct cpumask alloced_cpus_in_node;int numa_node;/* * Support for sub-channels. For high performance devices,
Signature · 1 line
1.7.4.1
PATCH 4/13 Drivers: hv: vmbus: Further improve CPU affiliation logic
In reply to K. Y. SrinivasanFrom: Dexuan Cui <[email protected]>
Keep track of CPU affiliations of sub-channels within the scope of the primary channel. This will allow us to better distribute the load amongst available CPUs.
--- drivers/hv/channel_mgmt.c | 20 ++++++++++++++++++-- include/linux/hyperv.h | 1 + 2 files changed, 19 insertions(+), 2 deletions(-)
drivers/hv/channel_mgmt.c +18 -2
@@ -447,8 +447,24 @@ static void init_vp_index(struct vmbus_channel *channel, const uuid_le *type_guicpumask_xor (&available_mask, alloced_mask,cpumask_of_node (primary->numa_node)); - cur_cpu =cpumask_next (-1 , &available_mask); -cpumask_set_cpu (cur_cpu, alloced_mask); + cur_cpu =-1 ; +while (true) { + cur_cpu =cpumask_next (cur_cpu, &available_mask); +if (cur_cpu >= nr_cpu_ids) { + cur_cpu =-1 ; +cpumask_copy (&available_mask, +cpumask_of_node (primary->numa_node)); +continue ; + } + +if (!cpumask_test_cpu (cur_cpu, + &primary->alloced_cpus_in_node)) { +cpumask_set_cpu (cur_cpu, + &primary->alloced_cpus_in_node); +cpumask_set_cpu (cur_cpu, alloced_mask); +break ; + } + } channel->target_cpu = cur_cpu; channel->target_vp = hv_context.vp_index[cur_cpu];
include/linux/hyperv.h +1 -0
@@ -699,6 +699,7 @@ struct vmbus_channel {/* * State to manage the CPU affiliation of channels. */ +struct cpumask alloced_cpus_in_node;int numa_node;/* * Support for sub-channels. For high performance devices,
Signature · 1 line
1.7.4.1
PATCH 5/13 Drivers: hv_vmbus: Fix signal to host condition
In reply to K. Y. SrinivasanFrom: Christopher Oo <[email protected]>
Fixes a bug where previously hv_ringbuffer_read would pass in the old number of bytes available to read instead of the expected old read index when calculating when to signal to the host that the ringbuffer is empty. Since the previous write size is already saved, also changes the hv_need_to_signal_on_read to use the previously read value rather than recalculating it.
--- drivers/hv/ring_buffer.c | 14 +++----------- 1 files changed, 3 insertions(+), 11 deletions(-)
drivers/hv/ring_buffer.c +3 -11
@@ -103,10 +103,9 @@ static bool hv_need_to_signal(u32 old_write, struct hv_ring_buffer_info *rbi)* there is room for the producer to send the pending packet. */ -static bool hv_need_to_signal_on_read (u32 old_rd, -struct hv_ring_buffer_info *rbi) +static bool hv_need_to_signal_on_read (u32 prev_write_sz, +struct hv_ring_buffer_info *rbi) { -u32 prev_write_sz;u32 cur_write_sz;u32 r_size;u32 write_loc = rbi->ring_buffer->write_index;
@@ -123,10 +122,6 @@ static bool hv_need_to_signal_on_read(u32 old_rd, cur_write_sz = write_loc >= read_loc ? r_size - (write_loc - read_loc) : read_loc - write_loc; - prev_write_sz = write_loc >= old_rd ? r_size - (write_loc - old_rd) : - old_rd - write_loc; - -if ((prev_write_sz < pending_sz) && (cur_write_sz >= pending_sz))return true;
@@ -517,7 +512,6 @@ int hv_ringbuffer_read(struct hv_ring_buffer_info *inring_info, void *buffer,u32 next_read_location =0 ;u64 prev_indices =0 ;unsigned long flags; -u32 old_read;if (buflen <=0 )return -EINVAL ;
@@ -528,8 +522,6 @@ int hv_ringbuffer_read(struct hv_ring_buffer_info *inring_info, void *buffer, &bytes_avail_toread, &bytes_avail_towrite); - old_read = bytes_avail_toread; -/* Make sure there is something to read */ if (bytes_avail_toread < buflen) {spin_unlock_irqrestore (&inring_info->ring_lock, flags);
@@ -560,7 +552,7 @@ int hv_ringbuffer_read(struct hv_ring_buffer_info *inring_info, void *buffer,spin_unlock_irqrestore (&inring_info->ring_lock, flags); - *signal =hv_need_to_signal_on_read (old_read, inring_info); + *signal =hv_need_to_signal_on_read (bytes_avail_towrite, inring_info);return 0 ; }
Signature · 1 line
1.7.4.1
PATCH 6/13 drivers/hv: Migrate to new 'set-state' interface
In reply to K. Y. SrinivasanFrom: Viresh Kumar <[email protected]>
Migrate hv driver to the new 'set-state' interface provided by clockevents core, the earlier 'set-mode' interface is marked obsolete now.
This also enables us to implement callbacks for new states of clockevent devices, for example: ONESHOT_STOPPED.
--- drivers/hv/hv.c | 45 +++++++++++++++++++-------------------------- 1 files changed, 19 insertions(+), 26 deletions(-)
drivers/hv/hv.c +19 -26
@@ -274,7 +274,7 @@ static int hv_ce_set_next_event(unsigned long delta, {cycle_t current_tick; -WARN_ON (evt->mode !=CLOCK_EVT_MODE_ONESHOT ); +WARN_ON (!clockevent_state_oneshot (evt));rdmsrl (HV_X64_MSR_TIME_REF_COUNT , current_tick); current_tick += delta;
@@ -282,31 +282,24 @@ static int hv_ce_set_next_event(unsigned long delta,return 0 ; } -static void hv_ce_setmode (enum clock_event_mode mode, -struct clock_event_device *evt) +static int hv_ce_shutdown (struct clock_event_device *evt) +{ +wrmsrl (HV_X64_MSR_STIMER0_COUNT ,0 ); +wrmsrl (HV_X64_MSR_STIMER0_CONFIG ,0 ); + +return 0 ; +} + +static int hv_ce_set_oneshot (struct clock_event_device *evt) {union hv_timer_config timer_cfg; -switch (mode) { -case CLOCK_EVT_MODE_PERIODIC : -/* unsupported */ -break ; - -case CLOCK_EVT_MODE_ONESHOT : - timer_cfg.enable =1 ; - timer_cfg.auto_enable =1 ; - timer_cfg.sintx =VMBUS_MESSAGE_SINT ; -wrmsrl (HV_X64_MSR_STIMER0_CONFIG , timer_cfg.as_uint64); -break ; - -case CLOCK_EVT_MODE_UNUSED : -case CLOCK_EVT_MODE_SHUTDOWN : -wrmsrl (HV_X64_MSR_STIMER0_COUNT ,0 ); -wrmsrl (HV_X64_MSR_STIMER0_CONFIG ,0 ); -break ; -case CLOCK_EVT_MODE_RESUME : -break ; - } + timer_cfg.enable =1 ; + timer_cfg.auto_enable =1 ; + timer_cfg.sintx =VMBUS_MESSAGE_SINT ; +wrmsrl (HV_X64_MSR_STIMER0_CONFIG , timer_cfg.as_uint64); + +return 0 ; }static void hv_init_clockevent_device (struct clock_event_device *dev,int cpu)
@@ -321,7 +314,8 @@ static void hv_init_clockevent_device(struct clock_event_device *dev, int cpu)* references to the hv_vmbus module making it impossible to unload. */ - dev->set_mode = hv_ce_setmode; + dev->set_state_shutdown = hv_ce_shutdown; + dev->set_state_oneshot = hv_ce_set_oneshot; dev->set_next_event = hv_ce_set_next_event; }
@@ -515,8 +509,7 @@ void hv_synic_cleanup(void *arg)/* Turn off clockevent device */ if (ms_hyperv.features &HV_X64_MSR_SYNTIMER_AVAILABLE ) -hv_ce_setmode (CLOCK_EVT_MODE_SHUTDOWN , - hv_context.clk_evt[cpu]); +hv_ce_shutdown (hv_context.clk_evt[cpu]);rdmsrl (HV_X64_MSR_SINT0 +VMBUS_MESSAGE_SINT , shared_sint.as_uint64);
Signature · 1 line
1.7.4.1
PATCH 7/13 Drivers: hv: vmbus: Implement a clocksource based on the TSC page
In reply to K. Y. SrinivasanThe current Hyper-V clock source is based on the per-partition reference counter and this counter is being accessed via s synthetic MSR - HV_X64_MSR_TIME_REF_COUNT. Hyper-V has a more efficient way of computing the per-partition reference counter value that does not involve reading a synthetic MSR. We implement a time source based on this mechanism.
--- arch/x86/include/uapi/asm/hyperv.h | 2 + drivers/hv/hv.c | 83 ++++++++++++++++++++++++++++++++++++ drivers/hv/hyperv_vmbus.h | 9 ++++ 3 files changed, 94 insertions(+), 0 deletions(-)
arch/x86/include/uapi/asm/hyperv.h +2 -0
@@ -27,6 +27,8 @@#define HV_X64_MSR_VP_RUNTIME_AVAILABLE (1 << 0)/* Partition Reference Counter (HV_X64_MSR_TIME_REF_COUNT) available*/ #define HV_X64_MSR_TIME_REF_COUNT_AVAILABLE (1 << 1) +/* Partition reference TSC MSR is available */ +#define HV_X64_MSR_REFERENCE_TSC_AVAILABLE (1 << 9)/* A partition's reference time stamp counter (TSC) page */ #define HV_X64_MSR_REFERENCE_TSC 0x40000021
drivers/hv/hv.c +83 -0
108 lines not on this page · raw message
drivers/hv/hyperv_vmbus.h +9 -0
@@ -517,6 +517,7 @@ struct hv_context {u64 guestid;void *hypercall_page; +void *tsc_page;bool synic_initialized;
@@ -560,6 +561,14 @@ struct hv_context {extern struct hv_context hv_context; +struct ms_hyperv_tsc_page { +volatile u32 tsc_sequence; +u32 reserved1; +volatile u64 tsc_scale; +volatile s64 tsc_offset; +u64 reserved2[509 ]; +}; +struct hv_ring_buffer_debug_info {u32 current_interrupt_mask;u32 current_read_index;
Signature · 1 line
1.7.4.1
PATCH 8/13 Drivers: hv: vmbus: add a sysfs attr to show the binding of channel/VP
In reply to K. Y. SrinivasanFrom: Dexuan Cui <[email protected]>
This is useful to analyze performance issue.
--- drivers/hv/vmbus_drv.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 38 insertions(+), 0 deletions(-)
drivers/hv/vmbus_drv.c +38 -0
@@ -441,6 +441,43 @@ static ssize_t in_write_bytes_avail_show(struct device *dev, }static DEVICE_ATTR_RO (in_write_bytes_avail); +static ssize_t channel_vp_mapping_show (struct device *dev, +struct device_attribute *dev_attr, +char *buf) +{ +struct hv_device *hv_dev =device_to_hv_device (dev); +struct vmbus_channel *channel = hv_dev->channel, *cur_sc; +unsigned long flags; +int buf_size =PAGE_SIZE , n_written, tot_written; +struct list_head *cur; + +if (!channel) +return -ENODEV ; + + tot_written =snprintf (buf, buf_size,"%u:%u\n" , + channel->offermsg.child_relid, channel->target_cpu); + +spin_lock_irqsave (&channel->lock, flags); + +list_for_each (cur, &channel->sc_list) { +if (tot_written >= buf_size -1 ) +break ; + + cur_sc = list_entry(cur, struct vmbus_channel, sc_list); + n_written =scnprintf (buf + tot_written, + buf_size - tot_written, +"%u:%u\n" , + cur_sc->offermsg.child_relid, + cur_sc->target_cpu); + tot_written += n_written; + } + +spin_unlock_irqrestore (&channel->lock, flags); + +return tot_written; +} +static DEVICE_ATTR_RO (channel_vp_mapping); +/* Set up per device attributes in /sys/bus/vmbus/devices/<bus device> */ static struct attribute *vmbus_attrs[] = { &dev_attr_id.attr,
@@ -465,6 +502,7 @@ static struct attribute *vmbus_attrs[] = { &dev_attr_in_write_index.attr, &dev_attr_in_read_bytes_avail.attr, &dev_attr_in_write_bytes_avail.attr, + &dev_attr_channel_vp_mapping.attr,NULL , };ATTRIBUTE_GROUPS (vmbus);
Signature · 1 line
1.7.4.1
PATCH 9/13 tools: hv: add a python script lsvmbus to list VMBus devices
In reply to K. Y. SrinivasanFrom: Dexuan Cui <[email protected]>
By default lsvmbus lists all the devices in the VMBus. With -v or -vv, more information is printed, including the VMBus Rel_ID, class ID, device ID and which channel is bound to which virtual processor, etc.
--- tools/hv/lsvmbus | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 101 insertions(+), 0 deletions(-) create mode 100644 tools/hv/lsvmbus
tools/hv/lsvmbus new file +101 -0
@@ -0,0 +1,101 @@ +#!/usr/bin/env python + +import os +from optparse import OptionParser + +parser = OptionParser() +parser.add_option("-v", "--verbose", dest="verbose", + help="print verbose messages. Try -vv, -vvv for \ + more verbose messages", action="count") + +(options, args) = parser.parse_args() + +verbose = 0 +if options.verbose is not None: + verbose = options.verbose + +vmbus_sys_path = '/sys/bus/vmbus/devices' +if not os.path.isdir(vmbus_sys_path): + print "%s doesn't exist: exiting..." % vmbus_sys_path + exit(-1) + +vmbus_dev_dict = { + '{0e0b6031-5213-4934-818b-38d90ced39db}' : '[Operating system shutdown]', + '{9527e630-d0ae-497b-adce-e80ab0175caf}' : '[Time Synchronization]', + '{57164f39-9115-4e78-ab55-382f3bd5422d}' : '[Heartbeat]', + '{a9a0f4e7-5a45-4d96-b827-8a841e8c03e6}' : '[Data Exchange]', + '{35fa2e29-ea23-4236-96ae-3a6ebacba440}' : '[Backup (volume checkpoint)]', + '{34d14be3-dee4-41c8-9ae7-6b174977c192}' : '[Guest services]', + '{525074dc-8985-46e2-8057-a307dc18a502}' : '[Dynamic Memory]', + '{cfa8b69e-5b4a-4cc0-b98b-8ba1a1f3f95a}' : 'Synthetic mouse', + '{f912ad6d-2b17-48ea-bd65-f927a61c7684}' : 'Synthetic keyboard', + '{da0a7802-e377-4aac-8e77-0558eb1073f8}' : 'Synthetic framebuffer adapter', + '{f8615163-df3e-46c5-913f-f2d2f965ed0e}' : 'Synthetic network adapter', + '{32412632-86cb-44a2-9b5c-50d1417354f5}' : 'Synthetic IDE Controller', + '{ba6163d9-04a1-4d29-b605-72e2ffb1dc7f}' : 'Synthetic SCSI Controller', + '{2f9bcc4a-0069-4af3-b76b-6fd0be528cda}' : 'Synthetic fiber channel adapter', + '{8c2eaf3d-32a7-4b09-ab99-bd1f1c86b501}' : 'Synthetic RDMA adapter', + '{276aacf4-ac15-426c-98dd-7521ad3f01fe}' : '[Reserved system device]', + '{f8e65716-3cb3-4a06-9a60-1889c5cccab5}' : '[Reserved system device]', + '{3375baf4-9e15-4b30-b765-67acb10d607b}' : '[Reserved system device]', +} + +def get_vmbus_dev_attr(dev_name, attr): + try: + f = open('%s/%s/%s' % (vmbus_sys_path, dev_name, attr), 'r') + lines = f.readlines() + f.close() + except IOError: + lines = [] + + return lines + +class VMBus_Dev: + pass + + +vmbus_dev_list = [] + +for f in os.listdir(vmbus_sys_path): + vmbus_id = get_vmbus_dev_attr(f, 'id')[0].strip() + class_id = get_vmbus_dev_attr(f, 'class_id')[0].strip() + device_id = get_vmbus_dev_attr(f, 'device_id')[0].strip() + dev_desc = vmbus_dev_dict.get(class_id, 'Unknown') + + chn_vp_mapping = get_vmbus_dev_attr(f, 'channel_vp_mapping') + chn_vp_mapping = [c.strip() for c in chn_vp_mapping] + chn_vp_mapping = sorted(chn_vp_mapping, + key = lambda c : int(c.split(':')[0])) + + chn_vp_mapping = ['\tRel_ID=%s, target_cpu=%s' % + (c.split(':')[0], c.split(':')[1]) + for c in chn_vp_mapping] + d = VMBus_Dev() + d.sysfs_path = '%s/%s' % (vmbus_sys_path, f) + d.vmbus_id = vmbus_id + d.class_id = class_id + d.device_id = device_id + d.dev_desc = dev_desc + d.chn_vp_mapping = '\n'.join(chn_vp_mapping) + if d.chn_vp_mapping: + d.chn_vp_mapping += '\n' + + vmbus_dev_list.append(d) + + +vmbus_dev_list = sorted(vmbus_dev_list, key = lambda d : int(d.vmbus_id)) + +format0 = '%2s: %s' +format1 = '%2s: Class_ID = %s - %s\n%s' +format2 = '%2s: Class_ID = %s - %s\n\tDevice_ID = %s\n\tSysfs path: %s\n%s' + +for d in vmbus_dev_list: + if verbose == 0: + print ('VMBUS ID ' + format0) % (d.vmbus_id, d.dev_desc) + elif verbose == 1: + print ('VMBUS ID ' + format1) % \ + (d.vmbus_id, d.class_id, d.dev_desc, d.chn_vp_mapping) + else: + print ('VMBUS ID ' + format2) % \ + (d.vmbus_id, d.class_id, d.dev_desc, \ + d.device_id, d.sysfs_path, d.chn_vp_mapping)
Signature · 1 line
1.7.4.1
PATCH 10/13 Drivers: hv: vmbus: document the VMBus sysfs files
In reply to K. Y. SrinivasanFrom: Dexuan Cui <[email protected]>
The 4 sysfs files should be stable ABIs to the user space.
--- Documentation/ABI/stable/sysfs-bus-vmbus | 29 +++++++++++++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 30 insertions(+), 0 deletions(-) create mode 100644 Documentation/ABI/stable/sysfs-bus-vmbus
Documentation/ABI/stable/sysfs-bus-vmbus new file +29 -0
@@ -0,0 +1,29 @@ +What: /sys/bus/vmbus/devices/vmbus_*/id +Date: Jul 2009 +KernelVersion: 2.6.31 +Contact: K. Y. Srinivasan <[email protected]> +Description: The VMBus child_relid of the device's primary channel +Users: tools/hv/lsvmbus + +What: /sys/bus/vmbus/devices/vmbus_*/class_id +Date: Jul 2009 +KernelVersion: 2.6.31 +Contact: K. Y. Srinivasan <[email protected]> +Description: The VMBus interface type GUID of the device +Users: tools/hv/lsvmbus + +What: /sys/bus/vmbus/devices/vmbus_*/device_id +Date: Jul 2009 +KernelVersion: 2.6.31 +Contact: K. Y. Srinivasan <[email protected]> +Description: The VMBus interface instance GUID of the device +Users: tools/hv/lsvmbus + +What: /sys/bus/vmbus/devices/vmbus_*/channel_vp_mapping +Date: Jul 2015 +KernelVersion: 4.2.0 +Contact: K. Y. Srinivasan <[email protected]> +Description: The mapping of which primary/sub channels are bound to which + Virtual Processors. + Format: <channel's child_relid:the bound cpu's number> +Users: tools/hv/lsvmbus
MAINTAINERS +1 -0
@@ -4956,6 +4956,7 @@ F: drivers/scsi/storvsc_drv.c F: drivers/video/fbdev/hyperv_fb.c F: include/linux/hyperv.h F: tools/hv/ +F: Documentation/ABI/stable/sysfs-bus-vmbus I2C OVER PARALLEL PORT M: Jean Delvare <[email protected]>
Signature · 1 line
1.7.4.1
PATCH 11/13 cpu-hotplug: convert cpu_hotplug_disabled to a counter
In reply to K. Y. SrinivasanFrom: Vitaly Kuznetsov <[email protected]>
As a prerequisite to exporting cpu_hotplug_enable/cpu_hotplug_disable functions to modules we need to convert cpu_hotplug_disabled to a counter to properly support disable -> disable -> enable call sequences. E.g. after Hyper-V vmbus module (which is supposed to be the first user of exported cpu_hotplug_enable/cpu_hotplug_disable) did cpu_hotplug_disable() hibernate path calls disable_nonboot_cpus() and if we hit an error in _cpu_down() enable_nonboot_cpus() will be called on the failure path (thus making cpu_hotplug_disabled = 0 and leaving cpu hotplug in 'enabled' state). Same problem is possible if more than 1 module use cpu_hotplug_disable/cpu_hotplug_enable on their load/unload paths. When one of these modules is been unloaded it is logical to leave cpu hotplug in 'disabled' state.
To support the change we need to increse cpu_hotplug_disabled counter in disable_nonboot_cpus() unconditionally as all users of disable_nonboot_cpus() are supposed to do enable_nonboot_cpus() in case an error was returned.
--- Documentation/power/suspend-and-cpuhotplug.txt | 6 +++--- kernel/cpu.c | 21 +++++++++++++-------- 2 files changed, 16 insertions(+), 11 deletions(-)
Documentation/power/suspend-and-cpuhotplug.txt +3 -3
@@ -72,7 +72,7 @@ More details follow: | v Disable regular cpu hotplug - by setting cpu_hotplug_disabled=1 + by increasing cpu_hotplug_disabled | v Release cpu_add_remove_lock
@@ -89,7 +89,7 @@ Resuming back is likewise, with the counterparts being (in the order of execution during resume): * enable_nonboot_cpus() which involves: | Acquire cpu_add_remove_lock - | Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug + | Decrease cpu_hotplug_disabled, thereby enabling regular cpu hotplug | Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop] | Release cpu_add_remove_lock v
@@ -120,7 +120,7 @@ after the entire cycle is complete (i.e., suspend + resume). Acquire cpu_add_remove_lock | v - If cpu_hotplug_disabled is 1 + If cpu_hotplug_disabled > 0 return gracefully | |
kernel/cpu.c +13 -8
@@ -191,14 +191,14 @@ void cpu_hotplug_done(void)void cpu_hotplug_disable (void ) {cpu_maps_update_begin (); - cpu_hotplug_disabled =1 ; + cpu_hotplug_disabled++;cpu_maps_update_done (); }void cpu_hotplug_enable (void ) {cpu_maps_update_begin (); - cpu_hotplug_disabled =0 ; +WARN_ON (--cpu_hotplug_disabled <0 );cpu_maps_update_done (); }
@@ -617,13 +617,18 @@ int disable_nonboot_cpus(void) } } -if (!error) { +if (!error)BUG_ON (num_online_cpus () >1 ); -/* Make sure the CPUs won't be enabled by someone else */ - cpu_hotplug_disabled =1 ; - }else { +else pr_err ("Non-boot CPUs are not disabled\n" ); - } + +/* +* Make sure the CPUs won't be enabled by someone else. We need to do +* this even in case of failure as all disable_nonboot_cpus() users are +* supposed to do enable_nonboot_cpus() on the failure path. +*/ + cpu_hotplug_disabled++; +cpu_maps_update_done ();return error; }
@@ -642,7 +647,7 @@ void __ref enable_nonboot_cpus(void)/* Allow everyone to use the CPU hotplug again */ cpu_maps_update_begin (); - cpu_hotplug_disabled =0 ; +WARN_ON (--cpu_hotplug_disabled <0 );if (cpumask_empty (frozen_cpus)) goto out;
Signature · 1 line
1.7.4.1
PATCH 12/13 cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
In reply to K. Y. SrinivasanFrom: Vitaly Kuznetsov <[email protected]>
Hyper-V module needs to disable cpu hotplug (offlining) as there is no support from hypervisor side to reassign already opened event channels to a different CPU. Currently it is been done by altering smp_ops.cpu_disable but it is hackish.
--- kernel/cpu.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/cpu.c +2 -1
@@ -194,6 +194,7 @@ void cpu_hotplug_disable(void) cpu_hotplug_disabled++;cpu_maps_update_done (); } +EXPORT_SYMBOL_GPL (cpu_hotplug_disable);void cpu_hotplug_enable (void ) {
@@ -201,7 +202,7 @@ void cpu_hotplug_enable(void)WARN_ON (--cpu_hotplug_disabled <0 ); cpu_maps_update_done(); } - +EXPORT_SYMBOL_GPL (cpu_hotplug_enable);#endif /* CONFIG_HOTPLUG_CPU */ /* Need to know about CPUs going up/down? */
Signature · 1 line
1.7.4.1
PATCH 13/13 Drivers: hv: vmbus: use cpu_hotplug_enable/disable
In reply to K. Y. SrinivasanFrom: Vitaly Kuznetsov <[email protected]>
Commit e513229b4c38 ("Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors") was altering smp_ops.cpu_disable to prevent CPU offlining. We can bo better by using cpu_hotplug_enable/disable functions instead of such hard-coding.
--- drivers/hv/vmbus_drv.c | 38 ++++---------------------------------- 1 files changed, 4 insertions(+), 34 deletions(-)
drivers/hv/vmbus_drv.c +4 -34
@@ -828,38 +828,6 @@ static void vmbus_isr(void) } } -#ifdef CONFIG_HOTPLUG_CPU -static int hyperv_cpu_disable (void ) -{ -return -ENOSYS ; -} - -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -static void *previous_cpu_disable; - -/* -* Offlining a CPU when running on newer hypervisors (WS2012R2, Win8, -* ...) is not supported at this moment as channel interrupts are -* distributed across all of them. -*/ - -if ((vmbus_proto_version ==VERSION_WS2008 ) || - (vmbus_proto_version ==VERSION_WIN7 )) -return ; - -if (vmbus_loaded) { - previous_cpu_disable = smp_ops.cpu_disable; - smp_ops.cpu_disable = hyperv_cpu_disable; -pr_notice ("CPU offlining is not supported by hypervisor\n" ); - }else if (previous_cpu_disable) - smp_ops.cpu_disable = previous_cpu_disable; -} -#else -static void hv_cpu_hotplug_quirk (bool vmbus_loaded) -{ -} -#endif /* * vmbus_bus_init -Main vmbus driver initialization routine.
@@ -901,7 +869,8 @@ static int vmbus_bus_init(int irq)if (ret) goto err_alloc; -hv_cpu_hotplug_quirk (true); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_disable ();/* * Only register if the crash MSRs are available
@@ -1364,7 +1333,8 @@ static void __exit vmbus_exit(void) }hv_synic_free ();acpi_bus_unregister_driver (&vmbus_acpi_driver); -hv_cpu_hotplug_quirk (false); +if (vmbus_proto_version >VERSION_WIN7 ) +cpu_hotplug_enable (); }
Signature · 1 line
1.7.4.1
Quoted 11 lines
-----Original Message-----
From: K. Y. Srinivasan [mailto:[email protected]]
Sent: Wednesday, August 5, 2015 12:52 AM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: KY Srinivasan <[email protected]>
Subject: [PATCH 00/13] Drivers: hv: vmbus: Miscellaneous fixes
Quoted 3 lines
Miscellaneous fixes/improvements. The patches that were sent over the last couple of months have been consolidated in this submission. All review comments have been addressed.
This is the remaining 13 matches being sent. I have fixed up Jake's name.
Regards,
K. Y
Quoted 44 lines
Christopher Oo (1):
Drivers:hv_vmbus: Fix signal to host conditionDexuan Cui (4):
Drivers: hv: vmbus: Further improve CPU affiliation logic
Drivers: hv: vmbus: add a sysfs attr to show the binding of
channel/VP
tools: hv: add a python script lsvmbus to list VMBus devices
Drivers: hv: vmbus: document the VMBus sysfs filesJake Oshins (2):
drivers:hv: Modifyhv_vmbusto search for all MMIO ranges available. drivers:hv: Move MMIO range picking fromhyper_fbtohv_vmbusK. Y. Srinivasan (2):
Drivers: hv: vmbus: Improve the CPU affiliation for channels
Drivers: hv: vmbus: Implement a clocksource based on the TSC pageViresh Kumar (1):
drivers/hv: Migrate to new 'set-state' interfaceVitaly Kuznetsov (3):
cpu-hotplug: convertcpu_hotplug_disabledto a counter
cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable
Drivers: hv: vmbus: use cpu_hotplug_enable/disableDocumentation/ABI/stable/sysfs-bus-vmbus | 29 +++ Documentation/power/suspend-and-cpuhotplug.txt | 6 +- MAINTAINERS | 1 + arch/x86/include/uapi/asm/hyperv.h | 2 + drivers/hv/channel_mgmt.c | 29 ++- drivers/hv/hv.c | 137 ++++++++++--- drivers/hv/hyperv_vmbus.h | 14 ++ drivers/hv/ring_buffer.c | 14 +- drivers/hv/vmbus_drv.c | 280 ++++++++++++++++++------ drivers/video/fbdev/hyperv_fb.c | 46 ++-- include/linux/hyperv.h | 7 +- kernel/cpu.c | 24 ++- tools/hv/lsvmbus | 101 +++++++++ 13 files changed, 544 insertions(+), 146 deletions(-) create mode 100644 Documentation/ABI/stable/sysfs-bus-vmbus create mode 100644 tools/hv/lsvmbus
Quoted 2 lines
--
1.7.4.1
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�