UIO driver for low speed Hyper-V devices
Subsystems (4)
v2 cover UIO driver for low speed Hyper-V devices
Hyper-V is adding some low speed "specialty" synthetic devices. This patch series propose the solution to support these devices. Instead of writing new kernel-level VMBus drivers for all of these devices, we propose a solution wherein these devices are made accessible to user space through a dedicated UIO-based hv_vmbus_client driver, allowing for efficient device handling via user space drivers. This solution aims to optimize the development process by eliminating the need to create individual kernel-level VMBus drivers for each device and provide flexibility to user space applications to control the ring buffer independently.
Since all these new synthetic devices are low speed devices, they don't support monitor bits and we must use vmbus_setevent() to enable interrupts from the host. The new uio driver supports all these requirements effectively. Additionally, this new driver also provide the support for having smaller/cutom ringbuffer size.
Furthermore, this patch series includes a revision of the fcopy application to leverage the new interface seamlessly along with removal of old driver and application. However, please note that the development of other similar drivers is still a work in progress, and will be shared as they become available.
[V2]
1. Update driver info in Documentation/driver-api/uio-howto.rst
2. Update ring_size sysfs info in Documentation/ABI/stable/sysfs-bus-vmbus
3. Remove DRIVER_VERSION
4. Remove refcnt
5. scnprintf -> sysfs_emit
6. sysfs_create_file -> ATTRIBUTE_GROUPS + ".driver.groups";
7. sysfs_create_bin_file -> device_create_bin_file
8. dev_notice -> dev_err
9. Removed invalid free of devm_ allocated data
10. Updated application with simpler sysfs path
Saurabh Sengar (5):
uio: Add hv_vmbus_client driver
tools: hv: Add vmbus_bufring
tools: hv: Add new fcopy application based on uio driver
tools: hv: Remove hv_fcopy_daemon
Drivers: hv: Remove fcopy driver
Documentation/ABI/stable/sysfs-bus-vmbus | 7 + Documentation/driver-api/uio-howto.rst | 46 +++ drivers/hv/Makefile | 2 +- drivers/hv/hv_fcopy.c | 427 -------------------- drivers/hv/hv_util.c | 12 - drivers/uio/Kconfig | 12 + drivers/uio/Makefile | 1 + drivers/uio/uio_hv_vmbus_client.c | 217 ++++++++++ tools/hv/Build | 3 +- tools/hv/Makefile | 10 +- tools/hv/hv_fcopy_daemon.c | 266 ------------ tools/hv/hv_fcopy_uio_daemon.c | 489 +++++++++++++++++++++++ tools/hv/vmbus_bufring.c | 322 +++++++++++++++ tools/hv/vmbus_bufring.h | 158 ++++++++ 14 files changed, 1260 insertions(+), 712 deletions(-) delete mode 100644 drivers/hv/hv_fcopy.c create mode 100644 drivers/uio/uio_hv_vmbus_client.c delete mode 100644 tools/hv/hv_fcopy_daemon.c create mode 100644 tools/hv/hv_fcopy_uio_daemon.c create mode 100644 tools/hv/vmbus_bufring.c create mode 100644 tools/hv/vmbus_bufring.h
Signature · 1 line
2.34.1
PATCH v2 1/5 uio: Add hv_vmbus_client driver
Generic VMBus driver that can be dynamically bound, to any simple low speed Hyper-V VMBus device. This driver supports single channel and uses hypercall instead of monitor bits to interrupt host, which is suitable for low speed devices. Additionally, the driver also provide the flexibility of custom ring buffer sizes and avoid memory allocation for gpadl to offer memory optimization.
---
[V2]
- Update driver info in Documentation/driver-api/uio-howto.rst
- Update ring_size sysfs info in Documentation/ABI/stable/sysfs-bus-vmbus
- Remove DRIVER_VERSION
- Remove refcnt
- scnprintf -> sysfs_emit
- sysfs_create_file -> ATTRIBUTE_GROUPS + ".driver.groups";
- sysfs_create_bin_file -> device_create_bin_file
- dev_notice -> dev_err
- remove MODULE_VERSION
Documentation/ABI/stable/sysfs-bus-vmbus | 7 + Documentation/driver-api/uio-howto.rst | 46 +++++ drivers/uio/Kconfig | 12 ++ drivers/uio/Makefile | 1 + drivers/uio/uio_hv_vmbus_client.c | 217 +++++++++++++++++++++++ 5 files changed, 283 insertions(+) create mode 100644 drivers/uio/uio_hv_vmbus_client.c
Documentation/ABI/stable/sysfs-bus-vmbus +7 -0
@@ -153,6 +153,13 @@ Contact: Stephen Hemminger <[email protected]> Description: Binary file created by uio_hv_generic for ring buffer Users: Userspace drivers +What: /sys/bus/vmbus/devices/<UUID>/ring_size +Date: June. 2023 +KernelVersion: 6.4 +Contact: Saurabh Sengar <[email protected]> +Description: File created by uio_hv_vmbus_client for setting device ring buffer size +Users: Userspace drivers + What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/intr_in_full Date: February 2019 KernelVersion: 5.0
Documentation/driver-api/uio-howto.rst +46 -0
52 lines not on this page · raw message
drivers/uio/Kconfig +12 -0
@@ -166,6 +166,18 @@ config UIO_HV_GENERIC If you compile this as a module, it will be called uio_hv_generic. +config UIO_HV_SLOW_DEVICES + tristate "Generic driver for low speed VMBus devices" + depends on HYPERV + help + Generic driver that you can bind, dynamically, to any simple + Hyper-V VMBus device with single channel and these devices + uses hypercall instead of monitor bits to interrupt host. This + driver also provide the flexibility of custom ring buffer sizes. + It is useful for slower VMbus devices. + + If you compile this as a module, it will be called uio_hv_vmbus_client. + config UIO_DFL tristate "Generic driver for DFL (Device Feature List) bus" depends on FPGA_DFL
drivers/uio/Makefile +1 -0
@@ -11,4 +11,5 @@ obj-$(CONFIG_UIO_PRUSS) += uio_pruss.o obj-$(CONFIG_UIO_MF624) += uio_mf624.o obj-$(CONFIG_UIO_FSL_ELBC_GPCM) += uio_fsl_elbc_gpcm.o obj-$(CONFIG_UIO_HV_GENERIC) += uio_hv_generic.o +obj-$(CONFIG_UIO_HV_SLOW_DEVICES) += uio_hv_vmbus_client.o obj-$(CONFIG_UIO_DFL) += uio_dfl.o
drivers/uio/uio_hv_vmbus_client.c new file +217 -0
217 lines not on this page · raw message
Signature · 1 line
2.34.1
Greg KH
maintainerRe: [PATCH v2 1/5] uio: Add hv_vmbus_client driver
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 157
Quoted 8 lines · Saurabh Sengar
Documentation/ABI/stable/sysfs-bus-vmbus +2
-0@@ -153,6 +153,13 @@ Contact: Stephen Hemminger <[email protected]> Description: Binary file created by uio_hv_generic for ring buffer Users: Userspace drivers +What: /sys/bus/vmbus/devices/<UUID>/ring_size +Date: June. 2023
No need for the "."
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 158
Quoted 1 line
Documentation/ABI/stable/sysfs-bus-vmbus +1
-0+KernelVersion: 6.4
6.4 will be released without this, sorry.
Quoted 23 lines
Documentation/ABI/stable/sysfs-bus-vmbus +4
-0+Contact: Saurabh Sengar <[email protected]> +Description: File created by uio_hv_vmbus_client for setting device ring buffer size +Users: Userspace drivers + What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/intr_in_full Date: February 2019 KernelVersion: 5.0Documentation/driver-api/uio-howto.rst +8
-0@@ -722,6 +722,52 @@ For example:: /sys/bus/vmbus/devices/3811fe4d-0fa0-4b62-981a-74fc1084c757/channels/21/ring +Generic Hyper-V driver for low speed devices +============================================ + +The generic driver is a kernel module named uio_hv_vmbus_client. It +supports slow devices on the Hyper-V VMBus similar to uio_hv_generic +for faster devices. This driver also gives flexibility of customized +ring buffer sizes. +
Documentation/driver-api/uio-howto.rst For example:: · line 742
Quoted 10 lines
Documentation/driver-api/uio-howto.rst +10
-0+Making the driver recognize the device +-------------------------------------- + +Since the driver does not declare any device GUID's, it will not get +loaded automatically and will not automatically bind to any devices, you +must load it and allocate id to the driver yourself. For example, to use +the fcopy device class GUID:: + + DEV_UUID=eb765408-105f-49b6-b4aa-c123b64d17d4 + driverctl -b vmbus set-override $DEV_UUID uio_hv_vmbus_client
Why are you adding a dependancy on a 300 line bash script that is not used by most distros?
Why not just show the "real" commands that you can use here that don't require an external tool not controlled by the kernel at all.
Quoted 5 lines
drivers/uio/uio_hv_vmbus_client.c new file +2
-0@@ -0,0 +1,217 @@ +// SPDX-License-Identifier: GPL-2.0 +/*
drivers/uio/uio_hv_vmbus_client.c line 12
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+* uio_hv_vmbus_client - UIO driver for low speed VMBus devices +* +* Copyright (c) 2023, Microsoft Corporation. +* +* Authors: +* Saurabh Sengar <[email protected]> +* +* Since the driver does not declare any device ids, you must allocate +* id and bind the device to the driver yourself. For example: +* driverctl -b vmbus set-override <dev uuid> uio_hv_vmbus_client
Again, no need to discuss driverctl.
drivers/uio/uio_hv_vmbus_client.c line 22
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+*/ + +#include <linux/device.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/uio_driver.h> +#include <linux/hyperv.h> + +#define DRIVER_AUTHOR "Saurabh Sengar <[email protected]>" +#define DRIVER_DESC "Generic UIO driver for low speed VMBus devices"
You only use these defines in one place, so why not just spell them out there, no need for 2 extra lines, right?
drivers/uio/uio_hv_vmbus_client.c line 25
Quoted 3 lines
drivers/uio/uio_hv_vmbus_client.c new file +3
-0+ +#define DEFAULT_HV_RING_SIZE VMBUS_RING_SIZE(3 * HV_HYP_PAGE_SIZE) +static int ring_size =DEFAULT_HV_RING_SIZE ;
You only use that #define in one place, why have it at all?
And you are defining a "global" variable that can be modified by an individual sysfs file for ANY device bound to this driver, messing with the other device's ring buffer size, right? This needs to be per-device, or explain in huge detail here why not.
Quoted 162 lines
drivers/uio/uio_hv_vmbus_client.c new file +162
-0+ +struct uio_hv_vmbus_dev { +struct uio_info info; +struct hv_device *device; +}; + +/* Sysfs API to allow mmap of the ring buffers */ +static int uio_hv_vmbus_mmap (struct file *filp,struct kobject *kobj, +struct bin_attribute *attr,struct vm_area_struct *vma) +{ + struct device *dev = container_of(kobj, struct device, kobj); + struct hv_device *hv_dev = container_of(dev, struct hv_device, device); +struct vmbus_channel *channel = hv_dev->channel; +void *ring_buffer =page_address (channel->ringbuffer_page); + +return vm_iomap_memory (vma,virt_to_phys (ring_buffer), + channel->ringbuffer_pagecount <<PAGE_SHIFT ); +} + +static const struct bin_attribute ring_buffer_bin_attr = { + .attr = { + .name ="ringbuffer" , + .mode =0600 , + }, + .mmap = uio_hv_vmbus_mmap, +}; + +/* +* This is the irqcontrol callback to be registered to uio_info. +* It can be used to disable/enable interrupt from user space processes. +* +* @param info +* pointer to uio_info. +* @param irq_state +* state value. 1 to enable interrupt, 0 to disable interrupt. +*/ +static int uio_hv_vmbus_irqcontrol (struct uio_info *info,s32 irq_state) +{ +struct uio_hv_vmbus_dev *pdata = info->priv; +struct hv_device *hv_dev = pdata->device; + +/* Issue a full memory barrier before triggering the notification */ +virt_mb (); + +vmbus_setevent (hv_dev->channel); +return 0 ; +} + +/* +* Callback from vmbus_event when something is in inbound ring. +*/ +static void uio_hv_vmbus_channel_cb (void *context) +{ +struct uio_hv_vmbus_dev *pdata = context; + +/* Issue a full memory barrier before sending the event to userspace */ +virt_mb (); + +uio_event_notify (&pdata->info); +} + +static int uio_hv_vmbus_open (struct uio_info *info,struct inode *inode) +{ + struct uio_hv_vmbus_dev *pdata = container_of(info, struct uio_hv_vmbus_dev, info); +struct hv_device *hv_dev = pdata->device; +struct vmbus_channel *channel = hv_dev->channel; +int ret; + + ret =vmbus_open (channel, ring_size, ring_size,NULL ,0 , + uio_hv_vmbus_channel_cb, pdata); +if (ret) { +dev_err (&hv_dev->device,"error %d when opening the channel\n" , ret); +return ret; + } + channel->inbound.ring_buffer->interrupt_mask =0 ; +set_channel_read_mode (channel,HV_CALL_ISR ); + + ret =device_create_bin_file (&hv_dev->device, &ring_buffer_bin_attr); +if (ret) +dev_err (&hv_dev->device,"sysfs create ring bin file failed; %d\n" , ret); + +return ret; +} + +/* VMbus primary channel is closed on last close */ +static int uio_hv_vmbus_release (struct uio_info *info,struct inode *inode) +{ + struct uio_hv_vmbus_dev *pdata = container_of(info, struct uio_hv_vmbus_dev, info); +struct hv_device *hv_dev = pdata->device; +struct vmbus_channel *channel = hv_dev->channel; + +device_remove_bin_file (&hv_dev->device, &ring_buffer_bin_attr); +vmbus_close (channel); + +return 0 ; +} + +static ssize_t ring_size_show (struct device *dev,struct device_attribute *attr, +char *buf) +{ +return sysfs_emit (buf,"%d\n" , ring_size); +} + +static ssize_t ring_size_store (struct device *dev,struct device_attribute *attr, +const char *buf,size_t count) +{ +unsigned int val; + +if (kstrtouint (buf,0 , &val) <0 ) +return -EINVAL ; + +if (val <HV_HYP_PAGE_SIZE ) +return -EINVAL ; + + ring_size = val; + +return count; +} + +static DEVICE_ATTR_RW (ring_size); + +static struct attribute *uio_hv_vmbus_client_attrs[] = { + &dev_attr_ring_size.attr, +NULL , +}; +ATTRIBUTE_GROUPS (uio_hv_vmbus_client); + +static int uio_hv_vmbus_probe (struct hv_device *dev,const struct hv_vmbus_device_id *dev_id) +{ +struct uio_hv_vmbus_dev *pdata; +int ret; +char *name =NULL ; + + pdata =devm_kzalloc (&dev->device,sizeof (*pdata),GFP_KERNEL ); +if (!pdata) +return -ENOMEM ; + + name =kasprintf (GFP_KERNEL ,"%pUl" , &dev->dev_instance); + +/* Fill general uio info */ + pdata->info.name = name;/* /sys/class/uio/uioX/name */ + pdata->info.version ="1" ; + pdata->info.irqcontrol = uio_hv_vmbus_irqcontrol; + pdata->info.open = uio_hv_vmbus_open; + pdata->info.release = uio_hv_vmbus_release; + pdata->info.irq =UIO_IRQ_CUSTOM ; + pdata->info.priv = pdata; + pdata->device = dev; + + ret =uio_register_device (&dev->device, &pdata->info); +if (ret) { +dev_err (&dev->device,"uio_hv_vmbus register failed\n" ); +return ret; + } + +hv_set_drvdata (dev, pdata); + +return 0 ; +} + +static void uio_hv_vmbus_remove (struct hv_device *dev) +{
drivers/uio/uio_hv_vmbus_client.c line 197
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+struct uio_hv_vmbus_dev *pdata =hv_get_drvdata (dev); + +if (pdata) +uio_unregister_device (&pdata->info); +} + +static struct hv_driver uio_hv_vmbus_drv = { + .driver.dev_groups = uio_hv_vmbus_client_groups, + .name ="uio_hv_vmbus_client" , + .id_table =NULL ,/* only dynamic id's */
No need to set this if it's NULL.
thanks,
greg k-h
RE: [EXTERNAL] Re: [PATCH v2 1/5] uio: Add hv_vmbus_client driver
Quoted 18 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Thursday, June 15, 2023 2:43 AM
To: Saurabh Sengar <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] Re: [PATCH v2 1/5] uio: Addhv_vmbus_clientdriverDocumentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger
Quoted 3 lines · Saurabh Sengar
Documentation/ABI/stable/sysfs-bus-vmbus +0
-0@@ -153,6 +153,13 @@ Contact: Stephen Hemminger
Quoted 7 lines
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 157
Quoted 5 lines
Documentation/ABI/stable/sysfs-bus-vmbus +2
-0Description: Binary file created by uio_hv_generic for ring buffer Users: Userspace drivers +What: /sys/bus/vmbus/devices/<UUID>/ring_size +Date: June. 2023No need for the "."
OK
Quoted 4 lines
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 158
Quoted 2 lines
Documentation/ABI/stable/sysfs-bus-vmbus +1
-0+KernelVersion: 6.46.4 will be released without this, sorry.
Ok will change it to 6.5.
Quoted 43 lines
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger · line 160
Quoted 3 lines
Documentation/ABI/stable/sysfs-bus-vmbus +2
-0+Contact: Saurabh Sengar <[email protected]> +Description: File created by uio_hv_vmbus_client for setting device ringbuffer size
Quoted 13 lines
Documentation/ABI/stable/sysfs-bus-vmbus +2
-0+Users: Userspace drivers + What: /sys/bus/vmbus/devices/<UUID>/channels/<N>/intr_in_full Date: February 2019 KernelVersion: 5.0Documentation/driver-api/uio-howto.rst +0
-0@@ -722,6 +722,52 @@ For example::Quoted 1 line
/sys/bus/vmbus/devices/3811fe4d-0fa0-4b62-981a-74fc1084c757/channels/2
Quoted 10 lines
1/ring
Documentation/driver-api/uio-howto.rst +8
-0+Generic Hyper-V driver for low speed devices +============================================ + +The generic driver is a kernel module named uio_hv_vmbus_client. It +supports slow devices on the Hyper-V VMBus similar to uio_hv_generic +for faster devices. This driver also gives flexibility of customized +ring buffer sizes. +Documentation/driver-api/uio-howto.rst For example:: · line 742
Quoted 10 lines
Documentation/driver-api/uio-howto.rst +10
-0+Making the driver recognize the device +-------------------------------------- + +Since the driver does not declare any device GUID's, it will not get +loaded automatically and will not automatically bind to any devices, +you must load it and allocate id to the driver yourself. For example, +to use the fcopy device class GUID:: + + DEV_UUID=eb765408-105f-49b6-b4aa-c123b64d17d4 + driverctl -b vmbus set-override $DEV_UUID uio_hv_vmbus_clientWhy are you adding a dependancy on a 300 line bash script that is not used by most distros?
Quoted 2 lines
Why not just show the "real" commands that you can use here that don't require an external tool not controlled by the kernel at all.
Ok will mention the regular "echo" commands as you suggested.
Quoted 11 lines
Quoted 11 lines
drivers/uio/uio_hv_vmbus_client.c new file +7
-0@@ -0,0 +1,217 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* uio_hv_vmbus_client - UIO driver for low speed VMBus devices +* +* Copyright (c) 2023, Microsoft Corporation. +* +* Authors:
Quoted 8 lines
drivers/uio/uio_hv_vmbus_client.c line 12
Quoted 6 lines
drivers/uio/uio_hv_vmbus_client.c new file +6
-0+* Saurabh Sengar <[email protected]> +* +* Since the driver does not declare any device ids, you must +allocate +* id and bind the device to the driver yourself. For example: +* driverctl -b vmbus set-override <dev uuid> uio_hv_vmbus_client Again, no need to discuss driverctl.
Noted.
Quoted 13 lines
drivers/uio/uio_hv_vmbus_client.c
Quoted 11 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+*/ + +#include <linux/device.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/uio_driver.h> +#include <linux/hyperv.h> + +#define DRIVER_AUTHOR "Saurabh Sengar <[email protected]>" +#define DRIVER_DESC "Generic UIO driver for low speed VMBusdevices"
Quoted 2 lines
You only use these defines in one place, so why not just spell them out there, no need for 2 extra lines, right?
Sure, will fix
Quoted 7 lines
drivers/uio/uio_hv_vmbus_client.c line 25
Quoted 3 lines
drivers/uio/uio_hv_vmbus_client.c new file +2
-0+ +#define DEFAULT_HV_RING_SIZE VMBUS_RING_SIZE(3 *
HV_HYP_PAGE_SIZE)drivers/uio/uio_hv_vmbus_client.c line 25
Quoted 1 line
drivers/uio/uio_hv_vmbus_client.c new file +1
-0+static int ring_size =DEFAULT_HV_RING_SIZE ;You only use that #define in one place, why have it at all?
Ok, will fix
Quoted 5 lines
And you are defining a "global" variable that can be modified by an individual sysfs file for ANY device bound to this driver, messing with the other device's ring buffer size, right? This needs to be per-device, or explain in huge detail here why not.
The global variable is expected to be set by userspace per device before opening, the particular uio device. For a particular Hyper-v device this value be same, and once device is open the ring buffer is allocated and there won't be any impact afterwards changing it. I can elaborate more of this in sysfs documentation.
Quoted 171 lines
drivers/uio/uio_hv_vmbus_client.c
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +9
-0+ +struct uio_hv_vmbus_dev { +struct uio_info info; +struct hv_device *device; +}; + +/* Sysfs API to allow mmap of the ring buffers */ static int +uio_hv_vmbus_mmap (struct file *filp,struct kobject *kobj, + struct bin_attribute *attr, struct vm_area_struct*vma) {
drivers/uio/uio_hv_vmbus_client.c line 37
Quoted 2 lines
drivers/uio/uio_hv_vmbus_client.c new file +2
-0+ struct device *dev = container_of(kobj, struct device, kobj); + struct hv_device *hv_dev = container_of(dev, struct hv_device,device);
Quoted 32 lines
drivers/uio/uio_hv_vmbus_client.c new file +32
-0+struct vmbus_channel *channel = hv_dev->channel; +void *ring_buffer =page_address (channel->ringbuffer_page); + +return vm_iomap_memory (vma,virt_to_phys (ring_buffer), + channel->ringbuffer_pagecount <<PAGE_SHIFT ); } + +static const struct bin_attribute ring_buffer_bin_attr = { + .attr = { + .name ="ringbuffer" , + .mode =0600 , + }, + .mmap = uio_hv_vmbus_mmap, +}; + +/* +* This is the irqcontrol callback to be registered to uio_info. +* It can be used to disable/enable interrupt from user space processes. +* +* @param info +* pointer to uio_info. +* @param irq_state +* state value. 1 to enable interrupt, 0 to disable interrupt. +*/ +static int uio_hv_vmbus_irqcontrol (struct uio_info *info,s32 +irq_state) { +struct uio_hv_vmbus_dev *pdata = info->priv; +struct hv_device *hv_dev = pdata->device; + +/* Issue a full memory barrier before triggering the notification */ +virt_mb (); + +vmbus_setevent (hv_dev->channel);drivers/uio/uio_hv_vmbus_client.c
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+return 0 ; +} + +/* +* Callback from vmbus_event when something is in inbound ring. +*/ +static void uio_hv_vmbus_channel_cb (void *context) { +struct uio_hv_vmbus_dev *pdata = context; + +/* Issue a full memory barrier before sending the event to userspace */
drivers/uio/uio_hv_vmbus_client.c
Quoted 8 lines
drivers/uio/uio_hv_vmbus_client.c new file +8
-0+virt_mb (); + +uio_event_notify (&pdata->info); +} + +static int uio_hv_vmbus_open (struct uio_info *info,struct inode +*inode) { + struct uio_hv_vmbus_dev *pdata = container_of(info, struct
uio_hv_vmbus_dev, info);drivers/uio/uio_hv_vmbus_client.c
Quoted 8 lines
drivers/uio/uio_hv_vmbus_client.c new file +8
-0+struct hv_device *hv_dev = pdata->device; +struct vmbus_channel *channel = hv_dev->channel; +int ret; + + ret =vmbus_open (channel, ring_size, ring_size,NULL ,0 , + uio_hv_vmbus_channel_cb, pdata); +if (ret) { + dev_err(&hv_dev->device, "error %d when opening thechannel\n", ret);
drivers/uio/uio_hv_vmbus_client.c
Quoted 6 lines
drivers/uio/uio_hv_vmbus_client.c new file +6
-0+return ret; + } + channel->inbound.ring_buffer->interrupt_mask =0 ; +set_channel_read_mode (channel,HV_CALL_ISR ); + + ret = device_create_bin_file(&hv_dev->device,
&ring_buffer_bin_attr);drivers/uio/uio_hv_vmbus_client.c line 103
Quoted 2 lines
drivers/uio/uio_hv_vmbus_client.c new file +2
-0+if (ret) + dev_err(&hv_dev->device, "sysfs create ring bin file failed;%d\n",
drivers/uio/uio_hv_vmbus_client.c
Quoted 8 lines
drivers/uio/uio_hv_vmbus_client.c new file +8
-0+ret); + +return ret; +} + +/* VMbus primary channel is closed on last close */ static int +uio_hv_vmbus_release (struct uio_info *info,struct inode *inode) { + struct uio_hv_vmbus_dev *pdata = container_of(info, struct
uio_hv_vmbus_dev, info);drivers/uio/uio_hv_vmbus_client.c
Quoted 10 lines
drivers/uio/uio_hv_vmbus_client.c new file +10
-0+struct hv_device *hv_dev = pdata->device; +struct vmbus_channel *channel = hv_dev->channel; + +device_remove_bin_file (&hv_dev->device, &ring_buffer_bin_attr); +vmbus_close (channel); + +return 0 ; +} + +static ssize_t ring_size_show(struct device *dev, struct device_attribute*attr,
drivers/uio/uio_hv_vmbus_client.c
Quoted 5 lines
drivers/uio/uio_hv_vmbus_client.c new file +5
-0+ char *buf) +{ +return sysfs_emit (buf,"%d\n" , ring_size); } + +static ssize_t ring_size_store(struct device *dev, struct device_attribute*attr,
Quoted 60 lines
drivers/uio/uio_hv_vmbus_client.c new file +60
-0+ const char *buf, size_t count) { + unsigned int val; + +if (kstrtouint (buf,0 , &val) <0 ) +return -EINVAL ; + +if (val <HV_HYP_PAGE_SIZE ) +return -EINVAL ; + + ring_size = val; + +return count; +} + +static DEVICE_ATTR_RW (ring_size); + +static struct attribute *uio_hv_vmbus_client_attrs[] = { + &dev_attr_ring_size.attr, +NULL , +}; +ATTRIBUTE_GROUPS (uio_hv_vmbus_client); + +static int uio_hv_vmbus_probe (struct hv_device *dev,const struct +hv_vmbus_device_id *dev_id) { +struct uio_hv_vmbus_dev *pdata; +int ret; +char *name =NULL ; + + pdata =devm_kzalloc (&dev->device,sizeof (*pdata),GFP_KERNEL ); +if (!pdata) +return -ENOMEM ; + + name =kasprintf (GFP_KERNEL ,"%pUl" , &dev->dev_instance); + +/* Fill general uio info */ + pdata->info.name = name;/* /sys/class/uio/uioX/name */ + pdata->info.version ="1" ; + pdata->info.irqcontrol = uio_hv_vmbus_irqcontrol; + pdata->info.open = uio_hv_vmbus_open; + pdata->info.release = uio_hv_vmbus_release; + pdata->info.irq =UIO_IRQ_CUSTOM ; + pdata->info.priv = pdata; + pdata->device = dev; + + ret =uio_register_device (&dev->device, &pdata->info); +if (ret) { +dev_err (&dev->device,"uio_hv_vmbus register failed\n" ); +return ret; + } + +hv_set_drvdata (dev, pdata); + +return 0 ; +} + +static void uio_hv_vmbus_remove (struct hv_device *dev) { +struct uio_hv_vmbus_dev *pdata =hv_get_drvdata (dev); + +if (pdata) +uio_unregister_device (&pdata->info);
Quoted 8 lines
drivers/uio/uio_hv_vmbus_client.c line 197
Quoted 6 lines
drivers/uio/uio_hv_vmbus_client.c new file +6
-0+} + +static struct hv_driver uio_hv_vmbus_drv = { + .driver.dev_groups = uio_hv_vmbus_client_groups, + .name ="uio_hv_vmbus_client" , + .id_table =NULL ,/* only dynamic id's */ No need to set this if it's NULL.
Ok.
Thanks for your review.
- Saurabh
Quoted 4 lines
thanks,
greg k-h
Greg KH
maintainerRe: [EXTERNAL] Re: [PATCH v2 1/5] uio: Add hv_vmbus_client driver
Quoted 9 lines · Saurabh Singh Sengar
Quoted 4 lines
And you are defining a "global" variable that can be modified by an individual sysfs file for ANY device bound to this driver, messing with the other device's ring buffer size, right? This needs to be per-device, or explain in huge detail here why not.
The global variable is expected to be set by userspace per device before opening, the particular uio device. For a particular Hyper-v device this value be same, and once device is open the ring buffer is allocated and there won't be any impact afterwards changing it. I can elaborate more of this in sysfs documentation.
That's totally confusing, please make this per-device properly, as you will find out when you try to document it, what you are describing is unlike any other per-device interface we have.
greg k-h
RE: [EXTERNAL] Re: [PATCH v2 1/5] uio: Add hv_vmbus_client driver
Quoted 24 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, June 20, 2023 12:37 PM
To: Saurabh Singh Sengar <[email protected]>
Cc: Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; linux- [email protected]; [email protected]; linux- [email protected]
Subject: Re: [EXTERNAL] Re: [PATCH v2 1/5] uio: Addhv_vmbus_clientdriverQuoted 9 lines · Saurabh Singh Sengar
Quoted 4 lines
And you are defining a "global" variable that can be modified by an individual sysfs file for ANY device bound to this driver, messing with the other device's ring buffer size, right? This needs to be per-device, or explain in huge detail here why not.
The global variable is expected to be set by userspace per device before opening, the particular uio device. For a particular Hyper-v device this value be same, and once device is open the ring buffer is allocated and there won't be any impact afterwards changing it. I can
Quoted 5 lines
elaborate more of this in sysfs documentation.
That's totally confusing, please make this per-device properly, as you will find out when you try to document it, what you are describing is unlike any other per-device interface we have.
Ok, will make this per device. Thanks.
- Saurabh
Quoted 2 lines
greg k-h
PATCH v2 2/5 tools: hv: Add vmbus_bufring
In reply to Saurabh SengarCommon userspace interface for read/write from VMBus ringbuffer. This implementation is open for use by any userspace driver or application seeking direct control over VMBus ring buffers. A significant part of this code is borrowed from DPDK.
---
[V2]
- simpler sysfs path, less parsing
tools/hv/vmbus_bufring.c | 322 +++++++++++++++++++++++++++++++++++++++ tools/hv/vmbus_bufring.h | 158 +++++++++++++++++++ 2 files changed, 480 insertions(+) create mode 100644 tools/hv/vmbus_bufring.c create mode 100644 tools/hv/vmbus_bufring.h
tools/hv/vmbus_bufring.c new file +322 -0
@@ -0,0 +1,322 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved. +*/ + +#include <errno.h> +#include <fcntl.h> +#include <emmintrin.h> +#include <linux/limits.h> +#include <stdbool.h> +#include <stdint.h> +#include <stdio.h> +#include <string.h> +#include <sys/mman.h> +#include <sys/uio.h> +#include <unistd.h> +#include "vmbus_bufring.h" + +#define rte_compiler_barrier () ({ asm volatile ("" : : : "memory"); }) + +#define rte_smp_rwmb () ({ asm volatile ("" : : : "memory"); }) + +#define VMBUS_RQST_ERROR 0xFFFFFFFFFFFFFFFF +#define ALIGN (val, align) ((typeof(val))((val) & (~((typeof(val))((align) - 1))))) + +void *vmbus_uio_map (char *sys_ring_buf_path,int size) +{ +void *map; +int fd; + + fd =open (sys_ring_buf_path,O_RDWR ); +if (fd <0 ) +return NULL ; + + map =mmap (NULL ,2 * size,PROT_READ |PROT_WRITE ,MAP_SHARED , fd,0 ); +close (fd); +if (map ==MAP_FAILED ) +return NULL ; + +return map; +} + +/* Increase bufring index by inc with wraparound */ +static inline uint32_t vmbus_br_idxinc (uint32_t idx,uint32_t inc,uint32_t sz) +{ + idx += inc; +if (idx >= sz) + idx -= sz; + +return idx; +} + +void vmbus_br_setup (struct vmbus_br *br,void *buf,unsigned int blen) +{ + br->vbr = buf; + br->windex = br->vbr->windex; + br->dsize = blen -sizeof (struct vmbus_bufring ); +} + +static inline __always_inline void +rte_smp_mb (void ) +{ + asmvolatile ("lock addl $0, -128(%%rsp); " :::"memory" ); +} + +static inline int +rte_atomic32_cmpset (volatile uint32_t *dst,uint32_t exp,uint32_t src) +{ +uint8_t res; + + asmvolatile ("lock ; " +"cmpxchgl %[src], %[dst];" +"sete %[res];" + : [res]"=a" (res),/* output */ + [dst]"=m" (*dst) + : [src]"r" (src),/* input */ +"a" (exp), +"m" (*dst) + :"memory" );/* no-clobber list */ +return res; +} + +static inline uint32_t +vmbus_txbr_copyto (const struct vmbus_br *tbr,uint32_t windex, +const void *src0,uint32_t cplen) +{ +uint8_t *br_data = tbr->vbr->data; +uint32_t br_dsize = tbr->dsize; +const uint8_t *src = src0; + +/* XXX use double mapping like Linux kernel? */ +if (cplen > br_dsize - windex) { +uint32_t fraglen = br_dsize - windex; + +/* Wrap-around detected */ +memcpy (br_data + windex, src, fraglen); +memcpy (br_data, src + fraglen, cplen - fraglen); + }else { +memcpy (br_data + windex, src, cplen); + } + +return vmbus_br_idxinc (windex, cplen, br_dsize); +} + +/* +* Write scattered channel packet to TX bufring. +* +* The offset of this channel packet is written as a 64bits value +* immediately after this channel packet. +* +* The write goes through three stages: +* 1. Reserve space in ring buffer for the new data. +* Writer atomically moves priv_write_index. +* 2. Copy the new data into the ring. +* 3. Update the tail of the ring (visible to host) that indicates +* next read location. Writer updates write_index +*/ +static int +vmbus_txbr_write (struct vmbus_br *tbr,const struct iovec iov[],int iovlen, +bool *need_sig) +{ +struct vmbus_bufring *vbr = tbr->vbr; +uint32_t ring_size = tbr->dsize; +uint32_t old_windex, next_windex, windex, total; +uint64_t save_windex; +int i; + + total =0 ; +for (i =0 ; i < iovlen; i++) + total += iov[i].iov_len; + total +=sizeof (save_windex); + +/* Reserve space in ring */ +do { +uint32_t avail; + +/* Get current free location */ + old_windex = tbr->windex; + +/* Prevent compiler reordering this with calculation */ +rte_compiler_barrier (); + + avail =vmbus_br_availwrite (tbr, old_windex); + +/* If not enough space in ring, then tell caller. */ +if (avail <= total) +return -EAGAIN ; + + next_windex =vmbus_br_idxinc (old_windex, total, ring_size); + +/* Atomic update of next write_index for other threads */ + }while (!rte_atomic32_cmpset (&tbr->windex, old_windex, next_windex)); + +/* Space from old..new is now reserved */ + windex = old_windex; +for (i =0 ; i < iovlen; i++) + windex =vmbus_txbr_copyto (tbr, windex, iov[i].iov_base, iov[i].iov_len); + +/* Set the offset of the current channel packet. */ + save_windex = ((uint64_t )old_windex) <<32 ; + windex =vmbus_txbr_copyto (tbr, windex, &save_windex, +sizeof (save_windex)); + +/* The region reserved should match region used */ +if (windex != next_windex) +return -EINVAL ; + +/* Ensure that data is available before updating host index */ +rte_smp_rwmb (); + +/* Checkin for our reservation. wait for our turn to update host */ +while (!rte_atomic32_cmpset (&vbr->windex, old_windex, next_windex)) +_mm_pause (); + +return 0 ; +} + +int rte_vmbus_chan_send (struct vmbus_br *txbr,uint16_t type,void *data, +uint32_t dlen,uint32_t flags) +{ +struct vmbus_chanpkt pkt; +unsigned int pktlen, pad_pktlen; +const uint32_t hlen =sizeof (pkt); +bool send_evt = false; +uint64_t pad =0 ; +struct iovec iov[3 ]; +int error; + + pktlen = hlen + dlen; + pad_pktlen =ALIGN (pktlen,sizeof (uint64_t )); + + pkt.hdr.type = type; + pkt.hdr.flags = flags; + pkt.hdr.hlen = hlen >>VMBUS_CHANPKT_SIZE_SHIFT ; + pkt.hdr.tlen = pad_pktlen >>VMBUS_CHANPKT_SIZE_SHIFT ; + pkt.hdr.xactid =VMBUS_RQST_ERROR ; + + iov[0 ].iov_base = &pkt; + iov[0 ].iov_len = hlen; + iov[1 ].iov_base = data; + iov[1 ].iov_len = dlen; + iov[2 ].iov_base = &pad; + iov[2 ].iov_len = pad_pktlen - pktlen; + + error =vmbus_txbr_write (txbr, iov,3 , &send_evt); + +return error; +} + +static inline uint32_t +vmbus_rxbr_copyfrom (const struct vmbus_br *rbr,uint32_t rindex, +void *dst0,size_t cplen) +{ +const uint8_t *br_data = rbr->vbr->data; +uint32_t br_dsize = rbr->dsize; +uint8_t *dst = dst0; + +if (cplen > br_dsize - rindex) { +uint32_t fraglen = br_dsize - rindex; + +/* Wrap-around detected. */ +memcpy (dst, br_data + rindex, fraglen); +memcpy (dst + fraglen, br_data, cplen - fraglen); + }else { +memcpy (dst, br_data + rindex, cplen); + } + +return vmbus_br_idxinc (rindex, cplen, br_dsize); +} + +/* Copy data from receive ring but don't change index */ +static int +vmbus_rxbr_peek (const struct vmbus_br *rbr,void *data,size_t dlen) +{ +uint32_t avail; + +/* +* The requested data and the 64bits channel packet +* offset should be there at least. +*/ + avail =vmbus_br_availread (rbr); +if (avail < dlen +sizeof (uint64_t )) +return -EAGAIN ; + +vmbus_rxbr_copyfrom (rbr, rbr->vbr->rindex, data, dlen); +return 0 ; +} + +/* +* Copy data from receive ring and change index +* NOTE: +* We assume (dlen + skip) == sizeof(channel packet). +*/ +static int +vmbus_rxbr_read (struct vmbus_br *rbr,void *data,size_t dlen,size_t skip) +{ +struct vmbus_bufring *vbr = rbr->vbr; +uint32_t br_dsize = rbr->dsize; +uint32_t rindex; + +if (vmbus_br_availread (rbr) < dlen + skip +sizeof (uint64_t )) +return -EAGAIN ; + +/* Record where host was when we started read (for debug) */ + rbr->windex = rbr->vbr->windex; + +/* +* Copy channel packet from RX bufring. +*/ + rindex =vmbus_br_idxinc (rbr->vbr->rindex, skip, br_dsize); + rindex =vmbus_rxbr_copyfrom (rbr, rindex, data, dlen); + +/* +* Discard this channel packet's 64bits offset, which is useless to us. +*/ + rindex =vmbus_br_idxinc (rindex,sizeof (uint64_t ), br_dsize); + +/* Update the read index _after_ the channel packet is fetched. */ +rte_compiler_barrier (); + + vbr->rindex = rindex; + +return 0 ; +} + +int rte_vmbus_chan_recv_raw (struct vmbus_br *rxbr, +void *data,uint32_t *len) +{ +struct vmbus_chanpkt_hdr pkt; +uint32_t dlen, bufferlen = *len; +int error; + + error =vmbus_rxbr_peek (rxbr, &pkt,sizeof (pkt)); +if (error) +return error; + +if (unlikely (pkt.hlen <VMBUS_CHANPKT_HLEN_MIN )) +/* XXX this channel is dead actually. */ +return -EIO ; + +if (unlikely (pkt.hlen > pkt.tlen)) +return -EIO ; + +/* Length are in quad words */ + dlen = pkt.tlen <<VMBUS_CHANPKT_SIZE_SHIFT ; + *len = dlen; + +/* If caller buffer is not large enough */ +if (unlikely (dlen > bufferlen)) +return -ENOBUFS ; + +/* Read data and skip packet header */ + error =vmbus_rxbr_read (rxbr, data, dlen,0 ); +if (error) +return error; + +/* Return the number of bytes read */ +return dlen +sizeof (uint64_t ); +}
tools/hv/vmbus_bufring.h new file +158 -0
158 lines not on this page · raw message
Signature · 1 line
2.34.1
Greg KH
maintainerRe: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
Quoted 4 lines · Saurabh Sengar
Common userspace interface for read/write from VMBus ringbuffer. This implementation is open for use by any userspace driver or application seeking direct control over VMBus ring buffers. A significant part of this code is borrowed from DPDK.
" "?
Anyway, this does not explain what this is at all.
And if you "borrowed" it from DPDK, that feels odd, are you sure you are allowed to do so?
Quoted 1 line
Not what a Link: tag is for, sorry.
Quoted 11 lines
Signed-off-by Saurabh Sengar <[email protected]>---
[V2]
- simpler sysfs path, less parsingtools/hv/vmbus_bufring.c | 322 +++++++++++++++++++++++++++++++++++++++ tools/hv/vmbus_bufring.h | 158 +++++++++++++++++++ 2 files changed, 480 insertions(+) create mode 100644 tools/hv/vmbus_bufring.c create mode 100644 tools/hv/vmbus_bufring.h
You add new files to the tools directory, yet say nothing about how to use them or even how to build them.
Why is there a .h file for a single .c file? That seems pointless, right?
tools/hv/vmbus_bufring.c line 6
Quoted 12 lines
tools/hv/vmbus_bufring.c new file +6
-0@@ -0,0 +1,322 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved.
No copyright for the work you did?
Quoted 8 lines
tools/hv/vmbus_bufring.c new file +8
-0+*/ + +#include <errno.h> +#include <fcntl.h> +#include <emmintrin.h> +#include <linux/limits.h> +#include <stdbool.h> +#include <stdint.h>
tools/hv/vmbus_bufring.c line 24
Quoted 10 lines
tools/hv/vmbus_bufring.c new file +10
-0+#include <stdio.h> +#include <string.h> +#include <sys/mman.h> +#include <sys/uio.h> +#include <unistd.h> +#include "vmbus_bufring.h" + +#define rte_compiler_barrier () ({ asm volatile ("" : : : "memory"); }) + +#define rte_smp_rwmb () ({ asm volatile ("" : : : "memory"); })
These aren't in any common header file already?
thanks,
greg k-h
RE: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
Quoted 14 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Thursday, June 15, 2023 2:46 AM
To: Saurabh Sengar <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Addvmbus_bufringOn Wed, Jun 14, 2023 at 11:15:09AM -0700, Saurabh Sengar wrote:
Quoted 8 lines
Quoted 4 lines
Common userspace interface for read/write from VMBus ringbuffer. This implementation is open for use by any userspace driver or application seeking direct control over VMBus ring buffers. A significant part of this code is borrowed from DPDK.
" "?
Anyway, this does not explain what this is at all.
I can elaborate more in next version.
Quoted 3 lines
And if you "borrowed" it from DPDK, that feels odd, are you sure you are allowed to do so?
I will confirm this internally before sending next version.
Quoted 10 lines
Quoted 4 lines
Link:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDPDK%2Fdpdk%2F&data=05%7C01%7Cssengar%40microsoft.com %7C79975
Quoted 1 line
a82b7b44c67b0b508db6d1c9301%7C72f988bf86f141af91ab2d7cd011db47%7 C1%7C0
Quoted 1 line
Quoted 8 lines
%7C638223741757437265%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLj AwMDAiLCJQ
Quoted 1 line
IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata =100fd
Quoted 1 line
FVed6C5lBrikWqkWwFpfH33LHF0H8fuRb0myL0%3D&reserved=0
Not what a Link: tag is for, sorry.
Will fix, thanks for pointing this.
Quoted 14 lines
Quoted 7 lines
Signed-off-by Saurabh Sengar <[email protected]>---
[V2]
- simpler sysfs path, less parsingQuoted 6 lines
tools/hv/vmbus_bufring.c | 322 +++++++++++++++++++++++++++++++++++++++ tools/hv/vmbus_bufring.h | 158 +++++++++++++++++++ 2 files changed, 480 insertions(+) create mode 100644 tools/hv/vmbus_bufring.c create mode 100644
tools/hv/vmbus_bufring.h
Quoted 4 lines
You add new files to the tools directory, yet say nothing about how to use them or even how to build them.
Why is there a .h file for a single .c file? That seems pointless, right?
This is a header file so that any userspace application can add it. This file Is used by fcopy application in [PATCH v2 3/5] of this patch series. If this is confusing, shall I merge 2/5 and 3/5 ? I thought better to keep the common library code as separate patch. Please let me know your opinion.
Quoted 14 lines
tools/hv/vmbus_bufring.c line 6
Quoted 12 lines
tools/hv/vmbus_bufring.c new renamed from tools/hv/vmbus_bufring.c +0
-0file mode 100644 index 000000000000..d44a06d45b03
tools/hv/vmbus_bufring.c new file +6
-0@@ -0,0 +1,322 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved. No copyright for the work you did?
I have added 2023 Microsoft Corp. Please let me know if I need to add anything more.
Quoted 21 lines
Quoted 9 lines
tools/hv/vmbus_bufring.c new file +8
-0+*/ + +#include <errno.h> +#include <fcntl.h> +#include <emmintrin.h> +#include <linux/limits.h> +#include <stdbool.h> +#include <stdint.h> tools/hv/vmbus_bufring.c
Quoted 10 lines
tools/hv/vmbus_bufring.c new file +10
-0+#include <stdio.h> +#include <string.h> +#include <sys/mman.h> +#include <sys/uio.h> +#include <unistd.h> +#include "vmbus_bufring.h" + +#define rte_compiler_barrier () ({ asm volatile ("" : : : "memory"); }) + +#define rte_smp_rwmb () ({ asm volatile ("" : : :"memory"); })
Quoted 1 line
These aren't in any common header file already?
I see every userspace application is maintaining their separate copy of this. Although I can remove this duplicate define and can use only one of these.
- Saurabh
Quoted 4 lines
thanks,
greg k-h
Greg KH
maintainerRe: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
Quoted 14 lines · Saurabh Singh Sengar
Quoted 5 lines
Quoted 5 lines
tools/hv/vmbus_bufring.c new renamed from tools/hv/vmbus_bufring.c +0
-0file mode 100644 index 000000000000..d44a06d45b03
tools/hv/vmbus_bufring.c new file +0
-0@@ -0,0 +1,322 @@Quoted 8 lines
tools/hv/vmbus_bufring.c line 6
Quoted 6 lines
tools/hv/vmbus_bufring.c new file +6
-0+// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved. No copyright for the work you did?
Quoted 2 lines
I have added 2023 Microsoft Corp. Please let me know if I need to add anything more.
Please contact your lawyers about EXACTLY what you need to do here, that's up to them, not me!
greg k-h
RE: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
Quoted 29 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, June 20, 2023 12:36 PM
To: Saurabh Singh Sengar <[email protected]>
Cc: Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; linux- [email protected]; [email protected]; linux- [email protected]
Subject: Re: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Addvmbus_bufringQuoted 14 lines · Saurabh Singh Sengar
Quoted 13 lines
tools/hv/vmbus_bufring.c line 6
Quoted 11 lines
tools/hv/vmbus_bufring.c new file +6
-0@@ -0,0 +1,322 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved. No copyright for the work you did?
Quoted 5 lines
Quoted 2 lines
I have added 2023 Microsoft Corp. Please let me know if I need to add anything more.
Please contact your lawyers about EXACTLY what you need to do here, that's up to them, not me!
I have cross verified this with our legal team and they are fine with it.
- Saurabh
Quoted 2 lines
greg k-h
Greg KH
maintainerRe: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
Quoted 29 lines · Saurabh Singh Sengar
Quoted 29 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, June 20, 2023 12:36 PM
To: Saurabh Singh Sengar <[email protected]>
Cc: Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; linux- [email protected]; [email protected]; linux- [email protected]
Subject: Re: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Addvmbus_bufringQuoted 14 lines · Saurabh Singh Sengar
Quoted 13 lines
tools/hv/vmbus_bufring.c line 6
Quoted 11 lines
tools/hv/vmbus_bufring.c new file +6
-0@@ -0,0 +1,322 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* +* Copyright (c) 2009-2012,2016,2023 Microsoft Corp. +* Copyright (c) 2012 NetApp Inc. +* Copyright (c) 2012 Citrix Inc. +* All rights reserved. No copyright for the work you did?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I have added 2023 Microsoft Corp. Please let me know if I need to add anything more.
Please contact your lawyers about EXACTLY what you need to do here, that's up to them, not me!
I have cross verified this with our legal team and they are fine with it.
Wonderful, please have them sign-off on the patch as well when you resubmit it.
thanks,
greg k-h
RE: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Add vmbus_bufring
In reply to Greg KHQuoted 19 lines
-----Original Message-----
From: Greg KH <[email protected]>
Sent: Thursday, June 15, 2023 2:46 AM
To: Saurabh Sengar <[email protected]>
Cc: KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; Michael Kelley (LINUX) <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] Re: [PATCH v2 2/5] tools: hv: Addvmbus_bufringQuoted 4 lines · Saurabh Sengar
Common userspace interface for read/write from VMBus ringbuffer. This implementation is open for use by any userspace driver or application seeking direct control over VMBus ring buffers. A significant part of this code is borrowed from DPDK.
Quoted 6 lines
" "?
Anyway, this does not explain what this is at all.
And if you "borrowed" it from DPDK, that feels odd, are you sure you are allowed to do so?
Yes, we have confirmed this with our legal team, as the previous code is covered under an appropriate open-source license, we are fine to reuse it here.
- Saurabh
PATCH v2 3/5 tools: hv: Add new fcopy application based on uio driver
In reply to Saurabh SengarNew fcopy application which utilizes uio_hv_vmbus_client driver
---
[V2]
- simpler sysfs path
tools/hv/Build | 3 +- tools/hv/Makefile | 10 +- tools/hv/hv_fcopy_uio_daemon.c | 489 +++++++++++++++++++++++++++++++++ 3 files changed, 496 insertions(+), 6 deletions(-) create mode 100644 tools/hv/hv_fcopy_uio_daemon.c
tools/hv/Build +2 -1
@@ -1,3 +1,4 @@ hv_kvp_daemon-y += hv_kvp_daemon.o hv_vss_daemon-y += hv_vss_daemon.o -hv_fcopy_daemon-y += hv_fcopy_daemon.o +hv_fcopy_uio_daemon-y += hv_fcopy_uio_daemon.o +hv_fcopy_uio_daemon-y += vmbus_bufring.o
tools/hv/Makefile +5 -5
@@ -17,7 +17,7 @@ MAKEFLAGS += -r override CFLAGS += -O2 -Wall -g -D_GNU_SOURCE -I$(OUTPUT)include -ALL_TARGETS := hv_kvp_daemon hv_vss_daemon hv_fcopy_daemon +ALL_TARGETS := hv_kvp_daemon hv_vss_daemon hv_fcopy_uio_daemon ALL_PROGRAMS := $(patsubst %,$(OUTPUT)%,$(ALL_TARGETS)) ALL_SCRIPTS := hv_get_dhcp_info.sh hv_get_dns_info.sh hv_set_ifconfig.sh
@@ -39,10 +39,10 @@ $(HV_VSS_DAEMON_IN): FORCE $(OUTPUT)hv_vss_daemon: $(HV_VSS_DAEMON_IN) $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ -HV_FCOPY_DAEMON_IN := $(OUTPUT)hv_fcopy_daemon-in.o -$(HV_FCOPY_DAEMON_IN): FORCE - $(Q)$(MAKE) $(build)=hv_fcopy_daemon -$(OUTPUT)hv_fcopy_daemon: $(HV_FCOPY_DAEMON_IN) +HV_FCOPY_UIO_DAEMON_IN := $(OUTPUT)hv_fcopy_uio_daemon-in.o +$(HV_FCOPY_UIO_DAEMON_IN): FORCE + $(Q)$(MAKE) $(build)=hv_fcopy_uio_daemon +$(OUTPUT)hv_fcopy_uio_daemon: $(HV_FCOPY_UIO_DAEMON_IN) $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ clean:
tools/hv/hv_fcopy_uio_daemon.c new file +489 -0
489 lines not on this page · raw message
Signature · 1 line
2.34.1
Greg KH
maintainerRe: [PATCH v2 3/5] tools: hv: Add new fcopy application based on uio driver
Quoted 1 line · Saurabh Sengar
New fcopy application which utilizes
uio_hv_vmbus_clientdriver
Please read the kernel documentation for how to write a good kernel changelog.
THen get another microsoft kernel developer to review this patchset before you send it out again, it needs some basic rework for simple things like this.
I've stopped reviewing this series here.
thanks,
greg k-h
PATCH v2 4/5 tools: hv: Remove hv_fcopy_daemon
In reply to Saurabh SengarAs the new fcopy application using uio is introduced, remove obsolete hv_fcopy_daemon.c
--- tools/hv/hv_fcopy_daemon.c | 266 ------------------------------------- 1 file changed, 266 deletions(-) delete mode 100644 tools/hv/hv_fcopy_daemon.c
tools/hv/hv_fcopy_daemon.c deleted +0 -266
@@ -1,266 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* -* An implementation of host to guest copy functionality for Linux. -* -* Copyright (C) 2014, Microsoft, Inc. -* -* Author : K. Y. Srinivasan <[email protected]> -*/ - - -#include <sys/types.h> -#include <stdio.h> -#include <stdlib.h> -#include <unistd.h> -#include <string.h> -#include <errno.h> -#include <linux/hyperv.h> -#include <linux/limits.h> -#include <syslog.h> -#include <sys/stat.h> -#include <fcntl.h> -#include <getopt.h> - -static int target_fd; -static char target_fname[PATH_MAX ]; -static unsigned long long filesize; - -static int hv_start_fcopy (struct hv_start_fcopy *smsg) -{ -int error =HV_E_FAIL ; -char *q, *p; - - filesize =0 ; - p = (char *)smsg->path_name; -snprintf (target_fname,sizeof (target_fname),"%s/%s" , - (char *)smsg->path_name, (char *)smsg->file_name); - -syslog (LOG_INFO ,"Target file name: %s" , target_fname); -/* -* Check to see if the path is already in place; if not, -* create if required. -*/ -while ((q =strchr (p,'/' )) !=NULL ) { -if (q == p) { - p++; -continue ; - } - *q ='\0' ; -if (access ((char *)smsg->path_name,F_OK )) { -if (smsg->copy_flags &CREATE_PATH ) { -if (mkdir ((char *)smsg->path_name,0755 )) { -syslog (LOG_ERR ,"Failed to create %s" , - (char *)smsg->path_name); - goto done; - } - }else { -syslog (LOG_ERR ,"Invalid path: %s" , - (char *)smsg->path_name); - goto done; - } - } - p = q +1 ; - *q ='/' ; - } - -if (!access (target_fname,F_OK )) { -syslog (LOG_INFO ,"File: %s exists" , target_fname); -if (!(smsg->copy_flags &OVER_WRITE )) { - error =HV_ERROR_ALREADY_EXISTS ; - goto done; - } - } - - target_fd =open (target_fname, -O_RDWR |O_CREAT |O_TRUNC |O_CLOEXEC ,0744 ); -if (target_fd ==-1 ) { -syslog (LOG_INFO ,"Open Failed: %s" ,strerror (errno)); - goto done; - } - - error =0 ; -done: -if (error) - target_fname[0 ] ='\0' ; -return error; -} - -static int hv_copy_data (struct hv_do_fcopy *cpmsg) -{ -ssize_t bytes_written; -int ret =0 ; - - bytes_written =pwrite (target_fd, cpmsg->data, cpmsg->size, - cpmsg->offset); - - filesize += cpmsg->size; -if (bytes_written != cpmsg->size) { -switch (errno) { -case ENOSPC : - ret =HV_ERROR_DISK_FULL ; -break ; -default : - ret =HV_E_FAIL ; -break ; - } -syslog (LOG_ERR ,"pwrite failed to write %llu bytes: %ld (%s)" , - filesize, (long )bytes_written,strerror (errno)); - } - -return ret; -} - -/* -* Reset target_fname to "" in the two below functions for hibernation: if -* the fcopy operation is aborted by hibernation, the daemon should remove the -* partially-copied file; to achieve this, the hv_utils driver always fakes a -* CANCEL_FCOPY message upon suspend, and later when the VM resumes back, -* the daemon calls hv_copy_cancel() to remove the file; if a file is copied -* successfully before suspend, hv_copy_finished() must reset target_fname to -* avoid that the file can be incorrectly removed upon resume, since the faked -* CANCEL_FCOPY message is spurious in this case. -*/ -static int hv_copy_finished (void ) -{ -close (target_fd); - target_fname[0 ] ='\0' ; -return 0 ; -} -static int hv_copy_cancel (void ) -{ -close (target_fd); -if (strlen (target_fname) >0 ) { -unlink (target_fname); - target_fname[0 ] ='\0' ; - } -return 0 ; - -} - -void print_usage (char *argv[]) -{ -fprintf (stderr,"Usage: %s [options]\n" -"Options are:\n" -" -n, --no-daemon stay in foreground, don't daemonize\n" -" -h, --help print this help\n" , argv[0 ]); -} - -int main (int argc,char *argv[]) -{ -int fcopy_fd =-1 ; -int error; -int daemonize =1 , long_index =0 , opt; -int version =FCOPY_CURRENT_VERSION ; -union { -struct hv_fcopy_hdr hdr; -struct hv_start_fcopy start; -struct hv_do_fcopy copy; -__u32 kernel_modver; - } buffer = { }; -int in_handshake; - -static struct option long_options[] = { - {"help" , no_argument,0 ,'h' }, - {"no-daemon" , no_argument,0 ,'n' }, - {0 ,0 ,0 ,0 } - }; - -while ((opt =getopt_long (argc, argv,"hn" , long_options, - &long_index)) !=-1 ) { -switch (opt) { -case 'n' : - daemonize =0 ; -break ; -case 'h' : -default : -print_usage (argv); -exit (EXIT_FAILURE ); - } - } - -if (daemonize &&daemon (1 ,0 )) { -syslog (LOG_ERR ,"daemon() failed; error: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); - } - -openlog ("HV_FCOPY" ,0 ,LOG_USER ); -syslog (LOG_INFO ,"starting; pid is:%d" ,getpid ()); - -reopen_fcopy_fd: -if (fcopy_fd !=-1 ) -close (fcopy_fd); -/* Remove any possible partially-copied file on error */ -hv_copy_cancel (); - in_handshake =1 ; - fcopy_fd =open ("/dev/vmbus/hv_fcopy" ,O_RDWR ); - -if (fcopy_fd <0 ) { -syslog (LOG_ERR ,"open /dev/vmbus/hv_fcopy failed; error: %d %s" , - errno,strerror (errno)); -exit (EXIT_FAILURE ); - } - -/* -* Register with the kernel. -*/ -if ((write (fcopy_fd, &version,sizeof (int ))) !=sizeof (int )) { -syslog (LOG_ERR ,"Registration failed: %s" ,strerror (errno)); -exit (EXIT_FAILURE ); - } - -while (1 ) { -/* -* In this loop we process fcopy messages after the -* handshake is complete. -*/ -ssize_t len; - - len =pread (fcopy_fd, &buffer,sizeof (buffer),0 ); -if (len <0 ) { -syslog (LOG_ERR ,"pread failed: %s" ,strerror (errno)); - goto reopen_fcopy_fd; - } - -if (in_handshake) { -if (len !=sizeof (buffer.kernel_modver)) { -syslog (LOG_ERR ,"invalid version negotiation" ); -exit (EXIT_FAILURE ); - } - in_handshake =0 ; -syslog (LOG_INFO ,"kernel module version: %u" , - buffer.kernel_modver); -continue ; - } - -switch (buffer.hdr.operation) { -case START_FILE_COPY : - error =hv_start_fcopy (&buffer.start); -break ; -case WRITE_TO_FILE : - error =hv_copy_data (&buffer.copy); -break ; -case COMPLETE_FCOPY : - error =hv_copy_finished (); -break ; -case CANCEL_FCOPY : - error =hv_copy_cancel (); -break ; - -default : - error =HV_E_FAIL ; -syslog (LOG_ERR ,"Unknown operation: %d" , - buffer.hdr.operation); - - } - -/* -* pwrite() may return an error due to the faked CANCEL_FCOPY -* message upon hibernation. Ignore the error by resetting the -* dev file, i.e. closing and re-opening it. -*/ -if (pwrite (fcopy_fd, &error,sizeof (int ),0 ) !=sizeof (int )) { -syslog (LOG_ERR ,"pwrite failed: %s" ,strerror (errno)); - goto reopen_fcopy_fd; - } - } -}
Signature · 1 line
2.34.1
PATCH v2 5/5 Drivers: hv: Remove fcopy driver
In reply to Saurabh SengarAs the new fcopy driver using uio is introduced, remove obsolete driver.
--- drivers/hv/Makefile | 2 +- drivers/hv/hv_fcopy.c | 427 ------------------------------------------ drivers/hv/hv_util.c | 12 -- 3 files changed, 1 insertion(+), 440 deletions(-) delete mode 100644 drivers/hv/hv_fcopy.c
drivers/hv/Makefile +1 -1
@@ -10,7 +10,7 @@ hv_vmbus-y := vmbus_drv.o \ hv.o connection.o channel.o \ channel_mgmt.o ring_buffer.o hv_trace.o hv_vmbus-$(CONFIG_HYPERV_TESTING) += hv_debugfs.o -hv_utils-y := hv_util.o hv_kvp.o hv_snapshot.o hv_fcopy.o hv_utils_transport.o +hv_utils-y := hv_util.o hv_kvp.o hv_snapshot.o hv_utils_transport.o # Code that must be built-in obj-$(subst m,y,$(CONFIG_HYPERV)) += hv_common.o
drivers/hv/hv_fcopy.c deleted +0 -427
427 lines not on this page · raw message
drivers/hv/hv_util.c +0 -12
@@ -154,14 +154,6 @@ static struct hv_util_service util_vss = { .util_deinit = hv_vss_deinit, }; -static struct hv_util_service util_fcopy = { - .util_cb = hv_fcopy_onchannelcallback, - .util_init = hv_fcopy_init, - .util_pre_suspend = hv_fcopy_pre_suspend, - .util_pre_resume = hv_fcopy_pre_resume, - .util_deinit = hv_fcopy_deinit, -}; -static void perform_shutdown (struct work_struct *dummy) {orderly_poweroff (true);
@@ -671,10 +663,6 @@ static const struct hv_vmbus_device_id id_table[] = { { HV_VSS_GUID, .driver_data = (unsigned long )&util_vss }, - /* File copy GUID */ - { HV_FCOPY_GUID, - .driver_data = (unsigned long )&util_fcopy - }, { }, };
Signature · 1 line
2.34.1