Add multiple address spaces support to VDUSE
Subsystems (4)
Eugenio Pérez
maintainerv12 cover Add multiple address spaces support to VDUSE
When used by vhost-vDPA bus driver for VM, the control virtqueue should be shadowed via userspace VMM (QEMU) instead of being assigned directly to Guest. This is because QEMU needs to know the device state in order to start and stop device correctly (e.g for Live Migration).
This requies to isolate the memory mapping for control virtqueue presented by vhost-vDPA to prevent guest from accessing it directly.
This series add support to multiple address spaces in VDUSE device allowing selective virtqueue isolation through address space IDs (ASID).
The VDUSE device needs to report:
* Number of virtqueue groups
* Association of each vq group with each virtqueue
* Number of address spaces supported.
Then, the vDPA driver can modify the ASID assigned to each VQ group to isolate the memory AS. This aligns VDUSE with vdpa_sim and nvidia mlx5 devices which already support ASID.
This helps to isolate the environments for the virtqueues that will not be assigned directly. E.g in the case of virtio-net, the control virtqueue will not be assigned directly to guest.
Also, to be able to test this patch, the user needs to manually revert the 56e71885b0349 ("vduse: Temporarily fail if control queue feature requested") commit.
Tested by creating a VDUSE device OVS with and without MQ, and live migrating between two hosts back and forth while maintaining ping alive in all the stages. All tested with and without lockdep, with the old VDUSE_IOTLB_GET_FD and the new VDUSE_IOTLB_GET_FD2, and with the VDUSE API 0 and 1.
A few workaround were needed in some parts to test live migration with vhost_vdpa:
* Do not enable CVQ before data vqs in QEMU, as VDUSE does not forward the enable message to the userland device. This will be solved in the future.
* Share the suspended state between all vhost devices in QEMU:
https://lists.nongnu.org/archive/html/qemu-devel/2025-11/msg02947.html
* Implement a fake VDUSE suspend vdpa operation callback that always returns true in the kernel. DPDK suspend the device at the first GET_VRING_BASE.
* Remove the CVQ blocker in ASID.
The driver vhost_vdpa was also tested with version 0, version 1 with the old ioctl, version 1 with the new ioctl but only one ASID, and version 1 with many ASID.
Also tested with virtio_vdpa with just one ASID, and by creating two threads that maps and unmaps memory randomly while ASID is changed [1].
[1] https://lore.kernel.org/lkml/CAJaqyWeDjBfzrA93riYTmefY4oLbctWWBMPcpR5Q6r50dUGOEw@mail.gmail.com
PATCH v12:
* Add API version 1 usage in Documentation/userspace-api/vduse.rst.
* Assume API version 0 by default if the VDUSE instance does not ask (Jason).
* Using scoped guards for vq group rwlock, so the one queue optimization is not missed (Jason proposed to factor them into helpers).
* Add the _v2 suffix to vduse_iova_range_v2 struct name fixing the doc (MST).
* Avoid dereferencing map_file if not assigned in vduse_dev_iotlb_entry function (MST).
* Avoid free_pages_exact(NULL, size) in case vduse_domain_alloc_coherent fails (MST).
* Add Fixes: tag (MST).
* s/verion/version/;s/accomodated/accommodated;s/behavious/behaviour in patch messages, and more rewording of them (MST).
* Remove trailing ; after a comment (Jason).
* Change the style of checking for vq group == 0 in VDUSE_VQ_SETUP dev ioctl if api_version < 1 (MST).
PATCH v11:
* Remove duplicated free_pages_exact in vduse_domain_free_coherent (Jason).
* Move the vduse_iotlb_entry_v2 argument to a new ioctl, as argument didn't match the previous VDUSE_IOTLB_GET_FD.
* Do not reset the vq group ASID in vq reset (Jason). Removed extra function vduse_set_group_asid_nomsg, not needed anymore.
* Do not take the vq groups lock if nas == 1.
* Move the asid < dev->nas check to vdpa core.
* Rename vq->vq_group to vq->group (Jason).
* Do not reset vq group at virtio reset (Jason).
PATCH v10:
* Back to rwlock version so stronger locks are used.
* Take out allocations from rwlock.
* Forbid changing ASID of a vq group after DRIVER_OK (Jason)
* Remove bad fetching again of domain variable in vduse_dev_max_mapping_size (Yongji).
* Remove unused vdev definition in vdpa map_ops callbacks (kernel test robot).
PATCH v9:
* Change to RCU.
PATCH v8:
* Revert the change from mutex to rwlock (MST).
PATCH v7:
* Fix not taking the write lock in the registering vdpa device error path (Jason).
PATCH v6:
* Make vdpa_dev_add use gotos for error handling (MST).
* s/(dev->api_version < 1) ?/(dev->api_version < VDUSE_API_VERSION_1) ?/ in group and nas handling at device creation (MST).
* Fix struct name not matching in the doc.
* s/sepparate/separate (MST).
PATCH v5:
* Properly return errno if copy_to_user returns >0 in VDUSE_IOTLB_GET_FD ioctl (Jason).
* Properly set domain bounce size to divide equally between nas (Jason).
* Revert core vdpa changes (Jason).
* Fix group == ngroup case in checking VQ_SETUP argument (Jason).
* Exclude "padding" member from the only >V1 members in vduse_dev_request.
PATCH v4:
* Consider config->nas == 0 and config->ngroups == 0 as a fail (Jason).
* Revert the "invalid vq group" concept and assume 0 if not set.
* Divide each domain bounce size between the device bounce size (Jason).
* Revert unneeded addr = NULL assignment (Jason)
* Change if (x && (y || z)) return to if (x) { if (y) return; if (z) return; } (Jason)
* Change a bad multiline comment, using @ caracter instead of * (Jason).
PATCH v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason). Adapt all the virtio_map_ops callbacks to it.
* Make setting status DRIVER_OK fail if vq group is not valid.
* Create the VDUSE_DEV_MAX_GROUPS and VDUSE_DEV_MAX_AS instead of using a magic number
* Remove the _int name suffix from struct vduse_vq_group.
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash the patch implementing the AS logic with the patch creating the vduse_as struct (Jason).
PATCH v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.
* Convert the use of mutex to rwlock (Xie Yongji).
PATCH v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)
* Using vduse_vq_group_int directly instead of an empty struct in union virtio_map.
RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs. Also, it's a dynamic array now. Same with ASID.
* Move the valid vq groups range check to vduse_validate_config.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Use of array_index_nospec in VDUSE device ioctls.
* Move the umem mutex to asid struct so there is no contention between ASIDs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.
* Fixes detected testing with OVS+VDUSE.
Eugenio Pérez (13):
vhost: move vdpa group bound check to vhost_vdpa
vduse: add v1 API definition
vduse: add vq group support
vduse: return internal vq group struct as map token
vdpa: document set_group_asid thread safety
vhost: forbid change vq groups ASID if DRIVER_OK is set
vduse: refactor vdpa_dev_add for goto err handling
vduse: remove unused vaddr parameter of vduse_domain_free_coherent
vduse: take out allocations from vduse_dev_alloc_coherent
vduse: merge tree search logic of IOTLB_GET_FD and IOTLB_GET_INFO
ioctls
vduse: add vq group asid support
vduse: bump version number
Documentation: Add documentation for VDUSE Address Space IDs
Documentation/userspace-api/vduse.rst | 49 +++ drivers/vdpa/mlx5/net/mlx5_vnet.c | 3 - drivers/vdpa/vdpa_sim/vdpa_sim.c | 6 - drivers/vdpa/vdpa_user/iova_domain.c | 27 +- drivers/vdpa/vdpa_user/iova_domain.h | 8 +- drivers/vdpa/vdpa_user/vduse_dev.c | 540 +++++++++++++++++++------- drivers/vhost/vdpa.c | 4 +- include/linux/vdpa.h | 4 +- include/linux/virtio.h | 6 +- include/uapi/linux/vduse.h | 77 +++- 10 files changed, 547 insertions(+), 177 deletions(-)
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 1/13 vhost: move vdpa group bound check to vhost_vdpa
Remove duplication by consolidating these here. This reduces the posibility of a parent driver missing them.
While we're at it, fix a bug in vdpa_sim where a valid ASID can be assigned to a group equal to ngroups, causing an out of bound write.
---
v12: Add Fixes tag (MST).
--- drivers/vdpa/mlx5/net/mlx5_vnet.c | 3 --- drivers/vdpa/vdpa_sim/vdpa_sim.c | 6 ------ drivers/vhost/vdpa.c | 2 +- 3 files changed, 1 insertion(+), 10 deletions(-)
drivers/vdpa/mlx5/net/mlx5_vnet.c +0 -3
@@ -3640,9 +3640,6 @@ static int mlx5_set_group_asid(struct vdpa_device *vdev, u32 group,struct mlx5_vdpa_dev *mvdev =to_mvdev (vdev);int err =0 ; -if (group >=MLX5_VDPA_NUMVQ_GROUPS ) -return -EINVAL ; - mvdev->mres.group2asid[group] = asid;mutex_lock (&mvdev->mres.lock);
drivers/vdpa/vdpa_sim/vdpa_sim.c +0 -6
@@ -606,12 +606,6 @@ static int vdpasim_set_group_asid(struct vdpa_device *vdpa, unsigned int group,struct vhost_iotlb *iommu;int i; -if (group > vdpasim->dev_attr.ngroups) -return -EINVAL ; - -if (asid >= vdpasim->dev_attr.nas) -return -EINVAL ; - iommu = &vdpasim->iommu[asid];mutex_lock (&vdpasim->mutex);
drivers/vhost/vdpa.c +1 -1
@@ -680,7 +680,7 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,case VHOST_VDPA_SET_GROUP_ASID :if (copy_from_user (&s, argp,sizeof (s)))return -EFAULT ; -if (s.num >= vdpa->nas) +if (idx >= vdpa->ngroups || s.num >= vdpa->nas)return -EINVAL ;if (!ops->set_group_asid)return -EOPNOTSUPP ;
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 2/13 vduse: add v1 API definition
In reply to Eugenio PérezThis allows the kernel to detect whether the userspace VDUSE device supports the VQ group and ASID features. VDUSE devices that don't set the V1 API will not receive the new messages, and vdpa device will be created with only one vq group and asid.
The next patches implement the new feature incrementally, only enabling the VDUSE device to set the V1 API version by the end of the series.
--- include/uapi/linux/vduse.h | 4 ++++ 1 file changed, 4 insertions(+)
include/uapi/linux/vduse.h +4 -0
@@ -10,6 +10,10 @@#define VDUSE_API_VERSION 0 +/* VQ groups and ASID support */ + +#define VDUSE_API_VERSION_1 1 +/* * Get the version of VDUSE API that kernel supported (VDUSE_API_VERSION). * This is used for future extension.
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 3/13 vduse: add vq group support
In reply to Eugenio PérezThis allows separate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API.
Allocating 3 vq groups as net is the device that need the most groups:
* Dataplane (guest passthrough)
* CVQ
* Shadowed vrings.
Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups.
---
v12:
* Change the style of checking for vq group == 0 in VDUSE_VQ_SETUP dev ioctl if api_version < 1 (MST).
v11:
* Rename vq->vq_group to vq->group (Jason).
* Do not reset vq group at virtio reset (Jason).
v6:
* s/sepparate/separate (MST).
* s/dev->api_version < 1/dev->api_version < VDUSE_API_VERSION_1
v5:
* Revert core vdpa changes (Jason).
* Fix group == ngroup case in checking VQ_SETUP argument (Jason).
v4:
* Revert the "invalid vq group" concept and assume 0 if not set (Jason).
* Make config->ngroups == 0 invalid (Jason).
v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason).
* Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number
v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.
v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)
RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Move the valid vq groups range check to vduse_validate_config.
RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.
--- drivers/vdpa/vdpa_user/vduse_dev.c | 47 ++++++++++++++++++++++++++---- include/uapi/linux/vduse.h | 12 ++++++-- 2 files changed, 51 insertions(+), 8 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +42 -5
@@ -39,6 +39,7 @@#define DRV_LICENSE "GPL v2"#define VDUSE_DEV_MAX (1U << MINORBITS) +#define VDUSE_DEV_MAX_GROUPS 0xffff#define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)#define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)#define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -58,6 +59,7 @@ struct vduse_virtqueue {struct vdpa_vq_state state;bool ready;bool kicked; +u32 group;spinlock_t kick_lock;spinlock_t irq_lock;struct eventfd_ctx *kickfd;
@@ -114,6 +116,7 @@ struct vduse_dev {u8 status;u32 vq_num;u32 vq_align; +u32 ngroups;struct vduse_umem *umem;struct mutex mem_lock;unsigned int bounce_size;
@@ -592,6 +595,16 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 idx,return 0 ; } +static u32 vduse_get_vq_group (struct vdpa_device *vdpa,u16 idx) +{ +struct vduse_dev *dev =vdpa_to_vduse (vdpa); + +if (dev->api_version <VDUSE_API_VERSION_1 ) +return 0 ; + +return dev->vqs[idx]->group; +} +static int vduse_vdpa_get_vq_state (struct vdpa_device *vdpa,u16 idx,struct vdpa_vq_state *state) {
@@ -789,6 +802,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .set_vq_cb = vduse_vdpa_set_vq_cb, .set_vq_num = vduse_vdpa_set_vq_num, .get_vq_size = vduse_vdpa_get_vq_size, + .get_vq_group = vduse_get_vq_group, .set_vq_ready = vduse_vdpa_set_vq_ready, .get_vq_ready = vduse_vdpa_get_vq_ready, .set_vq_state = vduse_vdpa_set_vq_state,
@@ -1252,12 +1266,24 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,if (config.index >= dev->vq_num)break ; -if (!is_mem_zero ((const char *)config.reserved, -sizeof (config.reserved))) +if (dev->api_version <VDUSE_API_VERSION_1 ) { +if (config.group) +break ; + }else { +if (config.group >= dev->ngroups) +break ; +if (dev->status &VIRTIO_CONFIG_S_DRIVER_OK ) +break ; + } + +if (config.reserved1 || + !is_mem_zero ((const char *)config.reserved2, +sizeof (config.reserved2)))break ; index =array_index_nospec (config.index, dev->vq_num); dev->vqs[index]->num_max = config.max_size; + dev->vqs[index]->group = config.group; ret =0 ;break ; }
@@ -1737,12 +1763,20 @@ static bool features_is_valid(struct vduse_dev_config *config)return true; } -static bool vduse_validate_config (struct vduse_dev_config *config) +static bool vduse_validate_config (struct vduse_dev_config *config, +u64 api_version) {if (!is_mem_zero ((const char *)config->reserved,sizeof (config->reserved)))return false; +if (api_version <VDUSE_API_VERSION_1 && config->ngroups) +return false; + +if (api_version >=VDUSE_API_VERSION_1 && + (!config->ngroups || config->ngroups >VDUSE_DEV_MAX_GROUPS )) +return false; +if (config->vq_align >PAGE_SIZE )return false;
@@ -1858,6 +1892,9 @@ static int vduse_create_dev(struct vduse_dev_config *config, dev->device_features = config->features; dev->device_id = config->device_id; dev->vendor_id = config->vendor_id; + dev->ngroups = (dev->api_version <VDUSE_API_VERSION_1 ) + ?1 + : config->ngroups; dev->name =kstrdup (config->name,GFP_KERNEL );if (!dev->name) goto err_str;
@@ -1936,7 +1973,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,break ; ret = -EINVAL ; -if (vduse_validate_config (&config) == false) +if (!vduse_validate_config (&config, control->api_version))break ; buf =vmemdup_user (argp + size, config.config_size);
@@ -2017,7 +2054,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name) vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev, &vduse_vdpa_config_ops, &vduse_map_ops, -1 ,1 , name, true); + dev->ngroups,1 , name, true);if (IS_ERR (vdev))return PTR_ERR (vdev);
include/uapi/linux/vduse.h +9 -3
@@ -31,6 +31,7 @@* @features: virtio features * @vq_num: the number of virtqueues * @vq_align: the allocation alignment of virtqueue's metadata +* @ngroups: number of vq groups that VDUSE device declares * @reserved: for future use, needs to be initialized to zero * @config_size: the size of the configuration space * @config: the buffer of the configuration space
@@ -45,7 +46,8 @@ struct vduse_dev_config {__u64 features;__u32 vq_num;__u32 vq_align; -__u32 reserved[13 ]; +__u32 ngroups;/* if VDUSE_API_VERSION >= 1 */ +__u32 reserved[12 ];__u32 config_size;__u8 config[]; };
@@ -122,14 +124,18 @@ struct vduse_config_data {* struct vduse_vq_config - basic configuration of a virtqueue * @index: virtqueue index * @max_size: the max size of virtqueue -* @reserved: for future use, needs to be initialized to zero +* @reserved1: for future use, needs to be initialized to zero +* @group: virtqueue group +* @reserved2: for future use, needs to be initialized to zero * * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue. */ struct vduse_vq_config {__u32 index;__u16 max_size; -__u16 reserved[13 ]; +__u16 reserved1; +__u32 group; +__u16 reserved2[10 ]; };/*
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 4/13 vduse: return internal vq group struct as map token
In reply to Eugenio PérezReturn the internal struct that represents the vq group as virtqueue map token, instead of the device. This allows the map functions to access the information per group.
At this moment all the virtqueues share the same vq group, that only can point to ASID 0. This change prepares the infrastructure for actual per-group address space handling
---
v4:
* Revert the "invalid vq group" concept, and assume 0 by default.
* Revert unnecesary blank line addition (Jason)
v3:
* Adapt all virtio_map_ops callbacks to handle empty tokens in case of invalid groups.
* Make setting status DRIVER_OK fail if vq group is not valid.
* Remove the _int name suffix from struct vduse_vq_group.
RFC v3:
* Make the vq groups a dynamic array to support an arbitrary number of them.
--- drivers/vdpa/vdpa_user/vduse_dev.c | 100 ++++++++++++++++++++++++++--- include/linux/virtio.h | 6 +- 2 files changed, 94 insertions(+), 12 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +91 -9
@@ -22,6 +22,7 @@#include <linux/uio.h> #include <linux/vdpa.h> #include <linux/nospec.h> +#include <linux/virtio.h> #include <linux/vmalloc.h> #include <linux/sched/mm.h> #include <uapi/linux/vduse.h>
@@ -85,6 +86,10 @@ struct vduse_umem {struct mm_struct *mm; }; +struct vduse_vq_group { +struct vduse_dev *dev; +}; +struct vduse_dev {struct vduse_vdpa *vdev;struct device *dev;
@@ -118,6 +123,7 @@ struct vduse_dev {u32 vq_align;u32 ngroups;struct vduse_umem *umem; +struct vduse_vq_group *groups;struct mutex mem_lock;unsigned int bounce_size;struct mutex domain_lock;
@@ -605,6 +611,17 @@ static u32 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)return dev->vqs[idx]->group; } +static union virtio_map vduse_get_vq_map (struct vdpa_device *vdpa,u16 idx) +{ +struct vduse_dev *dev =vdpa_to_vduse (vdpa); +u32 vq_group =vduse_get_vq_group (vdpa, idx); +union virtio_map ret = { + .group = &dev->groups[vq_group], + }; + +return ret; +} +static int vduse_vdpa_get_vq_state (struct vdpa_device *vdpa,u16 idx,struct vdpa_vq_state *state) {
@@ -825,6 +842,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, .set_map = vduse_vdpa_set_map, + .get_vq_map = vduse_get_vq_map, .free = vduse_vdpa_free, };
@@ -832,7 +850,14 @@ static void vduse_dev_sync_single_for_device(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return ; + + vdev = token.group->dev; + domain = vdev->domain;vduse_domain_sync_single_for_device (domain, dma_addr, size, dir); }
@@ -841,7 +866,14 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return ; + + vdev = token.group->dev; + domain = vdev->domain;vduse_domain_sync_single_for_cpu (domain, dma_addr, size, dir); }
@@ -851,7 +883,14 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page, enum dma_data_direction dir, unsigned long attrs) { -struct vduse_iova_domain *domain = token.iova_domain; +struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return DMA_MAPPING_ERROR ; + + vdev = token.group->dev; + domain = vdev->domain;return vduse_domain_map_page (domain, page, offset, size, dir, attrs); }
@@ -860,7 +899,14 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return ; + + vdev = token.group->dev; + domain = vdev->domain;return vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); }
@@ -868,11 +914,17 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,static void *vduse_dev_alloc_coherent (union virtio_map token,size_t size,dma_addr_t *dma_addr,gfp_t flag) { -struct vduse_iova_domain *domain = token.iova_domain; +struct vduse_dev *vdev; +struct vduse_iova_domain *domain;unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ; +if (!token.group) +return NULL ; + + vdev = token.group->dev; + domain = vdev->domain; addr =vduse_domain_alloc_coherent (domain, size, (dma_addr_t *)&iova, flag);if (!addr)
@@ -887,14 +939,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, void *vaddr, dma_addr_t dma_addr, unsigned long attrs) { - struct vduse_iova_domain *domain = token.iova_domain; + struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return ; + + vdev = token.group->dev; + domain = vdev->domain;vduse_domain_free_coherent (domain, size, vaddr, dma_addr, attrs); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) { -struct vduse_iova_domain *domain = token.iova_domain; +struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return false; + + vdev = token.group->dev; + domain = vdev->domain;return dma_addr < domain->bounce_size; }
@@ -908,7 +974,14 @@ static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)static size_t vduse_dev_max_mapping_size (union virtio_map token) { -struct vduse_iova_domain *domain = token.iova_domain; +struct vduse_dev *vdev; +struct vduse_iova_domain *domain; + +if (!token.group) +return 0 ; + + vdev = token.group->dev; + domain = vdev->domain;return domain->bounce_size; }
@@ -1726,6 +1799,7 @@ static int vduse_destroy_dev(char *name)if (dev->domain)vduse_domain_destroy (dev->domain);kfree (dev->name); +kfree (dev->groups);vduse_dev_destroy (dev);module_put (THIS_MODULE );
@@ -1895,6 +1969,13 @@ static int vduse_create_dev(struct vduse_dev_config *config, dev->ngroups = (dev->api_version <VDUSE_API_VERSION_1 ) ?1 : config->ngroups; + dev->groups =kcalloc (dev->ngroups,sizeof (dev->groups[0 ]), +GFP_KERNEL ); +if (!dev->groups) + goto err_vq_groups; +for (u32 i =0 ; i < dev->ngroups; ++i) + dev->groups[i].dev = dev; + dev->name =kstrdup (config->name,GFP_KERNEL );if (!dev->name) goto err_str;
@@ -1931,6 +2012,8 @@ static int vduse_create_dev(struct vduse_dev_config *config, err_idr:kfree (dev->name); err_str: +kfree (dev->groups); +err_vq_groups:vduse_dev_destroy (dev); err:return ret;
@@ -2092,7 +2175,6 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,return -ENOMEM ; } - dev->vdev->vdpa.vmap.iova_domain = dev->domain; ret =_vdpa_register_device (&dev->vdev->vdpa, dev->vq_num);if (ret) {put_device (&dev->vdev->vdpa.dev);
include/linux/virtio.h +3 -3
@@ -43,13 +43,13 @@ struct virtqueue {void *priv; }; -struct vduse_iova_domain ; +struct vduse_vq_group ;union virtio_map {/* Device that performs DMA */ struct device *dma_dev; -/* VDUSE specific mapping data */ -struct vduse_iova_domain *iova_domain; +/* VDUSE specific virtqueue group for doing map */ +struct vduse_vq_group *group; }; int virtqueue_add_outbuf(struct virtqueue *vq,
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 5/13 vdpa: document set_group_asid thread safety
In reply to Eugenio PérezDocument that the function races with the check of DRIVER_OK.
---
Requested at
https://lore.kernel.org/lkml/CACGkMEvXdV4ukZE6xhLL0sSN70G=AWVQgpRnH98Fr4btzMkK9g@mail.gmail.com/
v12: Reword comment (MST).
--- include/linux/vdpa.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/vdpa.h +3 -1
@@ -312,7 +312,9 @@ struct vdpa_map_file {* @idx: virtqueue index * Returns the affinity mask * @set_group_asid: Set address space identifier for a -* virtqueue group (optional) +* virtqueue group (optional). Caller must +* prevent this from being executed concurrently +* with set_status. * @vdev: vdpa device * @group: virtqueue group * @asid: address space id for this group
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 6/13 vhost: forbid change vq groups ASID if DRIVER_OK is set
In reply to Eugenio PérezOnly vdpa_sim support it. Forbid this behaviour as there is no use for it right now, we can always enable it in the future with a feature flag.
---
v12: Fix typo in patch message (MST).
--- drivers/vhost/vdpa.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/vhost/vdpa.c +2 -0
@@ -682,6 +682,8 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,return -EFAULT ;if (idx >= vdpa->ngroups || s.num >= vdpa->nas)return -EINVAL ; +if (ops->get_status (vdpa) &VIRTIO_CONFIG_S_DRIVER_OK ) +return -EBUSY ;if (!ops->set_group_asid)return -EOPNOTSUPP ;return ops->set_group_asid (vdpa, idx, s.num);
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 7/13 vduse: refactor vdpa_dev_add for goto err handling
In reply to Eugenio PérezNext patches introduce more error paths in this function. Refactor it so they can be accommodated through gotos.
---
v12: Fix typo in patch message (MST).
v6: New in v6.
--- drivers/vdpa/vdpa_user/vduse_dev.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +14 -8
@@ -2171,21 +2171,27 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, dev->bounce_size);mutex_unlock (&dev->domain_lock);if (!dev->domain) { -put_device (&dev->vdev->vdpa.dev); -return -ENOMEM ; + ret = -ENOMEM ; + goto domain_err; } ret =_vdpa_register_device (&dev->vdev->vdpa, dev->vq_num);if (ret) { -put_device (&dev->vdev->vdpa.dev); -mutex_lock (&dev->domain_lock); -vduse_domain_destroy (dev->domain); - dev->domain =NULL ; -mutex_unlock (&dev->domain_lock); -return ret; + goto register_err; }return 0 ; + +register_err: +mutex_lock (&dev->domain_lock); +vduse_domain_destroy (dev->domain); + dev->domain =NULL ; +mutex_unlock (&dev->domain_lock); + +domain_err: +put_device (&dev->vdev->vdpa.dev); + +return ret; }static void vdpa_dev_del (struct vdpa_mgmt_dev *mdev,struct vdpa_device *dev)
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 8/13 vduse: remove unused vaddr parameter of vduse_domain_free_coherent
In reply to Eugenio PérezWe will modify the function in next patches so let's clean it first.
--- drivers/vdpa/vdpa_user/iova_domain.c | 3 +-- drivers/vdpa/vdpa_user/iova_domain.h | 3 +-- drivers/vdpa/vdpa_user/vduse_dev.c | 2 +- 3 files changed, 3 insertions(+), 5 deletions(-)
drivers/vdpa/vdpa_user/iova_domain.c +1 -2
@@ -528,8 +528,7 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size, -void *vaddr,dma_addr_t dma_addr, -unsigned long attrs) +dma_addr_t dma_addr,unsigned long attrs) {struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;
drivers/vdpa/vdpa_user/iova_domain.h +1 -2
@@ -70,8 +70,7 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, gfp_t flag);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size, -void *vaddr,dma_addr_t dma_addr, -unsigned long attrs); +dma_addr_t dma_addr,unsigned long attrs);void vduse_domain_reset_bounce_map (struct vduse_iova_domain *domain);
drivers/vdpa/vdpa_user/vduse_dev.c +1 -1
@@ -948,7 +948,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, vdev = token.group->dev; domain = vdev->domain; -vduse_domain_free_coherent (domain, size, vaddr, dma_addr, attrs); +vduse_domain_free_coherent (domain, size, dma_addr, attrs); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr)
Signature · 1 line
2.52.0
Eugenio Pérez
maintainerPATCH v12 9/13 vduse: take out allocations from vduse_dev_alloc_coherent
In reply to Eugenio PérezThe function vduse_dev_alloc_coherent will be called under rwlock in next patches. Make it out of the lock to avoid increasing its fail rate.
---
v12:
* Avoid free_pages_exact(NULL, size) in case vduse_domain_alloc_coherent fails (MST).
v11: Remove duplicated call to free_pages_exact (Jason).
--- drivers/vdpa/vdpa_user/iova_domain.c | 24 +++++++----------------- drivers/vdpa/vdpa_user/iova_domain.h | 5 ++--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 26 deletions(-)
drivers/vdpa/vdpa_user/iova_domain.c +7 -17
@@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain,vduse_domain_free_iova (iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig) {struct iova_domain *iovad = &domain->consistent_iovad;unsigned long limit = domain->iova_limit;dma_addr_t iova =vduse_domain_alloc_iova (iovad, size, limit); -void *orig =alloc_pages_exact (size, flag); -if (!iova || !orig) - goto err; +if (!iova) +return DMA_MAPPING_ERROR ;spin_lock (&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1,
@@ -514,17 +512,12 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }spin_unlock (&domain->iotlb_lock); - *dma_addr = iova; +return iova; -return orig; err: - *dma_addr =DMA_MAPPING_ERROR ; -if (orig) -free_pages_exact (orig, size); -if (iova) -vduse_domain_free_iova (iovad, iova, size); +vduse_domain_free_iova (iovad, iova, size); -return NULL ; +return DMA_MAPPING_ERROR ; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,
@@ -533,7 +526,6 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;struct vdpa_map_file *map_file; -phys_addr_t pa;spin_lock (&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr,
@@ -545,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque;fput (map_file->file);kfree (map_file); - pa = map->addr;vhost_iotlb_map_free (domain->iotlb, map);spin_unlock (&domain->iotlb_lock);vduse_domain_free_iova (iovad, dma_addr, size); -free_pages_exact (phys_to_virt (pa), size); }static vm_fault_t vduse_domain_mmap_fault (struct vm_fault *vmf)
drivers/vdpa/vdpa_user/iova_domain.h +2 -3
@@ -65,9 +65,8 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size,dma_addr_t dma_addr,unsigned long attrs);
drivers/vdpa/vdpa_user/vduse_dev.c +11 -6
@@ -916,23 +916,27 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, {struct vduse_dev *vdev;struct vduse_iova_domain *domain; -unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ;if (!token.group)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - addr =vduse_domain_alloc_coherent (domain, size, - (dma_addr_t *)&iova, flag); + addr =alloc_pages_exact (size, flag);if (!addr)return NULL ; - *dma_addr = (dma_addr_t )iova; + vdev = token.group->dev; + domain = vdev->domain; + *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err;return addr; + +err: +free_pages_exact (addr, size); +return NULL ; } static void vduse_dev_free_coherent(union virtio_map token, size_t size,
@@ -949,6 +953,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, domain = vdev->domain;vduse_domain_free_coherent (domain, size, dma_addr, attrs); +free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr)
Signature · 1 line
2.52.0
Re: [PATCH v12 09/13] vduse: take out allocations from vduse_dev_alloc_coherent
Quoted 128 lines · Eugenio Pérez
The function
vduse_dev_alloc_coherentwill be called under rwlock in next patches. Make it out of the lock to avoid increasing its fail rate.Signed-off-by Eugenio Pérez <[email protected]>---
v12:
* Avoidfree_pages_exact(NULL, size)in casevduse_domain_alloc_coherentfails (MST).v11: Remove duplicated call to
free_pages_exact(Jason).--- drivers/vdpa/vdpa_user/iova_domain.c | 24 +++++++----------------- drivers/vdpa/vdpa_user/iova_domain.h | 5 ++--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 26 deletions(-)drivers/vdpa/vdpa_user/iova_domain.c +7
-17@@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain,vduse_domain_free_iova (iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig) {struct iova_domain *iovad = &domain->consistent_iovad;unsigned long limit = domain->iova_limit;dma_addr_t iova =vduse_domain_alloc_iova (iovad, size, limit); -void *orig =alloc_pages_exact (size, flag); -if (!iova || !orig) - goto err; +if (!iova) +return DMA_MAPPING_ERROR ;spin_lock (&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1,@@ -514,17 +512,12 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }spin_unlock (&domain->iotlb_lock); - *dma_addr = iova; +return iova; -return orig; err: - *dma_addr =DMA_MAPPING_ERROR ; -if (orig) -free_pages_exact (orig, size); -if (iova) -vduse_domain_free_iova (iovad, iova, size); +vduse_domain_free_iova (iovad, iova, size); -return NULL ; +return DMA_MAPPING_ERROR ; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,@@ -533,7 +526,6 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;struct vdpa_map_file *map_file; -phys_addr_t pa;spin_lock (&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr,@@ -545,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque;fput (map_file->file);kfree (map_file); - pa = map->addr;vhost_iotlb_map_free (domain->iotlb, map);spin_unlock (&domain->iotlb_lock);vduse_domain_free_iova (iovad, dma_addr, size); -free_pages_exact (phys_to_virt (pa), size); }static vm_fault_t vduse_domain_mmap_fault (struct vm_fault *vmf)drivers/vdpa/vdpa_user/iova_domain.h +2
-3@@ -65,9 +65,8 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size,dma_addr_t dma_addr,unsigned long attrs);drivers/vdpa/vdpa_user/vduse_dev.c +3
-6@@ -916,23 +916,27 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, {struct vduse_dev *vdev;struct vduse_iova_domain *domain; -unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ;if (!token.group)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - addr =vduse_domain_alloc_coherent (domain, size, - (dma_addr_t *)&iova, flag); + addr =alloc_pages_exact (size, flag);if (!addr)return NULL ; - *dma_addr = (dma_addr_t )iova; + vdev = token.group->dev; + domain = vdev->domain;
drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 940
Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +7
-0+ *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err;return addr; + +err: +free_pages_exact (addr, size); +return NULL ; }
I think the code tries to hide the iova domain specific things (e.g mappings and DMA_MAPPING_ERROR) from vduse core. Could we find a way to stick to that?
Thanks
Quoted 13 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1
-1static void vduse_dev_free_coherent(union virtio_map token, size_t size,@@ -949,6 +953,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, domain = vdev->domain;vduse_domain_free_coherent (domain, size, dma_addr, attrs); +free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) --2.52.0
Michael S. Tsirkin
maintainerRe: [PATCH v12 09/13] vduse: take out allocations from vduse_dev_alloc_coherent
Quoted 144 lines · Jason Wang
Quoted 128 lines · Eugenio Pérez
The function
vduse_dev_alloc_coherentwill be called under rwlock in next patches. Make it out of the lock to avoid increasing its fail rate.Signed-off-by Eugenio Pérez <[email protected]>---
v12:
* Avoidfree_pages_exact(NULL, size)in casevduse_domain_alloc_coherentfails (MST).v11: Remove duplicated call to
free_pages_exact(Jason).--- drivers/vdpa/vdpa_user/iova_domain.c | 24 +++++++----------------- drivers/vdpa/vdpa_user/iova_domain.h | 5 ++--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 26 deletions(-)drivers/vdpa/vdpa_user/iova_domain.c +7
-17@@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain,vduse_domain_free_iova (iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig) {struct iova_domain *iovad = &domain->consistent_iovad;unsigned long limit = domain->iova_limit;dma_addr_t iova =vduse_domain_alloc_iova (iovad, size, limit); -void *orig =alloc_pages_exact (size, flag); -if (!iova || !orig) - goto err; +if (!iova) +return DMA_MAPPING_ERROR ;spin_lock (&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1,@@ -514,17 +512,12 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }spin_unlock (&domain->iotlb_lock); - *dma_addr = iova; +return iova; -return orig; err: - *dma_addr =DMA_MAPPING_ERROR ; -if (orig) -free_pages_exact (orig, size); -if (iova) -vduse_domain_free_iova (iovad, iova, size); +vduse_domain_free_iova (iovad, iova, size); -return NULL ; +return DMA_MAPPING_ERROR ; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,@@ -533,7 +526,6 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;struct vdpa_map_file *map_file; -phys_addr_t pa;spin_lock (&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr,@@ -545,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque;fput (map_file->file);kfree (map_file); - pa = map->addr;vhost_iotlb_map_free (domain->iotlb, map);spin_unlock (&domain->iotlb_lock);vduse_domain_free_iova (iovad, dma_addr, size); -free_pages_exact (phys_to_virt (pa), size); }static vm_fault_t vduse_domain_mmap_fault (struct vm_fault *vmf)drivers/vdpa/vdpa_user/iova_domain.h +2
-3@@ -65,9 +65,8 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size,dma_addr_t dma_addr,unsigned long attrs);drivers/vdpa/vdpa_user/vduse_dev.c +3
-6@@ -916,23 +916,27 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, {struct vduse_dev *vdev;struct vduse_iova_domain *domain; -unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ;if (!token.group)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - addr =vduse_domain_alloc_coherent (domain, size, - (dma_addr_t *)&iova, flag); + addr =alloc_pages_exact (size, flag);if (!addr)return NULL ; - *dma_addr = (dma_addr_t )iova; + vdev = token.group->dev; + domain = vdev->domain;drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 940
Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +7
-0+ *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err;return addr; + +err: +free_pages_exact (addr, size); +return NULL ; }I think the code tries to hide the iova domain specific things (e.g mappings and
DMA_MAPPING_ERROR) from vduse core. Could we find a way to stick to that?
Quoted 1 line
Thanks
what do you suggest more spefically though?
Quoted 13 lines
Quoted 13 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1
-1static void vduse_dev_free_coherent(union virtio_map token, size_t size,@@ -949,6 +953,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, domain = vdev->domain;vduse_domain_free_coherent (domain, size, dma_addr, attrs); +free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) --2.52.0
Re: [PATCH v12 09/13] vduse: take out allocations from vduse_dev_alloc_coherent
Quoted 146 lines · Michael S. Tsirkin
Quoted 144 lines · Jason Wang
Quoted 128 lines · Eugenio Pérez
The function
vduse_dev_alloc_coherentwill be called under rwlock in next patches. Make it out of the lock to avoid increasing its fail rate.Signed-off-by Eugenio Pérez <[email protected]>---
v12:
* Avoidfree_pages_exact(NULL, size)in casevduse_domain_alloc_coherentfails (MST).v11: Remove duplicated call to
free_pages_exact(Jason).--- drivers/vdpa/vdpa_user/iova_domain.c | 24 +++++++----------------- drivers/vdpa/vdpa_user/iova_domain.h | 5 ++--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 26 deletions(-)drivers/vdpa/vdpa_user/iova_domain.c +7
-17@@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain,vduse_domain_free_iova (iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig) {struct iova_domain *iovad = &domain->consistent_iovad;unsigned long limit = domain->iova_limit;dma_addr_t iova =vduse_domain_alloc_iova (iovad, size, limit); -void *orig =alloc_pages_exact (size, flag); -if (!iova || !orig) - goto err; +if (!iova) +return DMA_MAPPING_ERROR ;spin_lock (&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1,@@ -514,17 +512,12 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }spin_unlock (&domain->iotlb_lock); - *dma_addr = iova; +return iova; -return orig; err: - *dma_addr =DMA_MAPPING_ERROR ; -if (orig) -free_pages_exact (orig, size); -if (iova) -vduse_domain_free_iova (iovad, iova, size); +vduse_domain_free_iova (iovad, iova, size); -return NULL ; +return DMA_MAPPING_ERROR ; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,@@ -533,7 +526,6 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;struct vdpa_map_file *map_file; -phys_addr_t pa;spin_lock (&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr,@@ -545,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque;fput (map_file->file);kfree (map_file); - pa = map->addr;vhost_iotlb_map_free (domain->iotlb, map);spin_unlock (&domain->iotlb_lock);vduse_domain_free_iova (iovad, dma_addr, size); -free_pages_exact (phys_to_virt (pa), size); }static vm_fault_t vduse_domain_mmap_fault (struct vm_fault *vmf)drivers/vdpa/vdpa_user/iova_domain.h +2
-3@@ -65,9 +65,8 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size,dma_addr_t dma_addr,unsigned long attrs);drivers/vdpa/vdpa_user/vduse_dev.c +3
-6@@ -916,23 +916,27 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, {struct vduse_dev *vdev;struct vduse_iova_domain *domain; -unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ;if (!token.group)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - addr =vduse_domain_alloc_coherent (domain, size, - (dma_addr_t *)&iova, flag); + addr =alloc_pages_exact (size, flag);if (!addr)return NULL ; - *dma_addr = (dma_addr_t )iova; + vdev = token.group->dev; + domain = vdev->domain;drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 940
Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +7
-0+ *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err;return addr; + +err: +free_pages_exact (addr, size); +return NULL ; }I think the code tries to hide the iova domain specific things (e.g mappings and
DMA_MAPPING_ERROR) from vduse core. Could we find a way to stick to that?
Quoted 4 lines
Quoted 1 line
Thanks
what do you suggest more spefically though?
I think it would be better to move the dma/map stuffs to the iova_domain.c
Thanks
Quoted 15 lines
Quoted 14 lines
Quoted 14 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1
-1static void vduse_dev_free_coherent(union virtio_map token, size_t size,@@ -949,6 +953,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, domain = vdev->domain;vduse_domain_free_coherent (domain, size, dma_addr, attrs); +free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) --2.52.0
Eugenio Perez Martin
maintainerRe: [PATCH v12 09/13] vduse: take out allocations from vduse_dev_alloc_coherent
Quoted 148 lines · Jason Wang
Quoted 146 lines · Michael S. Tsirkin
Quoted 144 lines · Jason Wang
Quoted 128 lines · Eugenio Pérez
The function
vduse_dev_alloc_coherentwill be called under rwlock in next patches. Make it out of the lock to avoid increasing its fail rate.Signed-off-by Eugenio Pérez <[email protected]>---
v12:
* Avoidfree_pages_exact(NULL, size)in casevduse_domain_alloc_coherentfails (MST).v11: Remove duplicated call to
free_pages_exact(Jason).--- drivers/vdpa/vdpa_user/iova_domain.c | 24 +++++++----------------- drivers/vdpa/vdpa_user/iova_domain.h | 5 ++--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++------ 3 files changed, 20 insertions(+), 26 deletions(-)drivers/vdpa/vdpa_user/iova_domain.c +7
-17@@ -493,17 +493,15 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain,vduse_domain_free_iova (iovad, dma_addr, size); } -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag) +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig) {struct iova_domain *iovad = &domain->consistent_iovad;unsigned long limit = domain->iova_limit;dma_addr_t iova =vduse_domain_alloc_iova (iovad, size, limit); -void *orig =alloc_pages_exact (size, flag); -if (!iova || !orig) - goto err; +if (!iova) +return DMA_MAPPING_ERROR ;spin_lock (&domain->iotlb_lock); if (vduse_iotlb_add_range(domain, (u64)iova, (u64)iova + size - 1,@@ -514,17 +512,12 @@ void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, }spin_unlock (&domain->iotlb_lock); - *dma_addr = iova; +return iova; -return orig; err: - *dma_addr =DMA_MAPPING_ERROR ; -if (orig) -free_pages_exact (orig, size); -if (iova) -vduse_domain_free_iova (iovad, iova, size); +vduse_domain_free_iova (iovad, iova, size); -return NULL ; +return DMA_MAPPING_ERROR ; } void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,@@ -533,7 +526,6 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size,struct iova_domain *iovad = &domain->consistent_iovad;struct vhost_iotlb_map *map;struct vdpa_map_file *map_file; -phys_addr_t pa;spin_lock (&domain->iotlb_lock); map = vhost_iotlb_itree_first(domain->iotlb, (u64)dma_addr,@@ -545,12 +537,10 @@ void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t size, map_file = (struct vdpa_map_file *)map->opaque;fput (map_file->file);kfree (map_file); - pa = map->addr;vhost_iotlb_map_free (domain->iotlb, map);spin_unlock (&domain->iotlb_lock);vduse_domain_free_iova (iovad, dma_addr, size); -free_pages_exact (phys_to_virt (pa), size); }static vm_fault_t vduse_domain_mmap_fault (struct vm_fault *vmf)drivers/vdpa/vdpa_user/iova_domain.h +2
-3@@ -65,9 +65,8 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *domain, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs); -void *vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, -size_t size,dma_addr_t *dma_addr, -gfp_t flag); +dma_addr_t vduse_domain_alloc_coherent (struct vduse_iova_domain *domain, +size_t size,void *orig);void vduse_domain_free_coherent (struct vduse_iova_domain *domain,size_t size,dma_addr_t dma_addr,unsigned long attrs);drivers/vdpa/vdpa_user/vduse_dev.c +3
-6@@ -916,23 +916,27 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, {struct vduse_dev *vdev;struct vduse_iova_domain *domain; -unsigned long iova;void *addr; *dma_addr =DMA_MAPPING_ERROR ;if (!token.group)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - addr =vduse_domain_alloc_coherent (domain, size, - (dma_addr_t *)&iova, flag); + addr =alloc_pages_exact (size, flag);if (!addr)return NULL ; - *dma_addr = (dma_addr_t )iova; + vdev = token.group->dev; + domain = vdev->domain;drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 940
Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +7
-0+ *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err;return addr; + +err: +free_pages_exact (addr, size); +return NULL ; }I think the code tries to hide the iova domain specific things (e.g mappings and
DMA_MAPPING_ERROR) from vduse core. Could we find a way to stick to that?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Thanks
what do you suggest more spefically though?
I think it would be better to move the dma/map stuffs to the
iova_domain.c
I'm not sure, DMA_MAPPING_ERROR was returned by parameter already before this series and virtio_ring.c uses it in other functions too. I guess we could divert more from the HW DMA and trust in return value in virtion_ring.c conditionals checking if vring_use_dma_api, but I think to include it here is trying to add too much to this series to be honest.
Would that work to do it on top in the future?
Quoted 18 lines
Thanks
Quoted 15 lines
Quoted 14 lines
Quoted 14 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1
-1static void vduse_dev_free_coherent(union virtio_map token, size_t size,@@ -949,6 +953,7 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, domain = vdev->domain;vduse_domain_free_coherent (domain, size, dma_addr, attrs); +free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) --2.52.0
Eugenio Pérez
maintainerPATCH v12 10/13 vduse: merge tree search logic of IOTLB_GET_FD and IOTLB_GET_INFO ioctls
In reply to Eugenio PérezThe next patch adds new ioctl with the ASID member per entry. Abstract these two so it can be build on top easily.
---
v12:
* Avoid dereferencing map_file if not assigned in vduse_dev_iotlb_entry function (MST).
v11: New in v11
--- drivers/vdpa/vdpa_user/vduse_dev.c | 102 ++++++++++++++++------------- 1 file changed, 56 insertions(+), 46 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +56 -46
@@ -1243,6 +1243,51 @@ static void vduse_vq_update_effective_cpu(struct vduse_virtqueue *vq) vq->irq_effective_cpu = curr_cpu; } +static int vduse_dev_iotlb_entry (struct vduse_dev *dev, +struct vduse_iotlb_entry *entry, +struct file **f,uint64_t *capability) +{ +int r = -EINVAL ; +struct vhost_iotlb_map *map; + +if (entry->start > entry->last) +return -EINVAL ; + +mutex_lock (&dev->domain_lock); +if (!dev->domain) + goto out; + +spin_lock (&dev->domain->iotlb_lock); + map =vhost_iotlb_itree_first (dev->domain->iotlb, entry->start, + entry->last); +if (map) { +if (f) { +const struct vdpa_map_file *map_file; + + map_file = (struct vdpa_map_file *)map->opaque; + entry->offset = map_file->offset; + *f =get_file (map_file->file); + } + entry->start = map->start; + entry->last = map->last; + entry->perm = map->perm; +if (capability) { + *capability =0 ; + +if (dev->domain->bounce_map && map->start ==0 && + map->last == dev->domain->bounce_size -1 ) + *capability |=VDUSE_IOVA_CAP_UMEM ; + } + + r =0 ; + } +spin_unlock (&dev->domain->iotlb_lock); + +out: +mutex_unlock (&dev->domain_lock); +return r; +} +static long vduse_dev_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {
@@ -1256,36 +1301,16 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,switch (cmd) {case VDUSE_IOTLB_GET_FD : {struct vduse_iotlb_entry entry; -struct vhost_iotlb_map *map; -struct vdpa_map_file *map_file;struct file *f =NULL ; ret = -EFAULT ;if (copy_from_user (&entry, argp,sizeof (entry)))break ; - ret = -EINVAL ; -if (entry.start > entry.last) + ret =vduse_dev_iotlb_entry (dev, &entry, &f,NULL ); +if (ret)break ; -mutex_lock (&dev->domain_lock); -if (!dev->domain) { -mutex_unlock (&dev->domain_lock); -break ; - } -spin_lock (&dev->domain->iotlb_lock); - map =vhost_iotlb_itree_first (dev->domain->iotlb, - entry.start, entry.last); -if (map) { - map_file = (struct vdpa_map_file *)map->opaque; - f =get_file (map_file->file); - entry.offset = map_file->offset; - entry.start = map->start; - entry.last = map->last; - entry.perm = map->perm; - } -spin_unlock (&dev->domain->iotlb_lock); -mutex_unlock (&dev->domain_lock); ret = -EINVAL ;if (!f)break ;
@@ -1475,41 +1500,26 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, }case VDUSE_IOTLB_GET_INFO : {struct vduse_iova_info info; -struct vhost_iotlb_map *map; +struct vduse_iotlb_entry entry; ret = -EFAULT ;if (copy_from_user (&info, argp,sizeof (info)))break ; - ret = -EINVAL ; -if (info.start > info.last) -break ; -if (!is_mem_zero ((const char *)info.reserved,sizeof (info.reserved)))break ; -mutex_lock (&dev->domain_lock); -if (!dev->domain) { -mutex_unlock (&dev->domain_lock); -break ; - } -spin_lock (&dev->domain->iotlb_lock); - map =vhost_iotlb_itree_first (dev->domain->iotlb, - info.start, info.last); -if (map) { - info.start = map->start; - info.last = map->last; - info.capability =0 ; -if (dev->domain->bounce_map && map->start ==0 && - map->last == dev->domain->bounce_size -1 ) - info.capability |=VDUSE_IOVA_CAP_UMEM ; - } -spin_unlock (&dev->domain->iotlb_lock); -mutex_unlock (&dev->domain_lock); -if (!map) + entry.start = info.start; + entry.last = info.last; + ret =vduse_dev_iotlb_entry (dev, &entry,NULL , + &info.capability); +if (ret <0 )break ; + info.start = entry.start; + info.last = entry.last; + ret = -EFAULT ;if (copy_to_user (argp, &info,sizeof (info)))break ;
Signature · 1 line
2.52.0
Re: [PATCH v12 10/13] vduse: merge tree search logic of IOTLB_GET_FD and IOTLB_GET_INFO ioctls
Quoted 5 lines · Eugenio Pérez
The next patch adds new ioctl with the ASID member per entry. Abstract these two so it can be build on top easily.
Signed-off-by Eugenio Pérez <[email protected]>
Thanks
Eugenio Pérez
maintainerPATCH v12 11/13 vduse: add vq group asid support
In reply to Eugenio PérezAdd support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.
The vq group to ASID association is protected by a rwlock now. But the mutex domain_lock keeps protecting the domains of all ASIDs, as some operations like the one related with the bounce buffer size still requires to lock all the ASIDs.
---
Future improvements can include performance optimizations on top like ore to RCU or thread synchronized atomics, or hardening by tracking ASID or ASID hashes on unused bits of the DMA address.
Tested virtio_vdpa by adding manually two threads in vduse_set_status:
one of them modifies the vq group 0 ASID and the other one map and unmap memory continuously. After a while, the two threads stop and the usual work continues. Test with version 0, version 1 with the old ioctl, and version 1 with the new ioctl.
Tested with vhost_vdpa by migrating a VM while ping on OVS+VDUSE. A few workaround were needed in some parts:
* Do not enable CVQ before data vqs in QEMU, as VDUSE does not forward the enable message to the userland device. This will be solved in the future.
* Share the suspended state between all vhost devices in QEMU:
https://lists.nongnu.org/archive/html/qemu-devel/2025-11/msg02947.html
* Implement a fake VDUSE suspend vdpa operation callback that always returns true in the kernel. DPDK suspend the device at the first GET_VRING_BASE.
* Remove the CVQ blocker in ASID.
The driver vhost_vdpa was also tested with version 0, version 1 with the old ioctl, version 1 with the new ioctl but only one ASID, and version 1 with many ASID.
---
v12:
* Using scoped guards for vq group rwlock, so the one queue optimization is not missed (Jason proposed to factor them into helpers).
* Add the _v2 suffix to vduse_iova_range_v2 struct name fixing the doc (MST).
* s/verion/version/ in patch message.
* Remove trailing ; after a comment (Jason).
v11:
* Remove duplicated free_pages_exact in vduse_domain_free_coherent (Jason).
* Do not take the vq groups lock if nas == 1.
* Do not reset the vq group ASID in vq reset (Jason). Removed extra function vduse_set_group_asid_nomsg, not needed anymore.
* Move the vduse_iotlb_entry_v2 argument to a new ioctl, as argument didn't match the previous VDUSE_IOTLB_GET_FD.
* Move the asid < dev->nas check to vdpa core.
v10:
* Back to rwlock version so stronger locks are used.
* Take out allocations from rwlock.
* Forbid changing ASID of a vq group after DRIVER_OK (Jason)
* Remove bad fetching again of domain variable in vduse_dev_max_mapping_size (Yongji).
* Remove unused vdev definition in vdpa map_ops callbacks (kernel test robot).
v9:
* Replace mutex with rwlock, as the vdpa map_ops can run from atomic context.
v8:
* Revert the mutex to rwlock change, it needs proper profiling to justify it.
v7:
* Take write lock in the error path (Jason).
v6:
* Make vdpa_dev_add use gotos for error handling (MST).
* s/(dev->api_version < 1) ?/(dev->api_version < VDUSE_API_VERSION_1) ?/ (MST).
* Fix struct name not matching in the doc.
v5:
* Properly return errno if copy_to_user returns >0 in VDUSE_IOTLB_GET_FD ioctl (Jason).
* Properly set domain bounce size to divide equally between nas (Jason).
* Exclude "padding" member from the only >V1 members in vduse_dev_request.
v4:
* Divide each domain bounce size between the device bounce size (Jason).
* revert unneeded addr = NULL assignment (Jason)
* Change if (x && (y || z)) return to if (x) { if (y) return; if (z) return; } (Jason)
* Change a bad multiline comment, using @ caracter instead of * (Jason).
* Consider config->nas == 0 as a fail (Jason).
v3:
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash with the patch creating the vduse_as struct (Jason).
* Create VDUSE_DEV_MAX_AS instead of comparing agains a magic number (Jason)
v2:
* Convert the use of mutex to rwlock.
RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
* Use of array_index_nospec in VDUSE device ioctls.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.
RFC v2:
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.
--- drivers/vdpa/vdpa_user/vduse_dev.c | 385 +++++++++++++++++++---------- include/uapi/linux/vduse.h | 63 ++++- 2 files changed, 312 insertions(+), 136 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +252 -133
@@ -9,6 +9,7 @@*/ #include "linux/virtio_net.h" +#include <linux/cleanup.h> #include <linux/init.h> #include <linux/module.h> #include <linux/cdev.h>
@@ -41,6 +42,7 @@#define VDUSE_DEV_MAX (1U << MINORBITS)#define VDUSE_DEV_MAX_GROUPS 0xffff +#define VDUSE_DEV_MAX_AS 0xffff#define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)#define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)#define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -86,7 +88,15 @@ struct vduse_umem {struct mm_struct *mm; }; +struct vduse_as { +struct vduse_iova_domain *domain; +struct vduse_umem *umem; +struct mutex mem_lock; +}; +struct vduse_vq_group { +rwlock_t as_lock; +struct vduse_as *as;/* Protected by as_lock */ struct vduse_dev *dev; };
@@ -94,7 +104,7 @@ struct vduse_dev {struct vduse_vdpa *vdev;struct device *dev;struct vduse_virtqueue **vqs; -struct vduse_iova_domain *domain; +struct vduse_as *as;char *name;struct mutex lock;spinlock_t msg_lock;
@@ -122,9 +132,8 @@ struct vduse_dev {u32 vq_num;u32 vq_align;u32 ngroups; -struct vduse_umem *umem; +u32 nas;struct vduse_vq_group *groups; -struct mutex mem_lock;unsigned int bounce_size;struct mutex domain_lock; };
@@ -314,7 +323,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)return vduse_dev_msg_sync (dev, &msg); } -static int vduse_dev_update_iotlb (struct vduse_dev *dev, +static int vduse_dev_update_iotlb (struct vduse_dev *dev,u32 asid,u64 start,u64 last) {struct vduse_dev_msg msg = {0 };
@@ -323,8 +332,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,return -EINVAL ; msg.req.type =VDUSE_UPDATE_IOTLB ; - msg.req.iova.start = start; - msg.req.iova.last = last; +if (dev->api_version <VDUSE_API_VERSION_1 ) { + msg.req.iova.start = start; + msg.req.iova.last = last; + }else { + msg.req.iova_v2.start = start; + msg.req.iova_v2.last = last; + msg.req.iova_v2.asid = asid; + }return vduse_dev_msg_sync (dev, &msg); }
@@ -439,11 +454,14 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)static void vduse_dev_reset (struct vduse_dev *dev) {int i; -struct vduse_iova_domain *domain = dev->domain;/* The coherent mappings are handled in vduse_dev_free_coherent() */ -if (domain && domain->bounce_map) -vduse_domain_reset_bounce_map (domain); +for (i =0 ; i < dev->nas; i++) { +struct vduse_iova_domain *domain = dev->as[i].domain; + +if (domain && domain->bounce_map) +vduse_domain_reset_bounce_map (domain); + }down_write (&dev->rwsem);
@@ -622,6 +640,42 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)return ret; } +DEFINE_GUARD(vq_group_as_read_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +read_lock (&_T->as_lock), + if (_T->dev->nas > 1) + read_unlock(&_T->as_lock)) + +DEFINE_GUARD(vq_group_as_write_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +write_lock (&_T->as_lock), + if (_T->dev->nas > 1) + write_unlock(&_T->as_lock)) + +static int vduse_set_group_asid (struct vdpa_device *vdpa,unsigned int group, +unsigned int asid) +{ +struct vduse_dev *dev =vdpa_to_vduse (vdpa); +struct vduse_dev_msg msg = {0 }; +int r; + +if (dev->api_version <VDUSE_API_VERSION_1 ) +return -EINVAL ; + + msg.req.type =VDUSE_SET_VQ_GROUP_ASID ; + msg.req.vq_group_asid.group = group; + msg.req.vq_group_asid.asid = asid; + + r =vduse_dev_msg_sync (dev, &msg); +if (r <0 ) +return r; + +guard (vq_group_as_write_lock)(&dev->groups[group]); + dev->groups[group].as = &dev->as[asid]; + +return 0 ; +} +static int vduse_vdpa_get_vq_state (struct vdpa_device *vdpa,u16 idx,struct vdpa_vq_state *state) {
@@ -793,13 +847,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,struct vduse_dev *dev =vdpa_to_vduse (vdpa);int ret; - ret =vduse_domain_set_map (dev->domain, iotlb); + ret =vduse_domain_set_map (dev->as[asid].domain, iotlb);if (ret)return ret; - ret =vduse_dev_update_iotlb (dev,0ULL ,ULLONG_MAX ); + ret =vduse_dev_update_iotlb (dev, asid,0ULL ,ULLONG_MAX );if (ret) { -vduse_domain_clear_map (dev->domain, iotlb); +vduse_domain_clear_map (dev->as[asid].domain, iotlb);return ret; }
@@ -842,6 +896,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, .set_map = vduse_vdpa_set_map, + .set_group_asid = vduse_set_group_asid, .get_vq_map = vduse_get_vq_map, .free = vduse_vdpa_free, };
@@ -850,15 +905,13 @@ static void vduse_dev_sync_single_for_device(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_device (domain, dma_addr, size, dir); }
@@ -866,15 +919,13 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_cpu (domain, dma_addr, size, dir); }
@@ -883,15 +934,13 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page, enum dma_data_direction dir, unsigned long attrs) { -struct vduse_dev *vdev;struct vduse_iova_domain *domain;if (!token.group)return DMA_MAPPING_ERROR ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;return vduse_domain_map_page (domain, page, offset, size, dir, attrs); }
@@ -899,23 +948,19 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - -return vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; +vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); }static void *vduse_dev_alloc_coherent (union virtio_map token,size_t size,dma_addr_t *dma_addr,gfp_t flag) { -struct vduse_dev *vdev; -struct vduse_iova_domain *domain;void *addr; *dma_addr =DMA_MAPPING_ERROR ;
@@ -926,11 +971,15 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,if (!addr)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); -if (*dma_addr ==DMA_MAPPING_ERROR ) - goto err; + { +struct vduse_iova_domain *domain; + +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; + *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err; + }return addr;
@@ -943,31 +992,27 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size, void *vaddr, dma_addr_t dma_addr, unsigned long attrs) { - struct vduse_dev *vdev; -struct vduse_iova_domain *domain; - if (!token.group) return; - vdev = token.group->dev; - domain = vdev->domain; + { +struct vduse_iova_domain *domain; + +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; +vduse_domain_free_coherent (domain, size, dma_addr, attrs); + } -vduse_domain_free_coherent (domain, size, dma_addr, attrs);free_pages_exact (vaddr, size); }static bool vduse_dev_need_sync (union virtio_map token,dma_addr_t dma_addr) { -struct vduse_dev *vdev; -struct vduse_iova_domain *domain; -if (!token.group)return false; - vdev = token.group->dev; - domain = vdev->domain; - -return dma_addr < domain->bounce_size; +guard (vq_group_as_read_lock)(token.group); +return dma_addr < token.group->as->domain->bounce_size; }static int vduse_dev_mapping_error (union virtio_map token,dma_addr_t dma_addr)
@@ -979,16 +1024,11 @@ static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)static size_t vduse_dev_max_mapping_size (union virtio_map token) { -struct vduse_dev *vdev; -struct vduse_iova_domain *domain; -if (!token.group)return 0 ; - vdev = token.group->dev; - domain = vdev->domain; - -return domain->bounce_size; +guard (vq_group_as_read_lock)(token.group); +return token.group->as->domain->bounce_size; }static const struct virtio_map_ops vduse_map_ops = {
@@ -1128,39 +1168,40 @@ static int vduse_dev_queue_irq_work(struct vduse_dev *dev,return ret; } -static int vduse_dev_dereg_umem (struct vduse_dev *dev, +static int vduse_dev_dereg_umem (struct vduse_dev *dev,u32 asid,u64 iova,u64 size) {int ret; -mutex_lock (&dev->mem_lock); +mutex_lock (&dev->as[asid].mem_lock); ret = -ENOENT ; -if (!dev->umem) +if (!dev->as[asid].umem) goto unlock; ret = -EINVAL ; -if (!dev->domain) +if (!dev->as[asid].domain) goto unlock; -if (dev->umem->iova != iova || size != dev->domain->bounce_size) +if (dev->as[asid].umem->iova != iova || + size != dev->as[asid].domain->bounce_size) goto unlock; -vduse_domain_remove_user_bounce_pages (dev->domain); -unpin_user_pages_dirty_lock (dev->umem->pages, - dev->umem->npages, true); -atomic64_sub (dev->umem->npages, &dev->umem->mm->pinned_vm); -mmdrop (dev->umem->mm); -vfree (dev->umem->pages); -kfree (dev->umem); - dev->umem =NULL ; +vduse_domain_remove_user_bounce_pages (dev->as[asid].domain); +unpin_user_pages_dirty_lock (dev->as[asid].umem->pages, + dev->as[asid].umem->npages, true); +atomic64_sub (dev->as[asid].umem->npages, &dev->as[asid].umem->mm->pinned_vm); +mmdrop (dev->as[asid].umem->mm); +vfree (dev->as[asid].umem->pages); +kfree (dev->as[asid].umem); + dev->as[asid].umem =NULL ; ret =0 ; unlock: -mutex_unlock (&dev->mem_lock); +mutex_unlock (&dev->as[asid].mem_lock);return ret; }static int vduse_dev_reg_umem (struct vduse_dev *dev, -u64 iova,u64 uaddr,u64 size) +u32 asid,u64 iova,u64 uaddr,u64 size) {struct page **page_list =NULL ;struct vduse_umem *umem =NULL ;
@@ -1168,14 +1209,14 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,unsigned long npages, lock_limit;int ret; -if (!dev->domain || !dev->domain->bounce_map || - size != dev->domain->bounce_size || +if (!dev->as[asid].domain || !dev->as[asid].domain->bounce_map || + size != dev->as[asid].domain->bounce_size || iova !=0 || uaddr & ~PAGE_MASK )return -EINVAL ; -mutex_lock (&dev->mem_lock); +mutex_lock (&dev->as[asid].mem_lock); ret = -EEXIST ; -if (dev->umem) +if (dev->as[asid].umem) goto unlock; ret = -ENOMEM ;
@@ -1199,7 +1240,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, goto out; } - ret =vduse_domain_add_user_bounce_pages (dev->domain, + ret =vduse_domain_add_user_bounce_pages (dev->as[asid].domain, page_list, pinned);if (ret) goto out;
@@ -1212,7 +1253,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev, umem->mm = current->mm;mmgrab (current->mm); - dev->umem = umem; + dev->as[asid].umem = umem; out:if (ret && pinned >0 )unpin_user_pages (page_list, pinned);
@@ -1223,7 +1264,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,vfree (page_list);kfree (umem); } -mutex_unlock (&dev->mem_lock); +mutex_unlock (&dev->as[asid].mem_lock);return ret; }
@@ -1244,44 +1285,47 @@ static void vduse_vq_update_effective_cpu(struct vduse_virtqueue *vq) }static int vduse_dev_iotlb_entry (struct vduse_dev *dev, -struct vduse_iotlb_entry *entry, +struct vduse_iotlb_entry_v2 *entry,struct file **f,uint64_t *capability) { +u32 asid;int r = -EINVAL ;struct vhost_iotlb_map *map; -if (entry->start > entry->last) +if (entry->v1.start > entry->v1.last || entry->asid >= dev->nas)return -EINVAL ; + asid =array_index_nospec (entry->asid, dev->nas);mutex_lock (&dev->domain_lock); -if (!dev->domain) + +if (!dev->as[asid].domain) goto out; -spin_lock (&dev->domain->iotlb_lock); - map =vhost_iotlb_itree_first (dev->domain->iotlb, entry->start, - entry->last); +spin_lock (&dev->as[asid].domain->iotlb_lock); + map =vhost_iotlb_itree_first (dev->as[asid].domain->iotlb, + entry->v1.start, entry->v1.last);if (map) {if (f) {const struct vdpa_map_file *map_file; map_file = (struct vdpa_map_file *)map->opaque; - entry->offset = map_file->offset; + entry->v1.offset = map_file->offset; *f =get_file (map_file->file); } - entry->start = map->start; - entry->last = map->last; - entry->perm = map->perm; + entry->v1.start = map->start; + entry->v1.last = map->last; + entry->v1.perm = map->perm;if (capability) { *capability =0 ; -if (dev->domain->bounce_map && map->start ==0 && - map->last == dev->domain->bounce_size -1 ) +if (dev->as[asid].domain->bounce_map && map->start ==0 && + map->last == dev->as[asid].domain->bounce_size -1 ) *capability |=VDUSE_IOVA_CAP_UMEM ; } r =0 ; } -spin_unlock (&dev->domain->iotlb_lock); +spin_unlock (&dev->as[asid].domain->iotlb_lock); out:mutex_unlock (&dev->domain_lock);
@@ -1299,12 +1343,29 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,return -EPERM ;switch (cmd) { -case VDUSE_IOTLB_GET_FD : { -struct vduse_iotlb_entry entry; +case VDUSE_IOTLB_GET_FD : +case VDUSE_IOTLB_GET_FD2 : { +struct vduse_iotlb_entry_v2 entry = {0 };struct file *f =NULL ; + ret = -ENOIOCTLCMD ; +if (dev->api_version <VDUSE_API_VERSION_1 && + cmd ==VDUSE_IOTLB_GET_FD2 ) +break ; + ret = -EFAULT ; -if (copy_from_user (&entry, argp,sizeof (entry))) +if (cmd ==VDUSE_IOTLB_GET_FD2 ) { +if (copy_from_user (&entry, argp,sizeof (entry))) +break ; + }else { +if (copy_from_user (&entry.v1, argp, +sizeof (entry.v1))) +break ; + } + + ret = -EINVAL ; +if (!is_mem_zero ((const char *)entry.reserved, +sizeof (entry.reserved)))break ; ret =vduse_dev_iotlb_entry (dev, &entry, &f,NULL );
@@ -1315,12 +1376,19 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,if (!f)break ; - ret = -EFAULT ; -if (copy_to_user (argp, &entry,sizeof (entry))) { +if (cmd ==VDUSE_IOTLB_GET_FD2 ) + ret =copy_to_user (argp, &entry, +sizeof (entry)); +else + ret =copy_to_user (argp, &entry.v1, +sizeof (entry.v1)); + +if (ret) { + ret = -EFAULT ;fput (f);break ; } - ret =receive_fd (f,NULL ,perm_to_file_flags (entry.perm)); + ret =receive_fd (f,NULL ,perm_to_file_flags (entry.v1.perm));fput (f);break ; }
@@ -1465,6 +1533,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, }case VDUSE_IOTLB_REG_UMEM : {struct vduse_iova_umem umem; +u32 asid; ret = -EFAULT ;if (copy_from_user (&umem, argp,sizeof (umem)))
@@ -1472,17 +1541,21 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, ret = -EINVAL ;if (!is_mem_zero ((const char *)umem.reserved, -sizeof (umem.reserved))) +sizeof (umem.reserved)) || + (dev->api_version <VDUSE_API_VERSION_1 && + umem.asid !=0 ) || umem.asid >= dev->nas)break ;mutex_lock (&dev->domain_lock); - ret =vduse_dev_reg_umem (dev, umem.iova, + asid =array_index_nospec (umem.asid, dev->nas); + ret =vduse_dev_reg_umem (dev, asid, umem.iova, umem.uaddr, umem.size);mutex_unlock (&dev->domain_lock);break ; }case VDUSE_IOTLB_DEREG_UMEM : {struct vduse_iova_umem umem; +u32 asid; ret = -EFAULT ;if (copy_from_user (&umem, argp,sizeof (umem)))
@@ -1490,17 +1563,22 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, ret = -EINVAL ;if (!is_mem_zero ((const char *)umem.reserved, -sizeof (umem.reserved))) +sizeof (umem.reserved)) || + (dev->api_version <VDUSE_API_VERSION_1 && + umem.asid !=0 ) || + umem.asid >= dev->nas)break ; +mutex_lock (&dev->domain_lock); - ret =vduse_dev_dereg_umem (dev, umem.iova, + asid =array_index_nospec (umem.asid, dev->nas); + ret =vduse_dev_dereg_umem (dev, asid, umem.iova, umem.size);mutex_unlock (&dev->domain_lock);break ; }case VDUSE_IOTLB_GET_INFO : {struct vduse_iova_info info; -struct vduse_iotlb_entry entry; +struct vduse_iotlb_entry_v2 entry; ret = -EFAULT ;if (copy_from_user (&info, argp,sizeof (info)))
@@ -1510,15 +1588,23 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd, sizeof(info.reserved)))break ; - entry.start = info.start; - entry.last = info.last; +if (dev->api_version <VDUSE_API_VERSION_1 ) { +if (info.asid) +break ; + }else if (info.asid >= dev->nas) +break ; + + entry.v1.start = info.start; + entry.v1.last = info.last; + entry.asid = info.asid; ret =vduse_dev_iotlb_entry (dev, &entry,NULL , &info.capability);if (ret <0 )break ; - info.start = entry.start; - info.last = entry.last; + info.start = entry.v1.start; + info.last = entry.v1.last; + info.asid = entry.asid; ret = -EFAULT ;if (copy_to_user (argp, &info,sizeof (info)))
@@ -1540,8 +1626,10 @@ static int vduse_dev_release(struct inode *inode, struct file *file)struct vduse_dev *dev = file->private_data;mutex_lock (&dev->domain_lock); -if (dev->domain) -vduse_dev_dereg_umem (dev,0 , dev->domain->bounce_size); +for (int i =0 ; i < dev->nas; i++) +if (dev->as[i].domain) +vduse_dev_dereg_umem (dev, i,0 , + dev->as[i].domain->bounce_size);mutex_unlock (&dev->domain_lock);spin_lock (&dev->msg_lock);/* Make sure the inflight messages can processed after reconncection */
@@ -1760,7 +1848,6 @@ static struct vduse_dev *vduse_dev_create(void)return NULL ;mutex_init (&dev->lock); -mutex_init (&dev->mem_lock);mutex_init (&dev->domain_lock);spin_lock_init (&dev->msg_lock);INIT_LIST_HEAD (&dev->send_list);
@@ -1811,8 +1898,11 @@ static int vduse_destroy_dev(char *name)idr_remove (&vduse_idr, dev->minor);kvfree (dev->config);vduse_dev_deinit_vqs (dev); -if (dev->domain) -vduse_domain_destroy (dev->domain); +for (int i =0 ; i < dev->nas; i++) { +if (dev->as[i].domain) +vduse_domain_destroy (dev->as[i].domain); + } +kfree (dev->as);kfree (dev->name);kfree (dev->groups);vduse_dev_destroy (dev);
@@ -1859,12 +1949,17 @@ static bool vduse_validate_config(struct vduse_dev_config *config, sizeof(config->reserved)))return false; -if (api_version <VDUSE_API_VERSION_1 && config->ngroups) +if (api_version <VDUSE_API_VERSION_1 && + (config->ngroups || config->nas))return false; -if (api_version >=VDUSE_API_VERSION_1 && - (!config->ngroups || config->ngroups >VDUSE_DEV_MAX_GROUPS )) -return false; +if (api_version >=VDUSE_API_VERSION_1 ) { +if (!config->ngroups || config->ngroups >VDUSE_DEV_MAX_GROUPS ) +return false; + +if (!config->nas || config->nas >VDUSE_DEV_MAX_AS ) +return false; + }if (config->vq_align >PAGE_SIZE )return false;
@@ -1929,7 +2024,8 @@ static ssize_t bounce_size_store(struct device *device, ret = -EPERM ;mutex_lock (&dev->domain_lock); -if (dev->domain) +/* Assuming that if the first domain is allocated, all are allocated */ +if (dev->as[0 ].domain) goto unlock; ret =kstrtouint (buf,10 , &bounce_size);
@@ -1981,6 +2077,14 @@ static int vduse_create_dev(struct vduse_dev_config *config, dev->device_features = config->features; dev->device_id = config->device_id; dev->vendor_id = config->vendor_id; + + dev->nas = (dev->api_version <VDUSE_API_VERSION_1 ) ?1 : config->nas; + dev->as =kcalloc (dev->nas,sizeof (dev->as[0 ]),GFP_KERNEL ); +if (!dev->as) + goto err_as; +for (int i =0 ; i < dev->nas; i++) +mutex_init (&dev->as[i].mem_lock); + dev->ngroups = (dev->api_version <VDUSE_API_VERSION_1 ) ?1 : config->ngroups;
@@ -1988,8 +2092,11 @@ static int vduse_create_dev(struct vduse_dev_config *config, GFP_KERNEL);if (!dev->groups) goto err_vq_groups; -for (u32 i =0 ; i < dev->ngroups; ++i) +for (u32 i =0 ; i < dev->ngroups; ++i) { dev->groups[i].dev = dev; +rwlock_init (&dev->groups[i].as_lock); + dev->groups[i].as = &dev->as[0 ]; + } dev->name =kstrdup (config->name,GFP_KERNEL );if (!dev->name)
@@ -2029,6 +2136,8 @@ static int vduse_create_dev(struct vduse_dev_config *config, err_str:kfree (dev->groups); err_vq_groups: +kfree (dev->as); +err_as:vduse_dev_destroy (dev); err:return ret;
@@ -2152,7 +2261,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name) vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev, &vduse_vdpa_config_ops, &vduse_map_ops, - dev->ngroups,1 , name, true); + dev->ngroups, dev->nas, name, true);if (IS_ERR (vdev))return PTR_ERR (vdev);
@@ -2167,7 +2276,8 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, const struct vdpa_dev_set_config *config) {struct vduse_dev *dev; -int ret; +size_t domain_bounce_size; +int ret, i;mutex_lock (&vduse_lock); dev =vduse_find_dev (name);
@@ -2181,29 +2291,38 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,return ret;mutex_lock (&dev->domain_lock); -if (!dev->domain) - dev->domain =vduse_domain_create (VDUSE_IOVA_SIZE -1 , - dev->bounce_size); -mutex_unlock (&dev->domain_lock); -if (!dev->domain) { - ret = -ENOMEM ; - goto domain_err; + ret =0 ; + + domain_bounce_size = dev->bounce_size / dev->nas; +for (i =0 ; i < dev->nas; ++i) { + dev->as[i].domain =vduse_domain_create (VDUSE_IOVA_SIZE -1 , + domain_bounce_size); +if (!dev->as[i].domain) { + ret = -ENOMEM ; + goto err; + } } +mutex_unlock (&dev->domain_lock); + ret =_vdpa_register_device (&dev->vdev->vdpa, dev->vq_num); -if (ret) { - goto register_err; - } +if (ret) + goto err_register;return 0 ; -register_err: +err_register:mutex_lock (&dev->domain_lock); -vduse_domain_destroy (dev->domain); - dev->domain =NULL ; + +err: +for (int j =0 ; j < i; j++) { +if (dev->as[j].domain) { +vduse_domain_destroy (dev->as[j].domain); + dev->as[j].domain =NULL ; + } + }mutex_unlock (&dev->domain_lock); -domain_err:put_device (&dev->vdev->vdpa.dev);return ret;
include/uapi/linux/vduse.h +60 -3
129 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH v12 11/13] vduse: add vq group asid support
Quoted 384 lines · Eugenio Pérez
Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.
The vq group to ASID association is protected by a rwlock now. But the mutex
domain_lockkeeps protecting the domains of all ASIDs, as some operations like the one related with the bounce buffer size still requires to lock all the ASIDs.Signed-off-by Eugenio Pérez <[email protected]>---
Future improvements can include performance optimizations on top like ore to RCU or thread synchronized atomics, or hardening by tracking ASID or ASID hashes on unused bits of the DMA address.Tested
virtio_vdpaby adding manually two threads invduse_set_status:
one of them modifies the vq group 0 ASID and the other one map and unmap memory continuously. After a while, the two threads stop and the usual work continues. Test with version 0, version 1 with the old ioctl, and version 1 with the new ioctl.Tested with
vhost_vdpaby migrating a VM while ping on OVS+VDUSE. A few workaround were needed in some parts:
* Do not enable CVQ before data vqs in QEMU, as VDUSE does not forward the enable message to the userland device. This will be solved in the future.
* Share the suspended state between all vhost devices in QEMU:
https://lists.nongnu.org/archive/html/qemu-devel/2025-11/msg02947.html
* Implement a fake VDUSE suspend vdpa operation callback that always returns true in the kernel. DPDK suspend the device at the firstGET_VRING_BASE.
* Remove the CVQ blocker in ASID.The driver
vhost_vdpawas also tested with version 0, version 1 with the old ioctl, version 1 with the new ioctl but only one ASID, and version 1 with many ASID.---
v12:
* Using scoped guards for vq group rwlock, so the one queue optimization is not missed (Jason proposed to factor them into helpers).
* Add the_v2suffix tovduse_iova_range_v2struct name fixing the doc (MST).
* s/verion/version/ in patch message.
* Remove trailing ; after a comment (Jason).v11:
* Remove duplicatedfree_pages_exactinvduse_domain_free_coherent(Jason).
* Do not take the vq groups lock if nas == 1.
* Do not reset the vq group ASID in vq reset (Jason). Removed extra functionvduse_set_group_asid_nomsg, not needed anymore.
* Move thevduse_iotlb_entry_v2argument to a new ioctl, as argument didn't match the previousVDUSE_IOTLB_GET_FD.
* Move the asid <dev->nascheck to vdpa core.v10:
* Back to rwlock version so stronger locks are used.
* Take out allocations from rwlock.
* Forbid changing ASID of a vq group afterDRIVER_OK(Jason)
* Remove bad fetching again of domain variable invduse_dev_max_mapping_size(Yongji).
* Remove unused vdev definition in vdpamap_opscallbacks (kernel test robot).v9:
* Replace mutex with rwlock, as the vdpamap_opscan run from atomic context.v8:
* Revert the mutex to rwlock change, it needs proper profiling to justify it.v7:
* Take write lock in the error path (Jason).v6:
* Makevdpa_dev_adduse gotos for error handling (MST).
* s/(dev->api_version < 1) ?/(dev->api_version <VDUSE_API_VERSION_1) ?/ (MST).
* Fix struct name not matching in the doc.v5:
* Properly return errno ifcopy_to_userreturns >0 inVDUSE_IOTLB_GET_FDioctl (Jason).
* Properly set domain bounce size to divide equally between nas (Jason).
* Exclude "padding" member from the only >V1 members invduse_dev_request.v4:
* Divide each domain bounce size between the device bounce size (Jason).
* revert unneeded addr = NULL assignment (Jason)
* Change if (x && (y || z)) return to if (x) { if (y) return; if (z) return; } (Jason)
* Change a bad multiline comment, using @ caracter instead of * (Jason).
* Considerconfig->nas== 0 as a fail (Jason).v3:
* Get the vduse domain through thevduse_asin the map functions (Jason).
* Squash with the patch creating thevduse_asstruct (Jason).
* CreateVDUSE_DEV_MAX_ASinstead of comparing agains a magic number (Jason)v2:
* Convert the use of mutex to rwlock.RFC v3:
* IncreaseVDUSE_MAX_VQ_GROUPSto0xffff(Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about mergingVDUSE_IOTLB_GET_FDioctl withVDUSE_IOTLB_GET_INFO.
* Use ofarray_index_nospecin VDUSE device ioctls.
* Embedvduse_iotlb_entryintovduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.RFC v2:
* Make iotlb entry the last one ofvduse_iotlb_entry_v2so the first part of the struct is the same.--- drivers/vdpa/vdpa_user/vduse_dev.c | 385 +++++++++++++++++++---------- include/uapi/linux/vduse.h | 63 ++++- 2 files changed, 312 insertions(+), 136 deletions(-)drivers/vdpa/vdpa_user/vduse_dev.c +76
-35@@ -9,6 +9,7 @@*/ #include "linux/virtio_net.h" +#include <linux/cleanup.h> #include <linux/init.h> #include <linux/module.h> #include <linux/cdev.h> @@ -41,6 +42,7 @@#define VDUSE_DEV_MAX (1U << MINORBITS)#define VDUSE_DEV_MAX_GROUPS 0xffff +#define VDUSE_DEV_MAX_AS 0xffff#define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)#define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)#define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)@@ -86,7 +88,15 @@ struct vduse_umem {struct mm_struct *mm; }; +struct vduse_as { +struct vduse_iova_domain *domain; +struct vduse_umem *umem; +struct mutex mem_lock; +}; +struct vduse_vq_group { +rwlock_t as_lock; +struct vduse_as *as;/* Protected by as_lock */ struct vduse_dev *dev; };@@ -94,7 +104,7 @@ struct vduse_dev {struct vduse_vdpa *vdev;struct device *dev;struct vduse_virtqueue **vqs; -struct vduse_iova_domain *domain; +struct vduse_as *as;char *name;struct mutex lock;spinlock_t msg_lock;@@ -122,9 +132,8 @@ struct vduse_dev {u32 vq_num;u32 vq_align;u32 ngroups; -struct vduse_umem *umem; +u32 nas;struct vduse_vq_group *groups; -struct mutex mem_lock;unsigned int bounce_size;struct mutex domain_lock; };@@ -314,7 +323,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)return vduse_dev_msg_sync (dev, &msg); } -static int vduse_dev_update_iotlb (struct vduse_dev *dev, +static int vduse_dev_update_iotlb (struct vduse_dev *dev,u32 asid,u64 start,u64 last) {struct vduse_dev_msg msg = {0 };@@ -323,8 +332,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,return -EINVAL ; msg.req.type =VDUSE_UPDATE_IOTLB ; - msg.req.iova.start = start; - msg.req.iova.last = last; +if (dev->api_version <VDUSE_API_VERSION_1 ) { + msg.req.iova.start = start; + msg.req.iova.last = last; + }else { + msg.req.iova_v2.start = start; + msg.req.iova_v2.last = last; + msg.req.iova_v2.asid = asid; + }return vduse_dev_msg_sync (dev, &msg); }@@ -439,11 +454,14 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)static void vduse_dev_reset (struct vduse_dev *dev) {int i; -struct vduse_iova_domain *domain = dev->domain;/* The coherent mappings are handled in vduse_dev_free_coherent() */ -if (domain && domain->bounce_map) -vduse_domain_reset_bounce_map (domain); +for (i =0 ; i < dev->nas; i++) { +struct vduse_iova_domain *domain = dev->as[i].domain; + +if (domain && domain->bounce_map) +vduse_domain_reset_bounce_map (domain); + }down_write (&dev->rwsem);@@ -622,6 +640,42 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)return ret; } +DEFINE_GUARD(vq_group_as_read_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +read_lock (&_T->as_lock), + if (_T->dev->nas > 1) + read_unlock(&_T->as_lock)) + +DEFINE_GUARD(vq_group_as_write_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +write_lock (&_T->as_lock), + if (_T->dev->nas > 1) + write_unlock(&_T->as_lock)) + +static int vduse_set_group_asid (struct vdpa_device *vdpa,unsigned int group, +unsigned int asid) +{ +struct vduse_dev *dev =vdpa_to_vduse (vdpa); +struct vduse_dev_msg msg = {0 }; +int r; + +if (dev->api_version <VDUSE_API_VERSION_1 ) +return -EINVAL ; + + msg.req.type =VDUSE_SET_VQ_GROUP_ASID ; + msg.req.vq_group_asid.group = group; + msg.req.vq_group_asid.asid = asid; + + r =vduse_dev_msg_sync (dev, &msg); +if (r <0 ) +return r; + +guard (vq_group_as_write_lock)(&dev->groups[group]); + dev->groups[group].as = &dev->as[asid]; + +return 0 ; +} +static int vduse_vdpa_get_vq_state (struct vdpa_device *vdpa,u16 idx,struct vdpa_vq_state *state) {@@ -793,13 +847,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,struct vduse_dev *dev =vdpa_to_vduse (vdpa);int ret; - ret =vduse_domain_set_map (dev->domain, iotlb); + ret =vduse_domain_set_map (dev->as[asid].domain, iotlb);if (ret)return ret; - ret =vduse_dev_update_iotlb (dev,0ULL ,ULLONG_MAX ); + ret =vduse_dev_update_iotlb (dev, asid,0ULL ,ULLONG_MAX );if (ret) { -vduse_domain_clear_map (dev->domain, iotlb); +vduse_domain_clear_map (dev->as[asid].domain, iotlb);return ret; }@@ -842,6 +896,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, .set_map = vduse_vdpa_set_map, + .set_group_asid = vduse_set_group_asid, .get_vq_map = vduse_get_vq_map, .free = vduse_vdpa_free, };@@ -850,15 +905,13 @@ static void vduse_dev_sync_single_for_device(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_device (domain, dma_addr, size, dir); }@@ -866,15 +919,13 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_cpu (domain, dma_addr, size, dir); }@@ -883,15 +934,13 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page, enum dma_data_direction dir, unsigned long attrs) { -struct vduse_dev *vdev;struct vduse_iova_domain *domain;if (!token.group)return DMA_MAPPING_ERROR ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;return vduse_domain_map_page (domain, page, offset, size, dir, attrs); }@@ -899,23 +948,19 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - -return vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; +vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); }static void *vduse_dev_alloc_coherent (union virtio_map token,size_t size,dma_addr_t *dma_addr,gfp_t flag) { -struct vduse_dev *vdev; -struct vduse_iova_domain *domain;void *addr; *dma_addr =DMA_MAPPING_ERROR ;@@ -926,11 +971,15 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,if (!addr)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); -if (*dma_addr ==DMA_MAPPING_ERROR )
drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 982
Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +9
-1- goto err; + { +struct vduse_iova_domain *domain; + +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; + *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err; + }
Nit: Having a standalone block probably means we have the chance to optimize the code, maybe using a helper?
Thanks
Eugenio Perez Martin
maintainerRe: [PATCH v12 11/13] vduse: add vq group asid support
Quoted 391 lines · Jason Wang
Quoted 389 lines · Eugenio Pérez
Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.
The vq group to ASID association is protected by a rwlock now. But the mutex
domain_lockkeeps protecting the domains of all ASIDs, as some operations like the one related with the bounce buffer size still requires to lock all the ASIDs.Signed-off-by Eugenio Pérez <[email protected]>---
Future improvements can include performance optimizations on top like ore to RCU or thread synchronized atomics, or hardening by tracking ASID or ASID hashes on unused bits of the DMA address.Tested
virtio_vdpaby adding manually two threads invduse_set_status:
one of them modifies the vq group 0 ASID and the other one map and unmap memory continuously. After a while, the two threads stop and the usual work continues. Test with version 0, version 1 with the old ioctl, and version 1 with the new ioctl.Tested with
vhost_vdpaby migrating a VM while ping on OVS+VDUSE. A few workaround were needed in some parts:
* Do not enable CVQ before data vqs in QEMU, as VDUSE does not forward the enable message to the userland device. This will be solved in the future.
* Share the suspended state between all vhost devices in QEMU:
https://lists.nongnu.org/archive/html/qemu-devel/2025-11/msg02947.html
* Implement a fake VDUSE suspend vdpa operation callback that always returns true in the kernel. DPDK suspend the device at the firstGET_VRING_BASE.
* Remove the CVQ blocker in ASID.The driver
vhost_vdpawas also tested with version 0, version 1 with the old ioctl, version 1 with the new ioctl but only one ASID, and version 1 with many ASID.---
v12:
* Using scoped guards for vq group rwlock, so the one queue optimization is not missed (Jason proposed to factor them into helpers).
* Add the_v2suffix tovduse_iova_range_v2struct name fixing the doc (MST).
* s/verion/version/ in patch message.
* Remove trailing ; after a comment (Jason).v11:
* Remove duplicatedfree_pages_exactinvduse_domain_free_coherent(Jason).
* Do not take the vq groups lock if nas == 1.
* Do not reset the vq group ASID in vq reset (Jason). Removed extra functionvduse_set_group_asid_nomsg, not needed anymore.
* Move thevduse_iotlb_entry_v2argument to a new ioctl, as argument didn't match the previousVDUSE_IOTLB_GET_FD.
* Move the asid <dev->nascheck to vdpa core.v10:
* Back to rwlock version so stronger locks are used.
* Take out allocations from rwlock.
* Forbid changing ASID of a vq group afterDRIVER_OK(Jason)
* Remove bad fetching again of domain variable invduse_dev_max_mapping_size(Yongji).
* Remove unused vdev definition in vdpamap_opscallbacks (kernel test robot).v9:
* Replace mutex with rwlock, as the vdpamap_opscan run from atomic context.v8:
* Revert the mutex to rwlock change, it needs proper profiling to justify it.v7:
* Take write lock in the error path (Jason).v6:
* Makevdpa_dev_adduse gotos for error handling (MST).
* s/(dev->api_version < 1) ?/(dev->api_version <VDUSE_API_VERSION_1) ?/ (MST).
* Fix struct name not matching in the doc.v5:
* Properly return errno ifcopy_to_userreturns >0 inVDUSE_IOTLB_GET_FDioctl (Jason).
* Properly set domain bounce size to divide equally between nas (Jason).
* Exclude "padding" member from the only >V1 members invduse_dev_request.v4:
* Divide each domain bounce size between the device bounce size (Jason).
* revert unneeded addr = NULL assignment (Jason)
* Change if (x && (y || z)) return to if (x) { if (y) return; if (z) return; } (Jason)
* Change a bad multiline comment, using @ caracter instead of * (Jason).
* Considerconfig->nas== 0 as a fail (Jason).v3:
* Get the vduse domain through thevduse_asin the map functions (Jason).
* Squash with the patch creating thevduse_asstruct (Jason).
* CreateVDUSE_DEV_MAX_ASinstead of comparing agains a magic number (Jason)v2:
* Convert the use of mutex to rwlock.RFC v3:
* IncreaseVDUSE_MAX_VQ_GROUPSto0xffff(Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about mergingVDUSE_IOTLB_GET_FDioctl withVDUSE_IOTLB_GET_INFO.
* Use ofarray_index_nospecin VDUSE device ioctls.
* Embedvduse_iotlb_entryintovduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.RFC v2:
* Make iotlb entry the last one ofvduse_iotlb_entry_v2so the first part of the struct is the same.--- drivers/vdpa/vdpa_user/vduse_dev.c | 385 +++++++++++++++++++---------- include/uapi/linux/vduse.h | 63 ++++- 2 files changed, 312 insertions(+), 136 deletions(-)drivers/vdpa/vdpa_user/vduse_dev.c +80
-36@@ -9,6 +9,7 @@*/ #include "linux/virtio_net.h" +#include <linux/cleanup.h> #include <linux/init.h> #include <linux/module.h> #include <linux/cdev.h> @@ -41,6 +42,7 @@#define VDUSE_DEV_MAX (1U << MINORBITS)#define VDUSE_DEV_MAX_GROUPS 0xffff +#define VDUSE_DEV_MAX_AS 0xffff#define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)#define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)#define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)@@ -86,7 +88,15 @@ struct vduse_umem {struct mm_struct *mm; }; +struct vduse_as { +struct vduse_iova_domain *domain; +struct vduse_umem *umem; +struct mutex mem_lock; +}; +struct vduse_vq_group { +rwlock_t as_lock; +struct vduse_as *as;/* Protected by as_lock */ struct vduse_dev *dev; };@@ -94,7 +104,7 @@ struct vduse_dev {struct vduse_vdpa *vdev;struct device *dev;struct vduse_virtqueue **vqs; -struct vduse_iova_domain *domain; +struct vduse_as *as;char *name;struct mutex lock;spinlock_t msg_lock;@@ -122,9 +132,8 @@ struct vduse_dev {u32 vq_num;u32 vq_align;u32 ngroups; -struct vduse_umem *umem; +u32 nas;struct vduse_vq_group *groups; -struct mutex mem_lock;unsigned int bounce_size;struct mutex domain_lock; };@@ -314,7 +323,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)return vduse_dev_msg_sync (dev, &msg); } -static int vduse_dev_update_iotlb (struct vduse_dev *dev, +static int vduse_dev_update_iotlb (struct vduse_dev *dev,u32 asid,u64 start,u64 last) {struct vduse_dev_msg msg = {0 };@@ -323,8 +332,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,return -EINVAL ; msg.req.type =VDUSE_UPDATE_IOTLB ; - msg.req.iova.start = start; - msg.req.iova.last = last; +if (dev->api_version <VDUSE_API_VERSION_1 ) { + msg.req.iova.start = start; + msg.req.iova.last = last; + }else { + msg.req.iova_v2.start = start; + msg.req.iova_v2.last = last; + msg.req.iova_v2.asid = asid; + }return vduse_dev_msg_sync (dev, &msg); }@@ -439,11 +454,14 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)static void vduse_dev_reset (struct vduse_dev *dev) {int i; -struct vduse_iova_domain *domain = dev->domain;/* The coherent mappings are handled in vduse_dev_free_coherent() */ -if (domain && domain->bounce_map) -vduse_domain_reset_bounce_map (domain); +for (i =0 ; i < dev->nas; i++) { +struct vduse_iova_domain *domain = dev->as[i].domain; + +if (domain && domain->bounce_map) +vduse_domain_reset_bounce_map (domain); + }down_write (&dev->rwsem);@@ -622,6 +640,42 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)return ret; } +DEFINE_GUARD(vq_group_as_read_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +read_lock (&_T->as_lock), + if (_T->dev->nas > 1) + read_unlock(&_T->as_lock)) + +DEFINE_GUARD(vq_group_as_write_lock, struct vduse_vq_group *, + if (_T->dev->nas > 1) +write_lock (&_T->as_lock), + if (_T->dev->nas > 1) + write_unlock(&_T->as_lock)) + +static int vduse_set_group_asid (struct vdpa_device *vdpa,unsigned int group, +unsigned int asid) +{ +struct vduse_dev *dev =vdpa_to_vduse (vdpa); +struct vduse_dev_msg msg = {0 }; +int r; + +if (dev->api_version <VDUSE_API_VERSION_1 ) +return -EINVAL ; + + msg.req.type =VDUSE_SET_VQ_GROUP_ASID ; + msg.req.vq_group_asid.group = group; + msg.req.vq_group_asid.asid = asid; + + r =vduse_dev_msg_sync (dev, &msg); +if (r <0 ) +return r; + +guard (vq_group_as_write_lock)(&dev->groups[group]); + dev->groups[group].as = &dev->as[asid]; + +return 0 ; +} +static int vduse_vdpa_get_vq_state (struct vdpa_device *vdpa,u16 idx,struct vdpa_vq_state *state) {@@ -793,13 +847,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,struct vduse_dev *dev =vdpa_to_vduse (vdpa);int ret; - ret =vduse_domain_set_map (dev->domain, iotlb); + ret =vduse_domain_set_map (dev->as[asid].domain, iotlb);if (ret)return ret; - ret =vduse_dev_update_iotlb (dev,0ULL ,ULLONG_MAX ); + ret =vduse_dev_update_iotlb (dev, asid,0ULL ,ULLONG_MAX );if (ret) { -vduse_domain_clear_map (dev->domain, iotlb); +vduse_domain_clear_map (dev->as[asid].domain, iotlb);return ret; }@@ -842,6 +896,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = { .get_vq_affinity = vduse_vdpa_get_vq_affinity, .reset = vduse_vdpa_reset, .set_map = vduse_vdpa_set_map, + .set_group_asid = vduse_set_group_asid, .get_vq_map = vduse_get_vq_map, .free = vduse_vdpa_free, };@@ -850,15 +905,13 @@ static void vduse_dev_sync_single_for_device(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_device (domain, dma_addr, size, dir); }@@ -866,15 +919,13 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;vduse_domain_sync_single_for_cpu (domain, dma_addr, size, dir); }@@ -883,15 +934,13 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page, enum dma_data_direction dir, unsigned long attrs) { -struct vduse_dev *vdev;struct vduse_iova_domain *domain;if (!token.group)return DMA_MAPPING_ERROR ; - vdev = token.group->dev; - domain = vdev->domain; - +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain;return vduse_domain_map_page (domain, page, offset, size, dir, attrs); }@@ -899,23 +948,19 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_dev *vdev; struct vduse_iova_domain *domain;if (!token.group)return ; - vdev = token.group->dev; - domain = vdev->domain; - -return vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); +guard (vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; +vduse_domain_unmap_page (domain, dma_addr, size, dir, attrs); }static void *vduse_dev_alloc_coherent (union virtio_map token,size_t size,dma_addr_t *dma_addr,gfp_t flag) { -struct vduse_dev *vdev; -struct vduse_iova_domain *domain;void *addr; *dma_addr =DMA_MAPPING_ERROR ;@@ -926,11 +971,15 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,if (!addr)return NULL ; - vdev = token.group->dev; - domain = vdev->domain; - *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); -if (*dma_addr ==DMA_MAPPING_ERROR ) - goto err; + { +struct vduse_iova_domain *domain; + +guard (vq_group_as_read_lock)(token.group);
Quoted 9 lines
drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 982
Quoted 5 lines
drivers/vdpa/vdpa_user/vduse_dev.c +5
-0+ domain = token.group->as->domain; + *dma_addr =vduse_domain_alloc_coherent (domain, size, addr); +if (*dma_addr ==DMA_MAPPING_ERROR ) + goto err; + }Nit: Having a standalone block probably means we have the chance to optimize the code, maybe using a helper?
What do you mean by optimize the code? Would you prefer vduse_dev_alloc_coherent function to be changed like this (untested)?
(quoted diff) +0 -0
b/drivers/vdpa/vdpa_user/vduse_dev.c
index d658f3e1cebf..3561d3c00c89 100644
drivers/vdpa/vdpa_user/vduse_dev.c +0 -0
@@ -911,11 +911,19 @@ static void vduse_dev_unmap_page(union
virtio_map token, dma_addr_t dma_addr,
return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
}
+static dma_addr_t *vduse_vq_group_alloc_coherent(struct vduse_vq_group *group,
+ size_t size, void *addr)
+{
+ struct vduse_iova_domain *domain;
+
+ guard(vq_group_as_read_lock)(token.group); + domain = token.group->as->domain; + return vduse_domain_alloc_coherent(domain, size, addr);
+}
+
static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
dma_addr_t *dma_addr, gfp_t flag)
{
- struct vduse_dev *vdev;
- struct vduse_iova_domain *domain;
void *addr;
*dma_addr = DMA_MAPPING_ERROR;
@@ -926,9 +934,7 @@ static void *vduse_dev_alloc_coherent(union
virtio_map token, size_t size,
if (!addr)
return NULL;- vdev = token.group->dev;
- domain = vdev->domain;
- *dma_addr = vduse_domain_alloc_coherent(domain, size, addr);
+ *dma_addr = vduse_vq_group_alloc_coherent(token.group, size, addr);
if (*dma_addr == DMA_MAPPING_ERROR)
goto err;---
I don't love single use functions but I'm ok with the change if that's what you want.
Eugenio Pérez
maintainerPATCH v12 12/13 vduse: bump version number
In reply to Eugenio PérezFinalize the series by advertising VDUSE API v1 support to userspace.
Now that all required infrastructure for v1 (ASIDs, VQ groups, update_iotlb_v2) is in place, VDUSE devices can opt in to the new features.
Assume API version 0 if the VDUSE instance does not call VDUSE_GET_API_VERSION to maintain compatibility.
---
v12:
* Assume API version 0 by default if the VDUSE instance does not ask (Jason).
--- drivers/vdpa/vdpa_user/vduse_dev.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +15 -2
@@ -52,6 +52,15 @@#define IRQ_UNBOUND -1 +/* +* VDUSE instance have not asked the vduse API version, so assume 0. +* +* Old devices may not ask for the device version and assume it is 0. Keep +* this value for these. From the moment the VDUSE instance ask for the +* version, convert to the latests supported one and continue regular flow +*/ +#define VDUSE_API_VERSION_NOT_ASKED U64_MAX +struct vduse_virtqueue {u16 index;u16 num_max;
@@ -2153,6 +2162,8 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,mutex_lock (&vduse_lock);switch (cmd) {case VDUSE_GET_API_VERSION : +if (control->api_version ==VDUSE_API_VERSION_NOT_ASKED ) + control->api_version =VDUSE_API_VERSION_1 ; ret =put_user (control->api_version, (u64 __user *)argp);break ;case VDUSE_SET_API_VERSION : {
@@ -2163,7 +2174,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,break ; ret = -EINVAL ; -if (api_version >VDUSE_API_VERSION ) +if (api_version >VDUSE_API_VERSION_1 )break ; ret =0 ;
@@ -2180,6 +2191,8 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,break ; ret = -EINVAL ; +if (control->api_version ==VDUSE_API_VERSION_NOT_ASKED ) + control->api_version =VDUSE_API_VERSION ;if (!vduse_validate_config (&config, control->api_version))break ;
@@ -2230,7 +2243,7 @@ static int vduse_open(struct inode *inode, struct file *file)if (!control)return -ENOMEM ; - control->api_version =VDUSE_API_VERSION ; + control->api_version =VDUSE_API_VERSION_NOT_ASKED ; file->private_data = control;return 0 ;
Signature · 1 line
2.52.0
Re: [PATCH v12 12/13] vduse: bump version number
Quoted 7 lines · Eugenio Pérez
Finalize the series by advertising VDUSE API v1 support to userspace.
Now that all required infrastructure for v1 (ASIDs, VQ groups,
update_iotlb_v2) is in place, VDUSE devices can opt in to the new features.
Quoted 5 lines
Assume API version 0 if the VDUSE instance does not call
VDUSE_GET_API_VERSIONto maintain compatibility.Signed-off-by Eugenio Pérez <[email protected]>---
Thanks
Eugenio Pérez
maintainerPATCH v12 13/13 Documentation: Add documentation for VDUSE Address Space IDs
In reply to Eugenio PérezAddress Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
---
v12: New in V12. Requested by Jason.
--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/userspace-api/vduse.rst +49 -0
@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled. +Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup it with ioctl(VDUSE_SET_API_VERSION) on `/dev/vduse/control` and +pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng. Similarly, you can use +ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVA +region of an specific ASID. Example usage: + +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; + + /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the + * cached mapping when the corresponding VDUSE_UPDATE_IOTLB + * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } + For more details on the uAPI, please see include/uapi/linux/vduse.h.
Signature · 1 line
2.52.0
Re: [PATCH v12 13/13] Documentation: Add documentation for VDUSE Address Space IDs
Quoted 26 lines · Eugenio Pérez
Address Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
Signed-off-by Eugenio Pérez <[email protected]>---
v12: New in V12. Requested by Jason.--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)Documentation/userspace-api/vduse.rst +0
-0@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled.
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 242
Quoted 10 lines
Documentation/userspace-api/vduse.rst +10
-0+Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup it with ioctl(VDUSE_SET_API_VERSION) on `/dev/vduse/control` and +pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng. Similarly, you can use
Should be "querying"
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 243
Quoted 1 line
Documentation/userspace-api/vduse.rst +1
-0+ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVA
Should we fail VDUSE_IOTLB_GET_FD on API version 1 or it is implied for ASID 0?
Quoted 28 lines
Documentation/userspace-api/vduse.rst +28
-0+region of an specific ASID. Example usage: + +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; +
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 281
Quoted 10 lines
Documentation/userspace-api/vduse.rst +10
-0+ /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the + * cached mapping when the corresponding VDUSE_UPDATE_IOTLB + * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } +
It looks like we missed the documentation of VDUSE_SET_VQ_GROUP_ASID.
Quoted 4 lines
Documentation/userspace-api/vduse.rst +0
-1For more details on the uAPI, please see include/uapi/linux/vduse.h. --2.52.0
Thanks
Eugenio Perez Martin
maintainerRe: [PATCH v12 13/13] Documentation: Add documentation for VDUSE Address Space IDs
Quoted 28 lines · Jason Wang
Quoted 26 lines · Eugenio Pérez
Address Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
Signed-off-by Eugenio Pérez <[email protected]>---
v12: New in V12. Requested by Jason.--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)Documentation/userspace-api/vduse.rst +0
-0@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled.
Quoted 13 lines
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 242
Quoted 10 lines
Documentation/userspace-api/vduse.rst +10
-0+Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup it with ioctl(VDUSE_SET_API_VERSION) on `/dev/vduse/control` and +pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng. Similarly, you can useShould be "querying"
Right, fixing the typos for the next version, thanks!
Quoted 36 lines
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 243
Quoted 1 line
Documentation/userspace-api/vduse.rst +1
-0+ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVAShould we fail
VDUSE_IOTLB_GET_FDon API version 1 or it is implied for ASID 0?Quoted 32 lines
Documentation/userspace-api/vduse.rst +32
-0+region of an specific ASID. Example usage: + +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; + + /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the + * cached mapping when the corresponding VDUSE_UPDATE_IOTLB
Quoted 9 lines
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 281
Quoted 6 lines
Documentation/userspace-api/vduse.rst +6
-0+ * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } +It looks like we missed the documentation of
VDUSE_SET_VQ_GROUP_ASID.
Sure, adding it for the next version. Thanks!
Eugenio Perez Martin
maintainerRe: [PATCH v12 13/13] Documentation: Add documentation for VDUSE Address Space IDs
In reply to Jason WangQuoted 41 lines · Jason Wang
Quoted 26 lines · Eugenio Pérez
Address Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
Signed-off-by Eugenio Pérez <[email protected]>---
v12: New in V12. Requested by Jason.--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)Documentation/userspace-api/vduse.rst +0
-0@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled.Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 242
Quoted 10 lines
Documentation/userspace-api/vduse.rst +10
-0+Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup it with ioctl(VDUSE_SET_API_VERSION) on `/dev/vduse/control` and +pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng. Similarly, you can useShould be "querying"
Quoted 4 lines
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 243
Quoted 1 line
Documentation/userspace-api/vduse.rst +1
-0+ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVAShould we fail
VDUSE_IOTLB_GET_FDon API version 1 or it is implied for ASID 0?
ASID 0 is implied at this moment, so future VDUSE devices don't need to worry about ASID configuration.
Quoted 48 lines
Quoted 28 lines
Documentation/userspace-api/vduse.rst +28
-0+region of an specific ASID. Example usage: + +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; +Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 281
Quoted 10 lines
Documentation/userspace-api/vduse.rst +10
-0+ /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the + * cached mapping when the corresponding VDUSE_UPDATE_IOTLB + * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } +It looks like we missed the documentation of
VDUSE_SET_VQ_GROUP_ASID.Quoted 4 lines
Documentation/userspace-api/vduse.rst +0
-1For more details on the uAPI, please see include/uapi/linux/vduse.h. --2.52.0
Thanks
Michael S. Tsirkin
maintainerRe: [PATCH v12 13/13] Documentation: Add documentation for VDUSE Address Space IDs
In reply to Eugenio PérezQuoted 5 lines · Eugenio Pérez
Address Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest
passthrough
Quoted 17 lines
intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
Signed-off-by Eugenio Pérez <[email protected]>---
v12: New in V12. Requested by Jason.--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)Documentation/userspace-api/vduse.rst +0
-0
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 237
Quoted 8 lines
Documentation/userspace-api/vduse.rst +5
-0@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled. +Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup it
Set it up
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows:
Quoted 6 lines
with
ioctl(VDUSE_SET_API_VERSION)on/dev/vduse/controlandDocumentation/userspace-api/vduse.rst +5
-0+pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng.
querying
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 244
Quoted 3 lines
Similarly, you can use
Documentation/userspace-api/vduse.rst +2
-0+ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVA +region of an specific ASID. Example usage:
a specific
Quoted 40 lines
Documentation/userspace-api/vduse.rst +37
-0+ +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; + + /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the + * cached mapping when the corresponding VDUSE_UPDATE_IOTLB + * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } + For more details on the uAPI, please see include/uapi/linux/vduse.h.Signature · 1 line
2.52.0
Eugenio Perez Martin
maintainerRe: [PATCH v12 13/13] Documentation: Add documentation for VDUSE Address Space IDs
Quoted 83 lines · Michael S. Tsirkin
Quoted 5 lines · Eugenio Pérez
Address Space IDs allows the VDUSE framework to support devices able to expose different virtqueues to different part of the drivers. For example, to let QEMU handle the net device control virtqueue, so QEMU always knows the state of the device like mac address or number of queues enabled, while leaving the dataplane passtrhough to the guest
passthrough
Quoted 17 lines
intact. This enables live migration.
Expands the VDUSE documentation to explain how to use the new ioctls or the new struct members of old ioctls.
Signed-off-by Eugenio Pérez <[email protected]>---
v12: New in V12. Requested by Jason.--- Documentation/userspace-api/vduse.rst | 49 +++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)Documentation/userspace-api/vduse.rst +0
-0Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 237
Quoted 8 lines
Documentation/userspace-api/vduse.rst +5
-0@@ -230,4 +230,53 @@ able to start the dataplane processing as follows: 5. Inject an interrupt for specific virtqueue with the VDUSE_INJECT_VQ_IRQ ioctl after the used ring is filled. +Enabling ASID (API version 1) +------------------------------ + +VDUSE supports per-address-space identifiers (ASIDs) starting with API +version 1. Setup itSet it up
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows:
Quoted 6 lines
with
ioctl(VDUSE_SET_API_VERSION)on/dev/vduse/controlandDocumentation/userspace-api/vduse.rst +5
-0+pass `VDUSE_API_VERSION_1` before creating a new VDUSE instance with +ioctl(VDUSE_CREATE_DEV). + +Afterwards, you can use the member asid of ioctl(VDUSE_IOTLB_GET_INFO) to +select the address space of the IOTLB you are queryng.querying
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 244
Quoted 3 lines
Similarly, you can use
Documentation/userspace-api/vduse.rst +2
-0+ioctl(VDUSE_IOTLB_GET_FD2) to obtain the file descriptor describing an IOVA +region of an specific ASID. Example usage:a specific
Quoted 30 lines
Documentation/userspace-api/vduse.rst +30
-0+ +.. code-block:: c + + static void *iova_to_va(int dev_fd, uint32_t asid, uint64_t iova, + uint64_t *len) + { + int fd; + void *addr; + size_t size; + struct vduse_iotlb_entry_v2 entry = { 0 }; + + entry.v1.start = iova; + entry.v1.last = iova; + entry.asid = asid; + + fd = ioctl(dev_fd, VDUSE_IOTLB_GET_FD2, &entry); + if (fd < 0) + return NULL; + + size = entry.v1.last - entry.v1.start + 1; + *len = entry.v1.last - iova + 1; + addr = mmap(0, size, perm_to_prot(entry.v1.perm), MAP_SHARED, + fd, entry.v1.offset); + close(fd); + if (addr == MAP_FAILED) + return NULL; + + /* + * Using some data structures such as linked list to store + * the iotlb mapping. The munmap(2) should be called for the
Quoted 8 lines
Documentation/userspace-api/vduse.rst able to start the dataplane processing as follows: · line 282
Quoted 8 lines
Documentation/userspace-api/vduse.rst +7
-0+ * cached mapping when the corresponding VDUSE_UPDATE_IOTLB + * message is received or the device is reset. + */ + + return addr + iova - entry.start; + } + For more details on the uAPI, please see include/uapi/linux/vduse.h.
Fixing typos for the next version, thanks!