Skip to contentSkip to conversation

Add multiple address spaces support to VDUSE

PATCH Superseded Eugenio Pérez 3 participants 18 messages Sep 19, 2025 – Sep 24, 2025

Subsystems (4)

v3 of 15 Superseded Latest is v15 →

lore
Share & tools
lore

Eugenio Pérez

author maintainer
#0 lore raw

v3 cover Add multiple address spaces support to VDUSE

When used by vhost-vDPA bus driver for VM, the control virtqueue should be shadowed via userspace VMM (QEMU) instead of being assigned directly to Guest. This is because QEMU needs to know the device state in order to start and stop device correctly (e.g for Live Migration).

This requies to isolate the memory mapping for control virtqueue presented by vhost-vDPA to prevent guest from accessing it directly.

This series add support to multiple address spaces in VDUSE device allowing selective virtqueue isolation through address space IDs (ASID).

The VDUSE device needs to report:
* Number of virtqueue groups
* Association of each vq group with each virtqueue
* Number of address spaces supported.

Then, the vDPA driver can modify the ASID assigned to each VQ group to isolate the memory AS. This aligns VDUSE with vdpa_sim and nvidia mlx5 devices which already support ASID.

This helps to isolate the environments for the virtqueues that will not be assigned directly. E.g in the case of virtio-net, the control virtqueue will not be assigned directly to guest.

Also, to be able to test this patch, the user needs to manually revert 56e71885b034 ("vduse: Temporarily fail if control queue feature requested").

PATCH v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason). Adapt all the virtio_map_ops callbacks to it.
* Make setting status DRIVER_OK fail if vq group is not valid.
* Create the VDUSE_DEV_MAX_GROUPS and VDUSE_DEV_MAX_AS instead of using a magic number
* Remove the _int name suffix from struct vduse_vq_group.
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash the patch implementing the AS logic with the patch creating the vduse_as struct (Jason).

PATCH v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.
* Convert the use of mutex to rwlock (Xie Yongji).

PATCH v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)
* Using vduse_vq_group_int directly instead of an empty struct in union virtio_map.

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs. Also, it's a dynamic array now. Same with ASID.
* Move the valid vq groups range check to vduse_validate_config.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Use of array_index_nospec in VDUSE device ioctls.
* Move the umem mutex to asid struct so there is no contention between ASIDs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.

RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.
* Fixes detected testing with OVS+VDUSE.

Eugenio Pérez (6):
vduse: make domain_lock an rwlock
vduse: add v1 API definition
vduse: add vq group support
vduse: return internal vq group struct as map token
vduse: add vq group asid support
vduse: bump version number

 drivers/vdpa/ifcvf/ifcvf_main.c    |   2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |   2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c   |   2 +-
 drivers/vdpa/vdpa_user/vduse_dev.c | 506 ++++++++++++++++++++++-------
 drivers/vhost/vdpa.c               |  11 +-
 include/linux/vdpa.h               |   5 +-
 include/linux/virtio.h             |   6 +-
 include/uapi/linux/vduse.h         |  65 +++-
 8 files changed, 467 insertions(+), 132 deletions(-)
Signature · 1 line
2.51.0
author maintainer
#1 lore raw

PATCH v3 1/6 vduse: make domain_lock an rwlock

It will be used in a few more scenarios read-only so make it more scalable.

---
v2: New in v2

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 41 +++++++++++++++---------------
 1 file changed, 21 insertions(+), 20 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +21 -20
index e7bced0b5542..2b6a8958ffe0 100644
@@ -14,6 +14,7 @@
 #include <linux/cdev.h>
 #include <linux/device.h>
 #include <linux/eventfd.h>
+#include <linux/rwlock.h>
 #include <linux/slab.h>
 #include <linux/wait.h>
 #include <linux/dma-map-ops.h>
@@ -117,7 +118,7 @@ struct vduse_dev {
 	struct vduse_umem *umem;
 	struct mutex mem_lock;
 	unsigned int bounce_size;
-	struct mutex domain_lock;
+	rwlock_t domain_lock;
 };
 
 struct vduse_dev_msg {
@@ -1176,9 +1177,9 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 		if (entry.start > entry.last)
 			break;
 
-		mutex_lock(&dev->domain_lock);
+		read_lock(&dev->domain_lock);
 		if (!dev->domain) {
-			mutex_unlock(&dev->domain_lock);
+			read_unlock(&dev->domain_lock);
 			break;
 		}
 		spin_lock(&dev->domain->iotlb_lock);
@@ -1193,7 +1194,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 			entry.perm = map->perm;
 		}
 		spin_unlock(&dev->domain->iotlb_lock);
-		mutex_unlock(&dev->domain_lock);
+		read_unlock(&dev->domain_lock);
 		ret = -EINVAL;
 		if (!f)
 			break;
@@ -1346,10 +1347,10 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 				 sizeof(umem.reserved)))
 			break;
 
-		mutex_lock(&dev->domain_lock);
+		write_lock(&dev->domain_lock);
 		ret = vduse_dev_reg_umem(dev, umem.iova,
 					 umem.uaddr, umem.size);
-		mutex_unlock(&dev->domain_lock);
+		write_unlock(&dev->domain_lock);
 		break;
 	}
 	case VDUSE_IOTLB_DEREG_UMEM: {
@@ -1363,10 +1364,10 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 		if (!is_mem_zero((const char *)umem.reserved,
 				 sizeof(umem.reserved)))
 			break;
-		mutex_lock(&dev->domain_lock);
+		write_lock(&dev->domain_lock);
 		ret = vduse_dev_dereg_umem(dev, umem.iova,
 					   umem.size);
-		mutex_unlock(&dev->domain_lock);
+		write_unlock(&dev->domain_lock);
 		break;
 	}
 	case VDUSE_IOTLB_GET_INFO: {
@@ -1385,9 +1386,9 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 				 sizeof(info.reserved)))
 			break;
 
-		mutex_lock(&dev->domain_lock);
+		read_lock(&dev->domain_lock);
 		if (!dev->domain) {
-			mutex_unlock(&dev->domain_lock);
+			read_unlock(&dev->domain_lock);
 			break;
 		}
 		spin_lock(&dev->domain->iotlb_lock);
@@ -1402,7 +1403,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 				info.capability |= VDUSE_IOVA_CAP_UMEM;
 		}
 		spin_unlock(&dev->domain->iotlb_lock);
-		mutex_unlock(&dev->domain_lock);
+		read_unlock(&dev->domain_lock);
 		if (!map)
 			break;
 
@@ -1425,10 +1426,10 @@ static int vduse_dev_release(struct inode *inode, struct file *file)
 {
 	struct vduse_dev *dev = file->private_data;
 
-	mutex_lock(&dev->domain_lock);
+	write_lock(&dev->domain_lock);
 	if (dev->domain)
 		vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size);
-	mutex_unlock(&dev->domain_lock);
+	write_unlock(&dev->domain_lock);
 	spin_lock(&dev->msg_lock);
 	/* Make sure the inflight messages can processed after reconncection */
 	list_splice_init(&dev->recv_list, &dev->send_list);
@@ -1647,7 +1648,7 @@ static struct vduse_dev *vduse_dev_create(void)
 
 	mutex_init(&dev->lock);
 	mutex_init(&dev->mem_lock);
-	mutex_init(&dev->domain_lock);
+	rwlock_init(&dev->domain_lock);
 	spin_lock_init(&dev->msg_lock);
 	INIT_LIST_HEAD(&dev->send_list);
 	INIT_LIST_HEAD(&dev->recv_list);
@@ -1805,7 +1806,7 @@ static ssize_t bounce_size_store(struct device *device,
 	int ret;
 
 	ret = -EPERM;
-	mutex_lock(&dev->domain_lock);
+	write_lock(&dev->domain_lock);
 	if (dev->domain)
 		goto unlock;
 
@@ -1821,7 +1822,7 @@ static ssize_t bounce_size_store(struct device *device,
 	dev->bounce_size = bounce_size & PAGE_MASK;
 	ret = count;
 unlock:
-	mutex_unlock(&dev->domain_lock);
+	write_unlock(&dev->domain_lock);
 	return ret;
 }
 
@@ -2045,11 +2046,11 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
 	if (ret)
 		return ret;
 
-	mutex_lock(&dev->domain_lock);
+	write_lock(&dev->domain_lock);
 	if (!dev->domain)
 		dev->domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
 						  dev->bounce_size);
-	mutex_unlock(&dev->domain_lock);
+	write_unlock(&dev->domain_lock);
 	if (!dev->domain) {
 		put_device(&dev->vdev->vdpa.dev);
 		return -ENOMEM;
@@ -2059,10 +2060,10 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
 	ret = _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num);
 	if (ret) {
 		put_device(&dev->vdev->vdpa.dev);
-		mutex_lock(&dev->domain_lock);
+		write_lock(&dev->domain_lock);
 		vduse_domain_destroy(dev->domain);
 		dev->domain = NULL;
-		mutex_unlock(&dev->domain_lock);
+		write_unlock(&dev->domain_lock);
 		return ret;
 	}
 
Signature · 1 line
2.51.0
author maintainer
#2 lore raw

PATCH v3 2/6 vduse: add v1 API definition

In reply to Eugenio Pérez

This allows the kernel to detect whether the userspace VDUSE device supports the VQ group and ASID features. VDUSE devices that don't set the V1 API will not receive the new messages, and vdpa device will be created with only one vq group and asid.

The next patches implement the new feature incrementally, only enabling the VDUSE device to set the V1 API version by the end of the series.

---
 include/uapi/linux/vduse.h | 4 ++++
 1 file changed, 4 insertions(+)
include/uapi/linux/vduse.h +4 -0
index 10ad71aa00d6..ccb92a1efce0 100644
@@ -10,6 +10,10 @@
 
 #define VDUSE_API_VERSION	0
 
+/* VQ groups and ASID support */
+
+#define VDUSE_API_VERSION_1	1
+
 /*
  * Get the version of VDUSE API that kernel supported (VDUSE_API_VERSION).
  * This is used for future extension.
Signature · 1 line
2.51.0
author maintainer
#3 lore raw

PATCH v3 3/6 vduse: add vq group support

In reply to Eugenio Pérez

This allows sepparate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API.

Allocating 3 vq groups as net is the device that need the most groups:
* Dataplane (guest passthrough)
* CVQ
* Shadowed vrings.

Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups.

---
v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason).
* Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number

v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.

v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Move the valid vq groups range check to vduse_validate_config.

RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.

---
 drivers/vdpa/ifcvf/ifcvf_main.c    |  2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |  2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c   |  2 +-
 drivers/vdpa/vdpa_user/vduse_dev.c | 51 +++++++++++++++++++++++++++---
 drivers/vhost/vdpa.c               | 11 +++++--
 include/linux/vdpa.h               |  5 +--
 include/uapi/linux/vduse.h         | 12 +++++--
 7 files changed, 69 insertions(+), 16 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +1 -1
index 6658dc74d915..ea6643760fb7 100644
@@ -559,7 +559,7 @@ static size_t ifcvf_vdpa_get_config_size(struct vdpa_device *vdpa_dev)
 	return  vf->config_size;
 }
 
-static u32 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
 	return 0;
 }
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -1
index 82034efb74fc..daab0ccb78b2 100644
@@ -2640,7 +2640,7 @@ static u32 mlx5_vdpa_get_vq_align(struct vdpa_device *vdev)
 	return PAGE_SIZE;
 }
 
-static u32 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
+static s64 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
 {
 	struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
 
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
index c1c6431950e1..fe4d65aaf6aa 100644
@@ -412,7 +412,7 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)
 	return VDPASIM_QUEUE_ALIGN;
 }
 
-static u32 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
 	/* RX and TX belongs to group 0, CVQ belongs to group 1 */
 	if (idx == 2)
drivers/vdpa/vdpa_user/vduse_dev.c +46 -5

132 lines not on this page · raw message

drivers/vhost/vdpa.c +8 -3
index 05a481e4c385..6305382eacbb 100644
@@ -22,6 +22,7 @@
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
 #include <linux/vhost.h>
+#include <linux/types.h>
 
 #include "vhost.h"
 
@@ -657,16 +658,20 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
 			return -EFAULT;
 		ops->set_vq_ready(vdpa, idx, s.num);
 		return 0;
-	case VHOST_VDPA_GET_VRING_GROUP:
+	case VHOST_VDPA_GET_VRING_GROUP: {
+		u64 group;
+
 		if (!ops->get_vq_group)
 			return -EOPNOTSUPP;
 		s.index = idx;
-		s.num = ops->get_vq_group(vdpa, idx);
-		if (s.num >= vdpa->ngroups)
+		group = ops->get_vq_group(vdpa, idx);
+		if (group >= vdpa->ngroups || group > U32_MAX || group < 0)
 			return -EIO;
 		else if (copy_to_user(argp, &s, sizeof(s)))
 			return -EFAULT;
+		s.num = group;
 		return 0;
+	}
 	case VHOST_VDPA_GET_VRING_DESC_GROUP:
 		if (!vhost_vdpa_has_desc_group(v))
 			return -EOPNOTSUPP;
include/linux/vdpa.h +3 -2
index 4cf21d6e9cfd..76a673ef7deb 100644
@@ -211,7 +211,8 @@ struct vdpa_map_file {
  *				virtqueue (optional)
  *				@vdev: vdpa device
  *				@idx: virtqueue index
- *				Returns u32: group id for this virtqueue
+ *				Returns i64: (32 bits wide) group id for this
+ *				virtqueue if >0. If <0, errno.
  * @get_vq_desc_group:		Get the group id for the descriptor table of
  *				a specific virtqueue (optional)
  *				@vdev: vdpa device
@@ -398,7 +399,7 @@ struct vdpa_config_ops {
 
 	/* Device ops */
 	u32 (*get_vq_align)(struct vdpa_device *vdev);
-	u32 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
+	s64 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
 	u32 (*get_vq_desc_group)(struct vdpa_device *vdev, u16 idx);
 	u64 (*get_device_features)(struct vdpa_device *vdev);
 	u64 (*get_backend_features)(const struct vdpa_device *vdev);
include/uapi/linux/vduse.h +9 -3
index ccb92a1efce0..a3d51cf6df3a 100644
@@ -31,6 +31,7 @@
  * @features: virtio features
  * @vq_num: the number of virtqueues
  * @vq_align: the allocation alignment of virtqueue's metadata
+ * @ngroups: number of vq groups that VDUSE device declares
  * @reserved: for future use, needs to be initialized to zero
  * @config_size: the size of the configuration space
  * @config: the buffer of the configuration space
@@ -45,7 +46,8 @@ struct vduse_dev_config {
 	__u64 features;
 	__u32 vq_num;
 	__u32 vq_align;
-	__u32 reserved[13];
+	__u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
+	__u32 reserved[12];
 	__u32 config_size;
 	__u8 config[];
 };
@@ -122,14 +124,18 @@ struct vduse_config_data {
  * struct vduse_vq_config - basic configuration of a virtqueue
  * @index: virtqueue index
  * @max_size: the max size of virtqueue
- * @reserved: for future use, needs to be initialized to zero
+ * @reserved1: for future use, needs to be initialized to zero
+ * @group: virtqueue group
+ * @reserved2: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
  */
 struct vduse_vq_config {
 	__u32 index;
 	__u16 max_size;
-	__u16 reserved[13];
+	__u16 reserved1;
+	__u32 group;
+	__u16 reserved2[10];
 };
 
 /*
Signature · 1 line
2.51.0
#4 lore raw

Re: [PATCH v3 3/6] vduse: add vq group support

kernel test robot noticed the following build errors:

Hi Eugenio,

kernel test robot noticed the following build errors:

[auto build test ERROR on mst-vhost/linux-next]
[also build test ERROR on next-20250919]
[cannot apply to linus/master v6.17-rc6]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Eugenio-P-rez/vduse-make-domain_lock-an-rwlock/20250919-174108
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
patch link:    https://lore.kernel.org/r/20250919093359.961296-4-eperezma%40redhat.com
patch subject: [PATCH v3 3/6] vduse: add vq group support
config: alpha-allyesconfig (https://download.01.org/0day-ci/archive/20250921/[email protected]/config)
compiler: alpha-linux-gcc (GCC) 15.1.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250921/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/vdpa/pds/vdpa_dev.c:590:35: error: initialization of 's64 (*)(struct vdpa_device *, u16)' {aka 'long long int (*)(struct vdpa_device *, short unsigned int)'} from incompatible pointer type 'u32 (*)(struct vdpa_device *, u16)' {aka 'unsigned int (*)(struct vdpa_device *, short unsigned int)'} [-Wincompatible-pointer-types]

590 | .get_vq_group = pds_vdpa_get_vq_group,
| ^~~~~~~~~~~~~~~~~~~~~
drivers/vdpa/pds/vdpa_dev.c:590:35: note: (near initialization for 'pds_vdpa_ops.get_vq_group') drivers/vdpa/pds/vdpa_dev.c:286:12: note: 'pds_vdpa_get_vq_group' declared here
286 | static u32 pds_vdpa_get_vq_group(struct vdpa_device *vdpa_dev, u16 idx)
| ^~~~~~~~~~~~~~~~~~~~~

vim +590 drivers/vdpa/pds/vdpa_dev.c

151cc834f3ddaf Shannon Nelson 2023-05-19  577  
151cc834f3ddaf Shannon Nelson 2023-05-19  578  static const struct vdpa_config_ops pds_vdpa_ops = {
151cc834f3ddaf Shannon Nelson 2023-05-19  579  	.set_vq_address		= pds_vdpa_set_vq_address,
151cc834f3ddaf Shannon Nelson 2023-05-19  580  	.set_vq_num		= pds_vdpa_set_vq_num,
151cc834f3ddaf Shannon Nelson 2023-05-19  581  	.kick_vq		= pds_vdpa_kick_vq,
151cc834f3ddaf Shannon Nelson 2023-05-19  582  	.set_vq_cb		= pds_vdpa_set_vq_cb,
151cc834f3ddaf Shannon Nelson 2023-05-19  583  	.set_vq_ready		= pds_vdpa_set_vq_ready,
151cc834f3ddaf Shannon Nelson 2023-05-19  584  	.get_vq_ready		= pds_vdpa_get_vq_ready,
151cc834f3ddaf Shannon Nelson 2023-05-19  585  	.set_vq_state		= pds_vdpa_set_vq_state,
151cc834f3ddaf Shannon Nelson 2023-05-19  586  	.get_vq_state		= pds_vdpa_get_vq_state,
151cc834f3ddaf Shannon Nelson 2023-05-19  587  	.get_vq_notification	= pds_vdpa_get_vq_notification,
151cc834f3ddaf Shannon Nelson 2023-05-19  588  	.get_vq_irq		= pds_vdpa_get_vq_irq,
151cc834f3ddaf Shannon Nelson 2023-05-19  589  	.get_vq_align		= pds_vdpa_get_vq_align,
151cc834f3ddaf Shannon Nelson 2023-05-19 @590  	.get_vq_group		= pds_vdpa_get_vq_group,
151cc834f3ddaf Shannon Nelson 2023-05-19  591  
151cc834f3ddaf Shannon Nelson 2023-05-19  592  	.get_device_features	= pds_vdpa_get_device_features,
151cc834f3ddaf Shannon Nelson 2023-05-19  593  	.set_driver_features	= pds_vdpa_set_driver_features,
151cc834f3ddaf Shannon Nelson 2023-05-19  594  	.get_driver_features	= pds_vdpa_get_driver_features,
151cc834f3ddaf Shannon Nelson 2023-05-19  595  	.set_config_cb		= pds_vdpa_set_config_cb,
151cc834f3ddaf Shannon Nelson 2023-05-19  596  	.get_vq_num_max		= pds_vdpa_get_vq_num_max,
151cc834f3ddaf Shannon Nelson 2023-05-19  597  	.get_device_id		= pds_vdpa_get_device_id,
151cc834f3ddaf Shannon Nelson 2023-05-19  598  	.get_vendor_id		= pds_vdpa_get_vendor_id,
151cc834f3ddaf Shannon Nelson 2023-05-19  599  	.get_status		= pds_vdpa_get_status,
151cc834f3ddaf Shannon Nelson 2023-05-19  600  	.set_status		= pds_vdpa_set_status,
151cc834f3ddaf Shannon Nelson 2023-05-19  601  	.reset			= pds_vdpa_reset,
151cc834f3ddaf Shannon Nelson 2023-05-19  602  	.get_config_size	= pds_vdpa_get_config_size,
151cc834f3ddaf Shannon Nelson 2023-05-19  603  	.get_config		= pds_vdpa_get_config,
151cc834f3ddaf Shannon Nelson 2023-05-19  604  	.set_config		= pds_vdpa_set_config,
151cc834f3ddaf Shannon Nelson 2023-05-19  605  };
25d1270b6e9ea8 Shannon Nelson 2023-05-19  606  static struct virtio_device_id pds_vdpa_id_table[] = {
25d1270b6e9ea8 Shannon Nelson 2023-05-19  607  	{VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID},
25d1270b6e9ea8 Shannon Nelson 2023-05-19  608  	{0},
25d1270b6e9ea8 Shannon Nelson 2023-05-19  609  };
25d1270b6e9ea8 Shannon Nelson 2023-05-19  610  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#5 lore raw

Re: [PATCH v3 3/6] vduse: add vq group support

In reply to Eugenio Pérez
Quoted 168 lines · Eugenio Pérez

This allows sepparate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API.

Allocating 3 vq groups as net is the device that need the most groups:
* Dataplane (guest passthrough)
* CVQ
* Shadowed vrings.

Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups.

---
v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason).
* Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number

v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.

v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Move the valid vq groups range check to vduse_validate_config.

RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.

---
 drivers/vdpa/ifcvf/ifcvf_main.c    |  2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |  2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c   |  2 +-
 drivers/vdpa/vdpa_user/vduse_dev.c | 51 +++++++++++++++++++++++++++---
 drivers/vhost/vdpa.c               | 11 +++++--
 include/linux/vdpa.h               |  5 +--
 include/uapi/linux/vduse.h         | 12 +++++--
 7 files changed, 69 insertions(+), 16 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +1 -1
index 6658dc74d915..ea6643760fb7 100644
@@ -559,7 +559,7 @@ static size_t ifcvf_vdpa_get_config_size(struct vdpa_device *vdpa_dev)
        return  vf->config_size;
 }
 
-static u32 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        return 0;
 }
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -1
index 82034efb74fc..daab0ccb78b2 100644
@@ -2640,7 +2640,7 @@ static u32 mlx5_vdpa_get_vq_align(struct vdpa_device *vdev)
        return PAGE_SIZE;
 }
 
-static u32 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
+static s64 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
 {
        struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
 
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
index c1c6431950e1..fe4d65aaf6aa 100644
@@ -412,7 +412,7 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)
        return VDPASIM_QUEUE_ALIGN;
 }
 
-static u32 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        /* RX and TX belongs to group 0, CVQ belongs to group 1 */
        if (idx == 2)
drivers/vdpa/vdpa_user/vduse_dev.c +20 -0
index 2b6a8958ffe0..7ddd81456f7b 100644
@@ -40,6 +40,7 @@
 #define DRV_LICENSE  "GPL v2"
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
+#define VDUSE_DEV_MAX_GROUPS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -49,6 +50,8 @@
 
 #define IRQ_UNBOUND -1
 
+#define VQ_GROUP_INVALID -1
+
 struct vduse_virtqueue {
        u16 index;
        u16 num_max;
@@ -59,6 +62,7 @@ struct vduse_virtqueue {
        struct vdpa_vq_state state;
        bool ready;
        bool kicked;
+       u32 vq_group;
        spinlock_t kick_lock;
        spinlock_t irq_lock;
        struct eventfd_ctx *kickfd;
@@ -115,6 +119,7 @@ struct vduse_dev {
        u8 status;
        u32 vq_num;
        u32 vq_align;
+       u32 ngroups;
        struct vduse_umem *umem;
        struct mutex mem_lock;
        unsigned int bounce_size;
@@ -456,6 +461,7 @@ static void vduse_dev_reset(struct vduse_dev *dev)
                vq->driver_addr = 0;
                vq->device_addr = 0;
                vq->num = 0;
+               vq->vq_group = VQ_GROUP_INVALID;
                memset(&vq->state, 0, sizeof(vq->state));
 
                spin_lock(&vq->kick_lock);
@@ -593,6 +599,19 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 idx,
        return 0;
 }
 
+static s64 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+
+       if (dev->api_version < VDUSE_API_VERSION_1)
+               return 0;
+
+       if (dev->vqs[idx]->vq_group == VQ_GROUP_INVALID)
+               return -EINVAL;
+
+       return dev->vqs[idx]->vq_group;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -790,6 +809,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .set_vq_cb              = vduse_vdpa_set_vq_cb,
        .set_vq_num             = vduse_vdpa_set_vq_num,
        .get_vq_size            = vduse_vdpa_get_vq_size,
+       .get_vq_group           = vduse_get_vq_group,
        .set_vq_ready           = vduse_vdpa_set_vq_ready,
        .get_vq_ready           = vduse_vdpa_get_vq_ready,
        .set_vq_state           = vduse_vdpa_set_vq_state,
@@ -1253,12 +1273,24 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
                if (config.index >= dev->vq_num)
                        break;
 

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1283

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +8 -2
-               if (!is_mem_zero((const char *)config.reserved,
-                                sizeof(config.reserved)))
+               if (dev->api_version < VDUSE_API_VERSION_1 && config.group)
+                       break;
+
+               if (dev->api_version >= VDUSE_API_VERSION_1) {
+                       if (config.group > dev->ngroups)
+                               break;
+                       if (dev->status & VIRTIO_CONFIG_S_DRIVER_OK)
+                               break;

This seems to be a new requirement for V1. Please explain why.

Quoted 36 lines
drivers/vdpa/vdpa_user/vduse_dev.c +15 -1
+               }
+
+               if (config.reserved1 ||
+                   !is_mem_zero((const char *)config.reserved2,
+                                sizeof(config.reserved2)))
                        break;
 
                index = array_index_nospec(config.index, dev->vq_num);
                dev->vqs[index]->num_max = config.max_size;
+               dev->vqs[index]->vq_group = config.group;
                ret = 0;
                break;
        }
@@ -1738,12 +1770,20 @@ static bool features_is_valid(struct vduse_dev_config *config)
        return true;
 }
 
-static bool vduse_validate_config(struct vduse_dev_config *config)
+static bool vduse_validate_config(struct vduse_dev_config *config,
+                                 u64 api_version)
 {
        if (!is_mem_zero((const char *)config->reserved,
                         sizeof(config->reserved)))
                return false;
 
+       if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+               return false;
+
+       if (api_version >= VDUSE_API_VERSION_1 &&
+           config->ngroups > VDUSE_DEV_MAX_GROUPS)
+               return false;
+
        if (config->vq_align > PAGE_SIZE)
                return false;
 

drivers/vdpa/vdpa_user/vduse_dev.c static int vduse_create_dev(struct vduse_dev_config *config, · line 1902

Quoted 4 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -0
@@ -1859,6 +1899,7 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        dev->device_features = config->features;
        dev->device_id = config->device_id;
        dev->vendor_id = config->vendor_id;
+       dev->ngroups = (dev->api_version < 1) ? 1 : (config->ngroups ?: 1);

Should we fail if config->ngroups is zero when version is 1?

Quoted 62 lines
drivers/vdpa/vdpa_user/vduse_dev.c +2 -2
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -1937,7 +1978,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,
                        break;
 
                ret = -EINVAL;
-               if (vduse_validate_config(&config) == false)
+               if (!vduse_validate_config(&config, control->api_version))
                        break;
 
                buf = vmemdup_user(argp + size, config.config_size);
@@ -2018,7 +2059,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
        vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
                                 &vduse_vdpa_config_ops, &vduse_map_ops,
-                                1, 1, name, true);
+                                dev->ngroups, 1, name, true);
        if (IS_ERR(vdev))
                return PTR_ERR(vdev);
 
drivers/vhost/vdpa.c +8 -3
index 05a481e4c385..6305382eacbb 100644
@@ -22,6 +22,7 @@
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
 #include <linux/vhost.h>
+#include <linux/types.h>
 
 #include "vhost.h"
 
@@ -657,16 +658,20 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
                        return -EFAULT;
                ops->set_vq_ready(vdpa, idx, s.num);
                return 0;
-       case VHOST_VDPA_GET_VRING_GROUP:
+       case VHOST_VDPA_GET_VRING_GROUP: {
+               u64 group;
+
                if (!ops->get_vq_group)
                        return -EOPNOTSUPP;
                s.index = idx;
-               s.num = ops->get_vq_group(vdpa, idx);
-               if (s.num >= vdpa->ngroups)
+               group = ops->get_vq_group(vdpa, idx);
+               if (group >= vdpa->ngroups || group > U32_MAX || group < 0)
                        return -EIO;
                else if (copy_to_user(argp, &s, sizeof(s)))
                        return -EFAULT;
+               s.num = group;
                return 0;
+       }
        case VHOST_VDPA_GET_VRING_DESC_GROUP:
                if (!vhost_vdpa_has_desc_group(v))
                        return -EOPNOTSUPP;
include/linux/vdpa.h +0 -0
index 4cf21d6e9cfd..76a673ef7deb 100644

include/linux/vdpa.h struct vdpa_map_file { · line 214

Quoted 5 lines
include/linux/vdpa.h +1 -1
@@ -211,7 +211,8 @@ struct vdpa_map_file {
  *                             virtqueue (optional)
  *                             @vdev: vdpa device
  *                             @idx: virtqueue index
- *                             Returns u32: group id for this virtqueue
+ *                             Returns i64: (32 bits wide) group id for this

Let's separate the u32->s64 changes. And maybe it's fine to say returns s64 and explain U32_MAX is the maximum value allowed.

Quoted 57 lines
include/linux/vdpa.h +2 -1
+ *                             virtqueue if >0. If <0, errno.
  * @get_vq_desc_group:         Get the group id for the descriptor table of
  *                             a specific virtqueue (optional)
  *                             @vdev: vdpa device
@@ -398,7 +399,7 @@ struct vdpa_config_ops {
 
        /* Device ops */
        u32 (*get_vq_align)(struct vdpa_device *vdev);
-       u32 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
+       s64 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
        u32 (*get_vq_desc_group)(struct vdpa_device *vdev, u16 idx);
        u64 (*get_device_features)(struct vdpa_device *vdev);
        u64 (*get_backend_features)(const struct vdpa_device *vdev);
include/uapi/linux/vduse.h +9 -4
index ccb92a1efce0..a3d51cf6df3a 100644
@@ -31,6 +31,7 @@
  * @features: virtio features
  * @vq_num: the number of virtqueues
  * @vq_align: the allocation alignment of virtqueue's metadata
+ * @ngroups: number of vq groups that VDUSE device declares
  * @reserved: for future use, needs to be initialized to zero
  * @config_size: the size of the configuration space
  * @config: the buffer of the configuration space
@@ -45,7 +46,8 @@ struct vduse_dev_config {
        __u64 features;
        __u32 vq_num;
        __u32 vq_align;
-       __u32 reserved[13];
+       __u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
+       __u32 reserved[12];
        __u32 config_size;
        __u8 config[];
 };
@@ -122,14 +124,18 @@ struct vduse_config_data {
  * struct vduse_vq_config - basic configuration of a virtqueue
  * @index: virtqueue index
  * @max_size: the max size of virtqueue
- * @reserved: for future use, needs to be initialized to zero
+ * @reserved1: for future use, needs to be initialized to zero
+ * @group: virtqueue group
+ * @reserved2: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
  */
 struct vduse_vq_config {
        __u32 index;
        __u16 max_size;
-       __u16 reserved[13];
+       __u16 reserved1;
+       __u32 group;
+       __u16 reserved2[10];
 };
 
 /*
--
Quoted 2 lines

2.51.0

Thanks

author maintainer
#6 lore raw

Re: [PATCH v3 3/6] vduse: add vq group support

Quoted 170 lines · Jason Wang
Quoted 168 lines · Eugenio Pérez

This allows sepparate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API.

Allocating 3 vq groups as net is the device that need the most groups:
* Dataplane (guest passthrough)
* CVQ
* Shadowed vrings.

Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups.

---
v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason).
* Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number

v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.

v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Move the valid vq groups range check to vduse_validate_config.

RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.

---
 drivers/vdpa/ifcvf/ifcvf_main.c    |  2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |  2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c   |  2 +-
 drivers/vdpa/vdpa_user/vduse_dev.c | 51 +++++++++++++++++++++++++++---
 drivers/vhost/vdpa.c               | 11 +++++--
 include/linux/vdpa.h               |  5 +--
 include/uapi/linux/vduse.h         | 12 +++++--
 7 files changed, 69 insertions(+), 16 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +1 -1
index 6658dc74d915..ea6643760fb7 100644
@@ -559,7 +559,7 @@ static size_t ifcvf_vdpa_get_config_size(struct vdpa_device *vdpa_dev)
        return  vf->config_size;
 }
 
-static u32 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        return 0;
 }
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -1
index 82034efb74fc..daab0ccb78b2 100644
@@ -2640,7 +2640,7 @@ static u32 mlx5_vdpa_get_vq_align(struct vdpa_device *vdev)
        return PAGE_SIZE;
 }
 
-static u32 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
+static s64 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
 {
        struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
 
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
index c1c6431950e1..fe4d65aaf6aa 100644
@@ -412,7 +412,7 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)
        return VDPASIM_QUEUE_ALIGN;
 }
 
-static u32 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        /* RX and TX belongs to group 0, CVQ belongs to group 1 */
        if (idx == 2)
drivers/vdpa/vdpa_user/vduse_dev.c +20 -0
index 2b6a8958ffe0..7ddd81456f7b 100644
@@ -40,6 +40,7 @@
 #define DRV_LICENSE  "GPL v2"
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
+#define VDUSE_DEV_MAX_GROUPS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -49,6 +50,8 @@
 
 #define IRQ_UNBOUND -1
 
+#define VQ_GROUP_INVALID -1
+
 struct vduse_virtqueue {
        u16 index;
        u16 num_max;
@@ -59,6 +62,7 @@ struct vduse_virtqueue {
        struct vdpa_vq_state state;
        bool ready;
        bool kicked;
+       u32 vq_group;
        spinlock_t kick_lock;
        spinlock_t irq_lock;
        struct eventfd_ctx *kickfd;
@@ -115,6 +119,7 @@ struct vduse_dev {
        u8 status;
        u32 vq_num;
        u32 vq_align;
+       u32 ngroups;
        struct vduse_umem *umem;
        struct mutex mem_lock;
        unsigned int bounce_size;
@@ -456,6 +461,7 @@ static void vduse_dev_reset(struct vduse_dev *dev)
                vq->driver_addr = 0;
                vq->device_addr = 0;
                vq->num = 0;
+               vq->vq_group = VQ_GROUP_INVALID;
                memset(&vq->state, 0, sizeof(vq->state));
 
                spin_lock(&vq->kick_lock);
@@ -593,6 +599,19 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 idx,
        return 0;
 }
 
+static s64 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+
+       if (dev->api_version < VDUSE_API_VERSION_1)
+               return 0;
+
+       if (dev->vqs[idx]->vq_group == VQ_GROUP_INVALID)
+               return -EINVAL;
+
+       return dev->vqs[idx]->vq_group;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -790,6 +809,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .set_vq_cb              = vduse_vdpa_set_vq_cb,
        .set_vq_num             = vduse_vdpa_set_vq_num,
        .get_vq_size            = vduse_vdpa_get_vq_size,
+       .get_vq_group           = vduse_get_vq_group,
        .set_vq_ready           = vduse_vdpa_set_vq_ready,
        .get_vq_ready           = vduse_vdpa_get_vq_ready,
        .set_vq_state           = vduse_vdpa_set_vq_state,
@@ -1253,12 +1273,24 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
                if (config.index >= dev->vq_num)
                        break;
 
Quoted 13 lines

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1283

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +8 -2
-               if (!is_mem_zero((const char *)config.reserved,
-                                sizeof(config.reserved)))
+               if (dev->api_version < VDUSE_API_VERSION_1 && config.group)
+                       break;
+
+               if (dev->api_version >= VDUSE_API_VERSION_1) {
+                       if (config.group > dev->ngroups)
+                               break;
+                       if (dev->status & VIRTIO_CONFIG_S_DRIVER_OK)
+                               break;

This seems to be a new requirement for V1. Please explain why.

Would a code comment like "The device is not allowed to change the VQ group after the driver sets DRIVER_OK" work? I'm tempted to add "VQ group or size", but it was not an error previously so I'm ok with leaving it only for vq groups.

Quoted 35 lines
Quoted 35 lines
drivers/vdpa/vdpa_user/vduse_dev.c +15 -1
+               }
+
+               if (config.reserved1 ||
+                   !is_mem_zero((const char *)config.reserved2,
+                                sizeof(config.reserved2)))
                        break;
 
                index = array_index_nospec(config.index, dev->vq_num);
                dev->vqs[index]->num_max = config.max_size;
+               dev->vqs[index]->vq_group = config.group;
                ret = 0;
                break;
        }
@@ -1738,12 +1770,20 @@ static bool features_is_valid(struct vduse_dev_config *config)
        return true;
 }
 
-static bool vduse_validate_config(struct vduse_dev_config *config)
+static bool vduse_validate_config(struct vduse_dev_config *config,
+                                 u64 api_version)
 {
        if (!is_mem_zero((const char *)config->reserved,
                         sizeof(config->reserved)))
                return false;
 
+       if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+               return false;
+
+       if (api_version >= VDUSE_API_VERSION_1 &&
+           config->ngroups > VDUSE_DEV_MAX_GROUPS)
+               return false;
+
        if (config->vq_align > PAGE_SIZE)
                return false;
 
Quoted 8 lines

drivers/vdpa/vdpa_user/vduse_dev.c static int vduse_create_dev(struct vduse_dev_config *config, · line 1902

Quoted 5 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -0
@@ -1859,6 +1899,7 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        dev->device_features = config->features;
        dev->device_id = config->device_id;
        dev->vendor_id = config->vendor_id;
+       dev->ngroups = (dev->api_version < 1) ? 1 : (config->ngroups ?: 1);

Should we fail if config->ngroups is zero when version is 1?

The devices that don't need to mess with vq groups are better with not needing to know anything about it in their codebases, in my opinion. Having to learn about what is a vq group for a newcomer to vDPA/VDUSE is setting the barrier higher for little benefit.

We can obtain the same advantage of reserving the value 0 with a new API version or VDUSE feature flags.

Having said that, I'm ok if you prefer this device creation to fail in that case.

Quoted 62 lines
Quoted 62 lines
drivers/vdpa/vdpa_user/vduse_dev.c +2 -2
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -1937,7 +1978,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,
                        break;
 
                ret = -EINVAL;
-               if (vduse_validate_config(&config) == false)
+               if (!vduse_validate_config(&config, control->api_version))
                        break;
 
                buf = vmemdup_user(argp + size, config.config_size);
@@ -2018,7 +2059,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
        vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
                                 &vduse_vdpa_config_ops, &vduse_map_ops,
-                                1, 1, name, true);
+                                dev->ngroups, 1, name, true);
        if (IS_ERR(vdev))
                return PTR_ERR(vdev);
 
drivers/vhost/vdpa.c +8 -3
index 05a481e4c385..6305382eacbb 100644
@@ -22,6 +22,7 @@
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
 #include <linux/vhost.h>
+#include <linux/types.h>
 
 #include "vhost.h"
 
@@ -657,16 +658,20 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
                        return -EFAULT;
                ops->set_vq_ready(vdpa, idx, s.num);
                return 0;
-       case VHOST_VDPA_GET_VRING_GROUP:
+       case VHOST_VDPA_GET_VRING_GROUP: {
+               u64 group;
+
                if (!ops->get_vq_group)
                        return -EOPNOTSUPP;
                s.index = idx;
-               s.num = ops->get_vq_group(vdpa, idx);
-               if (s.num >= vdpa->ngroups)
+               group = ops->get_vq_group(vdpa, idx);
+               if (group >= vdpa->ngroups || group > U32_MAX || group < 0)
                        return -EIO;
                else if (copy_to_user(argp, &s, sizeof(s)))
                        return -EFAULT;
+               s.num = group;
                return 0;
+       }
        case VHOST_VDPA_GET_VRING_DESC_GROUP:
                if (!vhost_vdpa_has_desc_group(v))
                        return -EOPNOTSUPP;
include/linux/vdpa.h +0 -0
index 4cf21d6e9cfd..76a673ef7deb 100644
@@ -211,7 +211,8 @@ struct vdpa_map_file {
Quoted 9 lines

include/linux/vdpa.h struct vdpa_map_file { · line 214

Quoted 5 lines
include/linux/vdpa.h +1 -1
  *                             virtqueue (optional)
  *                             @vdev: vdpa device
  *                             @idx: virtqueue index
- *                             Returns u32: group id for this virtqueue
+ *                             Returns i64: (32 bits wide) group id for this

Let's separate the u32->s64 changes. And maybe it's fine to say returns s64 and explain U32_MAX is the maximum value allowed.

Sure, sending a new version with that.

Quoted 62 lines
Quoted 57 lines
include/linux/vdpa.h +2 -1
+ *                             virtqueue if >0. If <0, errno.
  * @get_vq_desc_group:         Get the group id for the descriptor table of
  *                             a specific virtqueue (optional)
  *                             @vdev: vdpa device
@@ -398,7 +399,7 @@ struct vdpa_config_ops {
 
        /* Device ops */
        u32 (*get_vq_align)(struct vdpa_device *vdev);
-       u32 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
+       s64 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
        u32 (*get_vq_desc_group)(struct vdpa_device *vdev, u16 idx);
        u64 (*get_device_features)(struct vdpa_device *vdev);
        u64 (*get_backend_features)(const struct vdpa_device *vdev);
include/uapi/linux/vduse.h +9 -4
index ccb92a1efce0..a3d51cf6df3a 100644
@@ -31,6 +31,7 @@
  * @features: virtio features
  * @vq_num: the number of virtqueues
  * @vq_align: the allocation alignment of virtqueue's metadata
+ * @ngroups: number of vq groups that VDUSE device declares
  * @reserved: for future use, needs to be initialized to zero
  * @config_size: the size of the configuration space
  * @config: the buffer of the configuration space
@@ -45,7 +46,8 @@ struct vduse_dev_config {
        __u64 features;
        __u32 vq_num;
        __u32 vq_align;
-       __u32 reserved[13];
+       __u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
+       __u32 reserved[12];
        __u32 config_size;
        __u8 config[];
 };
@@ -122,14 +124,18 @@ struct vduse_config_data {
  * struct vduse_vq_config - basic configuration of a virtqueue
  * @index: virtqueue index
  * @max_size: the max size of virtqueue
- * @reserved: for future use, needs to be initialized to zero
+ * @reserved1: for future use, needs to be initialized to zero
+ * @group: virtqueue group
+ * @reserved2: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
  */
 struct vduse_vq_config {
        __u32 index;
        __u16 max_size;
-       __u16 reserved[13];
+       __u16 reserved1;
+       __u32 group;
+       __u16 reserved2[10];
 };
 
 /*
--
Quoted 2 lines

2.51.0

Thanks

#7 lore raw

Re: [PATCH v3 3/6] vduse: add vq group support

Quoted 186 lines · Eugenio Perez Martin
Quoted 170 lines · Jason Wang
Quoted 168 lines · Eugenio Pérez

This allows sepparate the different virtqueues in groups that shares the same address space. Asking the VDUSE device for the groups of the vq at the beginning as they're needed for the DMA API.

Allocating 3 vq groups as net is the device that need the most groups:
* Dataplane (guest passthrough)
* CVQ
* Shadowed vrings.

Future versions of the series can include dynamic allocation of the groups array so VDUSE can declare more groups.

---
v3:
* Make the default group an invalid group as long as VDUSE device does not set it to some valid u32 value. Modify the vdpa core to take that into account (Jason).
* Create the VDUSE_DEV_MAX_GROUPS instead of using a magic number

v2:
* Now the vq group is in vduse_vq_config struct instead of issuing one VDUSE message per vq.

v1:
* Fix: Remove BIT_ULL(VIRTIO_S_*), as _S_ is already the bit (Maxime)

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove the descs vq group capability as it will not be used and we can add it on top.
* Do not ask for vq groups in number of vq groups < 2.
* Move the valid vq groups range check to vduse_validate_config.

RFC v2:
* Cache group information in kernel, as we need to provide the vq map tokens properly.
* Add descs vq group to optimize SVQ forwarding and support indirect descriptors out of the box.

---
 drivers/vdpa/ifcvf/ifcvf_main.c    |  2 +-
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |  2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c   |  2 +-
 drivers/vdpa/vdpa_user/vduse_dev.c | 51 +++++++++++++++++++++++++++---
 drivers/vhost/vdpa.c               | 11 +++++--
 include/linux/vdpa.h               |  5 +--
 include/uapi/linux/vduse.h         | 12 +++++--
 7 files changed, 69 insertions(+), 16 deletions(-)
drivers/vdpa/ifcvf/ifcvf_main.c +1 -1
index 6658dc74d915..ea6643760fb7 100644
@@ -559,7 +559,7 @@ static size_t ifcvf_vdpa_get_config_size(struct vdpa_device *vdpa_dev)
        return  vf->config_size;
 }
 
-static u32 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 ifcvf_vdpa_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        return 0;
 }
drivers/vdpa/mlx5/net/mlx5_vnet.c +1 -1
index 82034efb74fc..daab0ccb78b2 100644
@@ -2640,7 +2640,7 @@ static u32 mlx5_vdpa_get_vq_align(struct vdpa_device *vdev)
        return PAGE_SIZE;
 }
 
-static u32 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
+static s64 mlx5_vdpa_get_vq_group(struct vdpa_device *vdev, u16 idx)
 {
        struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
 
drivers/vdpa/vdpa_sim/vdpa_sim.c +1 -1
index c1c6431950e1..fe4d65aaf6aa 100644
@@ -412,7 +412,7 @@ static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)
        return VDPASIM_QUEUE_ALIGN;
 }
 
-static u32 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+static s64 vdpasim_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 {
        /* RX and TX belongs to group 0, CVQ belongs to group 1 */
        if (idx == 2)
drivers/vdpa/vdpa_user/vduse_dev.c +20 -0
index 2b6a8958ffe0..7ddd81456f7b 100644
@@ -40,6 +40,7 @@
 #define DRV_LICENSE  "GPL v2"
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
+#define VDUSE_DEV_MAX_GROUPS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -49,6 +50,8 @@
 
 #define IRQ_UNBOUND -1
 
+#define VQ_GROUP_INVALID -1
+
 struct vduse_virtqueue {
        u16 index;
        u16 num_max;
@@ -59,6 +62,7 @@ struct vduse_virtqueue {
        struct vdpa_vq_state state;
        bool ready;
        bool kicked;
+       u32 vq_group;
        spinlock_t kick_lock;
        spinlock_t irq_lock;
        struct eventfd_ctx *kickfd;
@@ -115,6 +119,7 @@ struct vduse_dev {
        u8 status;
        u32 vq_num;
        u32 vq_align;
+       u32 ngroups;
        struct vduse_umem *umem;
        struct mutex mem_lock;
        unsigned int bounce_size;
@@ -456,6 +461,7 @@ static void vduse_dev_reset(struct vduse_dev *dev)
                vq->driver_addr = 0;
                vq->device_addr = 0;
                vq->num = 0;
+               vq->vq_group = VQ_GROUP_INVALID;
                memset(&vq->state, 0, sizeof(vq->state));
 
                spin_lock(&vq->kick_lock);
@@ -593,6 +599,19 @@ static int vduse_vdpa_set_vq_state(struct vdpa_device *vdpa, u16 idx,
        return 0;
 }
 
+static s64 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+
+       if (dev->api_version < VDUSE_API_VERSION_1)
+               return 0;
+
+       if (dev->vqs[idx]->vq_group == VQ_GROUP_INVALID)
+               return -EINVAL;
+
+       return dev->vqs[idx]->vq_group;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -790,6 +809,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .set_vq_cb              = vduse_vdpa_set_vq_cb,
        .set_vq_num             = vduse_vdpa_set_vq_num,
        .get_vq_size            = vduse_vdpa_get_vq_size,
+       .get_vq_group           = vduse_get_vq_group,
        .set_vq_ready           = vduse_vdpa_set_vq_ready,
        .get_vq_ready           = vduse_vdpa_get_vq_ready,
        .set_vq_state           = vduse_vdpa_set_vq_state,
@@ -1253,12 +1273,24 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
                if (config.index >= dev->vq_num)
                        break;
 
Quoted 13 lines

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1283

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +8 -2
-               if (!is_mem_zero((const char *)config.reserved,
-                                sizeof(config.reserved)))
+               if (dev->api_version < VDUSE_API_VERSION_1 && config.group)
+                       break;
+
+               if (dev->api_version >= VDUSE_API_VERSION_1) {
+                       if (config.group > dev->ngroups)
+                               break;
+                       if (dev->status & VIRTIO_CONFIG_S_DRIVER_OK)
+                               break;

This seems to be a new requirement for V1. Please explain why.

Quoted 4 lines

Would a code comment like "The device is not allowed to change the VQ group after the driver sets DRIVER_OK" work? I'm tempted to add "VQ group or size", but it was not an error previously so I'm ok with leaving it only for vq groups.

That's fine but I wonder why we check DRIVER_OK instead of FEATURES_OK.

Quoted 50 lines
Quoted 36 lines
Quoted 36 lines
drivers/vdpa/vdpa_user/vduse_dev.c +15 -1
+               }
+
+               if (config.reserved1 ||
+                   !is_mem_zero((const char *)config.reserved2,
+                                sizeof(config.reserved2)))
                        break;
 
                index = array_index_nospec(config.index, dev->vq_num);
                dev->vqs[index]->num_max = config.max_size;
+               dev->vqs[index]->vq_group = config.group;
                ret = 0;
                break;
        }
@@ -1738,12 +1770,20 @@ static bool features_is_valid(struct vduse_dev_config *config)
        return true;
 }
 
-static bool vduse_validate_config(struct vduse_dev_config *config)
+static bool vduse_validate_config(struct vduse_dev_config *config,
+                                 u64 api_version)
 {
        if (!is_mem_zero((const char *)config->reserved,
                         sizeof(config->reserved)))
                return false;
 
+       if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+               return false;
+
+       if (api_version >= VDUSE_API_VERSION_1 &&
+           config->ngroups > VDUSE_DEV_MAX_GROUPS)
+               return false;
+
        if (config->vq_align > PAGE_SIZE)
                return false;
 
Quoted 8 lines

drivers/vdpa/vdpa_user/vduse_dev.c static int vduse_create_dev(struct vduse_dev_config *config, · line 1902

Quoted 5 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -0
@@ -1859,6 +1899,7 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        dev->device_features = config->features;
        dev->device_id = config->device_id;
        dev->vendor_id = config->vendor_id;
+       dev->ngroups = (dev->api_version < 1) ? 1 : (config->ngroups ?: 1);

Should we fail if config->ngroups is zero when version is 1?

The devices that don't need to mess with vq groups are better with not needing to know anything about it in their codebases, in my opinion. Having to learn about what is a vq group for a newcomer to vDPA/VDUSE is setting the barrier higher for little benefit.

Quoted 5 lines

We can obtain the same advantage of reserving the value 0 with a new API version or VDUSE feature flags.

Having said that, I'm ok if you prefer this device creation to fail in that case.

I prefer to fail in this case.

Quoted 73 lines
Quoted 63 lines
Quoted 63 lines
drivers/vdpa/vdpa_user/vduse_dev.c +2 -2
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -1937,7 +1978,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,
                        break;
 
                ret = -EINVAL;
-               if (vduse_validate_config(&config) == false)
+               if (!vduse_validate_config(&config, control->api_version))
                        break;
 
                buf = vmemdup_user(argp + size, config.config_size);
@@ -2018,7 +2059,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
        vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
                                 &vduse_vdpa_config_ops, &vduse_map_ops,
-                                1, 1, name, true);
+                                dev->ngroups, 1, name, true);
        if (IS_ERR(vdev))
                return PTR_ERR(vdev);
 
drivers/vhost/vdpa.c +8 -3
index 05a481e4c385..6305382eacbb 100644
@@ -22,6 +22,7 @@
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
 #include <linux/vhost.h>
+#include <linux/types.h>
 
 #include "vhost.h"
 
@@ -657,16 +658,20 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
                        return -EFAULT;
                ops->set_vq_ready(vdpa, idx, s.num);
                return 0;
-       case VHOST_VDPA_GET_VRING_GROUP:
+       case VHOST_VDPA_GET_VRING_GROUP: {
+               u64 group;
+
                if (!ops->get_vq_group)
                        return -EOPNOTSUPP;
                s.index = idx;
-               s.num = ops->get_vq_group(vdpa, idx);
-               if (s.num >= vdpa->ngroups)
+               group = ops->get_vq_group(vdpa, idx);
+               if (group >= vdpa->ngroups || group > U32_MAX || group < 0)
                        return -EIO;
                else if (copy_to_user(argp, &s, sizeof(s)))
                        return -EFAULT;
+               s.num = group;
                return 0;
+       }
        case VHOST_VDPA_GET_VRING_DESC_GROUP:
                if (!vhost_vdpa_has_desc_group(v))
                        return -EOPNOTSUPP;
include/linux/vdpa.h +0 -0
index 4cf21d6e9cfd..76a673ef7deb 100644
@@ -211,7 +211,8 @@ struct vdpa_map_file {
Quoted 9 lines

include/linux/vdpa.h struct vdpa_map_file { · line 214

Quoted 5 lines
include/linux/vdpa.h +1 -1
  *                             virtqueue (optional)
  *                             @vdev: vdpa device
  *                             @idx: virtqueue index
- *                             Returns u32: group id for this virtqueue
+ *                             Returns i64: (32 bits wide) group id for this

Let's separate the u32->s64 changes. And maybe it's fine to say returns s64 and explain U32_MAX is the maximum value allowed.

Quoted 65 lines

Sure, sending a new version with that.

Quoted 62 lines
Quoted 57 lines
include/linux/vdpa.h +2 -1
+ *                             virtqueue if >0. If <0, errno.
  * @get_vq_desc_group:         Get the group id for the descriptor table of
  *                             a specific virtqueue (optional)
  *                             @vdev: vdpa device
@@ -398,7 +399,7 @@ struct vdpa_config_ops {
 
        /* Device ops */
        u32 (*get_vq_align)(struct vdpa_device *vdev);
-       u32 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
+       s64 (*get_vq_group)(struct vdpa_device *vdev, u16 idx);
        u32 (*get_vq_desc_group)(struct vdpa_device *vdev, u16 idx);
        u64 (*get_device_features)(struct vdpa_device *vdev);
        u64 (*get_backend_features)(const struct vdpa_device *vdev);
include/uapi/linux/vduse.h +9 -4
index ccb92a1efce0..a3d51cf6df3a 100644
@@ -31,6 +31,7 @@
  * @features: virtio features
  * @vq_num: the number of virtqueues
  * @vq_align: the allocation alignment of virtqueue's metadata
+ * @ngroups: number of vq groups that VDUSE device declares
  * @reserved: for future use, needs to be initialized to zero
  * @config_size: the size of the configuration space
  * @config: the buffer of the configuration space
@@ -45,7 +46,8 @@ struct vduse_dev_config {
        __u64 features;
        __u32 vq_num;
        __u32 vq_align;
-       __u32 reserved[13];
+       __u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
+       __u32 reserved[12];
        __u32 config_size;
        __u8 config[];
 };
@@ -122,14 +124,18 @@ struct vduse_config_data {
  * struct vduse_vq_config - basic configuration of a virtqueue
  * @index: virtqueue index
  * @max_size: the max size of virtqueue
- * @reserved: for future use, needs to be initialized to zero
+ * @reserved1: for future use, needs to be initialized to zero
+ * @group: virtqueue group
+ * @reserved2: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
  */
 struct vduse_vq_config {
        __u32 index;
        __u16 max_size;
-       __u16 reserved[13];
+       __u16 reserved1;
+       __u32 group;
+       __u16 reserved2[10];
 };
 
 /*
--
Quoted 2 lines

2.51.0

Thanks

Thanks

author maintainer
#8 lore raw

PATCH v3 4/6 vduse: return internal vq group struct as map token

In reply to Eugenio Pérez

Return the internal struct that represents the vq group as virtqueue map token, instead of the device. This allows the map functions to access the information per group.

At this moment all the virtqueues share the same vq group, that only can point to ASID 0. This change prepares the infrastructure for actual per-group address space handling

---
v3:
* Adapt all virtio_map_ops callbacks to handle empty tokens in case of invalid groups.
* Make setting status DRIVER_OK fail if vq group is not valid.
* Remove the _int name suffix from struct vduse_vq_group.

RFC v3:
* Make the vq groups a dynamic array to support an arbitrary number of them.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 111 ++++++++++++++++++++++++++---
 include/linux/virtio.h             |   6 +-
 2 files changed, 105 insertions(+), 12 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +102 -9
index 7ddd81456f7b..6fdabc3c910e 100644
@@ -23,6 +23,7 @@
 #include <linux/uio.h>
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
+#include <linux/virtio.h>
 #include <linux/vmalloc.h>
 #include <linux/sched/mm.h>
 #include <uapi/linux/vduse.h>
@@ -88,6 +89,10 @@ struct vduse_umem {
 	struct mm_struct *mm;
 };
 
+struct vduse_vq_group {
+	struct vduse_dev *dev;
+};
+
 struct vduse_dev {
 	struct vduse_vdpa *vdev;
 	struct device *dev;
@@ -121,6 +126,7 @@ struct vduse_dev {
 	u32 vq_align;
 	u32 ngroups;
 	struct vduse_umem *umem;
+	struct vduse_vq_group *groups;
 	struct mutex mem_lock;
 	unsigned int bounce_size;
 	rwlock_t domain_lock;
@@ -308,6 +314,13 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
 	msg.req.type = VDUSE_SET_STATUS;
 	msg.req.s.status = status;
 
+	if (dev->api_version >= VDUSE_API_VERSION_1 &&
+	    !(dev->status & VIRTIO_CONFIG_S_DRIVER_OK) &&
+	    status & VIRTIO_CONFIG_S_DRIVER_OK)
+		for (u32 i = 0; i < dev->vq_num; ++i)
+			if (dev->vqs[i]->vq_group == VQ_GROUP_INVALID)
+				return -EIO;
+
 	return vduse_dev_msg_sync(dev, &msg);
 }
 
@@ -612,6 +625,20 @@ static s64 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)
 	return dev->vqs[idx]->vq_group;
 }
 
+static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
+{
+	struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+	u32 vq_group = vduse_get_vq_group(vdpa, idx);
+	union virtio_map ret;
+
+	if (vq_group < 0)
+		ret.group = NULL;
+	else
+		ret.group = &dev->groups[vq_group];
+
+	return ret;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
 				struct vdpa_vq_state *state)
 {
@@ -832,6 +859,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
 	.get_vq_affinity	= vduse_vdpa_get_vq_affinity,
 	.reset			= vduse_vdpa_reset,
 	.set_map		= vduse_vdpa_set_map,
+	.get_vq_map		= vduse_get_vq_map,
 	.free			= vduse_vdpa_free,
 };
 
@@ -839,7 +867,14 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
 					     dma_addr_t dma_addr, size_t size,
 					     enum dma_data_direction dir)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	vduse_domain_sync_single_for_device(domain, dma_addr, size, dir);
 }
@@ -848,7 +883,14 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token,
 					     dma_addr_t dma_addr, size_t size,
 					     enum dma_data_direction dir)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir);
 }
@@ -858,7 +900,14 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
 				     enum dma_data_direction dir,
 				     unsigned long attrs)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return DMA_MAPPING_ERROR;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	return vduse_domain_map_page(domain, page, offset, size, dir, attrs);
 }
@@ -867,7 +916,14 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
 				 size_t size, enum dma_data_direction dir,
 				 unsigned long attrs)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
 }
@@ -875,11 +931,17 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
 static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
 				      dma_addr_t *dma_addr, gfp_t flag)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
 	unsigned long iova;
 	void *addr;
 
 	*dma_addr = DMA_MAPPING_ERROR;
+	if (!token.group)
+		return NULL;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 	addr = vduse_domain_alloc_coherent(domain, size,
 					   (dma_addr_t *)&iova, flag);
 	if (!addr)
@@ -894,14 +956,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size,
 				    void *vaddr, dma_addr_t dma_addr,
 				    unsigned long attrs)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs);
 }
 
 static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return false;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	return dma_addr < domain->bounce_size;
 }
@@ -915,7 +991,14 @@ static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)
 
 static size_t vduse_dev_max_mapping_size(union virtio_map token)
 {
-	struct vduse_iova_domain *domain = token.iova_domain;
+	struct vduse_dev *vdev;
+	struct vduse_iova_domain *domain;
+
+	if (!token.group)
+		return 0;
+
+	vdev = token.group->dev;
+	domain = vdev->domain;
 
 	return domain->bounce_size;
 }
@@ -1733,6 +1816,7 @@ static int vduse_destroy_dev(char *name)
 	if (dev->domain)
 		vduse_domain_destroy(dev->domain);
 	kfree(dev->name);
+	kfree(dev->groups);
 	vduse_dev_destroy(dev);
 	module_put(THIS_MODULE);
 
@@ -1899,7 +1983,15 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 	dev->device_features = config->features;
 	dev->device_id = config->device_id;
 	dev->vendor_id = config->vendor_id;
+
 	dev->ngroups = (dev->api_version < 1) ? 1 : (config->ngroups ?: 1);
+	dev->groups = kcalloc(dev->ngroups, sizeof(dev->groups[0]),
+			      GFP_KERNEL);
+	if (!dev->groups)
+		goto err_vq_groups;
+	for (u32 i = 0; i < dev->ngroups; ++i)
+		dev->groups[i].dev = dev;
+
 	dev->name = kstrdup(config->name, GFP_KERNEL);
 	if (!dev->name)
 		goto err_str;
@@ -1936,6 +2028,8 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 err_idr:
 	kfree(dev->name);
 err_str:
+	kfree(dev->groups);
+err_vq_groups:
 	vduse_dev_destroy(dev);
 err:
 	return ret;
@@ -2097,7 +2191,6 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
 		return -ENOMEM;
 	}
 
-	dev->vdev->vdpa.vmap.iova_domain = dev->domain;
 	ret = _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num);
 	if (ret) {
 		put_device(&dev->vdev->vdpa.dev);
include/linux/virtio.h +3 -3
index 96c66126c074..302109029700 100644
@@ -41,13 +41,13 @@ struct virtqueue {
 	void *priv;
 };
 
-struct vduse_iova_domain;
+struct vduse_vq_group;
 
 union virtio_map {
 	/* Device that performs DMA */
 	struct device *dma_dev;
-	/* VDUSE specific mapping data */
-	struct vduse_iova_domain *iova_domain;
+	/* VDUSE specific virtqueue group for doing map */
+	struct vduse_vq_group *group;
 };
 
 int virtqueue_add_outbuf(struct virtqueue *vq,
Signature · 1 line
2.51.0
#9 lore raw

Re: [PATCH v3 4/6] vduse: return internal vq group struct as map token

kernel test robot noticed the following build warnings:

Hi Eugenio,

kernel test robot noticed the following build warnings:

[auto build test WARNING on mst-vhost/linux-next]
[also build test WARNING on next-20250919]
[cannot apply to linus/master v6.17-rc6]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Eugenio-P-rez/vduse-make-domain_lock-an-rwlock/20250919-174108
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
patch link:    https://lore.kernel.org/r/20250919093359.961296-5-eperezma%40redhat.com
patch subject: [PATCH v3 4/6] vduse: return internal vq group struct as map token
config: microblaze-randconfig-r073-20250921 (https://download.01.org/0day-ci/archive/20250921/[email protected]/config)
compiler: microblaze-linux-gcc (GCC) 12.5.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

smatch warnings:
drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: unsigned 'vq_group' is never less than zero. drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: error code type promoted to positive: 'vq_group'

vim +/vq_group +634 drivers/vdpa/vdpa_user/vduse_dev.c

627
628 static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
629 {
630 struct vduse_dev *dev = vdpa_to_vduse(vdpa);
631 u32 vq_group = vduse_get_vq_group(vdpa, idx);
632 union virtio_map ret;
633

 > 634		if (vq_group < 0)

635 ret.group = NULL;
636 else
637 ret.group = &dev->groups[vq_group];
638
639 return ret;
640 }
641

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#10 lore raw

Re: [PATCH v3 4/6] vduse: return internal vq group struct as map token

Quoted 37 lines · kernel test robot

Hi Eugenio,

kernel test robot noticed the following build warnings:

[auto build test WARNING on mst-vhost/linux-next]
[also build test WARNING on next-20250919]
[cannot apply to linus/master v6.17-rc6]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Eugenio-P-rez/vduse-make-domain_lock-an-rwlock/20250919-174108
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
patch link:    https://lore.kernel.org/r/20250919093359.961296-5-eperezma%40redhat.com
patch subject: [PATCH v3 4/6] vduse: return internal vq group struct as map token
config: microblaze-randconfig-r073-20250921 (https://download.01.org/0day-ci/archive/20250921/[email protected]/config)
compiler: microblaze-linux-gcc (GCC) 12.5.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

smatch warnings:
drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: unsigned 'vq_group' is never less than zero. drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: error code type promoted to positive: 'vq_group'

vim +/vq_group +634 drivers/vdpa/vdpa_user/vduse_dev.c

   627
   628  static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
   629  {
   630          struct vduse_dev *dev = vdpa_to_vduse(vdpa);
   631          u32 vq_group = vduse_get_vq_group(vdpa, idx);
   632          union virtio_map ret;
   633
Quoted 9 lines
Quoted 1 line

634 if (vq_group < 0)

   635                  ret.group = NULL;
   636          else
   637                  ret.group = &dev->groups[vq_group];
   638
   639          return ret;
   640  }
   641

That's a good catch, thanks!

#11 lore raw

Re: [PATCH v3 4/6] vduse: return internal vq group struct as map token

In reply to Eugenio Pérez
Quoted 57 lines · Eugenio Pérez

Return the internal struct that represents the vq group as virtqueue map token, instead of the device. This allows the map functions to access the information per group.

At this moment all the virtqueues share the same vq group, that only can point to ASID 0. This change prepares the infrastructure for actual per-group address space handling

---
v3:
* Adapt all virtio_map_ops callbacks to handle empty tokens in case of invalid groups.
* Make setting status DRIVER_OK fail if vq group is not valid.
* Remove the _int name suffix from struct vduse_vq_group.

RFC v3:
* Make the vq groups a dynamic array to support an arbitrary number of them.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 111 ++++++++++++++++++++++++++---
 include/linux/virtio.h             |   6 +-
 2 files changed, 105 insertions(+), 12 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +6 -0
index 7ddd81456f7b..6fdabc3c910e 100644
@@ -23,6 +23,7 @@
 #include <linux/uio.h>
 #include <linux/vdpa.h>
 #include <linux/nospec.h>
+#include <linux/virtio.h>
 #include <linux/vmalloc.h>
 #include <linux/sched/mm.h>
 #include <uapi/linux/vduse.h>
@@ -88,6 +89,10 @@ struct vduse_umem {
        struct mm_struct *mm;
 };
 
+struct vduse_vq_group {
+       struct vduse_dev *dev;
+};
+
 struct vduse_dev {
        struct vduse_vdpa *vdev;
        struct device *dev;
@@ -121,6 +126,7 @@ struct vduse_dev {
        u32 vq_align;
        u32 ngroups;
        struct vduse_umem *umem;
+       struct vduse_vq_group *groups;
        struct mutex mem_lock;
        unsigned int bounce_size;
        rwlock_t domain_lock;

drivers/vdpa/vdpa_user/vduse_dev.c static int vduse_dev_set_status(struct vduse_dev *dev, u8 status) · line 319

Quoted 6 lines
drivers/vdpa/vdpa_user/vduse_dev.c +3 -0
@@ -308,6 +314,13 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
        msg.req.type = VDUSE_SET_STATUS;
        msg.req.s.status = status;
 
+       if (dev->api_version >= VDUSE_API_VERSION_1 &&
+           !(dev->status & VIRTIO_CONFIG_S_DRIVER_OK) &&
+           status & VIRTIO_CONFIG_S_DRIVER_OK)

I don't understand why the INVALID check is only valid in the 0->1 transition here.

Quoted 175 lines
drivers/vdpa/vdpa_user/vduse_dev.c +83 -8
+               for (u32 i = 0; i < dev->vq_num; ++i)
+                       if (dev->vqs[i]->vq_group == VQ_GROUP_INVALID)
+                               return -EIO;
+
        return vduse_dev_msg_sync(dev, &msg);
 }
 
@@ -612,6 +625,20 @@ static s64 vduse_get_vq_group(struct vdpa_device *vdpa, u16 idx)
        return dev->vqs[idx]->vq_group;
 }
 
+static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+       u32 vq_group = vduse_get_vq_group(vdpa, idx);
+       union virtio_map ret;
+
+       if (vq_group < 0)
+               ret.group = NULL;
+       else
+               ret.group = &dev->groups[vq_group];
+
+       return ret;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -832,6 +859,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .get_vq_affinity        = vduse_vdpa_get_vq_affinity,
        .reset                  = vduse_vdpa_reset,
        .set_map                = vduse_vdpa_set_map,
+       .get_vq_map             = vduse_get_vq_map,
        .free                   = vduse_vdpa_free,
 };
 
@@ -839,7 +867,14 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
                                             dma_addr_t dma_addr, size_t size,
                                             enum dma_data_direction dir)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        vduse_domain_sync_single_for_device(domain, dma_addr, size, dir);
 }
@@ -848,7 +883,14 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token,
                                             dma_addr_t dma_addr, size_t size,
                                             enum dma_data_direction dir)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir);
 }
@@ -858,7 +900,14 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
                                     enum dma_data_direction dir,
                                     unsigned long attrs)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return DMA_MAPPING_ERROR;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        return vduse_domain_map_page(domain, page, offset, size, dir, attrs);
 }
@@ -867,7 +916,14 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
                                 size_t size, enum dma_data_direction dir,
                                 unsigned long attrs)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
 }
@@ -875,11 +931,17 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
 static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
                                      dma_addr_t *dma_addr, gfp_t flag)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
        unsigned long iova;
        void *addr;
 
        *dma_addr = DMA_MAPPING_ERROR;
+       if (!token.group)
+               return NULL;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
        addr = vduse_domain_alloc_coherent(domain, size,
                                           (dma_addr_t *)&iova, flag);
        if (!addr)
@@ -894,14 +956,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size,
                                    void *vaddr, dma_addr_t dma_addr,
                                    unsigned long attrs)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs);
 }
 
 static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return false;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        return dma_addr < domain->bounce_size;
 }
@@ -915,7 +991,14 @@ static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)
 
 static size_t vduse_dev_max_mapping_size(union virtio_map token)
 {
-       struct vduse_iova_domain *domain = token.iova_domain;
+       struct vduse_dev *vdev;
+       struct vduse_iova_domain *domain;
+
+       if (!token.group)
+               return 0;
+
+       vdev = token.group->dev;
+       domain = vdev->domain;
 
        return domain->bounce_size;
 }
@@ -1733,6 +1816,7 @@ static int vduse_destroy_dev(char *name)
        if (dev->domain)
                vduse_domain_destroy(dev->domain);
        kfree(dev->name);
+       kfree(dev->groups);
        vduse_dev_destroy(dev);
        module_put(THIS_MODULE);
 

drivers/vdpa/vdpa_user/vduse_dev.c static int vduse_create_dev(struct vduse_dev_config *config, · line 1986

Quoted 4 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -0
@@ -1899,7 +1983,15 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        dev->device_features = config->features;
        dev->device_id = config->device_id;
        dev->vendor_id = config->vendor_id;
+

Unnecessary change?

Quoted 50 lines
drivers/vdpa/vdpa_user/vduse_dev.c +9 -1
        dev->ngroups = (dev->api_version < 1) ? 1 : (config->ngroups ?: 1);
+       dev->groups = kcalloc(dev->ngroups, sizeof(dev->groups[0]),
+                             GFP_KERNEL);
+       if (!dev->groups)
+               goto err_vq_groups;
+       for (u32 i = 0; i < dev->ngroups; ++i)
+               dev->groups[i].dev = dev;
+
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -1936,6 +2028,8 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 err_idr:
        kfree(dev->name);
 err_str:
+       kfree(dev->groups);
+err_vq_groups:
        vduse_dev_destroy(dev);
 err:
        return ret;
@@ -2097,7 +2191,6 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
                return -ENOMEM;
        }
 
-       dev->vdev->vdpa.vmap.iova_domain = dev->domain;
        ret = _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num);
        if (ret) {
                put_device(&dev->vdev->vdpa.dev);
include/linux/virtio.h +3 -4
index 96c66126c074..302109029700 100644
@@ -41,13 +41,13 @@ struct virtqueue {
        void *priv;
 };
 
-struct vduse_iova_domain;
+struct vduse_vq_group;
 
 union virtio_map {
        /* Device that performs DMA */
        struct device *dma_dev;
-       /* VDUSE specific mapping data */
-       struct vduse_iova_domain *iova_domain;
+       /* VDUSE specific virtqueue group for doing map */
+       struct vduse_vq_group *group;
 };
 
 int virtqueue_add_outbuf(struct virtqueue *vq,
--
Quoted 2 lines

2.51.0

Thanks

#12 lore raw

Re: [PATCH v3 4/6] vduse: return internal vq group struct as map token

In reply to Eugenio Pérez
kernel test robot noticed the following build warnings:

Hi Eugenio,

kernel test robot noticed the following build warnings:

[auto build test WARNING on mst-vhost/linux-next]
[also build test WARNING on next-20250919]
[cannot apply to linus/master v6.17-rc7]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Eugenio-P-rez/vduse-make-domain_lock-an-rwlock/20250919-174108
base:   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next
patch link:    https://lore.kernel.org/r/20250919093359.961296-5-eperezma%40redhat.com
patch subject: [PATCH v3 4/6] vduse: return internal vq group struct as map token
config: microblaze-randconfig-r073-20250921 (https://download.01.org/0day-ci/archive/20250922/[email protected]/config)
compiler: microblaze-linux-gcc (GCC) 12.5.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

smatch warnings:
drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: unsigned 'vq_group' is never less than zero. drivers/vdpa/vdpa_user/vduse_dev.c:634 vduse_get_vq_map() warn: error code type promoted to positive: 'vq_group'

vim +/vq_group +634 drivers/vdpa/vdpa_user/vduse_dev.c

627
628 static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
629 {
630 struct vduse_dev *dev = vdpa_to_vduse(vdpa);
631 u32 vq_group = vduse_get_vq_group(vdpa, idx);
632 union virtio_map ret;
633

 > 634		if (vq_group < 0)

635 ret.group = NULL;
636 else
637 ret.group = &dev->groups[vq_group];
638
639 return ret;
640 }
641

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#13 lore raw

PATCH v3 5/6 vduse: add vq group asid support

In reply to Eugenio Pérez

Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.

Now that the driver can change ASID in the middle of operation, the domain that each vq address point is also protected by domain_lock.

---
v3:
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash with the patch creating the vduse_as struct (Jason).
* Create VDUSE_DEV_MAX_AS instead of comparing agains a magic number (Jason)

v2:
* Convert the use of mutex to rwlock.

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
* Use of array_index_nospec in VDUSE device ioctls.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.

RFC v2:
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 327 ++++++++++++++++++++---------
 include/uapi/linux/vduse.h         |  51 ++++-
 2 files changed, 281 insertions(+), 97 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +233 -94
index 6fdabc3c910e..c6c70cfd4887 100644
@@ -42,6 +42,7 @@
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
 #define VDUSE_DEV_MAX_GROUPS 0xffff
+#define VDUSE_DEV_MAX_AS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -89,7 +90,14 @@ struct vduse_umem {
 	struct mm_struct *mm;
 };
 
+struct vduse_as {
+	struct vduse_iova_domain *domain;
+	struct vduse_umem *umem;
+	struct mutex mem_lock;
+};
+
 struct vduse_vq_group {
+	struct vduse_as *as;
 	struct vduse_dev *dev;
 };
 
@@ -97,7 +105,7 @@ struct vduse_dev {
 	struct vduse_vdpa *vdev;
 	struct device *dev;
 	struct vduse_virtqueue **vqs;
-	struct vduse_iova_domain *domain;
+	struct vduse_as *as;
 	char *name;
 	struct mutex lock;
 	spinlock_t msg_lock;
@@ -125,9 +133,8 @@ struct vduse_dev {
 	u32 vq_num;
 	u32 vq_align;
 	u32 ngroups;
-	struct vduse_umem *umem;
+	u32 nas;
 	struct vduse_vq_group *groups;
-	struct mutex mem_lock;
 	unsigned int bounce_size;
 	rwlock_t domain_lock;
 };
@@ -324,7 +331,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
 	return vduse_dev_msg_sync(dev, &msg);
 }
 
-static int vduse_dev_update_iotlb(struct vduse_dev *dev,
+static int vduse_dev_update_iotlb(struct vduse_dev *dev, u32 asid,
 				  u64 start, u64 last)
 {
 	struct vduse_dev_msg msg = { 0 };
@@ -333,8 +340,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,
 		return -EINVAL;
 
 	msg.req.type = VDUSE_UPDATE_IOTLB;
-	msg.req.iova.start = start;
-	msg.req.iova.last = last;
+	if (dev->api_version < VDUSE_API_VERSION_1) {
+		msg.req.iova.start = start;
+		msg.req.iova.last = last;
+	} else {
+		msg.req.iova_v2.start = start;
+		msg.req.iova_v2.last = last;
+		msg.req.iova_v2.asid = asid;
+	}
 
 	return vduse_dev_msg_sync(dev, &msg);
 }
@@ -446,14 +459,29 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)
 	return mask;
 }
 
+/* Force set the asid to a vq group without a message to the VDUSE device */
+static void vduse_set_group_asid_nomsg(struct vduse_dev *dev,
+				       unsigned int group, unsigned int asid)
+{
+	write_lock(&dev->domain_lock);
+	dev->groups[group].as = &dev->as[asid];
+	write_unlock(&dev->domain_lock);
+}
+
 static void vduse_dev_reset(struct vduse_dev *dev)
 {
 	int i;
-	struct vduse_iova_domain *domain = dev->domain;
 
 	/* The coherent mappings are handled in vduse_dev_free_coherent() */
-	if (domain && domain->bounce_map)
-		vduse_domain_reset_bounce_map(domain);
+	for (i = 0; i < dev->nas; i++) {
+		struct vduse_iova_domain *domain = dev->as[i].domain;
+
+		if (domain && domain->bounce_map)
+			vduse_domain_reset_bounce_map(domain);
+	}
+
+	for (i = 0; i < dev->ngroups; i++)
+		vduse_set_group_asid_nomsg(dev, i, 0);
 
 	down_write(&dev->rwsem);
 
@@ -639,6 +667,29 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
 	return ret;
 }
 
+static int vduse_set_group_asid(struct vdpa_device *vdpa, unsigned int group,
+				unsigned int asid)
+{
+	struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+	struct vduse_dev_msg msg = { 0 };
+	int r;
+
+	if (dev->api_version < VDUSE_API_VERSION_1 ||
+	    group >= dev->ngroups || asid >= dev->nas)
+		return -EINVAL;
+
+	msg.req.type = VDUSE_SET_VQ_GROUP_ASID;
+	msg.req.vq_group_asid.group = group;
+	msg.req.vq_group_asid.asid = asid;
+
+	r = vduse_dev_msg_sync(dev, &msg);
+	if (r < 0)
+		return r;
+
+	vduse_set_group_asid_nomsg(dev, group, asid);
+	return 0;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
 				struct vdpa_vq_state *state)
 {
@@ -810,13 +861,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,
 	struct vduse_dev *dev = vdpa_to_vduse(vdpa);
 	int ret;
 
-	ret = vduse_domain_set_map(dev->domain, iotlb);
+	ret = vduse_domain_set_map(dev->as[asid].domain, iotlb);
 	if (ret)
 		return ret;
 
-	ret = vduse_dev_update_iotlb(dev, 0ULL, ULLONG_MAX);
+	ret = vduse_dev_update_iotlb(dev, asid, 0ULL, ULLONG_MAX);
 	if (ret) {
-		vduse_domain_clear_map(dev->domain, iotlb);
+		vduse_domain_clear_map(dev->as[asid].domain, iotlb);
 		return ret;
 	}
 
@@ -859,6 +910,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
 	.get_vq_affinity	= vduse_vdpa_get_vq_affinity,
 	.reset			= vduse_vdpa_reset,
 	.set_map		= vduse_vdpa_set_map,
+	.set_group_asid		= vduse_set_group_asid,
 	.get_vq_map		= vduse_get_vq_map,
 	.free			= vduse_vdpa_free,
 };
@@ -874,9 +926,10 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
 		return;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
-
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
 	vduse_domain_sync_single_for_device(domain, dma_addr, size, dir);
+	read_unlock(&vdev->domain_lock);
 }
 
 static void vduse_dev_sync_single_for_cpu(union virtio_map token,
@@ -890,9 +943,10 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token,
 		return;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
-
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
 	vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir);
+	read_unlock(&vdev->domain_lock);
 }
 
 static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
@@ -902,14 +956,18 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
 {
 	struct vduse_dev *vdev;
 	struct vduse_iova_domain *domain;
+	dma_addr_t r;
 
 	if (!token.group)
 		return DMA_MAPPING_ERROR;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
+	r = vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+	read_unlock(&vdev->domain_lock);
 
-	return vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+	return r;
 }
 
 static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
@@ -923,9 +981,10 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
 		return;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
-
-	return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
+	vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+	read_unlock(&vdev->domain_lock);
 }
 
 static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
@@ -934,21 +993,21 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
 	struct vduse_dev *vdev;
 	struct vduse_iova_domain *domain;
 	unsigned long iova;
-	void *addr;
+	void *addr = NULL;
 
 	*dma_addr = DMA_MAPPING_ERROR;
 	if (!token.group)
 		return NULL;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
 	addr = vduse_domain_alloc_coherent(domain, size,
 					   (dma_addr_t *)&iova, flag);
-	if (!addr)
-		return NULL;
-
-	*dma_addr = (dma_addr_t)iova;
+	if (addr)
+		*dma_addr = (dma_addr_t)iova;
 
+	read_unlock(&vdev->domain_lock);
 	return addr;
 }
 
@@ -963,23 +1022,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size,
 		return;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
-
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
 	vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs);
+	read_unlock(&vdev->domain_lock);
 }
 
 static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr)
 {
 	struct vduse_dev *vdev;
 	struct vduse_iova_domain *domain;
+	size_t bounce_size;
 
 	if (!token.group)
 		return false;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
+	bounce_size = domain->bounce_size;
+	read_unlock(&vdev->domain_lock);
 
-	return dma_addr < domain->bounce_size;
+	return dma_addr < bounce_size;
 }
 
 static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)
@@ -993,14 +1057,18 @@ static size_t vduse_dev_max_mapping_size(union virtio_map token)
 {
 	struct vduse_dev *vdev;
 	struct vduse_iova_domain *domain;
+	size_t bounce_size;
 
 	if (!token.group)
 		return 0;
 
 	vdev = token.group->dev;
-	domain = vdev->domain;
+	read_lock(&vdev->domain_lock);
+	domain = token.group->as->domain;
+	bounce_size = domain->bounce_size;
+	read_unlock(&vdev->domain_lock);
 
-	return domain->bounce_size;
+	return bounce_size;
 }
 
 static const struct virtio_map_ops vduse_map_ops = {
@@ -1140,39 +1208,40 @@ static int vduse_dev_queue_irq_work(struct vduse_dev *dev,
 	return ret;
 }
 
-static int vduse_dev_dereg_umem(struct vduse_dev *dev,
+static int vduse_dev_dereg_umem(struct vduse_dev *dev, u32 asid,
 				u64 iova, u64 size)
 {
 	int ret;
 
-	mutex_lock(&dev->mem_lock);
+	mutex_lock(&dev->as[asid].mem_lock);
 	ret = -ENOENT;
-	if (!dev->umem)
+	if (!dev->as[asid].umem)
 		goto unlock;
 
 	ret = -EINVAL;
-	if (!dev->domain)
+	if (!dev->as[asid].domain)
 		goto unlock;
 
-	if (dev->umem->iova != iova || size != dev->domain->bounce_size)
+	if (dev->as[asid].umem->iova != iova ||
+	    size != dev->as[asid].domain->bounce_size)
 		goto unlock;
 
-	vduse_domain_remove_user_bounce_pages(dev->domain);
-	unpin_user_pages_dirty_lock(dev->umem->pages,
-				    dev->umem->npages, true);
-	atomic64_sub(dev->umem->npages, &dev->umem->mm->pinned_vm);
-	mmdrop(dev->umem->mm);
-	vfree(dev->umem->pages);
-	kfree(dev->umem);
-	dev->umem = NULL;
+	vduse_domain_remove_user_bounce_pages(dev->as[asid].domain);
+	unpin_user_pages_dirty_lock(dev->as[asid].umem->pages,
+				    dev->as[asid].umem->npages, true);
+	atomic64_sub(dev->as[asid].umem->npages, &dev->as[asid].umem->mm->pinned_vm);
+	mmdrop(dev->as[asid].umem->mm);
+	vfree(dev->as[asid].umem->pages);
+	kfree(dev->as[asid].umem);
+	dev->as[asid].umem = NULL;
 	ret = 0;
 unlock:
-	mutex_unlock(&dev->mem_lock);
+	mutex_unlock(&dev->as[asid].mem_lock);
 	return ret;
 }
 
 static int vduse_dev_reg_umem(struct vduse_dev *dev,
-			      u64 iova, u64 uaddr, u64 size)
+			      u32 asid, u64 iova, u64 uaddr, u64 size)
 {
 	struct page **page_list = NULL;
 	struct vduse_umem *umem = NULL;
@@ -1180,14 +1249,14 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
 	unsigned long npages, lock_limit;
 	int ret;
 
-	if (!dev->domain || !dev->domain->bounce_map ||
-	    size != dev->domain->bounce_size ||
+	if (!dev->as[asid].domain || !dev->as[asid].domain->bounce_map ||
+	    size != dev->as[asid].domain->bounce_size ||
 	    iova != 0 || uaddr & ~PAGE_MASK)
 		return -EINVAL;
 
-	mutex_lock(&dev->mem_lock);
+	mutex_lock(&dev->as[asid].mem_lock);
 	ret = -EEXIST;
-	if (dev->umem)
+	if (dev->as[asid].umem)
 		goto unlock;
 
 	ret = -ENOMEM;
@@ -1211,7 +1280,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
 		goto out;
 	}
 
-	ret = vduse_domain_add_user_bounce_pages(dev->domain,
+	ret = vduse_domain_add_user_bounce_pages(dev->as[asid].domain,
 						 page_list, pinned);
 	if (ret)
 		goto out;
@@ -1224,7 +1293,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
 	umem->mm = current->mm;
 	mmgrab(current->mm);
 
-	dev->umem = umem;
+	dev->as[asid].umem = umem;
 out:
 	if (ret && pinned > 0)
 		unpin_user_pages(page_list, pinned);
@@ -1235,7 +1304,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
 		vfree(page_list);
 		kfree(umem);
 	}
-	mutex_unlock(&dev->mem_lock);
+	mutex_unlock(&dev->as[asid].mem_lock);
 	return ret;
 }
 
@@ -1267,47 +1336,66 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
 	switch (cmd) {
 	case VDUSE_IOTLB_GET_FD: {
-		struct vduse_iotlb_entry entry;
+		struct vduse_iotlb_entry_v2 entry;
 		struct vhost_iotlb_map *map;
 		struct vdpa_map_file *map_file;
 		struct file *f = NULL;
+		u32 asid;
 
 		ret = -EFAULT;
-		if (copy_from_user(&entry, argp, sizeof(entry)))
-			break;
+		if (dev->api_version >= VDUSE_API_VERSION_1) {
+			if (copy_from_user(&entry, argp, sizeof(entry)))
+				break;
+		} else {
+			entry.asid = 0;
+			if (copy_from_user(&entry.v1, argp,
+					   sizeof(entry.v1)))
+				break;
+		}
 
 		ret = -EINVAL;
-		if (entry.start > entry.last)
+		if (entry.v1.start > entry.v1.last)
+			break;
+
+		if (entry.asid >= dev->nas)
 			break;
 
 		read_lock(&dev->domain_lock);
-		if (!dev->domain) {
+		asid = array_index_nospec(entry.asid, dev->nas);
+		if (!dev->as[asid].domain) {
 			read_unlock(&dev->domain_lock);
 			break;
 		}
-		spin_lock(&dev->domain->iotlb_lock);
-		map = vhost_iotlb_itree_first(dev->domain->iotlb,
-					      entry.start, entry.last);
+		spin_lock(&dev->as[asid].domain->iotlb_lock);
+		map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
+					      entry.v1.start, entry.v1.last);
 		if (map) {
 			map_file = (struct vdpa_map_file *)map->opaque;
 			f = get_file(map_file->file);
-			entry.offset = map_file->offset;
-			entry.start = map->start;
-			entry.last = map->last;
-			entry.perm = map->perm;
+			entry.v1.offset = map_file->offset;
+			entry.v1.start = map->start;
+			entry.v1.last = map->last;
+			entry.v1.perm = map->perm;
 		}
-		spin_unlock(&dev->domain->iotlb_lock);
+		spin_unlock(&dev->as[asid].domain->iotlb_lock);
 		read_unlock(&dev->domain_lock);
 		ret = -EINVAL;
 		if (!f)
 			break;
 
 		ret = -EFAULT;
-		if (copy_to_user(argp, &entry, sizeof(entry))) {
+		if (dev->api_version >= VDUSE_API_VERSION_1)
+			ret = copy_to_user(argp, &entry,
+					   sizeof(entry));
+		else
+			ret = copy_to_user(argp, &entry.v1,
+					   sizeof(entry.v1));
+
+		if (ret) {
 			fput(f);
 			break;
 		}
-		ret = receive_fd(f, NULL, perm_to_file_flags(entry.perm));
+		ret = receive_fd(f, NULL, perm_to_file_flags(entry.v1.perm));
 		fput(f);
 		break;
 	}
@@ -1452,6 +1540,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 	}
 	case VDUSE_IOTLB_REG_UMEM: {
 		struct vduse_iova_umem umem;
+		u32 asid;
 
 		ret = -EFAULT;
 		if (copy_from_user(&umem, argp, sizeof(umem)))
@@ -1459,17 +1548,21 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
 		ret = -EINVAL;
 		if (!is_mem_zero((const char *)umem.reserved,
-				 sizeof(umem.reserved)))
+				 sizeof(umem.reserved)) ||
+		    (dev->api_version < VDUSE_API_VERSION_1 &&
+		     umem.asid != 0) || umem.asid >= dev->nas)
 			break;
 
 		write_lock(&dev->domain_lock);
-		ret = vduse_dev_reg_umem(dev, umem.iova,
+		asid = array_index_nospec(umem.asid, dev->nas);
+		ret = vduse_dev_reg_umem(dev, asid, umem.iova,
 					 umem.uaddr, umem.size);
 		write_unlock(&dev->domain_lock);
 		break;
 	}
 	case VDUSE_IOTLB_DEREG_UMEM: {
 		struct vduse_iova_umem umem;
+		u32 asid;
 
 		ret = -EFAULT;
 		if (copy_from_user(&umem, argp, sizeof(umem)))
@@ -1477,10 +1570,15 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
 		ret = -EINVAL;
 		if (!is_mem_zero((const char *)umem.reserved,
-				 sizeof(umem.reserved)))
+				 sizeof(umem.reserved)) ||
+		    (dev->api_version < VDUSE_API_VERSION_1 &&
+		     umem.asid != 0) ||
+		     umem.asid >= dev->nas)
 			break;
+
 		write_lock(&dev->domain_lock);
-		ret = vduse_dev_dereg_umem(dev, umem.iova,
+		asid = array_index_nospec(umem.asid, dev->nas);
+		ret = vduse_dev_dereg_umem(dev, asid, umem.iova,
 					   umem.size);
 		write_unlock(&dev->domain_lock);
 		break;
@@ -1488,6 +1586,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 	case VDUSE_IOTLB_GET_INFO: {
 		struct vduse_iova_info info;
 		struct vhost_iotlb_map *map;
+		u32 asid;
 
 		ret = -EFAULT;
 		if (copy_from_user(&info, argp, sizeof(info)))
@@ -1501,23 +1600,31 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 				 sizeof(info.reserved)))
 			break;
 
+		if (dev->api_version < VDUSE_API_VERSION_1) {
+			if (info.asid)
+				break;
+		} else if (info.asid >= dev->nas)
+			break;
+
 		read_lock(&dev->domain_lock);
-		if (!dev->domain) {
+		asid = array_index_nospec(info.asid, dev->nas);
+		if (!dev->as[asid].domain) {
 			read_unlock(&dev->domain_lock);
 			break;
 		}
-		spin_lock(&dev->domain->iotlb_lock);
-		map = vhost_iotlb_itree_first(dev->domain->iotlb,
+		spin_lock(&dev->as[asid].domain->iotlb_lock);
+		map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
 					      info.start, info.last);
 		if (map) {
 			info.start = map->start;
 			info.last = map->last;
 			info.capability = 0;
-			if (dev->domain->bounce_map && map->start == 0 &&
-			    map->last == dev->domain->bounce_size - 1)
+			if (dev->as[asid].domain->bounce_map &&
+			    map->start == 0 &&
+			    map->last == dev->as[asid].domain->bounce_size - 1)
 				info.capability |= VDUSE_IOVA_CAP_UMEM;
 		}
-		spin_unlock(&dev->domain->iotlb_lock);
+		spin_unlock(&dev->as[asid].domain->iotlb_lock);
 		read_unlock(&dev->domain_lock);
 		if (!map)
 			break;
@@ -1542,8 +1649,10 @@ static int vduse_dev_release(struct inode *inode, struct file *file)
 	struct vduse_dev *dev = file->private_data;
 
 	write_lock(&dev->domain_lock);
-	if (dev->domain)
-		vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size);
+	for (int i = 0; i < dev->nas; i++)
+		if (dev->as[i].domain)
+			vduse_dev_dereg_umem(dev, i, 0,
+					     dev->as[i].domain->bounce_size);
 	write_unlock(&dev->domain_lock);
 	spin_lock(&dev->msg_lock);
 	/* Make sure the inflight messages can processed after reconncection */
@@ -1762,7 +1871,6 @@ static struct vduse_dev *vduse_dev_create(void)
 		return NULL;
 
 	mutex_init(&dev->lock);
-	mutex_init(&dev->mem_lock);
 	rwlock_init(&dev->domain_lock);
 	spin_lock_init(&dev->msg_lock);
 	INIT_LIST_HEAD(&dev->send_list);
@@ -1813,8 +1921,11 @@ static int vduse_destroy_dev(char *name)
 	idr_remove(&vduse_idr, dev->minor);
 	kvfree(dev->config);
 	vduse_dev_deinit_vqs(dev);
-	if (dev->domain)
-		vduse_domain_destroy(dev->domain);
+	for (int i = 0; i < dev->nas; i++) {
+		if (dev->as[i].domain)
+			vduse_domain_destroy(dev->as[i].domain);
+	}
+	kfree(dev->as);
 	kfree(dev->name);
 	kfree(dev->groups);
 	vduse_dev_destroy(dev);
@@ -1861,13 +1972,18 @@ static bool vduse_validate_config(struct vduse_dev_config *config,
 			 sizeof(config->reserved)))
 		return false;
 
-	if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+	if (api_version < VDUSE_API_VERSION_1 &&
+	    (config->ngroups || config->nas))
 		return false;
 
 	if (api_version >= VDUSE_API_VERSION_1 &&
 	    config->ngroups > VDUSE_DEV_MAX_GROUPS)
 		return false;
 
+	if (api_version >= VDUSE_API_VERSION_1 &&
+	    config->nas > VDUSE_DEV_MAX_AS)
+		return false;
+
 	if (config->vq_align > PAGE_SIZE)
 		return false;
 
@@ -1931,7 +2047,8 @@ static ssize_t bounce_size_store(struct device *device,
 
 	ret = -EPERM;
 	write_lock(&dev->domain_lock);
-	if (dev->domain)
+	/* Assuming that if the first domain is allocated, all are allocated */
+	if (dev->as[0].domain)
 		goto unlock;
 
 	ret = kstrtouint(buf, 10, &bounce_size);
@@ -1992,6 +2109,13 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 	for (u32 i = 0; i < dev->ngroups; ++i)
 		dev->groups[i].dev = dev;
 
+	dev->nas = (dev->api_version < 1) ? 1 : (config->nas ?: 1);
+	dev->as = kcalloc(dev->nas, sizeof(dev->as[0]), GFP_KERNEL);
+	if (!dev->as)
+		goto err_as;
+	for (int i = 0; i < dev->nas; i++)
+		mutex_init(&dev->as[i].mem_lock);
+
 	dev->name = kstrdup(config->name, GFP_KERNEL);
 	if (!dev->name)
 		goto err_str;
@@ -2028,6 +2152,8 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 err_idr:
 	kfree(dev->name);
 err_str:
+	kfree(dev->as);
+err_as:
 	kfree(dev->groups);
 err_vq_groups:
 	vduse_dev_destroy(dev);
@@ -2153,7 +2279,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
 	vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
 				 &vduse_vdpa_config_ops, &vduse_map_ops,
-				 dev->ngroups, 1, name, true);
+				 dev->ngroups, dev->nas, name, true);
 	if (IS_ERR(vdev))
 		return PTR_ERR(vdev);
 
@@ -2182,11 +2308,20 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
 		return ret;
 
 	write_lock(&dev->domain_lock);
-	if (!dev->domain)
-		dev->domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
-						  dev->bounce_size);
+	ret = 0;
+
+	for (int i = 0; i < dev->nas; ++i) {
+		dev->as[i].domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
+							dev->bounce_size);
+		if (!dev->as[i].domain) {
+			ret = -ENOMEM;
+			for (int j = 0; j < i; ++j)
+				vduse_domain_destroy(dev->as[j].domain);
+		}
+	}
+
 	write_unlock(&dev->domain_lock);
-	if (!dev->domain) {
+	if (ret == -ENOMEM) {
 		put_device(&dev->vdev->vdpa.dev);
 		return -ENOMEM;
 	}
@@ -2195,8 +2330,12 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
 	if (ret) {
 		put_device(&dev->vdev->vdpa.dev);
 		write_lock(&dev->domain_lock);
-		vduse_domain_destroy(dev->domain);
-		dev->domain = NULL;
+		for (int i = 0; i < dev->nas; i++) {
+			if (dev->as[i].domain) {
+				vduse_domain_destroy(dev->as[i].domain);
+				dev->as[i].domain = NULL;
+			}
+		}
 		write_unlock(&dev->domain_lock);
 		return ret;
 	}
include/uapi/linux/vduse.h +48 -3

117 lines not on this page · raw message

Signature · 1 line
2.51.0
#14 lore raw

Re: [PATCH v3 5/6] vduse: add vq group asid support

Quoted 200 lines · Eugenio Pérez

Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.

Now that the driver can change ASID in the middle of operation, the domain that each vq address point is also protected by domain_lock.

---
v3:
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash with the patch creating the vduse_as struct (Jason).
* Create VDUSE_DEV_MAX_AS instead of comparing agains a magic number (Jason)

v2:
* Convert the use of mutex to rwlock.

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
* Use of array_index_nospec in VDUSE device ioctls.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.

RFC v2:
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 327 ++++++++++++++++++++---------
 include/uapi/linux/vduse.h         |  51 ++++-
 2 files changed, 281 insertions(+), 97 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +64 -12
index 6fdabc3c910e..c6c70cfd4887 100644
@@ -42,6 +42,7 @@
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
 #define VDUSE_DEV_MAX_GROUPS 0xffff
+#define VDUSE_DEV_MAX_AS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -89,7 +90,14 @@ struct vduse_umem {
        struct mm_struct *mm;
 };
 
+struct vduse_as {
+       struct vduse_iova_domain *domain;
+       struct vduse_umem *umem;
+       struct mutex mem_lock;
+};
+
 struct vduse_vq_group {
+       struct vduse_as *as;
        struct vduse_dev *dev;
 };
 
@@ -97,7 +105,7 @@ struct vduse_dev {
        struct vduse_vdpa *vdev;
        struct device *dev;
        struct vduse_virtqueue **vqs;
-       struct vduse_iova_domain *domain;
+       struct vduse_as *as;
        char *name;
        struct mutex lock;
        spinlock_t msg_lock;
@@ -125,9 +133,8 @@ struct vduse_dev {
        u32 vq_num;
        u32 vq_align;
        u32 ngroups;
-       struct vduse_umem *umem;
+       u32 nas;
        struct vduse_vq_group *groups;
-       struct mutex mem_lock;
        unsigned int bounce_size;
        rwlock_t domain_lock;
 };
@@ -324,7 +331,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
        return vduse_dev_msg_sync(dev, &msg);
 }
 
-static int vduse_dev_update_iotlb(struct vduse_dev *dev,
+static int vduse_dev_update_iotlb(struct vduse_dev *dev, u32 asid,
                                  u64 start, u64 last)
 {
        struct vduse_dev_msg msg = { 0 };
@@ -333,8 +340,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,
                return -EINVAL;
 
        msg.req.type = VDUSE_UPDATE_IOTLB;
-       msg.req.iova.start = start;
-       msg.req.iova.last = last;
+       if (dev->api_version < VDUSE_API_VERSION_1) {
+               msg.req.iova.start = start;
+               msg.req.iova.last = last;
+       } else {
+               msg.req.iova_v2.start = start;
+               msg.req.iova_v2.last = last;
+               msg.req.iova_v2.asid = asid;
+       }
 
        return vduse_dev_msg_sync(dev, &msg);
 }
@@ -446,14 +459,29 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)
        return mask;
 }
 
+/* Force set the asid to a vq group without a message to the VDUSE device */
+static void vduse_set_group_asid_nomsg(struct vduse_dev *dev,
+                                      unsigned int group, unsigned int asid)
+{
+       write_lock(&dev->domain_lock);
+       dev->groups[group].as = &dev->as[asid];
+       write_unlock(&dev->domain_lock);
+}
+
 static void vduse_dev_reset(struct vduse_dev *dev)
 {
        int i;
-       struct vduse_iova_domain *domain = dev->domain;
 
        /* The coherent mappings are handled in vduse_dev_free_coherent() */
-       if (domain && domain->bounce_map)
-               vduse_domain_reset_bounce_map(domain);
+       for (i = 0; i < dev->nas; i++) {
+               struct vduse_iova_domain *domain = dev->as[i].domain;
+
+               if (domain && domain->bounce_map)
+                       vduse_domain_reset_bounce_map(domain);
+       }
+
+       for (i = 0; i < dev->ngroups; i++)
+               vduse_set_group_asid_nomsg(dev, i, 0);
 
        down_write(&dev->rwsem);
 
@@ -639,6 +667,29 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
        return ret;
 }
 
+static int vduse_set_group_asid(struct vdpa_device *vdpa, unsigned int group,
+                               unsigned int asid)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+       struct vduse_dev_msg msg = { 0 };
+       int r;
+
+       if (dev->api_version < VDUSE_API_VERSION_1 ||
+           group >= dev->ngroups || asid >= dev->nas)
+               return -EINVAL;
+
+       msg.req.type = VDUSE_SET_VQ_GROUP_ASID;
+       msg.req.vq_group_asid.group = group;
+       msg.req.vq_group_asid.asid = asid;
+
+       r = vduse_dev_msg_sync(dev, &msg);
+       if (r < 0)
+               return r;
+
+       vduse_set_group_asid_nomsg(dev, group, asid);
+       return 0;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -810,13 +861,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,
        struct vduse_dev *dev = vdpa_to_vduse(vdpa);
        int ret;
 
-       ret = vduse_domain_set_map(dev->domain, iotlb);
+       ret = vduse_domain_set_map(dev->as[asid].domain, iotlb);
        if (ret)
                return ret;
 
-       ret = vduse_dev_update_iotlb(dev, 0ULL, ULLONG_MAX);
+       ret = vduse_dev_update_iotlb(dev, asid, 0ULL, ULLONG_MAX);
        if (ret) {
-               vduse_domain_clear_map(dev->domain, iotlb);
+               vduse_domain_clear_map(dev->as[asid].domain, iotlb);
                return ret;
        }
 
@@ -859,6 +910,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .get_vq_affinity        = vduse_vdpa_get_vq_affinity,
        .reset                  = vduse_vdpa_reset,
        .set_map                = vduse_vdpa_set_map,
+       .set_group_asid         = vduse_set_group_asid,
        .get_vq_map             = vduse_get_vq_map,
        .free                   = vduse_vdpa_free,
 };

drivers/vdpa/vdpa_user/vduse_dev.c static void vduse_dev_sync_single_for_device(union virtio_map token, · line 929

Quoted 6 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -2
@@ -874,9 +926,10 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);

This introduce atomic operation in the datapath, I wonder if we can switch to use RCU instead.

Quoted 55 lines
drivers/vdpa/vdpa_user/vduse_dev.c +15 -7
+       domain = token.group->as->domain;
        vduse_domain_sync_single_for_device(domain, dma_addr, size, dir);
+       read_unlock(&vdev->domain_lock);
 }
 
 static void vduse_dev_sync_single_for_cpu(union virtio_map token,
@@ -890,9 +943,10 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir);
+       read_unlock(&vdev->domain_lock);
 }
 
 static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
@@ -902,14 +956,18 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       dma_addr_t r;
 
        if (!token.group)
                return DMA_MAPPING_ERROR;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       r = vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+       read_unlock(&vdev->domain_lock);
 
-       return vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+       return r;
 }
 
 static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
@@ -923,9 +981,10 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
-       return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+       read_unlock(&vdev->domain_lock);
 }
 
 static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,

drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 996

Quoted 5 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -1
@@ -934,21 +993,21 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
        unsigned long iova;
-       void *addr;
+       void *addr = NULL;

Nit: if we stick to no NULL assignment we can save some lines of code below.

Quoted 269 lines
drivers/vdpa/vdpa_user/vduse_dev.c +78 -48
        *dma_addr = DMA_MAPPING_ERROR;
        if (!token.group)
                return NULL;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        addr = vduse_domain_alloc_coherent(domain, size,
                                           (dma_addr_t *)&iova, flag);
-       if (!addr)
-               return NULL;
-
-       *dma_addr = (dma_addr_t)iova;
+       if (addr)
+               *dma_addr = (dma_addr_t)iova;
 
+       read_unlock(&vdev->domain_lock);
        return addr;
 }
 
@@ -963,23 +1022,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs);
+       read_unlock(&vdev->domain_lock);
 }
 
 static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr)
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       size_t bounce_size;
 
        if (!token.group)
                return false;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       bounce_size = domain->bounce_size;
+       read_unlock(&vdev->domain_lock);
 
-       return dma_addr < domain->bounce_size;
+       return dma_addr < bounce_size;
 }
 
 static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)
@@ -993,14 +1057,18 @@ static size_t vduse_dev_max_mapping_size(union virtio_map token)
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       size_t bounce_size;
 
        if (!token.group)
                return 0;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       bounce_size = domain->bounce_size;
+       read_unlock(&vdev->domain_lock);
 
-       return domain->bounce_size;
+       return bounce_size;
 }
 
 static const struct virtio_map_ops vduse_map_ops = {
@@ -1140,39 +1208,40 @@ static int vduse_dev_queue_irq_work(struct vduse_dev *dev,
        return ret;
 }
 
-static int vduse_dev_dereg_umem(struct vduse_dev *dev,
+static int vduse_dev_dereg_umem(struct vduse_dev *dev, u32 asid,
                                u64 iova, u64 size)
 {
        int ret;
 
-       mutex_lock(&dev->mem_lock);
+       mutex_lock(&dev->as[asid].mem_lock);
        ret = -ENOENT;
-       if (!dev->umem)
+       if (!dev->as[asid].umem)
                goto unlock;
 
        ret = -EINVAL;
-       if (!dev->domain)
+       if (!dev->as[asid].domain)
                goto unlock;
 
-       if (dev->umem->iova != iova || size != dev->domain->bounce_size)
+       if (dev->as[asid].umem->iova != iova ||
+           size != dev->as[asid].domain->bounce_size)
                goto unlock;
 
-       vduse_domain_remove_user_bounce_pages(dev->domain);
-       unpin_user_pages_dirty_lock(dev->umem->pages,
-                                   dev->umem->npages, true);
-       atomic64_sub(dev->umem->npages, &dev->umem->mm->pinned_vm);
-       mmdrop(dev->umem->mm);
-       vfree(dev->umem->pages);
-       kfree(dev->umem);
-       dev->umem = NULL;
+       vduse_domain_remove_user_bounce_pages(dev->as[asid].domain);
+       unpin_user_pages_dirty_lock(dev->as[asid].umem->pages,
+                                   dev->as[asid].umem->npages, true);
+       atomic64_sub(dev->as[asid].umem->npages, &dev->as[asid].umem->mm->pinned_vm);
+       mmdrop(dev->as[asid].umem->mm);
+       vfree(dev->as[asid].umem->pages);
+       kfree(dev->as[asid].umem);
+       dev->as[asid].umem = NULL;
        ret = 0;
 unlock:
-       mutex_unlock(&dev->mem_lock);
+       mutex_unlock(&dev->as[asid].mem_lock);
        return ret;
 }
 
 static int vduse_dev_reg_umem(struct vduse_dev *dev,
-                             u64 iova, u64 uaddr, u64 size)
+                             u32 asid, u64 iova, u64 uaddr, u64 size)
 {
        struct page **page_list = NULL;
        struct vduse_umem *umem = NULL;
@@ -1180,14 +1249,14 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
        unsigned long npages, lock_limit;
        int ret;
 
-       if (!dev->domain || !dev->domain->bounce_map ||
-           size != dev->domain->bounce_size ||
+       if (!dev->as[asid].domain || !dev->as[asid].domain->bounce_map ||
+           size != dev->as[asid].domain->bounce_size ||
            iova != 0 || uaddr & ~PAGE_MASK)
                return -EINVAL;
 
-       mutex_lock(&dev->mem_lock);
+       mutex_lock(&dev->as[asid].mem_lock);
        ret = -EEXIST;
-       if (dev->umem)
+       if (dev->as[asid].umem)
                goto unlock;
 
        ret = -ENOMEM;
@@ -1211,7 +1280,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
                goto out;
        }
 
-       ret = vduse_domain_add_user_bounce_pages(dev->domain,
+       ret = vduse_domain_add_user_bounce_pages(dev->as[asid].domain,
                                                 page_list, pinned);
        if (ret)
                goto out;
@@ -1224,7 +1293,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
        umem->mm = current->mm;
        mmgrab(current->mm);
 
-       dev->umem = umem;
+       dev->as[asid].umem = umem;
 out:
        if (ret && pinned > 0)
                unpin_user_pages(page_list, pinned);
@@ -1235,7 +1304,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
                vfree(page_list);
                kfree(umem);
        }
-       mutex_unlock(&dev->mem_lock);
+       mutex_unlock(&dev->as[asid].mem_lock);
        return ret;
 }
 
@@ -1267,47 +1336,66 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
        switch (cmd) {
        case VDUSE_IOTLB_GET_FD: {
-               struct vduse_iotlb_entry entry;
+               struct vduse_iotlb_entry_v2 entry;
                struct vhost_iotlb_map *map;
                struct vdpa_map_file *map_file;
                struct file *f = NULL;
+               u32 asid;
 
                ret = -EFAULT;
-               if (copy_from_user(&entry, argp, sizeof(entry)))
-                       break;
+               if (dev->api_version >= VDUSE_API_VERSION_1) {
+                       if (copy_from_user(&entry, argp, sizeof(entry)))
+                               break;
+               } else {
+                       entry.asid = 0;
+                       if (copy_from_user(&entry.v1, argp,
+                                          sizeof(entry.v1)))
+                               break;
+               }
 
                ret = -EINVAL;
-               if (entry.start > entry.last)
+               if (entry.v1.start > entry.v1.last)
+                       break;
+
+               if (entry.asid >= dev->nas)
                        break;
 
                read_lock(&dev->domain_lock);
-               if (!dev->domain) {
+               asid = array_index_nospec(entry.asid, dev->nas);
+               if (!dev->as[asid].domain) {
                        read_unlock(&dev->domain_lock);
                        break;
                }
-               spin_lock(&dev->domain->iotlb_lock);
-               map = vhost_iotlb_itree_first(dev->domain->iotlb,
-                                             entry.start, entry.last);
+               spin_lock(&dev->as[asid].domain->iotlb_lock);
+               map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
+                                             entry.v1.start, entry.v1.last);
                if (map) {
                        map_file = (struct vdpa_map_file *)map->opaque;
                        f = get_file(map_file->file);
-                       entry.offset = map_file->offset;
-                       entry.start = map->start;
-                       entry.last = map->last;
-                       entry.perm = map->perm;
+                       entry.v1.offset = map_file->offset;
+                       entry.v1.start = map->start;
+                       entry.v1.last = map->last;
+                       entry.v1.perm = map->perm;
                }
-               spin_unlock(&dev->domain->iotlb_lock);
+               spin_unlock(&dev->as[asid].domain->iotlb_lock);
                read_unlock(&dev->domain_lock);
                ret = -EINVAL;
                if (!f)
                        break;
 
                ret = -EFAULT;
-               if (copy_to_user(argp, &entry, sizeof(entry))) {
+               if (dev->api_version >= VDUSE_API_VERSION_1)
+                       ret = copy_to_user(argp, &entry,
+                                          sizeof(entry));
+               else
+                       ret = copy_to_user(argp, &entry.v1,
+                                          sizeof(entry.v1));
+
+               if (ret) {
                        fput(f);
                        break;
                }
-               ret = receive_fd(f, NULL, perm_to_file_flags(entry.perm));
+               ret = receive_fd(f, NULL, perm_to_file_flags(entry.v1.perm));
                fput(f);
                break;
        }
@@ -1452,6 +1540,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
        }
        case VDUSE_IOTLB_REG_UMEM: {
                struct vduse_iova_umem umem;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&umem, argp, sizeof(umem)))

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1553

Quoted 7 lines
drivers/vdpa/vdpa_user/vduse_dev.c +3 -1
@@ -1459,17 +1548,21 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
                ret = -EINVAL;
                if (!is_mem_zero((const char *)umem.reserved,
-                                sizeof(umem.reserved)))
+                                sizeof(umem.reserved)) ||
+                   (dev->api_version < VDUSE_API_VERSION_1 &&
+                    umem.asid != 0) || umem.asid >= dev->nas)

This allows umem.asid to be set even with version < 1?

Quoted 17 lines
drivers/vdpa/vdpa_user/vduse_dev.c +3 -1
                        break;
 
                write_lock(&dev->domain_lock);
-               ret = vduse_dev_reg_umem(dev, umem.iova,
+               asid = array_index_nospec(umem.asid, dev->nas);
+               ret = vduse_dev_reg_umem(dev, asid, umem.iova,
                                         umem.uaddr, umem.size);
                write_unlock(&dev->domain_lock);
                break;
        }
        case VDUSE_IOTLB_DEREG_UMEM: {
                struct vduse_iova_umem umem;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&umem, argp, sizeof(umem)))

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1576

Quoted 8 lines
drivers/vdpa/vdpa_user/vduse_dev.c +4 -1
@@ -1477,10 +1570,15 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
                ret = -EINVAL;
                if (!is_mem_zero((const char *)umem.reserved,
-                                sizeof(umem.reserved)))
+                                sizeof(umem.reserved)) ||
+                   (dev->api_version < VDUSE_API_VERSION_1 &&
+                    umem.asid != 0) ||
+                    umem.asid >= dev->nas)

And here?

Quoted 96 lines
drivers/vdpa/vdpa_user/vduse_dev.c +28 -13
                        break;
+
                write_lock(&dev->domain_lock);
-               ret = vduse_dev_dereg_umem(dev, umem.iova,
+               asid = array_index_nospec(umem.asid, dev->nas);
+               ret = vduse_dev_dereg_umem(dev, asid, umem.iova,
                                           umem.size);
                write_unlock(&dev->domain_lock);
                break;
@@ -1488,6 +1586,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
        case VDUSE_IOTLB_GET_INFO: {
                struct vduse_iova_info info;
                struct vhost_iotlb_map *map;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&info, argp, sizeof(info)))
@@ -1501,23 +1600,31 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
                                 sizeof(info.reserved)))
                        break;
 
+               if (dev->api_version < VDUSE_API_VERSION_1) {
+                       if (info.asid)
+                               break;
+               } else if (info.asid >= dev->nas)
+                       break;
+
                read_lock(&dev->domain_lock);
-               if (!dev->domain) {
+               asid = array_index_nospec(info.asid, dev->nas);
+               if (!dev->as[asid].domain) {
                        read_unlock(&dev->domain_lock);
                        break;
                }
-               spin_lock(&dev->domain->iotlb_lock);
-               map = vhost_iotlb_itree_first(dev->domain->iotlb,
+               spin_lock(&dev->as[asid].domain->iotlb_lock);
+               map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
                                              info.start, info.last);
                if (map) {
                        info.start = map->start;
                        info.last = map->last;
                        info.capability = 0;
-                       if (dev->domain->bounce_map && map->start == 0 &&
-                           map->last == dev->domain->bounce_size - 1)
+                       if (dev->as[asid].domain->bounce_map &&
+                           map->start == 0 &&
+                           map->last == dev->as[asid].domain->bounce_size - 1)
                                info.capability |= VDUSE_IOVA_CAP_UMEM;
                }
-               spin_unlock(&dev->domain->iotlb_lock);
+               spin_unlock(&dev->as[asid].domain->iotlb_lock);
                read_unlock(&dev->domain_lock);
                if (!map)
                        break;
@@ -1542,8 +1649,10 @@ static int vduse_dev_release(struct inode *inode, struct file *file)
        struct vduse_dev *dev = file->private_data;
 
        write_lock(&dev->domain_lock);
-       if (dev->domain)
-               vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size);
+       for (int i = 0; i < dev->nas; i++)
+               if (dev->as[i].domain)
+                       vduse_dev_dereg_umem(dev, i, 0,
+                                            dev->as[i].domain->bounce_size);
        write_unlock(&dev->domain_lock);
        spin_lock(&dev->msg_lock);
        /* Make sure the inflight messages can processed after reconncection */
@@ -1762,7 +1871,6 @@ static struct vduse_dev *vduse_dev_create(void)
                return NULL;
 
        mutex_init(&dev->lock);
-       mutex_init(&dev->mem_lock);
        rwlock_init(&dev->domain_lock);
        spin_lock_init(&dev->msg_lock);
        INIT_LIST_HEAD(&dev->send_list);
@@ -1813,8 +1921,11 @@ static int vduse_destroy_dev(char *name)
        idr_remove(&vduse_idr, dev->minor);
        kvfree(dev->config);
        vduse_dev_deinit_vqs(dev);
-       if (dev->domain)
-               vduse_domain_destroy(dev->domain);
+       for (int i = 0; i < dev->nas; i++) {
+               if (dev->as[i].domain)
+                       vduse_domain_destroy(dev->as[i].domain);
+       }
+       kfree(dev->as);
        kfree(dev->name);
        kfree(dev->groups);
        vduse_dev_destroy(dev);
@@ -1861,13 +1972,18 @@ static bool vduse_validate_config(struct vduse_dev_config *config,
                         sizeof(config->reserved)))
                return false;
 
-       if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+       if (api_version < VDUSE_API_VERSION_1 &&

drivers/vdpa/vdpa_user/vduse_dev.c static bool vduse_validate_config(struct vduse_dev_config *config, · line 1985

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +4 -0
+           (config->ngroups || config->nas))
                return false;
 
        if (api_version >= VDUSE_API_VERSION_1 &&
            config->ngroups > VDUSE_DEV_MAX_GROUPS)
                return false;
 
+       if (api_version >= VDUSE_API_VERSION_1 &&
+           config->nas > VDUSE_DEV_MAX_AS)
+               return false;

Nit: is this better to have:

if (api_version >= VDUSE_API_VERSION_1) {
      if () return false;
      if () return false;
}

?

drivers/vdpa/vdpa_user/vduse_dev.c static ssize_t bounce_size_store(struct device *device, · line 2050

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +2 -1
+
        if (config->vq_align > PAGE_SIZE)
                return false;
 
@@ -1931,7 +2047,8 @@ static ssize_t bounce_size_store(struct device *device,
 
        ret = -EPERM;
        write_lock(&dev->domain_lock);
-       if (dev->domain)
+       /* Assuming that if the first domain is allocated, all are allocated */

I don't fully understand the comment here.

Quoted 38 lines
drivers/vdpa/vdpa_user/vduse_dev.c +11 -1
+       if (dev->as[0].domain)
                goto unlock;
 
        ret = kstrtouint(buf, 10, &bounce_size);
@@ -1992,6 +2109,13 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        for (u32 i = 0; i < dev->ngroups; ++i)
                dev->groups[i].dev = dev;
 
+       dev->nas = (dev->api_version < 1) ? 1 : (config->nas ?: 1);
+       dev->as = kcalloc(dev->nas, sizeof(dev->as[0]), GFP_KERNEL);
+       if (!dev->as)
+               goto err_as;
+       for (int i = 0; i < dev->nas; i++)
+               mutex_init(&dev->as[i].mem_lock);
+
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -2028,6 +2152,8 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 err_idr:
        kfree(dev->name);
 err_str:
+       kfree(dev->as);
+err_as:
        kfree(dev->groups);
 err_vq_groups:
        vduse_dev_destroy(dev);
@@ -2153,7 +2279,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
        vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
                                 &vduse_vdpa_config_ops, &vduse_map_ops,
-                                dev->ngroups, 1, name, true);
+                                dev->ngroups, dev->nas, name, true);
        if (IS_ERR(vdev))
                return PTR_ERR(vdev);
 
@@ -2182,11 +2308,20 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
                return ret;

drivers/vdpa/vdpa_user/vduse_dev.c static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, · line 2315

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +5 -3
 
        write_lock(&dev->domain_lock);
-       if (!dev->domain)
-               dev->domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
-                                                 dev->bounce_size);
+       ret = 0;
+
+       for (int i = 0; i < dev->nas; ++i) {
+               dev->as[i].domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
+                                                       dev->bounce_size);

Per discussion previously, should we divide bounce_size here?

Quoted 62 lines
drivers/vdpa/vdpa_user/vduse_dev.c +14 -3
+               if (!dev->as[i].domain) {
+                       ret = -ENOMEM;
+                       for (int j = 0; j < i; ++j)
+                               vduse_domain_destroy(dev->as[j].domain);
+               }
+       }
+
        write_unlock(&dev->domain_lock);
-       if (!dev->domain) {
+       if (ret == -ENOMEM) {
                put_device(&dev->vdev->vdpa.dev);
                return -ENOMEM;
        }
@@ -2195,8 +2330,12 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
        if (ret) {
                put_device(&dev->vdev->vdpa.dev);
                write_lock(&dev->domain_lock);
-               vduse_domain_destroy(dev->domain);
-               dev->domain = NULL;
+               for (int i = 0; i < dev->nas; i++) {
+                       if (dev->as[i].domain) {
+                               vduse_domain_destroy(dev->as[i].domain);
+                               dev->as[i].domain = NULL;
+                       }
+               }
                write_unlock(&dev->domain_lock);
                return ret;
        }
include/uapi/linux/vduse.h +14 -1
index a3d51cf6df3a..da73c3f2c280 100644
@@ -47,7 +47,8 @@ struct vduse_dev_config {
        __u32 vq_num;
        __u32 vq_align;
        __u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
-       __u32 reserved[12];
+       __u32 nas; /* if VDUSE_API_VERSION >= 1 */
+       __u32 reserved[11];
        __u32 config_size;
        __u8 config[];
 };
@@ -82,6 +83,18 @@ struct vduse_iotlb_entry {
        __u8 perm;
 };
 
+/**
+ * struct vduse_iotlb_entry_v2 - entry of IOTLB to describe one IOVA region in an ASID
+ * @v1: the original vduse_iotlb_entry
+ * @asid: address space ID of the IOVA region
+ *
+ * Structure used by VDUSE_IOTLB_GET_FD ioctl to find an overlapped IOVA region.
+ */
+struct vduse_iotlb_entry_v2 {
+       struct vduse_iotlb_entry v1;
+       __u32 asid;
+};
+
 /*
  * Find the first IOVA region that overlaps with the range [start, last]
  * and return the corresponding file descriptor. Return -EINVAL means the

include/uapi/linux/vduse.h struct vduse_vq_state_packed { · line 184

Quoted 6 lines
include/uapi/linux/vduse.h +3 -0
@@ -166,6 +179,16 @@ struct vduse_vq_state_packed {
        __u16 last_used_idx;
 };
 
+/**
+ * struct vduse_vq_group - virtqueue group
+ @ @group: Index of the virtqueue group

I see double "@".

Quoted 93 lines
include/uapi/linux/vduse.h +31 -3
+ * @asid: Address space ID of the group
+ */
+struct vduse_vq_group_asid {
+       __u32 group;
+       __u32 asid;
+};
+
 /**
  * struct vduse_vq_info - information of a virtqueue
  * @index: virtqueue index
@@ -225,6 +248,7 @@ struct vduse_vq_eventfd {
  * @uaddr: start address of userspace memory, it must be aligned to page size
  * @iova: start of the IOVA region
  * @size: size of the IOVA region
+ * @asid: Address space ID of the IOVA region
  * @reserved: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM
@@ -234,7 +258,8 @@ struct vduse_iova_umem {
        __u64 uaddr;
        __u64 iova;
        __u64 size;
-       __u64 reserved[3];
+       __u32 asid;
+       __u32 reserved[5];
 };
 
 /* Register userspace memory for IOVA regions */
@@ -248,6 +273,7 @@ struct vduse_iova_umem {
  * @start: start of the IOVA region
  * @last: last of the IOVA region
  * @capability: capability of the IOVA region
+ * @asid: Address space ID of the IOVA region, only if device API version >= 1
  * @reserved: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_IOTLB_GET_INFO ioctl to get information of
@@ -258,7 +284,8 @@ struct vduse_iova_info {
        __u64 last;
 #define VDUSE_IOVA_CAP_UMEM (1 << 0)
        __u64 capability;
-       __u64 reserved[3];
+       __u32 asid; /* Only if device API version >= 1 */
+       __u32 reserved[5];
 };
 
 /*
@@ -280,6 +307,7 @@ enum vduse_req_type {
        VDUSE_GET_VQ_STATE,
        VDUSE_SET_STATUS,
        VDUSE_UPDATE_IOTLB,
+       VDUSE_SET_VQ_GROUP_ASID,
 };
 
 /**
@@ -314,6 +342,18 @@ struct vduse_iova_range {
        __u64 last;
 };
 
+/**
+ * struct vduse_iova_range - IOVA range [start, last] if API_VERSION >= 1
+ * @start: start of the IOVA range
+ * @last: last of the IOVA range
+ * @asid: address space ID of the IOVA range
+ */
+struct vduse_iova_range_v2 {
+       __u64 start;
+       __u64 last;
+       __u32 asid;
+};
+
 /**
  * struct vduse_dev_request - control request
  * @type: request type
@@ -322,6 +362,8 @@ struct vduse_iova_range {
  * @vq_state: virtqueue state, only index field is available
  * @s: device status
  * @iova: IOVA range for updating
+ * @iova_v2: IOVA range for updating if API_VERSION >= 1
+ * @vq_group_asid: ASID of a virtqueue group
  * @padding: padding
  *
  * Structure used by read(2) on /dev/vduse/$NAME.
@@ -334,6 +376,9 @@ struct vduse_dev_request {
                struct vduse_vq_state vq_state;
                struct vduse_dev_status s;
                struct vduse_iova_range iova;
+               /* Following members only if vduse api version >= 1 */;
+               struct vduse_iova_range_v2 iova_v2;
+               struct vduse_vq_group_asid vq_group_asid;
                __u32 padding[32];
        };
 };
--
Quoted 2 lines

2.51.0

Thanks

author maintainer
#15 lore raw

Re: [PATCH v3 5/6] vduse: add vq group asid support

Quoted 204 lines · Jason Wang
Quoted 202 lines · Eugenio Pérez

Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.

Now that the driver can change ASID in the middle of operation, the domain that each vq address point is also protected by domain_lock.

---
v3:
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash with the patch creating the vduse_as struct (Jason).
* Create VDUSE_DEV_MAX_AS instead of comparing agains a magic number (Jason)

v2:
* Convert the use of mutex to rwlock.

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
* Use of array_index_nospec in VDUSE device ioctls.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.

RFC v2:
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 327 ++++++++++++++++++++---------
 include/uapi/linux/vduse.h         |  51 ++++-
 2 files changed, 281 insertions(+), 97 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +64 -12
index 6fdabc3c910e..c6c70cfd4887 100644
@@ -42,6 +42,7 @@
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
 #define VDUSE_DEV_MAX_GROUPS 0xffff
+#define VDUSE_DEV_MAX_AS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -89,7 +90,14 @@ struct vduse_umem {
        struct mm_struct *mm;
 };
 
+struct vduse_as {
+       struct vduse_iova_domain *domain;
+       struct vduse_umem *umem;
+       struct mutex mem_lock;
+};
+
 struct vduse_vq_group {
+       struct vduse_as *as;
        struct vduse_dev *dev;
 };
 
@@ -97,7 +105,7 @@ struct vduse_dev {
        struct vduse_vdpa *vdev;
        struct device *dev;
        struct vduse_virtqueue **vqs;
-       struct vduse_iova_domain *domain;
+       struct vduse_as *as;
        char *name;
        struct mutex lock;
        spinlock_t msg_lock;
@@ -125,9 +133,8 @@ struct vduse_dev {
        u32 vq_num;
        u32 vq_align;
        u32 ngroups;
-       struct vduse_umem *umem;
+       u32 nas;
        struct vduse_vq_group *groups;
-       struct mutex mem_lock;
        unsigned int bounce_size;
        rwlock_t domain_lock;
 };
@@ -324,7 +331,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
        return vduse_dev_msg_sync(dev, &msg);
 }
 
-static int vduse_dev_update_iotlb(struct vduse_dev *dev,
+static int vduse_dev_update_iotlb(struct vduse_dev *dev, u32 asid,
                                  u64 start, u64 last)
 {
        struct vduse_dev_msg msg = { 0 };
@@ -333,8 +340,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,
                return -EINVAL;
 
        msg.req.type = VDUSE_UPDATE_IOTLB;
-       msg.req.iova.start = start;
-       msg.req.iova.last = last;
+       if (dev->api_version < VDUSE_API_VERSION_1) {
+               msg.req.iova.start = start;
+               msg.req.iova.last = last;
+       } else {
+               msg.req.iova_v2.start = start;
+               msg.req.iova_v2.last = last;
+               msg.req.iova_v2.asid = asid;
+       }
 
        return vduse_dev_msg_sync(dev, &msg);
 }
@@ -446,14 +459,29 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)
        return mask;
 }
 
+/* Force set the asid to a vq group without a message to the VDUSE device */
+static void vduse_set_group_asid_nomsg(struct vduse_dev *dev,
+                                      unsigned int group, unsigned int asid)
+{
+       write_lock(&dev->domain_lock);
+       dev->groups[group].as = &dev->as[asid];
+       write_unlock(&dev->domain_lock);
+}
+
 static void vduse_dev_reset(struct vduse_dev *dev)
 {
        int i;
-       struct vduse_iova_domain *domain = dev->domain;
 
        /* The coherent mappings are handled in vduse_dev_free_coherent() */
-       if (domain && domain->bounce_map)
-               vduse_domain_reset_bounce_map(domain);
+       for (i = 0; i < dev->nas; i++) {
+               struct vduse_iova_domain *domain = dev->as[i].domain;
+
+               if (domain && domain->bounce_map)
+                       vduse_domain_reset_bounce_map(domain);
+       }
+
+       for (i = 0; i < dev->ngroups; i++)
+               vduse_set_group_asid_nomsg(dev, i, 0);
 
        down_write(&dev->rwsem);
 
@@ -639,6 +667,29 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
        return ret;
 }
 
+static int vduse_set_group_asid(struct vdpa_device *vdpa, unsigned int group,
+                               unsigned int asid)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+       struct vduse_dev_msg msg = { 0 };
+       int r;
+
+       if (dev->api_version < VDUSE_API_VERSION_1 ||
+           group >= dev->ngroups || asid >= dev->nas)
+               return -EINVAL;
+
+       msg.req.type = VDUSE_SET_VQ_GROUP_ASID;
+       msg.req.vq_group_asid.group = group;
+       msg.req.vq_group_asid.asid = asid;
+
+       r = vduse_dev_msg_sync(dev, &msg);
+       if (r < 0)
+               return r;
+
+       vduse_set_group_asid_nomsg(dev, group, asid);
+       return 0;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -810,13 +861,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,
        struct vduse_dev *dev = vdpa_to_vduse(vdpa);
        int ret;
 
-       ret = vduse_domain_set_map(dev->domain, iotlb);
+       ret = vduse_domain_set_map(dev->as[asid].domain, iotlb);
        if (ret)
                return ret;
 
-       ret = vduse_dev_update_iotlb(dev, 0ULL, ULLONG_MAX);
+       ret = vduse_dev_update_iotlb(dev, asid, 0ULL, ULLONG_MAX);
        if (ret) {
-               vduse_domain_clear_map(dev->domain, iotlb);
+               vduse_domain_clear_map(dev->as[asid].domain, iotlb);
                return ret;
        }
 
@@ -859,6 +910,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .get_vq_affinity        = vduse_vdpa_get_vq_affinity,
        .reset                  = vduse_vdpa_reset,
        .set_map                = vduse_vdpa_set_map,
+       .set_group_asid         = vduse_set_group_asid,
        .get_vq_map             = vduse_get_vq_map,
        .free                   = vduse_vdpa_free,
 };
@@ -874,9 +926,10 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
                return;
 
Quoted 8 lines

drivers/vdpa/vdpa_user/vduse_dev.c static void vduse_dev_sync_single_for_device(union virtio_map token, · line 929

Quoted 4 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -2
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);

This introduce atomic operation in the datapath, I wonder if we can switch to use RCU instead.

I think it is a path that is worth exploring. The reason I chose rwlock (and just plain mutex before) is that vduse_domain_sync_single_for_device and vduse_domain_sync_single_for_cpu already take a read lock, so it seems the overhead was ok.

Can we do it on top of this series? I think both using RCU can improve the performance for sure, but just taking out one of them would not buy us a lot.

Quoted 53 lines
Quoted 53 lines
drivers/vdpa/vdpa_user/vduse_dev.c +15 -7
+       domain = token.group->as->domain;
        vduse_domain_sync_single_for_device(domain, dma_addr, size, dir);
+       read_unlock(&vdev->domain_lock);
 }
 
 static void vduse_dev_sync_single_for_cpu(union virtio_map token,
@@ -890,9 +943,10 @@ static void vduse_dev_sync_single_for_cpu(union virtio_map token,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir);
+       read_unlock(&vdev->domain_lock);
 }
 
 static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
@@ -902,14 +956,18 @@ static dma_addr_t vduse_dev_map_page(union virtio_map token, struct page *page,
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       dma_addr_t r;
 
        if (!token.group)
                return DMA_MAPPING_ERROR;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       r = vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+       read_unlock(&vdev->domain_lock);
 
-       return vduse_domain_map_page(domain, page, offset, size, dir, attrs);
+       return r;
 }
 
 static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
@@ -923,9 +981,10 @@ static void vduse_dev_unmap_page(union virtio_map token, dma_addr_t dma_addr,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
-       return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs);
+       read_unlock(&vdev->domain_lock);
 }
 
Quoted 10 lines

drivers/vdpa/vdpa_user/vduse_dev.c static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size, · line 996

Quoted 7 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -1
 static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
@@ -934,21 +993,21 @@ static void *vduse_dev_alloc_coherent(union virtio_map token, size_t size,
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
        unsigned long iova;
-       void *addr;
+       void *addr = NULL;

Nit: if we stick to no NULL assignment we can save some lines of code below.

Right, I guess I added it in a previous iteration of the code and now it is not needed anymore. Reverting!

Quoted 270 lines
Quoted 270 lines
drivers/vdpa/vdpa_user/vduse_dev.c +78 -48
        *dma_addr = DMA_MAPPING_ERROR;
        if (!token.group)
                return NULL;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        addr = vduse_domain_alloc_coherent(domain, size,
                                           (dma_addr_t *)&iova, flag);
-       if (!addr)
-               return NULL;
-
-       *dma_addr = (dma_addr_t)iova;
+       if (addr)
+               *dma_addr = (dma_addr_t)iova;
 
+       read_unlock(&vdev->domain_lock);
        return addr;
 }
 
@@ -963,23 +1022,28 @@ static void vduse_dev_free_coherent(union virtio_map token, size_t size,
                return;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
        vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs);
+       read_unlock(&vdev->domain_lock);
 }
 
 static bool vduse_dev_need_sync(union virtio_map token, dma_addr_t dma_addr)
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       size_t bounce_size;
 
        if (!token.group)
                return false;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       bounce_size = domain->bounce_size;
+       read_unlock(&vdev->domain_lock);
 
-       return dma_addr < domain->bounce_size;
+       return dma_addr < bounce_size;
 }
 
 static int vduse_dev_mapping_error(union virtio_map token, dma_addr_t dma_addr)
@@ -993,14 +1057,18 @@ static size_t vduse_dev_max_mapping_size(union virtio_map token)
 {
        struct vduse_dev *vdev;
        struct vduse_iova_domain *domain;
+       size_t bounce_size;
 
        if (!token.group)
                return 0;
 
        vdev = token.group->dev;
-       domain = vdev->domain;
+       read_lock(&vdev->domain_lock);
+       domain = token.group->as->domain;
+       bounce_size = domain->bounce_size;
+       read_unlock(&vdev->domain_lock);
 
-       return domain->bounce_size;
+       return bounce_size;
 }
 
 static const struct virtio_map_ops vduse_map_ops = {
@@ -1140,39 +1208,40 @@ static int vduse_dev_queue_irq_work(struct vduse_dev *dev,
        return ret;
 }
 
-static int vduse_dev_dereg_umem(struct vduse_dev *dev,
+static int vduse_dev_dereg_umem(struct vduse_dev *dev, u32 asid,
                                u64 iova, u64 size)
 {
        int ret;
 
-       mutex_lock(&dev->mem_lock);
+       mutex_lock(&dev->as[asid].mem_lock);
        ret = -ENOENT;
-       if (!dev->umem)
+       if (!dev->as[asid].umem)
                goto unlock;
 
        ret = -EINVAL;
-       if (!dev->domain)
+       if (!dev->as[asid].domain)
                goto unlock;
 
-       if (dev->umem->iova != iova || size != dev->domain->bounce_size)
+       if (dev->as[asid].umem->iova != iova ||
+           size != dev->as[asid].domain->bounce_size)
                goto unlock;
 
-       vduse_domain_remove_user_bounce_pages(dev->domain);
-       unpin_user_pages_dirty_lock(dev->umem->pages,
-                                   dev->umem->npages, true);
-       atomic64_sub(dev->umem->npages, &dev->umem->mm->pinned_vm);
-       mmdrop(dev->umem->mm);
-       vfree(dev->umem->pages);
-       kfree(dev->umem);
-       dev->umem = NULL;
+       vduse_domain_remove_user_bounce_pages(dev->as[asid].domain);
+       unpin_user_pages_dirty_lock(dev->as[asid].umem->pages,
+                                   dev->as[asid].umem->npages, true);
+       atomic64_sub(dev->as[asid].umem->npages, &dev->as[asid].umem->mm->pinned_vm);
+       mmdrop(dev->as[asid].umem->mm);
+       vfree(dev->as[asid].umem->pages);
+       kfree(dev->as[asid].umem);
+       dev->as[asid].umem = NULL;
        ret = 0;
 unlock:
-       mutex_unlock(&dev->mem_lock);
+       mutex_unlock(&dev->as[asid].mem_lock);
        return ret;
 }
 
 static int vduse_dev_reg_umem(struct vduse_dev *dev,
-                             u64 iova, u64 uaddr, u64 size)
+                             u32 asid, u64 iova, u64 uaddr, u64 size)
 {
        struct page **page_list = NULL;
        struct vduse_umem *umem = NULL;
@@ -1180,14 +1249,14 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
        unsigned long npages, lock_limit;
        int ret;
 
-       if (!dev->domain || !dev->domain->bounce_map ||
-           size != dev->domain->bounce_size ||
+       if (!dev->as[asid].domain || !dev->as[asid].domain->bounce_map ||
+           size != dev->as[asid].domain->bounce_size ||
            iova != 0 || uaddr & ~PAGE_MASK)
                return -EINVAL;
 
-       mutex_lock(&dev->mem_lock);
+       mutex_lock(&dev->as[asid].mem_lock);
        ret = -EEXIST;
-       if (dev->umem)
+       if (dev->as[asid].umem)
                goto unlock;
 
        ret = -ENOMEM;
@@ -1211,7 +1280,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
                goto out;
        }
 
-       ret = vduse_domain_add_user_bounce_pages(dev->domain,
+       ret = vduse_domain_add_user_bounce_pages(dev->as[asid].domain,
                                                 page_list, pinned);
        if (ret)
                goto out;
@@ -1224,7 +1293,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
        umem->mm = current->mm;
        mmgrab(current->mm);
 
-       dev->umem = umem;
+       dev->as[asid].umem = umem;
 out:
        if (ret && pinned > 0)
                unpin_user_pages(page_list, pinned);
@@ -1235,7 +1304,7 @@ static int vduse_dev_reg_umem(struct vduse_dev *dev,
                vfree(page_list);
                kfree(umem);
        }
-       mutex_unlock(&dev->mem_lock);
+       mutex_unlock(&dev->as[asid].mem_lock);
        return ret;
 }
 
@@ -1267,47 +1336,66 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
        switch (cmd) {
        case VDUSE_IOTLB_GET_FD: {
-               struct vduse_iotlb_entry entry;
+               struct vduse_iotlb_entry_v2 entry;
                struct vhost_iotlb_map *map;
                struct vdpa_map_file *map_file;
                struct file *f = NULL;
+               u32 asid;
 
                ret = -EFAULT;
-               if (copy_from_user(&entry, argp, sizeof(entry)))
-                       break;
+               if (dev->api_version >= VDUSE_API_VERSION_1) {
+                       if (copy_from_user(&entry, argp, sizeof(entry)))
+                               break;
+               } else {
+                       entry.asid = 0;
+                       if (copy_from_user(&entry.v1, argp,
+                                          sizeof(entry.v1)))
+                               break;
+               }
 
                ret = -EINVAL;
-               if (entry.start > entry.last)
+               if (entry.v1.start > entry.v1.last)
+                       break;
+
+               if (entry.asid >= dev->nas)
                        break;
 
                read_lock(&dev->domain_lock);
-               if (!dev->domain) {
+               asid = array_index_nospec(entry.asid, dev->nas);
+               if (!dev->as[asid].domain) {
                        read_unlock(&dev->domain_lock);
                        break;
                }
-               spin_lock(&dev->domain->iotlb_lock);
-               map = vhost_iotlb_itree_first(dev->domain->iotlb,
-                                             entry.start, entry.last);
+               spin_lock(&dev->as[asid].domain->iotlb_lock);
+               map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
+                                             entry.v1.start, entry.v1.last);
                if (map) {
                        map_file = (struct vdpa_map_file *)map->opaque;
                        f = get_file(map_file->file);
-                       entry.offset = map_file->offset;
-                       entry.start = map->start;
-                       entry.last = map->last;
-                       entry.perm = map->perm;
+                       entry.v1.offset = map_file->offset;
+                       entry.v1.start = map->start;
+                       entry.v1.last = map->last;
+                       entry.v1.perm = map->perm;
                }
-               spin_unlock(&dev->domain->iotlb_lock);
+               spin_unlock(&dev->as[asid].domain->iotlb_lock);
                read_unlock(&dev->domain_lock);
                ret = -EINVAL;
                if (!f)
                        break;
 
                ret = -EFAULT;
-               if (copy_to_user(argp, &entry, sizeof(entry))) {
+               if (dev->api_version >= VDUSE_API_VERSION_1)
+                       ret = copy_to_user(argp, &entry,
+                                          sizeof(entry));
+               else
+                       ret = copy_to_user(argp, &entry.v1,
+                                          sizeof(entry.v1));
+
+               if (ret) {
                        fput(f);
                        break;
                }
-               ret = receive_fd(f, NULL, perm_to_file_flags(entry.perm));
+               ret = receive_fd(f, NULL, perm_to_file_flags(entry.v1.perm));
                fput(f);
                break;
        }
@@ -1452,6 +1540,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
        }
        case VDUSE_IOTLB_REG_UMEM: {
                struct vduse_iova_umem umem;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&umem, argp, sizeof(umem)))
@@ -1459,17 +1548,21 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
Quoted 9 lines

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1553

Quoted 6 lines
drivers/vdpa/vdpa_user/vduse_dev.c +3 -1
                ret = -EINVAL;
                if (!is_mem_zero((const char *)umem.reserved,
-                                sizeof(umem.reserved)))
+                                sizeof(umem.reserved)) ||
+                   (dev->api_version < VDUSE_API_VERSION_1 &&
+                    umem.asid != 0) || umem.asid >= dev->nas)

This allows umem.asid to be set even with version < 1?

Not sure if I'm missing something. If (dev->api_version < VDUSE_API_VERSION_1 && umem.asid != 0), it takes the break.

Seems similar to comment [1].

Quoted 18 lines
Quoted 18 lines
drivers/vdpa/vdpa_user/vduse_dev.c +3 -1
                        break;
 
                write_lock(&dev->domain_lock);
-               ret = vduse_dev_reg_umem(dev, umem.iova,
+               asid = array_index_nospec(umem.asid, dev->nas);
+               ret = vduse_dev_reg_umem(dev, asid, umem.iova,
                                         umem.uaddr, umem.size);
                write_unlock(&dev->domain_lock);
                break;
        }
        case VDUSE_IOTLB_DEREG_UMEM: {
                struct vduse_iova_umem umem;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&umem, argp, sizeof(umem)))
@@ -1477,10 +1570,15 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
 
Quoted 10 lines

drivers/vdpa/vdpa_user/vduse_dev.c static long vduse_dev_ioctl(struct file *file, unsigned int cmd, · line 1576

Quoted 7 lines
drivers/vdpa/vdpa_user/vduse_dev.c +4 -1
                ret = -EINVAL;
                if (!is_mem_zero((const char *)umem.reserved,
-                                sizeof(umem.reserved)))
+                                sizeof(umem.reserved)) ||
+                   (dev->api_version < VDUSE_API_VERSION_1 &&
+                    umem.asid != 0) ||
+                    umem.asid >= dev->nas)

And here?

Same here.

Quoted 109 lines
Quoted 96 lines
drivers/vdpa/vdpa_user/vduse_dev.c +28 -13
                        break;
+
                write_lock(&dev->domain_lock);
-               ret = vduse_dev_dereg_umem(dev, umem.iova,
+               asid = array_index_nospec(umem.asid, dev->nas);
+               ret = vduse_dev_dereg_umem(dev, asid, umem.iova,
                                           umem.size);
                write_unlock(&dev->domain_lock);
                break;
@@ -1488,6 +1586,7 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
        case VDUSE_IOTLB_GET_INFO: {
                struct vduse_iova_info info;
                struct vhost_iotlb_map *map;
+               u32 asid;
 
                ret = -EFAULT;
                if (copy_from_user(&info, argp, sizeof(info)))
@@ -1501,23 +1600,31 @@ static long vduse_dev_ioctl(struct file *file, unsigned int cmd,
                                 sizeof(info.reserved)))
                        break;
 
+               if (dev->api_version < VDUSE_API_VERSION_1) {
+                       if (info.asid)
+                               break;
+               } else if (info.asid >= dev->nas)
+                       break;
+
                read_lock(&dev->domain_lock);
-               if (!dev->domain) {
+               asid = array_index_nospec(info.asid, dev->nas);
+               if (!dev->as[asid].domain) {
                        read_unlock(&dev->domain_lock);
                        break;
                }
-               spin_lock(&dev->domain->iotlb_lock);
-               map = vhost_iotlb_itree_first(dev->domain->iotlb,
+               spin_lock(&dev->as[asid].domain->iotlb_lock);
+               map = vhost_iotlb_itree_first(dev->as[asid].domain->iotlb,
                                              info.start, info.last);
                if (map) {
                        info.start = map->start;
                        info.last = map->last;
                        info.capability = 0;
-                       if (dev->domain->bounce_map && map->start == 0 &&
-                           map->last == dev->domain->bounce_size - 1)
+                       if (dev->as[asid].domain->bounce_map &&
+                           map->start == 0 &&
+                           map->last == dev->as[asid].domain->bounce_size - 1)
                                info.capability |= VDUSE_IOVA_CAP_UMEM;
                }
-               spin_unlock(&dev->domain->iotlb_lock);
+               spin_unlock(&dev->as[asid].domain->iotlb_lock);
                read_unlock(&dev->domain_lock);
                if (!map)
                        break;
@@ -1542,8 +1649,10 @@ static int vduse_dev_release(struct inode *inode, struct file *file)
        struct vduse_dev *dev = file->private_data;
 
        write_lock(&dev->domain_lock);
-       if (dev->domain)
-               vduse_dev_dereg_umem(dev, 0, dev->domain->bounce_size);
+       for (int i = 0; i < dev->nas; i++)
+               if (dev->as[i].domain)
+                       vduse_dev_dereg_umem(dev, i, 0,
+                                            dev->as[i].domain->bounce_size);
        write_unlock(&dev->domain_lock);
        spin_lock(&dev->msg_lock);
        /* Make sure the inflight messages can processed after reconncection */
@@ -1762,7 +1871,6 @@ static struct vduse_dev *vduse_dev_create(void)
                return NULL;
 
        mutex_init(&dev->lock);
-       mutex_init(&dev->mem_lock);
        rwlock_init(&dev->domain_lock);
        spin_lock_init(&dev->msg_lock);
        INIT_LIST_HEAD(&dev->send_list);
@@ -1813,8 +1921,11 @@ static int vduse_destroy_dev(char *name)
        idr_remove(&vduse_idr, dev->minor);
        kvfree(dev->config);
        vduse_dev_deinit_vqs(dev);
-       if (dev->domain)
-               vduse_domain_destroy(dev->domain);
+       for (int i = 0; i < dev->nas; i++) {
+               if (dev->as[i].domain)
+                       vduse_domain_destroy(dev->as[i].domain);
+       }
+       kfree(dev->as);
        kfree(dev->name);
        kfree(dev->groups);
        vduse_dev_destroy(dev);
@@ -1861,13 +1972,18 @@ static bool vduse_validate_config(struct vduse_dev_config *config,
                         sizeof(config->reserved)))
                return false;
 
-       if (api_version < VDUSE_API_VERSION_1 && config->ngroups)
+       if (api_version < VDUSE_API_VERSION_1 &&

drivers/vdpa/vdpa_user/vduse_dev.c static bool vduse_validate_config(struct vduse_dev_config *config, · line 1985

Quoted 10 lines
drivers/vdpa/vdpa_user/vduse_dev.c +4 -0
+           (config->ngroups || config->nas))
                return false;
 
        if (api_version >= VDUSE_API_VERSION_1 &&
            config->ngroups > VDUSE_DEV_MAX_GROUPS)
                return false;
 
+       if (api_version >= VDUSE_API_VERSION_1 &&
+           config->nas > VDUSE_DEV_MAX_AS)
+               return false;

Nit: is this better to have:

Quoted 7 lines
if (api_version >= VDUSE_API_VERSION_1) {
      if () return false;
      if () return false;
}

?

Sure, rewriting this.

Quoted 4 lines
Quoted 4 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -0
+
        if (config->vq_align > PAGE_SIZE)
                return false;
 
Quoted 9 lines

drivers/vdpa/vdpa_user/vduse_dev.c static ssize_t bounce_size_store(struct device *device, · line 2050

Quoted 6 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -1
@@ -1931,7 +2047,8 @@ static ssize_t bounce_size_store(struct device *device,
 
        ret = -EPERM;
        write_lock(&dev->domain_lock);
-       if (dev->domain)
+       /* Assuming that if the first domain is allocated, all are allocated */

I don't fully understand the comment here.

We're only checking for dev->as[0].domain, but all domains are allocated at the same time. So if dev->as[0].domain, all dev->as[i].domain when i < dev->nas are allocated.

Would you like me to rewrite the comment differently?

Quoted 39 lines
Quoted 39 lines
drivers/vdpa/vdpa_user/vduse_dev.c +11 -1
+       if (dev->as[0].domain)
                goto unlock;
 
        ret = kstrtouint(buf, 10, &bounce_size);
@@ -1992,6 +2109,13 @@ static int vduse_create_dev(struct vduse_dev_config *config,
        for (u32 i = 0; i < dev->ngroups; ++i)
                dev->groups[i].dev = dev;
 
+       dev->nas = (dev->api_version < 1) ? 1 : (config->nas ?: 1);
+       dev->as = kcalloc(dev->nas, sizeof(dev->as[0]), GFP_KERNEL);
+       if (!dev->as)
+               goto err_as;
+       for (int i = 0; i < dev->nas; i++)
+               mutex_init(&dev->as[i].mem_lock);
+
        dev->name = kstrdup(config->name, GFP_KERNEL);
        if (!dev->name)
                goto err_str;
@@ -2028,6 +2152,8 @@ static int vduse_create_dev(struct vduse_dev_config *config,
 err_idr:
        kfree(dev->name);
 err_str:
+       kfree(dev->as);
+err_as:
        kfree(dev->groups);
 err_vq_groups:
        vduse_dev_destroy(dev);
@@ -2153,7 +2279,7 @@ static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name)
 
        vdev = vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev,
                                 &vduse_vdpa_config_ops, &vduse_map_ops,
-                                dev->ngroups, 1, name, true);
+                                dev->ngroups, dev->nas, name, true);
        if (IS_ERR(vdev))
                return PTR_ERR(vdev);
 
@@ -2182,11 +2308,20 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
                return ret;
 
Quoted 12 lines

drivers/vdpa/vdpa_user/vduse_dev.c static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, · line 2315

Quoted 9 lines
drivers/vdpa/vdpa_user/vduse_dev.c +5 -3
        write_lock(&dev->domain_lock);
-       if (!dev->domain)
-               dev->domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
-                                                 dev->bounce_size);
+       ret = 0;
+
+       for (int i = 0; i < dev->nas; ++i) {
+               dev->as[i].domain = vduse_domain_create(VDUSE_IOVA_SIZE - 1,
+                                                       dev->bounce_size);

Per discussion previously, should we divide bounce_size here?

Right, I forgot it. And making it shared between vqs will cause contention between threads, at least for an atomic. So yes let's divide it.

Quoted 65 lines
Quoted 65 lines
drivers/vdpa/vdpa_user/vduse_dev.c +14 -3
+               if (!dev->as[i].domain) {
+                       ret = -ENOMEM;
+                       for (int j = 0; j < i; ++j)
+                               vduse_domain_destroy(dev->as[j].domain);
+               }
+       }
+
        write_unlock(&dev->domain_lock);
-       if (!dev->domain) {
+       if (ret == -ENOMEM) {
                put_device(&dev->vdev->vdpa.dev);
                return -ENOMEM;
        }
@@ -2195,8 +2330,12 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
        if (ret) {
                put_device(&dev->vdev->vdpa.dev);
                write_lock(&dev->domain_lock);
-               vduse_domain_destroy(dev->domain);
-               dev->domain = NULL;
+               for (int i = 0; i < dev->nas; i++) {
+                       if (dev->as[i].domain) {
+                               vduse_domain_destroy(dev->as[i].domain);
+                               dev->as[i].domain = NULL;
+                       }
+               }
                write_unlock(&dev->domain_lock);
                return ret;
        }
include/uapi/linux/vduse.h +14 -1
index a3d51cf6df3a..da73c3f2c280 100644
@@ -47,7 +47,8 @@ struct vduse_dev_config {
        __u32 vq_num;
        __u32 vq_align;
        __u32 ngroups; /* if VDUSE_API_VERSION >= 1 */
-       __u32 reserved[12];
+       __u32 nas; /* if VDUSE_API_VERSION >= 1 */
+       __u32 reserved[11];
        __u32 config_size;
        __u8 config[];
 };
@@ -82,6 +83,18 @@ struct vduse_iotlb_entry {
        __u8 perm;
 };
 
+/**
+ * struct vduse_iotlb_entry_v2 - entry of IOTLB to describe one IOVA region in an ASID
+ * @v1: the original vduse_iotlb_entry
+ * @asid: address space ID of the IOVA region
+ *
+ * Structure used by VDUSE_IOTLB_GET_FD ioctl to find an overlapped IOVA region.
+ */
+struct vduse_iotlb_entry_v2 {
+       struct vduse_iotlb_entry v1;
+       __u32 asid;
+};
+
 /*
  * Find the first IOVA region that overlaps with the range [start, last]
  * and return the corresponding file descriptor. Return -EINVAL means the
@@ -166,6 +179,16 @@ struct vduse_vq_state_packed {
        __u16 last_used_idx;
 };
 
Quoted 6 lines

include/uapi/linux/vduse.h struct vduse_vq_state_packed { · line 184

Quoted 3 lines
include/uapi/linux/vduse.h +3 -0
+/**
+ * struct vduse_vq_group - virtqueue group
+ @ @group: Index of the virtqueue group

I see double "@".

Ouch what a miss, thanks!

Quoted 89 lines
Quoted 89 lines
include/uapi/linux/vduse.h +31 -2
+ * @asid: Address space ID of the group
+ */
+struct vduse_vq_group_asid {
+       __u32 group;
+       __u32 asid;
+};
+
 /**
  * struct vduse_vq_info - information of a virtqueue
  * @index: virtqueue index
@@ -225,6 +248,7 @@ struct vduse_vq_eventfd {
  * @uaddr: start address of userspace memory, it must be aligned to page size
  * @iova: start of the IOVA region
  * @size: size of the IOVA region
+ * @asid: Address space ID of the IOVA region
  * @reserved: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM
@@ -234,7 +258,8 @@ struct vduse_iova_umem {
        __u64 uaddr;
        __u64 iova;
        __u64 size;
-       __u64 reserved[3];
+       __u32 asid;
+       __u32 reserved[5];
 };
 
 /* Register userspace memory for IOVA regions */
@@ -248,6 +273,7 @@ struct vduse_iova_umem {
  * @start: start of the IOVA region
  * @last: last of the IOVA region
  * @capability: capability of the IOVA region
+ * @asid: Address space ID of the IOVA region, only if device API version >= 1
  * @reserved: for future use, needs to be initialized to zero
  *
  * Structure used by VDUSE_IOTLB_GET_INFO ioctl to get information of
@@ -258,7 +284,8 @@ struct vduse_iova_info {
        __u64 last;
 #define VDUSE_IOVA_CAP_UMEM (1 << 0)
        __u64 capability;
-       __u64 reserved[3];
+       __u32 asid; /* Only if device API version >= 1 */
+       __u32 reserved[5];
 };
 
 /*
@@ -280,6 +307,7 @@ enum vduse_req_type {
        VDUSE_GET_VQ_STATE,
        VDUSE_SET_STATUS,
        VDUSE_UPDATE_IOTLB,
+       VDUSE_SET_VQ_GROUP_ASID,
 };
 
 /**
@@ -314,6 +342,18 @@ struct vduse_iova_range {
        __u64 last;
 };
 
+/**
+ * struct vduse_iova_range - IOVA range [start, last] if API_VERSION >= 1
+ * @start: start of the IOVA range
+ * @last: last of the IOVA range
+ * @asid: address space ID of the IOVA range
+ */
+struct vduse_iova_range_v2 {
+       __u64 start;
+       __u64 last;
+       __u32 asid;
+};
+
 /**
  * struct vduse_dev_request - control request
  * @type: request type
@@ -322,6 +362,8 @@ struct vduse_iova_range {
  * @vq_state: virtqueue state, only index field is available
  * @s: device status
  * @iova: IOVA range for updating
+ * @iova_v2: IOVA range for updating if API_VERSION >= 1
+ * @vq_group_asid: ASID of a virtqueue group
  * @padding: padding
  *
  * Structure used by read(2) on /dev/vduse/$NAME.
@@ -334,6 +376,9 @@ struct vduse_dev_request {
                struct vduse_vq_state vq_state;
                struct vduse_dev_status s;
                struct vduse_iova_range iova;
+               /* Following members only if vduse api version >= 1 */;
+               struct vduse_iova_range_v2 iova_v2;
+               struct vduse_vq_group_asid vq_group_asid;
Quoted 9 lines
Quoted 6 lines
include/uapi/linux/vduse.h +0 -1
                __u32 padding[32];
        };
 };
--

2.51.0

Thanks

[1] https://patchew.org/linux/[email protected]/[email protected]/#CACGkMEsZiX-M-PNGX8W7GprBJmiGi9Gz1=ayE=iMaP3WO3vr2Q@mail.gmail.com

#16 lore raw

Re: [PATCH v3 5/6] vduse: add vq group asid support

Quoted 221 lines · Eugenio Perez Martin
Quoted 204 lines · Jason Wang
Quoted 202 lines · Eugenio Pérez

Add support for assigning Address Space Identifiers (ASIDs) to each VQ group. This enables mapping each group into a distinct memory space.

Now that the driver can change ASID in the middle of operation, the domain that each vq address point is also protected by domain_lock.

---
v3:
* Get the vduse domain through the vduse_as in the map functions (Jason).
* Squash with the patch creating the vduse_as struct (Jason).
* Create VDUSE_DEV_MAX_AS instead of comparing agains a magic number (Jason)

v2:
* Convert the use of mutex to rwlock.

RFC v3:
* Increase VDUSE_MAX_VQ_GROUPS to 0xffff (Jason). It was set to a lower value to reduce memory consumption, but vqs are already limited to that value and userspace VDUSE is able to allocate that many vqs.
* Remove TODO about merging VDUSE_IOTLB_GET_FD ioctl with VDUSE_IOTLB_GET_INFO.
* Use of array_index_nospec in VDUSE device ioctls.
* Embed vduse_iotlb_entry into vduse_iotlb_entry_v2.
* Move the umem mutex to asid struct so there is no contention between ASIDs.

RFC v2:
* Make iotlb entry the last one of vduse_iotlb_entry_v2 so the first part of the struct is the same.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 327 ++++++++++++++++++++---------
 include/uapi/linux/vduse.h         |  51 ++++-
 2 files changed, 281 insertions(+), 97 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +64 -12
index 6fdabc3c910e..c6c70cfd4887 100644
@@ -42,6 +42,7 @@
 
 #define VDUSE_DEV_MAX (1U << MINORBITS)
 #define VDUSE_DEV_MAX_GROUPS 0xffff
+#define VDUSE_DEV_MAX_AS 0xffff
 #define VDUSE_MAX_BOUNCE_SIZE (1024 * 1024 * 1024)
 #define VDUSE_MIN_BOUNCE_SIZE (1024 * 1024)
 #define VDUSE_BOUNCE_SIZE (64 * 1024 * 1024)
@@ -89,7 +90,14 @@ struct vduse_umem {
        struct mm_struct *mm;
 };
 
+struct vduse_as {
+       struct vduse_iova_domain *domain;
+       struct vduse_umem *umem;
+       struct mutex mem_lock;
+};
+
 struct vduse_vq_group {
+       struct vduse_as *as;
        struct vduse_dev *dev;
 };
 
@@ -97,7 +105,7 @@ struct vduse_dev {
        struct vduse_vdpa *vdev;
        struct device *dev;
        struct vduse_virtqueue **vqs;
-       struct vduse_iova_domain *domain;
+       struct vduse_as *as;
        char *name;
        struct mutex lock;
        spinlock_t msg_lock;
@@ -125,9 +133,8 @@ struct vduse_dev {
        u32 vq_num;
        u32 vq_align;
        u32 ngroups;
-       struct vduse_umem *umem;
+       u32 nas;
        struct vduse_vq_group *groups;
-       struct mutex mem_lock;
        unsigned int bounce_size;
        rwlock_t domain_lock;
 };
@@ -324,7 +331,7 @@ static int vduse_dev_set_status(struct vduse_dev *dev, u8 status)
        return vduse_dev_msg_sync(dev, &msg);
 }
 
-static int vduse_dev_update_iotlb(struct vduse_dev *dev,
+static int vduse_dev_update_iotlb(struct vduse_dev *dev, u32 asid,
                                  u64 start, u64 last)
 {
        struct vduse_dev_msg msg = { 0 };
@@ -333,8 +340,14 @@ static int vduse_dev_update_iotlb(struct vduse_dev *dev,
                return -EINVAL;
 
        msg.req.type = VDUSE_UPDATE_IOTLB;
-       msg.req.iova.start = start;
-       msg.req.iova.last = last;
+       if (dev->api_version < VDUSE_API_VERSION_1) {
+               msg.req.iova.start = start;
+               msg.req.iova.last = last;
+       } else {
+               msg.req.iova_v2.start = start;
+               msg.req.iova_v2.last = last;
+               msg.req.iova_v2.asid = asid;
+       }
 
        return vduse_dev_msg_sync(dev, &msg);
 }
@@ -446,14 +459,29 @@ static __poll_t vduse_dev_poll(struct file *file, poll_table *wait)
        return mask;
 }
 
+/* Force set the asid to a vq group without a message to the VDUSE device */
+static void vduse_set_group_asid_nomsg(struct vduse_dev *dev,
+                                      unsigned int group, unsigned int asid)
+{
+       write_lock(&dev->domain_lock);
+       dev->groups[group].as = &dev->as[asid];
+       write_unlock(&dev->domain_lock);
+}
+
 static void vduse_dev_reset(struct vduse_dev *dev)
 {
        int i;
-       struct vduse_iova_domain *domain = dev->domain;
 
        /* The coherent mappings are handled in vduse_dev_free_coherent() */
-       if (domain && domain->bounce_map)
-               vduse_domain_reset_bounce_map(domain);
+       for (i = 0; i < dev->nas; i++) {
+               struct vduse_iova_domain *domain = dev->as[i].domain;
+
+               if (domain && domain->bounce_map)
+                       vduse_domain_reset_bounce_map(domain);
+       }
+
+       for (i = 0; i < dev->ngroups; i++)
+               vduse_set_group_asid_nomsg(dev, i, 0);
 
        down_write(&dev->rwsem);
 
@@ -639,6 +667,29 @@ static union virtio_map vduse_get_vq_map(struct vdpa_device *vdpa, u16 idx)
        return ret;
 }
 
+static int vduse_set_group_asid(struct vdpa_device *vdpa, unsigned int group,
+                               unsigned int asid)
+{
+       struct vduse_dev *dev = vdpa_to_vduse(vdpa);
+       struct vduse_dev_msg msg = { 0 };
+       int r;
+
+       if (dev->api_version < VDUSE_API_VERSION_1 ||
+           group >= dev->ngroups || asid >= dev->nas)
+               return -EINVAL;
+
+       msg.req.type = VDUSE_SET_VQ_GROUP_ASID;
+       msg.req.vq_group_asid.group = group;
+       msg.req.vq_group_asid.asid = asid;
+
+       r = vduse_dev_msg_sync(dev, &msg);
+       if (r < 0)
+               return r;
+
+       vduse_set_group_asid_nomsg(dev, group, asid);
+       return 0;
+}
+
 static int vduse_vdpa_get_vq_state(struct vdpa_device *vdpa, u16 idx,
                                struct vdpa_vq_state *state)
 {
@@ -810,13 +861,13 @@ static int vduse_vdpa_set_map(struct vdpa_device *vdpa,
        struct vduse_dev *dev = vdpa_to_vduse(vdpa);
        int ret;
 
-       ret = vduse_domain_set_map(dev->domain, iotlb);
+       ret = vduse_domain_set_map(dev->as[asid].domain, iotlb);
        if (ret)
                return ret;
 
-       ret = vduse_dev_update_iotlb(dev, 0ULL, ULLONG_MAX);
+       ret = vduse_dev_update_iotlb(dev, asid, 0ULL, ULLONG_MAX);
        if (ret) {
-               vduse_domain_clear_map(dev->domain, iotlb);
+               vduse_domain_clear_map(dev->as[asid].domain, iotlb);
                return ret;
        }
 
@@ -859,6 +910,7 @@ static const struct vdpa_config_ops vduse_vdpa_config_ops = {
        .get_vq_affinity        = vduse_vdpa_get_vq_affinity,
        .reset                  = vduse_vdpa_reset,
        .set_map                = vduse_vdpa_set_map,
+       .set_group_asid         = vduse_set_group_asid,
        .get_vq_map             = vduse_get_vq_map,
        .free                   = vduse_vdpa_free,
 };
@@ -874,9 +926,10 @@ static void vduse_dev_sync_single_for_device(union virtio_map token,
                return;
 
Quoted 8 lines

drivers/vdpa/vdpa_user/vduse_dev.c static void vduse_dev_sync_single_for_device(union virtio_map token, · line 929

Quoted 4 lines
drivers/vdpa/vdpa_user/vduse_dev.c +1 -2
        vdev = token.group->dev;
-       domain = vdev->domain;
-
+       read_lock(&vdev->domain_lock);

This introduce atomic operation in the datapath, I wonder if we can switch to use RCU instead.

I think it is a path that is worth exploring. The reason I chose rwlock (and just plain mutex before) is that vduse_domain_sync_single_for_device and vduse_domain_sync_single_for_cpu already take a read lock, so it seems the overhead was ok.

Quoted 3 lines

Can we do it on top of this series? I think both using RCU can improve the performance for sure, but just taking out one of them would not buy us a lot.

That's fine.

Thanks

author maintainer
#17 lore raw

PATCH v3 6/6 vduse: bump version number

In reply to Eugenio Pérez

Finalize the series by advertising VDUSE API v1 support to userspace.

Now that all required infrastructure for v1 (ASIDs, VQ groups, update_iotlb_v2) is in place, VDUSE devices can opt in to the new features.

---
 drivers/vdpa/vdpa_user/vduse_dev.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/vdpa/vdpa_user/vduse_dev.c +2 -2
index c6c70cfd4887..68d086fc19f0 100644
@@ -2181,7 +2181,7 @@ static long vduse_ioctl(struct file *file, unsigned int cmd,
 			break;
 
 		ret = -EINVAL;
-		if (api_version > VDUSE_API_VERSION)
+		if (api_version > VDUSE_API_VERSION_1)
 			break;
 
 		ret = 0;
@@ -2248,7 +2248,7 @@ static int vduse_open(struct inode *inode, struct file *file)
 	if (!control)
 		return -ENOMEM;
 
-	control->api_version = VDUSE_API_VERSION;
+	control->api_version = VDUSE_API_VERSION_1;
 	file->private_data = control;
 
 	return 0;
Signature · 1 line
2.51.0