Skip to contentSkip to conversation

vhost-user-gpu: Add blob resource and shared memory support

PATCH Superseded Dorinda Bassey 3 participants 9 messages Jul 16 – Jul 17

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover vhost-user-gpu: Add blob resource and shared memory support

Add blob resource support to the vhost-user-gpu device, enabling Vulkan pass-through via Venus with a vhost-user backend such as vhost-device-gpu.

Patches 1-2 forward the RESOURCE_BLOB and CONTEXT_INIT feature flags from the backend to the guest and register a shared memory region for SHMEM_MAP/UNMAP operations.

Patch 3 fixes a deadlock that occurs when the memory region transaction commit (needed so the KVM slot exists before the guest accesses the mapped region) triggers ADD_MEM_REG back to the backend while it is still blocked waiting for the SHMEM_MAP reply.

Dorinda Bassey (1):
vhost: Fix SHMEM_MAP transaction ordering and resulting deadlock

Matej Hrica (2):
vhost-user-gpu: Forward RESOURCE_BLOB and CONTEXT_INIT flags
vhost-user-gpu: Add shared memory region support

 hw/display/vhost-user-gpu.c | 22 ++++++++++++++++++++++
 hw/virtio/vhost-user.c      |  6 +++---
 hw/virtio/vhost.c           | 10 ++++++++++
 3 files changed, 35 insertions(+), 3 deletions(-)
Signature · 1 line
2.52.0
#1 lore raw

PATCH 1/3 vhost-user-gpu: Forward RESOURCE_BLOB and CONTEXT_INIT flags

From: Matej Hrica <[email protected]>

Forward VIRTIO_GPU_F_RESOURCE_BLOB and VIRTIO_GPU_F_CONTEXT_INIT received from a vhost backend to expose it to the guest.

Use vhost_dev_has_feature() to query the backend features, matching the existing pattern for RESOURCE_UUID.

This allows the guest to negotiate blob resource support and context initialization when the backend (e.g., vhost-device-gpu with Venus) advertises these capabilities.

---
 hw/display/vhost-user-gpu.c | 6 ++++++
 1 file changed, 6 insertions(+)
hw/display/vhost-user-gpu.c +6 -0
index 57360898ca0..06549c98e07 100644
@@ -643,6 +643,12 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_RESOURCE_UUID)) {
         g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED;
     }
+    if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_RESOURCE_BLOB)) {
+        g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_BLOB_ENABLED;
+    }
+    if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_CONTEXT_INIT)) {
+        g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED;
+    }
 
     if (!virtio_gpu_base_device_realize(qdev, NULL, NULL, errp)) {
         return;
Signature · 1 line
2.52.0
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] vhost-user-gpu: Forward RESOURCE_BLOB and CONTEXT_INIT flags

Quoted 9 lines · Dorinda Bassey

From: Matej Hrica <[email protected]>

Forward VIRTIO_GPU_F_RESOURCE_BLOB and VIRTIO_GPU_F_CONTEXT_INIT received from a vhost backend to expose it to the guest.

Use vhost_dev_has_feature() to query the backend features, matching the existing pattern for RESOURCE_UUID.

Quoted 7 lines

This allows the guest to negotiate blob resource support and context initialization when the backend (e.g., vhost-device-gpu with Venus) advertises these capabilities.

---

Nit: this patch maybe should logically come after the patch adding blob support (simple reordering).

Quoted 23 lines
 hw/display/vhost-user-gpu.c | 6 ++++++
 1 file changed, 6 insertions(+)
hw/display/vhost-user-gpu.c +6 -1
index 57360898ca0..06549c98e07 100644
@@ -643,6 +643,12 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_RESOURCE_UUID)) {
         g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED;
     }
+    if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_RESOURCE_BLOB)) {
+        g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_BLOB_ENABLED;
+    }
+    if (vhost_dev_has_feature(&g->vhost->dev, VIRTIO_GPU_F_CONTEXT_INIT)) {
+        g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED;
+    }
 
     if (!virtio_gpu_base_device_realize(qdev, NULL, NULL, errp)) {
         return;
--

2.52.0

#3 lore raw

Re: [PATCH 1/3] vhost-user-gpu: Forward RESOURCE_BLOB and CONTEXT_INIT flags

Fixed, Thanks!

Quoted 56 lines · Manos Pitsidianakis
Quoted 9 lines · Dorinda Bassey

From: Matej Hrica <[email protected]>

Forward VIRTIO_GPU_F_RESOURCE_BLOB and VIRTIO_GPU_F_CONTEXT_INIT received from a vhost backend to expose it to the guest.

Use vhost_dev_has_feature() to query the backend features, matching the existing pattern for RESOURCE_UUID.

Quoted 7 lines

This allows the guest to negotiate blob resource support and context initialization when the backend (e.g., vhost-device-gpu with Venus) advertises these capabilities.

---

Nit: this patch maybe should logically come after the patch adding blob support (simple reordering).

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev,

Quoted 8 lines
 hw/display/vhost-user-gpu.c | 6 ++++++
 1 file changed, 6 insertions(+)
hw/display/vhost-user-gpu.c +0 -0
index 57360898ca0..06549c98e07 100644
@@ -643,6 +643,12 @@ vhost_user_gpu_device_realize(DeviceState *qdev,

Error **errp)

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, · line 643

Quoted 1 line
hw/display/vhost-user-gpu.c +0 -0
     if (vhost_dev_has_feature(&g->vhost->dev,

VIRTIO_GPU_F_RESOURCE_UUID)) {

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, · line 644

Quoted 1 line
hw/display/vhost-user-gpu.c +0 -0
         g->parent_obj.conf.flags |= 1 <<

VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED;

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, · line 646

Quoted 2 lines
hw/display/vhost-user-gpu.c +1 -0
     }
+    if (vhost_dev_has_feature(&g->vhost->dev,

VIRTIO_GPU_F_RESOURCE_BLOB)) {

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)

Quoted 3 lines
hw/display/vhost-user-gpu.c +3 -0
+        g->parent_obj.conf.flags |= 1 << VIRTIO_GPU_FLAG_BLOB_ENABLED;
+    }
+    if (vhost_dev_has_feature(&g->vhost->dev,

VIRTIO_GPU_F_CONTEXT_INIT)) {

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, · line 650

Quoted 1 line
hw/display/vhost-user-gpu.c +1 -0
+        g->parent_obj.conf.flags |= 1 <<

VIRTIO_GPU_FLAG_CONTEXT_INIT_ENABLED;

Quoted 7 lines
hw/display/vhost-user-gpu.c +1 -1
+    }
 
     if (!virtio_gpu_base_device_realize(qdev, NULL, NULL, errp)) {
         return;
--

2.52.0

#4 lore raw

PATCH 2/3 vhost-user-gpu: Add shared memory region support

In reply to Dorinda Bassey

From: Matej Hrica <[email protected]>

Add a 'hostmem' property to the vhost-user-gpu which enables the shared memory in the parent object.

Register a container MR in the shmem_list for SHMEM_MAP/UNMAP operations. A container MR is used so that SHMEM_MAP FD-backed subregions each get their own KVM memory slot with the correct backing fd.

---
 hw/display/vhost-user-gpu.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
hw/display/vhost-user-gpu.c +16 -0
index 06549c98e07..7e4580aa895 100644
@@ -655,6 +655,21 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     }
 
     g->vhost_gpu_fd = -1;
+
+    /*
+     * If shared memory is enabled, register the hostmem region for
+     * SHMEM_MAP/UNMAP operations.
+     */
+    if (virtio_gpu_hostmem_enabled(g->parent_obj.conf)) {
+        VirtioSharedMemory *shmem = g_new0(VirtioSharedMemory, 1);
+        shmem->shmid = VIRTIO_GPU_SHM_ID_HOST_VISIBLE;
+        memory_region_init(&shmem->mr, OBJECT(vdev),
+                           "vhost-user-gpu-shmem",
+                           g->parent_obj.conf.hostmem);
+        memory_region_add_subregion(&g->parent_obj.hostmem, 0, &shmem->mr);
+        QTAILQ_INIT(&shmem->mmaps);
+        QSIMPLEQ_INSERT_TAIL(&vdev->shmem_list, shmem, entry);
+    }
 }
 
 static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
@@ -665,6 +680,7 @@ static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
 
 static const Property vhost_user_gpu_properties[] = {
     VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
+    DEFINE_PROP_SIZE("hostmem", VhostUserGPU, parent_obj.conf.hostmem, 0),
 };
 
 static void
Signature · 1 line
2.52.0
#5 lore raw

Re: [PATCH 2/3] vhost-user-gpu: Add shared memory region support

Quoted 30 lines · Dorinda Bassey

From: Matej Hrica <[email protected]>

Add a 'hostmem' property to the vhost-user-gpu which enables the shared memory in the parent object.

Register a container MR in the shmem_list for SHMEM_MAP/UNMAP operations. A container MR is used so that SHMEM_MAP FD-backed subregions each get their own KVM memory slot with the correct backing fd.

---
 hw/display/vhost-user-gpu.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
hw/display/vhost-user-gpu.c +5 -0
index 06549c98e07..7e4580aa895 100644
@@ -655,6 +655,21 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     }
 
     g->vhost_gpu_fd = -1;
+
+    /*
+     * If shared memory is enabled, register the hostmem region for
+     * SHMEM_MAP/UNMAP operations.
+     */

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp) · line 672

Quoted 10 lines
hw/display/vhost-user-gpu.c +10 -0
+    if (virtio_gpu_hostmem_enabled(g->parent_obj.conf)) {
+        VirtioSharedMemory *shmem = g_new0(VirtioSharedMemory, 1);
+        shmem->shmid = VIRTIO_GPU_SHM_ID_HOST_VISIBLE;
+        memory_region_init(&shmem->mr, OBJECT(vdev),
+                           "vhost-user-gpu-shmem",
+                           g->parent_obj.conf.hostmem);
+        memory_region_add_subregion(&g->parent_obj.hostmem, 0, &shmem->mr);
+        QTAILQ_INIT(&shmem->mmaps);
+        QSIMPLEQ_INSERT_TAIL(&vdev->shmem_list, shmem, entry);
+    }

Why not just use virtio_new_shmem_region()?

Quoted 14 lines
hw/display/vhost-user-gpu.c +1 -1
 }
 
 static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
@@ -665,6 +680,7 @@ static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
 
 static const Property vhost_user_gpu_properties[] = {
     VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
+    DEFINE_PROP_SIZE("hostmem", VhostUserGPU, parent_obj.conf.hostmem, 0),
 };
 
 static void
--

2.52.0

#6 lore raw

Re: [PATCH 2/3] vhost-user-gpu: Add shared memory region support

I used it now, Thanks!

Quoted 64 lines · Albert Esteve

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev,

Quoted 22 lines · Dorinda Bassey

From: Matej Hrica <[email protected]>

Add a 'hostmem' property to the vhost-user-gpu which enables the shared memory in the parent object.

Register a container MR in the shmem_list for SHMEM_MAP/UNMAP operations. A container MR is used so that SHMEM_MAP FD-backed subregions each get their own KVM memory slot with the correct backing fd.

---
 hw/display/vhost-user-gpu.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
hw/display/vhost-user-gpu.c +0 -0
index 06549c98e07..7e4580aa895 100644
@@ -655,6 +655,21 @@ vhost_user_gpu_device_realize(DeviceState *qdev,

Error **errp)

Quoted 5 lines
hw/display/vhost-user-gpu.c +2 -0
     }
 
     g->vhost_gpu_fd = -1;
+
+    /*

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)

Quoted 10 lines
hw/display/vhost-user-gpu.c +10 -0
+     * If shared memory is enabled, register the hostmem region for
+     * SHMEM_MAP/UNMAP operations.
+     */
+    if (virtio_gpu_hostmem_enabled(g->parent_obj.conf)) {
+        VirtioSharedMemory *shmem = g_new0(VirtioSharedMemory, 1);
+        shmem->shmid = VIRTIO_GPU_SHM_ID_HOST_VISIBLE;
+        memory_region_init(&shmem->mr, OBJECT(vdev),
+                           "vhost-user-gpu-shmem",
+                           g->parent_obj.conf.hostmem);
+        memory_region_add_subregion(&g->parent_obj.hostmem, 0,

&shmem->mr);

hw/display/vhost-user-gpu.c vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp) · line 672

Quoted 3 lines
hw/display/vhost-user-gpu.c +3 -0
+        QTAILQ_INIT(&shmem->mmaps);
+        QSIMPLEQ_INSERT_TAIL(&vdev->shmem_list, shmem, entry);
+    }

Why not just use virtio_new_shmem_region()?

hw/display/vhost-user-gpu.c static struct vhost_dev

Quoted 4 lines
hw/display/vhost-user-gpu.c +0 -0
 }
 
 static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
@@ -665,6 +680,7 @@ static struct vhost_dev

*vhost_user_gpu_get_vhost(VirtIODevice *vdev)

hw/display/vhost-user-gpu.c static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)

Quoted 4 lines
hw/display/vhost-user-gpu.c +1 -0
 static const Property vhost_user_gpu_properties[] = {
     VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
+    DEFINE_PROP_SIZE("hostmem", VhostUserGPU, parent_obj.conf.hostmem,

0),

Quoted 6 lines
hw/display/vhost-user-gpu.c +0 -1
 };
 
 static void
--

2.52.0

#7 lore raw

PATCH 3/3 vhost: Fix SHMEM_MAP transaction ordering and resulting deadlock

In reply to Dorinda Bassey

Commit the memory region transaction before sending the SHMEM_MAP reply so the KVM memory slot exists before the guest can access the mapped region.

This moves the commit to while the backend is still blocked waiting for the reply, which can trigger ADD_MEM_REG back to the backend and deadlock. Filter shmem mapping regions out of vhost_section() by checking the MR owner type to prevent this.

---
 hw/virtio/vhost-user.c |  6 +++---
 hw/virtio/vhost.c      | 10 ++++++++++
 2 files changed, 13 insertions(+), 3 deletions(-)
hw/virtio/vhost-user.c +3 -3
index 517cc4ca716..8d0f463aab8 100644
@@ -1998,17 +1998,17 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev *dev,
     }
 
 send_reply_commit:
-    /* Send reply and commit after transaction started */
+    /* Commit before reply so the KVM memory slot exists before guest access */
+    memory_region_transaction_commit();
+
     if (hdr->flags & VHOST_USER_NEED_REPLY_MASK) {
         payload->u64 = !!ret;
         hdr->size = sizeof(payload->u64);
         if (!vhost_user_send_resp(ioc, hdr, payload, &local_err)) {
             error_report_err(local_err);
-            memory_region_transaction_commit();
             return -EFAULT;
         }
     }
-    memory_region_transaction_commit();
     return 0;
 
 send_reply:
hw/virtio/vhost.c +10 -0
index af41841b529..e5c219be095 100644
@@ -636,6 +636,16 @@ static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section)
 {
     MemoryRegion *mr = section->mr;
 
+    /*
+     * Skip shmem mapping regions, they are managed via SHMEM_MAP/UNMAP.
+     * Including them triggers ADD_MEM_REG during the SHMEM_MAP transaction
+     * commit, deadlocking the backend.
+     */
+    if (object_dynamic_cast(memory_region_owner(mr),
+                            TYPE_VIRTIO_SHARED_MEMORY_MAPPING)) {
+        return false;
+    }
+
     if (memory_region_is_ram(mr) && !memory_region_is_rom(mr)) {
         uint8_t dirty_mask = memory_region_get_dirty_log_mask(mr);
         uint8_t handled_dirty;
Signature · 1 line
2.52.0
#8 lore raw

Re: [PATCH 3/3] vhost: Fix SHMEM_MAP transaction ordering and resulting deadlock

Hi Dorinda,

Quoted 9 lines · Dorinda Bassey

Commit the memory region transaction before sending the SHMEM_MAP reply so the KVM memory slot exists before the guest can access the mapped region.

This moves the commit to while the backend is still blocked waiting for the reply, which can trigger ADD_MEM_REG back to the backend and deadlock. Filter shmem mapping regions out of vhost_section() by checking the MR owner type to prevent this.

This is a nice alternative that prevents the deadlock without delaying the commit.

Quoted 12 lines
---
 hw/virtio/vhost-user.c |  6 +++---
 hw/virtio/vhost.c      | 10 ++++++++++
 2 files changed, 13 insertions(+), 3 deletions(-)
hw/virtio/vhost-user.c +0 -0
index 517cc4ca716..8d0f463aab8 100644

hw/virtio/vhost-user.c vhost_user_backend_handle_shmem_map(struct vhost_dev *dev, · line 2002

Quoted 6 lines
hw/virtio/vhost-user.c +2 -1
@@ -1998,17 +1998,17 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev *dev,
     }
 
 send_reply_commit:
-    /* Send reply and commit after transaction started */
+    /* Commit before reply so the KVM memory slot exists before guest access */
+    memory_region_transaction_commit();

Without the need to delay the commit, both the transaction_start() and transaction_commit() can be dropped entirely. virtio_add_shmem_map() already opens and commits a transaction via memory_region_add_subregion(). That also makes send_reply_commit redundant.

Something like this (untested, but should work fine with your filter below)?:

hw/virtio/vhost-user.c +1 -3
index d627351f45..55d8ce1f83 100644
@@ -1955,8 +1955,6 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev *dev,
         }
     }
 
-    memory_region_transaction_begin();
-
     /* Create VirtioSharedMemoryMapping object */
     VirtioSharedMemoryMapping *mapping = virtio_shared_memory_mapping_new(
         vu_mmap->shmid, fd, vu_mmap->fd_offset, vu_mmap->shm_offset,
@@ -1964,7 +1962,7 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev *dev,
 
     if (!mapping) {
         ret = -EFAULT;
-        goto send_reply_commit;
+        goto send_reply;
     }
 
     /* Add the mapping to the shared memory region */
@@ -1972,22 +1970,8 @@ vhost_user_backend_handle_shmem_map(struct

vhost_dev *dev,

         error_report("Failed to add shared memory mapping");
         object_unref(OBJECT(mapping));
         ret = -EFAULT;

- goto send_reply_commit;
- }
-
-send_reply_commit:
- /* Send reply and commit after transaction started */
- if (hdr->flags & VHOST_USER_NEED_REPLY_MASK) {

-        payload->u64 = !!ret;
-        hdr->size = sizeof(payload->u64);
-        if (!vhost_user_send_resp(ioc, hdr, payload, &local_err)) {
-            error_report_err(local_err);
-            memory_region_transaction_commit();
-            return -EFAULT;
-        }
+        goto send_reply;
     }

- memory_region_transaction_commit();
- return 0;

send_reply:

     if (hdr->flags & VHOST_USER_NEED_REPLY_MASK) {
         ...
     }
     return 0;
Quoted 22 lines
hw/virtio/vhost-user.c +1 -2
+
     if (hdr->flags & VHOST_USER_NEED_REPLY_MASK) {
         payload->u64 = !!ret;
         hdr->size = sizeof(payload->u64);
         if (!vhost_user_send_resp(ioc, hdr, payload, &local_err)) {
             error_report_err(local_err);
-            memory_region_transaction_commit();
             return -EFAULT;
         }
     }
-    memory_region_transaction_commit();
     return 0;
 
 send_reply:
hw/virtio/vhost.c +0 -0
index af41841b529..e5c219be095 100644
@@ -636,6 +636,16 @@ static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section)
 {
     MemoryRegion *mr = section->mr;
 

hw/virtio/vhost.c static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section) · line 648

Quoted 10 lines
hw/virtio/vhost.c +10 -0
+    /*
+     * Skip shmem mapping regions, they are managed via SHMEM_MAP/UNMAP.
+     * Including them triggers ADD_MEM_REG during the SHMEM_MAP transaction
+     * commit, deadlocking the backend.
+     */
+    if (object_dynamic_cast(memory_region_owner(mr),
+                            TYPE_VIRTIO_SHARED_MEMORY_MAPPING)) {
+        return false;
+    }
+

This is really nice. The filter looks correct to me.

Quoted 6 lines
hw/virtio/vhost.c +0 -1
     if (memory_region_is_ram(mr) && !memory_region_is_rom(mr)) {
         uint8_t dirty_mask = memory_region_get_dirty_log_mask(mr);
         uint8_t handled_dirty;
--

2.52.0