Skip to contentSkip to conversation

VirtIO-GPU driver fixes and improvements

PATCH Superseded Dmitry Osipenko 5 participants 23 messages Jun 30, 2022 – Jul 19, 2022

Subsystems (4)

v7 of 19 Superseded Latest is v19 →

lore
Share & tools
lore

Dmitry Osipenko

author maintainer
#0 lore raw

v7 cover VirtIO-GPU driver fixes and improvements

This series fixes few problem found in the VirtIO-GPU driver and makes couple improvements. The "DMA API usage improvement" patch will be needed later on when we will be about to add memory shrinker support to the driver, it also cleans up code nicely.

Changelog:

v7: - Factored out VirtIO fixes from [1] since I'll be working on the
dma-buf locking in a separate patchset now.

[1] https://lore.kernel.org/all/[email protected]/

- Added r-b from Thomas Hellström.

- Added more fixes-tags to the patches.

- The part of the v6 "Correct drm_gem_shmem_get_sg_table() error handling" patch got merged into linux-next recent from another patch [2], but that patch missed to zero out shmem->pages on error. Hence I updated my patch to fix the merged fix.

[2] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=c24968734abfed81c8f93dc5f44a7b7a9aecadfa

Dmitry Osipenko (9):
drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling drm/virtio: Check whether transferred 2D BO is shmem drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init()
error
drm/virtio: Unlock reservations on dma_resv_reserve_fences() error drm/virtio: Use appropriate atomic state in
virtio_gpu_plane_cleanup_fb()
drm/virtio: Simplify error handling of virtio_gpu_object_create() drm/virtio: Improve DMA API usage for shmem BOs drm/virtio: Use dev_is_pci()
drm/virtio: Return proper error codes instead of -1

 drivers/gpu/drm/virtio/virtgpu_drv.c    | 53 +++++---------------
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  5 +-
 drivers/gpu/drm/virtio/virtgpu_gem.c    |  4 +-
 drivers/gpu/drm/virtio/virtgpu_kms.c    |  7 ++-
 drivers/gpu/drm/virtio/virtgpu_object.c | 65 ++++++-------------------
 drivers/gpu/drm/virtio/virtgpu_plane.c  |  6 +--
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 21 ++++----
 7 files changed, 47 insertions(+), 114 deletions(-)
Signature · 1 line
2.36.1
author maintainer
#1 lore raw

PATCH v7 1/9 drm/virtio: Correct drm_gem_shmem_get_sg_table() error handling

Previous commit fixed checking of the ERR_PTR value returned by drm_gem_shmem_get_sg_table(), but it missed to zero out the shmem->pages, which will crash virtio_gpu_cleanup_object(). Add the missing zeroing of the shmem->pages.

Fixes c24968734abf ("drm/virtio: Fix NULL vs IS_ERR checking in virtio_gpu_object_shmem_init")
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/virtio/virtgpu_object.c +1 -0
index 1cc8f3fc8e4b..87b19b3b96e0 100644
@@ -170,6 +170,7 @@ static int virtio_gpu_object_shmem_init(struct virtio_gpu_device *vgdev,
 	shmem->pages = drm_gem_shmem_get_sg_table(&bo->base);
 	if (IS_ERR(shmem->pages)) {
 		drm_gem_shmem_unpin(&bo->base);
+		shmem->pages = NULL;
 		return PTR_ERR(shmem->pages);
 	}
 
Signature · 1 line
2.36.1
author maintainer
#2 lore raw

PATCH v7 2/9 drm/virtio: Check whether transferred 2D BO is shmem

In reply to Dmitry Osipenko

Transferred 2D BO always must be a shmem BO. Add check for that to prevent NULL dereference if userspace passes a VRAM BO.

---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/virtio/virtgpu_vq.c +1 -1
index b7529b2b9883..1262fd0b3bef 100644
@@ -597,7 +597,7 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
-	if (use_dma_api)
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
 		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
 					    shmem->pages, DMA_TO_DEVICE);
 
Signature · 1 line
2.36.1
author maintainer
#3 lore raw

PATCH v7 3/9 drm/virtio: Unlock reservations on virtio_gpu_object_shmem_init() error

In reply to Dmitry Osipenko

Unlock reservations in the error code path of virtio_gpu_object_create() to silence debug warning splat produced by ww_mutex_destroy(&obj->lock) when GEM is released with the held lock.

Fixes 30172efbfb84 ("drm/virtio: blob prep: refactor getting pages and attaching backing")
---
 drivers/gpu/drm/virtio/virtgpu_object.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/virtio/virtgpu_object.c +2 -0
index 87b19b3b96e0..75a159df0af6 100644
@@ -249,6 +249,8 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
 	if (ret != 0) {
+		if (fence)
+			virtio_gpu_array_unlock_resv(objs);
 		virtio_gpu_array_put_free(objs);
 		virtio_gpu_free_object(&shmem_obj->base);
 		return ret;
Signature · 1 line
2.36.1
author maintainer
#4 lore raw

PATCH v7 4/9 drm/virtio: Unlock reservations on dma_resv_reserve_fences() error

In reply to Dmitry Osipenko

Unlock reservations on dma_resv_reserve_fences() error to fix recursive locking of the reservations when this error happens.

Fixes c8d4c18bfbc4 ("dma-buf/drivers: make reserving a shared slot mandatory v4")
---
 drivers/gpu/drm/virtio/virtgpu_gem.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/gpu/drm/virtio/virtgpu_gem.c +3 -1
index 580a78809836..7db48d17ee3a 100644
@@ -228,8 +228,10 @@ int virtio_gpu_array_lock_resv(struct virtio_gpu_object_array *objs)
 
 	for (i = 0; i < objs->nents; ++i) {
 		ret = dma_resv_reserve_fences(objs->objs[i]->resv, 1);
-		if (ret)
+		if (ret) {
+			virtio_gpu_array_unlock_resv(objs);
 			return ret;
+		}
 	}
 	return ret;
 }
Signature · 1 line
2.36.1
author maintainer
#5 lore raw

PATCH v7 5/9 drm/virtio: Use appropriate atomic state in virtio_gpu_plane_cleanup_fb()

In reply to Dmitry Osipenko

Make virtio_gpu_plane_cleanup_fb() to clean the state which DRM core wants to clean up and not the current plane's state. Normally the older atomic state is cleaned up, but the newer state could also be cleaned up in case of aborted commits.

---
 drivers/gpu/drm/virtio/virtgpu_plane.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_plane.c +3 -3
index 6d3cc9e238a4..7148f3813d8b 100644
@@ -266,14 +266,14 @@ static int virtio_gpu_plane_prepare_fb(struct drm_plane *plane,
 }
 
 static void virtio_gpu_plane_cleanup_fb(struct drm_plane *plane,
-					struct drm_plane_state *old_state)
+					struct drm_plane_state *state)
 {
 	struct virtio_gpu_framebuffer *vgfb;
 
-	if (!plane->state->fb)
+	if (!state->fb)
 		return;
 
-	vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
+	vgfb = to_virtio_gpu_framebuffer(state->fb);
 	if (vgfb->fence) {
 		dma_fence_put(&vgfb->fence->f);
 		vgfb->fence = NULL;
Signature · 1 line
2.36.1
author maintainer
#6 lore raw

PATCH v7 6/9 drm/virtio: Simplify error handling of virtio_gpu_object_create()

In reply to Dmitry Osipenko

Change the order of SHMEM initialization and reservation locking to make code cleaner and to prepare for transitioning of the common GEM SHMEM code to use the GEM's reservation lock instead of the shmem.page_lock.

There is no need to lock reservation during allocation of the SHMEM pages because the lock is needed only to avoid racing with the async host-side allocation. Hence we can safely move the SHMEM initialization out of the reservation lock.

---
 drivers/gpu/drm/virtio/virtgpu_object.c | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)
drivers/gpu/drm/virtio/virtgpu_object.c +4 -9
index 75a159df0af6..62b4d075cfac 100644
@@ -235,6 +235,10 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 
 	bo->dumb = params->dumb;
 
+	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
+	if (ret != 0)
+		goto err_put_id;
+
 	if (fence) {
 		ret = -ENOMEM;
 		objs = virtio_gpu_array_alloc(1);
@@ -247,15 +251,6 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
 			goto err_put_objs;
 	}
 
-	ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents);
-	if (ret != 0) {
-		if (fence)
-			virtio_gpu_array_unlock_resv(objs);
-		virtio_gpu_array_put_free(objs);
-		virtio_gpu_free_object(&shmem_obj->base);
-		return ret;
-	}
-
 	if (params->blob) {
 		if (params->blob_mem == VIRTGPU_BLOB_MEM_GUEST)
 			bo->guest_blob = true;
Signature · 1 line
2.36.1
author maintainer
#7 lore raw

PATCH v7 7/9 drm/virtio: Improve DMA API usage for shmem BOs

In reply to Dmitry Osipenko

DRM API requires the DRM's driver to be backed with the device that can be used for generic DMA operations. The VirtIO-GPU device can't perform DMA operations if it uses PCI transport because PCI device driver creates a virtual VirtIO-GPU device that isn't associated with the PCI. Use PCI's GPU device for the DRM's device instead of the VirtIO-GPU device and drop DMA-related hacks from the VirtIO-GPU driver.

---
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 51 ++++++-----------------
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  5 +--
 drivers/gpu/drm/virtio/virtgpu_kms.c    |  7 ++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 55 +++++--------------------
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++---
 5 files changed, 32 insertions(+), 99 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +12 -39
index 5f25a8d15464..0141b7df97ec 100644
@@ -46,12 +46,11 @@ static int virtio_gpu_modeset = -1;
 MODULE_PARM_DESC(modeset, "Disable/Enable modesetting");
 module_param_named(modeset, virtio_gpu_modeset, int, 0400);
 
-static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vdev)
+static int virtio_gpu_pci_quirk(struct drm_device *dev)
 {
-	struct pci_dev *pdev = to_pci_dev(vdev->dev.parent);
+	struct pci_dev *pdev = to_pci_dev(dev->dev);
 	const char *pname = dev_name(&pdev->dev);
 	bool vga = (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA;
-	char unique[20];
 	int ret;
 
 	DRM_INFO("pci: %s detected at %s\n",
@@ -63,39 +62,7 @@ static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vd
 			return ret;
 	}
 
-	/*
-	 * Normally the drm_dev_set_unique() call is done by core DRM.
-	 * The following comment covers, why virtio cannot rely on it.
-	 *
-	 * Unlike the other virtual GPU drivers, virtio abstracts the
-	 * underlying bus type by using struct virtio_device.
-	 *
-	 * Hence the dev_is_pci() check, used in core DRM, will fail
-	 * and the unique returned will be the virtio_device "virtio0",
-	 * while a "pci:..." one is required.
-	 *
-	 * A few other ideas were considered:
-	 * - Extend the dev_is_pci() check [in drm_set_busid] to
-	 *   consider virtio.
-	 *   Seems like a bigger hack than what we have already.
-	 *
-	 * - Point drm_device::dev to the parent of the virtio_device
-	 *   Semantic changes:
-	 *   * Using the wrong device for i2c, framebuffer_alloc and
-	 *     prime import.
-	 *   Visual changes:
-	 *   * Helpers such as DRM_DEV_ERROR, dev_info, drm_printer,
-	 *     will print the wrong information.
-	 *
-	 * We could address the latter issues, by introducing
-	 * drm_device::bus_dev, ... which would be used solely for this.
-	 *
-	 * So for the moment keep things as-is, with a bulky comment
-	 * for the next person who feels like removing this
-	 * drm_dev_set_unique() quirk.
-	 */
-	snprintf(unique, sizeof(unique), "pci:%s", pname);
-	return drm_dev_set_unique(dev, unique);
+	return 0;
 }
 
 static int virtio_gpu_probe(struct virtio_device *vdev)
@@ -109,18 +76,24 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 	if (virtio_gpu_modeset == 0)
 		return -EINVAL;
 
-	dev = drm_dev_alloc(&driver, &vdev->dev);
+	/*
+	 * The virtio-gpu device is a virtual device that doesn't have DMA
+	 * ops assigned to it, nor DMA mask set and etc. Its parent device
+	 * is actual GPU device we want to use it for the DRM's device in
+	 * order to benefit from using generic DRM APIs.
+	 */
+	dev = drm_dev_alloc(&driver, vdev->dev.parent);
 	if (IS_ERR(dev))
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
 	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
-		ret = virtio_gpu_pci_quirk(dev, vdev);
+		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
 	}
 
-	ret = virtio_gpu_init(dev);
+	ret = virtio_gpu_init(vdev, dev);
 	if (ret)
 		goto err_free;
 
drivers/gpu/drm/virtio/virtgpu_drv.h +1 -4
index f80664cf98d0..9b98470593b0 100644
@@ -101,8 +101,6 @@ struct virtio_gpu_object {
 
 struct virtio_gpu_object_shmem {
 	struct virtio_gpu_object base;
-	struct sg_table *pages;
-	uint32_t mapped;
 };
 
 struct virtio_gpu_object_vram {
@@ -215,7 +213,6 @@ struct virtio_gpu_drv_cap_cache {
 };
 
 struct virtio_gpu_device {
-	struct device *dev;
 	struct drm_device *ddev;
 
 	struct virtio_device *vdev;
@@ -283,7 +280,7 @@ extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
 void virtio_gpu_create_context(struct drm_device *dev, struct drm_file *file);
 
 /* virtgpu_kms.c */
-int virtio_gpu_init(struct drm_device *dev);
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev);
 void virtio_gpu_deinit(struct drm_device *dev);
 void virtio_gpu_release(struct drm_device *dev);
 int virtio_gpu_driver_open(struct drm_device *dev, struct drm_file *file);
drivers/gpu/drm/virtio/virtgpu_kms.c +3 -4
index 3313b92db531..0d1e3eb61bee 100644
@@ -110,7 +110,7 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev,
 	vgdev->num_capsets = num_capsets;
 }
 
-int virtio_gpu_init(struct drm_device *dev)
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 {
 	static vq_callback_t *callbacks[] = {
 		virtio_gpu_ctrl_ack, virtio_gpu_cursor_ack
@@ -123,7 +123,7 @@ int virtio_gpu_init(struct drm_device *dev)
 	u32 num_scanouts, num_capsets;
 	int ret = 0;
 
-	if (!virtio_has_feature(dev_to_virtio(dev->dev), VIRTIO_F_VERSION_1))
+	if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
 		return -ENODEV;
 
 	vgdev = kzalloc(sizeof(struct virtio_gpu_device), GFP_KERNEL);
@@ -132,8 +132,7 @@ int virtio_gpu_init(struct drm_device *dev)
 
 	vgdev->ddev = dev;
 	dev->dev_private = vgdev;
-	vgdev->vdev = dev_to_virtio(dev->dev);
-	vgdev->dev = dev->dev;
+	vgdev->vdev = vdev;
 
 	spin_lock_init(&vgdev->display_info_lock);
 	spin_lock_init(&vgdev->resource_export_lock);
drivers/gpu/drm/virtio/virtgpu_object.c +11 -44

81 lines not on this page · raw message

drivers/gpu/drm/virtio/virtgpu_vq.c +5 -8
index 1262fd0b3bef..ee84256946ab 100644
@@ -595,11 +595,10 @@ void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_transfer_to_host_2d *cmd_p;
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
-	struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
 
 	if (virtio_gpu_is_shmem(bo) && use_dma_api)
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
@@ -1019,11 +1018,9 @@ void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev,
 	struct virtio_gpu_vbuffer *vbuf;
 	bool use_dma_api = !virtio_has_dma_quirk(vgdev->vdev);
 
-	if (virtio_gpu_is_shmem(bo) && use_dma_api) {
-		struct virtio_gpu_object_shmem *shmem = to_virtio_gpu_shmem(bo);
-		dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
-					    shmem->pages, DMA_TO_DEVICE);
-	}
+	if (virtio_gpu_is_shmem(bo) && use_dma_api)
+		dma_sync_sgtable_for_device(&vgdev->vdev->dev,
+					    bo->base.sgt, DMA_TO_DEVICE);
 
 	cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
 	memset(cmd_p, 0, sizeof(*cmd_p));
Signature · 1 line
2.36.1
maintainer
#8 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Hi,

drivers/gpu/drm/virtio/virtgpu_drv.c static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vd · line 95

Quoted 3 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +0 -3
-	 * So for the moment keep things as-is, with a bulky comment
-	 * for the next person who feels like removing this
-	 * drm_dev_set_unique() quirk.

Dragons lurking here. It's not the first attempt to ditch this, and so far all have been rolled back due to regressions. Specifically Xorg is notoriously picky if it doesn't find its expectations fulfilled.

Also note that pci is not the only virtio transport we have.

What kind of testing has this patch seen?

take care,
Gerd

author maintainer
#9 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Hello Gerd,

Quoted 9 lines · Gerd Hoffmann

Hi,

drivers/gpu/drm/virtio/virtgpu_drv.c static int virtio_gpu_pci_quirk(struct drm_device *dev, struct virtio_device *vd · line 95

Quoted 3 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +0 -3
-	 * So for the moment keep things as-is, with a bulky comment
-	 * for the next person who feels like removing this
-	 * drm_dev_set_unique() quirk.

Dragons lurking here. It's not the first attempt to ditch this, and so far all have been rolled back due to regressions. Specifically Xorg is notoriously picky if it doesn't find its expectations fulfilled.

I saw the previous attempt. Back then it was a mechanically created patch that didn't get any testing.

Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

Quoted 1 line

What kind of testing has this patch seen?

The Xorg and virgl work perfectly fine in Qemu (with and without IOMMU enabled in Qemu) and in crosvm (ChromeOS virtual machine).

Signature · 2 lines
Best regards,
Dmitry
maintainer
#10 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

take care,
Gerd

#11 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 8 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

Quoted 9 lines

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

As the last person who was there - the problem is indeed when using virtio on top of mmio. If that's no longer supported by the kernel then the hacky code-path can be dropped.

Even in that case, I would suggest keeping it a separate commit.

HTH
Emil

author maintainer
#12 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

In reply to Gerd Hoffmann
Quoted 8 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

Quoted 8 lines

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

I booted kernel with this patchset applied and everything is okay, Xorg works.

[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not
called
virtio-mmio LNRO0005:01: [drm] fb0: virtio_gpudrmfb frame buffer device

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

I'd appreciate if you could give s390x a test.. I never touched s390x and it will probably take some extra effort to get into it.

Signature · 2 lines
Best regards,
Dmitry
#13 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 31 lines · Dmitry Osipenko
Quoted 8 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

Quoted 8 lines

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

I booted kernel with this patchset applied and everything is okay, Xorg works.

[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not
called
virtio-mmio LNRO0005:01: [drm] fb0: virtio_gpudrmfb frame buffer device

Quoted 6 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

I'd appreciate if you could give s390x a test.. I never touched s390x and it will probably take some extra effort to get into it.

Adding Laszlo Ersek, who debugged and tested this the last time.

Laszlo Ersek do ypu have some tips for Dmitry? Xorg seems to be working on his end with the drm_drv_set_unique(... "pci:...") call removed.

Original patch can be found at:
https://lore.kernel.org/dri-devel/[email protected]/T/#mbc1a1bedc91d1855007188a725c5c75bbc771cf0

HTH
Emil

#14 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Hi Emil,

Quoted 39 lines · Emil Velikov
Quoted 31 lines · Dmitry Osipenko
Quoted 8 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

Quoted 8 lines

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

I booted kernel with this patchset applied and everything is okay, Xorg works.

[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not
called
virtio-mmio LNRO0005:01: [drm] fb0: virtio_gpudrmfb frame buffer device

Quoted 6 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

I'd appreciate if you could give s390x a test.. I never touched s390x and it will probably take some extra effort to get into it.

Quoted 8 lines

Adding Laszlo Ersek, who debugged and tested this the last time.

Laszlo Ersek do ypu have some tips for Dmitry? Xorg seems to be working on his end with the drm_drv_set_unique(... "pci:...") call removed.

Original patch can be found at:
https://lore.kernel.org/dri-devel/[email protected]/T/#mbc1a1bedc91d1855007188a725c5c75bbc771cf0

thanks for recalling this, but... I've moved to different projects, and I'm already scraping the bottom of the barrel for every chunk of time I can find :(

Laszlo

#15 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

In reply to Dmitry Osipenko
Quoted 16 lines · Dmitry Osipenko
Quoted 14 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

Quoted 7 lines
Quoted 2 lines

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

just a drive-by note, IME on aarch64 kernels, at least with crosvm, virtgpu is also a pci device.. the non-pci things in the guest kernel use dt, but devices on discoverable busses like pci don't need dt nodes (which is true also in the non-vm case)

BR,
-R

Quoted 17 lines

I booted kernel with this patchset applied and everything is okay, Xorg works.

[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not
called
virtio-mmio LNRO0005:01: [drm] fb0: virtio_gpudrmfb frame buffer device

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

I'd appreciate if you could give s390x a test.. I never touched s390x and it will probably take some extra effort to get into it.

--
Best regards,
Dmitry

author maintainer
#16 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 26 lines · Rob Clark
Quoted 16 lines · Dmitry Osipenko
Quoted 14 lines · Gerd Hoffmann

Hi,

Quoted 5 lines
Quoted 1 line

Also note that pci is not the only virtio transport we have.

The VirtIO indeed has other transports, but only PCI is really supported in case of the VirtIO-GPU in kernel and in Qemu/crosvm, AFAICT. Hence only the PCI transport was tested.

qemu -M microvm \
-global virtio-mmio.force-legacy=false \
-device virtio-gpu-device

Gives you a functional virtio-gpu device on virtio-mmio.

Quoted 7 lines
Quoted 2 lines

aarch64 virt machines support both pci and mmio too. s390x has virtio-gpu-ccw ...

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

Quoted 4 lines

just a drive-by note, IME on aarch64 kernels, at least with crosvm, virtgpu is also a pci device.. the non-pci things in the guest kernel use dt, but devices on discoverable busses like pci don't need dt nodes (which is true also in the non-vm case)

Sure, I was only looking how to test MMIO GPU on aarch64. Since I haven't a found a way to test MMIO back then, I concluded that the MMIO case wasn't really well supported.

Signature · 2 lines
Best regards,
Dmitry
maintainer
#17 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

In reply to Dmitry Osipenko

Hi,

Quoted 4 lines

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

aarch64 uses acpi by default (can be disabled via 'qemu -no-acpi'). Not fully sure about arm(v7).

Even with DT it should work because DT only describes the virtio-mmio 'slots', not the actual virtio devices.

Quoted 2 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

It's named 'virtio-gpu-gl-device'

take care,
Gerd

author maintainer
#18 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 10 lines · Gerd Hoffmann

Hi,

Quoted 4 lines

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

aarch64 uses acpi by default (can be disabled via 'qemu -no-acpi'). Not fully sure about arm(v7).

Quoted 7 lines

Even with DT it should work because DT only describes the virtio-mmio 'slots', not the actual virtio devices.

Quoted 2 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

It's named 'virtio-gpu-gl-device'

Ah, thanks again! Just quickly tested virtio-gpu-gl-device and everything works too for MMIO GPU on microvm, including virgl and Xorg (glamor).

[drm] features: +virgl +edid -resource_blob -host_visible
[drm] features: -context_init
[drm] number of scanouts: 1
[drm] number of cap sets: 2
[drm] cap set 0: id 1, max-version 1, max-size 308
[drm] cap set 1: id 2, max-version 2, max-size 696
[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not called

Signature · 2 lines
Best regards,
Dmitry
maintainer
#19 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 23 lines · Dmitry Osipenko
Quoted 10 lines · Gerd Hoffmann

Hi,

Quoted 4 lines

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

aarch64 uses acpi by default (can be disabled via 'qemu -no-acpi'). Not fully sure about arm(v7).

Quoted 7 lines

Even with DT it should work because DT only describes the virtio-mmio 'slots', not the actual virtio devices.

Quoted 2 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

It's named 'virtio-gpu-gl-device'

Ah, thanks again! Just quickly tested virtio-gpu-gl-device and everything works too for MMIO GPU on microvm, including virgl and Xorg (glamor).

Quoted 8 lines

[drm] features: +virgl +edid -resource_blob -host_visible
[drm] features: -context_init
[drm] number of scanouts: 1
[drm] number of cap sets: 2
[drm] cap set 0: id 1, max-version 1, max-size 308
[drm] cap set 1: id 2, max-version 2, max-size 696
[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not called

Cool. Havn't found the time to try s390x, so I'm taking that as good enough test that we don't have any unnoticed dependencies on pci.

Queued up. I'll go over a few more pending patches, and assuming no issues show up in testing this should land in drm-misc-next in a few hours.

take care,
Gerd

author maintainer
#20 lore raw

Re: [PATCH v7 7/9] drm/virtio: Improve DMA API usage for shmem BOs

Quoted 33 lines · Gerd Hoffmann
Quoted 23 lines · Dmitry Osipenko
Quoted 10 lines · Gerd Hoffmann

Hi,

Quoted 4 lines

Gerd, thank you very much! It's was indeed unclear to me how to test the MMIO GPU, but yours variant with microvm works! I was looking for trying aarch64 in the past, but it also was unclear how to do it since there is no DT support for the VirtIO-GPU, AFAICS.

aarch64 uses acpi by default (can be disabled via 'qemu -no-acpi'). Not fully sure about arm(v7).

Quoted 7 lines

Even with DT it should work because DT only describes the virtio-mmio 'slots', not the actual virtio devices.

Quoted 2 lines

There is no virgl support because it's a virtio-gpu-device and not virtio-gpu-device-gl that is PCI-only in Qemu. Hence everything seems good.

It's named 'virtio-gpu-gl-device'

Ah, thanks again! Just quickly tested virtio-gpu-gl-device and everything works too for MMIO GPU on microvm, including virgl and Xorg (glamor).

Quoted 8 lines

[drm] features: +virgl +edid -resource_blob -host_visible
[drm] features: -context_init
[drm] number of scanouts: 1
[drm] number of cap sets: 2
[drm] cap set 0: id 1, max-version 1, max-size 308
[drm] cap set 1: id 2, max-version 2, max-size 696
[drm] Initialized virtio_gpu 0.1.0 0 for LNRO0005:01 on minor 0 virtio-mmio LNRO0005:01: [drm] drm_plane_enable_fb_damage_clips() not called

Quoted 6 lines

Cool. Havn't found the time to try s390x, so I'm taking that as good enough test that we don't have any unnoticed dependencies on pci.

Queued up. I'll go over a few more pending patches, and assuming no issues show up in testing this should land in drm-misc-next in a few hours.

Great, thank you.

Signature · 2 lines
Best regards,
Dmitry
author maintainer
#21 lore raw

PATCH v7 8/9 drm/virtio: Use dev_is_pci()

In reply to Dmitry Osipenko

Use common dev_is_pci() helper to replace the strcmp("pci") used by driver.

---
 drivers/gpu/drm/virtio/virtgpu_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +1 -1
index 0141b7df97ec..0035affc3e59 100644
@@ -87,7 +87,7 @@ static int virtio_gpu_probe(struct virtio_device *vdev)
 		return PTR_ERR(dev);
 	vdev->priv = dev;
 
-	if (!strcmp(vdev->dev.parent->bus->name, "pci")) {
+	if (dev_is_pci(vdev->dev.parent)) {
 		ret = virtio_gpu_pci_quirk(dev);
 		if (ret)
 			goto err_free;
Signature · 1 line
2.36.1
author maintainer
#22 lore raw

PATCH v7 9/9 drm/virtio: Return proper error codes instead of -1

In reply to Dmitry Osipenko

Don't return -1 in error cases, return proper error code. The returned error codes propagate to error messages and to userspace and it's always good to have a meaningful error number for debugging purposes.

---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_vq.c +3 -3
index ee84256946ab..9ff8660b50ad 100644
@@ -322,7 +322,7 @@ static int virtio_gpu_queue_ctrl_sgs(struct virtio_gpu_device *vgdev,
 		if (fence && vbuf->objs)
 			virtio_gpu_array_unlock_resv(vbuf->objs);
 		free_vbuf(vgdev, vbuf);
-		return -1;
+		return -ENODEV;
 	}
 
 	if (vgdev->has_indirect)
@@ -386,7 +386,7 @@ static int virtio_gpu_queue_fenced_ctrl_buffer(struct virtio_gpu_device *vgdev,
 			if (!sgt) {
 				if (fence && vbuf->objs)
 					virtio_gpu_array_unlock_resv(vbuf->objs);
-				return -1;
+				return -ENOMEM;
 			}
 
 			elemcnt += sg_ents;
@@ -720,7 +720,7 @@ static int virtio_get_edid_block(void *data, u8 *buf,
 	size_t start = block * EDID_LENGTH;
 
 	if (start + len > le32_to_cpu(resp->size))
-		return -1;
+		return -EINVAL;
 	memcpy(buf, resp->edid + start, len);
 	return 0;
 }
Signature · 1 line
2.36.1