Skip to contentSkip to conversation

drm/virtio: introduce F_BLOB_ALIGNMENT support

PATCH Applied Sergio Lopez 4 participants 12 messages Apr 29 – Jul 15

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover drm/virtio: introduce F_BLOB_ALIGNMENT support

There's an increasing number of machines supporting multiple page sizes and on these machines the host and a guest can be running, each one, with a different page size.

For what pertains to virtio-gpu, this is not a problem if the page size of the guest happens to be bigger or equal than the host, but will potentially lead to failures in memory allocations and/or mappings otherwise.

To deal with this, the virtio-spec was extended to introduce with the VIRTIO_GPU_F_BLOB_ALIGNMENT feature [1]. If this feature is negotiated, we must use the "blob_alignment" field in the config to ensure every CREATE_BLOB and MAP_BLOB is properly aligned before sending it to the device.

We also introduce the VIRTGPU_PARAM_BLOB_ALIGNMENT parameter to allow userspace to query the alignment restrictions for blobs.

This supersedes "drm/virtio: introduce the HOST_PAGE_SIZE feature" [2].

[1] https://github.com/oasis-tcs/virtio-spec/commit/f9abfd55cb663837dda1153b826216dcf4d25b84
[2] https://lkml.org/lkml/2024/7/23/438

Changes in v2:
- Rebased.
- Moved blob size alignment validation to verify_blob(), rejecting misaligned sizes early in the ioctl path instead of checking in virtio_gpu_cmd_resource_create_blob() and virtio_gpu_cmd_map() (Dmitry Osipenko).

Sergio Lopez (3):
drm/virtio: support VIRTIO_GPU_F_BLOB_ALIGNMENT drm/virtio: honor blob_alignment requirements drm/virtio: add VIRTGPU_PARAM_BLOB_ALIGNMENT to params

 drivers/gpu/drm/virtio/virtgpu_drv.c   |  1 +
 drivers/gpu/drm/virtio/virtgpu_drv.h   |  2 ++
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 10 ++++++++++
 drivers/gpu/drm/virtio/virtgpu_kms.c   | 14 +++++++++++---
 include/uapi/drm/virtgpu_drm.h         |  1 +
 include/uapi/linux/virtio_gpu.h        |  9 +++++++++
 6 files changed, 34 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH v2 1/3 drm/virtio: support VIRTIO_GPU_F_BLOB_ALIGNMENT

Support VIRTIO_GPU_F_BLOB_ALIGNMENT, a feature that indicates the device provides a valid blob_alignment field in its configuration, and that both RESOURCE_CREATE_BLOB and RESOURCE_MAP_BLOB requests must be aligned to that value.

---
 drivers/gpu/drm/virtio/virtgpu_drv.c |  1 +
 drivers/gpu/drm/virtio/virtgpu_drv.h |  2 ++
 drivers/gpu/drm/virtio/virtgpu_kms.c | 14 +++++++++++---
 include/uapi/linux/virtio_gpu.h      |  9 +++++++++
 4 files changed, 23 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +1 -0
index a5ce96fb8a1d..812ee3f5e4aa 100644
@@ -163,6 +163,7 @@ static unsigned int features[] = {
 	VIRTIO_GPU_F_RESOURCE_UUID,
 	VIRTIO_GPU_F_RESOURCE_BLOB,
 	VIRTIO_GPU_F_CONTEXT_INIT,
+	VIRTIO_GPU_F_BLOB_ALIGNMENT,
 };
 static struct virtio_driver virtio_gpu_driver = {
 	.feature_table = features,
drivers/gpu/drm/virtio/virtgpu_drv.h +2 -0
index f17660a71a3e..d1fa386a5a99 100644
@@ -257,6 +257,7 @@ struct virtio_gpu_device {
 	bool has_resource_blob;
 	bool has_host_visible;
 	bool has_context_init;
+	bool has_blob_alignment;
 	struct virtio_shm_region host_visible_region;
 	struct drm_mm host_visible_mm;
 
@@ -270,6 +271,7 @@ struct virtio_gpu_device {
 	uint32_t num_capsets;
 	uint64_t capset_id_mask;
 	struct list_head cap_cache;
+	uint32_t blob_alignment;
 
 	/* protects uuid state when exporting */
 	spinlock_t resource_export_lock;
drivers/gpu/drm/virtio/virtgpu_kms.c +11 -3
index 80ba69b4860b..cfde9f573df6 100644
@@ -124,7 +124,7 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 	struct virtio_gpu_device *vgdev;
 	/* this will expand later */
 	struct virtqueue *vqs[2];
-	u32 num_scanouts, num_capsets;
+	u32 num_scanouts, num_capsets, blob_alignment;
 	int ret = 0;
 
 	if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
@@ -198,14 +198,22 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 	if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_F_CONTEXT_INIT))
 		vgdev->has_context_init = true;
 
+	if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_F_BLOB_ALIGNMENT)) {
+		vgdev->has_blob_alignment = true;
+		virtio_cread_le(vgdev->vdev, struct virtio_gpu_config,
+				blob_alignment, &blob_alignment);
+		vgdev->blob_alignment = blob_alignment;
+	}
+
 	DRM_INFO("features: %cvirgl %cedid %cresource_blob %chost_visible",
 		 vgdev->has_virgl_3d    ? '+' : '-',
 		 vgdev->has_edid        ? '+' : '-',
 		 vgdev->has_resource_blob ? '+' : '-',
 		 vgdev->has_host_visible ? '+' : '-');
 
-	DRM_INFO("features: %ccontext_init\n",
-		 vgdev->has_context_init ? '+' : '-');
+	DRM_INFO("features: %ccontext_init %cblob_alignment\n",
+		 vgdev->has_context_init ? '+' : '-',
+		 vgdev->has_blob_alignment ? '+' : '-');
 
 	ret = virtio_find_vqs(vgdev->vdev, 2, vqs, vqs_info, NULL);
 	if (ret) {
include/uapi/linux/virtio_gpu.h +9 -0
index be109777d10d..4f530d90058c 100644
@@ -64,6 +64,14 @@
  * context_init and multiple timelines
  */
 #define VIRTIO_GPU_F_CONTEXT_INIT        4
+/*
+ * The device provides a valid blob_alignment
+ * field in its configuration and both
+ * VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB and
+ * VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB requests
+ * must be aligned to that value.
+ */
+#define VIRTIO_GPU_F_BLOB_ALIGNMENT      5
 
 enum virtio_gpu_ctrl_type {
 	VIRTIO_GPU_UNDEFINED = 0,
@@ -365,6 +373,7 @@ struct virtio_gpu_config {
 	__le32 events_clear;
 	__le32 num_scanouts;
 	__le32 num_capsets;
+	__le32 blob_alignment;
 };
 
 /* simple formats for fbcon/X use */
Signature · 1 line
2.53.0
#2 lore raw

PATCH v2 2/3 drm/virtio: honor blob_alignment requirements

In reply to Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
Signature · 1 line
2.53.0
#3 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Quoted 23 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -1
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
--

2.53.0

#4 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 5 lines · Alyssa Ross
Quoted 4 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

Quoted 7 lines
Quoted 1 line

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Move fast and beak things, then it takes a wile to fix them) Do you understand who uses it maybe? Maybe if there's a PR for both sides it will be handled faster.

Quoted 23 lines
Quoted 23 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -1
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
--

2.53.0

#5 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 12 lines · Michael S. Tsirkin
Quoted 11 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

Quoted 5 lines
Quoted 1 line

https://issuetracker.google.com/issues/529852979

Move fast and beak things, then it takes a wile to fix them) Do you understand who uses it maybe? Maybe if there's a PR for both sides it will be handled faster.

I have no idea where the device side is. Doesn't seem to be in the Chromium kernel tree, nor are there any patches for it in Chromium Gerrit. Maybe Gurchetan can shed some light?

maintainer
#6 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

In reply to Alyssa Ross
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Quoted 20 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
Quoted 3 lines
Quoted 3 lines

--
2.53.0

Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.

Signature · 2 lines
Best regards,
Dmitry
#7 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Quoted 20 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

--
2.53.0

Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.

I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?

maintainer
#8 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 44 lines · Alyssa Ross
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Quoted 20 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

--
2.53.0

Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.

Quoted 4 lines

I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?

The GUEST_HANDLE should be used by Android [1]. The FENCE_PASSING shouldn't be used in production, though don't know for sure.

Crosvm should bump its experimental defines by +10 to prevent clash with upstream in future.

[1]
https://github.com/google/gfxstream/blob/main/host/vulkan/vk_decoder_global_state.cpp#L107

Signature · 2 lines
Best regards,
Dmitry
#9 lore raw

Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements

Quoted 50 lines · Dmitry Osipenko
Quoted 44 lines · Alyssa Ross
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez

If VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.

FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.

https://issuetracker.google.com/issues/529852979

Quoted 20 lines
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index c33c057365f8..d0c4edf1eaf4 100644
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+
+	if (vgdev->has_blob_alignment &&
+	    !IS_ALIGNED(params->size, vgdev->blob_alignment))
+		return -EINVAL;
+
 	return 0;
 }
 
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

--
2.53.0

Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.

Quoted 4 lines

I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?

Quoted 8 lines

The GUEST_HANDLE should be used by Android [1]. The FENCE_PASSING shouldn't be used in production, though don't know for sure.

Crosvm should bump its experimental defines by +10 to prevent clash with upstream in future.

[1]
https://github.com/google/gfxstream/blob/main/host/vulkan/vk_decoder_global_state.cpp#L107

Thanks for the extra info. :) They've agreed to go ahead and remove both of them, so the problem is solved.

#10 lore raw

PATCH v2 3/3 drm/virtio: add VIRTGPU_PARAM_BLOB_ALIGNMENT to params

In reply to Sergio Lopez

Add VIRTGPU_PARAM_BLOB_ALIGNMENT as a param that can be read with VIRTGPU_GETPARAM by userspace applications running in the guest to obtain the host's page size and find out the right alignment to be used in shared memory allocations.

---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++
 include/uapi/drm/virtgpu_drm.h         | 1 +
 2 files changed, 6 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
index d0c4edf1eaf4..146ff25396c2 100644
@@ -117,6 +117,11 @@ static int virtio_gpu_getparam_ioctl(struct drm_device *dev, void *data,
 	case VIRTGPU_PARAM_EXPLICIT_DEBUG_NAME:
 		value = vgdev->has_context_init ? 1 : 0;
 		break;
+	case VIRTGPU_PARAM_BLOB_ALIGNMENT:
+		if (!vgdev->has_blob_alignment)
+			return -ENOENT;
+		value = vgdev->blob_alignment;
+		break;
 	default:
 		return -EINVAL;
 	}
include/uapi/drm/virtgpu_drm.h +1 -0
index 9debb320c34b..7d4e4884d942 100644
@@ -98,6 +98,7 @@ struct drm_virtgpu_execbuffer {
 #define VIRTGPU_PARAM_CONTEXT_INIT 6 /* DRM_VIRTGPU_CONTEXT_INIT */
 #define VIRTGPU_PARAM_SUPPORTED_CAPSET_IDs 7 /* Bitmask of supported capability set ids */
 #define VIRTGPU_PARAM_EXPLICIT_DEBUG_NAME 8 /* Ability to set debug name from userspace */
+#define VIRTGPU_PARAM_BLOB_ALIGNMENT 9 /* Device alignment requirements for blobs */
 
 struct drm_virtgpu_getparam {
 	__u64 param;
Signature · 1 line
2.53.0
maintainer Applied
#11 lore raw
In reply to Sergio Lopez
Quoted 35 lines · Sergio Lopez

There's an increasing number of machines supporting multiple page sizes and on these machines the host and a guest can be running, each one, with a different page size.

For what pertains to virtio-gpu, this is not a problem if the page size of the guest happens to be bigger or equal than the host, but will potentially lead to failures in memory allocations and/or mappings otherwise.

To deal with this, the virtio-spec was extended to introduce with the VIRTIO_GPU_F_BLOB_ALIGNMENT feature [1]. If this feature is negotiated, we must use the "blob_alignment" field in the config to ensure every CREATE_BLOB and MAP_BLOB is properly aligned before sending it to the device.

We also introduce the VIRTGPU_PARAM_BLOB_ALIGNMENT parameter to allow userspace to query the alignment restrictions for blobs.

This supersedes "drm/virtio: introduce the HOST_PAGE_SIZE feature" [2].

[1] https://github.com/oasis-tcs/virtio-spec/commit/f9abfd55cb663837dda1153b826216dcf4d25b84
[2] https://lkml.org/lkml/2024/7/23/438

Changes in v2:
- Rebased.
- Moved blob size alignment validation to verify_blob(), rejecting misaligned sizes early in the ioctl path instead of checking in virtio_gpu_cmd_resource_create_blob() and virtio_gpu_cmd_map() (Dmitry Osipenko).

Sergio Lopez (3):
drm/virtio: support VIRTIO_GPU_F_BLOB_ALIGNMENT drm/virtio: honor blob_alignment requirements drm/virtio: add VIRTGPU_PARAM_BLOB_ALIGNMENT to params

Quoted 8 lines
 drivers/gpu/drm/virtio/virtgpu_drv.c   |  1 +
 drivers/gpu/drm/virtio/virtgpu_drv.h   |  2 ++
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 10 ++++++++++
 drivers/gpu/drm/virtio/virtgpu_kms.c   | 14 +++++++++++---
 include/uapi/drm/virtgpu_drm.h         |  1 +
 include/uapi/linux/virtio_gpu.h        |  9 +++++++++
 6 files changed, 34 insertions(+), 3 deletions(-)

Applied to misc-next, thanks

Signature · 2 lines
Best regards,
Dmitry