drm/virtio: introduce F_BLOB_ALIGNMENT support
Subsystems (4)
v2 cover drm/virtio: introduce F_BLOB_ALIGNMENT support
There's an increasing number of machines supporting multiple page sizes and on these machines the host and a guest can be running, each one, with a different page size.
For what pertains to virtio-gpu, this is not a problem if the page size of the guest happens to be bigger or equal than the host, but will potentially lead to failures in memory allocations and/or mappings otherwise.
To deal with this, the virtio-spec was extended to introduce with the VIRTIO_GPU_F_BLOB_ALIGNMENT feature [1]. If this feature is negotiated, we must use the "blob_alignment" field in the config to ensure every CREATE_BLOB and MAP_BLOB is properly aligned before sending it to the device.
We also introduce the VIRTGPU_PARAM_BLOB_ALIGNMENT parameter to allow userspace to query the alignment restrictions for blobs.
This supersedes "drm/virtio: introduce the HOST_PAGE_SIZE feature" [2].
[1] https://github.com/oasis-tcs/virtio-spec/commit/f9abfd55cb663837dda1153b826216dcf4d25b84
[2] https://lkml.org/lkml/2024/7/23/438
Changes in v2:
- Rebased.
- Moved blob size alignment validation to verify_blob(), rejecting misaligned sizes early in the ioctl path instead of checking in virtio_gpu_cmd_resource_create_blob() and virtio_gpu_cmd_map() (Dmitry Osipenko).
Sergio Lopez (3):
drm/virtio: support VIRTIO_GPU_F_BLOB_ALIGNMENT drm/virtio: honor blob_alignment requirements drm/virtio: add VIRTGPU_PARAM_BLOB_ALIGNMENT to params
drivers/gpu/drm/virtio/virtgpu_drv.c | 1 + drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++ drivers/gpu/drm/virtio/virtgpu_ioctl.c | 10 ++++++++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 14 +++++++++++--- include/uapi/drm/virtgpu_drm.h | 1 + include/uapi/linux/virtio_gpu.h | 9 +++++++++ 6 files changed, 34 insertions(+), 3 deletions(-)
Signature · 1 line
2.53.0
PATCH v2 1/3 drm/virtio: support VIRTIO_GPU_F_BLOB_ALIGNMENT
Support VIRTIO_GPU_F_BLOB_ALIGNMENT, a feature that indicates the device provides a valid blob_alignment field in its configuration, and that both RESOURCE_CREATE_BLOB and RESOURCE_MAP_BLOB requests must be aligned to that value.
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 1 + drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++ drivers/gpu/drm/virtio/virtgpu_kms.c | 14 +++++++++++--- include/uapi/linux/virtio_gpu.h | 9 +++++++++ 4 files changed, 23 insertions(+), 3 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +1 -0
@@ -163,6 +163,7 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_UUID ,VIRTIO_GPU_F_RESOURCE_BLOB ,VIRTIO_GPU_F_CONTEXT_INIT , + VIRTIO_GPU_F_BLOB_ALIGNMENT, };static struct virtio_driver virtio_gpu_driver = { .feature_table = features,
drivers/gpu/drm/virtio/virtgpu_drv.h +2 -0
@@ -257,6 +257,7 @@ struct virtio_gpu_device {bool has_resource_blob;bool has_host_visible;bool has_context_init; +bool has_blob_alignment;struct virtio_shm_region host_visible_region;struct drm_mm host_visible_mm;
@@ -270,6 +271,7 @@ struct virtio_gpu_device {uint32_t num_capsets;uint64_t capset_id_mask;struct list_head cap_cache; +uint32_t blob_alignment;/* protects uuid state when exporting */ spinlock_t resource_export_lock;
drivers/gpu/drm/virtio/virtgpu_kms.c +11 -3
@@ -124,7 +124,7 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)struct virtio_gpu_device *vgdev;/* this will expand later */ struct virtqueue *vqs[2 ]; -u32 num_scanouts, num_capsets; +u32 num_scanouts, num_capsets, blob_alignment;int ret =0 ;if (!virtio_has_feature (vdev,VIRTIO_F_VERSION_1 ))
@@ -198,14 +198,22 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)if (virtio_has_feature (vgdev->vdev,VIRTIO_GPU_F_CONTEXT_INIT )) vgdev->has_context_init = true; +if (virtio_has_feature (vgdev->vdev,VIRTIO_GPU_F_BLOB_ALIGNMENT )) { + vgdev->has_blob_alignment = true; + virtio_cread_le(vgdev->vdev, struct virtio_gpu_config, + blob_alignment, &blob_alignment); + vgdev->blob_alignment = blob_alignment; + } +DRM_INFO ("features: %cvirgl %cedid %cresource_blob %chost_visible" , vgdev->has_virgl_3d ?'+' :'-' , vgdev->has_edid ?'+' :'-' , vgdev->has_resource_blob ?'+' :'-' , vgdev->has_host_visible ?'+' :'-' ); -DRM_INFO ("features: %ccontext_init\n" , - vgdev->has_context_init ?'+' :'-' ); +DRM_INFO ("features: %ccontext_init %cblob_alignment\n" , + vgdev->has_context_init ?'+' :'-' , + vgdev->has_blob_alignment ?'+' :'-' ); ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL );if (ret) {
include/uapi/linux/virtio_gpu.h +9 -0
@@ -64,6 +64,14 @@* context_init and multiple timelines */ #define VIRTIO_GPU_F_CONTEXT_INIT 4 +/* +* The device provides a valid blob_alignment +* field in its configuration and both +* VIRTIO_GPU_CMD_RESOURCE_CREATE_BLOB and +* VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB requests +* must be aligned to that value. +*/ +#define VIRTIO_GPU_F_BLOB_ALIGNMENT 5enum virtio_gpu_ctrl_type {VIRTIO_GPU_UNDEFINED =0 ,
@@ -365,6 +373,7 @@ struct virtio_gpu_config {__le32 events_clear;__le32 num_scanouts;__le32 num_capsets; +__le32 blob_alignment; };/* simple formats for fbcon/X use */
Signature · 1 line
2.53.0
PATCH v2 2/3 drm/virtio: honor blob_alignment requirements
In reply to Sergio LopezIf VIRTIO_GPU_F_BLOB_ALIGNMENT has been negotiated, blob size must be aligned to blob_alignment. Validate this in verify_blob() so that invalid requests are rejected early.
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; }
Signature · 1 line
2.53.0
Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>
FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
https://issuetracker.google.com/issues/529852979
Quoted 23 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-1@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; } --2.53.0
Michael S. Tsirkin
maintainerRe: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 5 lines · Alyssa Ross
Quoted 4 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.
Quoted 7 lines
Quoted 1 line
Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
Move fast and beak things, then it takes a wile to fix them) Do you understand who uses it maybe? Maybe if there's a PR for both sides it will be handled faster.
Quoted 23 lines
Quoted 23 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-1@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; } --2.53.0
Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 12 lines · Michael S. Tsirkin
Quoted 11 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
Quoted 5 lines
Quoted 1 line
Move fast and beak things, then it takes a wile to fix them) Do you understand who uses it maybe? Maybe if there's a PR for both sides it will be handled faster.
I have no idea where the device side is. Doesn't seem to be in the Chromium kernel tree, nor are there any patches for it in Chromium Gerrit. Maybe Gurchetan can shed some light?
Dmitry Osipenko
maintainerRe: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
In reply to Alyssa RossQuoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
https://issuetracker.google.com/issues/529852979
Quoted 20 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-0@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; }
Quoted 3 lines
Quoted 3 lines
--
2.53.0
Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.
Signature · 2 lines
Best regards, Dmitry
Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
https://issuetracker.google.com/issues/529852979
Quoted 20 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-0@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; }
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
--
2.53.0Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.
I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?
Dmitry Osipenko
maintainerRe: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 44 lines · Alyssa Ross
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
https://issuetracker.google.com/issues/529852979
Quoted 20 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-0@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; }Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
--
2.53.0Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.
Quoted 4 lines
I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?
The GUEST_HANDLE should be used by Android [1]. The FENCE_PASSING shouldn't be used in production, though don't know for sure.
Crosvm should bump its experimental defines by +10 to prevent clash with upstream in future.
[1]
https://github.com/google/gfxstream/blob/main/host/vulkan/vk_decoder_global_state.cpp#L107
Signature · 2 lines
Best regards, Dmitry
Re: [PATCH v2 2/3] drm/virtio: honor blob_alignment requirements
Quoted 50 lines · Dmitry Osipenko
Quoted 44 lines · Alyssa Ross
Quoted 34 lines · Dmitry Osipenko
Quoted 33 lines · Alyssa Ross
Quoted 5 lines · Sergio Lopez
If
VIRTIO_GPU_F_BLOB_ALIGNMENThas been negotiated, blob size must be aligned toblob_alignment. Validate this inverify_blob()so that invalid requests are rejected early.Signed-off-by Sergio Lopez <[email protected]>FYI: this change breaks crosvm, which is squatting the 5 and 6 values of VIRTIO_GPU_F_* with different meanings. I've reported it as a crosvm bug, so hopefully it can be taken care of there.
https://issuetracker.google.com/issues/529852979
Quoted 20 lines
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+)drivers/gpu/drm/virtio/virtgpu_ioctl.c +5
-0@@ -489,6 +489,11 @@ static int verify_blob(struct virtio_gpu_device *vgdev, params->size = rc_blob->size; params->blob = true; params->blob_flags = rc_blob->blob_flags; + +if (vgdev->has_blob_alignment && + !IS_ALIGNED (params->size, vgdev->blob_alignment)) +return -EINVAL ; +return 0 ; }Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
--
2.53.0Thanks for the report. Indeed, crosvm will need to fix its experimental caps. CI will likely run into this problem first once it will update guest to 7.2+ kernel.
Quoted 4 lines
I sent
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/8064741 last week but still waiting to hear anything. Do you have any insight into whether it's still used / being pursued?
Quoted 8 lines
The
GUEST_HANDLEshould be used by Android [1]. TheFENCE_PASSINGshouldn't be used in production, though don't know for sure.Crosvm should bump its experimental defines by +10 to prevent clash with upstream in future.
[1]
https://github.com/google/gfxstream/blob/main/host/vulkan/vk_decoder_global_state.cpp#L107
Thanks for the extra info. :) They've agreed to go ahead and remove both of them, so the problem is solved.
PATCH v2 3/3 drm/virtio: add VIRTGPU_PARAM_BLOB_ALIGNMENT to params
In reply to Sergio LopezAdd VIRTGPU_PARAM_BLOB_ALIGNMENT as a param that can be read with VIRTGPU_GETPARAM by userspace applications running in the guest to obtain the host's page size and find out the right alignment to be used in shared memory allocations.
--- drivers/gpu/drm/virtio/virtgpu_ioctl.c | 5 +++++ include/uapi/drm/virtgpu_drm.h | 1 + 2 files changed, 6 insertions(+)
drivers/gpu/drm/virtio/virtgpu_ioctl.c +5 -0
@@ -117,6 +117,11 @@ static int virtio_gpu_getparam_ioctl(struct drm_device *dev, void *data,case VIRTGPU_PARAM_EXPLICIT_DEBUG_NAME : value = vgdev->has_context_init ?1 :0 ;break ; +case VIRTGPU_PARAM_BLOB_ALIGNMENT : +if (!vgdev->has_blob_alignment) +return -ENOENT ; + value = vgdev->blob_alignment; +break ;default :return -EINVAL ; }
include/uapi/drm/virtgpu_drm.h +1 -0
@@ -98,6 +98,7 @@ struct drm_virtgpu_execbuffer {#define VIRTGPU_PARAM_CONTEXT_INIT 6/* DRM_VIRTGPU_CONTEXT_INIT */ #define VIRTGPU_PARAM_SUPPORTED_CAPSET_IDs 7/* Bitmask of supported capability set ids */ #define VIRTGPU_PARAM_EXPLICIT_DEBUG_NAME 8/* Ability to set debug name from userspace */ +#define VIRTGPU_PARAM_BLOB_ALIGNMENT 9/* Device alignment requirements for blobs */ struct drm_virtgpu_getparam {__u64 param;
Signature · 1 line
2.53.0
Quoted 35 lines · Sergio Lopez
There's an increasing number of machines supporting multiple page sizes and on these machines the host and a guest can be running, each one, with a different page size.
For what pertains to virtio-gpu, this is not a problem if the page size of the guest happens to be bigger or equal than the host, but will potentially lead to failures in memory allocations and/or mappings otherwise.
To deal with this, the virtio-spec was extended to introduce with the
VIRTIO_GPU_F_BLOB_ALIGNMENTfeature [1]. If this feature is negotiated, we must use the "blob_alignment" field in the config to ensure everyCREATE_BLOBandMAP_BLOBis properly aligned before sending it to the device.We also introduce the
VIRTGPU_PARAM_BLOB_ALIGNMENTparameter to allow userspace to query the alignment restrictions for blobs.This supersedes "drm/virtio: introduce the
HOST_PAGE_SIZEfeature" [2].[1] https://github.com/oasis-tcs/virtio-spec/commit/f9abfd55cb663837dda1153b826216dcf4d25b84
[2] https://lkml.org/lkml/2024/7/23/438Changes in v2:
- Rebased.
- Moved blob size alignment validation toverify_blob(), rejecting misaligned sizes early in the ioctl path instead of checking invirtio_gpu_cmd_resource_create_blob()andvirtio_gpu_cmd_map()(Dmitry Osipenko).Sergio Lopez (3):
drm/virtio: supportVIRTIO_GPU_F_BLOB_ALIGNMENTdrm/virtio: honorblob_alignmentrequirements drm/virtio: addVIRTGPU_PARAM_BLOB_ALIGNMENTto params
Quoted 8 lines
drivers/gpu/drm/virtio/virtgpu_drv.c | 1 + drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++ drivers/gpu/drm/virtio/virtgpu_ioctl.c | 10 ++++++++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 14 +++++++++++--- include/uapi/drm/virtgpu_drm.h | 1 + include/uapi/linux/virtio_gpu.h | 9 +++++++++ 6 files changed, 34 insertions(+), 3 deletions(-)
Applied to misc-next, thanks
Signature · 2 lines
Best regards, Dmitry