Virtio-GPU S4 support
Subsystems (3)
v5 cover Virtio-GPU S4 support
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.
Second patch implements a mechanism for restoring virtio_gpu_object instances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.
Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)
Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)
Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation
drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 12 +++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 51 ++++++++++++++++++--- drivers/gpu/drm/virtio/virtgpu_object.c | 61 +++++++++++++++++++++++++ 4 files changed, 177 insertions(+), 7 deletions(-)
Signature · 1 line
2.34.1
PATCH v5 1/3 drm/virtio: Freeze and restore hooks to support suspend and resume
From: Dongwon Kim <[email protected]>
virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume
v2: 10ms sleep was added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 1 + drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++--- 3 files changed, 77 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +59 -1
@@ -163,6 +163,60 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_BLOB , VIRTIO_GPU_F_CONTEXT_INIT, }; + +#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error); +return error; + } + +flush_work (&vgdev->obj_free_work); +flush_work (&vgdev->ctrlq.dequeue_work); +flush_work (&vgdev->cursorq.dequeue_work); +flush_work (&vgdev->config_changed_work); + +wait_event (vgdev->ctrlq.ack_queue, + vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max); + +wait_event (vgdev->cursorq.ack_queue, + vgdev->cursorq.vq->num_free == vgdev->cursorq.vq->num_max); + + vdev->config->del_vqs (vdev); + +return 0 ; +} + +static int virtgpu_restore (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =virtio_gpu_find_vqs (vgdev); +if (error) { +DRM_ERROR ("failed to find virt queues\n" ); +return error; + } + +virtio_device_ready (vdev); + + error =drm_mode_config_helper_resume (dev); +if (error) { +DRM_ERROR ("resume error %d\n" , error); +return error; + } + +return 0 ; +} +#endif +static struct virtio_driver virtio_gpu_driver = { .feature_table = features, .feature_table_size = ARRAY_SIZE(features),
@@ -171,7 +225,11 @@ static struct virtio_driver virtio_gpu_driver = { .probe = virtio_gpu_probe, .remove = virtio_gpu_remove, .shutdown = virtio_gpu_shutdown, - .config_changed = virtio_gpu_config_changed + .config_changed = virtio_gpu_config_changed, +#ifdef CONFIG_PM_SLEEP + .freeze = virtgpu_freeze, + .restore = virtgpu_restore, +#endif };static int __init virtio_gpu_driver_init (void )
drivers/gpu/drm/virtio/virtgpu_drv.h +1 -0
@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);void virtio_gpu_release (struct drm_device *dev);int virtio_gpu_driver_open (struct drm_device *dev,struct drm_file *file);void virtio_gpu_driver_postclose (struct drm_device *dev,struct drm_file *file); +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev);/* virtgpu_gem.c */ int virtio_gpu_gem_object_open(struct drm_gem_object *obj,
drivers/gpu/drm/virtio/virtgpu_kms.c +17 -6
@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev, vgdev->num_capsets = num_capsets; } -int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev) {struct virtqueue_info vqs_info[] = { {"control" , virtio_gpu_ctrl_ack }, {"cursor" , virtio_gpu_cursor_ack }, }; -struct virtio_gpu_device *vgdev; -/* this will expand later */ struct virtqueue *vqs[2 ]; +int ret; + + ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); +if (ret) +return ret; + + vgdev->ctrlq.vq = vqs[0 ]; + vgdev->cursorq.vq = vqs[1 ]; + +return 0 ; +} + +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +{ +struct virtio_gpu_device *vgdev;u32 num_scanouts, num_capsets;int ret =0 ;
@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)DRM_INFO ("features: %ccontext_init\n" , vgdev->has_context_init ?'+' :'-' ); - ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); + ret =virtio_gpu_find_vqs (vgdev);if (ret) {DRM_ERROR ("failed to find virt queues\n" ); goto err_vqs; } - vgdev->ctrlq.vq = vqs[0 ]; - vgdev->cursorq.vq = vqs[1 ]; ret =virtio_gpu_alloc_vbufs (vgdev);if (ret) {DRM_ERROR ("failed to alloc vbufs\n" );
Signature · 1 line
2.34.1
PATCH v5 2/3 drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]From: Dongwon Kim <[email protected]>
When the host KVM/QEMU resumes from hibernation, it loses all graphics resources previously submitted by the guest OS, as the QEMU process is terminated during the suspend-resume cycle. This leads to invalid resource errors when the guest OS attempts to interact with the host using those resources after resumption.
To resolve this, the virtio-gpu driver now tracks all active virtio_gpu_objects and provides a mechanism to restore them by re-submitting the objects to QEMU when needed (e.g., during resume from hibernation).
v2: - Attach backing is done if bo->attached was set before
v3: - Restoration is no longer triggered via .restore; instead, it is handled
by a PM notifier only during hibernation.
v4: - Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)
- Protect restore list operations with a spinlock (Nirmoy Das)
- Initialize ret with 0 in virtio_gpu_object_restore_all (Nirmoy Das)
- Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.h | 9 ++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 2 + drivers/gpu/drm/virtio/virtgpu_object.c | 61 +++++++++++++++++++++++++ 3 files changed, 72 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.h +9 -0
@@ -98,6 +98,10 @@ struct virtio_gpu_object {int uuid_state;uuid_t uuid; + +/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head list; };#define gem_to_virtio_gpu_obj (gobj) \ container_of((gobj), struct virtio_gpu_object, base.base)
@@ -265,6 +269,8 @@ struct virtio_gpu_device {struct work_struct obj_free_work;spinlock_t obj_free_lock;struct list_head obj_free_list; +spinlock_t obj_restore_lock; +struct list_head obj_restore_list;struct virtio_gpu_drv_capset *capsets;uint32_t num_capsets;
@@ -479,6 +485,9 @@ bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);int virtio_gpu_resource_id_get (struct virtio_gpu_device *vgdev,uint32_t *resid); + +int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev); +/* virtgpu_prime.c */ int virtio_gpu_resource_assign_uuid (struct virtio_gpu_device *vgdev,struct virtio_gpu_object *bo);
drivers/gpu/drm/virtio/virtgpu_kms.c +2 -0
@@ -170,6 +170,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev) virtio_gpu_array_put_free_work);INIT_LIST_HEAD (&vgdev->obj_free_list);spin_lock_init (&vgdev->obj_free_lock); +INIT_LIST_HEAD (&vgdev->obj_restore_list); +spin_lock_init (&vgdev->obj_restore_lock);#ifdef __LITTLE_ENDIANif (virtio_has_feature (vgdev->vdev,VIRTIO_GPU_F_VIRGL ))
drivers/gpu/drm/virtio/virtgpu_object.c +61 -0
80 lines not on this page · raw message
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
drivers/gpu/drm/virtio/virtgpu_drv.h struct virtio_gpu_object { · line 104
Quoted 3 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.h +3
-0+/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head list;
These are bit too generic names for struct members that are supposed to be used for hibernation-restore only.
what about this variant:
struct virtio_gpu_object {
...
struct virtio_gpu_object_params params;
struct list_head restore_node;
};drivers/gpu/drm/virtio/virtgpu_object.c static void virtio_gpu_resource_id_put(struct virtio_gpu_device *vgdev, uint32_t · line 76
Quoted 13 lines
drivers/gpu/drm/virtio/virtgpu_object.c +13
-0+static void virtio_gpu_object_del_restore_list (struct virtio_gpu_device *vgdev, +struct virtio_gpu_object *bo) +{ +struct virtio_gpu_object *curr, *tmp; + +list_for_each_entry_safe (curr, tmp, &vgdev->obj_restore_list, list) { +if (bo == curr) { +spin_lock (&vgdev->obj_restore_lock); +list_del (&curr->list); +spin_unlock (&vgdev->obj_restore_lock); +break ; + } + }
1. The whole list_for_each_entry_safe() needs to be protected with a lock. But you don't need this iteration at all. Instead of using cleanup_object(), unconditionally remove object from list in virtio_gpu_free_object(), since we care only about shmem objects.
2. Use mutex instead of spinlock, we don't want problems with mem reclaim situations
static void virtio_gpu_free_object(struct drm_gem_object *obj) {
+ mutex_lock(&vgdev->obj_restore_lock);
+ list_del(&bo->restore_node);
+ mutex_unlock(&vgdev->obj_restore_lock);
if (bo->created) {
virtio_gpu_cmd_unref_resource(vgdev, bo);
virtio_gpu_notify(vgdev);
/* completion handler calls virtio_gpu_cleanup_object() */
return;
}
virtio_gpu_cleanup_object(bo);
}Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 2 lines · Dmitry Osipenko
2. Use mutex instead of spinlock, we don't want problems with mem reclaim situations
On the other hand, the virtio_gpu_object_restore_all() shouldn't race with any obj create/free, hence no need to protect it with a lock and then spinlock can be used to only add/remove objs from the restore_list.
Signature · 2 lines
Best regards, Dmitry
Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to Dmitry OsipenkoQuoted 43 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_drv.h struct virtio_gpu_object { · line 104
Quoted 3 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.h +3
-0+/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head list;These are bit too generic names for struct members that are supposed to be used for hibernation-restore only.
what about this variant:
struct virtio_gpu_object { ... struct virtio_gpu_object_params params; struct list_head restore_node; };drivers/gpu/drm/virtio/virtgpu_object.c static void virtio_gpu_resource_id_put(struct virtio_gpu_device *vgdev, uint32_t · line 76
Quoted 13 lines
drivers/gpu/drm/virtio/virtgpu_object.c +13
-0+static void virtio_gpu_object_del_restore_list (struct virtio_gpu_device *vgdev, +struct virtio_gpu_object *bo) +{ +struct virtio_gpu_object *curr, *tmp; + +list_for_each_entry_safe (curr, tmp, &vgdev->obj_restore_list, list) { +if (bo == curr) { +spin_lock (&vgdev->obj_restore_lock); +list_del (&curr->list); +spin_unlock (&vgdev->obj_restore_lock); +break ; + } + }1. The whole
list_for_each_entry_safe()needs to be protected with a lock. But you don't need this iteration at all. Instead of usingcleanup_object(), unconditionally remove object from list invirtio_gpu_free_object(), since we care only about shmem objects.2. Use mutex instead of spinlock, we don't want problems with mem reclaim situations
static void
virtio_gpu_free_object(struct drm_gem_object *obj){(quoted diff) +3
-0+ mutex_lock(&vgdev->obj_restore_lock); + list_del(&bo->restore_node); + mutex_unlock(&vgdev->obj_restore_lock);
Quoted 9 lines
if (bo->created) {
virtio_gpu_cmd_unref_resource(vgdev, bo);
virtio_gpu_notify(vgdev);
/* completion handler calls virtio_gpu_cleanup_object() */
return;
}
virtio_gpu_cleanup_object(bo);
}+1 I have exactly same comments regarding the locks here.
RE: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Nirmoy,
Quoted 4 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objects
Quoted 16 lines
Quoted 15 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_drv.h struct virtio_gpu_object { · line 104
Quoted 3 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.h +3
-0+/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head list;These are bit too generic names for struct members that are supposed to be used for hibernation-restore only.
what about this variant:
struct virtio_gpu_object { ... struct virtio_gpu_object_params params; struct list_head restore_node; };
Yeah, makes sense.
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
+static void virtio_gpu_object_del_restore_list(struct
virtio_gpu_device*vgdev,
Quoted 16 lines
drivers/gpu/drm/virtio/virtgpu_object.c static void virtio_gpu_resource_id_put(struct virtio_gpu_device *vgdev, uint32_t · line 76
Quoted 11 lines
drivers/gpu/drm/virtio/virtgpu_object.c +11
-0+ struct virtio_gpu_object *bo) { +struct virtio_gpu_object *curr, *tmp; + +list_for_each_entry_safe (curr, tmp, &vgdev->obj_restore_list, list) { +if (bo == curr) { +spin_lock (&vgdev->obj_restore_lock); +list_del (&curr->list); +spin_unlock (&vgdev->obj_restore_lock); +break ; + } + }1. The whole
list_for_each_entry_safe()needs to be protected with a lock. But you don't need this iteration at all. Instead of usingcleanup_object(), unconditionally remove object from list invirtio_gpu_free_object(), since we care only about shmem objects.
Right, I haven’t thought about it too much. Thanks for pointing out the error here.
Quoted 2 lines
Quoted 2 lines
2. Use mutex instead of spinlock, we don't want problems with mem reclaim situations
I will take a look at it.
Quoted 6 lines
Quoted 6 lines
static void
virtio_gpu_free_object(struct drm_gem_object *obj){(quoted diff) +3
-0+ mutex_lock(&vgdev->obj_restore_lock); + list_del(&bo->restore_node); + mutex_unlock(&vgdev->obj_restore_lock);
Quoted 10 lines
Quoted 9 lines
if (bo->created) { virtio_gpu_cmd_unref_resource(vgdev, bo); virtio_gpu_notify(vgdev); /* completion handler calls virtio_gpu_cleanup_object() */ return; } virtio_gpu_cleanup_object(bo); }+1 I have exactly same comments regarding the locks here.
Thanks!
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]Quoted 15 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +15
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) +{ +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 10 lines
drivers/gpu/drm/virtio/virtgpu_object.c +10
-0+ ents, nents); + }else if (bo->params.virgl) { +virtio_gpu_cmd_resource_create_3d (vgdev, bo, &bo->params, +NULL ,NULL ); + +if (bo->attached) { + bo->attached = false; +virtio_gpu_object_attach (vgdev, bo, ents, nents); + } + }else {
No need to restore blob and 3d resources that we don't support restore of and that won't be in the obj_restore_list since only shmem objs are added to the list.
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +14
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) { +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c +3
-0+ ents, nents); + }else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0+ NULL, NULL); + +if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_object.c +2
-0+ } + }else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.
We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of backing pages for it (e.g. VRAM)?
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Thanks!
Quoted 3 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 44 lines · Dongwon
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +14
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) { +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c +3
-0+ ents, nents); + }else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0+ NULL, NULL); + +if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_object.c +2
-0+ } + }else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of backing pages for it (e.g. VRAM)?
Quoted 4 lines
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Thanks!
Good point, only host blobs should be excluded. We can't restore VRAM on resume as hostmem is lost on host side after entering S4.
Actually, now I don't see where imported dmabuf guest blob added to the restore_list in virtgpu_dma_buf_init_obj(). Please make sure restoring guest blobs tested properly.
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Dmitry,
Quoted 52 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 39 lines · Dongwon
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +14
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) { +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c +3
-0+ ents, nents); + }else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0+ NULL, NULL); + +if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_object.c +2
-0+ } + }else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.Quoted 4 lines
We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of
backing pages for it (e.g. VRAM)?
Quoted 4 lines
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Quoted 8 lines
Quoted 1 line
Thanks!
Good point, only host blobs should be excluded. We can't restore VRAM on resume as hostmem is lost on host side after entering S4.
Actually, now I don't see where imported dmabuf guest blob added to the
restore_listinvirtgpu_dma_buf_init_obj(). Please make sure restoring guest blobs tested properly.
Right, you got some critical point there.. I forgot about this object with imported dmabuf. We will have to store the case virtio_gpu_objects backed by imported dmabuf as well in case the backing storage is still valid. By the way, so far we are not using imported buffer as a guest framebuffer. All scanouts are originated from virtio_gpu itself, which would be the reason it has worked so far.
Anyhow, no matter whether it's imported or created by virtio-gpu driver, I guess storing only blobs with bo->attached=true would be enough assuming host blob doesn't have any backing pages. What do you think?
(I will also modify the code to store objects backed by imported dmabuf.)
Quoted 4 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 69 lines · Dongwon
Hi Dmitry,
Quoted 52 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 39 lines · Dongwon
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +14
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) { +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c +3
-0+ ents, nents); + }else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0+ NULL, NULL); + +if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_object.c +2
-0+ } + }else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.Quoted 4 lines
We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of
backing pages for it (e.g. VRAM)?
Quoted 4 lines
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Quoted 8 lines
Quoted 1 line
Thanks!
Good point, only host blobs should be excluded. We can't restore VRAM on resume as hostmem is lost on host side after entering S4.
Actually, now I don't see where imported dmabuf guest blob added to the
restore_listinvirtgpu_dma_buf_init_obj(). Please make sure restoring guest blobs tested properly.Right, you got some critical point there.. I forgot about this object with imported dmabuf. We will have to store the case
virtio_gpu_objectsbacked by imported dmabuf as well in case the backing storage is still valid. By the way, so far we are not using imported buffer as a guest framebuffer. All scanouts are originated fromvirtio_gpuitself, which would be the reason it has worked so far.
Quoted 4 lines
Anyhow, no matter whether it's imported or created by virtio-gpu driver, I guess storing only blobs with bo->attached=true would be enough assuming host blob doesn't have any backing pages. What do you think?
(I will also modify the code to store objects backed by imported dmabuf.)
The bo->attached only means whether pages are attached to shmem BO on host at the moment.
The whole state should be restored on resume - all shmem BOs re-created and pages attached to them when bo->attached=true.
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Dmitry,
Quoted 79 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 57 lines · Dongwon
Hi Dmitry,
Quoted 55 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 39 lines · Dongwon
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +14
-0+int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev) { +struct virtio_gpu_object *bo, *tmp; +struct virtio_gpu_mem_entry *ents; +unsigned int nents; +int ret =0 ; + +spin_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, list) { + ret =virtio_gpu_object_shmem_init (vgdev, bo, &ents, &nents); +if (ret) +break ; + +if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c +3
-0+ ents, nents); + }else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
Quoted 5 lines
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0+ NULL, NULL); + +if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
drivers/gpu/drm/virtio/virtgpu_object.c int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, · line 322
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_object.c +2
-0+ } + }else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.Quoted 5 lines
We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of
backing pages for it (e.g. VRAM)?
Quoted 5 lines
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Thanks!
Quoted 8 lines
Quoted 6 lines
Good point, only host blobs should be excluded. We can't restore VRAM on resume as hostmem is lost on host side after entering S4.
Actually, now I don't see where imported dmabuf guest blob added to the
restore_listinvirtgpu_dma_buf_init_obj(). Please make sure restoring guest blobs tested properly.Right, you got some critical point there.. I forgot about this object with
imported dmabuf.
Quoted 4 lines
We will have to store the case
virtio_gpu_objectsbacked by imported dmabuf as well in case the backing storage is still valid. By the way, so far we are not using imported buffer as a guest framebuffer. All scanouts are originated fromvirtio_gpuitself, which would be the reason ithas worked so far.
Quoted 4 lines
Anyhow, no matter whether it's imported or created by virtio-gpu driver, I guess storing only blobs with bo->attached=true would be enough assuming host blob doesn't have any backing pages. What do you
Quoted 8 lines
think?
Quoted 1 line
(I will also modify the code to store objects backed by imported dmabuf.)
The
bo->attachedonly means whether pages are attached to shmem BO on host at the moment.The whole state should be restored on resume - all shmem BOs re-created and pages attached to them when bo->attached=true.
Got it. By the way, regarding exclusion of host blob, I guess you meant only blobs with "!guest_blob && host3d_blob". If that is the case, it is automatically excluded as virtio_gpu_object_create is not even called. Or did you also mean blobs with both flags - host3d_blob and guest_blob set should also be excluded?
Quoted 4 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 87 lines · Dongwon
Hi Dmitry,
Quoted 85 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 57 lines · Dongwon
Hi Dmitry,
Quoted 55 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 39 lines · Dongwon
Hi Dmitry,
Quoted 29 lines
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring
virtio_gpu_objectsQuoted 19 lines · On 10/3/25 08:34, [email protected] wrote:
(quoted diff) +14
-0+int virtio_gpu_object_restore_all(struct virtio_gpu_device *vgdev) { + struct virtio_gpu_object *bo, *tmp; + struct virtio_gpu_mem_entry *ents; + unsigned int nents; + int ret = 0; + + spin_lock(&vgdev->obj_restore_lock); + list_for_each_entry_safe(bo, tmp, &vgdev->obj_restore_list, list) { + ret = virtio_gpu_object_shmem_init(vgdev, bo, &ents, &nents); + if (ret) + break; + + if (bo->params.blob) { + virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo-params,
(quoted diff) +3
-0+ ents, nents); + } else if (bo->params.virgl) { + virtio_gpu_cmd_resource_create_3d(vgdev, bo, &bo-params,
Review
Quoted 5 lines
(quoted diff) +5
-0+ NULL, NULL); + + if (bo->attached) { + bo->attached = false; + virtio_gpu_object_attach(vgdev, bo, ents,nents);
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ } + } else {No need to restore blob and 3d resources that we don't support restore of and that won't be in the
obj_restore_listsince only shmem objs are added to the list.Quoted 5 lines
We are very interested in restoration of blob as well. It is actually the primary type of resource we want to recover because those are used as guest frame buffer we render in QEMU. Can you tell me why it should be excluded? Would it be because of ambiguity of the location of
backing pages for it (e.g. VRAM)?
Quoted 5 lines
And 3D is not our primary interest so I don't have any issue excluding it but it would be great if you can explain the reason for it as well.
Thanks!
Quoted 8 lines
Quoted 6 lines
Good point, only host blobs should be excluded. We can't restore VRAM on resume as hostmem is lost on host side after entering S4.
Actually, now I don't see where imported dmabuf guest blob added to the
restore_listinvirtgpu_dma_buf_init_obj(). Please make sure restoring guest blobs tested properly.Right, you got some critical point there.. I forgot about this object with
imported dmabuf.
Quoted 4 lines
We will have to store the case
virtio_gpu_objectsbacked by imported dmabuf as well in case the backing storage is still valid. By the way, so far we are not using imported buffer as a guest framebuffer. All scanouts are originated fromvirtio_gpuitself, which would be the reason ithas worked so far.
Quoted 4 lines
Anyhow, no matter whether it's imported or created by virtio-gpu driver, I guess storing only blobs with bo->attached=true would be enough assuming host blob doesn't have any backing pages. What do you
think?
Quoted 1 line
(I will also modify the code to store objects backed by imported dmabuf.)
The
bo->attachedonly means whether pages are attached to shmem BO on host at the moment.
Quoted 7 lines
Quoted 2 lines
The whole state should be restored on resume - all shmem BOs re-created and pages attached to them when bo->attached=true.
Got it. By the way, regarding exclusion of host blob, I guess you meant only blobs with "!guest_blob &&
host3d_blob". If that is the case, it is automatically excluded asvirtio_gpu_object_createis not even called. Or did you also mean blobs with both flags -host3d_blobandguest_blobset should also be excluded?
There is no need to re-create resources on host that we can't restore, i.e. "!guest_blob && host3d_blob". Restore all BOs that can be fully restored on resume.
Also, reject hibernation on PM_HIBERNATION_PREPARE with a error msg if virgl is enabled.
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 2 lines · Dmitry Osipenko
Also, reject hibernation on
PM_HIBERNATION_PREPAREwith a error msg if virgl is enabled.
fail hibernation if hostmem is enabled in addition to virgl since both can't be restored
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Dmitry,
Quoted 8 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Tuesday, October 7, 2025 10:22 AM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoringvirtio_gpu_objects
Quoted 6 lines
Quoted 2 lines · Dmitry Osipenko
Also, reject hibernation on
PM_HIBERNATION_PREPAREwith a error msg if virgl is enabled.fail hibernation if hostmem is enabled in addition to virgl since both can't be restored
Yes, I understand it and am working on changes. But what do you think should be done If we really want to recover objects backed by host pages after hibernation. Is it just an impossible task In current design or it would be too complicated?
Quoted 4 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v5 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 14 lines · Dongwon
Hi Dmitry,
Quoted 12 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Tuesday, October 7, 2025 10:22 AM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]
Subject: Re: [PATCH v5 2/3] drm/virtio: Add support for saving and restoringvirtio_gpu_objectsQuoted 2 lines · Dmitry Osipenko
Also, reject hibernation on
PM_HIBERNATION_PREPAREwith a error msg if virgl is enabled.
Quoted 6 lines
Quoted 2 lines
fail hibernation if hostmem is enabled in addition to virgl since both can't be restored
Yes, I understand it and am working on changes. But what do you think should be done If we really want to recover objects backed by host pages after hibernation. Is it just an impossible task In current design or it would be too complicated?
To restore host blobs from hibernation, we'll need to copy them from host to guest. This can be done if all host blobs are mappable.
Depending on a context, some blobs may be not mappable, like in a case of secure memory, or when there is a need to relocate blob from a dedicate memory to make it mappable. Something new might need to be invented for that in virtio-gpu protocol.
Restoring blobs won't be enough for 3d contexts. The whole 3d context needs to be restored along with restored blobs. This is a next level of complexity.
Signature · 2 lines
Best regards, Dmitry
PATCH v5 3/3 drm/virtio: Add PM notifier to restore objects after hibernation
In reply to [email protected]From: Dongwon Kim <[email protected]>
Register a PM notifier in virtio-gpu to handle suspend/hibernate events. On PM_POST_HIBERNATION, restore all GPU objects so that the driver can properly recover after resume.
v2: Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.h | 2 ++ drivers/gpu/drm/virtio/virtgpu_kms.c | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.h +2 -0
@@ -277,6 +277,8 @@ struct virtio_gpu_device {uint64_t capset_id_mask;struct list_head cap_cache; +struct notifier_block pm_nb; +/* protects uuid state when exporting */ spinlock_t resource_export_lock;/* protects map state and host_visible_mm */
drivers/gpu/drm/virtio/virtgpu_kms.c +26 -0
@@ -26,6 +26,8 @@#include <linux/virtio.h> #include <linux/virtio_config.h> #include <linux/virtio_ring.h> +#include <linux/suspend.h> +#include <linux/pm_runtime.h> #include <drm/drm_file.h> #include <drm/drm_managed.h>
@@ -133,6 +135,23 @@ int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev)return 0 ; } +static int virtio_gpu_pm_notifier (struct notifier_block *nb,unsigned long mode, +void *data) +{ + struct virtio_gpu_device *vgdev = container_of(nb, struct virtio_gpu_device, pm_nb); +int ret; + +if (mode ==PM_POST_HIBERNATION ) { + ret =virtio_gpu_object_restore_all (vgdev); +if (ret) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return notifier_from_errno (ret); + } + } + +return NOTIFY_DONE ; +} +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) {struct virtio_gpu_device *vgdev;
@@ -269,6 +288,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)wait_event_timeout (vgdev->resp_wq, !vgdev->display_info_pending,5 *HZ ); } + + vgdev->pm_nb.notifier_call = virtio_gpu_pm_notifier; + ret =register_pm_notifier (&vgdev->pm_nb); +if (ret) + goto err_scanouts; +return 0 ; err_scanouts:
@@ -300,6 +325,7 @@ void virtio_gpu_deinit(struct drm_device *dev)flush_work (&vgdev->config_changed_work);virtio_reset_device (vgdev->vdev); vgdev->vdev->config->del_vqs (vgdev->vdev); +unregister_pm_notifier (&vgdev->pm_nb); }void virtio_gpu_release (struct drm_device *dev)
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerQuoted 51 lines · On 10/3/25 08:33, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernation
Quoted 6 lines
drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 12 +++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 51 ++++++++++++++++++--- drivers/gpu/drm/virtio/virtgpu_object.c | 61 +++++++++++++++++++++++++ 4 files changed, 177 insertions(+), 7 deletions(-)
Hi,
On v6 please use scripts/get_maintainer.pl and make sure that all virtio-gpu driver maintainers are emailed with the patches.
Signature · 2 lines
Best regards, Dmitry
How was this tested ? Example command+steps would be nice have in this cover letter.
Nirmoy
Quoted 57 lines · On 03.10.25 07:33, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernationdrivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 12 +++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 51 ++++++++++++++++++--- drivers/gpu/drm/virtio/virtgpu_object.c | 61 +++++++++++++++++++++++++ 4 files changed, 177 insertions(+), 7 deletions(-)
Hi Nirmoy,
This was tested with vanilla Ubuntu 24.04 LTS image updated with kernel 6.17 w/ this patch applied as a guest with QEMU launched with "-device virtio-vga (or virtio-gpu-pci)". Anyhow, it is a good idea to specify more details in the cover letter. I will include it in V6.
Thanks!
Quoted 74 lines
Subject: Re: [PATCH v5 0/3] Virtio-GPU S4 support
How was this tested ? Example command+steps would be nice have in this cover letter.
Nirmoy
Quoted 11 lines · On 03.10.25 07:33, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances.
Quoted 1 line
This is necessary because the host (QEMU) deletes all associated resources
during
Quoted 3 lines
the virtio-gpu reset, which occurs as part of hiberation/resume process.
Third patch adds pm-notifier to the driver that handles resubmission of virtio-
gpu
Quoted 3 lines
objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle
hibernation
Quoted 10 lines
scenarios without resource loss.
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in
S3
Quoted 1 line
case can be avoided by skipping virtio-gpu-reset in QEMU
(
hw/display/virtio-gpu.c).Quoted 1 line
To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be
attached to
Quoted 1 line
PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and
set to 'true'.
Quoted 26 lines
(e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernationdrivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 12 +++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 51 ++++++++++++++++++--- drivers/gpu/drm/virtio/virtgpu_object.c | 61 +++++++++++++++++++++++++ 4 files changed, 177 insertions(+), 7 deletions(-)