Virtio-GPU S4 support
Subsystems (3)
v6 cover Virtio-GPU S4 support
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.
Second patch implements a mechanism for restoring virtio_gpu_object instances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.
Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)
Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)
Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
v6: Include object backed by imported dmabuf
(Dmitry Osipenko)
Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)
Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)
Use mutex instead of spinlock when updating restore_list (Nirmoy Das)
Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.
Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.
Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation
drivers/gpu/drm/virtio/virtgpu_drv.c | 69 ++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 15 ++++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 53 ++++++++++++++++--- drivers/gpu/drm/virtio/virtgpu_object.c | 59 +++++++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_prime.c | 44 ++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_vram.c | 2 + 6 files changed, 235 insertions(+), 7 deletions(-)
Signature · 1 line
2.34.1
PATCH v6 1/3 drm/virtio: Freeze and restore hooks to support suspend and resume
From: Dongwon Kim <[email protected]>
virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume
v2: 10ms sleep was added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 1 + drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++--- 3 files changed, 77 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +59 -1
@@ -163,6 +163,60 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_BLOB , VIRTIO_GPU_F_CONTEXT_INIT, }; + +#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error); +return error; + } + +flush_work (&vgdev->obj_free_work); +flush_work (&vgdev->ctrlq.dequeue_work); +flush_work (&vgdev->cursorq.dequeue_work); +flush_work (&vgdev->config_changed_work); + +wait_event (vgdev->ctrlq.ack_queue, + vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max); + +wait_event (vgdev->cursorq.ack_queue, + vgdev->cursorq.vq->num_free == vgdev->cursorq.vq->num_max); + + vdev->config->del_vqs (vdev); + +return 0 ; +} + +static int virtgpu_restore (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =virtio_gpu_find_vqs (vgdev); +if (error) { +DRM_ERROR ("failed to find virt queues\n" ); +return error; + } + +virtio_device_ready (vdev); + + error =drm_mode_config_helper_resume (dev); +if (error) { +DRM_ERROR ("resume error %d\n" , error); +return error; + } + +return 0 ; +} +#endif +static struct virtio_driver virtio_gpu_driver = { .feature_table = features, .feature_table_size = ARRAY_SIZE(features),
@@ -171,7 +225,11 @@ static struct virtio_driver virtio_gpu_driver = { .probe = virtio_gpu_probe, .remove = virtio_gpu_remove, .shutdown = virtio_gpu_shutdown, - .config_changed = virtio_gpu_config_changed + .config_changed = virtio_gpu_config_changed, +#ifdef CONFIG_PM_SLEEP + .freeze = virtgpu_freeze, + .restore = virtgpu_restore, +#endif };static int __init virtio_gpu_driver_init (void )
drivers/gpu/drm/virtio/virtgpu_drv.h +1 -0
@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);void virtio_gpu_release (struct drm_device *dev);int virtio_gpu_driver_open (struct drm_device *dev,struct drm_file *file);void virtio_gpu_driver_postclose (struct drm_device *dev,struct drm_file *file); +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev);/* virtgpu_gem.c */ int virtio_gpu_gem_object_open(struct drm_gem_object *obj,
drivers/gpu/drm/virtio/virtgpu_kms.c +17 -6
@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev, vgdev->num_capsets = num_capsets; } -int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev) {struct virtqueue_info vqs_info[] = { {"control" , virtio_gpu_ctrl_ack }, {"cursor" , virtio_gpu_cursor_ack }, }; -struct virtio_gpu_device *vgdev; -/* this will expand later */ struct virtqueue *vqs[2 ]; +int ret; + + ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); +if (ret) +return ret; + + vgdev->ctrlq.vq = vqs[0 ]; + vgdev->cursorq.vq = vqs[1 ]; + +return 0 ; +} + +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +{ +struct virtio_gpu_device *vgdev;u32 num_scanouts, num_capsets;int ret =0 ;
@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)DRM_INFO ("features: %ccontext_init\n" , vgdev->has_context_init ?'+' :'-' ); - ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); + ret =virtio_gpu_find_vqs (vgdev);if (ret) {DRM_ERROR ("failed to find virt queues\n" ); goto err_vqs; } - vgdev->ctrlq.vq = vqs[0 ]; - vgdev->cursorq.vq = vqs[1 ]; ret =virtio_gpu_alloc_vbufs (vgdev);if (ret) {DRM_ERROR ("failed to alloc vbufs\n" );
Signature · 1 line
2.34.1
Re: [PATCH v6 1/3] drm/virtio: Freeze and restore hooks to support suspend and resume
Quoted 44 lines · On 27.10.25 21:53, [email protected] wrote:
From: Dongwon Kim <[email protected]>
virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume
v2: 10ms sleep was added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)Suggested-by Dmitry Osipenko <[email protected]>Tested-by Dmitry Osipenko <[email protected]>Cc Vivek Kasireddy <[email protected]>Signed-off-by Dongwon Kim <[email protected]>--- drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 1 + drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++--- 3 files changed, 77 insertions(+), 7 deletions(-)drivers/gpu/drm/virtio/virtgpu_drv.c +14
-0@@ -163,6 +163,60 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_BLOB , VIRTIO_GPU_F_CONTEXT_INIT, }; + +#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error); +return error; + } +
drivers/gpu/drm/virtio/virtgpu_drv.c static unsigned int features[] = { · line 189
Quoted 10 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +10
-0+flush_work (&vgdev->obj_free_work); +flush_work (&vgdev->ctrlq.dequeue_work); +flush_work (&vgdev->cursorq.dequeue_work); +flush_work (&vgdev->config_changed_work); + +wait_event (vgdev->ctrlq.ack_queue, + vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max); + +wait_event (vgdev->cursorq.ack_queue, + vgdev->cursorq.vq->num_free == vgdev->cursorq.vq->num_max);
This can wait forever, add a timeout here with wait_event_timeout().
Otherwise looks good to me.
Regards,
Nirmoy
Quoted 109 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +35
-1+ + vdev->config->del_vqs (vdev); + +return 0 ; +} + +static int virtgpu_restore (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =virtio_gpu_find_vqs (vgdev); +if (error) { +DRM_ERROR ("failed to find virt queues\n" ); +return error; + } + +virtio_device_ready (vdev); + + error =drm_mode_config_helper_resume (dev); +if (error) { +DRM_ERROR ("resume error %d\n" , error); +return error; + } + +return 0 ; +} +#endif +static struct virtio_driver virtio_gpu_driver = { .feature_table = features, .feature_table_size =ARRAY_SIZE (features),@@ -171,7 +225,11 @@ static struct virtio_driver virtio_gpu_driver = { .probe = virtio_gpu_probe, .remove = virtio_gpu_remove, .shutdown = virtio_gpu_shutdown, - .config_changed = virtio_gpu_config_changed + .config_changed = virtio_gpu_config_changed, +#ifdef CONFIG_PM_SLEEP + .freeze = virtgpu_freeze, + .restore = virtgpu_restore, +#endif };static int __init virtio_gpu_driver_init (void )drivers/gpu/drm/virtio/virtgpu_drv.h +1
-0@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);void virtio_gpu_release (struct drm_device *dev);int virtio_gpu_driver_open (struct drm_device *dev,struct drm_file *file);void virtio_gpu_driver_postclose (struct drm_device *dev,struct drm_file *file); +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev);/* virtgpu_gem.c */ int virtio_gpu_gem_object_open(struct drm_gem_object *obj,drivers/gpu/drm/virtio/virtgpu_kms.c +17
-6@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev, vgdev->num_capsets = num_capsets; } -int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev) {struct virtqueue_info vqs_info[] = { {"control" , virtio_gpu_ctrl_ack }, {"cursor" , virtio_gpu_cursor_ack }, }; -struct virtio_gpu_device *vgdev; -/* this will expand later */ struct virtqueue *vqs[2 ]; +int ret; + + ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); +if (ret) +return ret; + + vgdev->ctrlq.vq = vqs[0 ]; + vgdev->cursorq.vq = vqs[1 ]; + +return 0 ; +} + +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +{ +struct virtio_gpu_device *vgdev;u32 num_scanouts, num_capsets;int ret =0 ;@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)DRM_INFO ("features: %ccontext_init\n" , vgdev->has_context_init ?'+' :'-' ); - ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); + ret =virtio_gpu_find_vqs (vgdev);if (ret) {DRM_ERROR ("failed to find virt queues\n" ); goto err_vqs; } - vgdev->ctrlq.vq = vqs[0 ]; - vgdev->cursorq.vq = vqs[1 ]; ret =virtio_gpu_alloc_vbufs (vgdev);if (ret) {DRM_ERROR ("failed to alloc vbufs\n" );
RE: [PATCH v6 1/3] drm/virtio: Freeze and restore hooks to support suspend and resume
Hi Nirmoy,
Quoted 59 lines
-----Original Message-----
From: Nirmoy Das <[email protected]>
Sent: Wednesday, November 12, 2025 4:28 AM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 1/3] drm/virtio: Freeze and restore hooks to support suspend and resumeQuoted 49 lines · On 27.10.25 21:53, [email protected] wrote:
From: Dongwon Kim <[email protected]>
virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume
v2: 10ms sleep was added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)Suggested-by Dmitry Osipenko <[email protected]>Tested-by Dmitry Osipenko <[email protected]>Cc Vivek Kasireddy <[email protected]>Signed-off-by Dongwon Kim <[email protected]>--- drivers/gpu/drm/virtio/virtgpu_drv.c | 60 +++++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 1 + drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++--- 3 files changed, 77 insertions(+), 7 deletions(-)drivers/gpu/drm/virtio/virtgpu_drv.c +18
-0@@ -163,6 +163,60 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_BLOB , VIRTIO_GPU_F_CONTEXT_INIT, }; + +#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) { +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error); +return error; + } + +flush_work (&vgdev->obj_free_work); +flush_work (&vgdev->ctrlq.dequeue_work); +flush_work (&vgdev->cursorq.dequeue_work); +flush_work (&vgdev->config_changed_work); +
Quoted 8 lines
drivers/gpu/drm/virtio/virtgpu_drv.c static unsigned int features[] = {
Quoted 6 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +5
-0+wait_event (vgdev->ctrlq.ack_queue, + vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max); + +wait_event (vgdev->cursorq.ack_queue, + vgdev->cursorq.vq->num_free == vgdev->cursorq.vq- num_max);This can wait forever, add a timeout here with
wait_event_timeout().
Yeah, makes sense. I will update it.
Quoted 123 lines
Otherwise looks good to me.
Regards,
Nirmoy
Quoted 36 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +29
-0+ + vdev->config->del_vqs (vdev); + +return 0 ; +} + +static int virtgpu_restore (struct virtio_device *vdev) { +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =virtio_gpu_find_vqs (vgdev); +if (error) { +DRM_ERROR ("failed to find virt queues\n" ); +return error; + } + +virtio_device_ready (vdev); + + error =drm_mode_config_helper_resume (dev); +if (error) { +DRM_ERROR ("resume error %d\n" , error); +return error; + } + +return 0 ; +} +#endif +static struct virtio_driver virtio_gpu_driver = { .feature_table = features, .feature_table_size = ARRAY_SIZE(features), @@ -171,7 +225,11 @@static struct
virtio_drivervirtio_gpu_driver= {
.probe =virtio_gpu_probe,
.remove =virtio_gpu_remove,
.shutdown =virtio_gpu_shutdown,drivers/gpu/drm/virtio/virtgpu_drv.c
Quoted 2 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +1
-1- .config_changed = virtio_gpu_config_changed + .config_changed = virtio_gpu_config_changed, #ifdef
CONFIG_PM_SLEEPdrivers/gpu/drm/virtio/virtgpu_kms.c static void virtio_gpu_get_capsets(struct
Quoted 26 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +3
-0+ .freeze = virtgpu_freeze, + .restore = virtgpu_restore, +#endif }; static int __init virtio_gpu_driver_init(void) diff --git
a/drivers/gpu/drm/virtio/virtgpu_drv.hb/drivers/gpu/drm/virtio/virtgpu_drv.h
indexf17660a71a3e..1279f998c8e0100644drivers/gpu/drm/virtio/virtgpu_drv.h +1
-0@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);void virtio_gpu_release (struct drm_device *dev);int virtio_gpu_driver_open (struct drm_device *dev,struct drm_file *file);void virtio_gpu_driver_postclose (struct drm_device *dev,struct drm_file *file); +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev);/* virtgpu_gem.c */ int virtio_gpu_gem_object_open(struct
drm_gem_object*obj, diff
--gita/drivers/gpu/drm/virtio/virtgpu_kms.cb/drivers/gpu/drm/virtio/virtgpu_kms.c
index1c15cbf326b7..cbebe19c3fb3100644drivers/gpu/drm/virtio/virtgpu_kms.c +0
-0@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct
virtio_gpu_device*vgdev,drivers/gpu/drm/virtio/virtgpu_kms.c int virtio_gpu_init(struct virtio_device *vdev,
Quoted 33 lines
drivers/gpu/drm/virtio/virtgpu_kms.c +16
-3vgdev->num_capsets = num_capsets; } -int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev) {struct virtqueue_info vqs_info[] = { {"control" , virtio_gpu_ctrl_ack }, {"cursor" , virtio_gpu_cursor_ack }, }; -struct virtio_gpu_device *vgdev; -/* this will expand later */ struct virtqueue *vqs[2 ]; +int ret; + + ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); +if (ret) +return ret; + + vgdev->ctrlq.vq = vqs[0 ]; + vgdev->cursorq.vq = vqs[1 ]; + +return 0 ; +} + +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device +*dev) { +struct virtio_gpu_device *vgdev;u32 num_scanouts, num_capsets;int ret =0 ;@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev,struct
drm_device*dev)Quoted 14 lines
drivers/gpu/drm/virtio/virtgpu_kms.c +1
-3DRM_INFO ("features: %ccontext_init\n" , vgdev->has_context_init ?'+' :'-' ); - ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); + ret =virtio_gpu_find_vqs (vgdev);if (ret) {DRM_ERROR ("failed to find virt queues\n" ); goto err_vqs; } - vgdev->ctrlq.vq = vqs[0 ]; - vgdev->cursorq.vq = vqs[1 ]; ret =virtio_gpu_alloc_vbufs (vgdev);if (ret) {DRM_ERROR ("failed to alloc vbufs\n" );
PATCH v6 2/3 drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]From: Dongwon Kim <[email protected]>
When the host KVM/QEMU resumes from hibernation, it loses all graphics resources previously submitted by the guest OS, as the QEMU process is terminated during the suspend-resume cycle. This leads to invalid resource errors when the guest OS attempts to interact with the host using those resources after resumption.
To resolve this, virtio-gpu driver tracks all active virtio_gpu_objects and provides a mechanism to restore them by resubmitting the objects to QEMU when needed (e.g., during resume from hibernation).
v2: - Attach backing is done if bo->attached was set before
v3: - Restoration is no longer triggered via .restore; instead, it is
handled by a PM notifier only during hibernation.
v4: - Remove virtio_gpu_object from the restore list before freeing
the object to prevent an use-after-free situation. (Nirmoy Das)
- Protect restore list operations with a spinlock (Nirmoy Das)
- Initialize ret with 0 in virtio_gpu_object_restore_all (Nirmoy Das)
- Move restore list node into virtio_gpu_bo struct to reduce memory usage
(Dmitry Osipenko)
v5: - Include object backed by imported dmabuf
(Dmitry Osipenko)
- Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)
- Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)
- Use mutex instead of spinlock when updating restore_list (Nirmoy Das)
- Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.
--- drivers/gpu/drm/virtio/virtgpu_drv.h | 11 +++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 3 ++ drivers/gpu/drm/virtio/virtgpu_object.c | 59 +++++++++++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_prime.c | 44 ++++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_vram.c | 2 + 5 files changed, 119 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.h +11 -0
@@ -98,6 +98,10 @@ struct virtio_gpu_object {int uuid_state;uuid_t uuid; + +/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head restore_node; };#define gem_to_virtio_gpu_obj (gobj) \ container_of((gobj), struct virtio_gpu_object, base.base)
@@ -265,6 +269,8 @@ struct virtio_gpu_device {struct work_struct obj_free_work;spinlock_t obj_free_lock;struct list_head obj_free_list; +struct mutex obj_restore_lock; +struct list_head obj_restore_list;struct virtio_gpu_drv_capset *capsets;uint32_t num_capsets;
@@ -479,6 +485,9 @@ bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);int virtio_gpu_resource_id_get (struct virtio_gpu_device *vgdev,uint32_t *resid); + +int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev); +/* virtgpu_prime.c */ int virtio_gpu_resource_assign_uuid (struct virtio_gpu_device *vgdev,struct virtio_gpu_object *bo);
@@ -493,6 +502,8 @@ int virtgpu_dma_buf_import_sgt(struct virtio_gpu_mem_entry **ents,unsigned int *nents, struct virtio_gpu_object *bo, struct dma_buf_attachment *attach); +int virtgpu_dma_buf_obj_resubmit (struct virtio_gpu_device *vgdev, +struct virtio_gpu_object *bo);/* virtgpu_debugfs.c */ void virtio_gpu_debugfs_init (struct drm_minor *minor);
drivers/gpu/drm/virtio/virtgpu_kms.c +3 -0
@@ -170,6 +170,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev) virtio_gpu_array_put_free_work);INIT_LIST_HEAD (&vgdev->obj_free_list);spin_lock_init (&vgdev->obj_free_lock); +INIT_LIST_HEAD (&vgdev->obj_restore_list); +mutex_init (&vgdev->obj_restore_lock);#ifdef __LITTLE_ENDIANif (virtio_has_feature (vgdev->vdev,VIRTIO_GPU_F_VIRGL ))
@@ -298,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev)flush_work (&vgdev->config_changed_work);virtio_reset_device (vgdev->vdev); vgdev->vdev->config->del_vqs (vgdev->vdev); +mutex_destroy (&vgdev->obj_restore_lock); }void virtio_gpu_release (struct drm_device *dev)
drivers/gpu/drm/virtio/virtgpu_object.c +59 -0
80 lines not on this page · raw message
drivers/gpu/drm/virtio/virtgpu_prime.c +44 -0
62 lines not on this page · raw message
drivers/gpu/drm/virtio/virtgpu_vram.c +2 -0
@@ -200,6 +200,8 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev, obj = &vram->base.base.base; obj->funcs = &virtio_gpu_vram_funcs; +INIT_LIST_HEAD (&vram->base.restore_node); + params->size =PAGE_ALIGN (params->size);drm_gem_private_object_init (vgdev->ddev, obj, params->size);
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerRe: [PATCH v6 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
drivers/gpu/drm/virtio/virtgpu_prime.c static int virtgpu_dma_buf_init_obj(struct drm_device *dev, · line 270
Quoted 4 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_prime.c +4
-0+/* store the dmabuf imported object with its params to +* the restore list +*/ +memcpy (&bo->params, ¶ms,sizeof (params));
Nit: using memcpy is unnecessary for structs, "bo->params = params" would look cleaner
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v6 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Quoted 8 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_prime.c static int virtgpu_dma_buf_init_obj(struct drm_device *dev, · line 270
Quoted 4 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_prime.c +4
-0+/* store the dmabuf imported object with its params to +* the restore list +*/ +memcpy (&bo->params, ¶ms,sizeof (params));Nit: using memcpy is unnecessary for structs, "
bo->params= params" would look cleaner
Won't hurt to add helper for adding obj to a restore list since same code repeats two times.
virtio_gpu_add_object_to_restore_list(struct virtio_gpu_device *vgdev,
struct virtio_gpu_object *bo)
{
...
}
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v6 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
Hi Dmitry,
Quoted 8 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, November 12, 2025 3:07 AM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 2/3] drm/virtio: Add support for saving and restoringvirtio_gpu_objects
Quoted 9 lines
Quoted 8 lines · Dmitry Osipenko
drivers/gpu/drm/virtio/virtgpu_prime.c static int virtgpu_dma_buf_init_obj(struct drm_device *dev, · line 270
Quoted 4 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_prime.c +4
-0+/* store the dmabuf imported object with its params to +* the restore list +*/ +memcpy (&bo->params, ¶ms,sizeof (params));Nit: using memcpy is unnecessary for structs, "
bo->params= params" would look cleaner
Yeah, you are right. I will update it.
Quoted 8 lines
Won't hurt to add helper for adding obj to a restore list since same code repeats two times.
virtio_gpu_add_object_to_restore_list(struct
virtio_gpu_device*vgdev,struct virtio_gpu_object *bo) {...
}
Ok, I will.
Quoted 4 lines
--
Best regards,
Dmitry
PATCH v6 3/3 drm/virtio: Add PM notifier to restore objects after hibernation
In reply to [email protected]From: Dongwon Kim <[email protected]>
Register a PM notifier in virtio-gpu to handle suspend/hibernate events. On PM_POST_HIBERNATION, restore all GPU objects so that the driver can properly recover after resume.
v2: Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
v3: Objects for virgl usecase can't be recovered after resume so
blocking S4 when virgl is enabled
(Dmitry Osipenko)
v4: Restoring objects in the PM notifier is too late, as virtio-gpu
message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 9 +++++++++ drivers/gpu/drm/virtio/virtgpu_drv.h | 3 +++ drivers/gpu/drm/virtio/virtgpu_kms.c | 27 +++++++++++++++++++++++++++ 3 files changed, 39 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.c +9 -0
@@ -207,6 +207,15 @@ static int virtgpu_restore(struct virtio_device *vdev)virtio_device_ready (vdev); +if (vgdev->hibernation) { + vgdev->hibernation = false; + error =virtio_gpu_object_restore_all (vgdev); +if (error) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return error; + } + } + error =drm_mode_config_helper_resume (dev);if (error) {DRM_ERROR ("resume error %d\n" , error);
drivers/gpu/drm/virtio/virtgpu_drv.h +3 -0
@@ -261,6 +261,7 @@ struct virtio_gpu_device {bool has_resource_blob;bool has_host_visible;bool has_context_init; +bool hibernation;struct virtio_shm_region host_visible_region;struct drm_mm host_visible_mm;
@@ -277,6 +278,8 @@ struct virtio_gpu_device {uint64_t capset_id_mask;struct list_head cap_cache; +struct notifier_block pm_nb; +/* protects uuid state when exporting */ spinlock_t resource_export_lock;/* protects map state and host_visible_mm */
drivers/gpu/drm/virtio/virtgpu_kms.c +27 -0
@@ -26,6 +26,8 @@#include <linux/virtio.h> #include <linux/virtio_config.h> #include <linux/virtio_ring.h> +#include <linux/suspend.h> +#include <linux/pm_runtime.h> #include <drm/drm_file.h> #include <drm/drm_managed.h>
@@ -133,6 +135,24 @@ int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev)return 0 ; } +static int virtio_gpu_pm_notifier (struct notifier_block *nb,unsigned long mode, +void *data) +{ +struct virtio_gpu_device *vgdev =container_of (nb, + struct virtio_gpu_device, + pm_nb); + +if (mode ==PM_HIBERNATION_PREPARE ) { +if (vgdev->has_virgl_3d) { +DRM_ERROR ("S4 not allowed when VIRGL is enabled\n" ); +return notifier_from_errno (-EPERM ); + } + vgdev->hibernation = true; + } + +return NOTIFY_DONE ; +} +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) {struct virtio_gpu_device *vgdev;
@@ -269,6 +289,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)wait_event_timeout (vgdev->resp_wq, !vgdev->display_info_pending,5 *HZ ); } + + vgdev->pm_nb.notifier_call = virtio_gpu_pm_notifier; + ret =register_pm_notifier (&vgdev->pm_nb); +if (ret) + goto err_scanouts; +return 0 ; err_scanouts:
@@ -301,6 +327,7 @@ void virtio_gpu_deinit(struct drm_device *dev)virtio_reset_device (vgdev->vdev); vgdev->vdev->config->del_vqs (vgdev->vdev);mutex_destroy (&vgdev->obj_restore_lock); +unregister_pm_notifier (&vgdev->pm_nb); }void virtio_gpu_release (struct drm_device *dev)
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Quoted 6 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +0
-0@@ -207,6 +207,15 @@ static int virtgpu_restore(struct virtio_device *vdev)
drivers/gpu/drm/virtio/virtgpu_drv.c static int virtgpu_restore(struct virtio_device *vdev) · line 217
Quoted 10 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +8
-0virtio_device_ready (vdev); +if (vgdev->hibernation) { + vgdev->hibernation = false; + error =virtio_gpu_object_restore_all (vgdev); +if (error) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return error; + } + }
What if hibernation was aborted and didn't happen? In that case restoring of objects will fail. Please investigate how to handle aborted hibernation properly.
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Quoted 9 lines · Dmitry Osipenko
Quoted 8 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +0
-0@@ -207,6 +207,15 @@ static int virtgpu_restore(struct virtio_device *vdev)virtio_device_ready (vdev);
Quoted 12 lines
drivers/gpu/drm/virtio/virtgpu_drv.c static int virtgpu_restore(struct virtio_device *vdev) · line 217
Quoted 8 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +8
-0+if (vgdev->hibernation) { + vgdev->hibernation = false; + error =virtio_gpu_object_restore_all (vgdev); +if (error) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return error; + } + }What if hibernation was aborted and didn't happen? In that case restoring of objects will fail. Please investigate how to handle aborted hibernation properly.
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Hi Dmitry,
Quoted 33 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, November 12, 2025 3:12 AM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 11 lines · Dmitry Osipenko
Quoted 10 lines · On 10/27/25 23:53, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +0
-0@@ -207,6 +207,15 @@ static int virtgpu_restore(struct virtio_device *vdev)virtio_device_ready (vdev);Quoted 12 lines
drivers/gpu/drm/virtio/virtgpu_drv.c static int virtgpu_restore(struct virtio_device *vdev) · line 217
Quoted 8 lines
drivers/gpu/drm/virtio/virtgpu_drv.c +8
-0+if (vgdev->hibernation) { + vgdev->hibernation = false; + error =virtio_gpu_object_restore_all (vgdev); +if (error) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return error; + } + }What if hibernation was aborted and didn't happen? In that case restoring of objects will fail. Please investigate how to handle aborted hibernation properly.
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just run. In this case, virtio_gpu_object_restore_all won't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any of virtio-gpu resets here as hibernation failed.
Quoted 4 lines
--
Best regards,
Dmitry
Thanks!
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Quoted 8 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just run. In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any of virtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor from host. Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
1. Destroy stored host resources upon hibernation
2. Extend hibernation core [1] with addition of PM_HIBERNATION_UNPREPARE event that will be invoked when hibernation fails, while PM_POST_HIBERNATION will be invoked only after a successful hibernation
You may start with implementing the first option right away and later implement the second.
[1]
https://elixir.bootlin.com/linux/v6.17.7/source/kernel/power/hibernate.c#L827
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Hi Dmitry,
Quoted 27 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 14, 2025 5:16 AM
To: Kim, Dongwon <[email protected]>; dri- [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 3 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just
run.
Quoted 5 lines
In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any ofvirtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor from host. Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
Quoted 7 lines
1. Destroy stored host resources upon hibernation 2. Extend hibernation core [1] with addition of
PM_HIBERNATION_UNPREPAREevent that will be invoked when hibernation fails, whilePM_POST_HIBERNATIONwill be invoked only after a successful hibernationYou may start with implementing the first option right away and later implement the second.
I am testing the first method - send unref message to QEMU so that connected resources can be all removed from QEMU. But I found out there is cb that clean up the object once this unref is processed and response is received. It means that we can't just remove the resource/bo only in QEMU side. There is a way to do this though. We can simply add one more fuction in virtgpu_vq.c that handles unref but with cb = NULL. Is this what you want to try or do you have any better ideas?
Quoted 8 lines
[1]
https://elixir.bootlin.com/linux/v6.17.7/source/kernel/power/hibernate.c#L
827--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Hi,
Quoted 37 lines · Dongwon
Hi Dmitry,
Quoted 27 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 14, 2025 5:16 AM
To: Kim, Dongwon <[email protected]>; dri- [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 3 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just
run.
Quoted 5 lines
In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any ofvirtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor from host. Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
Quoted 7 lines
1. Destroy stored host resources upon hibernation 2. Extend hibernation core [1] with addition of
PM_HIBERNATION_UNPREPAREevent that will be invoked when hibernation fails, whilePM_POST_HIBERNATIONwill be invoked only after a successful hibernationYou may start with implementing the first option right away and later implement the second.
Quoted 6 lines
I am testing the first method - send unref message to QEMU so that connected resources can be all removed from QEMU. But I found out there is cb that clean up the object once this unref is processed and response is received. It means that we can't just remove the resource/bo only in QEMU side. There is a way to do this though. We can simply add one more fuction in
virtgpu_vq.cthat handles unref but with cb = NULL. Is this what you want to try or do you have any better ideas?
The cb certainly shall not be invoked. You only need to send the "DETACH" cmd to host. This should be akin to what I did in my older patches adding memory shmem shrinker support to virtio-gpu, please see
[1] for inspiration.
[1]
https://lore.kernel.org/dri-devel/[email protected]/
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Quoted 46 lines · Dmitry Osipenko
Hi,
Quoted 37 lines · Dongwon
Hi Dmitry,
Quoted 27 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 14, 2025 5:16 AM
To: Kim, Dongwon <[email protected]>; dri- [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 3 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just
run.
Quoted 5 lines
In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any ofvirtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor from host. Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
Quoted 7 lines
1. Destroy stored host resources upon hibernation 2. Extend hibernation core [1] with addition of
PM_HIBERNATION_UNPREPAREevent that will be invoked when hibernation fails, whilePM_POST_HIBERNATIONwill be invoked only after a successful hibernationYou may start with implementing the first option right away and later implement the second.
Quoted 6 lines
I am testing the first method - send unref message to QEMU so that connected resources can be all removed from QEMU. But I found out there is cb that clean up the object once this unref is processed and response is received. It means that we can't just remove the resource/bo only in QEMU side. There is a way to do this though. We can simply add one more fuction in
virtgpu_vq.cthat handles unref but with cb = NULL. Is this what you want to try or do you have any better ideas?
Quoted 7 lines
The cb certainly shall not be invoked. You only need to send the "DETACH" cmd to host. This should be akin to what I did in my older patches adding memory shmem shrinker support to virtio-gpu, please see
[1] for inspiration.
Actually, in your case it will be the "UNREF" cmd, not "DETACH". But the idea in the same, you need to destroy resource only on host on suspend.
Signature · 2 lines
Best regards, Dmitry
RE: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Hi Dmitry,
Quoted 65 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 21, 2025 1:04 PM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 30 lines · Dmitry Osipenko
Hi,
Quoted 27 lines · Dongwon
Hi Dmitry,
Quoted 25 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 14, 2025 5:16 AM
To: Kim, Dongwon <[email protected]>; dri- [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 4 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just
run.
Quoted 6 lines
In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any ofvirtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor
from host.
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
1. Destroy stored host resources upon hibernation 2. Extend hibernation core [1] with addition of
PM_HIBERNATION_UNPREPAREevent that will be invoked when hibernation fails, whilePM_POST_HIBERNATIONwill be invoked only after a successful hibernationYou may start with implementing the first option right away and later implement the second.
Quoted 6 lines
Quoted 6 lines
I am testing the first method - send unref message to QEMU so that connected resources can be all removed from QEMU. But I found out there is cb that clean up the object once this unref is processed and response is received. It means that we can't just remove the resource/bo only in QEMU side. There is a way to do this though. We can simply add one more fuction in
virtgpu_vq.cthat handles unref but withcb = NULL. Is this what you want to try or do you have any better ideas?
Quoted 5 lines
The cb certainly shall not be invoked. You only need to send the "DETACH" cmd to host. This should be akin to what I did in my older patches adding memory shmem shrinker support to virtio-gpu, please see
[1] for inspiration.
Quoted 6 lines
Quoted 3 lines
[1]
https://lore.kernel.org/dri-devel/20231029230205.93277-26-dmitry.osipe
[email protected]/Actually, in your case it will be the "UNREF" cmd, not "DETACH". But the idea in the same, you need to destroy resource only on host on suspend.
Yes, this is the point. Current unref command always comes with a callback that cleans up virtio-gpu bo on the guest side. One way to avoid is to introduce another special version of unref that doesn't register any callback or.. I am wondering if we just make virtio-gpu driver ignores that callback in this specific situation (pre hibernation??). Either way looks like a dirty hack to me though. What do you think?
Quoted 4 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
Quoted 71 lines · Dongwon
Hi Dmitry,
Quoted 69 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 21, 2025 1:04 PM
To: Kim, Dongwon <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 30 lines · Dmitry Osipenko
Hi,
Quoted 27 lines · Dongwon
Hi Dmitry,
Quoted 25 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Friday, November 14, 2025 5:16 AM
To: Kim, Dongwon <[email protected]>; dri- [email protected]
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v6 3/3] drm/virtio: Add PM notifier to restore objects after hibernationQuoted 4 lines · Dongwon
Quoted 2 lines
One option could be to explicitly destroy all stored objs upon hibernation, that way the restoring will always work.
Yes, we can do it to avoid that corner case. Or maybe we can just let it just
run.
Quoted 6 lines
In this case,
virtio_gpu_object_restore_allwon't fail as shmem init will still work but QEMU will justsend back errored replies as all of those resources for BOs are still there in QEMU side but I think it won't break anything. Do you see any issues in doing this that I might be missing?? My assumption here is that the QEMU hasn't done any ofvirtio-gpu resets here as hibernation failed.
Correct, QEMU will emit a ton of "resource already exists" errors on aborted hibernation. There should be no errors neither from guest, nor
from host.
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Note that QEMU is not the only VMM using VirtIO-GPU.
Two options here:
1. Destroy stored host resources upon hibernation 2. Extend hibernation core [1] with addition of
PM_HIBERNATION_UNPREPAREevent that will be invoked when hibernation fails, whilePM_POST_HIBERNATIONwill be invoked only after a successful hibernationYou may start with implementing the first option right away and later implement the second.
Quoted 6 lines
Quoted 6 lines
I am testing the first method - send unref message to QEMU so that connected resources can be all removed from QEMU. But I found out there is cb that clean up the object once this unref is processed and response is received. It means that we can't just remove the resource/bo only in QEMU side. There is a way to do this though. We can simply add one more fuction in
virtgpu_vq.cthat handles unref but withcb = NULL. Is this what you want to try or do you have any better ideas?
Quoted 8 lines
The cb certainly shall not be invoked. You only need to send the "DETACH" cmd to host. This should be akin to what I did in my older patches adding memory shmem shrinker support to virtio-gpu, please see
[1] for inspiration.[1]
https://lore.kernel.org/dri-devel/20231029230205.93277-26-dmitry.osipe
[email protected]/
Quoted 8 lines
Quoted 2 lines
Actually, in your case it will be the "UNREF" cmd, not "DETACH". But the idea in the same, you need to destroy resource only on host on suspend.
Yes, this is the point. Current unref command always comes with a callback that cleans up virtio-gpu bo on the guest side. One way to avoid is to introduce another special version of unref that doesn't register any callback or.. I am wondering if we just make virtio-gpu driver ignores that callback in this specific situation (pre hibernation??). Either way looks like a dirty hack to me though. What do you think?
Add a new variant of virtio_gpu_cmd_unref_resource() that doesn't use callback and then unref a bulk of BOs in one go.
Signature · 2 lines
Best regards, Dmitry