Skip to contentSkip to conversation

Virtio-GPU S4 support

PATCH Superseded [email protected] 3 participants 14 messages Jan 7 – Apr 30

Subsystems (3)

v7 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v7 cover Virtio-GPU S4 support

From: Dongwon Kim <[email protected]>

This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.

First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.

Second patch implements a mechanism for restoring virtio_gpu_object instances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.

Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.

These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.

v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.

v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)

v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)

v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)

Protect restore list operations with a spinlock (Nirmoy Das)

Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)

Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)

v6: Include object backed by imported dmabuf
(Dmitry Osipenko)

Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)

Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)

Use mutex instead of spinlock when updating restore_list (Nirmoy Das)

Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.

Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.

v7: Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)

Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)

Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation

 drivers/gpu/drm/virtio/virtgpu_drv.c    | 74 +++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_drv.h    | 23 ++++++-
 drivers/gpu/drm/virtio/virtgpu_kms.c    | 54 ++++++++++++++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 83 ++++++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_prime.c  | 43 ++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++-
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  4 +-
 7 files changed, 280 insertions(+), 14 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v7 1/3 drm/virtio: Freeze and restore hooks to support suspend and resume

From: Dongwon Kim <[email protected]>

virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume

v2: 10ms sleep was added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.

v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)

v4: Change wait_event to wait_event_timeout to prevent permanent wait
(Nirmoy Das)

---
 drivers/gpu/drm/virtio/virtgpu_drv.c | 62 +++++++++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_drv.h |  1 +
 drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++---
 3 files changed, 79 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +61 -1
index 71c6ccad4b99..039fb8f18ead 100644
@@ -163,6 +163,62 @@ static unsigned int features[] = {
 	VIRTIO_GPU_F_RESOURCE_BLOB,
 	VIRTIO_GPU_F_CONTEXT_INIT,
 };
+
+#ifdef CONFIG_PM_SLEEP
+static int virtgpu_freeze(struct virtio_device *vdev)
+{
+	struct drm_device *dev = vdev->priv;
+	struct virtio_gpu_device *vgdev = dev->dev_private;
+	int error;
+
+	error = drm_mode_config_helper_suspend(dev);
+	if (error) {
+		DRM_ERROR("suspend error %d\n", error);
+		return error;
+	}
+
+	flush_work(&vgdev->obj_free_work);
+	flush_work(&vgdev->ctrlq.dequeue_work);
+	flush_work(&vgdev->cursorq.dequeue_work);
+	flush_work(&vgdev->config_changed_work);
+
+	wait_event_timeout(vgdev->ctrlq.ack_queue,
+		vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max,
+		5 * HZ);
+
+	wait_event_timeout(vgdev->cursorq.ack_queue,
+		vgdev->cursorq.vq->num_free == vgdev->cursorq.vq->num_max,
+		5 * HZ);
+
+	vdev->config->del_vqs(vdev);
+
+	return 0;
+}
+
+static int virtgpu_restore(struct virtio_device *vdev)
+{
+	struct drm_device *dev = vdev->priv;
+	struct virtio_gpu_device *vgdev = dev->dev_private;
+	int error;
+
+	error = virtio_gpu_find_vqs(vgdev);
+	if (error) {
+		DRM_ERROR("failed to find virt queues\n");
+		return error;
+	}
+
+	virtio_device_ready(vdev);
+
+	error = drm_mode_config_helper_resume(dev);
+	if (error) {
+		DRM_ERROR("resume error %d\n", error);
+		return error;
+	}
+
+	return 0;
+}
+#endif
+
 static struct virtio_driver virtio_gpu_driver = {
 	.feature_table = features,
 	.feature_table_size = ARRAY_SIZE(features),
@@ -171,7 +227,11 @@ static struct virtio_driver virtio_gpu_driver = {
 	.probe = virtio_gpu_probe,
 	.remove = virtio_gpu_remove,
 	.shutdown = virtio_gpu_shutdown,
-	.config_changed = virtio_gpu_config_changed
+	.config_changed = virtio_gpu_config_changed,
+#ifdef CONFIG_PM_SLEEP
+	.freeze = virtgpu_freeze,
+	.restore = virtgpu_restore,
+#endif
 };
 
 static int __init virtio_gpu_driver_init(void)
drivers/gpu/drm/virtio/virtgpu_drv.h +1 -0
index f17660a71a3e..1279f998c8e0 100644
@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);
 void virtio_gpu_release(struct drm_device *dev);
 int virtio_gpu_driver_open(struct drm_device *dev, struct drm_file *file);
 void virtio_gpu_driver_postclose(struct drm_device *dev, struct drm_file *file);
+int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev);
 
 /* virtgpu_gem.c */
 int virtio_gpu_gem_object_open(struct drm_gem_object *obj,
drivers/gpu/drm/virtio/virtgpu_kms.c +17 -6
index 1c15cbf326b7..cbebe19c3fb3 100644
@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev,
 	vgdev->num_capsets = num_capsets;
 }
 
-int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
+int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev)
 {
 	struct virtqueue_info vqs_info[] = {
 		{ "control", virtio_gpu_ctrl_ack },
 		{ "cursor", virtio_gpu_cursor_ack },
 	};
-	struct virtio_gpu_device *vgdev;
-	/* this will expand later */
 	struct virtqueue *vqs[2];
+	int ret;
+
+	ret = virtio_find_vqs(vgdev->vdev, 2, vqs, vqs_info, NULL);
+	if (ret)
+		return ret;
+
+	vgdev->ctrlq.vq = vqs[0];
+	vgdev->cursorq.vq = vqs[1];
+
+	return 0;
+}
+
+int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
+{
+	struct virtio_gpu_device *vgdev;
 	u32 num_scanouts, num_capsets;
 	int ret = 0;
 
@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 	DRM_INFO("features: %ccontext_init\n",
 		 vgdev->has_context_init ? '+' : '-');
 
-	ret = virtio_find_vqs(vgdev->vdev, 2, vqs, vqs_info, NULL);
+	ret = virtio_gpu_find_vqs(vgdev);
 	if (ret) {
 		DRM_ERROR("failed to find virt queues\n");
 		goto err_vqs;
 	}
-	vgdev->ctrlq.vq = vqs[0];
-	vgdev->cursorq.vq = vqs[1];
 	ret = virtio_gpu_alloc_vbufs(vgdev);
 	if (ret) {
 		DRM_ERROR("failed to alloc vbufs\n");
Signature · 1 line
2.34.1
maintainer
#2 lore raw

Re: [PATCH v7 1/3] drm/virtio: Freeze and restore hooks to support suspend and resume

drivers/gpu/drm/virtio/virtgpu_drv.c static unsigned int features[] = { · line 176

Quoted 10 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +10 -0
+#ifdef CONFIG_PM_SLEEP
+static int virtgpu_freeze(struct virtio_device *vdev)
+{
+	struct drm_device *dev = vdev->priv;
+	struct virtio_gpu_device *vgdev = dev->dev_private;
+	int error;
+
+	error = drm_mode_config_helper_suspend(dev);
+	if (error) {
+		DRM_ERROR("suspend error %d\n", error);

Nit: better to use DRM_DEV_ERROR everywhere to know which device produces the error

Signature · 2 lines
Best regards,
Dmitry
#3 lore raw

PATCH v7 2/3 drm/virtio: Add support for saving and restoring virtio_gpu_objects

In reply to [email protected]

From: Dongwon Kim <[email protected]>

When the host KVM/QEMU resumes from hibernation, it loses all graphics resources previously submitted by the guest OS, as the QEMU process is terminated during the suspend-resume cycle. This leads to invalid resource errors when the guest OS attempts to interact with the host using those resources after resumption.

To resolve this, the virtio-gpu driver now tracks all active virtio_gpu_objects and provides a mechanism to restore them by re-submitting the objects to QEMU when needed (e.g., during resume from hibernation).

v2: - Attach backing is done if bo->attached was set before

v3: - Restoration is no longer triggered via .restore; instead, it is handled
by a PM notifier only during hibernation.

v4: - Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)

- Protect restore list operations with a spinlock (Nirmoy Das)

- Initialize ret with 0 in virtio_gpu_object_restore_all (Nirmoy Das)

- Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)

v5: - Include object backed by imported dmabuf
(Dmitry Osipenko)

- Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)

- Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)

- Use mutex instead of spinlock when updating restore_list (Nirmoy Das)

- Initialize restore_node when virtio_gpu_object is created - this is to determine whether the object should be removed from the restore_list with 'list_empty' function when it is time to free the object as not all objects will be added to the list.

v6: - Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)

Cc Dmitry Osipenko <[email protected]>
Cc Vivek Kasireddy <[email protected]>
Cc Nirmoy Das <[email protected]>
---
 drivers/gpu/drm/virtio/virtgpu_drv.h    | 14 ++++++
 drivers/gpu/drm/virtio/virtgpu_kms.c    |  3 ++
 drivers/gpu/drm/virtio/virtgpu_object.c | 66 +++++++++++++++++++++++++
 drivers/gpu/drm/virtio/virtgpu_prime.c  | 41 +++++++++++++++
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  2 +
 5 files changed, 126 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.h +14 -0
index 1279f998c8e0..1632d92ee346 100644
@@ -98,6 +98,10 @@ struct virtio_gpu_object {
 
 	int uuid_state;
 	uuid_t uuid;
+
+	/* for restoration of objects after hibernation */
+	struct virtio_gpu_object_params params;
+	struct list_head restore_node;
 };
 #define gem_to_virtio_gpu_obj(gobj) \
 	container_of((gobj), struct virtio_gpu_object, base.base)
@@ -265,6 +269,8 @@ struct virtio_gpu_device {
 	struct work_struct obj_free_work;
 	spinlock_t obj_free_lock;
 	struct list_head obj_free_list;
+	struct mutex obj_restore_lock;
+	struct list_head obj_restore_list;
 
 	struct virtio_gpu_drv_capset *capsets;
 	uint32_t num_capsets;
@@ -479,6 +485,12 @@ bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);
 
 int virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
 			       uint32_t *resid);
+
+void virtio_gpu_add_object_to_restore_list(struct virtio_gpu_device *vgdev,
+					   struct virtio_gpu_object *bo);
+
+int virtio_gpu_object_restore_all(struct virtio_gpu_device *vgdev);
+
 /* virtgpu_prime.c */
 int virtio_gpu_resource_assign_uuid(struct virtio_gpu_device *vgdev,
 				    struct virtio_gpu_object *bo);
@@ -493,6 +505,8 @@ int virtgpu_dma_buf_import_sgt(struct virtio_gpu_mem_entry **ents,
 			       unsigned int *nents,
 			       struct virtio_gpu_object *bo,
 			       struct dma_buf_attachment *attach);
+int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
+				 struct virtio_gpu_object *bo);
 
 /* virtgpu_debugfs.c */
 void virtio_gpu_debugfs_init(struct drm_minor *minor);
drivers/gpu/drm/virtio/virtgpu_kms.c +3 -0
index cbebe19c3fb3..8ad79de70d85 100644
@@ -170,6 +170,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 		  virtio_gpu_array_put_free_work);
 	INIT_LIST_HEAD(&vgdev->obj_free_list);
 	spin_lock_init(&vgdev->obj_free_lock);
+	INIT_LIST_HEAD(&vgdev->obj_restore_list);
+	mutex_init(&vgdev->obj_restore_lock);
 
 #ifdef __LITTLE_ENDIAN
 	if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_F_VIRGL))
@@ -298,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev)
 	flush_work(&vgdev->config_changed_work);
 	virtio_reset_device(vgdev->vdev);
 	vgdev->vdev->config->del_vqs(vgdev->vdev);
+	mutex_destroy(&vgdev->obj_restore_lock);
 }
 
 void virtio_gpu_release(struct drm_device *dev)
drivers/gpu/drm/virtio/virtgpu_object.c +66 -0

87 lines not on this page · raw message

drivers/gpu/drm/virtio/virtgpu_prime.c +41 -0

59 lines not on this page · raw message

drivers/gpu/drm/virtio/virtgpu_vram.c +2 -0
index 5ad3b7c6f73c..dffba04849f5 100644
@@ -200,6 +200,8 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev,
 	obj = &vram->base.base.base;
 	obj->funcs = &virtio_gpu_vram_funcs;
 
+	INIT_LIST_HEAD(&vram->base.restore_node);
+
 	params->size = PAGE_ALIGN(params->size);
 	drm_gem_private_object_init(vgdev->ddev, obj, params->size);
 
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects

kernel test robot noticed the following build errors:

Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124
base:   https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next
patch link:    https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com
patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
config: x86_64-rhel-9.4 (https://download.01.org/0day-ci/archive/20260109/[email protected]/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260109/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

drivers/gpu/drm/virtio/virtgpu_prime.c: In function 'virtgpu_dma_buf_obj_resubmit':

>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:17: error: implicit declaration of function 'DRM_ERROR'; did you mean 'pud_ERROR'? [-Wimplicit-function-declaration]
     291 |                 DRM_ERROR("no sgt bound to virtio_gpu_object\n");
         |                 ^~~~~~~~~
         |                 pud_ERROR

vim +291 drivers/gpu/drm/virtio/virtgpu_prime.c

282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {

 > 291			DRM_ERROR("no sgt bound to virtio_gpu_object\n");

292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#5 lore raw

Re: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects

In reply to [email protected]
kernel test robot noticed the following build errors:

Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124
base:   https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next
patch link:    https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com
patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
config: hexagon-allmodconfig (https://download.01.org/0day-ci/archive/20260112/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260112/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:3: error: call to undeclared function 'DRM_ERROR'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     291 |                 DRM_ERROR("no sgt bound to virtio_gpu_object\n");
         |                 ^
   drivers/gpu/drm/virtio/virtgpu_prime.c:299:3: error: call to undeclared function 'DRM_ERROR'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     299 |                 DRM_ERROR("failed to allocate ent list\n");
         |                 ^

2 errors generated.

vim +/DRM_ERROR +291 drivers/gpu/drm/virtio/virtgpu_prime.c

282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {

 > 291			DRM_ERROR("no sgt bound to virtio_gpu_object\n");

292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#6 lore raw

Re: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects

In reply to [email protected]
kernel test robot noticed the following build errors:

Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124
base:   https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next
patch link:    https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com
patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
config: m68k-allmodconfig (https://download.01.org/0day-ci/archive/20260112/[email protected]/config)
compiler: m68k-linux-gcc (GCC) 15.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260112/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

drivers/gpu/drm/virtio/virtgpu_prime.c: In function 'virtgpu_dma_buf_obj_resubmit':

>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:17: error: implicit declaration of function 'DRM_ERROR'; did you mean 'DRM_IOR'? [-Wimplicit-function-declaration]
     291 |                 DRM_ERROR("no sgt bound to virtio_gpu_object\n");
         |                 ^~~~~~~~~
         |                 DRM_IOR

vim +291 drivers/gpu/drm/virtio/virtgpu_prime.c

282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {

 > 291			DRM_ERROR("no sgt bound to virtio_gpu_object\n");

292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
maintainer
#7 lore raw

Re: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects

In reply to [email protected]

drivers/gpu/drm/virtio/virtgpu_object.c void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) · line 72

Quoted 13 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +5 -0
index e6363c887500..8e8a8ec4a361 100644
@@ -65,6 +65,12 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
 {
 	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
 
+	if (!list_empty(&bo->restore_node)) {
+		mutex_lock(&vgdev->obj_restore_lock);
+		list_del(&bo->restore_node);
+		mutex_unlock(&vgdev->obj_restore_lock);
+	}

This looks fragile to me, I'd use list_del_init() unconditionally.

Also, looks wrong to delete object's node from the list by the virtio_gpu_cleanup_object() that is invoked from cmd-completion callback. You calling virtio_gpu_object_unref_all() that walks up obj_restore_list while obj already may be in process of async unref'ing?

Signature · 2 lines
Best regards,
Dmitry
#8 lore raw

PATCH v7 3/3 drm/virtio: Add PM notifier to restore objects after hibernation

In reply to [email protected]

From: Dongwon Kim <[email protected]>

Register a PM notifier in virtio-gpu to handle suspend/hibernate events. On PM_POST_HIBERNATION, restore all GPU objects so that the driver can properly recover after resume.

v2: Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)

v3: Objects for virgl usecase can't be recovered after resume so
blocking S4 when virgl is enabled
(Dmitry Osipenko)

v4: Restoring objects in the PM notifier is too late, as virtio-gpu
message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.

v5: Unreference all objects before hibernation so they can be removed
on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)

---
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 12 +++++++++++
 drivers/gpu/drm/virtio/virtgpu_drv.h    |  8 ++++++-
 drivers/gpu/drm/virtio/virtgpu_kms.c    | 28 +++++++++++++++++++++++++
 drivers/gpu/drm/virtio/virtgpu_object.c | 17 ++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_prime.c  |  2 +-
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++++++++---
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  2 +-
 7 files changed, 75 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +12 -0
index 039fb8f18ead..268f45f9a573 100644
@@ -177,6 +177,9 @@ static int virtgpu_freeze(struct virtio_device *vdev)
 		return error;
 	}
 
+	if (vgdev->hibernation)
+		virtio_gpu_object_unref_all(vgdev);
+
 	flush_work(&vgdev->obj_free_work);
 	flush_work(&vgdev->ctrlq.dequeue_work);
 	flush_work(&vgdev->cursorq.dequeue_work);
@@ -209,6 +212,15 @@ static int virtgpu_restore(struct virtio_device *vdev)
 
 	virtio_device_ready(vdev);
 
+	if (vgdev->hibernation) {
+		vgdev->hibernation = false;
+		error = virtio_gpu_object_restore_all(vgdev);
+		if (error) {
+			DRM_ERROR("Failed to recover virtio-gpu objects\n");
+			return error;
+		}
+	}
+
 	error = drm_mode_config_helper_resume(dev);
 	if (error) {
 		DRM_ERROR("resume error %d\n", error);
drivers/gpu/drm/virtio/virtgpu_drv.h +7 -1
index 1632d92ee346..ffddaaf9bc76 100644
@@ -261,6 +261,7 @@ struct virtio_gpu_device {
 	bool has_resource_blob;
 	bool has_host_visible;
 	bool has_context_init;
+	bool hibernation;
 	struct virtio_shm_region host_visible_region;
 	struct drm_mm host_visible_mm;
 
@@ -277,6 +278,8 @@ struct virtio_gpu_device {
 	uint64_t capset_id_mask;
 	struct list_head cap_cache;
 
+	struct notifier_block pm_nb;
+
 	/* protects uuid state when exporting */
 	spinlock_t resource_export_lock;
 	/* protects map state and host_visible_mm */
@@ -341,7 +344,8 @@ void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,
 				    struct virtio_gpu_object_array *objs,
 				    struct virtio_gpu_fence *fence);
 void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
-				   struct virtio_gpu_object *bo);
+				   struct virtio_gpu_object *bo,
+				   int no_cb);
 int virtio_gpu_panic_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
 					     uint64_t offset,
 					     uint32_t width, uint32_t height,
@@ -491,6 +495,8 @@ void virtio_gpu_add_object_to_restore_list(struct virtio_gpu_device *vgdev,
 
 int virtio_gpu_object_restore_all(struct virtio_gpu_device *vgdev);
 
+void virtio_gpu_object_unref_all(struct virtio_gpu_device *vgdev);
+
 /* virtgpu_prime.c */
 int virtio_gpu_resource_assign_uuid(struct virtio_gpu_device *vgdev,
 				    struct virtio_gpu_object *bo);
drivers/gpu/drm/virtio/virtgpu_kms.c +28 -0
index 8ad79de70d85..cc8aad1b72cd 100644
@@ -26,6 +26,8 @@
 #include <linux/virtio.h>
 #include <linux/virtio_config.h>
 #include <linux/virtio_ring.h>
+#include <linux/suspend.h>
+#include <linux/pm_runtime.h>
 
 #include <drm/drm_file.h>
 #include <drm/drm_managed.h>
@@ -133,6 +135,25 @@ int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev)
 	return 0;
 }
 
+static int virtio_gpu_pm_notifier(struct notifier_block *nb, unsigned long mode,
+				  void *data)
+{
+	struct virtio_gpu_device *vgdev = container_of(nb,
+						struct virtio_gpu_device,
+						pm_nb);
+
+	if (mode == PM_HIBERNATION_PREPARE) {
+		if (vgdev->has_virgl_3d) {
+			DRM_ERROR("S4 not allowed when VIRGL is enabled\n");
+			return notifier_from_errno(-EPERM);
+		}
+
+		vgdev->hibernation = true;
+	}
+
+	return NOTIFY_DONE;
+}
+
 int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 {
 	struct virtio_gpu_device *vgdev;
@@ -269,6 +290,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 		wait_event_timeout(vgdev->resp_wq, !vgdev->display_info_pending,
 				   5 * HZ);
 	}
+
+	vgdev->pm_nb.notifier_call = virtio_gpu_pm_notifier;
+	ret = register_pm_notifier(&vgdev->pm_nb);
+	if (ret)
+		goto err_scanouts;
+
 	return 0;
 
 err_scanouts:
@@ -301,6 +328,7 @@ void virtio_gpu_deinit(struct drm_device *dev)
 	virtio_reset_device(vgdev->vdev);
 	vgdev->vdev->config->del_vqs(vgdev->vdev);
 	mutex_destroy(&vgdev->obj_restore_lock);
+	unregister_pm_notifier(&vgdev->pm_nb);
 }
 
 void virtio_gpu_release(struct drm_device *dev)
drivers/gpu/drm/virtio/virtgpu_object.c +16 -1
index 8e8a8ec4a361..7c27c2fcd7ed 100644
@@ -98,7 +98,7 @@ static void virtio_gpu_free_object(struct drm_gem_object *obj)
 	struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;
 
 	if (bo->created) {
-		virtio_gpu_cmd_unref_resource(vgdev, bo);
+		virtio_gpu_cmd_unref_resource(vgdev, bo, false);
 		virtio_gpu_notify(vgdev);
 		/* completion handler calls virtio_gpu_cleanup_object() */
 		return;
@@ -337,3 +337,18 @@ int virtio_gpu_object_restore_all(struct virtio_gpu_device *vgdev)
 
 	return ret;
 }
+
+void virtio_gpu_object_unref_all(struct virtio_gpu_device *vgdev)
+{
+	struct virtio_gpu_object *bo, *tmp;
+
+	mutex_lock(&vgdev->obj_restore_lock);
+	list_for_each_entry_safe(bo, tmp, &vgdev->obj_restore_list,
+				 restore_node)
+		if (bo->created) {
+			virtio_gpu_cmd_unref_resource(vgdev, bo, true);
+			virtio_gpu_notify(vgdev);
+		}
+
+	mutex_unlock(&vgdev->obj_restore_lock);
+}
drivers/gpu/drm/virtio/virtgpu_prime.c +1 -1
index 088eaf000a9a..69f2186d0908 100644
@@ -218,7 +218,7 @@ static void virtgpu_dma_buf_free_obj(struct drm_gem_object *obj)
 	}
 
 	if (bo->created) {
-		virtio_gpu_cmd_unref_resource(vgdev, bo);
+		virtio_gpu_cmd_unref_resource(vgdev, bo, false);
 		virtio_gpu_notify(vgdev);
 		return;
 	}
drivers/gpu/drm/virtio/virtgpu_vq.c +10 -3
index 8181b22b9b46..fdd0166584bc 100644
@@ -625,14 +625,21 @@ static void virtio_gpu_cmd_unref_cb(struct virtio_gpu_device *vgdev,
 }
 
 void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
-				   struct virtio_gpu_object *bo)
+				   struct virtio_gpu_object *bo,
+				   int no_cb)
 {
 	struct virtio_gpu_resource_unref *cmd_p;
 	struct virtio_gpu_vbuffer *vbuf;
 	int ret;
 
-	cmd_p = virtio_gpu_alloc_cmd_cb(vgdev, &vbuf, sizeof(*cmd_p),
-					virtio_gpu_cmd_unref_cb);
+	if (no_cb) {
+		cmd_p = virtio_gpu_alloc_cmd_cb(vgdev, &vbuf, sizeof(*cmd_p),
+						NULL);
+	} else {
+		cmd_p = virtio_gpu_alloc_cmd_cb(vgdev, &vbuf, sizeof(*cmd_p),
+						virtio_gpu_cmd_unref_cb);
+	}
+
 	memset(cmd_p, 0, sizeof(*cmd_p));
 
 	cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_UNREF);
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
index dffba04849f5..5ec75bf03d0f 100644
@@ -18,7 +18,7 @@ static void virtio_gpu_vram_free(struct drm_gem_object *obj)
 		if (unmap)
 			virtio_gpu_cmd_unmap(vgdev, bo);
 
-		virtio_gpu_cmd_unref_resource(vgdev, bo);
+		virtio_gpu_cmd_unref_resource(vgdev, bo, false);
 		virtio_gpu_notify(vgdev);
 		return;
 	}
Signature · 1 line
2.34.1
maintainer
#9 lore raw

Re: [PATCH v7 3/3] drm/virtio: Add PM notifier to restore objects after hibernation

drivers/gpu/drm/virtio/virtgpu_drv.c static int virtgpu_freeze(struct virtio_device *vdev) · line 181

Quoted 8 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +2 -0
@@ -177,6 +177,9 @@ static int virtgpu_freeze(struct virtio_device *vdev)
 		return error;
 	}
 
+	if (vgdev->hibernation)
+		virtio_gpu_object_unref_all(vgdev);

Nit: function name doesn't reflect well what it's doing, I'd name it as virtio_gpu_unref_stored_objs

Signature · 2 lines
Best regards,
Dmitry
maintainer
#10 lore raw

Re: [PATCH v7 3/3] drm/virtio: Add PM notifier to restore objects after hibernation

In reply to [email protected]

drivers/gpu/drm/virtio/virtgpu_drv.h struct virtio_gpu_device { · line 264

Quoted 6 lines · On 1/7/26 21:27, [email protected] wrote:

+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h

drivers/gpu/drm/virtio/virtgpu_drv.h +1 -0
@@ -261,6 +261,7 @@ struct virtio_gpu_device {
 	bool has_resource_blob;
 	bool has_host_visible;
 	bool has_context_init;
+	bool hibernation;

Nit: bool hibernated;

Signature · 2 lines
Best regards,
Dmitry
maintainer
#11 lore raw
In reply to [email protected]
Quoted 85 lines · On 1/7/26 21:27, [email protected] wrote:

From: Dongwon Kim <[email protected]>

This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.

First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.

Second patch implements a mechanism for restoring virtio_gpu_object instances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.

Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.

These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.

v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.

v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)

v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)

v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)

Protect restore list operations with a spinlock (Nirmoy Das)

Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)

Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)

v6: Include object backed by imported dmabuf
(Dmitry Osipenko)

Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)

Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)

Use mutex instead of spinlock when updating restore_list (Nirmoy Das)

Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.

Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.

v7: Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)

Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)

Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation

Quoted 9 lines
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 74 +++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_drv.h    | 23 ++++++-
 drivers/gpu/drm/virtio/virtgpu_kms.c    | 54 ++++++++++++++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 83 ++++++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_prime.c  | 43 ++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++-
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  4 +-
 7 files changed, 280 insertions(+), 14 deletions(-)

Hello Kim,

Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.

Meanwhile there is a kernel bot bug report.

Signature · 2 lines
Best regards,
Dmitry
#12 lore raw

Hi Dmitry,

Quoted 113 lines

-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, February 25, 2026 8:06 AM
To: Kim, Dongwon <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Kasireddy, Vivek <[email protected]>
Subject: Re: [PATCH v7 0/3] Virtio-GPU S4 support

Quoted 7 lines · On 1/7/26 21:27, [email protected] wrote:

From: Dongwon Kim <[email protected]>

This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in
S4(hiberation) case.

Quoted 5 lines

First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.

Second patch implements a mechanism for restoring virtio_gpu_object

instances.

Quoted 2 lines

This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of

hiberation/resume process.

Quoted 7 lines

Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.

These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.

Quoted 8 lines

v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.

v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)

v4: New version of patchset only covers S4 case because loss of resources in

S3

Quoted 1 line

case can be avoided by skipping virtio-gpu-reset in QEMU

(hw/display/virtio-gpu.c).

Quoted 1 line

To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be

attached to

Quoted 1 line

PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set

to 'true'.

Quoted 55 lines
    (e.g. -device

virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)

v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)

Protect restore list operations with a spinlock (Nirmoy Das)

Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)

Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)

v6: Include object backed by imported dmabuf
(Dmitry Osipenko)

Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)

Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)

Use mutex instead of spinlock when updating restore_list (Nirmoy Das)

Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.

Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.

v7: Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)

Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)

Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation

Quoted 5 lines
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 74 +++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_drv.h    | 23 ++++++-
 drivers/gpu/drm/virtio/virtgpu_kms.c    | 54 ++++++++++++++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 83

++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_prime.c | 43

++++++++++++-

Quoted 4 lines
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++-
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  4 +-
 7 files changed, 280 insertions(+), 14 deletions(-)
Quoted 7 lines

Hello Kim,

Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.

Meanwhile there is a kernel bot bug report.

Thanks for the comment. No problem. I will wait for your test results then create V8 based on your feedback including what you just left for individual patches (function/variable names..etc.)

I thought I got an error report for the previous version but I will double check.

Quoted 4 lines

--
Best regards,
Dmitry

maintainer
#13 lore raw
Quoted 121 lines · Dongwon

Hi Dmitry,

Quoted 119 lines

-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, February 25, 2026 8:06 AM
To: Kim, Dongwon <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Kasireddy, Vivek <[email protected]>
Subject: Re: [PATCH v7 0/3] Virtio-GPU S4 support

Quoted 7 lines · On 1/7/26 21:27, [email protected] wrote:

From: Dongwon Kim <[email protected]>

This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in
S4(hiberation) case.

Quoted 5 lines

First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.

Second patch implements a mechanism for restoring virtio_gpu_object

instances.

Quoted 2 lines

This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of

hiberation/resume process.

Quoted 7 lines

Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.

These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.

Quoted 8 lines

v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.

v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)

v4: New version of patchset only covers S4 case because loss of resources in

S3

Quoted 1 line

case can be avoided by skipping virtio-gpu-reset in QEMU

(hw/display/virtio-gpu.c).

Quoted 1 line

To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be

attached to

Quoted 1 line

PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set

to 'true'.

Quoted 55 lines
    (e.g. -device

virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)

v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)

Protect restore list operations with a spinlock (Nirmoy Das)

Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)

Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)

v6: Include object backed by imported dmabuf
(Dmitry Osipenko)

Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)

Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)

Use mutex instead of spinlock when updating restore_list (Nirmoy Das)

Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.

Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.

v7: Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)

Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)

Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation

Quoted 5 lines
 drivers/gpu/drm/virtio/virtgpu_drv.c    | 74 +++++++++++++++++++++-
 drivers/gpu/drm/virtio/virtgpu_drv.h    | 23 ++++++-
 drivers/gpu/drm/virtio/virtgpu_kms.c    | 54 ++++++++++++++--
 drivers/gpu/drm/virtio/virtgpu_object.c | 83

++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_prime.c | 43

++++++++++++-

Quoted 4 lines
 drivers/gpu/drm/virtio/virtgpu_vq.c     | 13 +++-
 drivers/gpu/drm/virtio/virtgpu_vram.c   |  4 +-
 7 files changed, 280 insertions(+), 14 deletions(-)

Hello Kim,

Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.

Quoted 7 lines
Quoted 1 line

Meanwhile there is a kernel bot bug report.

Thanks for the comment. No problem. I will wait for your test results then create V8 based on your feedback including what you just left for individual patches (function/variable names..etc.)

I thought I got an error report for the previous version but I will double check.

Thanks for your patience and sending v8, will test it. There other patches in my queue, hopefully it will get untangled sooner.

Signature · 2 lines
Best regards,
Dmitry