Virtio-GPU S4 support
Subsystems (3)
v7 cover Virtio-GPU S4 support
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds virtgpu_freeze and virtgpu_restore functions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.
Second patch implements a mechanism for restoring virtio_gpu_object instances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.
Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)
Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)
Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
v6: Include object backed by imported dmabuf
(Dmitry Osipenko)
Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)
Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)
Use mutex instead of spinlock when updating restore_list (Nirmoy Das)
Initialize restore_node when virtio_gpu_object is created - this is to check if the node is in the list with 'list_empty' before removing it.
Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.
v7: Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)
Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)
Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoring virtio_gpu_objects drm/virtio: Add PM notifier to restore objects after hibernation
drivers/gpu/drm/virtio/virtgpu_drv.c | 74 +++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 23 ++++++- drivers/gpu/drm/virtio/virtgpu_kms.c | 54 ++++++++++++++-- drivers/gpu/drm/virtio/virtgpu_object.c | 83 ++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_prime.c | 43 ++++++++++++- drivers/gpu/drm/virtio/virtgpu_vq.c | 13 +++- drivers/gpu/drm/virtio/virtgpu_vram.c | 4 +- 7 files changed, 280 insertions(+), 14 deletions(-)
Signature · 1 line
2.34.1
PATCH v7 1/3 drm/virtio: Freeze and restore hooks to support suspend and resume
From: Dongwon Kim <[email protected]>
virtio device needs to delete before VM suspend happens then reinitialize all virtqueues again upon resume
v2: 10ms sleep was added in virtgpu_freeze to avoid the situation
the driver is locked up during resumption.
v3: Plain 10ms delay was replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)
v4: Change wait_event to wait_event_timeout to prevent permanent wait
(Nirmoy Das)
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 62 +++++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 1 + drivers/gpu/drm/virtio/virtgpu_kms.c | 23 ++++++++--- 3 files changed, 79 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +61 -1
@@ -163,6 +163,62 @@ static unsigned int features[] = {VIRTIO_GPU_F_RESOURCE_BLOB , VIRTIO_GPU_F_CONTEXT_INIT, }; + +#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error); +return error; + } + +flush_work (&vgdev->obj_free_work); +flush_work (&vgdev->ctrlq.dequeue_work); +flush_work (&vgdev->cursorq.dequeue_work); +flush_work (&vgdev->config_changed_work); + +wait_event_timeout (vgdev->ctrlq.ack_queue, + vgdev->ctrlq.vq->num_free == vgdev->ctrlq.vq->num_max, +5 *HZ ); + +wait_event_timeout (vgdev->cursorq.ack_queue, + vgdev->cursorq.vq->num_free == vgdev->cursorq.vq->num_max, +5 *HZ ); + + vdev->config->del_vqs (vdev); + +return 0 ; +} + +static int virtgpu_restore (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =virtio_gpu_find_vqs (vgdev); +if (error) { +DRM_ERROR ("failed to find virt queues\n" ); +return error; + } + +virtio_device_ready (vdev); + + error =drm_mode_config_helper_resume (dev); +if (error) { +DRM_ERROR ("resume error %d\n" , error); +return error; + } + +return 0 ; +} +#endif +static struct virtio_driver virtio_gpu_driver = { .feature_table = features, .feature_table_size = ARRAY_SIZE(features),
@@ -171,7 +227,11 @@ static struct virtio_driver virtio_gpu_driver = { .probe = virtio_gpu_probe, .remove = virtio_gpu_remove, .shutdown = virtio_gpu_shutdown, - .config_changed = virtio_gpu_config_changed + .config_changed = virtio_gpu_config_changed, +#ifdef CONFIG_PM_SLEEP + .freeze = virtgpu_freeze, + .restore = virtgpu_restore, +#endif };static int __init virtio_gpu_driver_init (void )
drivers/gpu/drm/virtio/virtgpu_drv.h +1 -0
@@ -300,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev);void virtio_gpu_release (struct drm_device *dev);int virtio_gpu_driver_open (struct drm_device *dev,struct drm_file *file);void virtio_gpu_driver_postclose (struct drm_device *dev,struct drm_file *file); +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev);/* virtgpu_gem.c */ int virtio_gpu_gem_object_open(struct drm_gem_object *obj,
drivers/gpu/drm/virtio/virtgpu_kms.c +17 -6
@@ -114,15 +114,28 @@ static void virtio_gpu_get_capsets(struct virtio_gpu_device *vgdev, vgdev->num_capsets = num_capsets; } -int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +int virtio_gpu_find_vqs (struct virtio_gpu_device *vgdev) {struct virtqueue_info vqs_info[] = { {"control" , virtio_gpu_ctrl_ack }, {"cursor" , virtio_gpu_cursor_ack }, }; -struct virtio_gpu_device *vgdev; -/* this will expand later */ struct virtqueue *vqs[2 ]; +int ret; + + ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); +if (ret) +return ret; + + vgdev->ctrlq.vq = vqs[0 ]; + vgdev->cursorq.vq = vqs[1 ]; + +return 0 ; +} + +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) +{ +struct virtio_gpu_device *vgdev;u32 num_scanouts, num_capsets;int ret =0 ;
@@ -206,13 +219,11 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)DRM_INFO ("features: %ccontext_init\n" , vgdev->has_context_init ?'+' :'-' ); - ret =virtio_find_vqs (vgdev->vdev,2 , vqs, vqs_info,NULL ); + ret =virtio_gpu_find_vqs (vgdev);if (ret) {DRM_ERROR ("failed to find virt queues\n" ); goto err_vqs; } - vgdev->ctrlq.vq = vqs[0 ]; - vgdev->cursorq.vq = vqs[1 ]; ret =virtio_gpu_alloc_vbufs (vgdev);if (ret) {DRM_ERROR ("failed to alloc vbufs\n" );
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerRe: [PATCH v7 1/3] drm/virtio: Freeze and restore hooks to support suspend and resume
drivers/gpu/drm/virtio/virtgpu_drv.c static unsigned int features[] = { · line 176
Quoted 10 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +10
-0+#ifdef CONFIG_PM_SLEEP +static int virtgpu_freeze (struct virtio_device *vdev) +{ +struct drm_device *dev = vdev->priv; +struct virtio_gpu_device *vgdev = dev->dev_private; +int error; + + error =drm_mode_config_helper_suspend (dev); +if (error) { +DRM_ERROR ("suspend error %d\n" , error);
Nit: better to use DRM_DEV_ERROR everywhere to know which device produces the error
Signature · 2 lines
Best regards, Dmitry
PATCH v7 2/3 drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]From: Dongwon Kim <[email protected]>
When the host KVM/QEMU resumes from hibernation, it loses all graphics resources previously submitted by the guest OS, as the QEMU process is terminated during the suspend-resume cycle. This leads to invalid resource errors when the guest OS attempts to interact with the host using those resources after resumption.
To resolve this, the virtio-gpu driver now tracks all active virtio_gpu_objects and provides a mechanism to restore them by re-submitting the objects to QEMU when needed (e.g., during resume from hibernation).
v2: - Attach backing is done if bo->attached was set before
v3: - Restoration is no longer triggered via .restore; instead, it is handled
by a PM notifier only during hibernation.
v4: - Remove virtio_gpu_object from the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)
- Protect restore list operations with a spinlock (Nirmoy Das)
- Initialize ret with 0 in virtio_gpu_object_restore_all (Nirmoy Das)
- Move restore list node into virtio_gpu_bo struct to reduce memory usage (Dmitry Osipenko)
v5: - Include object backed by imported dmabuf
(Dmitry Osipenko)
- Not storing virgl objects in the restore_list as virgl 3D objects are not recoverable.
(Dmitry Osipenko)
- Change the name 'list',a node in restore_list to 'restore_node' (Nirmoy Das)
- Use mutex instead of spinlock when updating restore_list (Nirmoy Das)
- Initialize restore_node when virtio_gpu_object is created - this is to determine whether the object should be removed from the restore_list with 'list_empty' function when it is time to free the object as not all objects will be added to the list.
v6: - Add a helper, virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.h | 14 ++++++ drivers/gpu/drm/virtio/virtgpu_kms.c | 3 ++ drivers/gpu/drm/virtio/virtgpu_object.c | 66 +++++++++++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_prime.c | 41 +++++++++++++++ drivers/gpu/drm/virtio/virtgpu_vram.c | 2 + 5 files changed, 126 insertions(+)
drivers/gpu/drm/virtio/virtgpu_drv.h +14 -0
@@ -98,6 +98,10 @@ struct virtio_gpu_object {int uuid_state;uuid_t uuid; + +/* for restoration of objects after hibernation */ +struct virtio_gpu_object_params params; +struct list_head restore_node; };#define gem_to_virtio_gpu_obj (gobj) \ container_of((gobj), struct virtio_gpu_object, base.base)
@@ -265,6 +269,8 @@ struct virtio_gpu_device {struct work_struct obj_free_work;spinlock_t obj_free_lock;struct list_head obj_free_list; +struct mutex obj_restore_lock; +struct list_head obj_restore_list;struct virtio_gpu_drv_capset *capsets;uint32_t num_capsets;
@@ -479,6 +485,12 @@ bool virtio_gpu_is_shmem(struct virtio_gpu_object *bo);int virtio_gpu_resource_id_get (struct virtio_gpu_device *vgdev,uint32_t *resid); + +void virtio_gpu_add_object_to_restore_list (struct virtio_gpu_device *vgdev, +struct virtio_gpu_object *bo); + +int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev); +/* virtgpu_prime.c */ int virtio_gpu_resource_assign_uuid (struct virtio_gpu_device *vgdev,struct virtio_gpu_object *bo);
@@ -493,6 +505,8 @@ int virtgpu_dma_buf_import_sgt(struct virtio_gpu_mem_entry **ents,unsigned int *nents, struct virtio_gpu_object *bo, struct dma_buf_attachment *attach); +int virtgpu_dma_buf_obj_resubmit (struct virtio_gpu_device *vgdev, +struct virtio_gpu_object *bo);/* virtgpu_debugfs.c */ void virtio_gpu_debugfs_init (struct drm_minor *minor);
drivers/gpu/drm/virtio/virtgpu_kms.c +3 -0
@@ -170,6 +170,8 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev) virtio_gpu_array_put_free_work);INIT_LIST_HEAD (&vgdev->obj_free_list);spin_lock_init (&vgdev->obj_free_lock); +INIT_LIST_HEAD (&vgdev->obj_restore_list); +mutex_init (&vgdev->obj_restore_lock);#ifdef __LITTLE_ENDIANif (virtio_has_feature (vgdev->vdev,VIRTIO_GPU_F_VIRGL ))
@@ -298,6 +300,7 @@ void virtio_gpu_deinit(struct drm_device *dev)flush_work (&vgdev->config_changed_work);virtio_reset_device (vgdev->vdev); vgdev->vdev->config->del_vqs (vgdev->vdev); +mutex_destroy (&vgdev->obj_restore_lock); }void virtio_gpu_release (struct drm_device *dev)
drivers/gpu/drm/virtio/virtgpu_object.c +66 -0
87 lines not on this page · raw message
drivers/gpu/drm/virtio/virtgpu_prime.c +41 -0
59 lines not on this page · raw message
drivers/gpu/drm/virtio/virtgpu_vram.c +2 -0
@@ -200,6 +200,8 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev, obj = &vram->base.base.base; obj->funcs = &virtio_gpu_vram_funcs; +INIT_LIST_HEAD (&vram->base.restore_node); + params->size =PAGE_ALIGN (params->size);drm_gem_private_object_init (vgdev->ddev, obj, params->size);
Signature · 1 line
2.34.1
kernel test robot
build botRe: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
kernel test robot noticed the following build errors:
Hi,
kernel test robot noticed the following build errors:
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124 base: https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next patch link: https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects config: x86_64-rhel-9.4 (https://download.01.org/0day-ci/archive/20260109/[email protected]/config) compiler: gcc-14 (Debian 14.2.0-19) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260109/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
drivers/gpu/drm/virtio/virtgpu_prime.c: In function 'virtgpu_dma_buf_obj_resubmit':
>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:17: error: implicit declaration of function 'DRM_ERROR'; did you mean 'pud_ERROR'? [-Wimplicit-function-declaration]
291 | DRM_ERROR("no sgt bound to virtio_gpu_object\n");
| ^~~~~~~~~
| pud_ERRORvim +291 drivers/gpu/drm/virtio/virtgpu_prime.c
282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {
> 291 DRM_ERROR("no sgt bound to virtio_gpu_object\n"); 292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]kernel test robot noticed the following build errors:
Hi,
kernel test robot noticed the following build errors:
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124 base: https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next patch link: https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects config: hexagon-allmodconfig (https://download.01.org/0day-ci/archive/20260112/[email protected]/config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260112/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:3: error: call to undeclared function 'DRM_ERROR'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
291 | DRM_ERROR("no sgt bound to virtio_gpu_object\n");
| ^
drivers/gpu/drm/virtio/virtgpu_prime.c:299:3: error: call to undeclared function 'DRM_ERROR'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
299 | DRM_ERROR("failed to allocate ent list\n");
| ^2 errors generated.
vim +/DRM_ERROR +291 drivers/gpu/drm/virtio/virtgpu_prime.c
282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {
> 291 DRM_ERROR("no sgt bound to virtio_gpu_object\n"); 292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]kernel test robot noticed the following build errors:
Hi,
kernel test robot noticed the following build errors:
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm/drm-next daeinki-drm-exynos/exynos-drm-next drm-i915/for-linux-next drm-i915/for-linux-next-fixes drm-tip/drm-tip linus/master v6.19-rc4 next-20260109] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/dongwon-kim-intel-com/drm-virtio-Freeze-and-restore-hooks-to-support-suspend-and-resume/20260108-024124 base: https://gitlab.freedesktop.org/drm/misc/kernel.git drm-misc-next patch link: https://lore.kernel.org/r/20260107182745.229481-3-dongwon.kim%40intel.com patch subject: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects config: m68k-allmodconfig (https://download.01.org/0day-ci/archive/20260112/[email protected]/config) compiler: m68k-linux-gcc (GCC) 15.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260112/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
drivers/gpu/drm/virtio/virtgpu_prime.c: In function 'virtgpu_dma_buf_obj_resubmit':
>> drivers/gpu/drm/virtio/virtgpu_prime.c:291:17: error: implicit declaration of function 'DRM_ERROR'; did you mean 'DRM_IOR'? [-Wimplicit-function-declaration]
291 | DRM_ERROR("no sgt bound to virtio_gpu_object\n");
| ^~~~~~~~~
| DRM_IORvim +291 drivers/gpu/drm/virtio/virtgpu_prime.c
282
283 int virtgpu_dma_buf_obj_resubmit(struct virtio_gpu_device *vgdev,
284 struct virtio_gpu_object *bo)
285 {
286 struct virtio_gpu_mem_entry *ents;
287 struct scatterlist *sl;
288 int i;
289
290 if (!bo->sgt) {
> 291 DRM_ERROR("no sgt bound to virtio_gpu_object\n"); 292 return -ENOMEM;
293 }
294
295 ents = kvmalloc_array(bo->sgt->nents,
296 sizeof(struct virtio_gpu_mem_entry),
297 GFP_KERNEL);
298 if (!ents) {
299 DRM_ERROR("failed to allocate ent list\n");
300 return -ENOMEM;
301 }
302
303 for_each_sgtable_dma_sg(bo->sgt, sl, i) {
304 ents[i].addr = cpu_to_le64(sg_dma_address(sl));
305 ents[i].length = cpu_to_le32(sg_dma_len(sl));
306 ents[i].padding = 0;
307 }
308
309 virtio_gpu_cmd_resource_create_blob(vgdev, bo, &bo->params,
310 ents, bo->sgt->nents);
311
312 return 0;
313 }
314
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Dmitry Osipenko
maintainerRe: [PATCH v7 2/3] drm/virtio: Add support for saving and restoring virtio_gpu_objects
In reply to [email protected]drivers/gpu/drm/virtio/virtgpu_object.c void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) · line 72
Quoted 13 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_object.c +5
-0@@ -65,6 +65,12 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo) {struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private; +if (!list_empty (&bo->restore_node)) { +mutex_lock (&vgdev->obj_restore_lock); +list_del (&bo->restore_node); +mutex_unlock (&vgdev->obj_restore_lock); + }
This looks fragile to me, I'd use list_del_init() unconditionally.
Also, looks wrong to delete object's node from the list by the virtio_gpu_cleanup_object() that is invoked from cmd-completion callback. You calling virtio_gpu_object_unref_all() that walks up obj_restore_list while obj already may be in process of async unref'ing?
Signature · 2 lines
Best regards, Dmitry
PATCH v7 3/3 drm/virtio: Add PM notifier to restore objects after hibernation
In reply to [email protected]From: Dongwon Kim <[email protected]>
Register a PM notifier in virtio-gpu to handle suspend/hibernate events. On PM_POST_HIBERNATION, restore all GPU objects so that the driver can properly recover after resume.
v2: Remove unused header - drm_atomic_helper.h
(Dmitry Osipenko)
v3: Objects for virgl usecase can't be recovered after resume so
blocking S4 when virgl is enabled
(Dmitry Osipenko)
v4: Restoring objects in the PM notifier is too late, as virtio-gpu
message communication begins in virtgpu_restore once virtqueues are re-established. To address this, a 'hibernation' flag is set during the PM_HIBERNATION_PREPARE phase in the notifier. This flag is then used in virtgpu_restore to detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.
v5: Unreference all objects before hibernation so they can be removed
on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)
--- drivers/gpu/drm/virtio/virtgpu_drv.c | 12 +++++++++++ drivers/gpu/drm/virtio/virtgpu_drv.h | 8 ++++++- drivers/gpu/drm/virtio/virtgpu_kms.c | 28 +++++++++++++++++++++++++ drivers/gpu/drm/virtio/virtgpu_object.c | 17 ++++++++++++++- drivers/gpu/drm/virtio/virtgpu_prime.c | 2 +- drivers/gpu/drm/virtio/virtgpu_vq.c | 13 +++++++++--- drivers/gpu/drm/virtio/virtgpu_vram.c | 2 +- 7 files changed, 75 insertions(+), 7 deletions(-)
drivers/gpu/drm/virtio/virtgpu_drv.c +12 -0
@@ -177,6 +177,9 @@ static int virtgpu_freeze(struct virtio_device *vdev)return error; } +if (vgdev->hibernation) +virtio_gpu_object_unref_all (vgdev); +flush_work (&vgdev->obj_free_work);flush_work (&vgdev->ctrlq.dequeue_work);flush_work (&vgdev->cursorq.dequeue_work);
@@ -209,6 +212,15 @@ static int virtgpu_restore(struct virtio_device *vdev)virtio_device_ready (vdev); +if (vgdev->hibernation) { + vgdev->hibernation = false; + error =virtio_gpu_object_restore_all (vgdev); +if (error) { +DRM_ERROR ("Failed to recover virtio-gpu objects\n" ); +return error; + } + } + error =drm_mode_config_helper_resume (dev);if (error) {DRM_ERROR ("resume error %d\n" , error);
drivers/gpu/drm/virtio/virtgpu_drv.h +7 -1
@@ -261,6 +261,7 @@ struct virtio_gpu_device {bool has_resource_blob;bool has_host_visible;bool has_context_init; +bool hibernation;struct virtio_shm_region host_visible_region;struct drm_mm host_visible_mm;
@@ -277,6 +278,8 @@ struct virtio_gpu_device {uint64_t capset_id_mask;struct list_head cap_cache; +struct notifier_block pm_nb; +/* protects uuid state when exporting */ spinlock_t resource_export_lock;/* protects map state and host_visible_mm */
@@ -341,7 +344,8 @@ void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,struct virtio_gpu_object_array *objs, struct virtio_gpu_fence *fence);void virtio_gpu_cmd_unref_resource (struct virtio_gpu_device *vgdev, -struct virtio_gpu_object *bo); +struct virtio_gpu_object *bo, +int no_cb); int virtio_gpu_panic_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev, uint64_t offset, uint32_t width, uint32_t height,
@@ -491,6 +495,8 @@ void virtio_gpu_add_object_to_restore_list(struct virtio_gpu_device *vgdev,int virtio_gpu_object_restore_all (struct virtio_gpu_device *vgdev); +void virtio_gpu_object_unref_all (struct virtio_gpu_device *vgdev); +/* virtgpu_prime.c */ int virtio_gpu_resource_assign_uuid (struct virtio_gpu_device *vgdev,struct virtio_gpu_object *bo);
drivers/gpu/drm/virtio/virtgpu_kms.c +28 -0
@@ -26,6 +26,8 @@#include <linux/virtio.h> #include <linux/virtio_config.h> #include <linux/virtio_ring.h> +#include <linux/suspend.h> +#include <linux/pm_runtime.h> #include <drm/drm_file.h> #include <drm/drm_managed.h>
@@ -133,6 +135,25 @@ int virtio_gpu_find_vqs(struct virtio_gpu_device *vgdev)return 0 ; } +static int virtio_gpu_pm_notifier (struct notifier_block *nb,unsigned long mode, +void *data) +{ +struct virtio_gpu_device *vgdev =container_of (nb, + struct virtio_gpu_device, + pm_nb); + +if (mode ==PM_HIBERNATION_PREPARE ) { +if (vgdev->has_virgl_3d) { +DRM_ERROR ("S4 not allowed when VIRGL is enabled\n" ); +return notifier_from_errno (-EPERM ); + } + + vgdev->hibernation = true; + } + +return NOTIFY_DONE ; +} +int virtio_gpu_init (struct virtio_device *vdev,struct drm_device *dev) {struct virtio_gpu_device *vgdev;
@@ -269,6 +290,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)wait_event_timeout (vgdev->resp_wq, !vgdev->display_info_pending,5 *HZ ); } + + vgdev->pm_nb.notifier_call = virtio_gpu_pm_notifier; + ret =register_pm_notifier (&vgdev->pm_nb); +if (ret) + goto err_scanouts; +return 0 ; err_scanouts:
@@ -301,6 +328,7 @@ void virtio_gpu_deinit(struct drm_device *dev)virtio_reset_device (vgdev->vdev); vgdev->vdev->config->del_vqs (vgdev->vdev);mutex_destroy (&vgdev->obj_restore_lock); +unregister_pm_notifier (&vgdev->pm_nb); }void virtio_gpu_release (struct drm_device *dev)
drivers/gpu/drm/virtio/virtgpu_object.c +16 -1
@@ -98,7 +98,7 @@ static void virtio_gpu_free_object(struct drm_gem_object *obj)struct virtio_gpu_device *vgdev = bo->base.base.dev->dev_private;if (bo->created) { -virtio_gpu_cmd_unref_resource (vgdev, bo); +virtio_gpu_cmd_unref_resource (vgdev, bo, false);virtio_gpu_notify (vgdev);/* completion handler calls virtio_gpu_cleanup_object() */ return ;
@@ -337,3 +337,18 @@ int virtio_gpu_object_restore_all(struct virtio_gpu_device *vgdev)return ret; } + +void virtio_gpu_object_unref_all (struct virtio_gpu_device *vgdev) +{ +struct virtio_gpu_object *bo, *tmp; + +mutex_lock (&vgdev->obj_restore_lock); +list_for_each_entry_safe (bo, tmp, &vgdev->obj_restore_list, + restore_node) +if (bo->created) { +virtio_gpu_cmd_unref_resource (vgdev, bo, true); +virtio_gpu_notify (vgdev); + } + +mutex_unlock (&vgdev->obj_restore_lock); +}
drivers/gpu/drm/virtio/virtgpu_prime.c +1 -1
@@ -218,7 +218,7 @@ static void virtgpu_dma_buf_free_obj(struct drm_gem_object *obj) }if (bo->created) { -virtio_gpu_cmd_unref_resource (vgdev, bo); +virtio_gpu_cmd_unref_resource (vgdev, bo, false);virtio_gpu_notify (vgdev);return ; }
drivers/gpu/drm/virtio/virtgpu_vq.c +10 -3
@@ -625,14 +625,21 @@ static void virtio_gpu_cmd_unref_cb(struct virtio_gpu_device *vgdev, }void virtio_gpu_cmd_unref_resource (struct virtio_gpu_device *vgdev, -struct virtio_gpu_object *bo) +struct virtio_gpu_object *bo, +int no_cb) {struct virtio_gpu_resource_unref *cmd_p;struct virtio_gpu_vbuffer *vbuf;int ret; - cmd_p =virtio_gpu_alloc_cmd_cb (vgdev, &vbuf,sizeof (*cmd_p), - virtio_gpu_cmd_unref_cb); +if (no_cb) { + cmd_p =virtio_gpu_alloc_cmd_cb (vgdev, &vbuf,sizeof (*cmd_p), +NULL ); + }else { + cmd_p =virtio_gpu_alloc_cmd_cb (vgdev, &vbuf,sizeof (*cmd_p), + virtio_gpu_cmd_unref_cb); + } +memset (cmd_p,0 ,sizeof (*cmd_p)); cmd_p->hdr.type =cpu_to_le32 (VIRTIO_GPU_CMD_RESOURCE_UNREF );
drivers/gpu/drm/virtio/virtgpu_vram.c +1 -1
@@ -18,7 +18,7 @@ static void virtio_gpu_vram_free(struct drm_gem_object *obj)if (unmap)virtio_gpu_cmd_unmap (vgdev, bo); -virtio_gpu_cmd_unref_resource (vgdev, bo); +virtio_gpu_cmd_unref_resource (vgdev, bo, false);virtio_gpu_notify (vgdev);return ; }
Signature · 1 line
2.34.1
Dmitry Osipenko
maintainerRe: [PATCH v7 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
drivers/gpu/drm/virtio/virtgpu_drv.c static int virtgpu_freeze(struct virtio_device *vdev) · line 181
Quoted 8 lines · On 1/7/26 21:27, [email protected] wrote:
drivers/gpu/drm/virtio/virtgpu_drv.c +2
-0@@ -177,6 +177,9 @@ static int virtgpu_freeze(struct virtio_device *vdev)return error; } +if (vgdev->hibernation) +virtio_gpu_object_unref_all (vgdev);
Nit: function name doesn't reflect well what it's doing, I'd name it as virtio_gpu_unref_stored_objs
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerRe: [PATCH v7 3/3] drm/virtio: Add PM notifier to restore objects after hibernation
In reply to [email protected]drivers/gpu/drm/virtio/virtgpu_drv.h struct virtio_gpu_device { · line 264
Quoted 6 lines · On 1/7/26 21:27, [email protected] wrote:
+++
b/drivers/gpu/drm/virtio/virtgpu_drv.hdrivers/gpu/drm/virtio/virtgpu_drv.h +1
-0@@ -261,6 +261,7 @@ struct virtio_gpu_device {bool has_resource_blob;bool has_host_visible;bool has_context_init; +bool hibernation;
Nit: bool hibernated;
Signature · 2 lines
Best regards, Dmitry
Dmitry Osipenko
maintainerQuoted 85 lines · On 1/7/26 21:27, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in S4(hiberation) case.
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances. This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of hiberation/resume process.Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in S3
case can be avoided by skipping virtio-gpu-reset in QEMU (hw/display/virtio-gpu.c). To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be attached to PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set to 'true'. (e.g. -device virtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)v6: Include object backed by imported dmabuf
(Dmitry Osipenko)Not storing virgl objects in the
restore_listas virgl 3D objects are not recoverable.
(Dmitry Osipenko)Change the name 'list',a node in
restore_listto 'restore_node' (Nirmoy Das)Use mutex instead of spinlock when updating
restore_list(Nirmoy Das)Initialize
restore_nodewhenvirtio_gpu_objectis created - this is to check if the node is in the list with 'list_empty' before removing it.Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in
virtgpu_restoreonce virtqueues are re-established. To address this, a 'hibernation' flag is set during thePM_HIBERNATION_PREPAREphase in the notifier. This flag is then used invirtgpu_restoreto detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.v7: Add a helper,
virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernation
Quoted 9 lines
drivers/gpu/drm/virtio/virtgpu_drv.c | 74 +++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 23 ++++++- drivers/gpu/drm/virtio/virtgpu_kms.c | 54 ++++++++++++++-- drivers/gpu/drm/virtio/virtgpu_object.c | 83 ++++++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_prime.c | 43 ++++++++++++- drivers/gpu/drm/virtio/virtgpu_vq.c | 13 +++- drivers/gpu/drm/virtio/virtgpu_vram.c | 4 +- 7 files changed, 280 insertions(+), 14 deletions(-)
Hello Kim,
Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.
Meanwhile there is a kernel bot bug report.
Signature · 2 lines
Best regards, Dmitry
Hi Dmitry,
Quoted 113 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, February 25, 2026 8:06 AM
To: Kim, Dongwon <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Kasireddy, Vivek <[email protected]>
Subject: Re: [PATCH v7 0/3] Virtio-GPU S4 supportQuoted 7 lines · On 1/7/26 21:27, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in
S4(hiberation) case.Quoted 5 lines
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances.
Quoted 2 lines
This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of
hiberation/resume process.
Quoted 7 lines
Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
Quoted 8 lines
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in
S3
Quoted 1 line
case can be avoided by skipping virtio-gpu-reset in QEMU
(
hw/display/virtio-gpu.c).Quoted 1 line
To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be
attached to
Quoted 1 line
PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set
to 'true'.
Quoted 55 lines
(e.g. -devicevirtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)v6: Include object backed by imported dmabuf
(Dmitry Osipenko)Not storing virgl objects in the
restore_listas virgl 3D objects are not recoverable.
(Dmitry Osipenko)Change the name 'list',a node in
restore_listto 'restore_node' (Nirmoy Das)Use mutex instead of spinlock when updating
restore_list(Nirmoy Das)Initialize
restore_nodewhenvirtio_gpu_objectis created - this is to check if the node is in the list with 'list_empty' before removing it.Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in
virtgpu_restoreonce virtqueues are re-established. To address this, a 'hibernation' flag is set during thePM_HIBERNATION_PREPAREphase in the notifier. This flag is then used invirtgpu_restoreto detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.v7: Add a helper,
virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernationQuoted 5 lines
drivers/gpu/drm/virtio/virtgpu_drv.c | 74 +++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 23 ++++++- drivers/gpu/drm/virtio/virtgpu_kms.c | 54 ++++++++++++++-- drivers/gpu/drm/virtio/virtgpu_object.c | 83++++++++++++++++++++++++-
drivers/gpu/drm/virtio/virtgpu_prime.c| 43++++++++++++-
Quoted 4 lines
drivers/gpu/drm/virtio/virtgpu_vq.c | 13 +++- drivers/gpu/drm/virtio/virtgpu_vram.c | 4 +- 7 files changed, 280 insertions(+), 14 deletions(-)
Quoted 7 lines
Hello Kim,
Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.
Meanwhile there is a kernel bot bug report.
Thanks for the comment. No problem. I will wait for your test results then create V8 based on your feedback including what you just left for individual patches (function/variable names..etc.)
I thought I got an error report for the previous version but I will double check.
Quoted 4 lines
--
Best regards,
Dmitry
Dmitry Osipenko
maintainerQuoted 121 lines · Dongwon
Hi Dmitry,
Quoted 119 lines
-----Original Message-----
From: Dmitry Osipenko <[email protected]>
Sent: Wednesday, February 25, 2026 8:06 AM
To: Kim, Dongwon <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Kasireddy, Vivek <[email protected]>
Subject: Re: [PATCH v7 0/3] Virtio-GPU S4 supportQuoted 7 lines · On 1/7/26 21:27, [email protected] wrote:
From: Dongwon Kim <[email protected]>
This patch series implements functions for .freeze and .restore hooks for virtio-gpu driver as well as pm-notifier to handle object restoration in
S4(hiberation) case.Quoted 5 lines
First patch adds
virtgpu_freezeandvirtgpu_restorefunctions. These functions handle the deletion of virtio queues before suspension and their recreation during the restoration process.Second patch implements a mechanism for restoring
virtio_gpu_objectinstances.
Quoted 2 lines
This is necessary because the host (QEMU) deletes all associated resources during the virtio-gpu reset, which occurs as part of
hiberation/resume process.
Quoted 7 lines
Third patch adds pm-notifier to the driver that handles resubmission of virtio-gpu objects to the QEMU once the guest resumes from hibenation.
These changes ensure that the virtio-gpu driver can properly handle hibernation scenarios without resource loss.
Quoted 8 lines
v2: 10ms sleep is added in
virtgpu_freezeto avoid the situation
the driver is locked up during resumption.v3: Plain 10ms delay (v2) is replaced with wait calls which wait until
the virtio queue is empty.
(Dmitry Osipenko)v4: New version of patchset only covers S4 case because loss of resources in
S3
Quoted 1 line
case can be avoided by skipping virtio-gpu-reset in QEMU
(
hw/display/virtio-gpu.c).Quoted 1 line
To skip virtio-gpu-reset (soft-reset), virtio-gpu-pci device should be
attached to
Quoted 1 line
PCIE bus AND a PCIE option, 'x-pcie-pm-no-soft-reset' should added and set
to 'true'.
Quoted 55 lines
(e.g. -devicevirtio-gpu-pci,bus=port,x-pcie-pm-no-soft-reset=true)
v5: Remove
virtio_gpu_objectfrom the restore list before freeing the object
to prevent an use-after-free situation.
(Nirmoy Das)Protect restore list operations with a spinlock (Nirmoy Das)
Move restore list node into
virtio_gpu_bostruct to reduce memory usage (Dmitry Osipenko)Remove unused header -
drm_atomic_helper.h
(Dmitry Osipenko)v6: Include object backed by imported dmabuf
(Dmitry Osipenko)Not storing virgl objects in the
restore_listas virgl 3D objects are not recoverable.
(Dmitry Osipenko)Change the name 'list',a node in
restore_listto 'restore_node' (Nirmoy Das)Use mutex instead of spinlock when updating
restore_list(Nirmoy Das)Initialize
restore_nodewhenvirtio_gpu_objectis created - this is to check if the node is in the list with 'list_empty' before removing it.Restoring objects in the PM notifier is too late, as virtio-gpu message communication begins in
virtgpu_restoreonce virtqueues are re-established. To address this, a 'hibernation' flag is set during thePM_HIBERNATION_PREPAREphase in the notifier. This flag is then used invirtgpu_restoreto detect if the system is resuming from S4, allowing objects to be recovered immediately after virtqueues are reconfigured.v7: Add a helper,
virtio_gpu_add_object_to_restore_list
(Dmitry Osipenko)Unreference all objects before hibernation so they can be removed on the host side, since they will be fully restored anyway. This prevents the situation where host-side hibernation fails (leaving all associated resources still alive) while the virtio-gpu driver still attempts to restore those objects.
(Dmitry Osipenko)Dongwon Kim (3):
drm/virtio: Freeze and restore hooks to support suspend and resume drm/virtio: Add support for saving and restoringvirtio_gpu_objectsdrm/virtio: Add PM notifier to restore objects after hibernationQuoted 5 lines
drivers/gpu/drm/virtio/virtgpu_drv.c | 74 +++++++++++++++++++++- drivers/gpu/drm/virtio/virtgpu_drv.h | 23 ++++++- drivers/gpu/drm/virtio/virtgpu_kms.c | 54 ++++++++++++++-- drivers/gpu/drm/virtio/virtgpu_object.c | 83++++++++++++++++++++++++-
drivers/gpu/drm/virtio/virtgpu_prime.c| 43++++++++++++-
Quoted 4 lines
drivers/gpu/drm/virtio/virtgpu_vq.c | 13 +++- drivers/gpu/drm/virtio/virtgpu_vram.c | 4 +- 7 files changed, 280 insertions(+), 14 deletions(-)Hello Kim,
Want let you know that I've seen the patches, but didn't have enough time to review and test them. Will try to do it sooner. Will leave couple comments for now.
Quoted 7 lines
Quoted 1 line
Meanwhile there is a kernel bot bug report.
Thanks for the comment. No problem. I will wait for your test results then create V8 based on your feedback including what you just left for individual patches (function/variable names..etc.)
I thought I got an error report for the previous version but I will double check.
Thanks for your patience and sending v8, will test it. There other patches in my queue, hopefully it will get untangled sooner.
Signature · 2 lines
Best regards, Dmitry