vdpa: Add resume operation
Subsystems (2)
v5 cover vdpa: Add resume operation
From: Sebastien Boeuf <[email protected]>
This series introduces a new operation for vdpa devices. It allows them to be resumed after they have been suspended. A new feature bit is introduced for devices to advertise their ability to be resumed after they have been suspended. This feature bit is different from the one advertising the ability to be suspended, meaning a device that can be suspended might not have the ability to be resumed.
Even if it is already possible to restore a device that has been suspended, which is very convenient for live migrating virtual machines, there is a major drawback as the device must be fully reset. There is no way to resume a device that has been suspended without having to configure the device again and without having to recreate the IOMMU mappings. This new operation aims at filling this gap by allowing the device to resume processing the virtqueue descriptors without having to reset it. This is particularly useful for performing virtual machine offline migration, also called snapshot/restore, as it allows a virtual machine to resume to a running state after it was paused and a snapshot of the entire system was taken.
Sebastien Boeuf (4):
vdpa: Add resume operation
vhost-vdpa: Introduce RESUME backend feature bit
vhost-vdpa: uAPI to resume the device
vdpa_sim: Implement resume vdpa op
drivers/vdpa/vdpa_sim/vdpa_sim.c | 28 ++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + drivers/vhost/vdpa.c | 34 +++++++++++++++++++++++++++++++- include/linux/vdpa.h | 6 +++++- include/uapi/linux/vhost.h | 8 ++++++++ include/uapi/linux/vhost_types.h | 2 ++ 6 files changed, 77 insertions(+), 2 deletions(-)
Signature · 13 lines
2.34.1 --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 5 208 026.16 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v5 1/4 vdpa: Add resume operation
From: Sebastien Boeuf <[email protected]>
Add a new operation to allow a vDPA device to be resumed after it has been suspended. Trying to resume a device that wasn't suspended will result in a no-op.
This operation is optional. If it's not implemented, the associated backend feature bit will not be exposed. And if the feature bit is not exposed, invoking this operation will return an error.
--- include/linux/vdpa.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
include/linux/vdpa.h +5 -1
@@ -219,7 +219,10 @@ struct vdpa_map_file {* @reset: Reset device * @vdev: vdpa device * Returns integer: success (0) or error (< 0) -* @suspend: Suspend or resume the device (optional) +* @suspend: Suspend the device (optional) +* @vdev: vdpa device +* Returns integer: success (0) or error (< 0) +* @resume: Resume the device (optional) * @vdev: vdpa device * Returns integer: success (0) or error (< 0) * @get_config_size: Get the size of the configuration space includes
@@ -324,6 +327,7 @@ struct vdpa_config_ops {void (*set_status)(struct vdpa_device *vdev,u8 status);int (*reset)(struct vdpa_device *vdev);int (*suspend)(struct vdpa_device *vdev); +int (*resume)(struct vdpa_device *vdev);size_t (*get_config_size)(struct vdpa_device *vdev);void (*get_config)(struct vdpa_device *vdev,unsigned int offset,void *buf,unsigned int len);
Signature · 13 lines
2.34.1 --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 5 208 026.16 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v5 2/4 vhost-vdpa: Introduce RESUME backend feature bit
In reply to [email protected]From: Sebastien Boeuf <[email protected]>
Userspace knows if the device can be resumed or not by checking this feature bit.
It's only exposed if the vdpa driver backend implements the resume() operation callback. Userspace trying to negotiate this feature when it hasn't been exposed will result in an error.
--- drivers/vhost/vdpa.c | 16 +++++++++++++++- include/uapi/linux/vhost_types.h | 2 ++ 2 files changed, 17 insertions(+), 1 deletion(-)
drivers/vhost/vdpa.c +15 -1
@@ -355,6 +355,14 @@ static bool vhost_vdpa_can_suspend(const struct vhost_vdpa *v)return ops->suspend; } +static bool vhost_vdpa_can_resume (const struct vhost_vdpa *v) +{ +struct vdpa_device *vdpa = v->vdpa; +const struct vdpa_config_ops *ops = vdpa->config; + +return ops->resume; +} +static long vhost_vdpa_get_features (struct vhost_vdpa *v,u64 __user *featurep) {struct vdpa_device *vdpa = v->vdpa;
@@ -602,11 +610,15 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,if (copy_from_user (&features, featurep,sizeof (features)))return -EFAULT ;if (features & ~(VHOST_VDPA_BACKEND_FEATURES | -BIT_ULL (VHOST_BACKEND_F_SUSPEND ))) +BIT_ULL (VHOST_BACKEND_F_SUSPEND ) | +BIT_ULL (VHOST_BACKEND_F_RESUME )))return -EOPNOTSUPP ;if ((features &BIT_ULL (VHOST_BACKEND_F_SUSPEND )) && !vhost_vdpa_can_suspend (v))return -EOPNOTSUPP ; +if ((features &BIT_ULL (VHOST_BACKEND_F_RESUME )) && + !vhost_vdpa_can_resume (v)) +return -EOPNOTSUPP ;vhost_set_backend_features (&v->vdev, features);return 0 ; }
@@ -658,6 +670,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep, features =VHOST_VDPA_BACKEND_FEATURES ;if (vhost_vdpa_can_suspend (v)) features |=BIT_ULL (VHOST_BACKEND_F_SUSPEND ); +if (vhost_vdpa_can_resume (v)) + features |=BIT_ULL (VHOST_BACKEND_F_RESUME );if (copy_to_user (featurep, &features,sizeof (features))) r = -EFAULT ;break ;
include/uapi/linux/vhost_types.h +2 -0
@@ -163,5 +163,7 @@ struct vhost_vdpa_iova_range {#define VHOST_BACKEND_F_IOTLB_ASID 0x3/* Device can be suspended */ #define VHOST_BACKEND_F_SUSPEND 0x4 +/* Device can be resumed */ +#define VHOST_BACKEND_F_RESUME 0x5#endif
Signature · 13 lines
2.34.1 --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 5 208 026.16 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v5 3/4 vhost-vdpa: uAPI to resume the device
In reply to [email protected]From: Sebastien Boeuf <[email protected]>
This new ioctl adds support for resuming the device from userspace.
This is required when trying to restore the device in a functioning state after it's been suspended. It is already possible to reset a suspended device, but that means the device must be reconfigured and all the IOMMU/IOTLB mappings must be recreated. This new operation allows the device to be resumed without going through a full reset.
This is particularly useful when trying to perform offline migration of a virtual machine (also known as snapshot/restore) as it allows the VMM to resume the virtual machine back to a running state after the snapshot is performed.
--- drivers/vhost/vdpa.c | 18 ++++++++++++++++++ include/uapi/linux/vhost.h | 8 ++++++++ 2 files changed, 26 insertions(+)
drivers/vhost/vdpa.c +18 -0
@@ -502,6 +502,21 @@ static long vhost_vdpa_suspend(struct vhost_vdpa *v)return ops->suspend (vdpa); } +/* After a successful return of this ioctl the device resumes processing +* virtqueue descriptors. The device becomes fully operational the same way it +* was before it was suspended. +*/ +static long vhost_vdpa_resume (struct vhost_vdpa *v) +{ +struct vdpa_device *vdpa = v->vdpa; +const struct vdpa_config_ops *ops = vdpa->config; + +if (!ops->resume) +return -EOPNOTSUPP ; + +return ops->resume (vdpa); +} +static long vhost_vdpa_vring_ioctl (struct vhost_vdpa *v,unsigned int cmd,void __user *argp) {
@@ -687,6 +702,9 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,case VHOST_VDPA_SUSPEND : r =vhost_vdpa_suspend (v);break ; +case VHOST_VDPA_RESUME : + r =vhost_vdpa_resume (v); +break ;default : r =vhost_dev_ioctl (&v->vdev, cmd, argp);if (r == -ENOIOCTLCMD )
include/uapi/linux/vhost.h +8 -0
@@ -180,4 +180,12 @@*/ #define VHOST_VDPA_SUSPEND _IO(VHOST_VIRTIO, 0x7D) +/* Resume a device so it can resume processing virtqueue requests +* +* After the return of this ioctl the device will have restored all the +* necessary states and it is fully operational to continue processing the +* virtqueue descriptors. +*/ +#define VHOST_VDPA_RESUME _IO(VHOST_VIRTIO, 0x7E) +#endif
Signature · 13 lines
2.34.1 --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 5 208 026.16 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v5 4/4 vdpa_sim: Implement resume vdpa op
In reply to [email protected]From: Sebastien Boeuf <[email protected]>
Implement resume operation for vdpa_sim devices, so vhost-vdpa will offer that backend feature and userspace can effectively resume the device.
--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 28 ++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + 2 files changed, 29 insertions(+)
drivers/vdpa/vdpa_sim/vdpa_sim.c +28 -0
@@ -357,6 +357,11 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +if (!vdpasim->running) { + vdpasim->pending_kick = true; +return ; + } +if (vq->ready)schedule_work (&vdpasim->work); }
@@ -527,6 +532,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa)return 0 ; } +static int vdpasim_resume (struct vdpa_device *vdpa) +{ +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); +int i; + +spin_lock (&vdpasim->lock); + vdpasim->running = true; + +if (vdpasim->pending_kick) { +/* Process pending descriptors */ +for (i =0 ; i < vdpasim->dev_attr.nvqs; ++i) +vdpasim_kick_vq (vdpa, i); + + vdpasim->pending_kick = false; + } + +spin_unlock (&vdpasim->lock); + +return 0 ; +} +static size_t vdpasim_get_config_size (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);
@@ -717,6 +743,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,
@@ -750,6 +777,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,
drivers/vdpa/vdpa_sim/vdpa_sim.h +1 -0
@@ -67,6 +67,7 @@ struct vdpasim {u64 features;u32 groups;bool running; +bool pending_kick;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; };
Signature · 13 lines
2.34.1 --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 5 208 026.16 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Re: [PATCH v5 4/4] vdpa_sim: Implement resume vdpa op
Quoted 18 lines · On Wed, Oct 26, 2022 at 11:09 PM <[email protected]> wrote:
From: Sebastien Boeuf <[email protected]>
Implement resume operation for
vdpa_simdevices, so vhost-vdpa will offer that backend feature and userspace can effectively resume the device.Signed-off-by Sebastien Boeuf <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 28 ++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + 2 files changed, 29 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-0
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) · line 362
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-0@@ -357,6 +357,11 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +if (!vdpasim->running) { + vdpasim->pending_kick = true; +return ;
I think we may hit here when the driver kicks vq before DRIVER_OK. Do we need to check device status in this case and resume?
Thanks
Quoted 71 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +25
-0+ } +if (vq->ready)schedule_work (&vdpasim->work); }@@ -527,6 +532,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa)return 0 ; } +static int vdpasim_resume (struct vdpa_device *vdpa) +{ +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); +int i; + +spin_lock (&vdpasim->lock); + vdpasim->running = true; + +if (vdpasim->pending_kick) { +/* Process pending descriptors */ +for (i =0 ; i < vdpasim->dev_attr.nvqs; ++i) +vdpasim_kick_vq (vdpa, i); + + vdpasim->pending_kick = false; + } + +spin_unlock (&vdpasim->lock); + +return 0 ; +} +static size_t vdpasim_get_config_size (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);@@ -717,6 +743,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,@@ -750,6 +777,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,drivers/vdpa/vdpa_sim/vdpa_sim.h +1
-1@@ -67,6 +67,7 @@ struct vdpasim {u64 features;u32 groups;bool running; +bool pending_kick;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; }; --2.34.1
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199R.C.S. NANTERRE
Capital: 5 208 026.16 Euros
Quoted 5 lines
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Michael S. Tsirkin
maintainerRe: [PATCH v5 4/4] vdpa_sim: Implement resume vdpa op
Quoted 22 lines · Jason Wang
Quoted 21 lines · On Wed, Oct 26, 2022 at 11:09 PM <[email protected]> wrote:
From: Sebastien Boeuf <[email protected]>
Implement resume operation for
vdpa_simdevices, so vhost-vdpa will offer that backend feature and userspace can effectively resume the device.Signed-off-by Sebastien Boeuf <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 28 ++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + 2 files changed, 29 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-0@@ -357,6 +357,11 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];
Quoted 8 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) · line 362
Quoted 3 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-0+if (!vdpasim->running) { + vdpasim->pending_kick = true; +return ;I think we may hit here when the driver kicks vq before
DRIVER_OK. Do we need to check device status in this case and resume?Thanks
Sebastien did you forget to reply here?
Quoted 71 lines
Quoted 71 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +25
-0+ } +if (vq->ready)schedule_work (&vdpasim->work); }@@ -527,6 +532,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa)return 0 ; } +static int vdpasim_resume (struct vdpa_device *vdpa) +{ +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); +int i; + +spin_lock (&vdpasim->lock); + vdpasim->running = true; + +if (vdpasim->pending_kick) { +/* Process pending descriptors */ +for (i =0 ; i < vdpasim->dev_attr.nvqs; ++i) +vdpasim_kick_vq (vdpa, i); + + vdpasim->pending_kick = false; + } + +spin_unlock (&vdpasim->lock); + +return 0 ; +} +static size_t vdpasim_get_config_size (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);@@ -717,6 +743,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,@@ -750,6 +777,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,drivers/vdpa/vdpa_sim/vdpa_sim.h +1
-1@@ -67,6 +67,7 @@ struct vdpasim {u64 features;u32 groups;bool running; +bool pending_kick;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; }; --2.34.1
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199R.C.S. NANTERRE
Capital: 5 208 026.16 Euros
Quoted 5 lines
Quoted 5 lines
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH v5 4/4] vdpa_sim: Implement resume vdpa op
Quoted 36 lines · Michael S. Tsirkin
Quoted 35 lines · Jason Wang
Quoted 22 lines · On Wed, Oct 26, 2022 at 11:09 PM <[email protected]> wrote:
From: Sebastien Boeuf <[email protected]>
Implement resume operation for
vdpa_simdevices, so vhost-vdpa will
offer that backend feature and userspace can effectively resume the
device.Signed-off-by Sebastien Boeuf <[email protected]>---
drivers/vdpa/vdpa_sim/vdpa_sim.c | 28
++++++++++++++++++++++++++++
drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 +2 files changed, 29 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-0drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_kick_vq(struct · line 363
Quoted 7 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-0@@ -357,6 +357,11 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) struct vdpasim *vdpasim = vdpa_to_sim(vdpa); struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; + if (!vdpasim->running) { + vdpasim->pending_kick = true; + return;I think we may hit here when the driver kicks vq before
DRIVER_OK. Do
we need to check device status in this case and resume?
Quoted 3 lines
Quoted 1 line
Thanks
Sebastien did you forget to reply here?
Ah yes sorry I got carried away with other things and forgot about this. And then I was on vacation.
I'll look at it this week.
Thanks,
Sebastien
Quoted 77 lines
Quoted 77 lines
Quoted 77 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +2
-0+ } + if (vq->ready) schedule_work(&vdpasim->work);}
drivers/vdpa/vdpa_sim/vdpa_sim.c +21
-0@@ -527,6 +532,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa) return 0; } +static int vdpasim_resume (struct vdpa_device *vdpa) +{ + struct vdpasim *vdpasim = vdpa_to_sim(vdpa); + int i; + + spin_lock(&vdpasim->lock); + vdpasim->running = true; + + if (vdpasim->pending_kick) { + /* Process pending descriptors */ + for (i = 0; i < vdpasim->dev_attr.nvqs; ++i) + vdpasim_kick_vq(vdpa, i); + + vdpasim->pending_kick = false; + } + + spin_unlock(&vdpasim->lock); + + return 0; +} + static size_t vdpasim_get_config_size(struct vdpa_device *vdpa) {struct vdpasim *vdpasim =
vdpa_to_sim(vdpa);drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -717,6 +743,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config,.set_config =
vdpasim_set_config,drivers/vdpa/vdpa_sim/vdpa_sim.c +1
-0@@ -750,6 +777,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config,.set_config =
vdpasim_set_config,drivers/vdpa/vdpa_sim/vdpa_sim.h +1
-1@@ -67,6 +67,7 @@ struct vdpasim { u64 features; u32 groups; bool running; + bool pending_kick; /* spinlock to synchronize iommu table */ spinlock_t iommu_lock; }; --2.34.1
-----------------------------------------------------------------
----
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199R.C.S. NANTERRE
Capital: 5 208 026.16 Euros
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199 R.C.S. NANTERRE
Capital: 5 208 026.16 Euros
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
A message that is not in the archive (sent off-list or not yet received): BY5PR11MB4401D90E9853727F06B309CFEAF79@BY5PR11MB4401.namprd11.prod.outlook.com
Re: [PATCH v5 4/4] vdpa_sim: Implement resume vdpa op
Quoted 11 lines · Sebastien
Hi Jason,
Are you suggesting we do something like the following:
if ((!vdpasim->running) && (
vdpasim->status==DRIVER_OK)) {vdpasim->pending_kick= true;
return;
}?
Yes.
Thanks
Quoted 125 lines
Thanks,
Sebastien
________________________________
From: Jason Wang <[email protected]>
Sent: Monday, November 7, 2022 8:43 AM
To: Boeuf, Sebastien <[email protected]>
Cc: [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>; [email protected] <[email protected]>
Subject: Re: [PATCH v5 4/4]vdpa_sim: Implement resume vdpa opQuoted 18 lines · On Wed, Oct 26, 2022 at 11:09 PM <[email protected]> wrote:
From: Sebastien Boeuf <[email protected]>
Implement resume operation for
vdpa_simdevices, so vhost-vdpa will offer that backend feature and userspace can effectively resume the device.Signed-off-by Sebastien Boeuf <[email protected]>--- drivers/vdpa/vdpa_sim/vdpa_sim.c | 28 ++++++++++++++++++++++++++++ drivers/vdpa/vdpa_sim/vdpa_sim.h | 1 + 2 files changed, 29 insertions(+)drivers/vdpa/vdpa_sim/vdpa_sim.c +0
-0drivers/vdpa/vdpa_sim/vdpa_sim.c static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx) · line 362
Quoted 6 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +3
-0@@ -357,6 +357,11 @@ static void vdpasim_kick_vq(struct vdpa_device *vdpa, u16 idx)struct vdpasim *vdpasim =vdpa_to_sim (vdpa);struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx]; +if (!vdpasim->running) { + vdpasim->pending_kick = true; +return ;I think we may hit here when the driver kicks vq before
DRIVER_OK. Do we need to check device status in this case and resume?Thanks
Quoted 71 lines
drivers/vdpa/vdpa_sim/vdpa_sim.c +25
-0+ } +if (vq->ready)schedule_work (&vdpasim->work); }@@ -527,6 +532,27 @@ static int vdpasim_suspend(struct vdpa_device *vdpa)return 0 ; } +static int vdpasim_resume (struct vdpa_device *vdpa) +{ +struct vdpasim *vdpasim =vdpa_to_sim (vdpa); +int i; + +spin_lock (&vdpasim->lock); + vdpasim->running = true; + +if (vdpasim->pending_kick) { +/* Process pending descriptors */ +for (i =0 ; i < vdpasim->dev_attr.nvqs; ++i) +vdpasim_kick_vq (vdpa, i); + + vdpasim->pending_kick = false; + } + +spin_unlock (&vdpasim->lock); + +return 0 ; +} +static size_t vdpasim_get_config_size (struct vdpa_device *vdpa) {struct vdpasim *vdpasim =vdpa_to_sim (vdpa);@@ -717,6 +743,7 @@ static const struct vdpa_config_ops vdpasim_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,@@ -750,6 +777,7 @@ static const struct vdpa_config_ops vdpasim_batch_config_ops = { .set_status = vdpasim_set_status, .reset = vdpasim_reset, .suspend = vdpasim_suspend, + .resume = vdpasim_resume, .get_config_size = vdpasim_get_config_size, .get_config = vdpasim_get_config, .set_config = vdpasim_set_config,drivers/vdpa/vdpa_sim/vdpa_sim.h +1
-1@@ -67,6 +67,7 @@ struct vdpasim {u64 features;u32 groups;bool running; +bool pending_kick;/* spinlock to synchronize iommu table */ spinlock_t iommu_lock; }; --2.34.1
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199R.C.S. NANTERRE
Capital: 5 208 026.16 EurosQuoted 5 lines
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
---------------------------------------------------------------------
Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France
Registration Number: 302 456 199R.C.S. NANTERRE
Capital: 5 208 026.16 Euros
Quoted 4 lines
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization