vhost-user-fs: Internal migration
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest4 patches 88 repliesIn review
Changes in v1 from the cover letter
- Patch 1 added
- Patch 2: Interface is different, now uses a pipe instead of shared memory (as suggested by Stefan); also, this is now a generic vhost-user interface, and not just for vhost-user-fs
- Patches 3 and 4: Because this is now supposed to be a generic migration method for vhost-user back-ends, most of the migration code has been moved from vhost-user-fs.c to vhost.c so it can be shared between different back-ends. The vhost-user-fs code is now a rather thin wrapper around the common code.
- Note also (as suggested by Anton) that the back-end’s migration state is now in a subsection, and that it is technically optional. “Technically” means that with this series, it is always used (unless the back-end doesn’t support migration, in which case migration is just blocked), but Anton’s series for external migration would make it optional. (I.e., the subsection would be skipped for external migration, and mandatorily included for internal migration.)
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | vhost: Re-enable vrings after setting featureshw/ include/ | +23 | 11 replies | |
| 2/4 | vhost-user: Interface for migration state transferhw/ hw/ include/ +1 more | +287 | 52 replies | |
| 3/4 | vhost: Add high-level state save/load functionshw/ include/ | +231 | 3 replies | |
| 4/4 | vhost-user-fs: Implement internal migrationhw/ | +100 | 0 replies |
Reviews trailers per patch
None of the 4 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Parav Pandit | Michael S. Tsirkin |
|---|---|---|
| Series | A-b | R-b |
| 1/4 | ||
| 2/4 | ||
| 3/4 | ||
| 4/4 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH 0/4] vhost-user-fs: Internal migration
RFC:
https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg04263.html
Hi,
Patch 2 of this series adds new vhost methods (only for vhost-user at this point) for transferring the back-end’s internal state to/from qemu during migration, so that this state can be stored in the migration stream. (This is what we call “internal migration”, because the state is internally available to qemu; this is in contrast to “external migration”, which Anton is working on, where the back-end’s state is handled by the back-end itself without involving qemu.)
For this, the state is handled as a binary blob by qemu, and it is transferred over a pipe that is established via a new vhost method.
Patch 3 adds two high-level helper functions to (A) fetch any vhost back-end’s internal state and store it in a migration stream (a QEMUFile), and (B) load such state from a migrations stream and send it to a vhost back-end. These build on the low-level interface introduced in patch 2.
Patch 4 then uses these functions to implement internal migration for vhost-user-fs. Note that this of course depends on support in the back-end (virtiofsd), which is not yet ready.
Finally, patch 1 fixes a bug around migrating vhost-user devices: To enable/disable logging[1], the VHOST_F_LOG_ALL feature must be set/cleared, via the SET_FEATURES call. Another, technically unrelated, feature exists, VHOST_USER_F_PROTOCOL_FEATURES, which indicates support for vhost-user protocol features. Naturally, qemu wants to keep that other feature enabled, so it will set it (when possible) in every SET_FEATURES call. However, a side effect of setting VHOST_USER_F_PROTOCOL_FEATURES is that all vrings are disabled. This causes any enabling (done at the start of migration) or disabling (done on the source after a cancelled/failed migration) of logging to make the back-end hang. Without patch 1, therefore, starting a migration will have any vhost-user back-end that supports both VHOST_F_LOG_ALL and VHOST_USER_F_PROTOCOL_FEATURES immediately hang completely, and unless execution is transferred to the destination, it will continue to hang.
[1] Logging here means logging writes to guest memory pages in a dirty bitmap so that these dirty pages are flushed to the destination. qemu cannot monitor the back-end’s writes to guest memory, so the back-end has to do so itself, and log its writes in a dirty bitmap shared with qemu.
Changes in v1 compared to the RFC:
- Patch 1 added
- Patch 2: Interface is different, now uses a pipe instead of shared memory (as suggested by Stefan); also, this is now a generic vhost-user interface, and not just for vhost-user-fs
- Patches 3 and 4: Because this is now supposed to be a generic migration method for vhost-user back-ends, most of the migration code has been moved from vhost-user-fs.c to vhost.c so it can be shared between different back-ends. The vhost-user-fs code is now a rather thin wrapper around the common code.
- Note also (as suggested by Anton) that the back-end’s migration state is now in a subsection, and that it is technically optional. “Technically” means that with this series, it is always used (unless the back-end doesn’t support migration, in which case migration is just blocked), but Anton’s series for external migration would make it optional. (I.e., the subsection would be skipped for external migration, and mandatorily included for internal migration.)
Hanna Czenczek (4):
vhost: Re-enable vrings after setting features
vhost-user: Interface for migration state transfer
vhost: Add high-level state save/load functions
vhost-user-fs: Implement internal migration
include/hw/virtio/vhost-backend.h | 24 +++ include/hw/virtio/vhost.h | 124 +++++++++++++++ hw/virtio/vhost-user-fs.c | 101 +++++++++++- hw/virtio/vhost-user.c | 147 ++++++++++++++++++ hw/virtio/vhost.c | 246 ++++++++++++++++++++++++++++++ 5 files changed, 641 insertions(+), 1 deletion(-)
Signature · 1 line
2.39.1