virtio-balloon: always indicate S_DONE when migration fails
PATCH v2 1/1 virtio-balloon: always indicate S_DONE when migration fails
If something goes wrong during precopy, before stopping the VM, we will never send a S_DONE indication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).
Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat /proc/meminfo) that there is basically no free memory left.
While at it, add similar locking to virtio_balloon_free_page_done() as done in virtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.
There is nothing to do in the PRECOPY_NOTIFY_COMPLETE case. Add some comments regarding S_DONE handling.
--- hw/virtio/virtio-balloon.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
hw/virtio/virtio-balloon.c +20 -6
@@ -628,8 +628,13 @@ static void virtio_balloon_free_page_done(VirtIOBalloon *s) {VirtIODevice *vdev =VIRTIO_DEVICE (s); - s->free_page_report_status =FREE_PAGE_REPORT_S_DONE ; -virtio_notify_config (vdev); +if (s->free_page_report_status !=FREE_PAGE_REPORT_S_DONE ) { +/* See virtio_balloon_free_page_stop() */ +qemu_mutex_lock (&s->free_page_lock); + s->free_page_report_status =FREE_PAGE_REPORT_S_DONE ; +qemu_mutex_unlock (&s->free_page_lock); +virtio_notify_config (vdev); + } } static int
@@ -653,17 +658,26 @@ virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)case PRECOPY_NOTIFY_SETUP :precopy_enable_free_page_optimization ();break ; -case PRECOPY_NOTIFY_COMPLETE : -case PRECOPY_NOTIFY_CLEANUP :case PRECOPY_NOTIFY_BEFORE_BITMAP_SYNC :virtio_balloon_free_page_stop (dev);break ;case PRECOPY_NOTIFY_AFTER_BITMAP_SYNC :if (vdev->vm_running) {virtio_balloon_free_page_start (dev); - }else { -virtio_balloon_free_page_done (dev); +break ; } +/* +* Set S_DONE before migrating the vmstate, so the guest will reuse +* all hinted pages once running on the destination. Fall through. +*/ +case PRECOPY_NOTIFY_CLEANUP : +/* +* Especially, if something goes wrong during precopy or if migration +* is canceled, we have to properly communicate S_DONE to the VM. +*/ +virtio_balloon_free_page_done (dev); +break ; +case PRECOPY_NOTIFY_COMPLETE :break ;default :virtio_error (vdev,"%s: %d reason unknown" , __func__, pnd->reason);
Signature · 6 lines
2.26.2 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 14 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.
Quoted 8 lines
There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
IIUC this is superceded by Alexander's patches right? If not pls rebase ...
Quoted 57 lines
--- hw/virtio/virtio-balloon.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)hw/virtio/virtio-balloon.c +20
-6@@ -628,8 +628,13 @@ static void virtio_balloon_free_page_done(VirtIOBalloon *s) {VirtIODevice *vdev =VIRTIO_DEVICE (s); - s->free_page_report_status =FREE_PAGE_REPORT_S_DONE ; -virtio_notify_config (vdev); +if (s->free_page_report_status !=FREE_PAGE_REPORT_S_DONE ) { +/* See virtio_balloon_free_page_stop() */ +qemu_mutex_lock (&s->free_page_lock); + s->free_page_report_status =FREE_PAGE_REPORT_S_DONE ; +qemu_mutex_unlock (&s->free_page_lock); +virtio_notify_config (vdev); + } } static int@@ -653,17 +658,26 @@ virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)case PRECOPY_NOTIFY_SETUP :precopy_enable_free_page_optimization ();break ; -case PRECOPY_NOTIFY_COMPLETE : -case PRECOPY_NOTIFY_CLEANUP :case PRECOPY_NOTIFY_BEFORE_BITMAP_SYNC :virtio_balloon_free_page_stop (dev);break ;case PRECOPY_NOTIFY_AFTER_BITMAP_SYNC :if (vdev->vm_running) {virtio_balloon_free_page_start (dev); - }else { -virtio_balloon_free_page_done (dev); +break ; } +/* +* Set S_DONE before migrating the vmstate, so the guest will reuse +* all hinted pages once running on the destination. Fall through. +*/ +case PRECOPY_NOTIFY_CLEANUP : +/* +* Especially, if something goes wrong during precopy or if migration +* is canceled, we have to properly communicate S_DONE to the VM. +*/ +virtio_balloon_free_page_done (dev); +break ; +case PRECOPY_NOTIFY_COMPLETE :break ;default :virtio_error (vdev,"%s: %d reason unknown" , __func__, pnd->reason);Signature · 1 line
2.26.2
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 18 lines · Michael S. Tsirkin
Quoted 17 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.
Quoted 7 lines
Quoted 5 lines
Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>IIUC this is superceded by Alexander's patches right?
Not that I know ... @Alex?
Quoted 2 lines
If not pls rebase ...
Signature · 8 lines
Thanks, David / dhildenb --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 25 lines · David Hildenbrand
Quoted 24 lines · Michael S. Tsirkin
Quoted 22 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 4 lines
Quoted 1 line
IIUC this is superceded by Alexander's patches right?
Not that I know ... @Alex?
Okay, I'm confused, that patch is already upstream (via your tree)?
dd8eeb9671fc ("virtio-balloon: always indicate S_DONE when migration fails")
Did you stumble over this mail by mistake again?
Signature · 8 lines
Thanks, David / dhildenb --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 29 lines · David Hildenbrand
Quoted 27 lines · David Hildenbrand
Quoted 25 lines · Michael S. Tsirkin
Quoted 14 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.Quoted 8 lines
There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>IIUC this is superceded by Alexander's patches right?
Quoted 8 lines
Quoted 2 lines
Not that I know ... @Alex?
Okay, I'm confused, that patch is already upstream (via your tree)?
dd8eeb9671fc("virtio-balloon: always indicateS_DONEwhen migration fails")Did you stumble over this mail by mistake again?
I agree. David's patches should be upstream, and if they aren't they should be applied before mine. Mine are meant to be a follow-on as I end the set with the "report"->"hint" rename for several fields. The idea is the fixes go first and the rename comes last.
Thanks.
- Alex
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 25 lines · David Hildenbrand
Quoted 24 lines · Michael S. Tsirkin
Quoted 22 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 6 lines
Quoted 1 line
IIUC this is superceded by Alexander's patches right?
Not that I know ... @Alex?
Quoted 2 lines
If not pls rebase ...
OK then I guess I was confused. This is older so I guess I should have applied this and asked Alex to rebase his patches, but I did the reverse.., Sorry about that. Could you rebase on top of the pci tree pls?
Thanks and sorry for messing up.
Quoted 6 lines
Signature · 4 lines
-- Thanks, David / dhildenb
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 28 lines · David Hildenbrand
Quoted 27 lines · David Hildenbrand
Quoted 25 lines · Michael S. Tsirkin
Quoted 14 lines · David Hildenbrand
If something goes wrong during precopy, before stopping the VM, we will never send a
S_DONEindication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat/proc/meminfo) that there is basically no free memory left.While at it, add similar locking to
virtio_balloon_free_page_done()as done invirtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.Quoted 8 lines
There is nothing to do in the
PRECOPY_NOTIFY_COMPLETEcase. Add some comments regardingS_DONEhandling.Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")Reviewed-by Alexander Duyck <[email protected]>Cc Wei Wang <[email protected]>Cc Alexander Duyck <[email protected]>Signed-off-by David Hildenbrand <[email protected]>IIUC this is superceded by Alexander's patches right?
Quoted 13 lines
Quoted 2 lines
Not that I know ... @Alex?
Okay, I'm confused, that patch is already upstream (via your tree)?
dd8eeb9671fc("virtio-balloon: always indicateS_DONEwhen migration fails")Did you stumble over this mail by mistake again?
Signature · 3 lines
Thanks, David / dhildenb
Oh. I guess that's what happened. I saw the code in the tree and thought it came in from Alex's patch.
Sorry about the noise.
Signature · 6 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]