virtio-balloon: fixes for PartialBalloonedPage
v2 cover virtio-balloon: fixes for PartialBalloonedPage
Some v4.1 (also v4.0 stable) fixes for virtio-balloon that only trigger when the pagesize is > BALLOON_PAGE_SIZE (4k).
v1 -> v2:
- Added "virtio-balloon: fix memory leak on unrealize()"
- Added "virtio-balloon: reset pbp on device resets"
David Hildenbrand (3):
virtio-balloon: fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE
virtio-balloon: fix memory leak on unrealize()
virtio-balloon: reset pbp on device resets
hw/virtio/virtio-balloon.c | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-)
Signature · 1 line
2.21.0
PATCH v2 1/3 virtio-balloon: fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE
We are using the wrong functions to set/clear bits, effectively touching multiple bits, writing out of range of the bitmap, resulting in memory corruptions. We have to use set_bit()/clear_bit() instead.
Can easily be reproduced by starting a qemu guest on hugetlbfs memory, inflating the balloon. QEMU crashes. This never could have worked properly - especially, also pages would have been discarded when the first sub-page would be inflated (the whole bitmap would be set).
While testing I realized, that on hugetlbfs it is pretty much impossible to discard a page - the guest just frees the 4k sub-pages in random order most of the time. I was only able to discard a hugepage a handful of times - so I hope that now works correctly.
host page size")
with inflates & deflates")
--- hw/virtio/virtio-balloon.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
hw/virtio/virtio-balloon.c +4 -6
@@ -94,9 +94,8 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, balloon->pbp->base = host_page_base; } -bitmap_set (balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , - subpages); +set_bit ((ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , + balloon->pbp->bitmap);if (bitmap_full (balloon->pbp->bitmap, subpages)) {/* We've accumulated a full host page, we can actually discard
@@ -140,9 +139,8 @@ static void balloon_deflate_page(VirtIOBalloon *balloon,* for a guest to do this in practice, but handle it anyway, * since getting it wrong could mean discarding memory the * guest is still using. */ -bitmap_clear (balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , - subpages); +clear_bit ((ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , + balloon->pbp->bitmap);if (bitmap_empty (balloon->pbp->bitmap, subpages)) {g_free (balloon->pbp);
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH-for-4.1 v2 1/3] virtio-balloon: fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE
Quoted 16 lines · David Hildenbrand
We are using the wrong functions to set/clear bits, effectively touching multiple bits, writing out of range of the bitmap, resulting in memory corruptions. We have to use set_bit()/clear_bit() instead.
Can easily be reproduced by starting a qemu guest on hugetlbfs memory, inflating the balloon. QEMU crashes. This never could have worked properly - especially, also pages would have been discarded when the first sub-page would be inflated (the whole bitmap would be set).
While testing I realized, that on hugetlbfs it is pretty much impossible to discard a page - the guest just frees the 4k sub-pages in random order most of the time. I was only able to discard a hugepage a handful of times - so I hope that now works correctly.
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Quoted 8 lines
Fixes b27b32391404 ("virtio-balloon: Fix possible guest memory corruptionwith inflates & deflates")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Ahem. You can pass me the brown paper bag now.
Quoted 32 lines
--- hw/virtio/virtio-balloon.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)hw/virtio/virtio-balloon.c +4
-6@@ -94,9 +94,8 @@ static void balloon_inflate_page(VirtIOBalloon *balloon, balloon->pbp->base = host_page_base; } -bitmap_set (balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , - subpages); +set_bit ((ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , + balloon->pbp->bitmap);if (bitmap_full (balloon->pbp->bitmap, subpages)) {/* We've accumulated a full host page, we can actually discard @@ -140,9 +139,8 @@ static void balloon_deflate_page(VirtIOBalloon *balloon,* for a guest to do this in practice, but handle it anyway, * since getting it wrong could mean discarding memory the * guest is still using. */ -bitmap_clear (balloon->pbp->bitmap, - (ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , - subpages); +clear_bit ((ram_offset - balloon->pbp->base) /BALLOON_PAGE_SIZE , + balloon->pbp->bitmap);if (bitmap_empty (balloon->pbp->bitmap, subpages)) {g_free (balloon->pbp);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [PATCH-for-4.1 v2 1/3] virtio-balloon: fix QEMU crashes on pagesize > BALLOON_PAGE_SIZE
Quoted 26 lines · David Gibson
Quoted 16 lines · David Hildenbrand
We are using the wrong functions to set/clear bits, effectively touching multiple bits, writing out of range of the bitmap, resulting in memory corruptions. We have to use set_bit()/clear_bit() instead.
Can easily be reproduced by starting a qemu guest on hugetlbfs memory, inflating the balloon. QEMU crashes. This never could have worked properly - especially, also pages would have been discarded when the first sub-page would be inflated (the whole bitmap would be set).
While testing I realized, that on hugetlbfs it is pretty much impossible to discard a page - the guest just frees the 4k sub-pages in random order most of the time. I was only able to discard a hugepage a handful of times - so I hope that now works correctly.
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Quoted 8 lines
Fixes b27b32391404 ("virtio-balloon: Fix possible guest memory corruptionwith inflates & deflates")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 2 lines
Ahem. You can pass me the brown paper bag now.
No worries, BUGs are inevitable.
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 2/3 virtio-balloon: fix memory leak on unrealize()
In reply to David HildenbrandWe could have tracking data for a pbp (PartiallyBalloonedPage) allocated. Let's free it.
host page size")
--- hw/virtio/virtio-balloon.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
hw/virtio/virtio-balloon.c +10 -6
@@ -40,6 +40,12 @@ struct PartiallyBalloonedPage {unsigned long bitmap[]; }; +static void virtio_balloon_reset_pbp (VirtIOBalloon *balloon) +{ +g_free (balloon->pbp); + balloon->pbp =NULL ; +} +static void balloon_inflate_page (VirtIOBalloon *balloon,MemoryRegion *mr,hwaddr offset) {
@@ -82,8 +88,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon,/* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); }if (!balloon->pbp) {
@@ -106,8 +111,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon,* has already reported them, and failing to discard a balloon * page is not fatal */ -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); } }
@@ -143,8 +147,7 @@ static void balloon_deflate_page(VirtIOBalloon *balloon, balloon->pbp->bitmap);if (bitmap_empty (balloon->pbp->bitmap, subpages)) { -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); } }
@@ -831,6 +834,7 @@ static void virtio_balloon_device_unrealize(DeviceState *dev, Error **errp)virtio_balloon_free_page_stop (s);precopy_remove_notifier (&s->free_page_report_notify); } +virtio_balloon_reset_pbp (s);balloon_stats_destroy_timer (s);qemu_remove_balloon_handler (s);virtio_cleanup (vdev);
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH-for-4.1 v2 2/3] virtio-balloon: fix memory leak on unrealize()
Quoted 11 lines · David Hildenbrand
We could have tracking data for a pbp (PartiallyBalloonedPage) allocated. Let's free it.
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 59 lines
--- hw/virtio/virtio-balloon.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)hw/virtio/virtio-balloon.c +10
-6@@ -40,6 +40,12 @@ struct PartiallyBalloonedPage {unsigned long bitmap[]; }; +static void virtio_balloon_reset_pbp (VirtIOBalloon *balloon) +{ +g_free (balloon->pbp); + balloon->pbp =NULL ; +} +static void balloon_inflate_page (VirtIOBalloon *balloon,MemoryRegion *mr,hwaddr offset) {@@ -82,8 +88,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon,/* We've partially ballooned part of a host page, but now * we're trying to balloon part of a different one. Too hard, * give up on the old partial page */ -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); }if (!balloon->pbp) {@@ -106,8 +111,7 @@ static void balloon_inflate_page(VirtIOBalloon *balloon,* has already reported them, and failing to discard a balloon * page is not fatal */ -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); } }@@ -143,8 +147,7 @@ static void balloon_deflate_page(VirtIOBalloon *balloon, balloon->pbp->bitmap);if (bitmap_empty (balloon->pbp->bitmap, subpages)) { -g_free (balloon->pbp); - balloon->pbp =NULL ; +virtio_balloon_reset_pbp (balloon); } }@@ -831,6 +834,7 @@ static void virtio_balloon_device_unrealize(DeviceState *dev, Error **errp)virtio_balloon_free_page_stop (s);precopy_remove_notifier (&s->free_page_report_notify); } +virtio_balloon_reset_pbp (s);balloon_stats_destroy_timer (s);qemu_remove_balloon_handler (s);virtio_cleanup (vdev);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
PATCH v2 3/3 virtio-balloon: reset pbp on device resets
In reply to David HildenbrandWhen a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
host page size")
--- hw/virtio/virtio-balloon.c | 1 + 1 file changed, 1 insertion(+)
hw/virtio/virtio-balloon.c +1 -0
@@ -847,6 +847,7 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)if (virtio_balloon_free_page_support (s)) {virtio_balloon_free_page_stop (s); } +virtio_balloon_reset_pbp (s);if (s->stats_vq_elem !=NULL ) {virtqueue_unpop (s->svq, s->stats_vq_elem,0 );
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH-for-4.1 v2 3/3] virtio-balloon: reset pbp on device resets
Quoted 10 lines · David Hildenbrand
When a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
Quoted 8 lines
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Can't something else remove a ramblock besides a reset?
Quoted 18 lines
--- hw/virtio/virtio-balloon.c | 1 + 1 file changed, 1 insertion(+)hw/virtio/virtio-balloon.c +1
-0@@ -847,6 +847,7 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)if (virtio_balloon_free_page_support (s)) {virtio_balloon_free_page_stop (s); } +virtio_balloon_reset_pbp (s);if (s->stats_vq_elem !=NULL ) {virtqueue_unpop (s->svq, s->stats_vq_elem,0 );Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH-for-4.1 v2 3/3] virtio-balloon: reset pbp on device resets
Quoted 20 lines · Michael S. Tsirkin
Quoted 10 lines · David Hildenbrand
When a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
Quoted 8 lines
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 1 line
Can't something else remove a ramblock besides a reset?
Yes, however this patch is not about ramblocks getting removed.
Take a close look, "balloon->pbp->rb" is only used as a token, it is never used besides for comparisons.
Signature · 3 lines
Thanks, David / dhildenb
Re: [Qemu-devel] [PATCH-for-4.1 v2 3/3] virtio-balloon: reset pbp on device resets
Quoted 21 lines · David Hildenbrand
Quoted 20 lines · Michael S. Tsirkin
Quoted 18 lines · David Hildenbrand
When a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 6 lines
Quoted 1 line
Can't something else remove a ramblock besides a reset?
Yes, however this patch is not about ramblocks getting removed.
Take a close look, "
balloon->pbp->rb" is only used as a token, it is never used besides for comparisons.
You are right but that's still not safe :)
E.g. the bit we are going to set could be out of range of the bitmap because the backing page size changed.
Quoted 5 lines
Signature · 3 lines
Thanks, David / dhildenb
Re: [Qemu-devel] [PATCH-for-4.1 v2 3/3] virtio-balloon: reset pbp on device resets
Quoted 26 lines · Michael S. Tsirkin
Quoted 25 lines · David Hildenbrand
Quoted 20 lines · Michael S. Tsirkin
Quoted 10 lines · David Hildenbrand
When a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
Quoted 8 lines
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Quoted 1 line
Can't something else remove a ramblock besides a reset?
Yes, however this patch is not about ramblocks getting removed.
Quoted 8 lines
Quoted 2 lines
Take a close look, "
balloon->pbp->rb" is only used as a token, it is never used besides for comparisons.You are right but that's still not safe :)
E.g. the bit we are going to set could be out of range of the bitmap because the backing page size changed.
As replied to the other thread, I agree. Will look into fixing this, too, tomorrow!
Signature · 3 lines
Thanks, David / dhildenb
Re: [Qemu-devel] [PATCH-for-4.1 v2 3/3] virtio-balloon: reset pbp on device resets
In reply to David HildenbrandQuoted 10 lines · David Hildenbrand
When a guest reboots (ordinary reboots, but also via kexec), it will happily reuse any system memory, including previously inflated memory.
We could have tracking data for a pbp (PartiallyBalloonedPage). It could happen that a new inflation request from the guest will result in a discard of such a pbp, although the guest is (again) reusing some memory.
We should reset the pbp on any device resets.
Quoted 8 lines
Fixes ed48c59875b6 ("virtio-balloon: Safely handle BALLOON_PAGE_SIZE <host page size")
Cc [email protected] #v4.0.0Cc Stefan Hajnoczi <[email protected]>Cc David Gibson <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Igor Mammedov <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 16 lines
--- hw/virtio/virtio-balloon.c | 1 + 1 file changed, 1 insertion(+)hw/virtio/virtio-balloon.c +1
-0@@ -847,6 +847,7 @@ static void virtio_balloon_device_reset(VirtIODevice *vdev)if (virtio_balloon_free_page_support (s)) {virtio_balloon_free_page_stop (s); } +virtio_balloon_reset_pbp (s);if (s->stats_vq_elem !=NULL ) {virtqueue_unpop (s->svq, s->stats_vq_elem,0 );
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson