xen-blkback: use balloon pages for persistent grants
Subsystems (2)
PATCH 1/1 xen-blkback: use balloon pages for persistent grants
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map operations change the mfn of the allocated page, and we can no longer pass it to __free_page without setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.
--- drivers/block/xen-blkback/blkback.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)
drivers/block/xen-blkback/blkback.c +4 -2
@@ -46,6 +46,7 @@#include <xen/xen.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/balloon.h> #include "common.h" /*
@@ -239,6 +240,7 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num) ret =gnttab_unmap_refs (unmap,NULL , pages, segs_to_unmap);BUG_ON (ret); +free_xenballooned_pages (segs_to_unmap, pages); segs_to_unmap =0 ; }
@@ -527,8 +529,8 @@ static int xen_blkbk_map(struct blkif_request *req, GFP_KERNEL);if (!persistent_gnt)return -ENOMEM ; - persistent_gnt->page =alloc_page (GFP_KERNEL ); -if (!persistent_gnt->page) { +if (alloc_xenballooned_pages (1 , &persistent_gnt->page, + false)) {kfree (persistent_gnt);return -ENOMEM ; }
Signature · 1 line
1.7.7.5 (Apple Git-26)
Quoted 9 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>
I think this patch is missing the following change in Kconfig, but gntdev doesn't depend on the balloon driver, which it also uses, so I'm not sure.
---
drivers/block/Kconfig +1 -1
@@ -488,7 +488,7 @@ config XEN_BLKDEV_FRONTEND config XEN_BLKDEV_BACKEND tristate "Xen block-device backend driver" - depends on XEN_BACKEND + depends on XEN_BACKEND && XEN_BALLOON help The block-device backend driver allows the kernel to export its block devices to other guests via a high-performance shared-memory
Quoted 9 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>
I think this patch is missing the following change in Kconfig, but gntdev doesn't depend on the balloon driver, which it also uses, so I'm not sure.
---
drivers/block/Kconfig +1 -1
@@ -488,7 +488,7 @@ config XEN_BLKDEV_FRONTEND config XEN_BLKDEV_BACKEND tristate "Xen block-device backend driver" - depends on XEN_BACKEND + depends on XEN_BACKEND && XEN_BALLOON help The block-device backend driver allows the kernel to export its block devices to other guests via a high-performance shared-memory
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't gnttab_set_unmap_op in free_persistent_gnts doing the right job of putting in the right mfn in? And then we could free the page?
Quoted 3 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.
Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
Quoted 42 lines
Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>--- drivers/block/xen-blkback/blkback.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)drivers/block/xen-blkback/blkback.c +4
-2@@ -46,6 +46,7 @@#include <xen/xen.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/balloon.h> #include "common.h" /* @@ -239,6 +240,7 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num) ret =gnttab_unmap_refs (unmap,NULL , pages, segs_to_unmap);BUG_ON (ret); +free_xenballooned_pages (segs_to_unmap, pages); segs_to_unmap =0 ; }@@ -527,8 +529,8 @@ static int xen_blkbk_map(struct blkif_request *req, GFP_KERNEL);if (!persistent_gnt)return -ENOMEM ; - persistent_gnt->page =alloc_page (GFP_KERNEL ); -if (!persistent_gnt->page) { +if (alloc_xenballooned_pages (1 , &persistent_gnt->page, + false)) {kfree (persistent_gnt);return -ENOMEM ; }Signature · 1 line
1.7.7.5 (Apple Git-26)
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't gnttab_set_unmap_op in free_persistent_gnts doing the right job of putting in the right mfn in? And then we could free the page?
Quoted 3 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.
Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
Quoted 42 lines
Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>--- drivers/block/xen-blkback/blkback.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)drivers/block/xen-blkback/blkback.c +4
-2@@ -46,6 +46,7 @@#include <xen/xen.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/balloon.h> #include "common.h" /* @@ -239,6 +240,7 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num) ret =gnttab_unmap_refs (unmap,NULL , pages, segs_to_unmap);BUG_ON (ret); +free_xenballooned_pages (segs_to_unmap, pages); segs_to_unmap =0 ; }@@ -527,8 +529,8 @@ static int xen_blkbk_map(struct blkif_request *req, GFP_KERNEL);if (!persistent_gnt)return -ENOMEM ; - persistent_gnt->page =alloc_page (GFP_KERNEL ); -if (!persistent_gnt->page) { +if (alloc_xenballooned_pages (1 , &persistent_gnt->page, + false)) {kfree (persistent_gnt);return -ENOMEM ; }Signature · 1 line
1.7.7.5 (Apple Git-26)
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't
gnttab_set_unmap_opinfree_persistent_gntsdoing the right job of putting in the right mfn in? And then we could free the page?
No, after gnttab_unmap_refs the page still points to the grant frame mfn. All the users of grant frames either have an internal buffer of pages that are reused (like blkback), or use balloon pages (like gntdev).
I could probably modify gnttab_map to store the mfn and set it back at gnttab_unmap, but that will surely require more work than this simple fix.
Quoted 10 lines
Quoted 5 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
No, we didn't recycle them AFAIK, we allocated them using alloc_page, passed them to gnttab_map, used them, and when closing the backend we only unmapped them, but they where never freed.
Quoted 42 lines
Quoted 42 lines
Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>--- drivers/block/xen-blkback/blkback.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)drivers/block/xen-blkback/blkback.c +4
-2@@ -46,6 +46,7 @@#include <xen/xen.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/balloon.h> #include "common.h" /* @@ -239,6 +240,7 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num) ret =gnttab_unmap_refs (unmap,NULL , pages, segs_to_unmap);BUG_ON (ret); +free_xenballooned_pages (segs_to_unmap, pages); segs_to_unmap =0 ; }@@ -527,8 +529,8 @@ static int xen_blkbk_map(struct blkif_request *req, GFP_KERNEL);if (!persistent_gnt)return -ENOMEM ; - persistent_gnt->page =alloc_page (GFP_KERNEL ); -if (!persistent_gnt->page) { +if (alloc_xenballooned_pages (1 , &persistent_gnt->page, + false)) {kfree (persistent_gnt);return -ENOMEM ; }Signature · 1 line
1.7.7.5 (Apple Git-26)
Quoted 9 lines · Roger Pau Monné
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't
gnttab_set_unmap_opinfree_persistent_gntsdoing the right job of putting in the right mfn in? And then we could free the page?
Quoted 6 lines
No, after
gnttab_unmap_refsthe page still points to the grant frame mfn. All the users of grant frames either have an internal buffer of pages that are reused (like blkback), or use balloon pages (like gntdev).I could probably modify
gnttab_mapto store the mfn and set it back atgnttab_unmap, but that will surely require more work than this simple fix.
Simple fix is what we want right now.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.
Quoted 7 lines
Quoted 3 lines
Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
No, we didn't recycle them AFAIK, we allocated them using
alloc_page, passed them tognttab_map, used them, and when closing the backend we only unmapped them, but they where never freed.
Ghastly!
Quoted 9 lines · Roger Pau Monné
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't
gnttab_set_unmap_opinfree_persistent_gntsdoing the right job of putting in the right mfn in? And then we could free the page?
Quoted 6 lines
No, after
gnttab_unmap_refsthe page still points to the grant frame mfn. All the users of grant frames either have an internal buffer of pages that are reused (like blkback), or use balloon pages (like gntdev).I could probably modify
gnttab_mapto store the mfn and set it back atgnttab_unmap, but that will surely require more work than this simple fix.
Simple fix is what we want right now.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.
Quoted 7 lines
Quoted 3 lines
Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
No, we didn't recycle them AFAIK, we allocated them using
alloc_page, passed them tognttab_map, used them, and when closing the backend we only unmapped them, but they where never freed.
Ghastly!
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · Roger Pau Monne
With current persistent grants implementation we are not freeing the persistent grants after we disconnect the device. Since grant map
Can you explain this in more details please? Isn't
gnttab_set_unmap_opinfree_persistent_gntsdoing the right job of putting in the right mfn in? And then we could free the page?
No, after gnttab_unmap_refs the page still points to the grant frame mfn. All the users of grant frames either have an internal buffer of pages that are reused (like blkback), or use balloon pages (like gntdev).
I could probably modify gnttab_map to store the mfn and set it back at gnttab_unmap, but that will surely require more work than this simple fix.
Quoted 10 lines
Quoted 5 lines
operations change the mfn of the allocated page, and we can no longer pass it to
__free_pagewithout setting the mfn to a sane value, use balloon grant pages instead, as the gntdev device does.Wow. I did not realize that we leaving such a huge memory leak behind! But I guess that was never an issue as we would recycle those persistent grants to other domains.
No, we didn't recycle them AFAIK, we allocated them using alloc_page, passed them to gnttab_map, used them, and when closing the backend we only unmapped them, but they where never freed.
Quoted 42 lines
Quoted 42 lines
Signed-off-by Roger Pau Monné <[email protected]>Cc Konrad Rzeszutek Wilk <[email protected]>--- drivers/block/xen-blkback/blkback.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)drivers/block/xen-blkback/blkback.c +4
-2@@ -46,6 +46,7 @@#include <xen/xen.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <xen/balloon.h> #include "common.h" /* @@ -239,6 +240,7 @@ static void free_persistent_gnts(struct rb_root *root, unsigned int num) ret =gnttab_unmap_refs (unmap,NULL , pages, segs_to_unmap);BUG_ON (ret); +free_xenballooned_pages (segs_to_unmap, pages); segs_to_unmap =0 ; }@@ -527,8 +529,8 @@ static int xen_blkbk_map(struct blkif_request *req, GFP_KERNEL);if (!persistent_gnt)return -ENOMEM ; - persistent_gnt->page =alloc_page (GFP_KERNEL ); -if (!persistent_gnt->page) { +if (alloc_xenballooned_pages (1 , &persistent_gnt->page, + false)) {kfree (persistent_gnt);return -ENOMEM ; }Signature · 1 line
1.7.7.5 (Apple Git-26)