Skip to contentSkip to conversation

xen/balloon: don't alloc page while non-preemptible

PATCH In review David Vrabel 2 participants 4 messages Sep 19, 2013 – Sep 25, 2013 Xen hypervisor interface

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen/balloon: don't alloc page while non-preemptible

From: David Vrabel <[email protected]>

get_balloon_scratch_page() disables preemption so we cannot call alloc_page() in between get/put_balloon_scratch_page(). Shuffle bits around in decrease_reservation() to avoid this.

---
 drivers/xen/balloon.c |   23 +++++++++++------------
 1 files changed, 11 insertions(+), 12 deletions(-)
drivers/xen/balloon.c +11 -12
index a50c6e3..b232908 100644
@@ -398,8 +398,6 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 	if (nr_pages > ARRAY_SIZE(frame_list))
 		nr_pages = ARRAY_SIZE(frame_list);
 
-	scratch_page = get_balloon_scratch_page();
-
 	for (i = 0; i < nr_pages; i++) {
 		page = alloc_page(gfp);
 		if (page == NULL) {
@@ -413,6 +411,12 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 
 		scrub_page(page);
 
+		/*
+		 * Ballooned out frames are effectively replaced with
+		 * a scratch frame.  Ensure direct mappings and the
+		 * p2m are consistent.
+		 */
+		scratch_page = get_balloon_scratch_page();
 #ifdef CONFIG_XEN_HAVE_PVMMU
 		if (xen_pv_domain() && !PageHighMem(page)) {
 			ret = HYPERVISOR_update_va_mapping(
@@ -422,24 +426,19 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 			BUG_ON(ret);
 		}
 #endif
-	}
-
-	/* Ensure that ballooned highmem pages don't have kmaps. */
-	kmap_flush_unused();
-	flush_tlb_all();
-
-	/* No more mappings: invalidate P2M and add to balloon. */
-	for (i = 0; i < nr_pages; i++) {
-		pfn = mfn_to_pfn(frame_list[i]);
 		if (!xen_feature(XENFEAT_auto_translated_physmap)) {
 			unsigned long p;
 			p = page_to_pfn(scratch_page);
 			__set_phys_to_machine(pfn, pfn_to_mfn(p));
 		}
+		put_balloon_scratch_page();
+
 		balloon_append(pfn_to_page(pfn));
 	}
 
-	put_balloon_scratch_page();
+	/* Ensure that ballooned highmem pages don't have kmaps. */
+	kmap_flush_unused();
+	flush_tlb_all();
 
 	set_xen_guest_handle(reservation.extent_start, frame_list);
 	reservation.nr_extents   = nr_pages;
Signature · 1 line
1.7.2.5
#1 lore raw
Quoted 54 lines · David Vrabel

From: David Vrabel <[email protected]>

get_balloon_scratch_page() disables preemption so we cannot call alloc_page() in between get/put_balloon_scratch_page(). Shuffle bits around in decrease_reservation() to avoid this.

---
 drivers/xen/balloon.c |   23 +++++++++++------------
 1 files changed, 11 insertions(+), 12 deletions(-)
drivers/xen/balloon.c +6 -11
index a50c6e3..b232908 100644
@@ -398,8 +398,6 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 	if (nr_pages > ARRAY_SIZE(frame_list))
 		nr_pages = ARRAY_SIZE(frame_list);
 
-	scratch_page = get_balloon_scratch_page();
-
 	for (i = 0; i < nr_pages; i++) {
 		page = alloc_page(gfp);
 		if (page == NULL) {
@@ -413,6 +411,12 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 
 		scrub_page(page);
 
+		/*
+		 * Ballooned out frames are effectively replaced with
+		 * a scratch frame.  Ensure direct mappings and the
+		 * p2m are consistent.
+		 */
+		scratch_page = get_balloon_scratch_page();
 #ifdef CONFIG_XEN_HAVE_PVMMU
 		if (xen_pv_domain() && !PageHighMem(page)) {
 			ret = HYPERVISOR_update_va_mapping(
@@ -422,24 +426,19 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp)
 			BUG_ON(ret);
 		}
 #endif
-	}
-
-	/* Ensure that ballooned highmem pages don't have kmaps. */
-	kmap_flush_unused();
-	flush_tlb_all();
-
-	/* No more mappings: invalidate P2M and add to balloon. */
-	for (i = 0; i < nr_pages; i++) {
-		pfn = mfn_to_pfn(frame_list[i]);
 		if (!xen_feature(XENFEAT_auto_translated_physmap)) {
 			unsigned long p;
 			p = page_to_pfn(scratch_page);
 			__set_phys_to_machine(pfn, pfn_to_mfn(p));

drivers/xen/balloon.c static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) · line 441

Quoted 10 lines
drivers/xen/balloon.c +5 -1
 		}
+		put_balloon_scratch_page();
+
 		balloon_append(pfn_to_page(pfn));
 	}
 
-	put_balloon_scratch_page();
+	/* Ensure that ballooned highmem pages don't have kmaps. */
+	kmap_flush_unused();
+	flush_tlb_all();

The change with possible side effects in this patch is that kmap_flush_unused and flush_tlb_all are now called after setting the p2m and after calling balloon_append instead of before.

However they are still called before the XENMEM_decrease_reservation hypercall, so I think it's OK.

#2 lore raw
Quoted 11 lines · Stefano Stabellini

drivers/xen/balloon.c static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) · line 441

Quoted 5 lines · David Vrabel
drivers/xen/balloon.c +3 -1
-	put_balloon_scratch_page();
+	/* Ensure that ballooned highmem pages don't have kmaps. */
+	kmap_flush_unused();
+	flush_tlb_all();

The change with possible side effects in this patch is that kmap_flush_unused and flush_tlb_all are now called after setting the p2m and after calling balloon_append instead of before.

Quoted 2 lines

However they are still called before the XENMEM_decrease_reservation hypercall, so I think it's OK.

Yes, the requirement is that the kmap cache is flushed before releasing the frame to Xen (in case it contains a kmap for a frame we're about to release).

I also wonder if the flush_tlb_all() is even necessary. Surely Xen has all the appropriate TLB flushes in the decrease_reservation hypercall. It can't possibly rely on the guest doing the right thing.

David

#3 lore raw
Quoted 15 lines · David Vrabel
Quoted 11 lines · Stefano Stabellini

drivers/xen/balloon.c static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) · line 441

Quoted 5 lines · David Vrabel
drivers/xen/balloon.c +3 -1
-	put_balloon_scratch_page();
+	/* Ensure that ballooned highmem pages don't have kmaps. */
+	kmap_flush_unused();
+	flush_tlb_all();

The change with possible side effects in this patch is that kmap_flush_unused and flush_tlb_all are now called after setting the p2m and after calling balloon_append instead of before.

Quoted 2 lines

However they are still called before the XENMEM_decrease_reservation hypercall, so I think it's OK.

Quoted 7 lines

Yes, the requirement is that the kmap cache is flushed before releasing the frame to Xen (in case it contains a kmap for a frame we're about to release).

I also wonder if the flush_tlb_all() is even necessary. Surely Xen has all the appropriate TLB flushes in the decrease_reservation hypercall. It can't possibly rely on the guest doing the right thing.

That would only be the case for PV guests, but yes, Xen has certainly a flush.