Skip to contentSkip to conversation

Drivers: hv: balloon

PATCH Superseded K. Y. Srinivasan 2 participants 11 messages Mar 16, 2013 – Mar 18, 2013

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Drivers: hv: balloon

Support 2M page allocations when memory is ballooned out of the guest. Hyper-V Dynamic Memory protocol is optimized around the ability to move memory in 2M chunks.

K. Y. Srinivasan (2):
mm: Export split_page()
Drivers: hv: balloon: Support 2M page allocations for ballooning

 drivers/hv/hv_balloon.c |   18 ++++++++++++++++--
 mm/page_alloc.c         |    1 +
 2 files changed, 17 insertions(+), 2 deletions(-)
Signature · 7 lines
1.7.4.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw

PATCH 1/2 mm: Export split_page()

The split_page() function will be very useful for balloon drivers. On Hyper-V, it will be very efficient to use 2M allocations in the guest as this (a) makes the ballooning protocol with the host that much more efficient and (b) moving memory in 2M chunks minimizes fragmentation in the host. Export the split_page() function to let the guest allocations be in 2M chunks while the host is free to return this memory at arbitrary granularity.

---
 mm/page_alloc.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
mm/page_alloc.c +1 -0
index 6cacfee..7e0ead6 100644
@@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 }
+EXPORT_SYMBOL_GPL(split_page);
 
 static int __isolate_free_page(struct page *page, unsigned int order)
 {
Signature · 7 lines
1.7.4.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

PATCH 2/2 Drivers: hv: balloon: Support 2M page allocations for ballooning

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

---
 drivers/hv/hv_balloon.c |   18 ++++++++++++++++--
 1 files changed, 16 insertions(+), 2 deletions(-)
drivers/hv/hv_balloon.c +16 -2
index 2cf7d4e..71655b4 100644
@@ -997,6 +997,14 @@ static int  alloc_balloon_pages(struct hv_dynmem_device *dm, int num_pages,
 
 		dm->num_pages_ballooned += alloc_unit;
 
+		/*
+		 * If we allocatted 2M pages; split them so we
+		 * can free them in any order we get.
+		 */
+
+		if (alloc_unit != 1)
+			split_page(pg, get_order(alloc_unit << PAGE_SHIFT));
+
 		bl_resp->range_count++;
 		bl_resp->range_array[i].finfo.start_page =
 			page_to_pfn(pg);
@@ -1023,9 +1031,10 @@ static void balloon_up(struct work_struct *dummy)
 
 
 	/*
-	 * Currently, we only support 4k allocations.
+	 * We will attempt 2M allocations. However, if we fail to
+	 * allocate 2M chunks, we will go back to 4k allocations.
 	 */
-	alloc_unit = 1;
+	alloc_unit = 512;
 
 	while (!done) {
 		bl_resp = (struct dm_balloon_response *)send_buffer;
@@ -1041,6 +1050,11 @@ static void balloon_up(struct work_struct *dummy)
 						bl_resp, alloc_unit,
 						 &alloc_error);
 
+		if ((alloc_error) && (alloc_unit != 1)) {
+			alloc_unit = 1;
+			continue;
+		}
+
 		if ((alloc_error) || (num_ballooned == num_pages)) {
 			bl_resp->more_pages = 0;
 			done = true;
Signature · 7 lines
1.7.4.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw

Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 4 lines · K. Y. Srinivasan

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

Maybe I am missing something but what is the advantage of 2M allocation when you split it up immediately so you are not using it as a huge page?

[...]

Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw

Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 8 lines · Michal Hocko
Quoted 4 lines · K. Y. Srinivasan

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

Maybe I am missing something but what is the advantage of 2M allocation when you split it up immediately so you are not using it as a huge page?

OK, it seems that Patch 1/2 says some advantages. That description should be part of this patch IMO.
You are saying that a) it reduces fragmentation on the host b) it makes ballooning more effective.

Have you measured any effects on the ability to allocate huge pages on the host? What prevents from depleting high order pages on the host with many guests? Also have you measured any effect on THP allocation success rate on the host?

Could you clarify how this helps ballooning when you split the page right after you allocate it?

Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw

RE: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

In reply to Michal Hocko
Quoted 13 lines

-----Original Message-----
From: Michal Hocko [mailto:[email protected]]
Sent: Monday, March 18, 2013 6:53 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 8 lines
Quoted 4 lines · K. Y. Srinivasan

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

Maybe I am missing something but what is the advantage of 2M allocation when you split it up immediately so you are not using it as a huge page?

The Hyper-V ballooning protocol specifies the pages being ballooned as page ranges -
start_pfn: number_of_pfns. So, when the guest balloon is inflating and I am able to allocate 2M pages, I will be able to represent 512 contiguous pages in one 64 bit entry and this makes the ballooning operation that much more efficient. The reason I split the page is that the host does not guarantee that when it returns the memory to the guest, it will return in any particular granularity and so I have to be able to free this memory in 4K granularity. This is the corner case that I will have to handle.

Regards,

K. Y

Quoted 6 lines

[...]
--
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw

Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 22 lines · KY Srinivasan
Quoted 13 lines

-----Original Message-----
From: Michal Hocko [mailto:[email protected]]
Sent: Monday, March 18, 2013 6:53 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 8 lines
Quoted 4 lines · K. Y. Srinivasan

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

Maybe I am missing something but what is the advantage of 2M allocation when you split it up immediately so you are not using it as a huge page?

Quoted 9 lines

The Hyper-V ballooning protocol specifies the pages being ballooned as page ranges - start_pfn: number_of_pfns. So, when the guest balloon is inflating and I am able to allocate 2M pages, I will be able to represent 512 contiguous pages in one 64 bit entry and this makes the ballooning operation that much more efficient. The reason I split the page is that the host does not guarantee that when it returns the memory to the guest, it will return in any particular granularity and so I have to be able to free this memory in 4K granularity. This is the corner case that I will have to handle.

Thanks for the clarification. I think this information would be valuable in the changelog.

Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#7 lore raw

RE: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 47 lines

-----Original Message-----
From: Michal Hocko [mailto:[email protected]]
Sent: Monday, March 18, 2013 10:13 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 10 lines · KY Srinivasan
Quoted 10 lines

-----Original Message-----
From: Michal Hocko [mailto:[email protected]]
Sent: Monday, March 18, 2013 6:53 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected];

[email protected]

Quoted 12 lines
Quoted 11 lines

Subject: Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 4 lines · K. Y. Srinivasan

While ballooning memory out of the guest, attempt 2M allocations first. If 2M allocations fail, then go for 4K allocations. In cases where we have performed 2M allocations, split this 2M page so that we can free this page at 4K granularity (when the host returns the memory).

Maybe I am missing something but what is the advantage of 2M allocation when you split it up immediately so you are not using it as a huge page?

Quoted 9 lines

The Hyper-V ballooning protocol specifies the pages being ballooned as page ranges - start_pfn: number_of_pfns. So, when the guest balloon is inflating and I am able to allocate 2M pages, I will be able to represent 512 contiguous pages in one 64 bit entry and this makes the ballooning operation that much more efficient. The reason I split the page is that the host does not guarantee that when it returns the memory to the guest, it will return in any particular granularity and so I have to be able to free this memory in 4K granularity. This is the corner case that I will have to handle.

Quoted 2 lines

Thanks for the clarification. I think this information would be valuable in the changelog.

Thanks Michal. I will resend the patches with the changes you have suggested. What is your recommendation with regards which tree the mm patch needs to go through; the Hyper-V balloon driver patch will go through Greg's tree.

Regards,

K. Y

Quoted 4 lines

--
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#8 lore raw

Re: [PATCH 2/2] Drivers: hv: balloon: Support 2M page allocations for ballooning

Quoted 3 lines · KY Srinivasan

What is your recommendation with regards which tree the mm patch needs to go through; the Hyper-V balloon driver patch will go through Greg's tree.

I would say via Andrew but there are dependencies between those two so a single tree would be less confusing /me thinks.

Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Acked-by on v1 1/2
#9 lore raw

Re: [PATCH 1/2] mm: Export split_page()

In reply to K. Y. Srinivasan
Quoted 8 lines · K. Y. Srinivasan

The split_page() function will be very useful for balloon drivers. On Hyper-V, it will be very efficient to use 2M allocations in the guest as this (a) makes the ballooning protocol with the host that much more efficient and (b) moving memory in 2M chunks minimizes fragmentation in the host. Export the split_page() function to let the guest allocations be in 2M chunks while the host is free to return this memory at arbitrary granularity.

I do not have any objections to exporting the symbol (at least we prevent drivers code from inventing their own split_page) but the Hyper-V specific description should go into Hyper-V patch IMO.

So for the export with a short note that the symbol will be used by Hyper-V

Quoted 24 lines
---
 mm/page_alloc.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
mm/page_alloc.c +1 -0
index 6cacfee..7e0ead6 100644
@@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 }
+EXPORT_SYMBOL_GPL(split_page);
 
 static int __isolate_free_page(struct page *page, unsigned int order)
 {
Signature · 7 lines
1.7.4.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#10 lore raw

RE: [PATCH 1/2] mm: Export split_page()

Quoted 21 lines

-----Original Message-----
From: Michal Hocko [mailto:[email protected]]
Sent: Monday, March 18, 2013 7:04 AM
To: KY Srinivasan
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 1/2] mm: Export split_page()

Quoted 4 lines · K. Y. Srinivasan

The split_page() function will be very useful for balloon drivers. On Hyper-V, it will be very efficient to use 2M allocations in the guest as this (a) makes the ballooning protocol with the host that much more efficient and (b) moving memory in 2M chunks minimizes fragmentation in the host. Export the

split_page()

Quoted 3 lines

function to let the guest allocations be in 2M chunks while the host is free to return this memory at arbitrary granularity.

Quoted 8 lines
Quoted 1 line

I do not have any objections to exporting the symbol (at least we prevent drivers code from inventing their own split_page) but the Hyper-V specific description should go into Hyper-V patch IMO.

So for the export with a short note that the symbol will be used by Hyper-V

Will do.

K. Y

Quoted 21 lines
Quoted 19 lines
---
 mm/page_alloc.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
mm/page_alloc.c +1 -1
index 6cacfee..7e0ead6 100644
@@ -1404,6 +1404,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 }
+EXPORT_SYMBOL_GPL(split_page);
 
 static int __isolate_free_page(struct page *page, unsigned int order)
 {
--

1.7.4.1

Quoted 10 lines
Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>