Drivers: hv: hv_balloon: avoid OOM killer on the ballooning path
cover Drivers: hv: hv_balloon: avoid OOM killer on the ballooning path
In some cases host asks us to overballoon and this triggers OOM killer which eventually kills everyone. The easiest way to get into such situation is to avoid onlining memory-hotplugged blocks. Address the issue twice:
- Report offline pages as used to the host so it won't ask us to overballoon;
- Avoid ballooning below the 'floor'.
Vitaly Kuznetsov (2):
Drivers: hv: hv_balloon: report offline pages as being used
Drivers: hv: hv_balloon: refuse to balloon below the floor
drivers/hv/hv_balloon.c | 44 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 35 insertions(+), 9 deletions(-)
Signature · 1 line
1.9.3
PATCH 1/2 Drivers: hv: hv_balloon: report offline pages as being used
When hot-added memory pages are not brought online or when some memory blocks are sent offline the subsequent ballooning process kills the guest with OOM killer. This happens as we don't report these pages as neither used nor free and apparently host algorythm considers them as being unused. Keep track of all online/offline operations and report all currently offline pages as being used so host won't try to balloon them out.
--- drivers/hv/hv_balloon.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)
drivers/hv/hv_balloon.c +24 -9
@@ -503,6 +503,8 @@ struct hv_dynmem_device {* Number of pages we have currently ballooned out. */ unsigned int num_pages_ballooned; +unsigned int num_pages_onlined; +unsigned int num_pages_added;/* * State to manage the ballooning (up) operation.
@@ -556,12 +558,15 @@ static void post_status(struct hv_dynmem_device *dm);static int hv_memory_notifier (struct notifier_block *nb,unsigned long val,void *v) { +struct memory_notify *mem = (struct memory_notify *)v; +switch (val) {case MEM_GOING_ONLINE :mutex_lock (&dm_device.ha_region_mutex);break ;case MEM_ONLINE : + dm_device.num_pages_onlined += mem->nr_pages;case MEM_CANCEL_ONLINE :mutex_unlock (&dm_device.ha_region_mutex);if (dm_device.ha_waiting) {
@@ -570,8 +575,12 @@ static int hv_memory_notifier(struct notifier_block *nb, unsigned long val, }break ; -case MEM_GOING_OFFLINE :case MEM_OFFLINE : +mutex_lock (&dm_device.ha_region_mutex); + dm_device.num_pages_onlined -= mem->nr_pages; +mutex_unlock (&dm_device.ha_region_mutex); +break ; +case MEM_GOING_OFFLINE :case MEM_CANCEL_OFFLINE :break ; }
@@ -896,6 +905,8 @@ static void hot_add_req(struct work_struct *dummy)if (do_hot_add) resp.page_count =process_hot_add (pg_start, pfn_cnt, rg_start, rg_sz); + + dm->num_pages_added += resp.page_count;mutex_unlock (&dm_device.ha_region_mutex);#endif /*
@@ -1009,17 +1020,21 @@ static void post_status(struct hv_dynmem_device *dm) status.hdr.trans_id =atomic_inc_return (&trans_id);/* -* The host expects the guest to report free memory. -* Further, the host expects the pressure information to -* include the ballooned out pages. -* For a given amount of memory that we are managing, we -* need to compute a floor below which we should not balloon. -* Compute this and add it to the pressure report. +* The host expects the guest to report free and committed memory. +* Furthermore, the host expects the pressure information to include +* the ballooned out pages. For a given amount of memory that we are +* managing we need to compute a floor below which we should not +* balloon. Compute this and add it to the pressure report. +* We also need to report all offline pages (num_pages_added - +* num_pages_onlined) as committed to the host, otherwise it can try +* asking us to balloon them out. */ status.num_avail = val.freeram; status.num_committed =vm_memory_committed () + - dm->num_pages_ballooned + -compute_balloon_floor (); + dm->num_pages_ballooned + + (dm->num_pages_added > dm->num_pages_onlined ? + dm->num_pages_added - dm->num_pages_onlined :0 ) + +compute_balloon_floor ();/* * If our transaction ID is no longer current, just don't
Signature · 1 line
1.9.3
RE: [PATCH 1/2] Drivers: hv: hv_balloon: report offline pages as being used
Quoted 42 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, February 19, 2015 8:27 AM
To: KY Srinivasan; [email protected]
Cc: Haiyang Zhang; [email protected]; Dexuan Cui
Subject: [PATCH 1/2] Drivers: hv:hv_balloon: report offline pages as being usedWhen hot-added memory pages are not brought online or when some memory blocks
are sent offline the subsequent ballooning process kills the guest with OOM killer. This happens as we don't report these pages as neither used nor free and apparently host algorythm considers them as being unused. Keep track of
all online/offline operations and report all currently offline pages as being used so host won't try to balloon them out.Signed-off-by Vitaly Kuznetsov <[email protected]>--- drivers/hv/hv_balloon.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)drivers/hv/hv_balloon.c +2
-0@@ -503,6 +503,8 @@ struct hv_dynmem_device {* Number of pages we have currently ballooned out. */ unsigned int num_pages_ballooned; +unsigned int num_pages_onlined; +unsigned int num_pages_added;/* * State to manage the ballooning (up) operation. @@ -556,12 +558,15 @@ static void post_status(struct hv_dynmem_device *dm);static int hv_memory_notifier (struct notifier_block *nb,unsigned long val,void *v) {
drivers/hv/hv_balloon.c static void post_status(struct hv_dynmem_device · line 571
Quoted 10 lines
drivers/hv/hv_balloon.c +3
-0+struct memory_notify *mem = (struct memory_notify *)v; +switch (val) {case MEM_GOING_ONLINE :mutex_lock (&dm_device.ha_region_mutex);break ;case MEM_ONLINE : + dm_device.num_pages_onlined += mem->nr_pages;case MEM_CANCEL_ONLINE :
Why are we not adjusting num_pages_onlined when we cancel the online Operation.
K. Y
Re: [PATCH 1/2] Drivers: hv: hv_balloon: report offline pages as being used
Quoted 47 lines · KY Srinivasan
Quoted 47 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, February 19, 2015 8:27 AM
To: KY Srinivasan; [email protected]
Cc: Haiyang Zhang; [email protected]; Dexuan Cui
Subject: [PATCH 1/2] Drivers: hv:hv_balloon: report offline pages as being usedWhen hot-added memory pages are not brought online or when some memory blocks
are sent offline the subsequent ballooning process kills the guest with OOM killer. This happens as we don't report these pages as neither used nor free and apparently host algorythm considers them as being unused. Keep track of
all online/offline operations and report all currently offline pages as being used so host won't try to balloon them out.Signed-off-by Vitaly Kuznetsov <[email protected]>--- drivers/hv/hv_balloon.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)drivers/hv/hv_balloon.c +4
-0@@ -503,6 +503,8 @@ struct hv_dynmem_device {* Number of pages we have currently ballooned out. */ unsigned int num_pages_ballooned; +unsigned int num_pages_onlined; +unsigned int num_pages_added;/* * State to manage the ballooning (up) operation. @@ -556,12 +558,15 @@ static void post_status(struct hv_dynmem_device *dm);static int hv_memory_notifier (struct notifier_block *nb,unsigned long val,void *v) { +struct memory_notify *mem = (struct memory_notify *)v; +switch (val) {case MEM_GOING_ONLINE :mutex_lock (&dm_device.ha_region_mutex);break ;
Quoted 6 lines
drivers/hv/hv_balloon.c static void post_status(struct hv_dynmem_device *dm); · line 570
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0case MEM_ONLINE : + dm_device.num_pages_onlined += mem->nr_pages;case MEM_CANCEL_ONLINE :Why are we not adjusting
num_pages_onlinedwhen we cancel the online Operation.
Because we didn't increase the number yet.
To my understanding, events come in the following order:
1) MEM_GOING_ONLINE - we just take the lock
2) MEM_ONLINE - and we increase nr_pages and drop the lock or
MEM_CANCEL_ONLINE - we just drop the lock (mem never was online so
nr_pages wasn't increased)
3) MEM_GOING_OFFLINE - we do nothing
4) MEM_OFFLINE - and we decrease nr_pages
or
MEM_CANCEL_OFFLINE - we do nothing (mem is still online, no need to
adjust nr_pages)
Quoted 2 lines
K. Y
Signature · 1 line
Vitaly
RE: [PATCH 1/2] Drivers: hv: hv_balloon: report offline pages as being used
Quoted 60 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Wednesday, February 25, 2015 8:56 AM
To: KY Srinivasan
Cc: [email protected]; Haiyang Zhang; linux- [email protected]; Dexuan Cui
Subject: Re: [PATCH 1/2] Drivers: hv:hv_balloon: report offline pages as being usedQuoted 32 lines · KY Srinivasan
Quoted 32 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, February 19, 2015 8:27 AM
To: KY Srinivasan; [email protected]
Cc: Haiyang Zhang; [email protected]; Dexuan Cui
Subject: [PATCH 1/2] Drivers: hv:hv_balloon: report offline pages as being usedWhen hot-added memory pages are not brought online or when some memory blocks are sent offline the subsequent ballooning process kills the guest with OOM killer. This happens as we don't report these pages as neither used nor free and apparently host algorythm considers them as being unused. Keep track of all online/offline operations and report all currently offline pages as being used so host won't try to balloon them out.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- drivers/hv/hv_balloon.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)drivers/hv/hv_balloon.c index renamed from drivers/hv/hv_balloon.c +0
-0a095b70..e4b4454 100644
drivers/hv/hv_balloon.c +2
-0@@ -503,6 +503,8 @@ struct hv_dynmem_device {* Number of pages we have currently ballooned out. */ unsigned int num_pages_ballooned; +unsigned int num_pages_onlined; +unsigned int num_pages_added;Quoted 3 lines
drivers/hv/hv_balloon.c static void post_status(struct
Quoted 3 lines
drivers/hv/hv_balloon.c +0
-0/* * State to manage the ballooning (up) operation. @@ -556,12 +558,15 @@ static void post_status(struct
hv_dynmem_deviceQuoted 11 lines
Quoted 11 lines
drivers/hv/hv_balloon.c +2
-0*dm); static int hv_memory_notifier(struct notifier_block *nb,unsigned long val,void *v) { +struct memory_notify *mem = (struct memory_notify *)v; +switch (val) {case MEM_GOING_ONLINE :mutex_lock (&dm_device.ha_region_mutex);break ;
Quoted 8 lines
Quoted 6 lines
drivers/hv/hv_balloon.c static void post_status(struct hv_dynmem_device *dm); · line 570
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0case MEM_ONLINE : + dm_device.num_pages_onlined += mem->nr_pages;case MEM_CANCEL_ONLINE :Why are we not adjusting
num_pages_onlinedwhen we cancel the online Operation.Because we didn't increase the number yet.
Thanks; my mistake.
K. Y
PATCH 2/2 Drivers: hv: hv_balloon: refuse to balloon below the floor
In reply to Vitaly KuznetsovWhen host asks us to balloon up we need to be sure we're not committing suicide by overballooning. Use already existent 'floor' metric as our lowest possible value for free ram.
--- drivers/hv/hv_balloon.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
drivers/hv/hv_balloon.c +11 -0
@@ -1138,6 +1138,8 @@ static void balloon_up(struct work_struct *dummy)bool alloc_error;bool done = false;int i; +struct sysinfo val; +unsigned long floor;/* The host balloons pages in 2M granularity. */ WARN_ON_ONCE (num_pages %PAGES_IN_2M !=0 );
@@ -1148,6 +1150,15 @@ static void balloon_up(struct work_struct *dummy)*/ alloc_unit =512 ; +si_meminfo (&val); + floor =compute_balloon_floor (); + +/* Refuse to balloon below the floor, keep the 2M granularity. */ +if (val.freeram - num_pages < floor) { + num_pages = val.freeram > floor ? (val.freeram - floor) :0 ; + num_pages -= num_pages %PAGES_IN_2M ; + } +while (!done) { bl_resp = (struct dm_balloon_response *)send_buffer;memset (send_buffer,0 ,PAGE_SIZE );
Signature · 1 line
1.9.3