vmw_balloon: addressing bugs and issues
Subsystems (2)
cover vmw_balloon: addressing bugs and issues
This patch-set addresses minor issues and bugs in the VMware balloon driver. Patches 3-7 address bugs that were not reported by customers, but we would like to get them to 4.18 if possible.
We will send separately later a patch-set with new features to be set on top of this set.
Nadav Amit (7):
vmw_balloon: update copyright message
vmw_balloon: update maintainers list
vmw_balloon: fix inflation of 64-bit GFNs
vmw_balloon: do not use 2MB without batching
vmw_balloon: VMCI_DOORBELL_SET does not check status
vmw_balloon: fix VMCI use when balloon built into kernel
vmw_balloon: remove inflation rate limiting
MAINTAINERS | 2 +- drivers/misc/vmw_balloon.c | 167 ++++++++++++------------------------- 2 files changed, 56 insertions(+), 113 deletions(-)
Signature · 1 line
2.17.0
PATCH 1/7 vmw_balloon: update copyright message
Removing the paragraph about writing to the Free Software Foundation's mailing address from the sample GPL notice according to checkpatch request.
In addition, updating the year and adding a license tag.
--- drivers/misc/vmw_balloon.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
drivers/misc/vmw_balloon.c +2 -4
@@ -1,7 +1,7 @@/* * VMware Balloon driver. * -* Copyright (C) 2000-2014, VMware, Inc. All Rights Reserved. +* Copyright (C) 2000-2018, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the
@@ -13,9 +13,7 @@* NON INFRINGEMENT. See the GNU General Public License for more * details. * -* You should have received a copy of the GNU General Public License -* along with this program; if not, write to the Free Software -* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +* SPDX-License-Identifier: GPL-2.0 * * Maintained by: Xavier Deguillard <[email protected]> * Philip Moltmann <[email protected]>
Signature · 1 line
2.17.0
Greg Kroah-Hartman
maintainerRe: [PATCH 1/7] vmw_balloon: update copyright message
Quoted 26 lines · Nadav Amit
Removing the paragraph about writing to the Free Software Foundation's mailing address from the sample GPL notice according to checkpatch request.
In addition, updating the year and adding a license tag.
Reviewed-by Xavier Deguillard <[email protected]>Signed-off-by Nadav Amit <[email protected]>--- drivers/misc/vmw_balloon.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)drivers/misc/vmw_balloon.c +1
-1@@ -1,7 +1,7 @@/* * VMware Balloon driver. * -* Copyright (C) 2000-2014, VMware, Inc. All Rights Reserved. +* Copyright (C) 2000-2018, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the
drivers/misc/vmw_balloon.c line 16
Quoted 7 lines
drivers/misc/vmw_balloon.c +1
-3@@ -13,9 +13,7 @@* NON INFRINGEMENT. See the GNU General Public License for more * details. * -* You should have received a copy of the GNU General Public License -* along with this program; if not, write to the Free Software -* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +* SPDX-License-Identifier: GPL-2.0
This goes as the first line of the file, please see the documentation files that describes how to do this properly.
Thanks,
greg k-h
Re: [PATCH 1/7] vmw_balloon: update copyright message
Quoted 29 lines · Greg Kroah-Hartman
Quoted 28 lines · Nadav Amit
Removing the paragraph about writing to the Free Software Foundation's mailing address from the sample GPL notice according to checkpatch request.
In addition, updating the year and adding a license tag.
Reviewed-by Xavier Deguillard <[email protected]>Signed-off-by Nadav Amit <[email protected]>---
drivers/misc/vmw_balloon.c| 6 ++----1 file changed, 2 insertions(+), 4 deletions(-)drivers/misc/vmw_balloon.c +1
-1@@ -1,7 +1,7 @@/* * VMware Balloon driver. * -* Copyright (C) 2000-2014, VMware, Inc. All Rights Reserved. +* Copyright (C) 2000-2018, VMware, Inc. All Rights Reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License as published by the @@ -13,9 +13,7 @@* NON INFRINGEMENT. See the GNU General Public License for more * details.
Quoted 8 lines
drivers/misc/vmw_balloon.c line 16
Quoted 5 lines
drivers/misc/vmw_balloon.c +1
-3* -* You should have received a copy of the GNU General Public License -* along with this program; if not, write to the Free Software -* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. +* SPDX-License-Identifier: GPL-2.0 This goes as the first line of the file, please see the documentation files that describes how to do this properly.
Of course. I was sure I fixed it... I will resend.
PATCH 2/7 vmw_balloon: update maintainers list
In reply to Nadav AmitPhilip Moltman is no longer a maintainer of the VMware balloon. Setting Nadav Amit as one instead.
--- MAINTAINERS | 2 +- drivers/misc/vmw_balloon.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
MAINTAINERS +1 -1
@@ -15136,7 +15136,7 @@ F: include/linux/vme* VMWARE BALLOON DRIVER M: Xavier Deguillard <[email protected]> -M: Philip Moltmann <[email protected]> +M: Nadav Amit <[email protected]> M: "VMware, Inc." <[email protected]> L: [email protected] S: Maintained
drivers/misc/vmw_balloon.c +1 -1
@@ -16,7 +16,7 @@* SPDX-License-Identifier: GPL-2.0 * * Maintained by: Xavier Deguillard <[email protected]> -* Philip Moltmann <[email protected]> +* Nadav Amit <[email protected]> */ /*
Signature · 1 line
2.17.0
PATCH 3/7 vmw_balloon: fix inflation of 64-bit GFNs
In reply to Nadav AmitWhen balloon batching is not supported by the hypervisor, the guest frame number (GFN) must fit in 32-bit. However, due to a bug, this check was mistakenly ignored. In practice, when total RAM is greater than 16TB, the balloon does not work currently, making this bug unlikely to happen.
vmballoon_reserve_page.")
--- drivers/misc/vmw_balloon.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
drivers/misc/vmw_balloon.c +7 -6
@@ -448,7 +448,7 @@ static int vmballoon_send_lock_page(struct vmballoon *b, unsigned long pfn, pfn32 = (u32 )pfn;if (pfn32 != pfn) -return -1 ; +return -EINVAL ;STATS_INC (b->stats.lock[false]);
@@ -458,7 +458,7 @@ static int vmballoon_send_lock_page(struct vmballoon *b, unsigned long pfn,pr_debug ("%s - ppn %lx, hv returns %ld\n" , __func__, pfn, status);STATS_INC (b->stats.lock_fail[false]); -return 1 ; +return -EIO ; } static int vmballoon_send_batched_lock(struct vmballoon *b,
@@ -595,11 +595,12 @@ static int vmballoon_lock_page(struct vmballoon *b, unsigned int num_pages, locked =vmballoon_send_lock_page (b,page_to_pfn (page), &hv_status, target); -if (locked >0 ) { +if (locked) {STATS_INC (b->stats.refused_alloc[false]); -if (hv_status ==VMW_BALLOON_ERROR_RESET || - hv_status ==VMW_BALLOON_ERROR_PPN_NOTNEEDED ) { +if (locked == -EIO && + (hv_status ==VMW_BALLOON_ERROR_RESET || + hv_status ==VMW_BALLOON_ERROR_PPN_NOTNEEDED )) {vmballoon_free_page (page, false);return -EIO ; }
@@ -615,7 +616,7 @@ static int vmballoon_lock_page(struct vmballoon *b, unsigned int num_pages, }else {vmballoon_free_page (page, false); } -return -EIO ; +return locked; }/* track allocated page */
Signature · 1 line
2.17.0
Greg Kroah-Hartman
maintainerRe: [PATCH 3/7] vmw_balloon: fix inflation of 64-bit GFNs
Quoted 9 lines · Nadav Amit
When balloon batching is not supported by the hypervisor, the guest frame number (GFN) must fit in 32-bit. However, due to a bug, this check was mistakenly ignored. In practice, when total RAM is greater than 16TB, the balloon does not work currently, making this bug unlikely to happen.
Fixes ef0f8f112984 ("VMware balloon: partially inline
vmballoon_reserve_page.")
Why have a bugfix as the 3rd patch of the series? Please put it first, so that it can get into Linus's tree sooner.
Also, no need to line-wrap this, and drop the blank line.
And finally, does this need to go to the stable kernels? If so, please add the correct line for that.
thanks,
greg k-h
Re: [PATCH 3/7] vmw_balloon: fix inflation of 64-bit GFNs
Quoted 10 lines · Greg Kroah-Hartman
Quoted 8 lines · Nadav Amit
When balloon batching is not supported by the hypervisor, the guest frame number (GFN) must fit in 32-bit. However, due to a bug, this check was mistakenly ignored. In practice, when total RAM is greater than 16TB, the balloon does not work currently, making this bug unlikely to happen.
Fixes ef0f8f112984 ("VMware balloon: partially inline
vmballoon_reserve_page.")
Quoted 7 lines
Why have a bugfix as the 3rd patch of the series? Please put it first, so that it can get into Linus's tree sooner.
Also, no need to line-wrap this, and drop the blank line.
And finally, does this need to go to the stable kernels? If so, please add the correct line for that.
I will fix the message. Patches 3-7 deal with minor bugs, some of which have been there since day one, so I didn’t put a “Fixes” tag.
I’ll follow your direction and put patches 1-2 at the end.
PATCH 4/7 vmw_balloon: do not use 2MB without batching
In reply to Nadav AmitIf the hypervisor sets 2MB batching is on, while batching is cleared, the balloon code breaks. In this case the legacy mechanism is used with 2MB page. The VM would report a 2MB page is ballooned, and the hypervisor would only take the first 4KB.
While the hypervisor should not report such settings, make the code more robust by not enabling 2MB support without batching.
--- drivers/misc/vmw_balloon.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/misc/vmw_balloon.c +7 -1
@@ -339,7 +339,13 @@ static bool vmballoon_send_start(struct vmballoon *b, unsigned long req_caps) success = false; } -if (b->capabilities &VMW_BALLOON_BATCHED_2M_CMDS ) +/* +* 2MB pages are only supported with batching. If batching is for some +* reason disabled, do not use 2MB pages, since otherwise the legacy +* mechanism is used with 2MB pages, causing a failure. +*/ +if ((b->capabilities &VMW_BALLOON_BATCHED_2M_CMDS ) && + (b->capabilities &VMW_BALLOON_BATCHED_CMDS )) b->supported_page_sizes =2 ;else b->supported_page_sizes =1 ;
Signature · 1 line
2.17.0
PATCH 5/7 vmw_balloon: VMCI_DOORBELL_SET does not check status
In reply to Nadav AmitWhen vmballoon_vmci_init() sets a doorbell using VMCI_DOORBELL_SET, for some reason it does not consider the status and looks at the result. However, the hypervisor does not update the result - it updates the status.
--- drivers/misc/vmw_balloon.c | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-)
drivers/misc/vmw_balloon.c +19 -18
@@ -1034,29 +1034,30 @@ static void vmballoon_vmci_cleanup(struct vmballoon *b)*/ static int vmballoon_vmci_init (struct vmballoon *b) { -int error =0 ; +unsigned long error, dummy; -if ((b->capabilities &VMW_BALLOON_SIGNALLED_WAKEUP_CMD ) !=0 ) { - error =vmci_doorbell_create (&b->vmci_doorbell, -VMCI_FLAG_DELAYED_CB , -VMCI_PRIVILEGE_FLAG_RESTRICTED , - vmballoon_doorbell, b); - -if (error ==VMCI_SUCCESS ) { -VMWARE_BALLOON_CMD (VMCI_DOORBELL_SET , - b->vmci_doorbell.context, - b->vmci_doorbell.resource, error); -STATS_INC (b->stats.doorbell_set); - } - } +if ((b->capabilities &VMW_BALLOON_SIGNALLED_WAKEUP_CMD ) ==0 ) +return 0 ; -if (error !=0 ) { -vmballoon_vmci_cleanup (b); + error =vmci_doorbell_create (&b->vmci_doorbell,VMCI_FLAG_DELAYED_CB , +VMCI_PRIVILEGE_FLAG_RESTRICTED , + vmballoon_doorbell, b); -return -EIO ; - } +if (error !=VMCI_SUCCESS ) + goto fail; + + error =VMWARE_BALLOON_CMD (VMCI_DOORBELL_SET , b->vmci_doorbell.context, + b->vmci_doorbell.resource, dummy); + +STATS_INC (b->stats.doorbell_set); + +if (error !=VMW_BALLOON_SUCCESS ) + goto fail;return 0 ; +fail: +vmballoon_vmci_cleanup (b); +return -EIO ; }/*
Signature · 1 line
2.17.0
PATCH 6/7 vmw_balloon: fix VMCI use when balloon built into kernel
In reply to Nadav AmitCurrently, when all modules, including VMCI and VMware balloon are built into the kernel, the initialization of the balloon happens before the VMCI is probed. As a result, the balloon fails to initialize the VMCI doorbell, which it uses to get asynchronous requests for balloon size changes.
The problem can be seen in the logs, in the form of the following message:
"vmw_balloon: failed to initialize vmci doorbell"
The driver would work correctly but slightly less efficiently, probing for requests periodically. This patch changes the balloon to be initialized using late_initcall() instead of module_init() to address this issue. It does not address a situation in which VMCI is built as a module and the balloon is built into the kernel.
--- drivers/misc/vmw_balloon.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/misc/vmw_balloon.c +8 -1
@@ -1295,7 +1295,14 @@ static int __init vmballoon_init(void)return 0 ; } -module_init (vmballoon_init); + +/* +* Using late_initcall() instead of module_init() allows the balloon to use the +* VMCI doorbell even when the balloon is built into the kernel. Otherwise the +* VMCI is probed only after the balloon is initialized. If the balloon is used +* as a module, late_initcall() is equivalent to module_init(). +*/ +late_initcall (vmballoon_init); static void __exit vmballoon_exit(void) {
Signature · 1 line
2.17.0
PATCH 7/7 vmw_balloon: remove inflation rate limiting
In reply to Nadav AmitSince commit 33d268ed0019 ("VMware balloon: Do not limit the amount of frees and allocations in non-sleep mode."), the allocations are not increased, and therefore balloon inflation rate limiting is in practice broken.
While we can restore rate limiting, in practice we see that it can result in adverse effect, as the hypervisor throttles down the VM if it does not respond well enough, or alternatively causes it to perform very poorly as the host swaps out the VM memory. Throttling the VM down can even have a cascading effect, in which the VM reclaims memory even slower and consequentially throttled down even further.
We therefore remove all the rate limiting mechanisms, including the slow allocation cycles, as they are likely to do more harm than good.
--- drivers/misc/vmw_balloon.c | 92 +++++--------------------------------- 1 file changed, 11 insertions(+), 81 deletions(-)
drivers/misc/vmw_balloon.c +11 -81
@@ -52,25 +52,6 @@ MODULE_ALIAS("dmi:*:svnVMware*:*");MODULE_ALIAS ("vmware_vmmemctl" );MODULE_LICENSE ("GPL" ); -/* -* Various constants controlling rate of inflaint/deflating balloon, -* measured in pages. -*/ - -/* -* Rates of memory allocaton when guest experiences memory pressure -* (driver performs sleeping allocations). -*/ -#define VMW_BALLOON_RATE_ALLOC_MIN 512U -#define VMW_BALLOON_RATE_ALLOC_MAX 2048U -#define VMW_BALLOON_RATE_ALLOC_INC 16U - -/* -* When guest is under memory pressure, use a reduced page allocation -* rate for next several cycles. -*/ -#define VMW_BALLOON_SLOW_CYCLES 4 -/* * Use __GFP_HIGHMEM to allow pages from HIGHMEM zone. We don't * allow wait (__GFP_RECLAIM) for NOSLEEP page allocations. Use
@@ -282,12 +263,6 @@ struct vmballoon {/* reset flag */ bool reset_required; -/* adjustment rates (pages per second) */ -unsigned int rate_alloc; - -/* slowdown page allocations for next few cycles */ -unsigned int slow_allocation_cycles; -unsigned long capabilities;struct vmballoon_batch_page *batch_page;
@@ -795,8 +770,6 @@ static void vmballoon_add_batched_page(struct vmballoon *b, int idx,*/ static void vmballoon_inflate (struct vmballoon *b) { -unsigned rate; -unsigned int allocations =0 ;unsigned int num_pages =0 ;int error =0 ;gfp_t flags =VMW_PAGE_ALLOC_NOSLEEP ;
@@ -823,17 +796,9 @@ static void vmballoon_inflate(struct vmballoon *b)* Start with no sleep allocation rate which may be higher * than sleeping allocation rate. */ -if (b->slow_allocation_cycles) { - rate = b->rate_alloc; - is_2m_pages = false; - }else { - rate =UINT_MAX ; - is_2m_pages = - b->supported_page_sizes ==VMW_BALLOON_NUM_PAGE_SIZES ; - } + is_2m_pages = b->supported_page_sizes ==VMW_BALLOON_NUM_PAGE_SIZES ; -pr_debug ("%s - goal: %d, no-sleep rate: %u, sleep rate: %d\n" , - __func__, b->target - b->size, rate, b->rate_alloc); +pr_debug ("%s - goal: %d" , __func__, b->target - b->size);while (!b->reset_required && b->size + num_pages * vmballoon_page_size(is_2m_pages)
@@ -866,31 +831,24 @@ static void vmballoon_inflate(struct vmballoon *b)if (flags ==VMW_PAGE_ALLOC_CANSLEEP ) {/* * CANSLEEP page allocation failed, so guest -* is under severe memory pressure. Quickly -* decrease allocation rate. +* is under severe memory pressure. We just log +* the event, but do not stop the inflation +* due to its negative impact on performance. */ - b->rate_alloc =max (b->rate_alloc /2 , -VMW_BALLOON_RATE_ALLOC_MIN );STATS_INC (b->stats.sleep_alloc_fail);break ; }/* * NOSLEEP page allocation failed, so the guest is -* under memory pressure. Let us slow down page -* allocations for next few cycles so that the guest -* gets out of memory pressure. Also, if we already -* allocated b->rate_alloc pages, let's pause, -* otherwise switch to sleeping allocations. +* under memory pressure. Slowing down page alloctions +* seems to be reasonable, but doing so might actually +* cause the hypervisor to throttle us down, resulting +* in degraded performance. We will count on the +* scheduler and standard memory management mechanisms +* for now. */ - b->slow_allocation_cycles =VMW_BALLOON_SLOW_CYCLES ; - -if (allocations >= b->rate_alloc) -break ; - flags =VMW_PAGE_ALLOC_CANSLEEP ; -/* Lower rate for sleeping allocations. */ - rate = b->rate_alloc;continue ; }
@@ -904,28 +862,11 @@ static void vmballoon_inflate(struct vmballoon *b) }cond_resched (); - -if (allocations >= rate) { -/* We allocated enough pages, let's take a break. */ -break ; - } }if (num_pages >0 ) b->ops->lock (b, num_pages, is_2m_pages, &b->target); -/* -* We reached our goal without failures so try increasing -* allocation rate. -*/ -if (error ==0 && allocations >= b->rate_alloc) { -unsigned int mult = allocations / b->rate_alloc; - - b->rate_alloc = -min (b->rate_alloc + mult *VMW_BALLOON_RATE_ALLOC_INC , -VMW_BALLOON_RATE_ALLOC_MAX ); - } -vmballoon_release_refused_pages (b, true);vmballoon_release_refused_pages (b, false); }
@@ -1120,9 +1061,6 @@ static void vmballoon_work(struct work_struct *work)if (b->reset_required)vmballoon_reset (b); -if (b->slow_allocation_cycles >0 ) - b->slow_allocation_cycles--; -if (!b->reset_required &&vmballoon_send_get_target (b, &target)) {/* update target, adjust size */ b->target = target;
@@ -1166,11 +1104,6 @@ static int vmballoon_debug_show(struct seq_file *f, void *offset)"current: %8d pages\n" , b->target, b->size); -/* format rate info */ -seq_printf (f, -"rateSleepAlloc: %8d pages/sec\n" , - b->rate_alloc); -seq_printf (f,"\n" "timer: %8u\n"
@@ -1277,9 +1210,6 @@ static int __init vmballoon_init(void)INIT_LIST_HEAD (&balloon.page_sizes[is_2m_pages].refused_pages); } -/* initialize rates */ - balloon.rate_alloc =VMW_BALLOON_RATE_ALLOC_MAX ; -INIT_DELAYED_WORK (&balloon.dwork, vmballoon_work); error =vmballoon_debugfs_init (&balloon);
Signature · 1 line
2.17.0