x86: respect memory size limits
Subsystems (4)
Juergen Gross
maintainercover x86: respect memory size limits
On a customer system running Xen a boot problem was observed due to the kernel not respecting the memory size limit imposed by the Xen hypervisor.
During analysis I found the same problem should be able to occur on bare metal in case the memory would be limited via the "mem=" boot parameter.
The system this problem has been observed on has tons of memory added via PCI. So while in the E820 map the not to be used memory has been wiped out the additional PCI memory is detected during ACPI scan and it is added via __add_memory().
This small series tries to repair the issue by testing the imposed memory limit during the memory hotplug process and refusing to add it in case the limit is being violated.
I've chosen to refuse adding the complete memory chunk in case the limit is reached instead of adding only some of the memory, as I thought this would result in less problems (e.g. avoiding to add only parts of a 128MB memory bar which might be difficult to remove later).
Juergen Gross (2):
x86: respect memory size limiting via mem= parameter x86/xen: dont add memory above max allowed allocation
arch/x86/kernel/e820.c | 5 +++++ arch/x86/xen/setup.c | 5 +++++ include/linux/memory_hotplug.h | 2 ++ mm/memory_hotplug.c | 6 ++++++ 4 files changed, 18 insertions(+)
Signature · 1 line
2.16.4
Juergen Gross
maintainerPATCH 1/2 x86: respect memory size limiting via mem= parameter
When limiting memory size via kernel parameter "mem=" this should be respected even in case of memory made accessible via a PCI card.
Today this kind of memory won't be made usable in initial memory setup as the memory won't be visible in E820 map, but it might be added when adding PCI devices due to corresponding ACPI table entries.
Not respecting "mem=" can be corrected by adding a global max_mem_size variable set by parse_memopt() which will result in rejecting adding memory areas resulting in a memory size above the allowed limit.
--- arch/x86/kernel/e820.c | 5 +++++ include/linux/memory_hotplug.h | 2 ++ mm/memory_hotplug.c | 6 ++++++ 3 files changed, 13 insertions(+)
arch/x86/kernel/e820.c +5 -0
@@ -14,6 +14,7 @@#include <linux/acpi.h> #include <linux/firmware-map.h> #include <linux/sort.h> +#include <linux/memory_hotplug.h> #include <asm/e820/api.h> #include <asm/setup.h>
@@ -881,6 +882,10 @@ static int __init parse_memopt(char *p)e820__range_remove (mem_size,ULLONG_MAX - mem_size,E820_TYPE_RAM ,1 ); +#ifdef CONFIG_MEMORY_HOTPLUG + max_mem_size = mem_size; +#endif +return 0 ; }early_param ("mem" , parse_memopt);
include/linux/memory_hotplug.h +2 -0
@@ -98,6 +98,8 @@ extern void __online_page_free(struct page *page);extern int try_online_node (int nid); +extern u64 max_mem_size; +extern bool memhp_auto_online;/* If movable_node boot option specified */ extern bool movable_node_enabled;
mm/memory_hotplug.c +6 -0
@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); } +u64 max_mem_size =-1 ; +/* add this memory to iomem resource */ static struct resource *register_memory_resource (u64 start,u64 size) {struct resource *res, *conflict; + +if (start + size > max_mem_size) +return ERR_PTR (-E2BIG ); + res =kzalloc (sizeof (struct resource ),GFP_KERNEL );if (!res)return ERR_PTR (-ENOMEM );
Signature · 1 line
2.16.4
Re: [PATCH 1/2] x86: respect memory size limiting via mem= parameter
mm/memory_hotplug.c void mem_hotplug_done(void) · line 99
Quoted 13 lines · Juergen Gross
mm/memory_hotplug.c +1
-0@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); } +u64 max_mem_size =-1 ;
This may be pedantic, but I'd rather see U64_MAX used here.
Juergen Gross
maintainerRe: [PATCH 1/2] x86: respect memory size limiting via mem= parameter
Quoted 12 lines · William Kucharski
Quoted 12 lines · Juergen Gross
mm/memory_hotplug.c +0
-0@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); }
Quoted 3 lines
mm/memory_hotplug.c void mem_hotplug_done(void) · line 99
Quoted 1 line
mm/memory_hotplug.c +1
-0+u64 max_mem_size =-1 ;This may be pedantic, but I'd rather see
U64_MAXused here.
Fine with me. Will change.
Juergen
Juergen Gross
maintainerRe: [PATCH 1/2] x86: respect memory size limiting via mem= parameter
In reply to William KucharskiQuoted 12 lines · William Kucharski
Quoted 12 lines · Juergen Gross
mm/memory_hotplug.c +0
-0@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); }
Quoted 3 lines
mm/memory_hotplug.c void mem_hotplug_done(void) · line 99
Quoted 1 line
mm/memory_hotplug.c +1
-0+u64 max_mem_size =-1 ;This may be pedantic, but I'd rather see
U64_MAXused here.
Fine with me. Will change.
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH 1/2] x86: respect memory size limiting via mem= parameter
In reply to Juergen Grossmm/memory_hotplug.c void mem_hotplug_done(void) · line 99
Quoted 13 lines · Juergen Gross
mm/memory_hotplug.c +1
-0@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); } +u64 max_mem_size =-1 ;
This may be pedantic, but I'd rather see U64_MAX used here.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerPATCH 1/2 x86: respect memory size limiting via mem= parameter
In reply to Juergen GrossWhen limiting memory size via kernel parameter "mem=" this should be respected even in case of memory made accessible via a PCI card.
Today this kind of memory won't be made usable in initial memory setup as the memory won't be visible in E820 map, but it might be added when adding PCI devices due to corresponding ACPI table entries.
Not respecting "mem=" can be corrected by adding a global max_mem_size variable set by parse_memopt() which will result in rejecting adding memory areas resulting in a memory size above the allowed limit.
--- arch/x86/kernel/e820.c | 5 +++++ include/linux/memory_hotplug.h | 2 ++ mm/memory_hotplug.c | 6 ++++++ 3 files changed, 13 insertions(+)
arch/x86/kernel/e820.c +5 -0
@@ -14,6 +14,7 @@#include <linux/acpi.h> #include <linux/firmware-map.h> #include <linux/sort.h> +#include <linux/memory_hotplug.h> #include <asm/e820/api.h> #include <asm/setup.h>
@@ -881,6 +882,10 @@ static int __init parse_memopt(char *p)e820__range_remove (mem_size,ULLONG_MAX - mem_size,E820_TYPE_RAM ,1 ); +#ifdef CONFIG_MEMORY_HOTPLUG + max_mem_size = mem_size; +#endif +return 0 ; }early_param ("mem" , parse_memopt);
include/linux/memory_hotplug.h +2 -0
@@ -98,6 +98,8 @@ extern void __online_page_free(struct page *page);extern int try_online_node (int nid); +extern u64 max_mem_size; +extern bool memhp_auto_online;/* If movable_node boot option specified */ extern bool movable_node_enabled;
mm/memory_hotplug.c +6 -0
@@ -96,10 +96,16 @@ void mem_hotplug_done(void)cpus_read_unlock (); } +u64 max_mem_size =-1 ; +/* add this memory to iomem resource */ static struct resource *register_memory_resource (u64 start,u64 size) {struct resource *res, *conflict; + +if (start + size > max_mem_size) +return ERR_PTR (-E2BIG ); + res =kzalloc (sizeof (struct resource ),GFP_KERNEL );if (!res)return ERR_PTR (-ENOMEM );
Signature · 7 lines
2.16.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerPATCH 2/2 x86/xen: dont add memory above max allowed allocation
In reply to Juergen GrossDon't allow memory to be added above the allowed maximum allocation limit set by Xen.
Trying to do so would result in cases like the following:
[ 584.559652] ------------[ cut here ]------------ [ 584.564897] WARNING: CPU: 2 PID: 1 at ../arch/x86/xen/multicalls.c:129 xen_alloc_pte+0x1c7/0x390() [ 584.575151] Modules linked in: [ 584.578643] Supported: Yes [ 584.581750] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 4.4.120-92.70-default #1 [ 584.590000] Hardware name: Cisco Systems Inc UCSC-C460-M4/UCSC-C460-M4, BIOS C460M4.4.0.1b.0.0629181419 06/29/2018 [ 584.601862] 0000000000000000 ffffffff813175a0 0000000000000000 ffffffff8184777c [ 584.610200] ffffffff8107f4e1 ffff880487eb7000 ffff8801862b79c0 ffff88048608d290 [ 584.618537] 0000000000487eb7 ffffea0000000201 ffffffff81009de7 ffffffff81068561 [ 584.626876] Call Trace: [ 584.629699] [<ffffffff81019ad9>] dump_trace+0x59/0x340 [ 584.635645] [<ffffffff81019eaa>] show_stack_log_lvl+0xea/0x170 [ 584.642391] [<ffffffff8101ac51>] show_stack+0x21/0x40 [ 584.648238] [<ffffffff813175a0>] dump_stack+0x5c/0x7c [ 584.654085] [<ffffffff8107f4e1>] warn_slowpath_common+0x81/0xb0 [ 584.660932] [<ffffffff81009de7>] xen_alloc_pte+0x1c7/0x390 [ 584.667289] [<ffffffff810647f0>] pmd_populate_kernel.constprop.6+0x40/0x80 [ 584.675241] [<ffffffff815ecfe8>] phys_pmd_init+0x210/0x255 [ 584.681587] [<ffffffff815ed207>] phys_pud_init+0x1da/0x247 [ 584.687931] [<ffffffff815edb3b>] kernel_physical_mapping_init+0xf5/0x1d4 [ 584.695682] [<ffffffff815e9bdd>] init_memory_mapping+0x18d/0x380 [ 584.702631] [<ffffffff81064699>] arch_add_memory+0x59/0xf0
--- arch/x86/xen/setup.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/xen/setup.c +5 -0
@@ -12,6 +12,7 @@#include <linux/memblock.h> #include <linux/cpuidle.h> #include <linux/cpufreq.h> +#include <linux/memory_hotplug.h> #include <asm/elf.h> #include <asm/vdso.h>
@@ -785,6 +786,10 @@ char * __init xen_memory_setup(void)/* How many extra pages do we need due to remapping? */ max_pages +=xen_foreach_remap_area (max_pfn, xen_count_remap_pages); +#ifdef CONFIG_MEMORY_HOTPLUG + max_mem_size =PFN_PHYS (max_pages); +#endif +if (max_pages > max_pfn) extra_pages += max_pages - max_pfn;
Signature · 1 line
2.16.4
Re: [Xen-devel] [PATCH 2/2] x86/xen: dont add memory above max allowed allocation
Quoted 3 lines
Quoted 1 line
Quoted 1 line · On 22.01.19 at 09:06, <[email protected]> wrote:
Don't allow memory to be added above the allowed maximum allocation limit set by Xen.
This reads as if the hypervisor was imposing a limit here, but looking at xen_get_max_pages(), xen_foreach_remap_area(), and xen_count_remap_pages() I take it that it's a restriction enforced by the Xen subsystem in Linux. Furthermore from the cover letter I imply that the observed issue was on a Dom0, yet xen_get_max_pages()'s use of XENMEM_maximum_reservation wouldn't impose any limit there at all (without use of the hypervisor option "dom0_mem=max:..."), would it?
Jan
Re: [PATCH 2/2] x86/xen: dont add memory above max allowed allocation
In reply to Juergen GrossQuoted 3 lines
Quoted 1 line
Quoted 1 line · On 22.01.19 at 09:06, <[email protected]> wrote:
Don't allow memory to be added above the allowed maximum allocation limit set by Xen.
This reads as if the hypervisor was imposing a limit here, but looking at xen_get_max_pages(), xen_foreach_remap_area(), and xen_count_remap_pages() I take it that it's a restriction enforced by the Xen subsystem in Linux. Furthermore from the cover letter I imply that the observed issue was on a Dom0, yet xen_get_max_pages()'s use of XENMEM_maximum_reservation wouldn't impose any limit there at all (without use of the hypervisor option "dom0_mem=max:..."), would it?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerPATCH 2/2 x86/xen: dont add memory above max allowed allocation
In reply to Juergen GrossDon't allow memory to be added above the allowed maximum allocation limit set by Xen.
Trying to do so would result in cases like the following:
[ 584.559652] ------------[ cut here ]------------ [ 584.564897] WARNING: CPU: 2 PID: 1 at ../arch/x86/xen/multicalls.c:129 xen_alloc_pte+0x1c7/0x390() [ 584.575151] Modules linked in: [ 584.578643] Supported: Yes [ 584.581750] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 4.4.120-92.70-default #1 [ 584.590000] Hardware name: Cisco Systems Inc UCSC-C460-M4/UCSC-C460-M4, BIOS C460M4.4.0.1b.0.0629181419 06/29/2018 [ 584.601862] 0000000000000000 ffffffff813175a0 0000000000000000 ffffffff8184777c [ 584.610200] ffffffff8107f4e1 ffff880487eb7000 ffff8801862b79c0 ffff88048608d290 [ 584.618537] 0000000000487eb7 ffffea0000000201 ffffffff81009de7 ffffffff81068561 [ 584.626876] Call Trace: [ 584.629699] [<ffffffff81019ad9>] dump_trace+0x59/0x340 [ 584.635645] [<ffffffff81019eaa>] show_stack_log_lvl+0xea/0x170 [ 584.642391] [<ffffffff8101ac51>] show_stack+0x21/0x40 [ 584.648238] [<ffffffff813175a0>] dump_stack+0x5c/0x7c [ 584.654085] [<ffffffff8107f4e1>] warn_slowpath_common+0x81/0xb0 [ 584.660932] [<ffffffff81009de7>] xen_alloc_pte+0x1c7/0x390 [ 584.667289] [<ffffffff810647f0>] pmd_populate_kernel.constprop.6+0x40/0x80 [ 584.675241] [<ffffffff815ecfe8>] phys_pmd_init+0x210/0x255 [ 584.681587] [<ffffffff815ed207>] phys_pud_init+0x1da/0x247 [ 584.687931] [<ffffffff815edb3b>] kernel_physical_mapping_init+0xf5/0x1d4 [ 584.695682] [<ffffffff815e9bdd>] init_memory_mapping+0x18d/0x380 [ 584.702631] [<ffffffff81064699>] arch_add_memory+0x59/0xf0
--- arch/x86/xen/setup.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/xen/setup.c +5 -0
@@ -12,6 +12,7 @@#include <linux/memblock.h> #include <linux/cpuidle.h> #include <linux/cpufreq.h> +#include <linux/memory_hotplug.h> #include <asm/elf.h> #include <asm/vdso.h>
@@ -785,6 +786,10 @@ char * __init xen_memory_setup(void)/* How many extra pages do we need due to remapping? */ max_pages +=xen_foreach_remap_area (max_pfn, xen_count_remap_pages); +#ifdef CONFIG_MEMORY_HOTPLUG + max_mem_size =PFN_PHYS (max_pages); +#endif +if (max_pages > max_pfn) extra_pages += max_pages - max_pfn;
Signature · 7 lines
2.16.4 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel