mm/memory_hotplug: Pre-validate the address range with platform
Subsystems (7)
v3 cover mm/memory_hotplug: Pre-validate the address range with platform
This series adds a mechanism allowing platforms to weigh in and prevalidate incoming address range before proceeding further with the memory hotplug. This helps prevent potential platform errors for the given address range, down the hotplug call chain, which inevitably fails the hotplug itself.
This mechanism was suggested by David Hildenbrand during another discussion with respect to a memory hotplug fix on arm64 platform.
https://lore.kernel.org/linux-arm-kernel/[email protected]/
This mechanism focuses on the addressibility aspect and not [sub] section alignment aspect. Hence check_hotplug_memory_range() and check_pfn_span() have been left unchanged. Wondering if all these can still be unified in an expanded memhp_range_allowed() check, that can be called from multiple memory hot add and remove paths.
This series applies on v5.11-rc4 and has been tested on arm64. But only build tested on s390.
Changes in V3
- Updated the commit message in [PATCH 1/3]
- Replaced 1 with 'true' and 0 with 'false' in memhp_range_allowed()
- Updated memhp_range.end as VMEM_MAX_PHYS - 1 and updated vmem_add_mapping() on s390
- Changed memhp_range_allowed() behaviour in __add_pages()
- Updated __add_pages() to return E2BIG when memhp_range_allowed() fails for non-linear mapping based requests
Changes in V2:
https://lore.kernel.org/linux-mm/[email protected]/
- Changed s390 version per Heiko and updated the commit message
- Called memhp_range_allowed() only for arch_add_memory() in pagemap_range()
- Exported the symbol memhp_get_pluggable_range()
Changes in V1:
https://lore.kernel.org/linux-mm/[email protected]/
- Fixed build problems with (MEMORY_HOTPLUG & !MEMORY_HOTREMOVE)
- Added missing prototype for arch_get_mappable_range()
- Added VM_BUG_ON() check for memhp_range_allowed() in arch_add_memory() per David
Changes in RFC V2:
https://lore.kernel.org/linux-mm/[email protected]/
Incorporated all review feedbacks from David.
- Added additional range check in __segment_load() on s390 which was lost
- Changed is_private init in pagemap_range()
- Moved the framework into mm/memory_hotplug.c
- Made arch_get_addressable_range() a __weak function
- Renamed arch_get_addressable_range() as arch_get_mappable_range()
- Callback arch_get_mappable_range() only handles range requiring linear mapping
- Merged multiple memhp_range_allowed() checks in register_memory_resource()
- Replaced WARN() with pr_warn() in memhp_range_allowed()
- Replaced error return code ERANGE with E2BIG
Changes in RFC V1:
https://lore.kernel.org/linux-mm/[email protected]/
Anshuman Khandual (3):
mm/memory_hotplug: Prevalidate the address range being added with platform arm64/mm: Define arch_get_mappable_range()
s390/mm: Define arch_get_mappable_range()
David Hildenbrand (1):
virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
arch/arm64/mm/mmu.c | 15 +++---- arch/s390/mm/init.c | 1 + arch/s390/mm/vmem.c | 15 ++++++- drivers/virtio/virtio_mem.c | 40 +++++++++++------ include/linux/memory_hotplug.h | 10 +++++ mm/memory_hotplug.c | 79 ++++++++++++++++++++++++++-------- mm/memremap.c | 6 +++ 7 files changed, 125 insertions(+), 41 deletions(-)
Signature · 1 line
2.20.1
PATCH v3 1/3 mm/memory_hotplug: Prevalidate the address range being added with platform
This introduces memhp_range_allowed() which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Then memhp_range_allowed() calls memhp_get_pluggable_range() which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callback arch_get_mappable_range() which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.
This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant check_hotplug_memory_addressable() check in __add_pages() but instead adds a VM_BUG_ON() check which would ensure that the range has been validated with memhp_range_allowed() earlier in the call chain. Besides memhp_get_pluggable_range() also can be used by potential memory hotplug callers to avail the allowed physical range which would go through on a given platform.
This does not really add any new range check in generic memory hotplug but instead compensates for lost checks in arch_add_memory() where applicable and check_hotplug_memory_addressable(), with unified memhp_range_allowed().
--- include/linux/memory_hotplug.h | 10 +++++ mm/memory_hotplug.c | 79 ++++++++++++++++++++++++++-------- mm/memremap.c | 6 +++ 3 files changed, 76 insertions(+), 19 deletions(-)
include/linux/memory_hotplug.h +10 -0
@@ -70,6 +70,9 @@ typedef int __bitwise mhp_t;*/ #define MEMHP_MERGE_RESOURCE ((__force mhp_t)BIT(0)) +bool memhp_range_allowed (u64 start,u64 size,bool need_mapping); +struct range memhp_get_pluggable_range (bool need_mapping); +/* * Extended parameters for memory hotplug: * altmap: alternative allocator for memmap array (optional)
@@ -281,6 +284,13 @@ static inline bool movable_node_is_enabled(void) }#endif /* ! CONFIG_MEMORY_HOTPLUG */ +/* +* Keep this declaration outside CONFIG_MEMORY_HOTPLUG as some +* platforms might override and use arch_get_mappable_range() +* for internal non memory hotplug purposes. +*/ +struct range arch_get_mappable_range (void ); +#if defined(CONFIG_MEMORY_HOTPLUG ) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT )/* * pgdat resizing functions
mm/memory_hotplug.c +60 -19
103 lines not on this page · raw message
mm/memremap.c +6 -0
@@ -253,6 +253,12 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params, goto err_kasan; } +if (!memhp_range_allowed (range->start,range_len (range), true)) { + error = -ERANGE ; +mem_hotplug_done (); + goto err_add_memory; + } + error =arch_add_memory (nid, range->start,range_len (range), params); }
Signature · 1 line
2.20.1
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 9 lines · Anshuman Khandual
This introduces
memhp_range_allowed()which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Thenmemhp_range_allowed()callsmemhp_get_pluggable_range()which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callbackarch_get_mappable_range()which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.
Quoted 3 lines
This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant
check_hotplug_memory_addressable()check in__add_pages()but instead adds aVM_BUG_ON()check which would
In this patch, you keep the __add_pages() checks. But as discussed, we could perform it in mm/memremap.c:pagemap_range() insted and convert it to a VM_BUG_ON().
Apart from that looks good to me.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 10 lines · David Hildenbrand
Quoted 9 lines · Anshuman Khandual
This introduces
memhp_range_allowed()which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Thenmemhp_range_allowed()callsmemhp_get_pluggable_range()which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callbackarch_get_mappable_range()which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.
Quoted 7 lines
Quoted 3 lines
This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant
check_hotplug_memory_addressable()check in__add_pages()but instead adds aVM_BUG_ON()check which wouldIn this patch, you keep the
__add_pages()checks. But as discussed, we could perform it in mm/memremap.c:pagemap_range() insted and convert it to aVM_BUG_ON().
Just to be sure, will the following change achieve what you are suggesting here. pagemap_range() after this change, will again be the same like the V1 series.
--- mm/memory_hotplug.c | 3 +-- mm/memremap.c | 12 +++++------- 2 files changed, 6 insertions(+), 9 deletions(-)
mm/memory_hotplug.c +1 -2
@@ -304,8 +304,7 @@ int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages,if (WARN_ON_ONCE (!params->pgprot.pgprot))return -EINVAL ; -if (!memhp_range_allowed (PFN_PHYS (pfn), nr_pages *PAGE_SIZE , false)) -return -E2BIG ; +VM_BUG_ON (!memhp_range_allowed (PFN_PHYS (pfn), nr_pages *PAGE_SIZE , false));if (altmap) {/*
mm/memremap.c +5 -7
@@ -185,6 +185,7 @@ static void dev_pagemap_percpu_release(struct percpu_ref *ref)static int pagemap_range (struct dev_pagemap *pgmap,struct mhp_params *params,int range_id,int nid) { +const bool is_private = pgmap->type ==MEMORY_DEVICE_PRIVATE ;struct range *range = &pgmap->ranges[range_id];struct dev_pagemap *conflict_pgmap;int error, is_ram;
@@ -230,6 +231,9 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params,if (error) goto err_pfn_remap; +if (!memhp_range_allowed (range->start,range_len (range), !is_private)) + goto err_pfn_remap; +mem_hotplug_begin ();/*
@@ -243,7 +247,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params,* the CPU, we do want the linear mapping and thus use * arch_add_memory(). */ -if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) { +if (is_private) { error =add_pages (nid,PHYS_PFN (range->start),PHYS_PFN (range_len (range)), params); }else {
@@ -253,12 +257,6 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params, goto err_kasan; } -if (!memhp_range_allowed (range->start,range_len (range), true)) { - error = -ERANGE ; -mem_hotplug_done (); - goto err_add_memory; - } - error =arch_add_memory (nid, range->start,range_len (range), params); }
Signature · 1 line
2.20.1
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 17 lines · Anshuman Khandual
Quoted 14 lines · David Hildenbrand
Quoted 12 lines · Anshuman Khandual
This introduces
memhp_range_allowed()which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Thenmemhp_range_allowed()callsmemhp_get_pluggable_range()which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callbackarch_get_mappable_range()which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant
check_hotplug_memory_addressable()check in__add_pages()but instead adds aVM_BUG_ON()check which would
Quoted 7 lines
Quoted 3 lines
In this patch, you keep the
__add_pages()checks. But as discussed, we could perform it in mm/memremap.c:pagemap_range() insted and convert it to aVM_BUG_ON().Just to be sure, will the following change achieve what you are suggesting here.
pagemap_range()after this change, will again be the same like the V1 series.
Yeah, as we used to have in v1. Maybe other reviewers (@Oscar?) have a different opinion.
If you decide to leave as-is, please fixup the patch description. Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Oscar Salvador
maintainerRe: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 22 lines · David Hildenbrand
Quoted 21 lines · Anshuman Khandual
Quoted 17 lines · David Hildenbrand
Quoted 9 lines · Anshuman Khandual
This introduces
memhp_range_allowed()which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Thenmemhp_range_allowed()callsmemhp_get_pluggable_range()which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callbackarch_get_mappable_range()which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.Quoted 3 lines
This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant
check_hotplug_memory_addressable()check in__add_pages()but instead adds aVM_BUG_ON()check which wouldIn this patch, you keep the
__add_pages()checks. But as discussed, we could perform it in mm/memremap.c:pagemap_range() insted and convert it to aVM_BUG_ON().
Quoted 6 lines
Quoted 3 lines
Just to be sure, will the following change achieve what you are suggesting here.
pagemap_range()after this change, will again be the same like the V1 series.Yeah, as we used to have in v1. Maybe other reviewers (@Oscar?) have a different opinion.
No, I think that placing the check in pagemap_range() out of the if-else makes much more sense.
Actually, unless my memory fails me that is what I suggested in v2.
I plan to have a look at the series later this week as I am fairly busy atm.
Thanks
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
In reply to Anshuman KhandualQuoted 3 lines · Anshuman Khandual
Just to be sure, will the following change achieve what you are suggesting here.
pagemap_range()after this change, will again be the same like the V1 series.
With below diff on top it looks good to me:
The only nit I would have is whether the declaration of arch_get_mappable_range should be in include/linux/memory_hotplug.h.
As you pointed out, arch_get_mappable_range() might be used by the platform for other purposes, and since you are defining it out of CONFIG_MEMORY_HOTPLUG anyway.
Would include/linu/memory.h be a better fit?
As I said, nothing to bikeshed about, just my thoughts.
Quoted 66 lines
--- mm/memory_hotplug.c | 3 +-- mm/memremap.c | 12 +++++------- 2 files changed, 6 insertions(+), 9 deletions(-)mm/memory_hotplug.c +1
-2@@ -304,8 +304,7 @@ int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages,if (WARN_ON_ONCE (!params->pgprot.pgprot))return -EINVAL ; -if (!memhp_range_allowed (PFN_PHYS (pfn), nr_pages *PAGE_SIZE , false)) -return -E2BIG ; +VM_BUG_ON (!memhp_range_allowed (PFN_PHYS (pfn), nr_pages *PAGE_SIZE , false));if (altmap) {/* mm/memremap.c +5
-7@@ -185,6 +185,7 @@ static void dev_pagemap_percpu_release(struct percpu_ref *ref)static int pagemap_range (struct dev_pagemap *pgmap,struct mhp_params *params,int range_id,int nid) { +const bool is_private = pgmap->type ==MEMORY_DEVICE_PRIVATE ;struct range *range = &pgmap->ranges[range_id];struct dev_pagemap *conflict_pgmap;int error, is_ram;@@ -230,6 +231,9 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params,if (error) goto err_pfn_remap; +if (!memhp_range_allowed (range->start,range_len (range), !is_private)) + goto err_pfn_remap; +mem_hotplug_begin ();/* @@ -243,7 +247,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params,* the CPU, we do want the linear mapping and thus use * arch_add_memory(). */ -if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) { +if (is_private) { error =add_pages (nid,PHYS_PFN (range->start),PHYS_PFN (range_len (range)), params); }else {@@ -253,12 +257,6 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params, goto err_kasan; } -if (!memhp_range_allowed (range->start,range_len (range), true)) { - error = -ERANGE ; -mem_hotplug_done (); - goto err_add_memory; - } - error =arch_add_memory (nid, range->start,range_len (range), params); }Signature · 1 line
2.20.1
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
In reply to Anshuman KhandualQuoted 16 lines
mm/memory_hotplug.c +15
-0+/* +* Platforms should define arch_get_mappable_range() that provides +* maximum possible addressable physical memory range for which the +* linear mapping could be created. The platform returned address +* range must adhere to these following semantics. +* +* - range.start <= range.end +* - Range includes both end points [range.start..range.end] +* +* There is also a fallback definition provided here, allowing the +* entire possible physical address range in case any platform does +* not define arch_get_mappable_range(). +*/ +struct range __weak arch_get_mappable_range (void ) +{
mm/memory_hotplug.c int add_memory_driver_managed(int nid, u64 start, u64 size, · line 1193
Quoted 10 lines
mm/memory_hotplug.c +10
-0+struct range memhp_range = { + .start =0UL , + .end =-1ULL , + }; +return memhp_range; +} + +struct range memhp_get_pluggable_range (bool need_mapping) +{ +const u64 max_phys = (1ULL << (MAX_PHYSMEM_BITS +1 )) -1 ;
Sorry, thought about that line a bit more, and I think this is just wrong (took me longer to realize as it should). The old code used this calculation to print the limit only (in a wrong way), let's recap:
Assume MAX_PHYSMEM_BITS=32
max_phys = (1ULL << (32 + 1)) - 1 = 0x1ffffffffull;
Ehm, these are 33 bit.
OTOH, old code checked for
if (max_addr >> MAX_PHYSMEM_BITS) {Which makes sense, because
0x1ffffffffull >> 32 = 1
results in "true", meaning it's to big, while
0xffffffffull >> 32 = 0
correctly results in "false", meaning the address is fine.
So, this should just be
const u64 max_phys = 1ULL << MAX_PHYSMEM_BITS;
(similarly as calculated in virito-mem code, or in kernel/resource.c)
Quoted 16 lines
mm/memory_hotplug.c +16
-0+struct range memhp_range; + +if (need_mapping) { + memhp_range =arch_get_mappable_range (); +if (memhp_range.start > max_phys) { + memhp_range.start =0 ; + memhp_range.end =0 ; + } + memhp_range.end =min_t (u64, memhp_range.end, max_phys); + }else { + memhp_range.start =0 ; + memhp_range.end = max_phys; + } +return memhp_range; +} +EXPORT_SYMBOL_GPL (memhp_get_pluggable_range);
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 53 lines · David Hildenbrand
Quoted 16 lines
mm/memory_hotplug.c +15
-0+/* +* Platforms should define arch_get_mappable_range() that provides +* maximum possible addressable physical memory range for which the +* linear mapping could be created. The platform returned address +* range must adhere to these following semantics. +* +* - range.start <= range.end +* - Range includes both end points [range.start..range.end] +* +* There is also a fallback definition provided here, allowing the +* entire possible physical address range in case any platform does +* not define arch_get_mappable_range(). +*/ +struct range __weak arch_get_mappable_range (void ) +{mm/memory_hotplug.c int add_memory_driver_managed(int nid, u64 start, u64 size, · line 1193
Quoted 10 lines
mm/memory_hotplug.c +10
-0+struct range memhp_range = { + .start =0UL , + .end =-1ULL , + }; +return memhp_range; +} + +struct range memhp_get_pluggable_range (bool need_mapping) +{ +const u64 max_phys = (1ULL << (MAX_PHYSMEM_BITS +1 )) -1 ;Sorry, thought about that line a bit more, and I think this is just wrong (took me longer to realize as it should). The old code used this calculation to print the limit only (in a wrong way), let's recap:
Assume MAX_PHYSMEM_BITS=32
max_phys = (1ULL << (32 + 1)) - 1 = 0x1ffffffffull;Ehm, these are 33 bit.
OTOH, old code checked for
if (max_addr >> MAX_PHYSMEM_BITS) {Which makes sense, because
0x1ffffffffull >> 32 = 1results in "true", meaning it's to big, while
0xffffffffull >> 32 = 0correctly results in "false", meaning the address is fine.
Quoted 5 lines
So, this should just be
const u64
max_phys= 1ULL <<MAX_PHYSMEM_BITS;(similarly as calculated in virito-mem code, or in
kernel/resource.c)
Should this be 1ULL << MAX_PHYSMEM_BITS - 1 instead ? Currently there are three usage for this variable in the function.
- if (mhp_range.start > max_phys)
- mhp_range.end = min_t(u64, mhp_range.end, max_phys)
- mhp_range.end = max_phys
mhp_range.end being always inclusive on the higher end and could be maximum (1ULL << MAX_PHYSMEM_BITS - 1) which is 0xFFFFFFFF instead of 0x100000000 when (1ULL << MAX_PHYSMEM_BITS) is followed for a 32 bit system. This seems consistent with the default fallback (range.end = -1ULL) defined above.
Quoted 20 lines
Quoted 18 lines
mm/memory_hotplug.c +16
-0+struct range memhp_range; + +if (need_mapping) { + memhp_range =arch_get_mappable_range (); +if (memhp_range.start > max_phys) { + memhp_range.start =0 ; + memhp_range.end =0 ; + } + memhp_range.end =min_t (u64, memhp_range.end, max_phys); + }else { + memhp_range.start =0 ; + memhp_range.end = max_phys; + } +return memhp_range; +} +EXPORT_SYMBOL_GPL (memhp_get_pluggable_range);
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 55 lines · Anshuman Khandual
Quoted 53 lines · David Hildenbrand
Quoted 16 lines
mm/memory_hotplug.c +15
-0+/* +* Platforms should define arch_get_mappable_range() that provides +* maximum possible addressable physical memory range for which the +* linear mapping could be created. The platform returned address +* range must adhere to these following semantics. +* +* - range.start <= range.end +* - Range includes both end points [range.start..range.end] +* +* There is also a fallback definition provided here, allowing the +* entire possible physical address range in case any platform does +* not define arch_get_mappable_range(). +*/ +struct range __weak arch_get_mappable_range (void ) +{mm/memory_hotplug.c int add_memory_driver_managed(int nid, u64 start, u64 size, · line 1193
Quoted 10 lines
mm/memory_hotplug.c +10
-0+struct range memhp_range = { + .start =0UL , + .end =-1ULL , + }; +return memhp_range; +} + +struct range memhp_get_pluggable_range (bool need_mapping) +{ +const u64 max_phys = (1ULL << (MAX_PHYSMEM_BITS +1 )) -1 ;Sorry, thought about that line a bit more, and I think this is just wrong (took me longer to realize as it should). The old code used this calculation to print the limit only (in a wrong way), let's recap:
Assume MAX_PHYSMEM_BITS=32
max_phys = (1ULL << (32 + 1)) - 1 = 0x1ffffffffull;Ehm, these are 33 bit.
OTOH, old code checked for
if (max_addr >> MAX_PHYSMEM_BITS) {Which makes sense, because
0x1ffffffffull >> 32 = 1results in "true", meaning it's to big, while
0xffffffffull >> 32 = 0correctly results in "false", meaning the address is fine.
Quoted 7 lines
Quoted 5 lines
So, this should just be
const u64
max_phys= 1ULL <<MAX_PHYSMEM_BITS;(similarly as calculated in virito-mem code, or in
kernel/resource.c)Should this be 1ULL <<
MAX_PHYSMEM_BITS- 1 instead ? Currently there are
Yes, obviously, sorry, forgot the -1.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH V3 1/3] mm/memory_hotplug: Prevalidate the address range being added with platform
Quoted 58 lines · David Hildenbrand
Quoted 57 lines · Anshuman Khandual
Quoted 55 lines · David Hildenbrand
Quoted 16 lines
mm/memory_hotplug.c +15
-0+/* +* Platforms should define arch_get_mappable_range() that provides +* maximum possible addressable physical memory range for which the +* linear mapping could be created. The platform returned address +* range must adhere to these following semantics. +* +* - range.start <= range.end +* - Range includes both end points [range.start..range.end] +* +* There is also a fallback definition provided here, allowing the +* entire possible physical address range in case any platform does +* not define arch_get_mappable_range(). +*/ +struct range __weak arch_get_mappable_range (void ) +{mm/memory_hotplug.c int add_memory_driver_managed(int nid, u64 start, u64 size, · line 1193
Quoted 10 lines
mm/memory_hotplug.c +10
-0+struct range memhp_range = { + .start =0UL , + .end =-1ULL , + }; +return memhp_range; +} + +struct range memhp_get_pluggable_range (bool need_mapping) +{ +const u64 max_phys = (1ULL << (MAX_PHYSMEM_BITS +1 )) -1 ;Sorry, thought about that line a bit more, and I think this is just wrong (took me longer to realize as it should). The old code used this calculation to print the limit only (in a wrong way), let's recap:
Assume MAX_PHYSMEM_BITS=32
max_phys = (1ULL << (32 + 1)) - 1 = 0x1ffffffffull;Ehm, these are 33 bit.
OTOH, old code checked for
if (max_addr >> MAX_PHYSMEM_BITS) {Which makes sense, because
0x1ffffffffull >> 32 = 1results in "true", meaning it's to big, while
0xffffffffull >> 32 = 0correctly results in "false", meaning the address is fine.
So, this should just be
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
const u64
max_phys= 1ULL <<MAX_PHYSMEM_BITS;(similarly as calculated in virito-mem code, or in
kernel/resource.c)Should this be 1ULL <<
MAX_PHYSMEM_BITS- 1 instead ? Currently there areYes, obviously, sorry, forgot the -1.
const u64 max_phys = (1ULL << MAX_PHYSMEM_BITS) - 1;
to be precise.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v3 2/3 arm64/mm: Define arch_get_mappable_range()
In reply to Anshuman KhandualThis overrides arch_get_mappable_range() on arm64 platform which will be used with recently added generic framework. It drops inside_linear_region() and subsequent check in arch_add_memory() which are no longer required. It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called.
--- arch/arm64/mm/mmu.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
arch/arm64/mm/mmu.c +7 -8
@@ -1442,16 +1442,19 @@ static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size)free_empty_tables (start, end,PAGE_OFFSET ,PAGE_END ); } -static bool inside_linear_region (u64 start,u64 size) +struct range arch_get_mappable_range (void ) { +struct range memhp_range; +/* * Linear mapping region is the range [PAGE_OFFSET..(PAGE_END - 1)] * accommodating both its ends but excluding PAGE_END. Max physical * range which can be mapped inside this linear mapping range, must * also be derived from its end points. */ -return start >=__pa (_PAGE_OFFSET (vabits_actual)) && - (start + size -1 ) <=__pa (PAGE_END -1 ); + memhp_range.start =__pa (_PAGE_OFFSET (vabits_actual)); + memhp_range.end =__pa (PAGE_END -1 ); +return memhp_range; } int arch_add_memory(int nid, u64 start, u64 size,
@@ -1459,11 +1462,7 @@ int arch_add_memory(int nid, u64 start, u64 size, {int ret, flags =0 ; -if (!inside_linear_region (start, size)) { -pr_err ("[%llx %llx] is outside linear mapping region\n" , start, start + size); -return -EINVAL ; - } - +VM_BUG_ON (!memhp_range_allowed (start, size, true));if (rodata_full ||debug_pagealloc_enabled ()) flags =NO_BLOCK_MAPPINGS |NO_CONT_MAPPINGS ;
Signature · 1 line
2.20.1
Re: [PATCH V3 2/3] arm64/mm: Define arch_get_mappable_range()
Quoted 49 lines · Anshuman Khandual
This overrides
arch_get_mappable_range()on arm64 platform which will be used with recently added generic framework. It dropsinside_linear_region()and subsequent check inarch_add_memory()which are no longer required. It also adds aVM_BUG_ON()check that would ensure thatmemhp_range_allowed()has already been called.Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Mark Rutland <[email protected]>Cc David Hildenbrand <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>--- arch/arm64/mm/mmu.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)arch/arm64/mm/mmu.c +6
-3@@ -1442,16 +1442,19 @@ static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size)free_empty_tables (start, end,PAGE_OFFSET ,PAGE_END ); } -static bool inside_linear_region (u64 start,u64 size) +struct range arch_get_mappable_range (void ) { +struct range memhp_range; +/* * Linear mapping region is the range [PAGE_OFFSET..(PAGE_END - 1)] * accommodating both its ends but excluding PAGE_END. Max physical * range which can be mapped inside this linear mapping range, must * also be derived from its end points. */ -return start >=__pa (_PAGE_OFFSET (vabits_actual)) && - (start + size -1 ) <=__pa (PAGE_END -1 ); + memhp_range.start =__pa (_PAGE_OFFSET (vabits_actual)); + memhp_range.end =__pa (PAGE_END -1 ); +return memhp_range; } int arch_add_memory(int nid, u64 start, u64 size,@@ -1459,11 +1462,7 @@ int arch_add_memory(int nid, u64 start, u64 size, {int ret, flags =0 ;
arch/arm64/mm/mmu.c int arch_add_memory(int nid, u64 start, u64 size, · line 1468
Quoted 10 lines
arch/arm64/mm/mmu.c +1
-5-if (!inside_linear_region (start, size)) { -pr_err ("[%llx %llx] is outside linear mapping region\n" , start, start + size); -return -EINVAL ; - } - +VM_BUG_ON (!memhp_range_allowed (start, size, true));if (rodata_full ||debug_pagealloc_enabled ()) flags =NO_BLOCK_MAPPINGS |NO_CONT_MAPPINGS ;
Signature · 3 lines
Thanks, David / dhildenb
PATCH v3 3/3 s390/mm: Define arch_get_mappable_range()
In reply to Anshuman KhandualThis overrides arch_get_mappabble_range() on s390 platform which will be used with recently added generic framework. It modifies the existing range check in vmem_add_mapping() using arch_get_mappable_range(). It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called on the hotplug path.
--- arch/s390/mm/init.c | 1 + arch/s390/mm/vmem.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
arch/s390/mm/init.c +1 -0
@@ -297,6 +297,7 @@ int arch_add_memory(int nid, u64 start, u64 size,if (WARN_ON_ONCE (params->pgprot.pgprot !=PAGE_KERNEL .pgprot))return -EINVAL ; +VM_BUG_ON (!memhp_range_allowed (start, size, true)); rc =vmem_add_mapping (start, size);if (rc)return rc;
arch/s390/mm/vmem.c +14 -1
@@ -4,6 +4,7 @@* Author(s): Heiko Carstens <[email protected]> */ +#include <linux/memory_hotplug.h> #include <linux/memblock.h> #include <linux/pfn.h> #include <linux/mm.h>
@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)mutex_unlock (&vmem_mutex); } +struct range arch_get_mappable_range (void ) +{ +struct range memhp_range; + + memhp_range.start =0 ; + memhp_range.end =VMEM_MAX_PHYS -1 ; +return memhp_range; +} +int vmem_add_mapping (unsigned long start,unsigned long size) { +struct range range;int ret; -if (start + size >VMEM_MAX_PHYS || + range =arch_get_mappable_range (); +if (start < range.start || + start + size > range.end +1 || start + size < start)return -ERANGE ;
Signature · 1 line
2.20.1
Re: [PATCH V3 3/3] s390/mm: Define arch_get_mappable_range()
Quoted 43 lines · Anshuman Khandual
This overrides
arch_get_mappabble_range()on s390 platform which will be used with recently added generic framework. It modifies the existing range check invmem_add_mapping()usingarch_get_mappable_range(). It also adds aVM_BUG_ON()check that would ensure thatmemhp_range_allowed()has already been called on the hotplug path.Cc Heiko Carstens <[email protected]>Cc Vasily Gorbik <[email protected]>Cc David Hildenbrand <[email protected]>Acked-by Heiko Carstens <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>--- arch/s390/mm/init.c | 1 + arch/s390/mm/vmem.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/s390/mm/init.c +1
-0@@ -297,6 +297,7 @@ int arch_add_memory(int nid, u64 start, u64 size,if (WARN_ON_ONCE (params->pgprot.pgprot !=PAGE_KERNEL .pgprot))return -EINVAL ; +VM_BUG_ON (!memhp_range_allowed (start, size, true)); rc =vmem_add_mapping (start, size);if (rc)return rc;arch/s390/mm/vmem.c +1
-0@@ -4,6 +4,7 @@* Author(s): Heiko Carstens <[email protected]> */ +#include <linux/memory_hotplug.h> #include <linux/memblock.h> #include <linux/pfn.h> #include <linux/mm.h>
arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 538
Quoted 6 lines
arch/s390/mm/vmem.c +3
-0@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)mutex_unlock (&vmem_mutex); } +struct range arch_get_mappable_range (void ) +{ +struct range memhp_range;
You could do:
memhp_range = {
.start = 0,
.end = VMEM_MAX_PHYS - 1,
};Similar in the arm64 patch.
arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 550
Quoted 13 lines
arch/s390/mm/vmem.c +8
-1+ + memhp_range.start =0 ; + memhp_range.end =VMEM_MAX_PHYS -1 ; +return memhp_range; +} +int vmem_add_mapping (unsigned long start,unsigned long size) { +struct range range;int ret; - if (start + size > VMEM_MAX_PHYS || + range =arch_get_mappable_range ();
You could do
struct range range = arch_get_mappable_range();
Quoted 6 lines
arch/s390/mm/vmem.c +2
-0+if (start < range.start || + start + size > range.end +1 || start + size < start)return -ERANGE ;
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH V3 3/3] s390/mm: Define arch_get_mappable_range()
Quoted 53 lines · David Hildenbrand
Quoted 43 lines · Anshuman Khandual
This overrides
arch_get_mappabble_range()on s390 platform which will be used with recently added generic framework. It modifies the existing range check invmem_add_mapping()usingarch_get_mappable_range(). It also adds aVM_BUG_ON()check that would ensure thatmemhp_range_allowed()has already been called on the hotplug path.Cc Heiko Carstens <[email protected]>Cc Vasily Gorbik <[email protected]>Cc David Hildenbrand <[email protected]>Acked-by Heiko Carstens <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>--- arch/s390/mm/init.c | 1 + arch/s390/mm/vmem.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/s390/mm/init.c +1
-0@@ -297,6 +297,7 @@ int arch_add_memory(int nid, u64 start, u64 size,if (WARN_ON_ONCE (params->pgprot.pgprot !=PAGE_KERNEL .pgprot))return -EINVAL ; +VM_BUG_ON (!memhp_range_allowed (start, size, true)); rc =vmem_add_mapping (start, size);if (rc)return rc;arch/s390/mm/vmem.c +1
-0@@ -4,6 +4,7 @@* Author(s): Heiko Carstens <[email protected]> */ +#include <linux/memory_hotplug.h> #include <linux/memblock.h> #include <linux/pfn.h> #include <linux/mm.h> arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 538
Quoted 6 lines
arch/s390/mm/vmem.c +3
-0@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)mutex_unlock (&vmem_mutex); } +struct range arch_get_mappable_range (void ) +{ +struct range memhp_range;You could do:
Quoted 6 lines
memhp_range = { .start = 0, .end = VMEM_MAX_PHYS - 1, };Similar in the arm64 patch.
There is a comment block just before this assignment on arm64. Also it seems like code style preference and Heiko had originally agreed on this particular patch. Could we just leave it unchanged please ?
Quoted 12 lines
Quoted 12 lines
arch/s390/mm/vmem.c +7
-0+ + memhp_range.start =0 ; + memhp_range.end =VMEM_MAX_PHYS -1 ; +return memhp_range; +} +int vmem_add_mapping (unsigned long start,unsigned long size) { +struct range range;int ret;
Quoted 6 lines
arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 550
Quoted 2 lines
arch/s390/mm/vmem.c +1
-1- if (start + size > VMEM_MAX_PHYS || + range =arch_get_mappable_range ();You could do
struct range range =
arch_get_mappable_range();
Sure, will change this though.
Quoted 9 lines
Quoted 7 lines
arch/s390/mm/vmem.c +2
-0+if (start < range.start || + start + size > range.end +1 || start + size < start)return -ERANGE ;
Re: [PATCH V3 3/3] s390/mm: Define arch_get_mappable_range()
Quoted 61 lines · Anshuman Khandual
Quoted 58 lines · David Hildenbrand
Quoted 43 lines · Anshuman Khandual
This overrides
arch_get_mappabble_range()on s390 platform which will be used with recently added generic framework. It modifies the existing range check invmem_add_mapping()usingarch_get_mappable_range(). It also adds aVM_BUG_ON()check that would ensure thatmemhp_range_allowed()has already been called on the hotplug path.Cc Heiko Carstens <[email protected]>Cc Vasily Gorbik <[email protected]>Cc David Hildenbrand <[email protected]>Acked-by Heiko Carstens <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>--- arch/s390/mm/init.c | 1 + arch/s390/mm/vmem.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)arch/s390/mm/init.c +1
-0@@ -297,6 +297,7 @@ int arch_add_memory(int nid, u64 start, u64 size,if (WARN_ON_ONCE (params->pgprot.pgprot !=PAGE_KERNEL .pgprot))return -EINVAL ; +VM_BUG_ON (!memhp_range_allowed (start, size, true)); rc =vmem_add_mapping (start, size);if (rc)return rc;arch/s390/mm/vmem.c +1
-0@@ -4,6 +4,7 @@* Author(s): Heiko Carstens <[email protected]> */ +#include <linux/memory_hotplug.h> #include <linux/memblock.h> #include <linux/pfn.h> #include <linux/mm.h> arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 538
Quoted 6 lines
arch/s390/mm/vmem.c +3
-0@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)mutex_unlock (&vmem_mutex); } +struct range arch_get_mappable_range (void ) +{ +struct range memhp_range;You could do:
memhp_range = { .start = 0, .end = VMEM_MAX_PHYS - 1, };
Quoted 5 lines
Quoted 1 line
Similar in the arm64 patch.
There is a comment block just before this assignment on arm64. Also it seems like code style preference and Heiko had originally agreed on this particular patch. Could we just leave it unchanged please ?
That's not how review works. But as I said, "You could do".
Signature · 3 lines
Thanks, David / dhildenb
RFC PATCH 1/1 virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
In reply to Anshuman KhandualFrom: David Hildenbrand <[email protected]>
Right now, we only check against MAX_PHYSMEM_BITS - but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device operation.
So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.
In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.
--- drivers/virtio/virtio_mem.c | 40 +++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 13 deletions(-)
drivers/virtio/virtio_mem.c +27 -13
@@ -2222,7 +2222,7 @@ static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm)*/ static void virtio_mem_refresh_config (struct virtio_mem *vm) { -const uint64_t phys_limit =1UL <<MAX_PHYSMEM_BITS ; +const struct range pluggable_range =memhp_get_pluggable_range (true);uint64_t new_plugged_size, usable_region_size, end_addr;/* the plugged_size is just a reflection of what _we_ did previously */
@@ -2234,15 +2234,25 @@ static void virtio_mem_refresh_config(struct virtio_mem *vm)/* calculate the last usable memory block id */ virtio_cread_le(vm->vdev, struct virtio_mem_config, usable_region_size, &usable_region_size); - end_addr = vm->addr + usable_region_size; - end_addr =min (end_addr, phys_limit); + end_addr =min (vm->addr + usable_region_size -1 , + pluggable_range.end); -if (vm->in_sbm) - vm->sbm.last_usable_mb_id = -virtio_mem_phys_to_mb_id (end_addr) -1 ; -else - vm->bbm.last_usable_bb_id = -virtio_mem_phys_to_bb_id (vm, end_addr) -1 ; +if (vm->in_sbm) { + vm->sbm.last_usable_mb_id =virtio_mem_phys_to_mb_id (end_addr); +if (!IS_ALIGNED (end_addr +1 ,memory_block_size_bytes ())) + vm->sbm.last_usable_mb_id--; + }else { + vm->bbm.last_usable_bb_id =virtio_mem_phys_to_bb_id (vm, + end_addr); +if (!IS_ALIGNED (end_addr +1 , vm->bbm.bb_size)) + vm->bbm.last_usable_bb_id--; + } +/* +* If we cannot plug any of our device memory (e.g., nothing in the +* usable region is addressable), the last usable memory block id will +* be smaller than the first usable memory block id. We'll stop +* attempting to add memory with -ENOSPC from our main loop. +*/ /* see if there is a request to change the size */ virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size,
@@ -2364,6 +2374,7 @@ static int virtio_mem_init_vq(struct virtio_mem *vm)static int virtio_mem_init (struct virtio_mem *vm) { +const struct range pluggable_range =memhp_get_pluggable_range (true);const uint64_t phys_limit =1UL <<MAX_PHYSMEM_BITS ;uint64_t sb_size, addr;uint16_t node_id;
@@ -2405,9 +2416,10 @@ static int virtio_mem_init(struct virtio_mem *vm)if (!IS_ALIGNED (vm->addr + vm->region_size,memory_block_size_bytes ()))dev_warn (&vm->vdev->dev,"The alignment of the physical end address can make some memory unusable.\n" ); -if (vm->addr + vm->region_size > phys_limit) +if (vm->addr < pluggable_range.start || + vm->addr + vm->region_size -1 > pluggable_range.end)dev_warn (&vm->vdev->dev, -"Some memory is not addressable. This can make some memory unusable.\n" ); +"Some device memory is not addressable/pluggable. This can make some memory unusable.\n" );/* * We want subblocks to span at least MAX_ORDER_NR_PAGES and
@@ -2429,7 +2441,8 @@ static int virtio_mem_init(struct virtio_mem *vm) vm->sbm.sb_size;/* Round up to the next full memory block */ - addr = vm->addr +memory_block_size_bytes () -1 ; + addr =max_t (uint64_t, vm->addr, pluggable_range.start) + +memory_block_size_bytes () -1 ; vm->sbm.first_mb_id =virtio_mem_phys_to_mb_id (addr); vm->sbm.next_mb_id = vm->sbm.first_mb_id; }else {
@@ -2450,7 +2463,8 @@ static int virtio_mem_init(struct virtio_mem *vm) }/* Round up to the next aligned big block */ - addr = vm->addr + vm->bbm.bb_size -1 ; + addr =max_t (uint64_t, vm->addr, pluggable_range.start) + + vm->bbm.bb_size -1 ; vm->bbm.first_bb_id =virtio_mem_phys_to_bb_id (vm, addr); vm->bbm.next_bb_id = vm->bbm.first_bb_id; }
Signature · 1 line
2.20.1
Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
Quoted 29 lines · Anshuman Khandual
From: David Hildenbrand <[email protected]>
Right now, we only check against
MAX_PHYSMEM_BITS- but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like-E2BIGor-ERANGEfromadd_memory_driver_managed(), stopping device operation.So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.
In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.
Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Wei Yang <[email protected]>Cc Andrew Morton <[email protected]>Cc teawater <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Jonathan Cameron <[email protected]>
Quoted 8 lines
Cc Vasily Gorbik <[email protected]>Cc Will Deacon <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Mark Rutland <[email protected]>Cc Heiko Carstens <[email protected]>Cc Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>
Hello David,
As your original patch was in the RFC state, I have just maintained the same here as well. But once you test this patch along with the new series, please do let me know if this needs to be converted to a normal PATCH instead. Thank you.
- Anshuman
Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
Quoted 41 lines · Anshuman Khandual
Quoted 29 lines · Anshuman Khandual
From: David Hildenbrand <[email protected]>
Right now, we only check against
MAX_PHYSMEM_BITS- but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like-E2BIGor-ERANGEfromadd_memory_driver_managed(), stopping device operation.So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.
In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.
Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Wei Yang <[email protected]>Cc Andrew Morton <[email protected]>Cc teawater <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Jonathan Cameron <[email protected]>Quoted 8 lines
Cc Vasily Gorbik <[email protected]>Cc Will Deacon <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Mark Rutland <[email protected]>Cc Heiko Carstens <[email protected]>Cc Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>
Quoted 6 lines
Hello David,
As your original patch was in the RFC state, I have just maintained the same here as well. But once you test this patch along with the new series, please do let me know if this needs to be converted to a normal PATCH instead. Thank you.
I'll give it a churn on x86-64, where not that much should change. It will be interesting to test with arm64 in such corner cases in the future.
Thanks
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
In reply to Anshuman KhandualQuoted 41 lines · Anshuman Khandual
Quoted 29 lines · Anshuman Khandual
From: David Hildenbrand <[email protected]>
Right now, we only check against
MAX_PHYSMEM_BITS- but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like-E2BIGor-ERANGEfromadd_memory_driver_managed(), stopping device operation.So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.
In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.
Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Wei Yang <[email protected]>Cc Andrew Morton <[email protected]>Cc teawater <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Jonathan Cameron <[email protected]>Quoted 8 lines
Cc Vasily Gorbik <[email protected]>Cc Will Deacon <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Mark Rutland <[email protected]>Cc Heiko Carstens <[email protected]>Cc Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>
Quoted 6 lines
Hello David,
As your original patch was in the RFC state, I have just maintained the same here as well. But once you test this patch along with the new series, please do let me know if this needs to be converted to a normal PATCH instead. Thank you.
Yes, you can drop the RFC part. I assume you'll send another revision, I'll do another test there, thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug
Quoted 44 lines · David Hildenbrand
Quoted 43 lines · Anshuman Khandual
Quoted 29 lines · Anshuman Khandual
From: David Hildenbrand <[email protected]>
Right now, we only check against
MAX_PHYSMEM_BITS- but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like-E2BIGor-ERANGEfromadd_memory_driver_managed(), stopping device operation.So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.
In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.
Cc "Michael S. Tsirkin" <[email protected]>Cc Jason Wang <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Wei Yang <[email protected]>Cc Andrew Morton <[email protected]>Cc teawater <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Pankaj Gupta <[email protected]>Cc Jonathan Cameron <[email protected]>Quoted 8 lines
Cc Vasily Gorbik <[email protected]>Cc Will Deacon <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Mark Rutland <[email protected]>Cc Heiko Carstens <[email protected]>Cc Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Signed-off-by Anshuman Khandual <[email protected]>Hello David,
Quoted 7 lines
Quoted 4 lines
As your original patch was in the RFC state, I have just maintained the same here as well. But once you test this patch along with the new series, please do let me know if this needs to be converted to a normal PATCH instead. Thank you.
Yes, you can drop the RFC part. I assume you'll send another revision, I'll do another test there, thanks!
Sure, will drop the RFC in next version.
Quoted 19 lines · Anshuman Khandual
This series adds a mechanism allowing platforms to weigh in and prevalidate incoming address range before proceeding further with the memory hotplug. This helps prevent potential platform errors for the given address range, down the hotplug call chain, which inevitably fails the hotplug itself.
This mechanism was suggested by David Hildenbrand during another discussion with respect to a memory hotplug fix on arm64 platform.
https://lore.kernel.org/linux-arm-kernel/[email protected]/
This mechanism focuses on the addressibility aspect and not [sub] section alignment aspect. Hence
check_hotplug_memory_range()andcheck_pfn_span()have been left unchanged. Wondering if all these can still be unified in an expandedmemhp_range_allowed()check, that can be called from multiple memory hot add and remove paths.This series applies on v5.11-rc4 and has been tested on arm64. But only build tested on s390.
Quoted 7 lines
Changes in V3
- Updated the commit message in [PATCH 1/3]
- Replaced 1 with 'true' and 0 with 'false' inmemhp_range_allowed()
- Updated memhp_range.end asVMEM_MAX_PHYS- 1 and updatedvmem_add_mapping()on s390
- Changedmemhp_range_allowed()behaviour in__add_pages()
- Updated__add_pages()to return E2BIG whenmemhp_range_allowed()fails for non-linear mapping based requests
Minor thing, we should make up our mind if we want to call stuff internally "memhp_" or "mhp". I prefer the latter, because it is shorter.
Signature · 3 lines
Thanks, David / dhildenb
Oscar Salvador
maintainerQuoted 2 lines · David Hildenbrand
Minor thing, we should make up our mind if we want to call stuff internally "
memhp_" or "mhp". I prefer the latter, because it is shorter.
I would rather use the latter as well. I used that in [1]. MEMHP_MERGE_RESOURCE should be renamed if we agree on that.
[1] https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/
Signature · 2 lines
Oscar Salvador SUSE L3
Quoted 5 lines · Oscar Salvador
Quoted 2 lines · David Hildenbrand
Minor thing, we should make up our mind if we want to call stuff internally "
memhp_" or "mhp". I prefer the latter, because it is shorter.I would rather use the latter as well. I used that in [1].
Okay, will change all that is 'memhp' as 'mhp' instead.
Quoted 5 lines
MEMHP_MERGE_RESOURCEshould be renamed if we agree on that.[1] https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/
Quoted 15 lines · Anshuman Khandual
Quoted 5 lines · Oscar Salvador
Quoted 2 lines · David Hildenbrand
Minor thing, we should make up our mind if we want to call stuff internally "
memhp_" or "mhp". I prefer the latter, because it is shorter.I would rather use the latter as well. I used that in [1].
Okay, will change all that is 'memhp' as 'mhp' instead.
Quoted 4 lines
MEMHP_MERGE_RESOURCEshould be renamed if we agree on that.[1] https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/
While replacing 'memhp' as 'mhp' in this series, noticed there are some more 'memhp' scattered around the code from earlier. A mix of both 'memhp' and 'mhp' might not be a good idea. Hence should we just change these remaining 'memhp' as 'mhp' as well and possibly also MEMHP_MERGE_RESOURCE as suggested earlier, in a subsequent clean up patch ? Would there be a problem with memhp_default_state being a command line parameter ?
From a tree wide search -
Documentation/admin-guide/kernel-parameters.txt: memhp_default_state=online/offline
drivers/base/memory.c:int memhp_online_type_from_str(const char *str)
drivers/base/memory.c: const int online_type = memhp_online_type_from_str(buf); drivers/base/memory.c: online_type_to_str[memhp_default_online_type]); drivers/base/memory.c: const int online_type = memhp_online_type_from_str(buf); drivers/base/memory.c: memhp_default_online_type = online_type;
include/linux/memory_hotplug.h:extern int memhp_online_type_from_str(const char *str); include/linux/memory_hotplug.h:extern int memhp_default_online_type; mm/memory_hotplug.c:int memhp_default_online_type = MMOP_OFFLINE; mm/memory_hotplug.c:int memhp_default_online_type = MMOP_ONLINE; mm/memory_hotplug.c:static int __init setup_memhp_default_state(char *str)
mm/memory_hotplug.c: const int online_type = memhp_online_type_from_str(str);
mm/memory_hotplug.c: memhp_default_online_type = online_type;
mm/memory_hotplug.c:__setup("memhp_default_state=", setup_memhp_default_state);
mm/memory_hotplug.c: mem->online_type = memhp_default_online_type;
mm/memory_hotplug.c: if (memhp_default_online_type != MMOP_OFFLINE)Quoted 15 lines · Anshuman Khandual
Quoted 13 lines · Anshuman Khandual
Quoted 5 lines · Oscar Salvador
Quoted 2 lines · David Hildenbrand
Minor thing, we should make up our mind if we want to call stuff internally "
memhp_" or "mhp". I prefer the latter, because it is shorter.I would rather use the latter as well. I used that in [1].
Okay, will change all that is 'memhp' as 'mhp' instead.
Quoted 2 lines
MEMHP_MERGE_RESOURCEshould be renamed if we agree on that.
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
[1] https://patchwork.kernel.org/project/linux-mm/cover/[email protected]/
While replacing 'memhp' as 'mhp' in this series, noticed there are some more 'memhp' scattered around the code from earlier. A mix of both 'memhp' and 'mhp' might not be a good idea. Hence should we just change these remaining 'memhp' as 'mhp' as well and possibly also
MEMHP_MERGE_RESOURCEas suggested earlier, in a subsequent
As mentioned in another thread to Oscar, I already have a cleanup patch for that one lying around, part of a bigger series. Might just send that one out separately earlier.
Quoted 2 lines
clean up patch ? Would there be a problem with
memhp_default_statebeing a command line parameter ?
Yes, that one we should not change, to not break existing cmdlines without good reason. We could change the memhp_default_online_type/memhp_online_type_from_str/... thingies, though.
Feel free to send a patch, thanks!
Signature · 3 lines
Thanks, David / dhildenb