Skip to contentSkip to conversation

mm/hotplug: Pre-validate the address range with platform

PATCH Superseded Anshuman Khandual 4 participants 16 messages Dec 17, 2020 – Jan 13, 2021

Subsystems (7)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover mm/hotplug: Pre-validate the address range with platform

This series adds a mechanism allowing platforms to weigh in and prevalidate incoming address range before proceeding further with the memory hotplug. This helps prevent potential platform errors for the given address range, down the hotplug call chain, which inevitably fails the hotplug itself.

This mechanism was suggested by David Hildenbrand during another discussion with respect to a memory hotplug fix on arm64 platform.

https://lore.kernel.org/linux-arm-kernel/[email protected]/

This mechanism focuses on the addressibility aspect and not [sub] section alignment aspect. Hence check_hotplug_memory_range() and check_pfn_span() have been left unchanged. Wondering if all these can still be unified in an expanded memhp_range_allowed() check, that can be called from multiple memory hot add and remove paths.

This series applies on v5.10 and has been tested on arm64. But only build tested on s390.

Changes in V2:

- Changed s390 version per Heiko and updated the commit message
- Called memhp_range_allowed() only for arch_add_memory() in pagemap_range()
- Exported the symbol memhp_get_pluggable_range()

Changes in V1:

https://lore.kernel.org/linux-mm/[email protected]/

- Fixed build problems with (MEMORY_HOTPLUG & !MEMORY_HOTREMOVE)
- Added missing prototype for arch_get_mappable_range()
- Added VM_BUG_ON() check for memhp_range_allowed() in arch_add_memory() per David

Changes in RFC V2:

https://lore.kernel.org/linux-mm/[email protected]/

Incorporated all review feedbacks from David.

- Added additional range check in __segment_load() on s390 which was lost
- Changed is_private init in pagemap_range()
- Moved the framework into mm/memory_hotplug.c
- Made arch_get_addressable_range() a __weak function
- Renamed arch_get_addressable_range() as arch_get_mappable_range()
- Callback arch_get_mappable_range() only handles range requiring linear mapping
- Merged multiple memhp_range_allowed() checks in register_memory_resource()
- Replaced WARN() with pr_warn() in memhp_range_allowed()
- Replaced error return code ERANGE with E2BIG

Changes in RFC V1:

https://lore.kernel.org/linux-mm/[email protected]/

Cc Heiko Carstens <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Mark Rutland <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>

Anshuman Khandual (3):
mm/hotplug: Prevalidate the address range being added with platform arm64/mm: Define arch_get_mappable_range()
s390/mm: Define arch_get_mappable_range()

 arch/arm64/mm/mmu.c            | 15 +++----
 arch/s390/mm/init.c            |  1 +
 arch/s390/mm/vmem.c            | 15 ++++++-
 include/linux/memory_hotplug.h | 10 +++++
 mm/memory_hotplug.c            | 79 +++++++++++++++++++++++++---------
 mm/memremap.c                  |  6 +++
 6 files changed, 97 insertions(+), 29 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 1/3 mm/hotplug: Prevalidate the address range being added with platform

This introduces memhp_range_allowed() which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Then memhp_range_allowed() calls memhp_get_pluggable_range() which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callback arch_get_mappable_range() which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.

This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant check_hotplug_memory_addressable() check in __add_pages() but instead adds a VM_BUG_ON() check which would ensure that the range has been validated with memhp_range_allowed() earlier in the call chain. Besides memhp_get_pluggable_range() also can be used by potential memory hotplug callers to avail the allowed physical range which would go through on a given platform.

This does not really add any new range check in generic memory hotplug but instead compensates for lost checks in arch_add_memory() where applicable and check_hotplug_memory_addressable(), with unified memhp_range_allowed().

---
 include/linux/memory_hotplug.h | 10 +++++
 mm/memory_hotplug.c            | 79 +++++++++++++++++++++++++---------
 mm/memremap.c                  |  6 +++
 3 files changed, 75 insertions(+), 20 deletions(-)
include/linux/memory_hotplug.h +10 -0
index 551093b74596..8d72354758c8 100644
@@ -70,6 +70,9 @@ typedef int __bitwise mhp_t;
  */
 #define MEMHP_MERGE_RESOURCE	((__force mhp_t)BIT(0))
 
+bool memhp_range_allowed(u64 start, u64 size, bool need_mapping);
+struct range memhp_get_pluggable_range(bool need_mapping);
+
 /*
  * Extended parameters for memory hotplug:
  * altmap: alternative allocator for memmap array (optional)
@@ -281,6 +284,13 @@ static inline bool movable_node_is_enabled(void)
 }
 #endif /* ! CONFIG_MEMORY_HOTPLUG */
 
+/*
+ * Keep this declaration outside CONFIG_MEMORY_HOTPLUG as some
+ * platforms might override and use arch_get_mappable_range()
+ * for internal non memory hotplug purposes.
+ */
+struct range arch_get_mappable_range(void);
+
 #if defined(CONFIG_MEMORY_HOTPLUG) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
 /*
  * pgdat resizing functions
mm/memory_hotplug.c +59 -20

103 lines not on this page · raw message

mm/memremap.c +6 -0
index 16b2fb482da1..8c5500030592 100644
@@ -253,6 +253,12 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params,
 			goto err_kasan;
 		}
 
+		if (!memhp_range_allowed(range->start, range_len(range), 1)) {
+			error = -ERANGE;
+			mem_hotplug_done();
+			goto err_add_memory;
+		}
+
 		error = arch_add_memory(nid, range->start, range_len(range),
 					params);
 	}
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

Quoted 17 lines · Anshuman Khandual

This introduces memhp_range_allowed() which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Then memhp_range_allowed() calls memhp_get_pluggable_range() which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callback arch_get_mappable_range() which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.

This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant check_hotplug_memory_addressable() check in __add_pages() but instead adds a VM_BUG_ON() check which would ensure that the range has been validated with memhp_range_allowed() earlier in the call chain. Besides memhp_get_pluggable_range() also can be used by potential memory hotplug callers to avail the allowed physical range which would go through on a given platform.

Quoted 4 lines

This does not really add any new range check in generic memory hotplug but instead compensates for lost checks in arch_add_memory() where applicable and check_hotplug_memory_addressable(), with unified memhp_range_allowed().

Subject s/mm\/hotplug/mm\/memory_hotplug/

Everywhere in this patch: Use "true/false" for boolean values.

Quoted 17 lines
---
 include/linux/memory_hotplug.h | 10 +++++
 mm/memory_hotplug.c            | 79 +++++++++++++++++++++++++---------
 mm/memremap.c                  |  6 +++
 3 files changed, 75 insertions(+), 20 deletions(-)
include/linux/memory_hotplug.h +0 -0
index 551093b74596..8d72354758c8 100644

include/linux/memory_hotplug.h typedef int __bitwise mhp_t; · line 74

Quoted 5 lines
include/linux/memory_hotplug.h +2 -0
@@ -70,6 +70,9 @@ typedef int __bitwise mhp_t;
  */
 #define MEMHP_MERGE_RESOURCE	((__force mhp_t)BIT(0))
 
+bool memhp_range_allowed(u64 start, u64 size, bool need_mapping);
+struct range memhp_get_pluggable_range(bool need_mapping);

AFAIKs, all memhp_get_pluggable_range() users pass "1".

What about the "add_pages()-only" path?

Signature · 3 lines
Thanks,

David / dhildenb
maintainer
#3 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

Quoted 3 lines · David Hildenbrand

AFAIKs, all memhp_get_pluggable_range() users pass "1".

What about the "add_pages()-only" path?

I guess you refer to memremap_pages(), right?
If so, moving the added memhp_range_allowed() check above the if-else might do the trick

Signature · 2 lines
Oscar Salvador
SUSE L3
#4 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

Quoted 6 lines · Oscar Salvador
Quoted 3 lines · David Hildenbrand

AFAIKs, all memhp_get_pluggable_range() users pass "1".

What about the "add_pages()-only" path?

I guess you refer to memremap_pages(), right?

Right, via pagemap_range().

Quoted 3 lines

If so, moving the added memhp_range_allowed() check above the if-else might do the trick

We had that code in the earlier version. But dropped it, as we did not want to add any new checks in the generic code. Can add it back if that is preferred.

#5 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

Quoted 10 lines · Anshuman Khandual
Quoted 6 lines · Oscar Salvador
Quoted 3 lines · David Hildenbrand

AFAIKs, all memhp_get_pluggable_range() users pass "1".

What about the "add_pages()-only" path?

I guess you refer to memremap_pages(), right?

Quoted 8 lines

Right, via pagemap_range().

Quoted 3 lines

If so, moving the added memhp_range_allowed() check above the if-else might do the trick

We had that code in the earlier version. But dropped it, as we did not want to add any new checks in the generic code. Can add it back if that is preferred.

I remember discussing replacing the check in __add_pages() instead. But I don't really care where the check ends up being. As discussed, at some point, we should provide versions of add_pages() and arch_add_pages() that don't immediately end in arch-code.

Signature · 3 lines
Thanks,

David / dhildenb
#6 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

Quoted 13 lines · David Hildenbrand
Quoted 12 lines · Anshuman Khandual
Quoted 6 lines · Oscar Salvador
Quoted 3 lines · David Hildenbrand

AFAIKs, all memhp_get_pluggable_range() users pass "1".

What about the "add_pages()-only" path?

I guess you refer to memremap_pages(), right?

Right, via pagemap_range().

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

If so, moving the added memhp_range_allowed() check above the if-else might do the trick

We had that code in the earlier version. But dropped it, as we did not want to add any new checks in the generic code. Can add it back if that is preferred.

I remember discussing replacing the check in __add_pages() instead. But

The proposed change for __add_pages() now seems misleading. Instead of VM_BUG_ON(), memhp_range_allowed() should be checked directly for a non linear mapping i.e with 'false' argument and return prematurely in case that check fails.

s/VM_BUG_ON(!memhp_range_allowed(.., 1)/!memhp_range_allowed(.., 0)/

/*
* Reasonably generic function for adding memory. It is
* expected that archs that support memory hotplug will
@@ -317,10 +304,7 @@ int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages,
if (WARN_ON_ONCE(!params->pgprot.pgprot))
return -EINVAL;

- err = check_hotplug_memory_addressable(pfn, nr_pages);
- if (err)
- return err;
-
+ VM_BUG_ON(!memhp_range_allowed(PFN_PHYS(pfn), nr_pages * PAGE_SIZE, 1));
if (altmap) {
/*
* Validate altmap is within bounds of the total request
@@ -1181,6 +1165,61 @@ int add_memory_driver_managed(int nid, u64 start, u64 size,

Quoted 3 lines

I don't really care where the check ends up being. As discussed, at some point, we should provide versions of add_pages() and arch_add_pages() that don't immediately end in arch-code.

Sure. But for now, AFAICS the above replacement should be sufficient.

#7 lore raw

Re: [PATCH V2 1/3] mm/hotplug: Prevalidate the address range being added with platform

In reply to David Hildenbrand
Quoted 18 lines · David Hildenbrand
Quoted 17 lines · Anshuman Khandual

This introduces memhp_range_allowed() which can be called in various memory hotplug paths to prevalidate the address range which is being added, with the platform. Then memhp_range_allowed() calls memhp_get_pluggable_range() which provides applicable address range depending on whether linear mapping is required or not. For ranges that require linear mapping, it calls a new arch callback arch_get_mappable_range() which the platform can override. So the new callback, in turn provides the platform an opportunity to configure acceptable memory hotplug address ranges in case there are constraints.

This mechanism will help prevent platform specific errors deep down during hotplug calls. This drops now redundant check_hotplug_memory_addressable() check in __add_pages() but instead adds a VM_BUG_ON() check which would ensure that the range has been validated with memhp_range_allowed() earlier in the call chain. Besides memhp_get_pluggable_range() also can be used by potential memory hotplug callers to avail the allowed physical range which would go through on a given platform.

Quoted 6 lines
Quoted 4 lines

This does not really add any new range check in generic memory hotplug but instead compensates for lost checks in arch_add_memory() where applicable and check_hotplug_memory_addressable(), with unified memhp_range_allowed().

Subject s/mm\/hotplug/mm\/memory_hotplug/

Sure, will do.

Quoted 2 lines

Everywhere in this patch: Use "true/false" for boolean values.

Sure, will change.

Quoted 21 lines
Quoted 21 lines
---
 include/linux/memory_hotplug.h | 10 +++++
 mm/memory_hotplug.c            | 79 +++++++++++++++++++++++++---------
 mm/memremap.c                  |  6 +++
 3 files changed, 75 insertions(+), 20 deletions(-)
include/linux/memory_hotplug.h +0 -0
index 551093b74596..8d72354758c8 100644
@@ -70,6 +70,9 @@ typedef int __bitwise mhp_t;
  */
 #define MEMHP_MERGE_RESOURCE	((__force mhp_t)BIT(0))
 
Quoted 4 lines

include/linux/memory_hotplug.h typedef int __bitwise mhp_t; · line 74

Quoted 2 lines
include/linux/memory_hotplug.h +2 -0
+bool memhp_range_allowed(u64 start, u64 size, bool need_mapping);
+struct range memhp_get_pluggable_range(bool need_mapping);

AFAIKs, all memhp_get_pluggable_range() users pass "1".

Right.

Quoted 2 lines

What about the "add_pages()-only" path?

I had dropped memhp_range_allowed() changes for add_pages() in pagemap_range() because you had mentioned not to add any new checks in the generic code. Will add it back if that is preferred.

#8 lore raw

PATCH v2 2/3 arm64/mm: Define arch_get_mappable_range()

In reply to Anshuman Khandual

This overrides arch_get_mappable_range() on arm64 platform which will be used with recently added generic framework. It drops inside_linear_region() and subsequent check in arch_add_memory() which are no longer required. It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called.

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Mark Rutland <[email protected]>
Cc David Hildenbrand <[email protected]>
---
 arch/arm64/mm/mmu.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
arch/arm64/mm/mmu.c +7 -8
index ca692a815731..54705dcacb4e 100644
@@ -1444,16 +1444,19 @@ static void __remove_pgd_mapping(pgd_t *pgdir, unsigned long start, u64 size)
 	free_empty_tables(start, end, PAGE_OFFSET, PAGE_END);
 }
 
-static bool inside_linear_region(u64 start, u64 size)
+struct range arch_get_mappable_range(void)
 {
+	struct range memhp_range;
+
 	/*
 	 * Linear mapping region is the range [PAGE_OFFSET..(PAGE_END - 1)]
 	 * accommodating both its ends but excluding PAGE_END. Max physical
 	 * range which can be mapped inside this linear mapping range, must
 	 * also be derived from its end points.
 	 */
-	return start >= __pa(_PAGE_OFFSET(vabits_actual)) &&
-	       (start + size - 1) <= __pa(PAGE_END - 1);
+	memhp_range.start = __pa(_PAGE_OFFSET(vabits_actual));
+	memhp_range.end =  __pa(PAGE_END - 1);
+	return memhp_range;
 }
 
 int arch_add_memory(int nid, u64 start, u64 size,
@@ -1461,11 +1464,7 @@ int arch_add_memory(int nid, u64 start, u64 size,
 {
 	int ret, flags = 0;
 
-	if (!inside_linear_region(start, size)) {
-		pr_err("[%llx %llx] is outside linear mapping region\n", start, start + size);
-		return -EINVAL;
-	}
-
+	VM_BUG_ON(!memhp_range_allowed(start, size, 1));
 	if (rodata_full || debug_pagealloc_enabled())
 		flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
 
Signature · 1 line
2.20.1
#9 lore raw

PATCH v2 3/3 s390/mm: Define arch_get_mappable_range()

In reply to Anshuman Khandual

This overrides arch_get_mappabble_range() on s390 platform which will be used with recently added generic framework. It modifies the existing range check in vmem_add_mapping() using arch_get_mappable_range(). It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called on the hotplug path.

---
 arch/s390/mm/init.c |  1 +
 arch/s390/mm/vmem.c | 15 ++++++++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
arch/s390/mm/init.c +1 -0
index 77767850d0d0..e0e78234ae57 100644
@@ -291,6 +291,7 @@ int arch_add_memory(int nid, u64 start, u64 size,
 	if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot))
 		return -EINVAL;
 
+	VM_BUG_ON(!memhp_range_allowed(start, size, 1));
 	rc = vmem_add_mapping(start, size);
 	if (rc)
 		return rc;
arch/s390/mm/vmem.c +14 -1
index b239f2ba93b0..e10e563ad2b4 100644
@@ -4,6 +4,7 @@
  *    Author(s): Heiko Carstens <[email protected]>
  */
 
+#include <linux/memory_hotplug.h>
 #include <linux/memblock.h>
 #include <linux/pfn.h>
 #include <linux/mm.h>
@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)
 	mutex_unlock(&vmem_mutex);
 }
 
+struct range arch_get_mappable_range(void)
+{
+	struct range memhp_range;
+
+	memhp_range.start = 0;
+	memhp_range.end =  VMEM_MAX_PHYS;
+	return memhp_range;
+}
+
 int vmem_add_mapping(unsigned long start, unsigned long size)
 {
+	struct range range;
 	int ret;
 
-	if (start + size > VMEM_MAX_PHYS ||
+	range = arch_get_mappable_range();
+	if (start < range.start ||
+	    start + size > range.end ||
 	    start + size < start)
 		return -ERANGE;
 
Signature · 1 line
2.20.1
#10 lore raw

Re: [PATCH V2 3/3] s390/mm: Define arch_get_mappable_range()

Quoted 23 lines · Anshuman Khandual

This overrides arch_get_mappabble_range() on s390 platform which will be used with recently added generic framework. It modifies the existing range check in vmem_add_mapping() using arch_get_mappable_range(). It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called on the hotplug path.

---
 arch/s390/mm/init.c |  1 +
 arch/s390/mm/vmem.c | 15 ++++++++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
arch/s390/mm/init.c +0 -0
index 77767850d0d0..e0e78234ae57 100644

arch/s390/mm/init.c int arch_add_memory(int nid, u64 start, u64 size, · line 294

Quoted 4 lines
arch/s390/mm/init.c +1 -0
@@ -291,6 +291,7 @@ int arch_add_memory(int nid, u64 start, u64 size,
 	if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot))
 		return -EINVAL;
 
+	VM_BUG_ON(!memhp_range_allowed(start, size, 1));

s/1/true/

Quoted 16 lines
arch/s390/mm/init.c +0 -0
 	rc = vmem_add_mapping(start, size);
 	if (rc)
 		return rc;
arch/s390/mm/vmem.c +1 -0
index b239f2ba93b0..e10e563ad2b4 100644
@@ -4,6 +4,7 @@
  *    Author(s): Heiko Carstens <[email protected]>
  */
 
+#include <linux/memory_hotplug.h>
 #include <linux/memblock.h>
 #include <linux/pfn.h>
 #include <linux/mm.h>

arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 541

Quoted 9 lines
arch/s390/mm/vmem.c +6 -0
@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)
 	mutex_unlock(&vmem_mutex);
 }
 
+struct range arch_get_mappable_range(void)
+{
+	struct range memhp_range;
+
+	memhp_range.start = 0;
+	memhp_range.end =  VMEM_MAX_PHYS;

s/ / /

IIRC, the range is inclusive? "VMEM_MAX_PHYS - 1" then, and adjust the check below.

Quoted 3 lines
arch/s390/mm/vmem.c +3 -0
+	return memhp_range;
+}
+
Signature · 3 lines
Thanks,

David / dhildenb
#11 lore raw

Re: [PATCH V2 3/3] s390/mm: Define arch_get_mappable_range()

Quoted 27 lines · David Hildenbrand
Quoted 26 lines · Anshuman Khandual

This overrides arch_get_mappabble_range() on s390 platform which will be used with recently added generic framework. It modifies the existing range check in vmem_add_mapping() using arch_get_mappable_range(). It also adds a VM_BUG_ON() check that would ensure that memhp_range_allowed() has already been called on the hotplug path.

---
 arch/s390/mm/init.c |  1 +
 arch/s390/mm/vmem.c | 15 ++++++++++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
arch/s390/mm/init.c +0 -0
index 77767850d0d0..e0e78234ae57 100644
@@ -291,6 +291,7 @@ int arch_add_memory(int nid, u64 start, u64 size,
 	if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot))
 		return -EINVAL;
 
Quoted 3 lines

arch/s390/mm/init.c int arch_add_memory(int nid, u64 start, u64 size, · line 294

Quoted 1 line
arch/s390/mm/init.c +1 -0
+	VM_BUG_ON(!memhp_range_allowed(start, size, 1));

s/1/true/

Sure, will do.

Quoted 29 lines
Quoted 17 lines
arch/s390/mm/init.c +0 -0
 	rc = vmem_add_mapping(start, size);
 	if (rc)
 		return rc;
arch/s390/mm/vmem.c +1 -0
index b239f2ba93b0..e10e563ad2b4 100644
@@ -4,6 +4,7 @@
  *    Author(s): Heiko Carstens <[email protected]>
  */
 
+#include <linux/memory_hotplug.h>
 #include <linux/memblock.h>
 #include <linux/pfn.h>
 #include <linux/mm.h>

arch/s390/mm/vmem.c void vmem_remove_mapping(unsigned long start, unsigned long size) · line 541

Quoted 9 lines
arch/s390/mm/vmem.c +6 -0
@@ -532,11 +533,23 @@ void vmem_remove_mapping(unsigned long start, unsigned long size)
 	mutex_unlock(&vmem_mutex);
 }
 
+struct range arch_get_mappable_range(void)
+{
+	struct range memhp_range;
+
+	memhp_range.start = 0;
+	memhp_range.end =  VMEM_MAX_PHYS;

s/ / /

Quoted 2 lines

IIRC, the range is inclusive? "VMEM_MAX_PHYS - 1" then, and adjust the check below.

Right, the memory range is inclusive at both ends. Hence will update the code here.

arch/s390/mm/vmem.c +2 -2
index 21cabe3473d7..afc39ff1cc8d 100644
@@ -538,7 +538,7 @@ struct range arch_get_mappable_range(void)
        struct range memhp_range;
 
        memhp_range.start = 0;
-       memhp_range.end =  VMEM_MAX_PHYS;
+       memhp_range.end =  VMEM_MAX_PHYS - 1;
        return memhp_range;
 }
 
@@ -549,7 +549,7 @@ int vmem_add_mapping(unsigned long start, unsigned long size)
 
        range = arch_get_mappable_range();
        if (start < range.start ||
-           start + size > range.end ||
+           start + size > range.end + 1 ||
            start + size < start)
                return -ERANGE;
 

(start + size - 1) is valid only when it is upto the inclusive range.end value.

#12 lore raw
In reply to Anshuman Khandual
Quoted 66 lines · Anshuman Khandual

This series adds a mechanism allowing platforms to weigh in and prevalidate incoming address range before proceeding further with the memory hotplug. This helps prevent potential platform errors for the given address range, down the hotplug call chain, which inevitably fails the hotplug itself.

This mechanism was suggested by David Hildenbrand during another discussion with respect to a memory hotplug fix on arm64 platform.

https://lore.kernel.org/linux-arm-kernel/[email protected]/

This mechanism focuses on the addressibility aspect and not [sub] section alignment aspect. Hence check_hotplug_memory_range() and check_pfn_span() have been left unchanged. Wondering if all these can still be unified in an expanded memhp_range_allowed() check, that can be called from multiple memory hot add and remove paths.

This series applies on v5.10 and has been tested on arm64. But only build tested on s390.

Changes in V2:

- Changed s390 version per Heiko and updated the commit message
- Called memhp_range_allowed() only for arch_add_memory() in pagemap_range()
- Exported the symbol memhp_get_pluggable_range()

Changes in V1:

https://lore.kernel.org/linux-mm/[email protected]/

- Fixed build problems with (MEMORY_HOTPLUG & !MEMORY_HOTREMOVE)
- Added missing prototype for arch_get_mappable_range()
- Added VM_BUG_ON() check for memhp_range_allowed() in arch_add_memory() per David

Changes in RFC V2:

https://lore.kernel.org/linux-mm/[email protected]/

Incorporated all review feedbacks from David.

- Added additional range check in __segment_load() on s390 which was lost
- Changed is_private init in pagemap_range()
- Moved the framework into mm/memory_hotplug.c
- Made arch_get_addressable_range() a __weak function
- Renamed arch_get_addressable_range() as arch_get_mappable_range()
- Callback arch_get_mappable_range() only handles range requiring linear mapping
- Merged multiple memhp_range_allowed() checks in register_memory_resource()
- Replaced WARN() with pr_warn() in memhp_range_allowed()
- Replaced error return code ERANGE with E2BIG

Changes in RFC V1:

https://lore.kernel.org/linux-mm/[email protected]/

Cc Heiko Carstens <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Mark Rutland <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Andrew Morton <[email protected]>
Quoted 4 lines

Anshuman Khandual (3):
mm/hotplug: Prevalidate the address range being added with platform arm64/mm: Define arch_get_mappable_range()
s390/mm: Define arch_get_mappable_range()

Thanks, I'm planning on reviewing + sending a virtio-mem patch to use memhp_get_mappable_range() in the new year. I assume we also have restrictions when it comes to x86-64, will have a look.

Signature · 3 lines
Thanks,

David / dhildenb
#13 lore raw

RFC PATCH 1/1 virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug

In reply to Anshuman Khandual

Right now, we only check against MAX_PHYSMEM_BITS - but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device operation.

So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.

In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Pankaj Gupta <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Wei Yang <[email protected]>
Cc Andrew Morton <[email protected]>
Cc teawater <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Pankaj Gupta <[email protected]>
Cc Jonathan Cameron <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Michal Hocko <[email protected]>

---

This is an example how virito-mem intends to use an interface like memhp_get_pluggable_range() once around. See:

"[PATCH V2 0/3] mm/hotplug: Pre-validate the address range with platform"
https://lkml.kernel.org/r/[email protected]

@Anshuman, feel free to pick up and carry this patch. I'll retest the final result / new versions of you series.

---
 drivers/virtio/virtio_mem.c | 40 +++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 13 deletions(-)
drivers/virtio/virtio_mem.c +27 -13
index 9fc9ec4a25f5..1fe40b2d7b6d 100644
@@ -2222,7 +2222,7 @@ static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm)
  */
 static void virtio_mem_refresh_config(struct virtio_mem *vm)
 {
-	const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS;
+	const struct range pluggable_range = memhp_get_pluggable_range(true);
 	uint64_t new_plugged_size, usable_region_size, end_addr;
 
 	/* the plugged_size is just a reflection of what _we_ did previously */
@@ -2234,15 +2234,25 @@ static void virtio_mem_refresh_config(struct virtio_mem *vm)
 	/* calculate the last usable memory block id */
 	virtio_cread_le(vm->vdev, struct virtio_mem_config,
 			usable_region_size, &usable_region_size);
-	end_addr = vm->addr + usable_region_size;
-	end_addr = min(end_addr, phys_limit);
+	end_addr = min(vm->addr + usable_region_size - 1,
+		       pluggable_range.end);
 
-	if (vm->in_sbm)
-		vm->sbm.last_usable_mb_id =
-					 virtio_mem_phys_to_mb_id(end_addr) - 1;
-	else
-		vm->bbm.last_usable_bb_id =
-				     virtio_mem_phys_to_bb_id(vm, end_addr) - 1;
+	if (vm->in_sbm) {
+		vm->sbm.last_usable_mb_id = virtio_mem_phys_to_mb_id(end_addr);
+		if (!IS_ALIGNED(end_addr + 1, memory_block_size_bytes()))
+			vm->sbm.last_usable_mb_id--;
+	} else {
+		vm->bbm.last_usable_bb_id = virtio_mem_phys_to_bb_id(vm,
+								     end_addr);
+		if (!IS_ALIGNED(end_addr + 1, vm->bbm.bb_size))
+			vm->bbm.last_usable_bb_id--;
+	}
+	/*
+	 * If we cannot plug any of our device memory (e.g., nothing in the
+	 * usable region is addressable), the last usable memory block id will
+	 * be smaller than the first usable memory block id. We'll stop
+	 * attempting to add memory with -ENOSPC from our main loop.
+	 */
 
 	/* see if there is a request to change the size */
 	virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size,
@@ -2364,6 +2374,7 @@ static int virtio_mem_init_vq(struct virtio_mem *vm)
 
 static int virtio_mem_init(struct virtio_mem *vm)
 {
+	const struct range pluggable_range = memhp_get_pluggable_range(true);
 	const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS;
 	uint64_t sb_size, addr;
 	uint16_t node_id;
@@ -2405,9 +2416,10 @@ static int virtio_mem_init(struct virtio_mem *vm)
 	if (!IS_ALIGNED(vm->addr + vm->region_size, memory_block_size_bytes()))
 		dev_warn(&vm->vdev->dev,
 			 "The alignment of the physical end address can make some memory unusable.\n");
-	if (vm->addr + vm->region_size > phys_limit)
+	if (vm->addr < pluggable_range.start ||
+	    vm->addr + vm->region_size - 1 > pluggable_range.end)
 		dev_warn(&vm->vdev->dev,
-			 "Some memory is not addressable. This can make some memory unusable.\n");
+			 "Some device memory is not addressable/pluggable. This can make some memory unusable.\n");
 
 	/*
 	 * We want subblocks to span at least MAX_ORDER_NR_PAGES and
@@ -2429,7 +2441,8 @@ static int virtio_mem_init(struct virtio_mem *vm)
 				     vm->sbm.sb_size;
 
 		/* Round up to the next full memory block */
-		addr = vm->addr + memory_block_size_bytes() - 1;
+		addr = max_t(uint64_t, vm->addr, pluggable_range.start) +
+		       memory_block_size_bytes() - 1;
 		vm->sbm.first_mb_id = virtio_mem_phys_to_mb_id(addr);
 		vm->sbm.next_mb_id = vm->sbm.first_mb_id;
 	} else {
@@ -2450,7 +2463,8 @@ static int virtio_mem_init(struct virtio_mem *vm)
 		}
 
 		/* Round up to the next aligned big block */
-		addr = vm->addr + vm->bbm.bb_size - 1;
+		addr = max_t(uint64_t, vm->addr, pluggable_range.start) +
+		       vm->bbm.bb_size - 1;
 		vm->bbm.first_bb_id = virtio_mem_phys_to_bb_id(vm, addr);
 		vm->bbm.next_bb_id = vm->bbm.first_bb_id;
 	}
Signature · 1 line
2.29.2
#14 lore raw

Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug

[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi David,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on linus/master]
[also build test ERROR on v5.11-rc3 next-20210111]
[cannot apply to linux/master hnaz-linux-mm/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/David-Hildenbrand/virtio-mem-check-against-memhp_get_pluggable_range-which-memory-we-can-hotplug/20210111-205003
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 7c53f6b671f4aba70ff15e1b05148b10d58c2837
config: x86_64-rhel (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/a53d8904d125a1020b44f1bf11961f781da124f3
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review David-Hildenbrand/virtio-mem-check-against-memhp_get_pluggable_range-which-memory-we-can-hotplug/20210111-205003
        git checkout a53d8904d125a1020b44f1bf11961f781da124f3
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

drivers/virtio/virtio_mem.c: In function 'virtio_mem_refresh_config':

>> drivers/virtio/virtio_mem.c:2225:39: error: implicit declaration of function 'memhp_get_pluggable_range' [-Werror=implicit-function-declaration]
    2225 |  const struct range pluggable_range = memhp_get_pluggable_range(true);
         |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~
>> drivers/virtio/virtio_mem.c:2225:39: error: invalid initializer
   drivers/virtio/virtio_mem.c: In function 'virtio_mem_init':
   drivers/virtio/virtio_mem.c:2377:39: error: invalid initializer
    2377 |  const struct range pluggable_range = memhp_get_pluggable_range(true);
         |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/virtio/virtio_mem.c:2378:17: warning: unused variable 'phys_limit' [-Wunused-variable]
    2378 |  const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS;
         |                 ^~~~~~~~~~

cc1: some warnings being treated as errors

vim +/memhp_get_pluggable_range +2225 drivers/virtio/virtio_mem.c

2219
2220 /*
2221 * Update all parts of the config that could have changed.
2222 */
2223 static void virtio_mem_refresh_config(struct virtio_mem *vm)
2224 {

> 2225		const struct range pluggable_range = memhp_get_pluggable_range(true);

2226 uint64_t new_plugged_size, usable_region_size, end_addr;
2227
2228 /* the plugged_size is just a reflection of what _we_ did previously */
2229 virtio_cread_le(vm->vdev, struct virtio_mem_config, plugged_size,
2230 &new_plugged_size);
2231 if (WARN_ON_ONCE(new_plugged_size != vm->plugged_size))
2232 vm->plugged_size = new_plugged_size;
2233
2234 /* calculate the last usable memory block id */
2235 virtio_cread_le(vm->vdev, struct virtio_mem_config,
2236 usable_region_size, &usable_region_size);
2237 end_addr = min(vm->addr + usable_region_size - 1,
2238 pluggable_range.end);
2239
2240 if (vm->in_sbm) {
2241 vm->sbm.last_usable_mb_id = virtio_mem_phys_to_mb_id(end_addr);
2242 if (!IS_ALIGNED(end_addr + 1, memory_block_size_bytes()))
2243 vm->sbm.last_usable_mb_id--;
2244 } else {
2245 vm->bbm.last_usable_bb_id = virtio_mem_phys_to_bb_id(vm,
2246 end_addr);
2247 if (!IS_ALIGNED(end_addr + 1, vm->bbm.bb_size))
2248 vm->bbm.last_usable_bb_id--;
2249 }
2250 /*
2251 * If we cannot plug any of our device memory (e.g., nothing in the
2252 * usable region is addressable), the last usable memory block id will
2253 * be smaller than the first usable memory block id. We'll stop
2254 * attempting to add memory with -ENOSPC from our main loop.
2255 */
2256
2257 /* see if there is a request to change the size */
2258 virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size,
2259 &vm->requested_size);
2260
2261 dev_info(&vm->vdev->dev, "plugged size: 0x%llx", vm->plugged_size);
2262 dev_info(&vm->vdev->dev, "requested size: 0x%llx", vm->requested_size);
2263 }
2264

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#15 lore raw

Re: [PATCH RFC] virtio-mem: check against memhp_get_pluggable_range() which memory we can hotplug

In reply to David Hildenbrand
Quoted 36 lines · David Hildenbrand

Right now, we only check against MAX_PHYSMEM_BITS - but turns out there are more restrictions of which memory we can actually hotplug, especially om arm64 or s390x once we support them: we might receive something like -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device operation.

So, check right when initializing the device which memory we can add, warning the user. Try only adding actually pluggable ranges: in the worst case, no memory provided by our device is pluggable.

In the usual case, we expect all device memory to be pluggable, and in corner cases only some memory at the end of the device-managed memory region to not be pluggable.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Jason Wang <[email protected]>
Cc Pankaj Gupta <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Wei Yang <[email protected]>
Cc Andrew Morton <[email protected]>
Cc teawater <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Pankaj Gupta <[email protected]>
Cc Jonathan Cameron <[email protected]>
Cc Vasily Gorbik <[email protected]>
Cc Will Deacon <[email protected]>
Cc Ard Biesheuvel <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Heiko Carstens <[email protected]>
Cc Michal Hocko <[email protected]>

---

Quoted 8 lines

This is an example how virito-mem intends to use an interface like memhp_get_pluggable_range() once around. See:

"[PATCH V2 0/3] mm/hotplug: Pre-validate the address range with platform"
https://lkml.kernel.org/r/[email protected]

@Anshuman, feel free to pick up and carry this patch. I'll retest the final result / new versions of you series.

Makes sense, will carry this patch in the series.