mm/memory_hotplug: Factor out memory block devicehandling
Subsystems (11)
v3 cover mm/memory_hotplug: Factor out memory block devicehandling
We only want memory block devices for memory to be onlined/offlined (add/remove from the buddy). This is required so user space can online/offline memory and kdump gets notified about newly onlined memory.
Let's factor out creation/removal of memory block devices. This helps to further cleanup arch_add_memory/arch_remove_memory() and to make implementation of new features easier - especially sub-section memory hot add from Dan.
Anshuman Khandual is currently working on arch_remove_memory(). I added a temporary solution via "arm64/mm: Add temporary arch_remove_memory() implementation", that is sufficient as a firsts tep in the context of this series. (we don't cleanup page tables in case anything goes wrong already)
Did a quick sanity test with DIMM plug/unplug, making sure all devices and sysfs links properly get added/removed. Compile tested on s390x and x86-64.
Based on next/master.
Next refactoring on my list will be making sure that remove_memory() will never deal with zones / access "struct pages". Any kind of zone handling will have to be done when offlining system memory / before removing device memory. I am thinking about remove_pfn_range_from_zone()", du undo everything "move_pfn_range_to_zone()" did.
v2 -> v3:
- Add "s390x/mm: Fail when an altmap is used for arch_add_memory()"
- Add "arm64/mm: Add temporary arch_remove_memory() implementation"
- Add "drivers/base/memory: Pass a block_id to init_memory_block()"
- Various changes to "mm/memory_hotplug: Create memory block devices after arch_add_memory()" and "mm/memory_hotplug: Create memory block devices after arch_add_memory()" due to switching from sections to block_id's.
v1 -> v2:
- s390x/mm: Implement arch_remove_memory()
-- remove mapping after "__remove_pages"
David Hildenbrand (11):
mm/memory_hotplug: Simplify and fix check_hotplug_memory_range() s390x/mm: Fail when an altmap is used for arch_add_memory() s390x/mm: Implement arch_remove_memory()
arm64/mm: Add temporary arch_remove_memory() implementation drivers/base/memory: Pass a block_id to init_memory_block() mm/memory_hotplug: Allow arch_remove_pages() without
CONFIG_MEMORY_HOTREMOVE
mm/memory_hotplug: Create memory block devices after arch_add_memory() mm/memory_hotplug: Drop MHP_MEMBLOCK_API mm/memory_hotplug: Remove memory block devices before
arch_remove_memory()
mm/memory_hotplug: Make unregister_memory_block_under_nodes() never
fail
mm/memory_hotplug: Remove "zone" parameter from
sparse_remove_one_section
arch/arm64/mm/mmu.c | 17 +++++ arch/ia64/mm/init.c | 2 - arch/powerpc/mm/mem.c | 2 - arch/s390/mm/init.c | 18 +++-- arch/sh/mm/init.c | 2 - arch/x86/mm/init_32.c | 2 - arch/x86/mm/init_64.c | 2 - drivers/base/memory.c | 134 +++++++++++++++++++-------------- drivers/base/node.c | 27 +++---- include/linux/memory.h | 6 +- include/linux/memory_hotplug.h | 12 +-- include/linux/node.h | 7 +- mm/memory_hotplug.c | 44 +++++------ mm/sparse.c | 10 +-- 14 files changed, 140 insertions(+), 145 deletions(-)
Signature · 1 line
2.20.1
PATCH v3 1/11 mm/memory_hotplug: Simplify and fix check_hotplug_memory_range()
By converting start and size to page granularity, we actually ignore unaligned parts within a page instead of properly bailing out with an error.
--- mm/memory_hotplug.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
mm/memory_hotplug.c +3 -8
@@ -1051,16 +1051,11 @@ int try_online_node(int nid)static int check_hotplug_memory_range (u64 start,u64 size) { -unsigned long block_sz =memory_block_size_bytes (); -u64 block_nr_pages = block_sz >>PAGE_SHIFT ; -u64 nr_pages = size >>PAGE_SHIFT ; -u64 start_pfn =PFN_DOWN (start); -/* memory range must be block size aligned */ -if (!nr_pages || !IS_ALIGNED (start_pfn, block_nr_pages) || - !IS_ALIGNED (nr_pages, block_nr_pages)) { +if (!size || !IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())) {pr_err ("Block size [%#lx] unaligned hotplug range: start %#llx, size %#llx" , - block_sz, start, size); +memory_block_size_bytes (), start, size);return -EINVAL ; }
Signature · 1 line
2.20.1
Re: [PATCH v3 01/11] mm/memory_hotplug: Simplify and fix check_hotplug_memory_range()
Quoted 5 lines · David Hildenbrand
By converting start and size to page granularity, we actually ignore unaligned parts within a page instead of properly bailing out with an error.
Quoted 12 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Re: [PATCH v3 01/11] mm/memory_hotplug: Simplify and fix check_hotplug_memory_range()
In reply to David HildenbrandQuoted 4 lines · David Hildenbrand
By converting start and size to page granularity, we actually ignore unaligned parts within a page instead of properly bailing out with an error.
Quoted 12 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 01/11] mm/memory_hotplug: Simplify and fix check_hotplug_memory_range()
In reply to David Hildenbrand[Sorry for a really late response]
Quoted 3 lines · David Hildenbrand
By converting start and size to page granularity, we actually ignore unaligned parts within a page instead of properly bailing out with an error.
I do not expect any code path would ever provide an unaligned address and even if it did then rounding that to a pfn doesn't sound like a terrible thing to do. Anyway this removes few lines so why not.
Quoted 13 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Reviewed-by Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 31 lines
--- mm/memory_hotplug.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)mm/memory_hotplug.c +3
-8@@ -1051,16 +1051,11 @@ int try_online_node(int nid)static int check_hotplug_memory_range (u64 start,u64 size) { -unsigned long block_sz =memory_block_size_bytes (); -u64 block_nr_pages = block_sz >>PAGE_SHIFT ; -u64 nr_pages = size >>PAGE_SHIFT ; -u64 start_pfn =PFN_DOWN (start); -/* memory range must be block size aligned */ -if (!nr_pages || !IS_ALIGNED (start_pfn, block_nr_pages) || - !IS_ALIGNED (nr_pages, block_nr_pages)) { +if (!size || !IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())) {pr_err ("Block size [%#lx] unaligned hotplug range: start %#llx, size %#llx" , - block_sz, start, size); +memory_block_size_bytes (), start, size);return -EINVAL ; }Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 2/11 s390x/mm: Fail when an altmap is used for arch_add_memory()
In reply to David HildenbrandZONE_DEVICE is not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.
--- arch/s390/mm/init.c | 3 +++ 1 file changed, 3 insertions(+)
arch/s390/mm/init.c +3 -0
@@ -226,6 +226,9 @@ int arch_add_memory(int nid, u64 start, u64 size,unsigned long size_pages =PFN_DOWN (size);int rc; +if (WARN_ON_ONCE (restrictions->altmap)) +return -EINVAL ; + rc =vmem_add_mapping (start, size);if (rc)return rc;
Signature · 1 line
2.20.1
Re: [PATCH v3 02/11] s390x/mm: Fail when an altmap is used for arch_add_memory()
Quoted 4 lines · David Hildenbrand
ZONE_DEVICEis not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.
Quoted 10 lines
Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Oscar Salvador <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 02/11] s390x/mm: Fail when an altmap is used for arch_add_memory()
In reply to David HildenbrandQuoted 3 lines · David Hildenbrand
ZONE_DEVICEis not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.
Why do we need this? Sure ZONE_DEVICE is not supported for s390 and so might be the case for other arches which support hotplug. I do not see much point in adding warning to each of them.
Quoted 31 lines
Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Oscar Salvador <[email protected]>Suggested-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/s390/mm/init.c | 3 +++ 1 file changed, 3 insertions(+)arch/s390/mm/init.c +3
-0@@ -226,6 +226,9 @@ int arch_add_memory(int nid, u64 start, u64 size,unsigned long size_pages =PFN_DOWN (size);int rc; +if (WARN_ON_ONCE (restrictions->altmap)) +return -EINVAL ; + rc =vmem_add_mapping (start, size);if (rc)return rc;Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 02/11] s390x/mm: Fail when an altmap is used for arch_add_memory()
Quoted 8 lines · Michal Hocko
Quoted 3 lines · David Hildenbrand
ZONE_DEVICEis not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.Why do we need this? Sure
ZONE_DEVICEis not supported for s390 and so might be the case for other arches which support hotplug. I do not see much point in adding warning to each of them.
I would drop this one. If there is a strong reason to have something like that it should come with a better explanation and it can be done on top.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 02/11] s390x/mm: Fail when an altmap is used for arch_add_memory()
Quoted 6 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
Quoted 3 lines · David Hildenbrand
ZONE_DEVICEis not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.
Quoted 8 lines
Quoted 3 lines
Why do we need this? Sure
ZONE_DEVICEis not supported for s390 and so might be the case for other arches which support hotplug. I do not see much point in adding warning to each of them.I would drop this one. If there is a strong reason to have something like that it should come with a better explanation and it can be done on top.
This was requested by Dan and I agree it is the right thing to do. In the context of paravirtualized devices (e.g., virtio-pmem), it makes sense to block functionality an arch does not support.
I'll leave the decision to Andrew.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 02/11] s390x/mm: Fail when an altmap is used for arch_add_memory()
Quoted 11 lines · David Hildenbrand
Quoted 10 lines · Michal Hocko
Quoted 8 lines · Michal Hocko
Quoted 3 lines · David Hildenbrand
ZONE_DEVICEis not yet supported, fail if an altmap is passed, so we don't forget arch_add_memory()/arch_remove_memory() when unlocking support.Why do we need this? Sure
ZONE_DEVICEis not supported for s390 and so might be the case for other arches which support hotplug. I do not see much point in adding warning to each of them.
Quoted 6 lines
Quoted 4 lines
I would drop this one. If there is a strong reason to have something like that it should come with a better explanation and it can be done on top.
This was requested by Dan and I agree it is the right thing to do.
This is probably a matter of taste. I would argue that altmap doesn't really equal ZONE_DEVICE. This is more a mechanism to use an alternative memmap allocator. Sure ZONE_DEVICE is the only in tree user of the feature but I really do not see why the arh specific code should care about it. The lack of altmap allocator is handled in the sparse code so this is just adding an early check which might confuse people in future.
Quoted 3 lines
In
the context of paravirtualized devices (e.g., virtio-pmem), it makes sense to block functionality an arch does not support.
Then block it on the config dependences.
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 3/11 s390x/mm: Implement arch_remove_memory()
In reply to David HildenbrandWill come in handy when wanting to handle errors after arch_add_memory().
--- arch/s390/mm/init.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
arch/s390/mm/init.c +7 -6
@@ -243,12 +243,13 @@ int arch_add_memory(int nid, u64 start, u64 size,void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) { -/* -* There is no hardware or firmware interface which could trigger a -* hot memory remove on s390. So there is nothing that needs to be -* implemented. -*/ -BUG (); +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +vmem_remove_mapping (start, size); }#endif #endif /* CONFIG_MEMORY_HOTPLUG */
Signature · 1 line
2.20.1
Re: [PATCH v3 03/11] s390x/mm: Implement arch_remove_memory()
Quoted 2 lines · David Hildenbrand
Will come in handy when wanting to handle errors after
arch_add_memory().
I do not understand this. Why do you add a code for something that is not possible on this HW (based on the comment - is it still valid btw?)
Quoted 40 lines
Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Oscar Salvador <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/s390/mm/init.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)arch/s390/mm/init.c +7
-6@@ -243,12 +243,13 @@ int arch_add_memory(int nid, u64 start, u64 size,void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) { -/* -* There is no hardware or firmware interface which could trigger a -* hot memory remove on s390. So there is nothing that needs to be -* implemented. -*/ -BUG (); +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +vmem_remove_mapping (start, size); }#endif #endif /* CONFIG_MEMORY_HOTPLUG */ Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 03/11] s390x/mm: Implement arch_remove_memory()
Quoted 6 lines · Michal Hocko
Quoted 2 lines · David Hildenbrand
Will come in handy when wanting to handle errors after
arch_add_memory().I do not understand this. Why do you add a code for something that is not possible on this HW (based on the comment - is it still valid btw?)
Same as the previous patch (drop it).
Quoted 44 lines
Quoted 40 lines
Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Oscar Salvador <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/s390/mm/init.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)arch/s390/mm/init.c +7
-6@@ -243,12 +243,13 @@ int arch_add_memory(int nid, u64 start, u64 size,void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) { -/* -* There is no hardware or firmware interface which could trigger a -* hot memory remove on s390. So there is nothing that needs to be -* implemented. -*/ -BUG (); +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +vmem_remove_mapping (start, size); }#endif #endif /* CONFIG_MEMORY_HOTPLUG */ Signature · 1 line
2.20.1Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 03/11] s390x/mm: Implement arch_remove_memory()
Quoted 9 lines · Michal Hocko
Quoted 6 lines · Michal Hocko
Quoted 2 lines · David Hildenbrand
Will come in handy when wanting to handle errors after
arch_add_memory().I do not understand this. Why do you add a code for something that is not possible on this HW (based on the comment - is it still valid btw?)
Same as the previous patch (drop it).
No. As the description says, this will be needed to handle errors in patch 6 cleanly.
And BTW, with paravirtualied devices like virtio-pmem and virtio-mem, this will also see some other users in the future.
Thanks.
Quoted 46 lines
Quoted 45 lines
Quoted 41 lines
Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Oscar Salvador <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/s390/mm/init.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)arch/s390/mm/init.c +7
-6@@ -243,12 +243,13 @@ int arch_add_memory(int nid, u64 start, u64 size,void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) { -/* -* There is no hardware or firmware interface which could trigger a -* hot memory remove on s390. So there is nothing that needs to be -* implemented. -*/ -BUG (); +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +vmem_remove_mapping (start, size); }#endif #endif /* CONFIG_MEMORY_HOTPLUG */ Signature · 1 line
2.20.1Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 3 lines
Thanks, David / dhildenb
PATCH v3 4/11 arm64/mm: Add temporary arch_remove_memory() implementation
In reply to David HildenbrandA proper arch_remove_memory() implementation is on its way, which also cleanly removes page tables in arch_add_memory() in case something goes wrong.
As we want to use arch_remove_memory() in case something goes wrong during memory hotplug after arch_add_memory() finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.
We will remove CONFIG_MEMORY_HOTREMOVE around this code in follow up patches.
--- arch/arm64/mm/mmu.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
arch/arm64/mm/mmu.c +19 -0
@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used +* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +} +#endif #endif
Signature · 1 line
2.20.1
Re: [PATCH v3 04/11] arm64/mm: Add temporary arch_remove_memory() implementation
Quoted 3 lines · David Hildenbrand
A proper
arch_remove_memory()implementation is on its way, which also cleanly removes page tables inarch_add_memory()in case something goes wrong.
Would this be better to understand?
removes page tables created in arch_add_memory
Quoted 39 lines
As we want to use
arch_remove_memory()in case something goes wrong during memory hotplug afterarch_add_memory()finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.We will remove
CONFIG_MEMORY_HOTREMOVEaround this code in follow up patches.Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Mark Rutland <[email protected]>Cc Andrew Morton <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Chintan Pandya <[email protected]>Cc Mike Rapoport <[email protected]>Cc Jun Yao <[email protected]>Cc Yu Zhao <[email protected]>Cc Robin Murphy <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
arch/arm64/mm/mmu.c| 19 +++++++++++++++++++1 file changed, 19 insertions(+)arch/arm64/mm/mmu.c +6
-0@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ;
arch/arm64/mm/mmu.c int arch_add_memory(int nid, u64 start, u64 size, · line 1102
Quoted 10 lines
arch/arm64/mm/mmu.c +10
-0+struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used +* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn));
Compared with arch_remove_memory in x86. If altmap is not NULL, zone will be retrieved from page related to altmap. Not sure why this is not the same?
Quoted 6 lines
arch/arm64/mm/mmu.c +3
-0+__remove_pages (zone, start_pfn, nr_pages, altmap); +} +#endif #endif Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 04/11] arm64/mm: Add temporary arch_remove_memory() implementation
Quoted 8 lines · Wei Yang
Quoted 3 lines · David Hildenbrand
A proper
arch_remove_memory()implementation is on its way, which also cleanly removes page tables inarch_add_memory()in case something goes wrong.Would this be better to understand?
removes page tables created in
arch_add_memory
That's not what this sentence expresses. Have a look at arch_add_memory(), in case __add_pages() fails, the page tables are not removed. This will also be fixed by Anshuman in the same shot.
Quoted 45 lines
Quoted 45 lines
As we want to use
arch_remove_memory()in case something goes wrong during memory hotplug afterarch_add_memory()finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.We will remove
CONFIG_MEMORY_HOTREMOVEaround this code in follow up patches.Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Mark Rutland <[email protected]>Cc Andrew Morton <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Chintan Pandya <[email protected]>Cc Mike Rapoport <[email protected]>Cc Jun Yao <[email protected]>Cc Yu Zhao <[email protected]>Cc Robin Murphy <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
arch/arm64/mm/mmu.c| 19 +++++++++++++++++++1 file changed, 19 insertions(+)arch/arm64/mm/mmu.c +11
-0@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used
Quoted 8 lines
arch/arm64/mm/mmu.c int arch_add_memory(int nid, u64 start, u64 size, · line 1102
Quoted 5 lines
arch/arm64/mm/mmu.c +5
-0+* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn));Compared with
arch_remove_memoryin x86. If altmap is not NULL, zone will be retrieved from page related to altmap. Not sure why this is not the same?
This is a minimal implementation, sufficient for this use case here. A full implementation is in the works. For now, this function will not be used with an altmap (ZONE_DEVICE is not esupported for arm64 yet).
Thanks!
Quoted 8 lines
Quoted 7 lines
arch/arm64/mm/mmu.c +3
-0+__remove_pages (zone, start_pfn, nr_pages, altmap); +} +#endif #endif Signature · 1 line
2.20.1
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 04/11] arm64/mm: Add temporary arch_remove_memory() implementation
Quoted 68 lines · David Hildenbrand
Quoted 8 lines · Wei Yang
Quoted 3 lines · David Hildenbrand
A proper
arch_remove_memory()implementation is on its way, which also cleanly removes page tables inarch_add_memory()in case something goes wrong.Would this be better to understand?
removes page tables created in
arch_add_memoryThat's not what this sentence expresses. Have a look at
arch_add_memory(), in case__add_pages()fails, the page tables are not removed. This will also be fixed by Anshuman in the same shot.Quoted 45 lines
Quoted 45 lines
As we want to use
arch_remove_memory()in case something goes wrong during memory hotplug afterarch_add_memory()finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.We will remove
CONFIG_MEMORY_HOTREMOVEaround this code in follow up patches.Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Mark Rutland <[email protected]>Cc Andrew Morton <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Chintan Pandya <[email protected]>Cc Mike Rapoport <[email protected]>Cc Jun Yao <[email protected]>Cc Yu Zhao <[email protected]>Cc Robin Murphy <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
arch/arm64/mm/mmu.c| 19 +++++++++++++++++++1 file changed, 19 insertions(+)arch/arm64/mm/mmu.c +11
-0@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used Quoted 8 lines
arch/arm64/mm/mmu.c int arch_add_memory(int nid, u64 start, u64 size, · line 1102
Quoted 5 lines
arch/arm64/mm/mmu.c +5
-0+* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn));Compared with
arch_remove_memoryin x86. If altmap is not NULL, zone will be retrieved from page related to altmap. Not sure why this is not the same?
Quoted 3 lines
This is a minimal implementation, sufficient for this use case here. A full implementation is in the works. For now, this function will not be used with an altmap (
ZONE_DEVICEis not esupported for arm64 yet).
FWIW the other pieces of ZONE_DEVICE are now due to land in parallel, but as long as we don't throw the ARCH_ENABLE_MEMORY_HOTREMOVE switch then there should still be no issue. Besides, given that we should consistently ignore the altmap everywhere at the moment, it may even work out regardless.
One thing stands out about the failure path thing, though - if __add_pages() did fail, can it still be guaranteed to have initialised the memmap such that page_zone() won't return nonsense? Last time I looked that was still a problem when removing memory which had been successfully added, but never onlined (although I do know that particular case was already being discussed at the time, and I've not been paying the greatest attention since).
Robin.
Re: [PATCH v3 04/11] arm64/mm: Add temporary arch_remove_memory() implementation
Quoted 73 lines · Robin Murphy
Quoted 68 lines · David Hildenbrand
Quoted 8 lines · Wei Yang
Quoted 3 lines · David Hildenbrand
A proper
arch_remove_memory()implementation is on its way, which also cleanly removes page tables inarch_add_memory()in case something goes wrong.Would this be better to understand?
removes page tables created in
arch_add_memoryThat's not what this sentence expresses. Have a look at
arch_add_memory(), in case__add_pages()fails, the page tables are not removed. This will also be fixed by Anshuman in the same shot.Quoted 45 lines
Quoted 45 lines
As we want to use
arch_remove_memory()in case something goes wrong during memory hotplug afterarch_add_memory()finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.We will remove
CONFIG_MEMORY_HOTREMOVEaround this code in follow up patches.Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Mark Rutland <[email protected]>Cc Andrew Morton <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Chintan Pandya <[email protected]>Cc Mike Rapoport <[email protected]>Cc Jun Yao <[email protected]>Cc Yu Zhao <[email protected]>Cc Robin Murphy <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
arch/arm64/mm/mmu.c| 19 +++++++++++++++++++1 file changed, 19 insertions(+)arch/arm64/mm/mmu.c +11
-0@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used Quoted 8 lines
arch/arm64/mm/mmu.c int arch_add_memory(int nid, u64 start, u64 size, · line 1102
Quoted 5 lines
arch/arm64/mm/mmu.c +5
-0+* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn));Compared with
arch_remove_memoryin x86. If altmap is not NULL, zone will be retrieved from page related to altmap. Not sure why this is not the same?Quoted 3 lines
This is a minimal implementation, sufficient for this use case here. A full implementation is in the works. For now, this function will not be used with an altmap (
ZONE_DEVICEis not esupported for arm64 yet).
Quoted 5 lines
FWIW the other pieces of
ZONE_DEVICEare now due to land in parallel, but as long as we don't throw theARCH_ENABLE_MEMORY_HOTREMOVEswitch then there should still be no issue. Besides, given that we should consistently ignore the altmap everywhere at the moment, it may even work out regardless.
Thanks for the info.
Quoted 4 lines
One thing stands out about the failure path thing, though - if
__add_pages()did fail, can it still be guaranteed to have initialised the memmap such thatpage_zone()won't return nonsense? Last time I
if __add_pages() fails, then arch_add_memory() fails and arch_remove_memory() will not be called in the context of this series. Only if it succeeded.
Quoted 4 lines
looked that was still a problem when removing memory which had been successfully added, but never onlined (although I do know that particular case was already being discussed at the time, and I've not been paying the greatest attention since).
Yes, that part is next on my list. It works but is ugly. The memory removal process should not care about zones at all.
Slowly moving into the right direction :)
Quoted 3 lines
Robin.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 04/11] arm64/mm: Add temporary arch_remove_memory() implementation
In reply to David HildenbrandQuoted 11 lines · David Hildenbrand
A proper
arch_remove_memory()implementation is on its way, which also cleanly removes page tables inarch_add_memory()in case something goes wrong.As we want to use
arch_remove_memory()in case something goes wrong during memory hotplug afterarch_add_memory()finished, let's add a temporary hack that is sufficient enough until we get a proper implementation that cleans up page table entries.We will remove
CONFIG_MEMORY_HOTREMOVEaround this code in follow up patches.
I would drop this one as well (like s390 counterpart).
Quoted 46 lines
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Mark Rutland <[email protected]>Cc Andrew Morton <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Chintan Pandya <[email protected]>Cc Mike Rapoport <[email protected]>Cc Jun Yao <[email protected]>Cc Yu Zhao <[email protected]>Cc Robin Murphy <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/arm64/mm/mmu.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)arch/arm64/mm/mmu.c +19
-0@@ -1084,4 +1084,23 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } +#ifdef CONFIG_MEMORY_HOTREMOVE +void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +unsigned long start_pfn = start >>PAGE_SHIFT ; +unsigned long nr_pages = size >>PAGE_SHIFT ; +struct zone *zone; + +/* +* FIXME: Cleanup page tables (also in arch_add_memory() in case +* adding fails). Until then, this function should only be used +* during memory hotplug (adding memory), not for memory +* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be +* unlocked yet. +*/ + zone =page_zone (pfn_to_page (start_pfn)); +__remove_pages (zone, start_pfn, nr_pages, altmap); +} +#endif #endif Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 5/11 drivers/base/memory: Pass a block_id to init_memory_block()
In reply to David HildenbrandWe'll rework hotplug_memory_register() shortly, so it no longer consumes pass a section.
--- drivers/base/memory.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/base/memory.c +7 -7
@@ -651,21 +651,18 @@ int register_memory(struct memory_block *memory)return ret; } -static int init_memory_block (struct memory_block **memory, -struct mem_section *section,unsigned long state) +static int init_memory_block (struct memory_block **memory,int block_id, +unsigned long state) {struct memory_block *mem;unsigned long start_pfn; -int scn_nr;int ret =0 ; mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ; - scn_nr =__section_nr (section); - mem->start_section_nr -base_memory_block_id (scn_nr) * sections_per_block; + mem->start_section_nr = block_id * sections_per_block; mem->end_section_nr = mem->start_section_nr + sections_per_block -1 ; mem->state = state; start_pfn =section_nr_to_pfn (mem->start_section_nr);
@@ -694,7 +691,8 @@ static int add_memory_block(int base_section_nr)if (section_count =0 )return 0 ; - ret =init_memory_block (&mem,__nr_to_section (section_nr),MEM_ONLINE ); + ret =init_memory_block (&mem,base_memory_block_id (base_section_nr), +MEM_ONLINE );if (ret)return ret; mem->section_count = section_count;
@@ -707,6 +705,7 @@ static int add_memory_block(int base_section_nr)*/ int hotplug_memory_register (int nid,struct mem_section *section) { +int block_id =base_memory_block_id (__section_nr (section));int ret =0 ;struct memory_block *mem;
@@ -717,7 +716,7 @@ int hotplug_memory_register(int nid, struct mem_section *section) mem->section_count++;put_device (&mem->dev); }else { - ret =init_memory_block (&mem, section,MEM_OFFLINE ); + ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) goto out; mem->section_count++;
Signature · 1 line
2.20.1
Re: [PATCH v3 05/11] drivers/base/memory: Pass a block_id to init_memory_block()
Quoted 39 lines · David Hildenbrand
We'll rework
hotplug_memory_register()shortly, so it no longer consumes pass a section.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/base/memory.c| 15 +++++++--------1 file changed, 7 insertions(+), 8 deletions(-)drivers/base/memory.c +3
-5@@ -651,21 +651,18 @@ int register_memory(struct memory_block *memory)return ret; } -static int init_memory_block (struct memory_block **memory, -struct mem_section *section,unsigned long state) +static int init_memory_block (struct memory_block **memory,int block_id, +unsigned long state) {struct memory_block *mem;unsigned long start_pfn; -int scn_nr;int ret =0 ; mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ; - scn_nr =__section_nr (section); - mem->start_section_nr >-base_memory_block_id (scn_nr) * sections_per_block; + mem->start_section_nr = block_id * sections_per_block; mem->end_section_nr = mem->start_section_nr + sections_per_block -1 ; mem->state = state; start_pfn =section_nr_to_pfn (mem->start_section_nr);
drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 695
Quoted 6 lines
drivers/base/memory.c +2
-1@@ -694,7 +691,8 @@ static int add_memory_block(int base_section_nr)if (section_count =0 )return 0 ; - ret =init_memory_block (&mem,__nr_to_section (section_nr),MEM_ONLINE ); + ret =init_memory_block (&mem,base_memory_block_id (base_section_nr), +MEM_ONLINE );
If my understanding is correct, section_nr could be removed too.
Quoted 22 lines
drivers/base/memory.c +2
-1if (ret)return ret; mem->section_count = section_count;@@ -707,6 +705,7 @@ static int add_memory_block(int base_section_nr)*/ int hotplug_memory_register (int nid,struct mem_section *section) { +int block_id =base_memory_block_id (__section_nr (section));int ret =0 ;struct memory_block *mem;@@ -717,7 +716,7 @@ int hotplug_memory_register(int nid, struct mem_section *section) mem->section_count++;put_device (&mem->dev); }else { - ret =init_memory_block (&mem, section,MEM_OFFLINE ); + ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) goto out; mem->section_count++;Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 05/11] drivers/base/memory: Pass a block_id to init_memory_block()
Quoted 41 lines · Wei Yang
Quoted 40 lines · David Hildenbrand
We'll rework
hotplug_memory_register()shortly, so it no longer consumes pass a section.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/base/memory.c| 15 +++++++--------1 file changed, 7 insertions(+), 8 deletions(-)drivers/base/memory.c +3
-5@@ -651,21 +651,18 @@ int register_memory(struct memory_block *memory)return ret; } -static int init_memory_block (struct memory_block **memory, -struct mem_section *section,unsigned long state) +static int init_memory_block (struct memory_block **memory,int block_id, +unsigned long state) {struct memory_block *mem;unsigned long start_pfn; -int scn_nr;int ret =0 ; mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ; - scn_nr =__section_nr (section); - mem->start_section_nr >> -base_memory_block_id (scn_nr) * sections_per_block; + mem->start_section_nr = block_id * sections_per_block; mem->end_section_nr = mem->start_section_nr + sections_per_block -1 ; mem->state = state; start_pfn =section_nr_to_pfn (mem->start_section_nr);@@ -694,7 +691,8 @@ static int add_memory_block(int base_section_nr)
Quoted 7 lines
drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 695
Quoted 5 lines
drivers/base/memory.c +2
-1if (section_count =0 )return 0 ; - ret =init_memory_block (&mem,__nr_to_section (section_nr),MEM_ONLINE ); + ret =init_memory_block (&mem,base_memory_block_id (base_section_nr), +MEM_ONLINE );If my understanding is correct,
section_nrcould be removed too.
Yes you are, this has already been addressed in linux-next.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 05/11] drivers/base/memory: Pass a block_id to init_memory_block()
In reply to David HildenbrandQuoted 6 lines · David Hildenbrand
We'll rework
hotplug_memory_register()shortly, so it no longer consumes pass a section.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 61 lines
--- drivers/base/memory.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)drivers/base/memory.c +7
-7@@ -651,21 +651,18 @@ int register_memory(struct memory_block *memory)return ret; } -static int init_memory_block (struct memory_block **memory, -struct mem_section *section,unsigned long state) +static int init_memory_block (struct memory_block **memory,int block_id, +unsigned long state) {struct memory_block *mem;unsigned long start_pfn; -int scn_nr;int ret =0 ; mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ; - scn_nr =__section_nr (section); - mem->start_section_nr > -base_memory_block_id (scn_nr) * sections_per_block; + mem->start_section_nr = block_id * sections_per_block; mem->end_section_nr = mem->start_section_nr + sections_per_block -1 ; mem->state = state; start_pfn =section_nr_to_pfn (mem->start_section_nr);@@ -694,7 +691,8 @@ static int add_memory_block(int base_section_nr)if (section_count =0 )return 0 ; - ret =init_memory_block (&mem,__nr_to_section (section_nr),MEM_ONLINE ); + ret =init_memory_block (&mem,base_memory_block_id (base_section_nr), +MEM_ONLINE );if (ret)return ret; mem->section_count = section_count;@@ -707,6 +705,7 @@ static int add_memory_block(int base_section_nr)*/ int hotplug_memory_register (int nid,struct mem_section *section) { +int block_id =base_memory_block_id (__section_nr (section));int ret =0 ;struct memory_block *mem;@@ -717,7 +716,7 @@ int hotplug_memory_register(int nid, struct mem_section *section) mem->section_count++;put_device (&mem->dev); }else { - ret =init_memory_block (&mem, section,MEM_OFFLINE ); + ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) goto out; mem->section_count++;Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 6/11 mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
In reply to David HildenbrandWe want to improve error handling while adding memory by allowing to use arch_remove_memory() and __remove_pages() even if CONFIG_MEMORY_HOTREMOVE is not set to e.g., implement something like:
arch_add_memory()
rc = do_something();
if (rc) {
arch_remove_memory();
}We won't get rid of CONFIG_MEMORY_HOTREMOVE for now, as it will require quite some dependencies for memory offlining.
--- arch/arm64/mm/mmu.c | 2 -- arch/ia64/mm/init.c | 2 -- arch/powerpc/mm/mem.c | 2 -- arch/s390/mm/init.c | 2 -- arch/sh/mm/init.c | 2 -- arch/x86/mm/init_32.c | 2 -- arch/x86/mm/init_64.c | 2 -- drivers/base/memory.c | 2 -- include/linux/memory.h | 2 -- include/linux/memory_hotplug.h | 2 -- mm/memory_hotplug.c | 2 -- mm/sparse.c | 6 ------ 12 files changed, 28 deletions(-)
arch/arm64/mm/mmu.c +0 -2
@@ -1084,7 +1084,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -1103,4 +1102,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif
arch/ia64/mm/init.c +0 -2
@@ -681,7 +681,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -693,4 +692,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif
arch/powerpc/mm/mem.c +0 -2
@@ -130,7 +130,6 @@ int __ref arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void __ref arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -164,7 +163,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, pr_warn("Hash collision while resizing HPT\n"); }#endif -#endif /* CONFIG_MEMORY_HOTPLUG */ #ifndef CONFIG_NEED_MULTIPLE_NODES void __init mem_topology_setup (void )
arch/s390/mm/init.c +0 -2
@@ -239,7 +239,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return rc; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -251,5 +250,4 @@ void arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); vmem_remove_mapping(start, size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */
arch/sh/mm/init.c +0 -2
@@ -429,7 +429,6 @@ int memory_add_physaddr_to_nid(u64 addr)EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid);#endif -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -440,5 +439,4 @@ void arch_remove_memory(int nid, u64 start, u64 size, zone =page_zone (pfn_to_page (start_pfn)); __remove_pages(zone, start_pfn, nr_pages, altmap); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */
arch/x86/mm/init_32.c +0 -2
@@ -860,7 +860,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {
@@ -872,7 +871,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif int kernel_set_to_readonly__read_mostly ;
arch/x86/mm/init_64.c +0 -2
@@ -1196,7 +1196,6 @@ void __ref vmemmap_free(unsigned long start, unsigned long end,remove_pagetable (start, end, false, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void __meminit kernel_physical_mapping_remove (unsigned long start,unsigned long end) {
@@ -1221,7 +1220,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); kernel_physical_mapping_remove(start, start + size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ static struct kcore_list kcore_vsyscall;
drivers/base/memory.c +0 -2
@@ -727,7 +727,6 @@ int hotplug_memory_register(int nid, struct mem_section *section)return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE static void unregister_memory (struct memory_block *memory) {
@@ -766,7 +765,6 @@ void unregister_memory_section(struct mem_section *section) out_unlock:mutex_unlock (&mem_sysfs_mutex); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ /* return true if the memory block is offlined, otherwise, return false */ bool is_memblock_offlined (struct memory_block *mem)
include/linux/memory.h +0 -2
@@ -112,9 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int hotplug_memory_register (int nid,struct mem_section *section); -#ifdef CONFIG_MEMORY_HOTREMOVE extern void unregister_memory_section (struct mem_section *); -#endif extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);
include/linux/memory_hotplug.h +0 -2
@@ -123,12 +123,10 @@ static inline bool movable_node_is_enabled(void)return movable_node_enabled; } -#ifdef CONFIG_MEMORY_HOTREMOVE extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap);extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -#endif /* CONFIG_MEMORY_HOTREMOVE */ /* * Do we want sysfs memblock files created. This will allow userspace to online
mm/memory_hotplug.c +0 -2
@@ -318,7 +318,6 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn,return err; } -#ifdef CONFIG_MEMORY_HOTREMOVE /* find the smallest valid pfn in the range [start_pfn, end_pfn) */ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone, unsigned long start_pfn,
@@ -582,7 +581,6 @@ void __remove_pages(struct zone *zone, unsigned long phys_start_pfn,set_zone_contiguous (zone); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ int set_online_page_callback (online_page_callback_t callback) {
mm/sparse.c +0 -6
@@ -604,7 +604,6 @@ static void __kfree_section_memmap(struct page *memmap,vmemmap_free (start, end, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long start = (unsigned long )memmap;
@@ -612,7 +611,6 @@ static void free_map_bootmem(struct page *memmap) vmemmap_free(start, end, NULL); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #else static struct page *__kmalloc_section_memmap (void ) {
@@ -651,7 +649,6 @@ static void __kfree_section_memmap(struct page *memmap, get_order(sizeof(struct page) * PAGES_PER_SECTION)); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long maps_section_nr, removing_section_nr, i;
@@ -681,7 +678,6 @@ static void free_map_bootmem(struct page *memmap) put_page_bootmem(page); } } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_SPARSEMEM_VMEMMAP */ /**
@@ -746,7 +742,6 @@ int __meminit sparse_add_one_section(int nid, unsigned long start_pfn,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE#ifdef CONFIG_MEMORY_FAILURE static void clear_hwpoisoned_pages (struct page *memmap,int nr_pages) {
@@ -823,5 +818,4 @@ void sparse_remove_one_section(struct zone *zone, struct mem_section *ms, PAGES_PER_SECTION - map_offset); free_section_usemap(memmap, usemap, altmap); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_MEMORY_HOTPLUG */
Signature · 1 line
2.20.1
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 5 lines · David Hildenbrand
We want to improve error handling while adding memory by allowing to use
arch_remove_memory()and__remove_pages()even ifCONFIG_MEMORY_HOTREMOVEis not set to e.g., implement something like:
Quoted 8 lines
arch_add_memory() rc = do_something(); if (rc) { arch_remove_memory(); }We won't get rid of
CONFIG_MEMORY_HOTREMOVEfor now, as it will require quite some dependencies for memory offlining.
I like this simplification, we should really get rid of CONFIG_MEMORY_HOTREMOVE.
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
In reply to David HildenbrandAllow arch_remove_pages() or arch_remove_memory()?
And want to confirm the kernel build on affected arch succeed?
Quoted 325 lines · David Hildenbrand
We want to improve error handling while adding memory by allowing to use
arch_remove_memory()and__remove_pages()even ifCONFIG_MEMORY_HOTREMOVEis not set to e.g., implement something like:arch_add_memory() rc = do_something(); if (rc) { arch_remove_memory(); }We won't get rid of
CONFIG_MEMORY_HOTREMOVEfor now, as it will require quite some dependencies for memory offlining.Cc Tony Luck <[email protected]>Cc Fenghua Yu <[email protected]>Cc Benjamin Herrenschmidt <[email protected]>Cc Paul Mackerras <[email protected]>Cc Michael Ellerman <[email protected]>Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Yoshinori Sato <[email protected]>Cc Rich Felker <[email protected]>Cc Dave Hansen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc Christophe Leroy <[email protected]>Cc Nicholas Piggin <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Rob Herring <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Andrew Banman <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Qian Cai <[email protected]>Cc Mathieu Malaterre <[email protected]>Cc Baoquan He <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
arch/arm64/mm/mmu.c | 2 -- arch/ia64/mm/init.c | 2 -- arch/powerpc/mm/mem.c | 2 -- arch/s390/mm/init.c | 2 -- arch/sh/mm/init.c | 2 -- arch/x86/mm/init_32.c | 2 -- arch/x86/mm/init_64.c | 2 -- drivers/base/memory.c | 2 -- include/linux/memory.h | 2 -- include/linux/memory_hotplug.h | 2 -- mm/memory_hotplug.c | 2 -- mm/sparse.c | 6 ------ 12 files changed, 28 deletions(-)arch/arm64/mm/mmu.c +0
-2@@ -1084,7 +1084,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -1103,4 +1102,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif arch/ia64/mm/init.c +0
-2@@ -681,7 +681,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -693,4 +692,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif arch/powerpc/mm/mem.c +0
-2@@ -130,7 +130,6 @@ int __ref arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void __ref arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -164,7 +163,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, pr_warn("Hash collision while resizing HPT\n"); }#endif -#endif /* CONFIG_MEMORY_HOTPLUG */ #ifndef CONFIG_NEED_MULTIPLE_NODES void __init mem_topology_setup (void )arch/s390/mm/init.c +0
-2@@ -239,7 +239,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return rc; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -251,5 +250,4 @@ void arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); vmem_remove_mapping(start, size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ arch/sh/mm/init.c +0
-2@@ -429,7 +429,6 @@ int memory_add_physaddr_to_nid(u64 addr)EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid);#endif -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -440,5 +439,4 @@ void arch_remove_memory(int nid, u64 start, u64 size, zone =page_zone (pfn_to_page (start_pfn)); __remove_pages(zone, start_pfn, nr_pages, altmap); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ arch/x86/mm/init_32.c +0
-2@@ -860,7 +860,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -872,7 +871,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif int kernel_set_to_readonly__read_mostly ;arch/x86/mm/init_64.c +0
-1@@ -1196,7 +1196,6 @@ void __ref vmemmap_free(unsigned long start, unsigned long end,remove_pagetable (start, end, false, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void
__meminitkernel_physical_mapping_remove(unsigned long start, unsigned long end){arch/x86/mm/init_64.c +0
-1@@ -1221,7 +1220,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); kernel_physical_mapping_remove(start, start + size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ static struct kcore_list kcore_vsyscall;drivers/base/memory.c +0
-1@@ -727,7 +727,6 @@ int hotplug_memory_register(int nid, struct mem_section *section)return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE static void
unregister_memory(struct memory_block *memory)
{drivers/base/memory.c +0
-0@@ -766,7 +765,6 @@ void unregister_memory_section(struct mem_section *section)
out_unlock:mutex_unlock(&mem_sysfs_mutex); }-#endif /*
CONFIG_MEMORY_HOTREMOVE*//* return true if the memory block is offlined, otherwise, return false */ bool
is_memblock_offlined(struct memory_block *mem)include/linux/memory.h +0
-2@@ -112,9 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int hotplug_memory_register (int nid,struct mem_section *section); -#ifdef CONFIG_MEMORY_HOTREMOVE extern void unregister_memory_section (struct mem_section *); -#endif extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);include/linux/memory_hotplug.h +0
-2@@ -123,12 +123,10 @@ static inline bool movable_node_is_enabled(void)return movable_node_enabled; } -#ifdef CONFIG_MEMORY_HOTREMOVE extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap);extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -#endif /* CONFIG_MEMORY_HOTREMOVE */ /* * Do we want sysfs memblock files created. This will allow userspace to online mm/memory_hotplug.c +0
-2@@ -318,7 +318,6 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn,return err; } -#ifdef CONFIG_MEMORY_HOTREMOVE /* find the smallest valid pfn in the range [start_pfn, end_pfn) */ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone, unsigned long start_pfn,@@ -582,7 +581,6 @@ void __remove_pages(struct zone *zone, unsigned long phys_start_pfn,set_zone_contiguous (zone); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ int set_online_page_callback (online_page_callback_t callback) {mm/sparse.c +0
-6@@ -604,7 +604,6 @@ static void __kfree_section_memmap(struct page *memmap,vmemmap_free (start, end, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long start = (unsigned long )memmap;@@ -612,7 +611,6 @@ static void free_map_bootmem(struct page *memmap) vmemmap_free(start, end, NULL); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #else static struct page *__kmalloc_section_memmap (void ) {@@ -651,7 +649,6 @@ static void __kfree_section_memmap(struct page *memmap, get_order(sizeof(struct page) * PAGES_PER_SECTION)); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long maps_section_nr, removing_section_nr, i;@@ -681,7 +678,6 @@ static void free_map_bootmem(struct page *memmap) put_page_bootmem(page); } } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_SPARSEMEM_VMEMMAP */ /** @@ -746,7 +742,6 @@ int __meminit sparse_add_one_section(int nid, unsigned long start_pfn,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE#ifdef CONFIG_MEMORY_FAILURE static void clear_hwpoisoned_pages (struct page *memmap,int nr_pages) {@@ -823,5 +818,4 @@ void sparse_remove_one_section(struct zone *zone, struct mem_section *ms, PAGES_PER_SECTION - map_offset); free_section_usemap(memmap, usemap, altmap); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_MEMORY_HOTPLUG */ Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 1 line · Wei Yang
Allow
arch_remove_pages()orarch_remove_memory()?
Looks like I merged __remove_pages() and arch_remove_memory().
@Andrew, can you fix this up to
"mm/memory_hotplug: Allow arch_remove_memory() without CONFIG_MEMORY_HOTREMOVE"
? Thanks!
Quoted 2 lines
And want to confirm the kernel build on affected arch succeed?
I compile-tested on s390x and x86. As the patches are in linux-next for some time, I think the other builds are also fine.
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 12 lines · David Hildenbrand
Quoted 1 line · Wei Yang
Allow
arch_remove_pages()orarch_remove_memory()?Looks like I merged
__remove_pages()andarch_remove_memory().@Andrew, can you fix this up to
"
mm/memory_hotplug: Allowarch_remove_memory()withoutCONFIG_MEMORY_HOTREMOVE"? Thanks!
Already merged?
Quoted 6 lines
Quoted 2 lines
And want to confirm the kernel build on affected arch succeed?
I compile-tested on s390x and x86. As the patches are in linux-next for some time, I think the other builds are also fine.
Yep, sounds good~
Quoted 7 lines
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 8 lines · Wei Yang
Quoted 7 lines · David Hildenbrand
Quoted 1 line · Wei Yang
Allow
arch_remove_pages()orarch_remove_memory()?Looks like I merged
__remove_pages()andarch_remove_memory().@Andrew, can you fix this up to
Quoted 7 lines
Quoted 5 lines
"
mm/memory_hotplug: Allowarch_remove_memory()withoutCONFIG_MEMORY_HOTREMOVE"? Thanks!
Already merged?
Andrew picked it up, but it's not in linus' tree yet.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
In reply to David HildenbrandQuoted 4 lines · David Hildenbrand
We want to improve error handling while adding memory by allowing to use
arch_remove_memory()and__remove_pages()even ifCONFIG_MEMORY_HOTREMOVEis not set to e.g., implement something like:
Quoted 8 lines
arch_add_memory() rc = do_something(); if (rc) { arch_remove_memory(); }We won't get rid of
CONFIG_MEMORY_HOTREMOVEfor now, as it will require quite some dependencies for memory offlining.
If we cannot really remove CONFIG_MEMORY_HOTREMOVE altogether then why not simply add an empty placeholder for arch_remove_memory when the config is disabled?
Quoted 313 lines
Cc Tony Luck <[email protected]>Cc Fenghua Yu <[email protected]>Cc Benjamin Herrenschmidt <[email protected]>Cc Paul Mackerras <[email protected]>Cc Michael Ellerman <[email protected]>Cc Martin Schwidefsky <[email protected]>Cc Heiko Carstens <[email protected]>Cc Yoshinori Sato <[email protected]>Cc Rich Felker <[email protected]>Cc Dave Hansen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Andrew Morton <[email protected]>Cc Michal Hocko <[email protected]>Cc Mike Rapoport <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc Christophe Leroy <[email protected]>Cc Nicholas Piggin <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Rob Herring <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Andrew Banman <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Qian Cai <[email protected]>Cc Mathieu Malaterre <[email protected]>Cc Baoquan He <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Anshuman Khandual <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/arm64/mm/mmu.c | 2 -- arch/ia64/mm/init.c | 2 -- arch/powerpc/mm/mem.c | 2 -- arch/s390/mm/init.c | 2 -- arch/sh/mm/init.c | 2 -- arch/x86/mm/init_32.c | 2 -- arch/x86/mm/init_64.c | 2 -- drivers/base/memory.c | 2 -- include/linux/memory.h | 2 -- include/linux/memory_hotplug.h | 2 -- mm/memory_hotplug.c | 2 -- mm/sparse.c | 6 ------ 12 files changed, 28 deletions(-)arch/arm64/mm/mmu.c +0
-2@@ -1084,7 +1084,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start >>PAGE_SHIFT , size >>PAGE_SHIFT , restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -1103,4 +1102,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif arch/ia64/mm/init.c +0
-2@@ -681,7 +681,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -693,4 +692,3 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif arch/powerpc/mm/mem.c +0
-2@@ -130,7 +130,6 @@ int __ref arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void __ref arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -164,7 +163,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, pr_warn("Hash collision while resizing HPT\n"); }#endif -#endif /* CONFIG_MEMORY_HOTPLUG */ #ifndef CONFIG_NEED_MULTIPLE_NODES void __init mem_topology_setup (void )arch/s390/mm/init.c +0
-2@@ -239,7 +239,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return rc; } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -251,5 +250,4 @@ void arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); vmem_remove_mapping(start, size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ arch/sh/mm/init.c +0
-2@@ -429,7 +429,6 @@ int memory_add_physaddr_to_nid(u64 addr)EXPORT_SYMBOL_GPL (memory_add_physaddr_to_nid);#endif -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -440,5 +439,4 @@ void arch_remove_memory(int nid, u64 start, u64 size, zone =page_zone (pfn_to_page (start_pfn)); __remove_pages(zone, start_pfn, nr_pages, altmap); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ arch/x86/mm/init_32.c +0
-2@@ -860,7 +860,6 @@ int arch_add_memory(int nid, u64 start, u64 size,return __add_pages (nid, start_pfn, nr_pages, restrictions); } -#ifdef CONFIG_MEMORY_HOTREMOVE void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap) {@@ -872,7 +871,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, __remove_pages(zone, start_pfn, nr_pages, altmap); }#endif -#endif int kernel_set_to_readonly__read_mostly ;arch/x86/mm/init_64.c +0
-2@@ -1196,7 +1196,6 @@ void __ref vmemmap_free(unsigned long start, unsigned long end,remove_pagetable (start, end, false, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void __meminit kernel_physical_mapping_remove (unsigned long start,unsigned long end) {@@ -1221,7 +1220,6 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size,__remove_pages (zone, start_pfn, nr_pages, altmap); kernel_physical_mapping_remove(start, start + size); } -#endif #endif /* CONFIG_MEMORY_HOTPLUG */ static struct kcore_list kcore_vsyscall;drivers/base/memory.c +0
-2@@ -727,7 +727,6 @@ int hotplug_memory_register(int nid, struct mem_section *section)return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE static void unregister_memory (struct memory_block *memory) {@@ -766,7 +765,6 @@ void unregister_memory_section(struct mem_section *section) out_unlock:mutex_unlock (&mem_sysfs_mutex); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ /* return true if the memory block is offlined, otherwise, return false */ bool is_memblock_offlined (struct memory_block *mem)include/linux/memory.h +0
-2@@ -112,9 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int hotplug_memory_register (int nid,struct mem_section *section); -#ifdef CONFIG_MEMORY_HOTREMOVE extern void unregister_memory_section (struct mem_section *); -#endif extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);include/linux/memory_hotplug.h +0
-2@@ -123,12 +123,10 @@ static inline bool movable_node_is_enabled(void)return movable_node_enabled; } -#ifdef CONFIG_MEMORY_HOTREMOVE extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap);extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -#endif /* CONFIG_MEMORY_HOTREMOVE */ /* * Do we want sysfs memblock files created. This will allow userspace to online mm/memory_hotplug.c +0
-2@@ -318,7 +318,6 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn,return err; } -#ifdef CONFIG_MEMORY_HOTREMOVE /* find the smallest valid pfn in the range [start_pfn, end_pfn) */ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone, unsigned long start_pfn,@@ -582,7 +581,6 @@ void __remove_pages(struct zone *zone, unsigned long phys_start_pfn,set_zone_contiguous (zone); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ int set_online_page_callback (online_page_callback_t callback) {mm/sparse.c +0
-6@@ -604,7 +604,6 @@ static void __kfree_section_memmap(struct page *memmap,vmemmap_free (start, end, altmap); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long start = (unsigned long )memmap;@@ -612,7 +611,6 @@ static void free_map_bootmem(struct page *memmap) vmemmap_free(start, end, NULL); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #else static struct page *__kmalloc_section_memmap (void ) {@@ -651,7 +649,6 @@ static void __kfree_section_memmap(struct page *memmap, get_order(sizeof(struct page) * PAGES_PER_SECTION)); } -#ifdef CONFIG_MEMORY_HOTREMOVE static void free_map_bootmem (struct page *memmap) {unsigned long maps_section_nr, removing_section_nr, i;@@ -681,7 +678,6 @@ static void free_map_bootmem(struct page *memmap) put_page_bootmem(page); } } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_SPARSEMEM_VMEMMAP */ /** @@ -746,7 +742,6 @@ int __meminit sparse_add_one_section(int nid, unsigned long start_pfn,return ret; } -#ifdef CONFIG_MEMORY_HOTREMOVE#ifdef CONFIG_MEMORY_FAILURE static void clear_hwpoisoned_pages (struct page *memmap,int nr_pages) {@@ -823,5 +818,4 @@ void sparse_remove_one_section(struct zone *zone, struct mem_section *ms, PAGES_PER_SECTION - map_offset); free_section_usemap(memmap, usemap, altmap); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ #endif /* CONFIG_MEMORY_HOTPLUG */ Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 11 lines · Michal Hocko
Quoted 10 lines · David Hildenbrand
We want to improve error handling while adding memory by allowing to use
arch_remove_memory()and__remove_pages()even ifCONFIG_MEMORY_HOTREMOVEis not set to e.g., implement something like:arch_add_memory() rc = do_something(); if (rc) { arch_remove_memory(); }
Quoted 6 lines
Quoted 2 lines
We won't get rid of
CONFIG_MEMORY_HOTREMOVEfor now, as it will require quite some dependencies for memory offlining.If we cannot really remove
CONFIG_MEMORY_HOTREMOVEaltogether then why not simply add an empty placeholder forarch_remove_memorywhen the config is disabled?
In other words, can we replace this by something as simple as:
include/linux/memory_hotplug.h +14 -0
@@ -128,6 +128,20 @@ extern void arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap);extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); +#else +/* +* Allow code using +* arch_add_memory(); +* rc = do_something(); +* if (rc) +* arch_remove_memory(); +* +* without ifdefery. +*/ +static inline void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +}#endif /* CONFIG_MEMORY_HOTREMOVE */ /*
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
Quoted 37 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 10 lines · David Hildenbrand
We want to improve error handling while adding memory by allowing to use
arch_remove_memory()and__remove_pages()even ifCONFIG_MEMORY_HOTREMOVEis not set to e.g., implement something like:arch_add_memory() rc = do_something(); if (rc) { arch_remove_memory(); }Quoted 6 lines
Quoted 2 lines
We won't get rid of
CONFIG_MEMORY_HOTREMOVEfor now, as it will require quite some dependencies for memory offlining.If we cannot really remove
CONFIG_MEMORY_HOTREMOVEaltogether then why not simply add an empty placeholder forarch_remove_memorywhen the config is disabled?In other words, can we replace this by something as simple as:
include/linux/memory_hotplug.h +7
-0@@ -128,6 +128,20 @@ extern void arch_remove_memory(int nid, u64 start, u64 size, struct vmem_altmap *altmap);extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); +#else +/* +* Allow code using +* arch_add_memory(); +* rc = do_something(); +* if (rc) +* arch_remove_memory();
include/linux/memory_hotplug.h extern void arch_remove_memory(int nid, u64 start, u64 size, · line 147
Quoted 10 lines
include/linux/memory_hotplug.h +7
-0+* +* without ifdefery. +*/ +static inline void arch_remove_memory (int nid,u64 start,u64 size, +struct vmem_altmap *altmap) +{ +}#endif /* CONFIG_MEMORY_HOTREMOVE */ /*
A system configured without CONFIG_MEMORY_HOTREMOVE should not suddenly behave worse than before when adding of memory fails. What you suggest result in that.
The goal should be to force architectures to properly implement arch_remove_memory() right from the start - which is the case for all architectures after this patch set *except* arm, for which a proper implementation is on the way.
So I'm leaving it like it is. arch_remove_memory() will be mandatory for architectures implementing arch_add_memory().
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 06/11] mm/memory_hotplug: Allow arch_remove_pages() without CONFIG_MEMORY_HOTREMOVE
On Mon 15-07-19 12:54:20, David Hildenbrand wrote:
[...]
Quoted 2 lines
So I'm leaving it like it is.
arch_remove_memory()will be mandatory for architectures implementingarch_add_memory().
I do agree that removing CONFIG_MEMORY_HOTREMOVE makes some sense. But this patch being a mid step should be simpler rather than going half way to get there. I would have liked the above for the purpose of this patch more and then go with another one to remove the config altogether. But Andrew has already sent his patch bomb including this series to Linus so this is all moot.
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 7/11 mm/memory_hotplug: Create memory block devices after arch_add_memory()
In reply to David HildenbrandOnly memory to be added to the buddy and to be onlined/offlined by user space using /sys/devices/system/memory/... needs (and should have!) memory block devices.
Factor out creation of memory block devices. Create all devices after arch_add_memory() succeeded. We can later drop the want_memblock parameter, because it is now effectively stale.
Only after memory block devices have been added, memory can be onlined by user space. This implies, that memory is not visible to user space at all before arch_add_memory() succeeded.
While at it
- use WARN_ON_ONCE instead of BUG_ON in moved unregister_memory()
- introduce find_memory_block_by_id() to search via block id
- Use find_memory_block_by_id() in init_memory_block() to catch duplicates
--- drivers/base/memory.c | 82 +++++++++++++++++++++++++++--------------- include/linux/memory.h | 2 +- mm/memory_hotplug.c | 15 ++++---- 3 files changed, 63 insertions(+), 36 deletions(-)
drivers/base/memory.c +54 -28
@@ -39,6 +39,11 @@ static inline int base_memory_block_id(int section_nr)return section_nr / sections_per_block; } +static inline int pfn_to_block_id (unsigned long pfn) +{ +return base_memory_block_id (pfn_to_section_nr (pfn)); +} +static int memory_subsys_online (struct device *dev);static int memory_subsys_offline (struct device *dev);
@@ -582,10 +587,9 @@ int __weak arch_get_memory_phys_device(unsigned long start_pfn)* A reference for the returned object is held and the reference for the * hinted object is released. */ -struct memory_block *find_memory_block_hinted (struct mem_section *section, -struct memory_block *hint) +static struct memory_block *find_memory_block_by_id (int block_id, +struct memory_block *hint) { -int block_id =base_memory_block_id (__section_nr (section));struct device *hintdev = hint ? &hint->dev :NULL ;struct device *dev;
@@ -597,6 +601,14 @@ struct memory_block *find_memory_block_hinted(struct mem_section *section,return to_memory_block (dev); } +struct memory_block *find_memory_block_hinted (struct mem_section *section, +struct memory_block *hint) +{ +int block_id =base_memory_block_id (__section_nr (section)); + +return find_memory_block_by_id (block_id, hint); +} +/* * For now, we have a linear search to go find the appropriate * memory_block corresponding to a particular phys_index. If
@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ; + mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + } mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;
@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block; + } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } } - -out:mutex_unlock (&mem_sysfs_mutex);return ret; } -static void -unregister_memory (struct memory_block *memory) -{ -BUG_ON (memory->dev.bus != &memory_subsys); - -/* drop the ref. we got via find_memory_block() */ -put_device (&memory->dev); -device_unregister (&memory->dev); -} -void unregister_memory_section (struct mem_section *section) {struct memory_block *mem;
include/linux/memory.h +1 -1
@@ -111,7 +111,7 @@ extern int register_memory_notifier(struct notifier_block *nb);extern void unregister_memory_notifier (struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb); -int hotplug_memory_register (int nid,struct mem_section *section); +int create_memory_block_devices (unsigned long start,unsigned long size);extern void unregister_memory_section (struct mem_section *);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);
mm/memory_hotplug.c +8 -7
@@ -259,13 +259,7 @@ static int __meminit __add_section(int nid, unsigned long phys_start_pfn,return -EEXIST ; ret =sparse_add_one_section (nid, phys_start_pfn, altmap); -if (ret <0 ) -return ret; - -if (!want_memblock) -return 0 ; - -return hotplug_memory_register (nid,__pfn_to_section (phys_start_pfn)); +return ret <0 ? ret :0 ; }/*
@@ -1107,6 +1101,13 @@ int __ref add_memory_resource(int nid, struct resource *res)if (ret <0 ) goto error; +/* create memory block devices after memory was added */ + ret =create_memory_block_devices (start, size); +if (ret) { +arch_remove_memory (nid, start, size,NULL ); + goto error; + } +if (new_node) {/* If sysfs file of new node can't be created, cpu on the node * can't be hot-added. There is no rollback way now.
Signature · 1 line
2.20.1
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
Quoted 19 lines · David Hildenbrand
Only memory to be added to the buddy and to be onlined/offlined by user space using
/sys/devices/system/memory/... needs (and should have!) memory block devices.Factor out creation of memory block devices. Create all devices after
arch_add_memory()succeeded. We can later drop thewant_memblockparameter, because it is now effectively stale.Only after memory block devices have been added, memory can be onlined by user space. This implies, that memory is not visible to user space at all before
arch_add_memory()succeeded.While at it
- useWARN_ON_ONCEinstead ofBUG_ONin movedunregister_memory()
- introducefind_memory_block_by_id()to search via block id
- Usefind_memory_block_by_id()ininit_memory_block()to catch duplicates
Quoted 15 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Ingo Molnar <[email protected]>Cc Andrew Banman <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
LGTM
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
In reply to David HildenbrandQuoted 12 lines · David Hildenbrand
Only memory to be added to the buddy and to be onlined/offlined by user space using
/sys/devices/system/memory/... needs (and should have!) memory block devices.Factor out creation of memory block devices. Create all devices after
arch_add_memory()succeeded. We can later drop thewant_memblockparameter, because it is now effectively stale.Only after memory block devices have been added, memory can be onlined by user space. This implies, that memory is not visible to user space at all before
arch_add_memory()succeeded.
Quoted 5 lines
While at it
- useWARN_ON_ONCEinstead ofBUG_ONin movedunregister_memory()
- introducefind_memory_block_by_id()to search via block id
- Usefind_memory_block_by_id()ininit_memory_block()to catch duplicates
Generally looks good to me besides two tiny comments.
Quoted 67 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Ingo Molnar <[email protected]>Cc Andrew Banman <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/base/memory.c | 82 +++++++++++++++++++++++++++--------------- include/linux/memory.h | 2 +- mm/memory_hotplug.c | 15 ++++---- 3 files changed, 63 insertions(+), 36 deletions(-)drivers/base/memory.c +15
-3@@ -39,6 +39,11 @@ static inline int base_memory_block_id(int section_nr)return section_nr / sections_per_block; } +static inline int pfn_to_block_id (unsigned long pfn) +{ +return base_memory_block_id (pfn_to_section_nr (pfn)); +} +static int memory_subsys_online (struct device *dev);static int memory_subsys_offline (struct device *dev);@@ -582,10 +587,9 @@ int __weak arch_get_memory_phys_device(unsigned long start_pfn)* A reference for the returned object is held and the reference for the * hinted object is released. */ -struct memory_block *find_memory_block_hinted (struct mem_section *section, -struct memory_block *hint) +static struct memory_block *find_memory_block_by_id (int block_id, +struct memory_block *hint) { -int block_id =base_memory_block_id (__section_nr (section));struct device *hintdev = hint ? &hint->dev :NULL ;struct device *dev;@@ -597,6 +601,14 @@ struct memory_block *find_memory_block_hinted(struct mem_section *section,return to_memory_block (dev); } +struct memory_block *find_memory_block_hinted (struct mem_section *section, +struct memory_block *hint) +{ +int block_id =base_memory_block_id (__section_nr (section)); + +return find_memory_block_by_id (block_id, hint); +} +/* * For now, we have a linear search to go find the appropriate * memory_block corresponding to a particular phys_index. If
drivers/base/memory.c static int init_memory_block(struct memory_block **memory, int block_id, · line 677
Quoted 8 lines
drivers/base/memory.c +5
-0@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ; + mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + }
find_memory_block_by_id() is not that close to the main idea in this patch. Would it be better to split this part?
Quoted 53 lines
drivers/base/memory.c +25
-13mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block;
drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 761
Quoted 10 lines
drivers/base/memory.c +9
-0+ } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } }
Would it be better to do this in reverse order?
And unregister_memory() would free mem, so it is still necessary to set section_count to 0?
Quoted 67 lines
drivers/base/memory.c +0
-12- -out:mutex_unlock (&mem_sysfs_mutex);return ret; } -static void -unregister_memory (struct memory_block *memory) -{ -BUG_ON (memory->dev.bus != &memory_subsys); - -/* drop the ref. we got via find_memory_block() */ -put_device (&memory->dev); -device_unregister (&memory->dev); -} -void unregister_memory_section (struct mem_section *section) {struct memory_block *mem;include/linux/memory.h +1
-1@@ -111,7 +111,7 @@ extern int register_memory_notifier(struct notifier_block *nb);extern void unregister_memory_notifier (struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb); -int hotplug_memory_register (int nid,struct mem_section *section); +int create_memory_block_devices (unsigned long start,unsigned long size);extern void unregister_memory_section (struct mem_section *);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);mm/memory_hotplug.c +8
-7@@ -259,13 +259,7 @@ static int __meminit __add_section(int nid, unsigned long phys_start_pfn,return -EEXIST ; ret =sparse_add_one_section (nid, phys_start_pfn, altmap); -if (ret <0 ) -return ret; - -if (!want_memblock) -return 0 ; - -return hotplug_memory_register (nid,__pfn_to_section (phys_start_pfn)); +return ret <0 ? ret :0 ; }/* @@ -1107,6 +1101,13 @@ int __ref add_memory_resource(int nid, struct resource *res)if (ret <0 ) goto error; +/* create memory block devices after memory was added */ + ret =create_memory_block_devices (start, size); +if (ret) { +arch_remove_memory (nid, start, size,NULL ); + goto error; + } +if (new_node) {/* If sysfs file of new node can't be created, cpu on the node * can't be hot-added. There is no rollback way now. Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
Quoted 7 lines
Quoted 7 lines
/*
* For now, we have a linear search to go find the appropriate
*memory_blockcorresponding to a particularphys_index. Ifdrivers/base/memory.c +0
-0@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ;
Quoted 8 lines
drivers/base/memory.c static int init_memory_block(struct memory_block **memory, int block_id, · line 677
Quoted 5 lines
drivers/base/memory.c +5
-0+ mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + }
find_memory_block_by_id()is not that close to the main idea in this patch. Would it be better to split this part?
I played with that but didn't like the temporary results (e.g. having to export find_memory_block_by_id()). I'll stick to this for now.
Quoted 67 lines
Quoted 54 lines
drivers/base/memory.c +25
-13mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block;drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 761
Quoted 10 lines
drivers/base/memory.c +9
-0+ } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } }Would it be better to do this in reverse order?
Quoted 2 lines
And
unregister_memory()would free mem, so it is still necessary to setsection_countto 0?
1. I kept the existing behavior (setting it to 0) for now. I am planning to eventually remove the section count completely (it could be beneficial to detect removing of partially populated memory blocks).
2. Reverse order: We would have to start with "block_id - 1", I don't like that better.
Thanks for having a look!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
Quoted 86 lines · David Hildenbrand
Quoted 7 lines
Quoted 7 lines
/*
* For now, we have a linear search to go find the appropriate
*memory_blockcorresponding to a particularphys_index. Ifdrivers/base/memory.c +0
-0@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ;Quoted 8 lines
drivers/base/memory.c static int init_memory_block(struct memory_block **memory, int block_id, · line 677
Quoted 5 lines
drivers/base/memory.c +5
-0+ mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + }
find_memory_block_by_id()is not that close to the main idea in this patch. Would it be better to split this part?I played with that but didn't like the temporary results (e.g. having to export
find_memory_block_by_id()). I'll stick to this for now.Quoted 67 lines
Quoted 54 lines
drivers/base/memory.c +25
-13mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block;drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 761
Quoted 10 lines
drivers/base/memory.c +9
-0+ } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } }Would it be better to do this in reverse order?
Quoted 6 lines
Quoted 2 lines
And
unregister_memory()would free mem, so it is still necessary to setsection_countto 0?1. I kept the existing behavior (setting it to 0) for now. I am planning to eventually remove the section count completely (it could be beneficial to detect removing of partially populated memory blocks).
Correction: We already use it to block offlining of partially populated memory blocks \o/
Quoted 6 lines
2. Reverse order: We would have to start with "
block_id- 1", I don't like that better.Thanks for having a look!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
Quoted 90 lines · David Hildenbrand
Quoted 89 lines · David Hildenbrand
Quoted 7 lines
Quoted 7 lines
/*
* For now, we have a linear search to go find the appropriate
*memory_blockcorresponding to a particularphys_index. Ifdrivers/base/memory.c +0
-0@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ;Quoted 8 lines
drivers/base/memory.c static int init_memory_block(struct memory_block **memory, int block_id, · line 677
Quoted 5 lines
drivers/base/memory.c +5
-0+ mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + }
find_memory_block_by_id()is not that close to the main idea in this patch. Would it be better to split this part?I played with that but didn't like the temporary results (e.g. having to export
find_memory_block_by_id()). I'll stick to this for now.Quoted 69 lines
Quoted 54 lines
drivers/base/memory.c +25
-13mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block;drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 761
Quoted 10 lines
drivers/base/memory.c +9
-0+ } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } }Would it be better to do this in reverse order?
And
unregister_memory()would free mem, so it is still necessary to setsection_countto 0?
Quoted 6 lines
Quoted 3 lines
1. I kept the existing behavior (setting it to 0) for now. I am planning to eventually remove the section count completely (it could be beneficial to detect removing of partially populated memory blocks).
Correction: We already use it to block offlining of partially populated memory blocks \o/
Would you mind letting me know where we leverage this?
Quoted 14 lines
Quoted 7 lines
2. Reverse order: We would have to start with "
block_id- 1", I don't like that better.Thanks for having a look!
Signature · 3 lines
Thanks, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
Quoted 91 lines · Wei Yang
Quoted 90 lines · David Hildenbrand
Quoted 89 lines · David Hildenbrand
Quoted 7 lines
Quoted 7 lines
/*
* For now, we have a linear search to go find the appropriate
*memory_blockcorresponding to a particularphys_index. Ifdrivers/base/memory.c +0
-0@@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ;Quoted 8 lines
drivers/base/memory.c static int init_memory_block(struct memory_block **memory, int block_id, · line 677
Quoted 5 lines
drivers/base/memory.c +5
-0+ mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + }
find_memory_block_by_id()is not that close to the main idea in this patch. Would it be better to split this part?I played with that but didn't like the temporary results (e.g. having to export
find_memory_block_by_id()). I'll stick to this for now.Quoted 69 lines
Quoted 54 lines
drivers/base/memory.c +25
-13mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block;drivers/base/memory.c static int add_memory_block(int base_section_nr) · line 761
Quoted 10 lines
drivers/base/memory.c +9
-0+ } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } }Would it be better to do this in reverse order?
And
unregister_memory()would free mem, so it is still necessary to setsection_countto 0?
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
1. I kept the existing behavior (setting it to 0) for now. I am planning to eventually remove the section count completely (it could be beneficial to detect removing of partially populated memory blocks).
Correction: We already use it to block offlining of partially populated memory blocks \o/
Would you mind letting me know where we leverage this?
Sure:
drivers/base/memory.c:memory_subsys_offline()
if (mem->section_count != sections_per_block)
return -EINVAL;
I would have expected such checks in the offline_pages() function instead.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 07/11] mm/memory_hotplug: Create memory block devices after arch_add_memory()
In reply to David HildenbrandQuoted 11 lines · David Hildenbrand
Only memory to be added to the buddy and to be onlined/offlined by user space using
/sys/devices/system/memory/... needs (and should have!) memory block devices.Factor out creation of memory block devices. Create all devices after
arch_add_memory()succeeded. We can later drop thewant_memblockparameter, because it is now effectively stale.Only after memory block devices have been added, memory can be onlined by user space. This implies, that memory is not visible to user space at all before
arch_add_memory()succeeded.
I like the memblock API to go away from the low level hotplug handling. The current implementation is just too convoluted and I remember I was fighting with subtle expectations wired deep in call chains when touching that code in the past (some memblocks didn't get created etc.). Maybe those have been addressed in the meantime.
Quoted 2 lines
While at it
- useWARN_ON_ONCEinstead ofBUG_ONin movedunregister_memory()
This would better be a separate patch with an explanation
Quoted 4 lines
- introduce
find_memory_block_by_id()to search via block id
- Usefind_memory_block_by_id()ininit_memory_block()to catch duplicates
Quoted 15 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Ingo Molnar <[email protected]>Cc Andrew Banman <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Qian Cai <[email protected]>Cc Wei Yang <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Other than that looks good to me.
Quoted 190 lines
--- drivers/base/memory.c | 82 +++++++++++++++++++++++++++--------------- include/linux/memory.h | 2 +- mm/memory_hotplug.c | 15 ++++---- 3 files changed, 63 insertions(+), 36 deletions(-)drivers/base/memory.c +54
-28@@ -39,6 +39,11 @@ static inline int base_memory_block_id(int section_nr)return section_nr / sections_per_block; } +static inline int pfn_to_block_id (unsigned long pfn) +{ +return base_memory_block_id (pfn_to_section_nr (pfn)); +} +static int memory_subsys_online (struct device *dev);static int memory_subsys_offline (struct device *dev);@@ -582,10 +587,9 @@ int __weak arch_get_memory_phys_device(unsigned long start_pfn)* A reference for the returned object is held and the reference for the * hinted object is released. */ -struct memory_block *find_memory_block_hinted (struct mem_section *section, -struct memory_block *hint) +static struct memory_block *find_memory_block_by_id (int block_id, +struct memory_block *hint) { -int block_id =base_memory_block_id (__section_nr (section));struct device *hintdev = hint ? &hint->dev :NULL ;struct device *dev;@@ -597,6 +601,14 @@ struct memory_block *find_memory_block_hinted(struct mem_section *section,return to_memory_block (dev); } +struct memory_block *find_memory_block_hinted (struct mem_section *section, +struct memory_block *hint) +{ +int block_id =base_memory_block_id (__section_nr (section)); + +return find_memory_block_by_id (block_id, hint); +} +/* * For now, we have a linear search to go find the appropriate * memory_block corresponding to a particular phys_index. If @@ -658,6 +670,11 @@ static int init_memory_block(struct memory_block **memory, int block_id,unsigned long start_pfn;int ret =0 ; + mem =find_memory_block_by_id (block_id,NULL ); +if (mem) { +put_device (&mem->dev); +return -EEXIST ; + } mem =kzalloc (sizeof (*mem),GFP_KERNEL );if (!mem)return -ENOMEM ;@@ -699,44 +716,53 @@ static int add_memory_block(int base_section_nr)return 0 ; } +static void unregister_memory (struct memory_block *memory) +{ +if (WARN_ON_ONCE (memory->dev.bus != &memory_subsys)) +return ; + +/* drop the ref. we got via find_memory_block() */ +put_device (&memory->dev); +device_unregister (&memory->dev); +} +/* -* need an interface for the VM to add new memory regions, -* but without onlining it. +* Create memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* will be initialized as offline. */ -int hotplug_memory_register (int nid,struct mem_section *section) +int create_memory_block_devices (unsigned long start,unsigned long size) { -int block_id =base_memory_block_id (__section_nr (section)); -int ret =0 ; +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +unsigned long block_id; +int ret =0 ; -mutex_lock (&mem_sysfs_mutex); +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ()))) +return -EINVAL ; - mem =find_memory_block (section); -if (mem) { - mem->section_count++; -put_device (&mem->dev); - }else { +mutex_lock (&mem_sysfs_mutex); +for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret =init_memory_block (&mem, block_id,MEM_OFFLINE );if (ret) - goto out; - mem->section_count++; +break ; + mem->section_count = sections_per_block; + } +if (ret) { + end_block_id = block_id; +for (block_id = start_block_id; block_id != end_block_id; + block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); + mem->section_count =0 ; +unregister_memory (mem); + } } - -out:mutex_unlock (&mem_sysfs_mutex);return ret; } -static void -unregister_memory (struct memory_block *memory) -{ -BUG_ON (memory->dev.bus != &memory_subsys); - -/* drop the ref. we got via find_memory_block() */ -put_device (&memory->dev); -device_unregister (&memory->dev); -} -void unregister_memory_section (struct mem_section *section) {struct memory_block *mem;include/linux/memory.h +1
-1@@ -111,7 +111,7 @@ extern int register_memory_notifier(struct notifier_block *nb);extern void unregister_memory_notifier (struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb); -int hotplug_memory_register (int nid,struct mem_section *section); +int create_memory_block_devices (unsigned long start,unsigned long size);extern void unregister_memory_section (struct mem_section *);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);mm/memory_hotplug.c +8
-7@@ -259,13 +259,7 @@ static int __meminit __add_section(int nid, unsigned long phys_start_pfn,return -EEXIST ; ret =sparse_add_one_section (nid, phys_start_pfn, altmap); -if (ret <0 ) -return ret; - -if (!want_memblock) -return 0 ; - -return hotplug_memory_register (nid,__pfn_to_section (phys_start_pfn)); +return ret <0 ? ret :0 ; }/* @@ -1107,6 +1101,13 @@ int __ref add_memory_resource(int nid, struct resource *res)if (ret <0 ) goto error; +/* create memory block devices after memory was added */ + ret =create_memory_block_devices (start, size); +if (ret) { +arch_remove_memory (nid, start, size,NULL ); + goto error; + } +if (new_node) {/* If sysfs file of new node can't be created, cpu on the node * can't be hot-added. There is no rollback way now. Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 8/11 mm/memory_hotplug: Drop MHP_MEMBLOCK_API
In reply to David HildenbrandNo longer needed, the callers of arch_add_memory() can handle this manually.
--- include/linux/memory_hotplug.h | 8 -------- mm/memory_hotplug.c | 9 +++------ 2 files changed, 3 insertions(+), 14 deletions(-)
include/linux/memory_hotplug.h +0 -8
@@ -128,14 +128,6 @@ extern void arch_remove_memory(int nid, u64 start, u64 size,extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -/* -* Do we want sysfs memblock files created. This will allow userspace to online -* and offline memory explicitly. Lack of this bit means that the caller has to -* call move_pfn_range_to_zone to finish the initialization. -*/ - -#define MHP_MEMBLOCK_API (1<<0) -/* reasonably generic interface to expand the physical pages */ extern int __add_pages (int nid,unsigned long start_pfn,unsigned long nr_pages,struct mhp_restrictions *restrictions);
mm/memory_hotplug.c +3 -6
@@ -251,7 +251,7 @@ void __init register_page_bootmem_info_node(struct pglist_data *pgdat)#endif /* CONFIG_HAVE_BOOTMEM_INFO_NODE */ static int __meminit __add_section (int nid,unsigned long phys_start_pfn, -struct vmem_altmap *altmap,bool want_memblock) +struct vmem_altmap *altmap) {int ret;
@@ -294,8 +294,7 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn, }for (i = start_sec; i <= end_sec; i++) { - err =__add_section (nid,section_nr_to_pfn (i), altmap, - restrictions->flags &MHP_MEMBLOCK_API ); + err =__add_section (nid,section_nr_to_pfn (i), altmap);/* * EEXIST is finally dealt with by ioresource collision
@@ -1067,9 +1066,7 @@ static int online_memory_block(struct memory_block *mem, void *arg)*/ int __ref add_memory_resource (int nid,struct resource *res) { -struct mhp_restrictions restrictions = { - .flags =MHP_MEMBLOCK_API , - }; +struct mhp_restrictions restrictions = {};u64 start, size;bool new_node = false;int ret;
Signature · 1 line
2.20.1
Re: [PATCH v3 08/11] mm/memory_hotplug: Drop MHP_MEMBLOCK_API
Quoted 14 lines · David Hildenbrand
No longer needed, the callers of
arch_add_memory()can handle this manually.Cc Andrew Morton <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Wei Yang <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Qian Cai <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 60 lines
---
include/linux/memory_hotplug.h| 8 --------mm/memory_hotplug.c| 9 +++------2 files changed, 3 insertions(+), 14 deletions(-)include/linux/memory_hotplug.h +0
-8@@ -128,14 +128,6 @@ extern void arch_remove_memory(int nid, u64 start, u64 size,extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -/* -* Do we want sysfs memblock files created. This will allow userspace to online -* and offline memory explicitly. Lack of this bit means that the caller has to -* call move_pfn_range_to_zone to finish the initialization. -*/ - -#define MHP_MEMBLOCK_API (1<<0) -/* reasonably generic interface to expand the physical pages */ extern int __add_pages (int nid,unsigned long start_pfn,unsigned long nr_pages,struct mhp_restrictions *restrictions);mm/memory_hotplug.c +3
-6@@ -251,7 +251,7 @@ void __init register_page_bootmem_info_node(struct pglist_data *pgdat)#endif /* CONFIG_HAVE_BOOTMEM_INFO_NODE */ static int __meminit __add_section (int nid,unsigned long phys_start_pfn, -struct vmem_altmap *altmap,bool want_memblock) +struct vmem_altmap *altmap) {int ret;@@ -294,8 +294,7 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn, }for (i = start_sec; i <= end_sec; i++) { - err =__add_section (nid,section_nr_to_pfn (i), altmap, - restrictions->flags &MHP_MEMBLOCK_API ); + err =__add_section (nid,section_nr_to_pfn (i), altmap);/* * EEXIST is finally dealt with by ioresource collision @@ -1067,9 +1066,7 @@ static int online_memory_block(struct memory_block *mem, void *arg)*/ int __ref add_memory_resource (int nid,struct resource *res) { -struct mhp_restrictions restrictions = { - .flags =MHP_MEMBLOCK_API , - }; +struct mhp_restrictions restrictions = {};u64 start, size;bool new_node = false;int ret;Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 08/11] mm/memory_hotplug: Drop MHP_MEMBLOCK_API
In reply to David HildenbrandQuoted 14 lines · David Hildenbrand
No longer needed, the callers of
arch_add_memory()can handle this manually.Cc Andrew Morton <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Wei Yang <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Qian Cai <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 61 lines
--- include/linux/memory_hotplug.h | 8 -------- mm/memory_hotplug.c | 9 +++------ 2 files changed, 3 insertions(+), 14 deletions(-)include/linux/memory_hotplug.h +0
-8@@ -128,14 +128,6 @@ extern void arch_remove_memory(int nid, u64 start, u64 size,extern void __remove_pages (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); -/* -* Do we want sysfs memblock files created. This will allow userspace to online -* and offline memory explicitly. Lack of this bit means that the caller has to -* call move_pfn_range_to_zone to finish the initialization. -*/ - -#define MHP_MEMBLOCK_API (1<<0) -/* reasonably generic interface to expand the physical pages */ extern int __add_pages (int nid,unsigned long start_pfn,unsigned long nr_pages,struct mhp_restrictions *restrictions);mm/memory_hotplug.c +3
-6@@ -251,7 +251,7 @@ void __init register_page_bootmem_info_node(struct pglist_data *pgdat)#endif /* CONFIG_HAVE_BOOTMEM_INFO_NODE */ static int __meminit __add_section (int nid,unsigned long phys_start_pfn, -struct vmem_altmap *altmap,bool want_memblock) +struct vmem_altmap *altmap) {int ret;@@ -294,8 +294,7 @@ int __ref __add_pages(int nid, unsigned long phys_start_pfn, }for (i = start_sec; i <= end_sec; i++) { - err =__add_section (nid,section_nr_to_pfn (i), altmap, - restrictions->flags &MHP_MEMBLOCK_API ); + err =__add_section (nid,section_nr_to_pfn (i), altmap);/* * EEXIST is finally dealt with by ioresource collision @@ -1067,9 +1066,7 @@ static int online_memory_block(struct memory_block *mem, void *arg)*/ int __ref add_memory_resource (int nid,struct resource *res) { -struct mhp_restrictions restrictions = { - .flags =MHP_MEMBLOCK_API , - }; +struct mhp_restrictions restrictions = {};u64 start, size;bool new_node = false;int ret;Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 9/11 mm/memory_hotplug: Remove memory block devices before arch_remove_memory()
In reply to David HildenbrandLet's factor out removing of memory block devices, which is only necessary for memory added via add_memory() and friends that created memory block devices. Remove the devices before calling arch_remove_memory().
This finishes factoring out memory block device handling from arch_add_memory() and arch_remove_memory().
--- drivers/base/memory.c | 37 ++++++++++++++++++------------------- drivers/base/node.c | 11 ++++++----- include/linux/memory.h | 2 +- include/linux/node.h | 6 ++---- mm/memory_hotplug.c | 5 +++-- 5 files changed, 30 insertions(+), 31 deletions(-)
drivers/base/memory.c +18 -19
@@ -763,32 +763,31 @@ int create_memory_block_devices(unsigned long start, unsigned long size)return ret; } -void unregister_memory_section (struct mem_section *section) +/* +* Remove memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* have to be offline. +*/ +void remove_memory_block_devices (unsigned long start,unsigned long size) { +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +const int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +int block_id; -if (WARN_ON_ONCE (!present_section (section))) +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())))return ;mutex_lock (&mem_sysfs_mutex); - -/* -* Some users of the memory hotplug do not want/need memblock to -* track all sections. Skip over those. -*/ - mem =find_memory_block (section); -if (!mem) - goto out_unlock; - -unregister_mem_sect_under_nodes (mem,__section_nr (section)); - - mem->section_count--; -if (mem->section_count =0 ) +for (block_id = start_block_id; block_id != end_block_id; block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); +if (WARN_ON_ONCE (!mem)) +continue ; + mem->section_count =0 ; +unregister_memory_block_under_nodes (mem);unregister_memory (mem); -else -put_device (&mem->dev); - -out_unlock: + }mutex_unlock (&mem_sysfs_mutex); }
drivers/base/node.c +6 -5
@@ -801,9 +801,10 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)return 0 ; } -/* unregister memory section under all nodes that it spans */ -int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +/* +* Unregister memory block device under all nodes that it spans. +*/ +int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn;
@@ -816,8 +817,8 @@ int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,return -ENOMEM ;nodes_clear (*unlinked_nodes); - sect_start_pfn =section_nr_to_pfn (phys_index); - sect_end_pfn = sect_start_pfn +PAGES_PER_SECTION -1 ; + sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); + sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {int nid;
include/linux/memory.h +1 -1
@@ -112,7 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int create_memory_block_devices (unsigned long start,unsigned long size); -extern void unregister_memory_section (struct mem_section *); +void remove_memory_block_devices (unsigned long start,unsigned long size);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);
include/linux/node.h +2 -4
@@ -139,8 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index); +extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,
@@ -176,8 +175,7 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {return 0 ; }
mm/memory_hotplug.c +3 -2
@@ -520,8 +520,6 @@ static void __remove_section(struct zone *zone, struct mem_section *ms,if (WARN_ON_ONCE (!valid_section (ms)))return ; -unregister_memory_section (ms); - scn_nr =__section_nr (ms); start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn);
@@ -1845,6 +1843,9 @@ void __ref __remove_memory(int nid, u64 start, u64 size)memblock_free (start, size);memblock_remove (start, size); +/* remove memory block devices before removing memory */ +remove_memory_block_devices (start, size); +arch_remove_memory (nid, start, size,NULL );__release_memory_resource (start, size);
Signature · 1 line
2.20.1
Re: [PATCH v3 09/11] mm/memory_hotplug: Remove memory block devices before arch_remove_memory()
Quoted 70 lines · David Hildenbrand
Let's factor out removing of memory block devices, which is only necessary for memory added via
add_memory()and friends that created memory block devices. Remove the devices before callingarch_remove_memory().This finishes factoring out memory block device handling from
arch_add_memory()andarch_remove_memory().Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Andrew Banman <[email protected]>Cc Ingo Molnar <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc Oscar Salvador <[email protected]>Cc Jonathan Cameron <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/base/memory.c | 37 ++++++++++++++++++------------------- drivers/base/node.c | 11 ++++++----- include/linux/memory.h | 2 +- include/linux/node.h | 6 ++---- mm/memory_hotplug.c | 5 +++-- 5 files changed, 30 insertions(+), 31 deletions(-)drivers/base/memory.c +11
-10@@ -763,32 +763,31 @@ int create_memory_block_devices(unsigned long start, unsigned long size)return ret; } -void unregister_memory_section (struct mem_section *section) +/* +* Remove memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* have to be offline. +*/ +void remove_memory_block_devices (unsigned long start,unsigned long size) { +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +const int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +int block_id; -if (WARN_ON_ONCE (!present_section (section))) +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())))return ;mutex_lock (&mem_sysfs_mutex); - -/* -* Some users of the memory hotplug do not want/need memblock to -* track all sections. Skip over those. -*/ - mem =find_memory_block (section); -if (!mem) - goto out_unlock;
drivers/base/memory.c int create_memory_block_devices(unsigned long start, unsigned long size) · line 787
Quoted 10 lines
drivers/base/memory.c +5
-5- -unregister_mem_sect_under_nodes (mem,__section_nr (section)); - - mem->section_count--; -if (mem->section_count =0 ) +for (block_id = start_block_id; block_id != end_block_id; block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); +if (WARN_ON_ONCE (!mem)) +continue ; + mem->section_count =0 ;
Is this step necessary?
Quoted 101 lines
drivers/base/memory.c +2
-4+unregister_memory_block_under_nodes (mem);unregister_memory (mem); - else - put_device(&mem->dev); - -out_unlock: + }mutex_unlock (&mem_sysfs_mutex); }drivers/base/node.c +6
-5@@ -801,9 +801,10 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)return 0 ; } -/* unregister memory section under all nodes that it spans */ -int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +/* +* Unregister memory block device under all nodes that it spans. +*/ +int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn;@@ -816,8 +817,8 @@ int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,return -ENOMEM ;nodes_clear (*unlinked_nodes); - sect_start_pfn =section_nr_to_pfn (phys_index); - sect_end_pfn = sect_start_pfn +PAGES_PER_SECTION -1 ; + sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); + sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {int nid;include/linux/memory.h +1
-1@@ -112,7 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int create_memory_block_devices (unsigned long start,unsigned long size); -extern void unregister_memory_section (struct mem_section *); +void remove_memory_block_devices (unsigned long start,unsigned long size);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);include/linux/node.h +2
-4@@ -139,8 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index); +extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,@@ -176,8 +175,7 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {return 0 ; }mm/memory_hotplug.c +3
-2@@ -520,8 +520,6 @@ static void __remove_section(struct zone *zone, struct mem_section *ms,if (WARN_ON_ONCE (!valid_section (ms)))return ; -unregister_memory_section (ms); - scn_nr =__section_nr (ms); start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn);@@ -1845,6 +1843,9 @@ void __ref __remove_memory(int nid, u64 start, u64 size)memblock_free (start, size);memblock_remove (start, size); +/* remove memory block devices before removing memory */ +remove_memory_block_devices (start, size); +arch_remove_memory (nid, start, size,NULL );__release_memory_resource (start, size);Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 09/11] mm/memory_hotplug: Remove memory block devices before arch_remove_memory()
Quoted 75 lines · Wei Yang
Quoted 74 lines · David Hildenbrand
Let's factor out removing of memory block devices, which is only necessary for memory added via
add_memory()and friends that created memory block devices. Remove the devices before callingarch_remove_memory().This finishes factoring out memory block device handling from
arch_add_memory()andarch_remove_memory().Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Andrew Banman <[email protected]>Cc Ingo Molnar <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc Oscar Salvador <[email protected]>Cc Jonathan Cameron <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/base/memory.c | 37 ++++++++++++++++++------------------- drivers/base/node.c | 11 ++++++----- include/linux/memory.h | 2 +- include/linux/node.h | 6 ++---- mm/memory_hotplug.c | 5 +++-- 5 files changed, 30 insertions(+), 31 deletions(-)drivers/base/memory.c +11
-14@@ -763,32 +763,31 @@ int create_memory_block_devices(unsigned long start, unsigned long size)return ret; } -void unregister_memory_section (struct mem_section *section) +/* +* Remove memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* have to be offline. +*/ +void remove_memory_block_devices (unsigned long start,unsigned long size) { +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +const int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +int block_id; -if (WARN_ON_ONCE (!present_section (section))) +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())))return ;mutex_lock (&mem_sysfs_mutex); - -/* -* Some users of the memory hotplug do not want/need memblock to -* track all sections. Skip over those. -*/ - mem =find_memory_block (section); -if (!mem) - goto out_unlock; - -unregister_mem_sect_under_nodes (mem,__section_nr (section)); - - mem->section_count--;
Quoted 8 lines
drivers/base/memory.c int create_memory_block_devices(unsigned long start, unsigned long size) · line 787
Quoted 6 lines
drivers/base/memory.c +5
-1-if (mem->section_count =0 ) +for (block_id = start_block_id; block_id != end_block_id; block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); +if (WARN_ON_ONCE (!mem)) +continue ; + mem->section_count =0 ;Is this step necessary?
It's what the previous code does, it might not be - I'll leave it like that for now. As mentioned in another reply, I might remove the section_count completely, eventually.
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 09/11] mm/memory_hotplug: Remove memory block devices before arch_remove_memory()
In reply to David HildenbrandQuoted 7 lines · David Hildenbrand
Let's factor out removing of memory block devices, which is only necessary for memory added via
add_memory()and friends that created memory block devices. Remove the devices before callingarch_remove_memory().This finishes factoring out memory block device handling from
arch_add_memory()andarch_remove_memory().
OK, this makes sense again. Just a nit. Calling find_memory_block_by_id for each memory block looks a bit suboptimal, especially when we are removing consequent physical memblocks. I have to confess that I do not know how expensive is the search and I also expect that there won't be that many memblocks in the removed range anyway as large setups have large memblocks.
Quoted 11 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc David Hildenbrand <[email protected]>Cc Andrew Morton <[email protected]>Cc Andrew Banman <[email protected]>Cc Ingo Molnar <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>
Quoted 8 lines
Cc Oscar Salvador <[email protected]>Cc Jonathan Cameron <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Arun KS <[email protected]>Cc Mathieu Malaterre <[email protected]>Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Other than that looks good to me.
Quoted 155 lines
--- drivers/base/memory.c | 37 ++++++++++++++++++------------------- drivers/base/node.c | 11 ++++++----- include/linux/memory.h | 2 +- include/linux/node.h | 6 ++---- mm/memory_hotplug.c | 5 +++-- 5 files changed, 30 insertions(+), 31 deletions(-)drivers/base/memory.c +18
-19@@ -763,32 +763,31 @@ int create_memory_block_devices(unsigned long start, unsigned long size)return ret; } -void unregister_memory_section (struct mem_section *section) +/* +* Remove memory block devices for the given memory area. Start and size +* have to be aligned to memory block granularity. Memory block devices +* have to be offline. +*/ +void remove_memory_block_devices (unsigned long start,unsigned long size) { +const int start_block_id =pfn_to_block_id (PFN_DOWN (start)); +const int end_block_id =pfn_to_block_id (PFN_DOWN (start + size));struct memory_block *mem; +int block_id; -if (WARN_ON_ONCE (!present_section (section))) +if (WARN_ON_ONCE (!IS_ALIGNED (start,memory_block_size_bytes ()) || + !IS_ALIGNED (size,memory_block_size_bytes ())))return ;mutex_lock (&mem_sysfs_mutex); - -/* -* Some users of the memory hotplug do not want/need memblock to -* track all sections. Skip over those. -*/ - mem =find_memory_block (section); -if (!mem) - goto out_unlock; - -unregister_mem_sect_under_nodes (mem,__section_nr (section)); - - mem->section_count--; -if (mem->section_count =0 ) +for (block_id = start_block_id; block_id != end_block_id; block_id++) { + mem =find_memory_block_by_id (block_id,NULL ); +if (WARN_ON_ONCE (!mem)) +continue ; + mem->section_count =0 ; +unregister_memory_block_under_nodes (mem);unregister_memory (mem); -else -put_device (&mem->dev); - -out_unlock: + }mutex_unlock (&mem_sysfs_mutex); }drivers/base/node.c +6
-5@@ -801,9 +801,10 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)return 0 ; } -/* unregister memory section under all nodes that it spans */ -int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +/* +* Unregister memory block device under all nodes that it spans. +*/ +int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn;@@ -816,8 +817,8 @@ int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,return -ENOMEM ;nodes_clear (*unlinked_nodes); - sect_start_pfn =section_nr_to_pfn (phys_index); - sect_end_pfn = sect_start_pfn +PAGES_PER_SECTION -1 ; + sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); + sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {int nid;include/linux/memory.h +1
-1@@ -112,7 +112,7 @@ extern void unregister_memory_notifier(struct notifier_block *nb);extern int register_memory_isolate_notifier (struct notifier_block *nb);extern void unregister_memory_isolate_notifier (struct notifier_block *nb);int create_memory_block_devices (unsigned long start,unsigned long size); -extern void unregister_memory_section (struct mem_section *); +void remove_memory_block_devices (unsigned long start,unsigned long size);extern int memory_dev_init (void );extern int memory_notify (unsigned long val,void *v);extern int memory_isolate_notify (unsigned long val,void *v);include/linux/node.h +2
-4@@ -139,8 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index); +extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,@@ -176,8 +175,7 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_mem_sect_under_nodes (struct memory_block *mem_blk, -unsigned long phys_index) +static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) {return 0 ; }mm/memory_hotplug.c +3
-2@@ -520,8 +520,6 @@ static void __remove_section(struct zone *zone, struct mem_section *ms,if (WARN_ON_ONCE (!valid_section (ms)))return ; -unregister_memory_section (ms); - scn_nr =__section_nr (ms); start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn);@@ -1845,6 +1843,9 @@ void __ref __remove_memory(int nid, u64 start, u64 size)memblock_free (start, size);memblock_remove (start, size); +/* remove memory block devices before removing memory */ +remove_memory_block_devices (start, size); +arch_remove_memory (nid, start, size,NULL );__release_memory_resource (start, size);Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 09/11] mm/memory_hotplug: Remove memory block devices before arch_remove_memory()
Quoted 9 lines · Michal Hocko
Quoted 7 lines · David Hildenbrand
Let's factor out removing of memory block devices, which is only necessary for memory added via
add_memory()and friends that created memory block devices. Remove the devices before callingarch_remove_memory().This finishes factoring out memory block device handling from
arch_add_memory()andarch_remove_memory().
Quoted 7 lines
OK, this makes sense again. Just a nit. Calling
find_memory_block_by_idfor each memory block looks a bit suboptimal, especially when we are removing consequent physical memblocks. I have to confess that I do not know how expensive is the search and I also expect that there won't be that many memblocks in the removed range anyway as large setups have large memblocks.
The devices are not allocated sequentially, so there is no easy way to look them up.
There is a comment for find_memory_block():
"For now, we have a linear search to go find the appropriate memory_block corresponding to a particular phys_index. If this gets to be a real problem, we can always use a radix tree or something here."
So if this becomes a problem, we need a separate data structure to speed up the lookup. (IOW, this was already the same in the old code)
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v3 10/11 mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
In reply to David HildenbrandWe really don't want anything during memory hotunplug to fail. We always pass a valid memory block device, that check can go. Avoid allocating memory and eventually failing. As we are always called under lock, we can use a static piece of memory. This avoids having to put the structure onto the stack, having to guess about the stack size of callers.
Patch inspired by a patch from Oscar Salvador.
In the future, there might be no need to iterate over nodes at all. mem->nid should tell us exactly what to remove. Memory block devices with mixed nodes (added during boot) should properly fenced off and never removed.
--- drivers/base/node.c | 18 +++++------------- include/linux/node.h | 5 ++--- 2 files changed, 7 insertions(+), 16 deletions(-)
drivers/base/node.c +5 -13
@@ -803,20 +803,14 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)/* * Unregister memory block device under all nodes that it spans. +* Has to be called with mem_sysfs_mutex held (due to unlinked_nodes). */ -int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn; +static nodemask_t unlinked_nodes; -if (!mem_blk) { -NODEMASK_FREE (unlinked_nodes); -return -EFAULT ; - } -if (!unlinked_nodes) -return -ENOMEM ; -nodes_clear (*unlinked_nodes); - +nodes_clear (unlinked_nodes); sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {
@@ -827,15 +821,13 @@ int unregister_memory_block_under_nodes(struct memory_block *mem_blk)continue ;if (!node_online (nid))continue ; -if (node_test_and_set (nid, *unlinked_nodes)) +if (node_test_and_set (nid, unlinked_nodes))continue ;sysfs_remove_link (&node_devices[nid]->dev.kobj,kobject_name (&mem_blk->dev.kobj));sysfs_remove_link (&mem_blk->dev.kobj,kobject_name (&node_devices[nid]->dev.kobj)); } -NODEMASK_FREE (unlinked_nodes); -return 0 ; }int link_mem_sections (int nid,unsigned long start_pfn,unsigned long end_pfn)
include/linux/node.h +2 -3
@@ -139,7 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); +extern void unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,
@@ -175,9 +175,8 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +static inline void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -return 0 ; } static inline void register_hugetlbfs_with_node(node_registration_func_t reg,
Signature · 1 line
2.20.1
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 14 lines · David Hildenbrand
We really don't want anything during memory hotunplug to fail. We always pass a valid memory block device, that check can go. Avoid allocating memory and eventually failing. As we are always called under lock, we can use a static piece of memory. This avoids having to put the structure onto the stack, having to guess about the stack size of callers.
Patch inspired by a patch from Oscar Salvador.
In the future, there might be no need to iterate over nodes at all.
mem->nidshould tell us exactly what to remove. Memory block devices with mixed nodes (added during boot) should properly fenced off and never removed.
Quoted 11 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc Andrew Morton <[email protected]>Cc Jonathan Cameron <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 77 lines
---
drivers/base/node.c| 18 +++++-------------include/linux/node.h| 5 ++---2 files changed, 7 insertions(+), 16 deletions(-)drivers/base/node.c +5
-13@@ -803,20 +803,14 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)/* * Unregister memory block device under all nodes that it spans. +* Has to be called with mem_sysfs_mutex held (due to unlinked_nodes). */ -int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn; +static nodemask_t unlinked_nodes; -if (!mem_blk) { -NODEMASK_FREE (unlinked_nodes); -return -EFAULT ; - } -if (!unlinked_nodes) -return -ENOMEM ; -nodes_clear (*unlinked_nodes); - +nodes_clear (unlinked_nodes); sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {@@ -827,15 +821,13 @@ int unregister_memory_block_under_nodes(struct memory_block *mem_blk)continue ;if (!node_online (nid))continue ; -if (node_test_and_set (nid, *unlinked_nodes)) +if (node_test_and_set (nid, unlinked_nodes))continue ;sysfs_remove_link (&node_devices[nid]->dev.kobj,kobject_name (&mem_blk->dev.kobj));sysfs_remove_link (&mem_blk->dev.kobj,kobject_name (&node_devices[nid]->dev.kobj)); } -NODEMASK_FREE (unlinked_nodes); -return 0 ; }int link_mem_sections (int nid,unsigned long start_pfn,unsigned long end_pfn)include/linux/node.h +2
-3@@ -139,7 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); +extern void unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,@@ -175,9 +175,8 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +static inline void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -return 0 ; } static inline void register_hugetlbfs_with_node(node_registration_func_t reg,Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
In reply to David HildenbrandQuoted 14 lines · David Hildenbrand
We really don't want anything during memory hotunplug to fail. We always pass a valid memory block device, that check can go. Avoid allocating memory and eventually failing. As we are always called under lock, we can use a static piece of memory. This avoids having to put the structure onto the stack, having to guess about the stack size of callers.
Patch inspired by a patch from Oscar Salvador.
In the future, there might be no need to iterate over nodes at all.
mem->nidshould tell us exactly what to remove. Memory block devices with mixed nodes (added during boot) should properly fenced off and never removed.
Quoted 11 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc Andrew Morton <[email protected]>Cc Jonathan Cameron <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
In reply to David HildenbrandQuoted 7 lines · David Hildenbrand
We really don't want anything during memory hotunplug to fail. We always pass a valid memory block device, that check can go. Avoid allocating memory and eventually failing. As we are always called under lock, we can use a static piece of memory. This avoids having to put the structure onto the stack, having to guess about the stack size of callers.
Quoted 6 lines
Patch inspired by a patch from Oscar Salvador.
In the future, there might be no need to iterate over nodes at all.
mem->nidshould tell us exactly what to remove. Memory block devices with mixed nodes (added during boot) should properly fenced off and never removed.
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
Quoted 11 lines
Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Alex Deucher <[email protected]>Cc "David S. Miller" <[email protected]>Cc Mark Brown <[email protected]>Cc Chris Wilson <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc Andrew Morton <[email protected]>Cc Jonathan Cameron <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Anyway
Quoted 77 lines
--- drivers/base/node.c | 18 +++++------------- include/linux/node.h | 5 ++--- 2 files changed, 7 insertions(+), 16 deletions(-)drivers/base/node.c +5
-13@@ -803,20 +803,14 @@ int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)/* * Unregister memory block device under all nodes that it spans. +* Has to be called with mem_sysfs_mutex held (due to unlinked_nodes). */ -int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -NODEMASK_ALLOC (nodemask_t, unlinked_nodes,GFP_KERNEL );unsigned long pfn, sect_start_pfn, sect_end_pfn; +static nodemask_t unlinked_nodes; -if (!mem_blk) { -NODEMASK_FREE (unlinked_nodes); -return -EFAULT ; - } -if (!unlinked_nodes) -return -ENOMEM ; -nodes_clear (*unlinked_nodes); - +nodes_clear (unlinked_nodes); sect_start_pfn =section_nr_to_pfn (mem_blk->start_section_nr); sect_end_pfn =section_nr_to_pfn (mem_blk->end_section_nr);for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {@@ -827,15 +821,13 @@ int unregister_memory_block_under_nodes(struct memory_block *mem_blk)continue ;if (!node_online (nid))continue ; -if (node_test_and_set (nid, *unlinked_nodes)) +if (node_test_and_set (nid, unlinked_nodes))continue ;sysfs_remove_link (&node_devices[nid]->dev.kobj,kobject_name (&mem_blk->dev.kobj));sysfs_remove_link (&mem_blk->dev.kobj,kobject_name (&node_devices[nid]->dev.kobj)); } -NODEMASK_FREE (unlinked_nodes); -return 0 ; }int link_mem_sections (int nid,unsigned long start_pfn,unsigned long end_pfn)include/linux/node.h +2
-3@@ -139,7 +139,7 @@ extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);extern int unregister_cpu_under_node (unsigned int cpu,unsigned int nid);extern int register_mem_sect_under_node (struct memory_block *mem_blk,void *arg); -extern int unregister_memory_block_under_nodes (struct memory_block *mem_blk); +extern void unregister_memory_block_under_nodes (struct memory_block *mem_blk); extern int register_memory_node_under_compute_node(unsigned int mem_nid, unsigned int cpu_nid,@@ -175,9 +175,8 @@ static inline int register_mem_sect_under_node(struct memory_block *mem_blk, {return 0 ; } -static inline int unregister_memory_block_under_nodes (struct memory_block *mem_blk) +static inline void unregister_memory_block_under_nodes (struct memory_block *mem_blk) { -return 0 ; } static inline void register_hugetlbfs_with_node(node_registration_func_t reg,Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
Oscar Salvador
maintainerRe: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 8 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
Quoted 6 lines
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
Quoted 7 lines
Quoted 3 lines
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
So what's the suggestion? Dropping the nodemask_t completely and calling sysfs_remove_link() on already potentially removed links?
Of course, we can also just use mem_blk->nid and rest assured that it will never be called for memory blocks belonging to multiple nodes.
Signature · 3 lines
Thanks, David / dhildenb
Oscar Salvador
maintainerRe: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 21 lines · David Hildenbrand
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
Quoted 7 lines
Quoted 3 lines
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
Quoted 5 lines
So what's the suggestion? Dropping the
nodemask_tcompletely and callingsysfs_remove_link()on already potentially removed links?Of course, we can also just use
mem_blk->nidand rest assured that it will never be called for memory blocks belonging to multiple nodes.
Hi David,
While it is easy to construct a scenario where a memblock belongs to multiple nodes, I have to confess that I yet have not seen that in a real-world scenario.
Given said that, I think that the less risky way is to just drop the nodemask_t and do not care about calling sysfs_remove_link() for already removed links. As I said, sysfs_remove_link() will silently fail when it fails to find the symlink, so I do not think it is a big deal.
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
Quoted 30 lines · Oscar Salvador
Quoted 21 lines · David Hildenbrand
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
Quoted 7 lines
Quoted 3 lines
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
Quoted 5 lines
So what's the suggestion? Dropping the
nodemask_tcompletely and callingsysfs_remove_link()on already potentially removed links?Of course, we can also just use
mem_blk->nidand rest assured that it will never be called for memory blocks belonging to multiple nodes.Hi David,
Quoted 9 lines
While it is easy to construct a scenario where a memblock belongs to multiple nodes, I have to confess that I yet have not seen that in a real-world scenario.
Given said that, I think that the less risky way is to just drop the
nodemask_tand do not care about callingsysfs_remove_link()for already removed links. As I said,sysfs_remove_link()will silently fail when it fails to find the symlink, so I do not think it is a big deal.
As far as I can tell we
a) don't allow offlining of memory that belongs to multiple nodes already (as pointed out by Michael recently)
b) users cannot add memory blocks that belong to multiple nodes via add_memory()
So I don't see a way how remove_memory() (and even offline_pages()) could ever succeed on such memory blocks.
I think it should be fine to limit it to one node here. (if not, I guess we would have a different BUG that would actually allow to remove such memory blocks)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
In reply to Oscar SalvadorQuoted 30 lines · Oscar Salvador
Quoted 21 lines · David Hildenbrand
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
Quoted 7 lines
Quoted 3 lines
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
Quoted 5 lines
So what's the suggestion? Dropping the
nodemask_tcompletely and callingsysfs_remove_link()on already potentially removed links?Of course, we can also just use
mem_blk->nidand rest assured that it will never be called for memory blocks belonging to multiple nodes.Hi David,
Quoted 9 lines
While it is easy to construct a scenario where a memblock belongs to multiple nodes, I have to confess that I yet have not seen that in a real-world scenario.
Given said that, I think that the less risky way is to just drop the
nodemask_tand do not care about callingsysfs_remove_link()for already removed links. As I said,sysfs_remove_link()will silently fail when it fails to find the symlink, so I do not think it is a big deal.
As far as I can tell we
a) don't allow offlining of memory that belongs to multiple nodes already (as pointed out by Michal recently)
b) users cannot add memory blocks that belong to multiple nodes via add_memory()
So I don't see a way how remove_memory() (and even offline_pages()) could ever succeed on such memory blocks.
I think it should be fine to limit it to one node here. (if not, I guess we would have a different BUG that would actually allow to remove such memory blocks)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v3 10/11] mm/memory_hotplug: Make unregister_memory_block_under_nodes() never fail
In reply to David HildenbrandQuoted 17 lines · David Hildenbrand
Quoted 16 lines · Michal Hocko
Quoted 14 lines · Oscar Salvador
Quoted 6 lines · Michal Hocko
Yeah, we do not allow to offline multi zone (node) ranges so the current code seems to be over engineered.
Anyway, I am wondering why do we have to strictly check for already removed nodes links. Is the sysfs code going to complain we we try to remove again?
No, sysfs will silently "fail" if the symlink has already been removed. At least that is what I saw last time I played with it.
I guess the question is what if sysfs handling changes in the future and starts dropping warnings when trying to remove a symlink is not there. Maybe that is unlikely to happen?
Quoted 6 lines
Quoted 3 lines
And maybe we handle it then rather than have a static allocation that everybody with hotremove configured has to pay for.
So what's the suggestion? Dropping the
nodemask_tcompletely and callingsysfs_remove_link()on already potentially removed links?
Yes. In a follow up patch.
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v3 11/11 mm/memory_hotplug: Remove "zone" parameter from sparse_remove_one_section
In reply to David HildenbrandThe parameter is unused, so let's drop it. Memory removal paths should never care about zones. This is the job of memory offlining and will require more refactorings.
--- include/linux/memory_hotplug.h | 2 +- mm/memory_hotplug.c | 2 +- mm/sparse.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-)
include/linux/memory_hotplug.h +1 -1
@@ -346,7 +346,7 @@ extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_one_section (int nid,unsigned long start_pfn,struct vmem_altmap *altmap); -extern void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, +extern void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset,struct vmem_altmap *altmap);extern struct page *sparse_decode_mem_map (unsigned long coded_mem_map,unsigned long pnum);
mm/memory_hotplug.c +1 -1
@@ -524,7 +524,7 @@ static void __remove_section(struct zone *zone, struct mem_section *ms, start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn); -sparse_remove_one_section (zone, ms, map_offset, altmap); +sparse_remove_one_section (ms, map_offset, altmap); }/**
mm/sparse.c +2 -2
@@ -800,8 +800,8 @@ static void free_section_usemap(struct page *memmap, unsigned long *usemap,free_map_bootmem (memmap); } -void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, -unsigned long map_offset,struct vmem_altmap *altmap) +void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset, +struct vmem_altmap *altmap) {struct page *memmap =NULL ;unsigned long *usemap =NULL ;
Signature · 1 line
2.20.1
Re: [PATCH v3 11/11] mm/memory_hotplug: Remove "zone" parameter from sparse_remove_one_section
Quoted 6 lines · David Hildenbrand
The parameter is unused, so let's drop it. Memory removal paths should never care about zones. This is the job of memory offlining and will require more refactorings.
Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 49 lines
---
include/linux/memory_hotplug.h | 2 +- mm/memory_hotplug.c | 2 +- mm/sparse.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-)include/linux/memory_hotplug.h +1
-1@@ -346,7 +346,7 @@ extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_one_section (int nid,unsigned long start_pfn,struct vmem_altmap *altmap); -extern void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, +extern void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset,struct vmem_altmap *altmap);extern struct page *sparse_decode_mem_map (unsigned long coded_mem_map,unsigned long pnum);mm/memory_hotplug.c +1
-1@@ -524,7 +524,7 @@ static void __remove_section(struct zone *zone, struct mem_section *ms, start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn); -sparse_remove_one_section (zone, ms, map_offset, altmap); +sparse_remove_one_section (ms, map_offset, altmap); }/** mm/sparse.c +2
-2@@ -800,8 +800,8 @@ static void free_section_usemap(struct page *memmap, unsigned long *usemap,free_map_bootmem (memmap); } -void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, -unsigned long map_offset,struct vmem_altmap *altmap) +void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset, +struct vmem_altmap *altmap) {struct page *memmap =NULL ;unsigned long *usemap =NULL ;Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v3 11/11] mm/memory_hotplug: Remove "zone" parameter from sparse_remove_one_section
In reply to David HildenbrandQuoted 6 lines · David Hildenbrand
The parameter is unused, so let's drop it. Memory removal paths should never care about zones. This is the job of memory offlining and will require more refactorings.
Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v3 11/11] mm/memory_hotplug: Remove "zone" parameter from sparse_remove_one_section
In reply to David HildenbrandQuoted 6 lines · David Hildenbrand
The parameter is unused, so let's drop it. Memory removal paths should never care about zones. This is the job of memory offlining and will require more refactorings.
Reviewed-by Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 49 lines
--- include/linux/memory_hotplug.h | 2 +- mm/memory_hotplug.c | 2 +- mm/sparse.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-)include/linux/memory_hotplug.h +1
-1@@ -346,7 +346,7 @@ extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_one_section (int nid,unsigned long start_pfn,struct vmem_altmap *altmap); -extern void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, +extern void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset,struct vmem_altmap *altmap);extern struct page *sparse_decode_mem_map (unsigned long coded_mem_map,unsigned long pnum);mm/memory_hotplug.c +1
-1@@ -524,7 +524,7 @@ static void __remove_section(struct zone *zone, struct mem_section *ms, start_pfn =section_nr_to_pfn ((unsigned long )scn_nr);__remove_zone (zone, start_pfn); -sparse_remove_one_section (zone, ms, map_offset, altmap); +sparse_remove_one_section (ms, map_offset, altmap); }/** mm/sparse.c +2
-2@@ -800,8 +800,8 @@ static void free_section_usemap(struct page *memmap, unsigned long *usemap,free_map_bootmem (memmap); } -void sparse_remove_one_section (struct zone *zone,struct mem_section *ms, -unsigned long map_offset,struct vmem_altmap *altmap) +void sparse_remove_one_section (struct mem_section *ms,unsigned long map_offset, +struct vmem_altmap *altmap) {struct page *memmap =NULL ;unsigned long *usemap =NULL ;Signature · 1 line
2.20.1
Signature · 2 lines
Michal Hocko SUSE Labs
IMHO, there is some typo.
s/devicehandling/device handling/
Quoted 4 lines · David Hildenbrand
We only want memory block devices for memory to be onlined/offlined (add/remove from the buddy). This is required so user space can online/offline memory and kdump gets notified about newly onlined memory.
Quoted 8 lines
Let's factor out creation/removal of memory block devices. This helps to further cleanup arch_add_memory/arch_remove_memory() and to make implementation of new features easier - especially sub-section memory hot add from Dan.
Anshuman Khandual is currently working on
arch_remove_memory(). I added a temporary solution via "arm64/mm: Add temporaryarch_remove_memory()implementation", that is sufficient as a firsts tep in the context of
s/firsts tep/first step/
Quoted 7 lines
this series. (we don't cleanup page tables in case anything goes wrong already)
Did a quick sanity test with DIMM plug/unplug, making sure all devices and sysfs links properly get added/removed. Compile tested on s390x and x86-64.
Quoted 7 lines
Based on next/master.
Next refactoring on my list will be making sure that
remove_memory()will never deal with zones / access "struct pages". Any kind of zone handling will have to be done when offlining system memory / before removing device memory. I am thinking aboutremove_pfn_range_from_zone()", du undo everything "move_pfn_range_to_zone()" did.
what is "du undo"? I may not get it.
Quoted 49 lines
v2 -> v3:
- Add "s390x/mm: Fail when an altmap is used forarch_add_memory()"
- Add "arm64/mm: Add temporaryarch_remove_memory()implementation"
- Add "drivers/base/memory: Pass ablock_idtoinit_memory_block()"
- Various changes to "mm/memory_hotplug: Create memory block devices afterarch_add_memory()" and "mm/memory_hotplug: Create memory block devices afterarch_add_memory()" due to switching from sections to block_id's.v1 -> v2:
- s390x/mm: Implementarch_remove_memory()
-- remove mapping after "__remove_pages"David Hildenbrand (11):
mm/memory_hotplug: Simplify and fixcheck_hotplug_memory_range()s390x/mm: Fail when an altmap is used forarch_add_memory()s390x/mm: Implementarch_remove_memory()
arm64/mm: Add temporaryarch_remove_memory()implementationdrivers/base/memory: Pass ablock_idtoinit_memory_block()mm/memory_hotplug: Allowarch_remove_pages()without
CONFIG_MEMORY_HOTREMOVE
mm/memory_hotplug: Create memory block devices afterarch_add_memory()mm/memory_hotplug: DropMHP_MEMBLOCK_API
mm/memory_hotplug: Remove memory block devices before
arch_remove_memory()
mm/memory_hotplug: Makeunregister_memory_block_under_nodes()never
fail
mm/memory_hotplug: Remove "zone" parameter from
sparse_remove_one_sectionarch/arm64/mm/mmu.c | 17 +++++ arch/ia64/mm/init.c | 2 - arch/powerpc/mm/mem.c | 2 - arch/s390/mm/init.c | 18 +++-- arch/sh/mm/init.c | 2 - arch/x86/mm/init_32.c | 2 - arch/x86/mm/init_64.c | 2 - drivers/base/memory.c | 134 +++++++++++++++++++-------------- drivers/base/node.c | 27 +++---- include/linux/memory.h | 6 +- include/linux/memory_hotplug.h | 12 +-- include/linux/node.h | 7 +- mm/memory_hotplug.c | 44 +++++------ mm/sparse.c | 10 +-- 14 files changed, 140 insertions(+), 145 deletions(-)Signature · 1 line
2.20.1
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 1 line · Wei Yang
IMHO, there is some typo.
Yes, thanks.
Quoted 28 lines
s/devicehandling/device handling/
Quoted 4 lines · David Hildenbrand
We only want memory block devices for memory to be onlined/offlined (add/remove from the buddy). This is required so user space can online/offline memory and kdump gets notified about newly onlined memory.
Quoted 8 lines
Let's factor out creation/removal of memory block devices. This helps to further cleanup arch_add_memory/arch_remove_memory() and to make implementation of new features easier - especially sub-section memory hot add from Dan.
Anshuman Khandual is currently working on
arch_remove_memory(). I added a temporary solution via "arm64/mm: Add temporaryarch_remove_memory()implementation", that is sufficient as a firsts tep in the context ofs/firsts tep/first step/
Quoted 9 lines
this series. (we don't cleanup page tables in case anything goes wrong already)
Did a quick sanity test with DIMM plug/unplug, making sure all devices and sysfs links properly get added/removed. Compile tested on s390x and x86-64.
Based on next/master.
Quoted 7 lines
Quoted 5 lines
Next refactoring on my list will be making sure that
remove_memory()will never deal with zones / access "struct pages". Any kind of zone handling will have to be done when offlining system memory / before removing device memory. I am thinking aboutremove_pfn_range_from_zone()", du undo everything "move_pfn_range_to_zone()" did.what is "du undo"? I may not get it.
to undo ;)
Signature · 3 lines
Thanks, David / dhildenb