mm/memory_hotplug: Shrink zones before removing memory
Subsystems (10)
v6 cover mm/memory_hotplug: Shrink zones before removing memory
This series fixes the access of uninitialized memmaps when shrinking zones/nodes and when removing memory. Also, it contains all fixes for crashes that can be triggered when removing certain namespace using memunmap_pages() - ZONE_DEVICE, reported by Aneesh.
We stop trying to shrink ZONE_DEVICE, as it's buggy, fixing it would be more involved (we don't have SECTION_IS_ONLINE as an indicator), and shrinking is only of limited use (set_zone_contiguous() cannot detect the ZONE_DEVICE as contiguous).
We continue shrinking !ZONE_DEVICE zones, however, I reduced the amount of code to a minimum. Shrinking is especially necessary to keep zone->contiguous set where possible, especially, on memory unplug of DIMMs at zone boundaries.
--------------------------------------------------------------------------
Zones are now properly shrunk when offlining memory blocks or when onlining failed. This allows to properly shrink zones on memory unplug even if the separate memory blocks of a DIMM were onlined to different zones or re-onlined to a different zone after offlining.
Example:
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 0
present 0
managed 0
:/# echo "online_movable" > /sys/devices/system/memory/memory41/state
:/# echo "online_movable" > /sys/devices/system/memory/memory43/state
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 98304
present 65536
managed 65536
:/# echo 0 > /sys/devices/system/memory/memory43/online
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 32768
present 32768
managed 32768
:/# echo 0 > /sys/devices/system/memory/memory41/online
:/# cat /proc/zoneinfo
Node 1, zone Movable
spanned 0
present 0
managed 0--------------------------------------------------------------------------
I tested this with DIMMs on x86. I didn't test the ZONE_DEVICE part.
v4 -> v6:
- "mm/memunmap: Don't access uninitialized memmap in memunmap_pages()" -- Minimize code changes, rephrase subject and description
- "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()" -- Add ifdef to make it compile without ZONE_DEVICE
v4 -> v5:
- "mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()" -- Add more details why ZONE_DEVICE is special
- Include two patches from Aneesh
-- "mm/memunmap: Use the correct start and end pfn when removing pages
from zone"
-- "mm/memmap_init: Update variable name in memmap_init_zone"
v3 -> v4:
- Drop "mm/memremap: Get rid of memmap_init_zone_device()"
-- As Alexander noticed, it was messy either way
- Drop "mm/memory_hotplug: Exit early in __remove_pages() on BUGs"
- Drop "mm: Exit early in set_zone_contiguous() if already contiguous"
- Drop "mm/memory_hotplug: Optimize zone shrinking code when checking for holes"
- Merged "mm/memory_hotplug: Remove pages from a zone before removing memory" and "mm/memory_hotplug: Remove zone parameter from __remove_pages()" into "mm/memory_hotplug: Shrink zones when offlining memory"
- Added "mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()"
- Stop shrinking ZONE_DEVICE
- Reshuffle patches, moving all fixes to the front. Add Fixes: tags.
- Change subject/description of various patches
- Minor changes (too many to mention)
Aneesh Kumar K.V (2):
mm/memunmap: Don't access uninitialized memmap in memunmap_pages() mm/memmap_init: Update variable name in memmap_init_zone
David Hildenbrand (8):
mm/memory_hotplug: Don't access uninitialized memmaps in
shrink_pgdat_span()
mm/memory_hotplug: Don't access uninitialized memmaps in
shrink_zone_span()
mm/memory_hotplug: Shrink zones when offlining memory mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone() mm/memory_hotplug: We always have a zone in
find_(smallest|biggest)_section_pfn
mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span() mm/memory_hotplug: Drop local variables in shrink_zone_span() mm/memory_hotplug: Cleanup __remove_pages()
arch/arm64/mm/mmu.c | 4 +- arch/ia64/mm/init.c | 4 +- arch/powerpc/mm/mem.c | 3 +- arch/s390/mm/init.c | 4 +- arch/sh/mm/init.c | 4 +- arch/x86/mm/init_32.c | 4 +- arch/x86/mm/init_64.c | 4 +- include/linux/memory_hotplug.h | 7 +- mm/memory_hotplug.c | 186 ++++++++++++--------------------- mm/memremap.c | 13 ++- mm/page_alloc.c | 8 +- 11 files changed, 90 insertions(+), 151 deletions(-)
Signature · 1 line
2.21.0
PATCH v6 1/10 mm/memunmap: Don't access uninitialized memmap in memunmap_pages()
From: "Aneesh Kumar K.V" <[email protected]>
With an altmap, the memmap falling into the reserved altmap space are not initialized and, therefore, contain a garbage NID and a garbage zone. Make sure to read the NID/zone from a memmap that was initialzed.
This fixes a kernel crash that is observed when destroying a namespace:
[ 81.356173] kernel BUG at include/linux/mm.h:1107!
cpu 0x1: Vector: 700 (Program Check) at [c000000274087890]
pc: c0000000004b9728: memunmap_pages+0x238/0x340
lr: c0000000004b9724: memunmap_pages+0x234/0x340
...
pid = 3669, comm = ndctlkernel BUG at include/linux/mm.h:1107!
[c000000274087ba0] c0000000009e3500 devm_action_release+0x30/0x50 [c000000274087bc0] c0000000009e4758 release_nodes+0x268/0x2d0 [c000000274087c30] c0000000009dd144 device_release_driver_internal+0x174/0x240 [c000000274087c70] c0000000009d9dfc unbind_store+0x13c/0x190 [c000000274087cb0] c0000000009d8a24 drv_attr_store+0x44/0x60 [c000000274087cd0] c0000000005a7470 sysfs_kf_write+0x70/0xa0 [c000000274087d10] c0000000005a5cac kernfs_fop_write+0x1ac/0x290 [c000000274087d60] c0000000004be45c __vfs_write+0x3c/0x70 [c000000274087d80] c0000000004c26e4 vfs_write+0xe4/0x200 [c000000274087dd0] c0000000004c2a6c ksys_write+0x7c/0x140 [c000000274087e20] c00000000000bbd0 system_call+0x5c/0x68
--- mm/memremap.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
mm/memremap.c +7 -4
@@ -123,6 +123,7 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)void memunmap_pages (struct dev_pagemap *pgmap) {struct resource *res = &pgmap->res; +struct page *first_page;unsigned long pfn;int nid;
@@ -131,14 +132,16 @@ void memunmap_pages(struct dev_pagemap *pgmap)put_page (pfn_to_page (pfn));dev_pagemap_cleanup (pgmap); +/* make sure to access a memmap that was actually initialized */ + first_page =pfn_to_page (pfn_first (pgmap)); +/* pages are dead and unused, undo the arch mapping */ - nid =page_to_nid (pfn_to_page (PHYS_PFN (res->start))); + nid =page_to_nid (first_page);mem_hotplug_begin ();if (pgmap->type =MEMORY_DEVICE_PRIVATE ) { - pfn =PHYS_PFN (res->start); -__remove_pages (page_zone (pfn_to_page (pfn)), pfn, -PHYS_PFN (resource_size (res)),NULL ); +__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +PHYS_PFN (resource_size (res)),NULL ); }else {arch_remove_memory (nid, res->start,resource_size (res),pgmap_altmap (pgmap));
Signature · 1 line
2.21.0
Re: [PATCH v6 01/10] mm/memunmap: Don't access uninitialized memmap in memunmap_pages()
Hello David,
patch 05/10 is missing in the patch series.
Quoted 52 lines · David Hildenbrand
From: "Aneesh Kumar K.V" <[email protected]>
With an altmap, the memmap falling into the reserved altmap space are not initialized and, therefore, contain a garbage NID and a garbage zone. Make sure to read the NID/zone from a memmap that was initialzed.
This fixes a kernel crash that is observed when destroying a namespace:
[ 81.356173] kernel BUG at include/linux/mm.h:1107! cpu 0x1: Vector: 700 (Program Check) at [c000000274087890] pc: c0000000004b9728: memunmap_pages+0x238/0x340 lr: c0000000004b9724: memunmap_pages+0x234/0x340 ... pid = 3669, comm = ndctlkernel BUG at
include/linux/mm.h:1107!
[c000000274087ba0]c0000000009e3500devm_action_release+0x30/0x50[c000000274087bc0]c0000000009e4758release_nodes+0x268/0x2d0[c000000274087c30]c0000000009dd144device_release_driver_internal+0x174/0x240[c000000274087c70]c0000000009d9dfcunbind_store+0x13c/0x190[c000000274087cb0]c0000000009d8a24drv_attr_store+0x44/0x60[c000000274087cd0]c0000000005a7470sysfs_kf_write+0x70/0xa0[c000000274087d10]c0000000005a5cackernfs_fop_write+0x1ac/0x290[c000000274087d60]c0000000004be45c__vfs_write+0x3c/0x70[c000000274087d80]c0000000004c26e4vfs_write+0xe4/0x200[c000000274087dd0]c0000000004c2a6cksys_write+0x7c/0x140[c000000274087e20]c00000000000bbd0system_call+0x5c/0x68Cc Dan Williams <[email protected]>Cc Andrew Morton <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Ira Weiny <[email protected]>Signed-off-by Aneesh Kumar K.V <[email protected]>[ minimze code changes, rephrase description ]Signed-off-by David Hildenbrand <[email protected]>--- mm/memremap.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)mm/memremap.c +1
-0@@ -123,6 +123,7 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)void memunmap_pages (struct dev_pagemap *pgmap) {struct resource *res = &pgmap->res; +struct page *first_page;unsigned long pfn;int nid;
mm/memremap.c void memunmap_pages(struct dev_pagemap *pgmap) · line 139
Quoted 9 lines
mm/memremap.c +4
-1@@ -131,14 +132,16 @@ void memunmap_pages(struct dev_pagemap *pgmap)put_page (pfn_to_page (pfn));dev_pagemap_cleanup (pgmap); +/* make sure to access a memmap that was actually initialized */ + first_page =pfn_to_page (pfn_first (pgmap)); +/* pages are dead and unused, undo the arch mapping */ - nid =page_to_nid (pfn_to_page (PHYS_PFN (res->start))); + nid =page_to_nid (first_page);
Why we need 'nid = page_to_nid(first_page)' we didnt use it anymore in this function ?
Quoted 14 lines
mm/memremap.c +2
-3mem_hotplug_begin ();if (pgmap->type =MEMORY_DEVICE_PRIVATE ) { - pfn =PHYS_PFN (res->start); -__remove_pages (page_zone (pfn_to_page (pfn)), pfn, -PHYS_PFN (resource_size (res)),NULL ); +__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +PHYS_PFN (resource_size (res)),NULL ); }else {arch_remove_memory (nid, res->start,resource_size (res),pgmap_altmap (pgmap));Signature · 1 line
2.21.0
Best regards
Damian
Re: [PATCH v6 01/10] mm/memunmap: Don't access uninitialized memmap in memunmap_pages()
Quoted 4 lines · Damian Tometzki
Hello David,
patch 05/10 is missing in the patch series.
Hi Damian,
not really. Could be that lkml is slow today. E.g., check
https://marc.info/?l=linux-mm&m7035222620403&w=2
and especially
https://marc.info/?l=linux-mm&m7035225120440&w=2
All mails popped up on the mm list.
Quoted 57 lines · David Hildenbrand
Quoted 55 lines
From: "Aneesh Kumar K.V" <[email protected]>
With an altmap, the memmap falling into the reserved altmap space are not initialized and, therefore, contain a garbage NID and a garbage zone. Make sure to read the NID/zone from a memmap that was initialzed.
This fixes a kernel crash that is observed when destroying a namespace:
[ 81.356173] kernel BUG at include/linux/mm.h:1107! cpu 0x1: Vector: 700 (Program Check) at [c000000274087890] pc: c0000000004b9728: memunmap_pages+0x238/0x340 lr: c0000000004b9724: memunmap_pages+0x234/0x340 ... pid = 3669, comm = ndctlkernel BUG at
include/linux/mm.h:1107!
[c000000274087ba0]c0000000009e3500devm_action_release+0x30/0x50[c000000274087bc0]c0000000009e4758release_nodes+0x268/0x2d0[c000000274087c30]c0000000009dd144device_release_driver_internal+0x174/0x240[c000000274087c70]c0000000009d9dfcunbind_store+0x13c/0x190[c000000274087cb0]c0000000009d8a24drv_attr_store+0x44/0x60[c000000274087cd0]c0000000005a7470sysfs_kf_write+0x70/0xa0[c000000274087d10]c0000000005a5cackernfs_fop_write+0x1ac/0x290[c000000274087d60]c0000000004be45c__vfs_write+0x3c/0x70[c000000274087d80]c0000000004c26e4vfs_write+0xe4/0x200[c000000274087dd0]c0000000004c2a6cksys_write+0x7c/0x140[c000000274087e20]c00000000000bbd0system_call+0x5c/0x68Cc Dan Williams <[email protected]>Cc Andrew Morton <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Ira Weiny <[email protected]>Signed-off-by Aneesh Kumar K.V <[email protected]>[ minimze code changes, rephrase description ]Signed-off-by David Hildenbrand <[email protected]>--- mm/memremap.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)mm/memremap.c +1
-0@@ -123,6 +123,7 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)void memunmap_pages (struct dev_pagemap *pgmap) {struct resource *res = &pgmap->res; +struct page *first_page;unsigned long pfn;int nid;@@ -131,14 +132,16 @@ void memunmap_pages(struct dev_pagemap *pgmap)put_page (pfn_to_page (pfn));dev_pagemap_cleanup (pgmap);
Quoted 8 lines
mm/memremap.c void memunmap_pages(struct dev_pagemap *pgmap) · line 139
Quoted 6 lines
mm/memremap.c +4
-1+/* make sure to access a memmap that was actually initialized */ + first_page =pfn_to_page (pfn_first (pgmap)); +/* pages are dead and unused, undo the arch mapping */ - nid =page_to_nid (pfn_to_page (PHYS_PFN (res->start))); + nid =page_to_nid (first_page);Why we need 'nid =
page_to_nid(first_page)' we didnt use it anymore in this function ?
Please see ...
Quoted 11 lines
mm/memremap.c void memunmap_pages(struct dev_pagemap *pgmap) · line 146
Quoted 11 lines
mm/memremap.c +2
-3mem_hotplug_begin ();if (pgmap->type =MEMORY_DEVICE_PRIVATE ) { - pfn =PHYS_PFN (res->start); -__remove_pages (page_zone (pfn_to_page (pfn)), pfn, -PHYS_PFN (resource_size (res)),NULL ); +__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +PHYS_PFN (resource_size (res)),NULL ); }else { arch_remove_memory(nid, res->start, resource_size(res),
^ here
:)
Quoted 1 line
Quoted 1 line
mm/memremap.c +0
-0pgmap_altmap(pgmap));
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 01/10] mm/memunmap: Don't access uninitialized memmap in memunmap_pages()
In reply to David HildenbrandQuoted 63 lines · David Hildenbrand
From: "Aneesh Kumar K.V" <[email protected]>
With an altmap, the memmap falling into the reserved altmap space are not initialized and, therefore, contain a garbage NID and a garbage zone. Make sure to read the NID/zone from a memmap that was initialzed.
This fixes a kernel crash that is observed when destroying a namespace:
[ 81.356173] kernel BUG at include/linux/mm.h:1107! cpu 0x1: Vector: 700 (Program Check) at [c000000274087890] pc: c0000000004b9728: memunmap_pages+0x238/0x340 lr: c0000000004b9724: memunmap_pages+0x234/0x340 ... pid = 3669, comm = ndctlkernel BUG at
include/linux/mm.h:1107!
[c000000274087ba0]c0000000009e3500devm_action_release+0x30/0x50[c000000274087bc0]c0000000009e4758release_nodes+0x268/0x2d0[c000000274087c30]c0000000009dd144device_release_driver_internal+0x174/0x240[c000000274087c70]c0000000009d9dfcunbind_store+0x13c/0x190[c000000274087cb0]c0000000009d8a24drv_attr_store+0x44/0x60[c000000274087cd0]c0000000005a7470sysfs_kf_write+0x70/0xa0[c000000274087d10]c0000000005a5cackernfs_fop_write+0x1ac/0x290[c000000274087d60]c0000000004be45c__vfs_write+0x3c/0x70[c000000274087d80]c0000000004c26e4vfs_write+0xe4/0x200[c000000274087dd0]c0000000004c2a6cksys_write+0x7c/0x140[c000000274087e20]c00000000000bbd0system_call+0x5c/0x68Cc Dan Williams <[email protected]>Cc Andrew Morton <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Ira Weiny <[email protected]>Signed-off-by Aneesh Kumar K.V <[email protected]>[ minimze code changes, rephrase description ]Signed-off-by David Hildenbrand <[email protected]>--- mm/memremap.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)mm/memremap.c +5
-1@@ -123,6 +123,7 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)void memunmap_pages (struct dev_pagemap *pgmap) {struct resource *res = &pgmap->res; +struct page *first_page;unsigned long pfn;int nid;@@ -131,14 +132,16 @@ void memunmap_pages(struct dev_pagemap *pgmap)put_page (pfn_to_page (pfn));dev_pagemap_cleanup (pgmap); +/* make sure to access a memmap that was actually initialized */ + first_page =pfn_to_page (pfn_first (pgmap)); +/* pages are dead and unused, undo the arch mapping */ - nid =page_to_nid (pfn_to_page (PHYS_PFN (res->start))); + nid =page_to_nid (first_page);
mm/memremap.c void memunmap_pages(struct dev_pagemap *pgmap) · line 147
Quoted 10 lines
mm/memremap.c +2
-3mem_hotplug_begin ();if (pgmap->type =MEMORY_DEVICE_PRIVATE ) { - pfn =PHYS_PFN (res->start); -__remove_pages (page_zone (pfn_to_page (pfn)), pfn, -PHYS_PFN (resource_size (res)),NULL ); +__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +PHYS_PFN (resource_size (res)),NULL ); }else {arch_remove_memory (nid, res->start,resource_size (res),pgmap_altmap (pgmap));
@Andrew, can you add
arch_remove_memory")
(which basically introduced the nid = page_to_nid(first_page))
The "page_zone(pfn_to_page(pfn)" was introduced by 69324b8f4833 ("mm,devm_memremap_pages: add MEMORY_DEVICE_PRIVATE support"), however, I think we will never have driver reserved memory with MEMORY_DEVICE_PRIVATE (no altmap AFAIKS).
Also, I think
makes sense.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v6 2/10 mm/memmap_init: Update variable name in memmap_init_zone
In reply to David HildenbrandFrom: "Aneesh Kumar K.V" <[email protected]>
The third argument is actually number of pages. Changes the variable name from size to nr_pages to indicate this better.
No functional change in this patch.
--- mm/page_alloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_alloc.c +4 -4
@@ -5936,10 +5936,10 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,#ifdef CONFIG_ZONE_DEVICE void __ref memmap_init_zone_device (struct zone *zone,unsigned long start_pfn, -unsigned long size, +unsigned long nr_pages,struct dev_pagemap *pgmap) { -unsigned long pfn, end_pfn = start_pfn + size; +unsigned long pfn, end_pfn = start_pfn + nr_pages;struct pglist_data *pgdat = zone->zone_pgdat;struct vmem_altmap *altmap =pgmap_altmap (pgmap);unsigned long zone_idx =zone_idx (zone);
@@ -5956,7 +5956,7 @@ void __ref memmap_init_zone_device(struct zone *zone,*/ if (altmap) { start_pfn = altmap->base_pfn +vmem_altmap_offset (altmap); - size = end_pfn - start_pfn; + nr_pages = end_pfn - start_pfn; }for (pfn = start_pfn; pfn < end_pfn; pfn++) {
@@ -6003,7 +6003,7 @@ void __ref memmap_init_zone_device(struct zone *zone, }pr_info ("%s initialised %lu pages in %ums\n" , __func__, - size,jiffies_to_msecs (jiffies - start)); + nr_pages,jiffies_to_msecs (jiffies - start)); }#endif
Signature · 1 line
2.21.0
PATCH v6 3/10 mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span()
In reply to David HildenbrandWe might use the nid of memmaps that were never initialized. For example, if the memmap was poisoned, we will crash the kernel in pfn_to_nid() right now. Let's use the calculated boundaries of the separate zones instead. This now also avoids having to iterate over a whole bunch of subsections again, after shrinking one zone.
Before commit d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug"), the memmap was initialized to 0 and the node was set to the right value. After that commit, the node might be garbage.
We'll have to fix shrink_zone_span() next.
--- mm/memory_hotplug.c | 72 ++++++++++----------------------------------- 1 file changed, 15 insertions(+), 57 deletions(-)
mm/memory_hotplug.c +15 -57
@@ -436,67 +436,25 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,zone_span_writeunlock (zone); } -static void shrink_pgdat_span (struct pglist_data *pgdat, -unsigned long start_pfn,unsigned long end_pfn) +static void update_pgdat_span (struct pglist_data *pgdat) { -unsigned long pgdat_start_pfn = pgdat->node_start_pfn; -unsigned long p =pgdat_end_pfn (pgdat);/* pgdat_end_pfn namespace clash */ -unsigned long pgdat_end_pfn = p; -unsigned long pfn; -int nid = pgdat->node_id; - -if (pgdat_start_pfn = start_pfn) { -/* -* If the section is smallest section in the pgdat, it need -* shrink pgdat->node_start_pfn and pgdat->node_spanned_pages. -* In this case, we find second smallest valid mem_section -* for shrinking zone. -*/ - pfn =find_smallest_section_pfn (nid,NULL , end_pfn, - pgdat_end_pfn); -if (pfn) { - pgdat->node_start_pfn = pfn; - pgdat->node_spanned_pages = pgdat_end_pfn - pfn; - } - }else if (pgdat_end_pfn = end_pfn) { -/* -* If the section is biggest section in the pgdat, it need -* shrink pgdat->node_spanned_pages. -* In this case, we find second biggest valid mem_section for -* shrinking zone. -*/ - pfn =find_biggest_section_pfn (nid,NULL , pgdat_start_pfn, - start_pfn); -if (pfn) - pgdat->node_spanned_pages = pfn - pgdat_start_pfn +1 ; - } - -/* -* If the section is not biggest or smallest mem_section in the pgdat, -* it only creates a hole in the pgdat. So in this case, we need not -* change the pgdat. -* But perhaps, the pgdat has only hole data. Thus it check the pgdat -* has only hole or not. -*/ - pfn = pgdat_start_pfn; -for (; pfn < pgdat_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_valid (pfn))) -continue ; - -if (pfn_to_nid (pfn) != nid) -continue ; +unsigned long node_start_pfn =0 , node_end_pfn =0 ; +struct zone *zone; -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; +for (zone = pgdat->node_zones; + zone < pgdat->node_zones +MAX_NR_ZONES ; zone++) { +unsigned long zone_end_pfn = zone->zone_start_pfn + + zone->spanned_pages; -/* If we find valid section, we have nothing to do */ -return ; +/* No need to lock the zones, they can't change. */ +if (zone_end_pfn > node_end_pfn) + node_end_pfn = zone_end_pfn; +if (zone->zone_start_pfn < node_start_pfn) + node_start_pfn = zone->zone_start_pfn; } -/* The pgdat has no valid section */ - pgdat->node_start_pfn =0 ; - pgdat->node_spanned_pages =0 ; + pgdat->node_start_pfn = node_start_pfn; + pgdat->node_spanned_pages = node_end_pfn - node_start_pfn; } static void __remove_zone(struct zone *zone, unsigned long start_pfn,
@@ -507,7 +465,7 @@ static void __remove_zone(struct zone *zone, unsigned long start_pfn,pgdat_resize_lock (zone->zone_pgdat, &flags);shrink_zone_span (zone, start_pfn, start_pfn + nr_pages); -shrink_pgdat_span (pgdat, start_pfn, start_pfn + nr_pages); +update_pgdat_span (pgdat);pgdat_resize_unlock (zone->zone_pgdat, &flags); }
Signature · 1 line
2.21.0
Re: [PATCH v6 03/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_pgdat_span()
Quoted 12 lines · David Hildenbrand
We might use the nid of memmaps that were never initialized. For example, if the memmap was poisoned, we will crash the kernel in
pfn_to_nid()right now. Let's use the calculated boundaries of the separate zones instead. This now also avoids having to iterate over a whole bunch of subsections again, after shrinking one zone.Before commit
d0dc12e86b31("mm/memory_hotplug: optimize memory hotplug"), the memmap was initialized to 0 and the node was set to the right value. After that commit, the node might be garbage.We'll have to fix
shrink_zone_span()next.
Quoted 8 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")
@Andrew, can you convert that to
and add
Signature · 3 lines
Thanks, David / dhildenb
PATCH v6 4/10 mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()
In reply to David HildenbrandLet's limit shrinking to !ZONE_DEVICE so we can fix the current code. We should never try to touch the memmap of offline sections where we could have uninitialized memmaps and could trigger BUGs when calling page_to_nid() on poisoned pages.
There is no reliable way to distinguish an uninitialized memmap from an initialized memmap that belongs to ZONE_DEVICE, as we don't have anything like SECTION_IS_ONLINE we can use similar to pfn_to_online_section() for !ZONE_DEVICE memory. E.g., set_zone_contiguous() similarly relies on pfn_to_online_section() and will therefore never set a ZONE_DEVICE zone consecutive. Stopping to shrink the ZONE_DEVICE therefore results in no observable changes, besides /proc/zoneinfo indicating different boundaries - something we can totally live with.
Before commit d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug"), the memmap was initialized with 0 and the node with the right value. So the zone might be wrong but not garbage. After that commit, both the zone and the node will be garbage when touching uninitialized memmaps.
--- mm/memory_hotplug.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
mm/memory_hotplug.c +13 -3
@@ -331,7 +331,7 @@ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone, unsigned long end_pfn) {for (; start_pfn < end_pfn; start_pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_valid (start_pfn))) +if (unlikely (!pfn_to_online_page (start_pfn)))continue ;if (unlikely (pfn_to_nid (start_pfn) != nid))
@@ -356,7 +356,7 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,/* pfn is the end pfn of a memory section. */ pfn = end_pfn -1 ;for (; pfn >= start_pfn; pfn -=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_valid (pfn))) +if (unlikely (!pfn_to_online_page (pfn)))continue ;if (unlikely (pfn_to_nid (pfn) != nid))
@@ -415,7 +415,7 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,*/ pfn = zone_start_pfn;for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_valid (pfn))) +if (unlikely (!pfn_to_online_page (pfn)))continue ;if (page_zone (pfn_to_page (pfn)) != zone)
@@ -463,6 +463,16 @@ static void __remove_zone(struct zone *zone, unsigned long start_pfn,struct pglist_data *pgdat = zone->zone_pgdat;unsigned long flags; +#ifdef CONFIG_ZONE_DEVICE +/* +* Zone shrinking code cannot properly deal with ZONE_DEVICE. So +* we will not try to shrink the zones - which is okay as +* set_zone_contiguous() cannot deal with ZONE_DEVICE either way. +*/ +if (zone_idx (zone) =ZONE_DEVICE ) +return ; +#endif +pgdat_resize_lock (zone->zone_pgdat, &flags);shrink_zone_span (zone, start_pfn, start_pfn + nr_pages);update_pgdat_span (pgdat);
Signature · 1 line
2.21.0
Re: [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()
Quoted 21 lines · David Hildenbrand
Let's limit shrinking to !ZONE_DEVICE so we can fix the current code. We should never try to touch the memmap of offline sections where we could have uninitialized memmaps and could trigger BUGs when calling
page_to_nid()on poisoned pages.There is no reliable way to distinguish an uninitialized memmap from an initialized memmap that belongs to
ZONE_DEVICE, as we don't have anything likeSECTION_IS_ONLINEwe can use similar topfn_to_online_section()for !ZONE_DEVICE memory. E.g.,set_zone_contiguous()similarly relies onpfn_to_online_section()and will therefore never set aZONE_DEVICEzone consecutive. Stopping to shrink theZONE_DEVICEtherefore results in no observable changes, besides/proc/zoneinfoindicating different boundaries - something we can totally live with.Before commit
d0dc12e86b31("mm/memory_hotplug: optimize memory hotplug"), the memmap was initialized with 0 and the node with the right value. So the zone might be wrong but not garbage. After that commit, both the zone and the node will be garbage when touching uninitialized memmaps.
Quoted 7 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")
@Andrew, can you convert that to
memory to zones until online") # visible after d0dc12e86b319
and add
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()
Quoted 10 lines · On Mon, 14 Oct 2019 11:32:13 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 1 line
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")@Andrew, can you convert that to
Fixes f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotaddedmemory to zones until online") # visible after
d0dc12e86b319and add
Cc [email protected] # v4.13+
Done, thanks.
Re: [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()
Quoted 6 lines · Andrew Morton
Quoted 4 lines · On Mon, 14 Oct 2019 11:32:13 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 1 line
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")@Andrew, can you convert that to
Quoted 9 lines
Quoted 6 lines
Fixes f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotaddedmemory to zones until online") # visible after
d0dc12e86b319and add
Cc [email protected] # v4.13+Done, thanks.
Just a note that Toshiki reported a BUG (race between delayed initialization of ZONE_DEVICE memmaps without holding the memory hotplug lock and concurrent zone shrinking).
https://lkml.org/lkml/2019/11/14/1040
"Iteration of create and destroy namespace causes the panic as below:
[ 41.207694] kernel BUG at mm/page_alloc.c:535! [ 41.208109] invalid opcode: 0000 [#1] SMP PTI [ 41.208508] CPU: 7 PID: 2766 Comm: ndctl Not tainted 5.4.0-rc4 #6 [ 41.209064] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014 [ 41.210175] RIP: 0010:set_pfnblock_flags_mask+0x95/0xf0 [ 41.210643] Code: 04 41 83 e2 3c 48 8d 04 a8 48 c1 e0 07 48 03 04 dd e0 59 55 bb 48 8b 58 68 48 39 da 73 0e 48 c7 c6 70 ac 11 bb e8 1b b2 fd ff <0f> 0b 48 03 58 78 48 39 da 73 e9 49 01 ca b9 3f 00 00 00 4f 8d 0c [ 41.212354] RSP: 0018:ffffac0d41557c80 EFLAGS: 00010246 [ 41.212821] RAX: 000000000000004a RBX: 0000000000244a00 RCX: 0000000000000000 [ 41.213459] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffbb1197dc [ 41.214100] RBP: 000000000000000c R08: 0000000000000439 R09: 0000000000000059 [ 41.214736] R10: 0000000000000000 R11: ffffac0d41557b08 R12: ffff8be475ea72b0 [ 41.215376] R13: 000000000000fa00 R14: 0000000000250000 R15: 00000000fffc0bb5 [ 41.216008] FS: 00007f30862ab600(0000) GS:ffff8be57bc40000(0000) knlGS:0000000000000000 [ 41.216771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 41.217299] CR2: 000055e824d0d508 CR3: 0000000231dac000 CR4: 00000000000006e0 [ 41.217934] Call Trace: [ 41.218225] memmap_init_zone_device+0x165/0x17c [ 41.218642] memremap_pages+0x4c1/0x540 [ 41.218989] devm_memremap_pages+0x1d/0x60 [ 41.219367] pmem_attach_disk+0x16b/0x600 [nd_pmem] [ 41.219804] ? devm_nsio_enable+0xb8/0xe0 [ 41.220172] nvdimm_bus_probe+0x69/0x1c0 [ 41.220526] really_probe+0x1c2/0x3e0 [ 41.220856] driver_probe_device+0xb4/0x100 [ 41.221238] device_driver_attach+0x4f/0x60 [ 41.221611] bind_store+0xc9/0x110 [ 41.221919] kernfs_fop_write+0x116/0x190 [ 41.222326] vfs_write+0xa5/0x1a0 [ 41.222626] ksys_write+0x59/0xd0 [ 41.222927] do_syscall_64+0x5b/0x180 [ 41.223264] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 41.223714] RIP: 0033:0x7f30865d0ed8 [ 41.224037] Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 45 78 0d 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 55 [ 41.225920] RSP: 002b:00007fffe5d30a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 41.226608] RAX: ffffffffffffffda RBX: 000055e824d07f40 RCX: 00007f30865d0ed8 [ 41.227242] RDX: 0000000000000007 RSI: 000055e824d07f40 RDI: 0000000000000004 [ 41.227870] RBP: 0000000000000007 R08: 0000000000000007 R09: 0000000000000006 [ 41.228753] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000004 [ 41.229419] R13: 00007f30862ab528 R14: 0000000000000001 R15: 000055e824d07f40
While creating a namespace and initializing memmap, if you destroy the namespace and shrink the zone, it will initialize the memmap outside the zone and trigger VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page) in set_pfnblock_flags_mask()."
This BUG is also mitigated by this commit, where we for now stop to shrink the ZONE_DEVICE zone until we can do it in a safe and clean way.
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v6 04/10] mm/memory_hotplug: Don't access uninitialized memmaps in shrink_zone_span()
Quoted 71 lines · On Tue, 19 Nov 2019 15:16:22 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 6 lines · Andrew Morton
Quoted 4 lines · On Mon, 14 Oct 2019 11:32:13 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 1 line
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")@Andrew, can you convert that to
Quoted 9 lines
Quoted 6 lines
Fixes f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotaddedmemory to zones until online") # visible after
d0dc12e86b319and add
Cc [email protected] # v4.13+Done, thanks.
Just a note that Toshiki reported a BUG (race between delayed initialization of
ZONE_DEVICEmemmaps without holding the memory hotplug lock and concurrent zone shrinking).https://lkml.org/lkml/2019/11/14/1040
"Iteration of create and destroy namespace causes the panic as below:
[ 41.207694] kernel BUG at mm/page_alloc.c:535! [ 41.208109] invalid opcode: 0000 [#1] SMP PTI [ 41.208508] CPU: 7 PID: 2766 Comm: ndctl Not tainted 5.4.0-rc4 #6 [ 41.209064] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.0-0-g63451fca13-prebuilt.qemu-project.org 04/01/2014 [ 41.210175] RIP: 0010:set_pfnblock_flags_mask+0x95/0xf0 [ 41.210643] Code: 04 41 83 e2 3c 48 8d 04 a8 48 c1 e0 07 48 03 04 dd e0 59 55 bb 48 8b 58 68 48 39 da 73 0e 48 c7 c6 70 ac 11 bb e8 1b b2 fd ff <0f> 0b 48 03 58 78 48 39 da 73 e9 49 01 ca b9 3f 00 00 00 4f 8d 0c [ 41.212354] RSP: 0018:ffffac0d41557c80 EFLAGS: 00010246 [ 41.212821] RAX: 000000000000004a RBX: 0000000000244a00 RCX: 0000000000000000 [ 41.213459] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffbb1197dc [ 41.214100] RBP: 000000000000000c R08: 0000000000000439 R09: 0000000000000059 [ 41.214736] R10: 0000000000000000 R11: ffffac0d41557b08 R12: ffff8be475ea72b0 [ 41.215376] R13: 000000000000fa00 R14: 0000000000250000 R15: 00000000fffc0bb5 [ 41.216008] FS: 00007f30862ab600(0000) GS:ffff8be57bc40000(0000) knlGS:0000000000000000 [ 41.216771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 41.217299] CR2: 000055e824d0d508 CR3: 0000000231dac000 CR4: 00000000000006e0 [ 41.217934] Call Trace: [ 41.218225] memmap_init_zone_device+0x165/0x17c [ 41.218642] memremap_pages+0x4c1/0x540 [ 41.218989] devm_memremap_pages+0x1d/0x60 [ 41.219367] pmem_attach_disk+0x16b/0x600 [nd_pmem] [ 41.219804] ? devm_nsio_enable+0xb8/0xe0 [ 41.220172] nvdimm_bus_probe+0x69/0x1c0 [ 41.220526] really_probe+0x1c2/0x3e0 [ 41.220856] driver_probe_device+0xb4/0x100 [ 41.221238] device_driver_attach+0x4f/0x60 [ 41.221611] bind_store+0xc9/0x110 [ 41.221919] kernfs_fop_write+0x116/0x190 [ 41.222326] vfs_write+0xa5/0x1a0 [ 41.222626] ksys_write+0x59/0xd0 [ 41.222927] do_syscall_64+0x5b/0x180 [ 41.223264] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 41.223714] RIP: 0033:0x7f30865d0ed8 [ 41.224037] Code: 89 02 48 c7 c0 ff ff ff ff eb b3 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 8d 05 45 78 0d 00 8b 00 85 c0 75 17 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 58 c3 0f 1f 80 00 00 00 00 41 54 49 89 d4 55 [ 41.225920] RSP: 002b:00007fffe5d30a78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 41.226608] RAX: ffffffffffffffda RBX: 000055e824d07f40 RCX: 00007f30865d0ed8 [ 41.227242] RDX: 0000000000000007 RSI: 000055e824d07f40 RDI: 0000000000000004 [ 41.227870] RBP: 0000000000000007 R08: 0000000000000007 R09: 0000000000000006 [ 41.228753] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000004 [ 41.229419] R13: 00007f30862ab528 R14: 0000000000000001 R15: 000055e824d07f40While creating a namespace and initializing memmap, if you destroy the namespace and shrink the zone, it will initialize the memmap outside the zone and trigger
VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page)inset_pfnblock_flags_mask()."
Quoted 4 lines
This BUG is also mitigated by this commit, where we for now stop to shrink the
ZONE_DEVICEzone until we can do it in a safe and clean way.
OK, thanks. I updated the changelog, added Reported-by:Toshiki and shall squeeze this fix into 5.4.
PATCH v6 5/10 mm/memory_hotplug: Shrink zones when offlining memory
In reply to David HildenbrandWe currently try to shrink a single zone when removing memory. We use the zone of the first page of the memory we are removing. If that memmap was never initialized (e.g., memory was never onlined), we will read garbage and can trigger kernel BUGs (due to a stale pointer):
:/# [ 23.912993] BUG: unable to handle page fault for address: 000000000000353d [ 23.914219] #PF: supervisor write access in kernel mode [ 23.915199] #PF: error_code(0x0002) - not-present page [ 23.916160] PGD 0 P4D 0 [ 23.916627] Oops: 0002 [#1] SMP PTI [ 23.917256] CPU: 1 PID: 7 Comm: kworker/u8:0 Not tainted 5.3.0-rc5-next-20190820+ #317 [ 23.918900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.4 [ 23.921194] Workqueue: kacpi_hotplug acpi_hotplug_work_fn [ 23.922249] RIP: 0010:clear_zone_contiguous+0x5/0x10 [ 23.923173] Code: 48 89 c6 48 89 c3 e8 2a fe ff ff 48 85 c0 75 cf 5b 5d c3 c6 85 fd 05 00 00 01 5b 5d c3 0f 1f 840 [ 23.926876] RSP: 0018:ffffad2400043c98 EFLAGS: 00010246 [ 23.927928] RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000000000 [ 23.929458] RDX: 0000000000200000 RSI: 0000000000140000 RDI: 0000000000002f40 [ 23.930899] RBP: 0000000140000000 R08: 0000000000000000 R09: 0000000000000001 [ 23.932362] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000140000 [ 23.933603] R13: 0000000000140000 R14: 0000000000002f40 R15: ffff9e3e7aff3680 [ 23.934913] FS: 0000000000000000(0000) GS:ffff9e3e7bb00000(0000) knlGS:0000000000000000 [ 23.936294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 23.937481] CR2: 000000000000353d CR3: 0000000058610000 CR4: 00000000000006e0 [ 23.938687] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 23.939889] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 23.941168] Call Trace: [ 23.941580] __remove_pages+0x4b/0x640 [ 23.942303] ? mark_held_locks+0x49/0x70 [ 23.943149] arch_remove_memory+0x63/0x8d [ 23.943921] try_remove_memory+0xdb/0x130 [ 23.944766] ? walk_memory_blocks+0x7f/0x9e [ 23.945616] __remove_memory+0xa/0x11 [ 23.946274] acpi_memory_device_remove+0x70/0x100 [ 23.947308] acpi_bus_trim+0x55/0x90 [ 23.947914] acpi_device_hotplug+0x227/0x3a0 [ 23.948714] acpi_hotplug_work_fn+0x1a/0x30 [ 23.949433] process_one_work+0x221/0x550 [ 23.950190] worker_thread+0x50/0x3b0 [ 23.950993] kthread+0x105/0x140 [ 23.951644] ? process_one_work+0x550/0x550 [ 23.952508] ? kthread_park+0x80/0x80 [ 23.953367] ret_from_fork+0x3a/0x50 [ 23.954025] Modules linked in: [ 23.954613] CR2: 000000000000353d [ 23.955248] ---[ end trace 93d982b1fb3e1a69 ]---
Instead, shrink the zones when offlining memory or when onlining failed. Introduce and use remove_pfn_range_from_zone(() for that. We now properly shrink the zones, even if we have DIMMs whereby
- Some memory blocks fall into no zone (never onlined)
- Some memory blocks fall into multiple zones (offlined+re-onlined)
- Multiple memory blocks that fall into different zones
Drop the zone parameter (with a potential dubious value) from __remove_pages() and __remove_section().
--- arch/arm64/mm/mmu.c | 4 +--- arch/ia64/mm/init.c | 4 +--- arch/powerpc/mm/mem.c | 3 +-- arch/s390/mm/init.c | 4 +--- arch/sh/mm/init.c | 4 +--- arch/x86/mm/init_32.c | 4 +--- arch/x86/mm/init_64.c | 4 +--- include/linux/memory_hotplug.h | 7 +++++-- mm/memory_hotplug.c | 31 ++++++++++++++++--------------- mm/memremap.c | 2 +- 10 files changed, 29 insertions(+), 38 deletions(-)
arch/arm64/mm/mmu.c +1 -3
@@ -1069,7 +1069,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone;/* * FIXME: Cleanup page tables (also in arch_add_memory() in case
@@ -1078,7 +1077,6 @@ void arch_remove_memory(int nid, u64 start, u64 size,* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be * unlocked yet. */ - zone =page_zone (pfn_to_page (start_pfn)); - __remove_pages(zone, start_pfn, nr_pages, altmap); + __remove_pages(start_pfn, nr_pages, altmap); }#endif
arch/ia64/mm/init.c +1 -3
@@ -689,9 +689,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif
arch/powerpc/mm/mem.c +1 -2
@@ -130,10 +130,9 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap);int ret; -__remove_pages (page_zone (page), start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);/* Remove htab bolted mappings for this section of memory */ start = (unsigned long )__va (start);
arch/s390/mm/init.c +1 -3
@@ -291,10 +291,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);vmem_remove_mapping (start, size); }#endif /* CONFIG_MEMORY_HOTPLUG */
arch/sh/mm/init.c +1 -3
@@ -434,9 +434,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn =PFN_DOWN (start);unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif /* CONFIG_MEMORY_HOTPLUG */
arch/x86/mm/init_32.c +1 -3
@@ -865,10 +865,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif
arch/x86/mm/init_64.c +1 -3
@@ -1212,10 +1212,8 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap); -struct zone *zone =page_zone (page); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);kernel_physical_mapping_remove (start, start + size); }#endif /* CONFIG_MEMORY_HOTPLUG */
include/linux/memory_hotplug.h +5 -2
@@ -126,8 +126,8 @@ static inline bool movable_node_is_enabled(void)extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap); -extern void __remove_pages (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages,struct vmem_altmap *altmap); +extern void __remove_pages (unsigned long start_pfn,unsigned long nr_pages, +struct vmem_altmap *altmap);/* reasonably generic interface to expand the physical pages */ extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,
@@ -346,6 +346,9 @@ extern int add_memory(int nid, u64 start, u64 size);extern int add_memory_resource (int nid,struct resource *resource);extern void move_pfn_range_to_zone (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); +extern void remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages);extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_section (int nid,unsigned long pfn,unsigned long nr_pages,struct vmem_altmap *altmap);
mm/memory_hotplug.c +16 -15
@@ -457,8 +457,9 @@ static void update_pgdat_span(struct pglist_data *pgdat) pgdat->node_spanned_pages = node_end_pfn - node_start_pfn; } -static void __remove_zone (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages) +void __ref remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages) {struct pglist_data *pgdat = zone->zone_pgdat;unsigned long flags;
@@ -473,28 +474,30 @@ static void __remove_zone(struct zone *zone, unsigned long start_pfn,return ;#endif +clear_zone_contiguous (zone); +pgdat_resize_lock (zone->zone_pgdat, &flags);shrink_zone_span (zone, start_pfn, start_pfn + nr_pages);update_pgdat_span (pgdat);pgdat_resize_unlock (zone->zone_pgdat, &flags); + +set_zone_contiguous (zone); } -static void __remove_section (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,unsigned long map_offset, -struct vmem_altmap *altmap) +static void __remove_section (unsigned long pfn,unsigned long nr_pages, +unsigned long map_offset, +struct vmem_altmap *altmap) {struct mem_section *ms =__nr_to_section (pfn_to_section_nr (pfn));if (WARN_ON_ONCE (!valid_section (ms)))return ; -__remove_zone (zone, pfn, nr_pages);sparse_remove_section (ms, pfn, nr_pages, map_offset, altmap); }/** -* __remove_pages() - remove sections of pages from a zone -* @zone: zone from which pages need to be removed +* __remove_pages() - remove sections of pages * @pfn: starting pageframe (must be aligned to start of a section) * @nr_pages: number of pages to remove (must be multiple of section size) * @altmap: alternative device page map or %NULL if default memmap is used
@@ -504,16 +507,14 @@ static void __remove_section(struct zone *zone, unsigned long pfn,* sure that pages are marked reserved and zones are adjust properly by * calling offline_pages(). */ -void __remove_pages (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,struct vmem_altmap *altmap) +void __remove_pages (unsigned long pfn,unsigned long nr_pages, +struct vmem_altmap *altmap) {unsigned long map_offset =0 ;unsigned long nr, start_sec, end_sec; map_offset =vmem_altmap_offset (altmap); -clear_zone_contiguous (zone); -if (check_pfn_span (pfn, nr_pages,"remove" ))return ;
@@ -525,13 +526,11 @@ void __remove_pages(struct zone *zone, unsigned long pfn,cond_resched (); pfns =min (nr_pages,PAGES_PER_SECTION - (pfn & ~PAGE_SECTION_MASK )); -__remove_section (zone, pfn, pfns, map_offset, altmap); +__remove_section (pfn, pfns, map_offset, altmap); pfn += pfns; nr_pages -= pfns; map_offset =0 ; } - -set_zone_contiguous (zone); }int set_online_page_callback (online_page_callback_t callback)
@@ -859,6 +858,7 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ (unsigned long long ) pfn <<PAGE_SHIFT , (((unsigned long long) pfn + nr_pages) << PAGE_SHIFT) - 1);memory_notify (MEM_CANCEL_ONLINE , &arg); +remove_pfn_range_from_zone (zone, pfn, nr_pages);mem_hotplug_done ();return ret; }
@@ -1605,6 +1605,7 @@ static int __ref __offline_pages(unsigned long start_pfn,writeback_set_ratelimit ();memory_notify (MEM_OFFLINE , &arg); +remove_pfn_range_from_zone (zone, start_pfn, nr_pages);mem_hotplug_done ();return 0 ;
mm/memremap.c +1 -1
@@ -140,7 +140,7 @@ void memunmap_pages(struct dev_pagemap *pgmap)mem_hotplug_begin ();if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) { -__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +__remove_pages (PHYS_PFN (res->start),PHYS_PFN (resource_size (res)),NULL ); }else { arch_remove_memory(nid, res->start, resource_size(res),
Signature · 1 line
2.21.0
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 100 lines · David Hildenbrand
We currently try to shrink a single zone when removing memory. We use the zone of the first page of the memory we are removing. If that memmap was never initialized (e.g., memory was never onlined), we will read garbage and can trigger kernel BUGs (due to a stale pointer):
:/# [ 23.912993] BUG: unable to handle page fault for address: 000000000000353d [ 23.914219] #PF: supervisor write access in kernel mode [ 23.915199] #PF: error_code(0x0002) - not-present page [ 23.916160] PGD 0 P4D 0 [ 23.916627] Oops: 0002 [#1] SMP PTI [ 23.917256] CPU: 1 PID: 7 Comm: kworker/u8:0 Not tainted 5.3.0-rc5-next-20190820+ #317 [ 23.918900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.4 [ 23.921194] Workqueue: kacpi_hotplug acpi_hotplug_work_fn [ 23.922249] RIP: 0010:clear_zone_contiguous+0x5/0x10 [ 23.923173] Code: 48 89 c6 48 89 c3 e8 2a fe ff ff 48 85 c0 75 cf 5b 5d c3 c6 85 fd 05 00 00 01 5b 5d c3 0f 1f 840 [ 23.926876] RSP: 0018:ffffad2400043c98 EFLAGS: 00010246 [ 23.927928] RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000000000 [ 23.929458] RDX: 0000000000200000 RSI: 0000000000140000 RDI: 0000000000002f40 [ 23.930899] RBP: 0000000140000000 R08: 0000000000000000 R09: 0000000000000001 [ 23.932362] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000140000 [ 23.933603] R13: 0000000000140000 R14: 0000000000002f40 R15: ffff9e3e7aff3680 [ 23.934913] FS: 0000000000000000(0000) GS:ffff9e3e7bb00000(0000) knlGS:0000000000000000 [ 23.936294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 23.937481] CR2: 000000000000353d CR3: 0000000058610000 CR4: 00000000000006e0 [ 23.938687] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 23.939889] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 23.941168] Call Trace: [ 23.941580] __remove_pages+0x4b/0x640 [ 23.942303] ? mark_held_locks+0x49/0x70 [ 23.943149] arch_remove_memory+0x63/0x8d [ 23.943921] try_remove_memory+0xdb/0x130 [ 23.944766] ? walk_memory_blocks+0x7f/0x9e [ 23.945616] __remove_memory+0xa/0x11 [ 23.946274] acpi_memory_device_remove+0x70/0x100 [ 23.947308] acpi_bus_trim+0x55/0x90 [ 23.947914] acpi_device_hotplug+0x227/0x3a0 [ 23.948714] acpi_hotplug_work_fn+0x1a/0x30 [ 23.949433] process_one_work+0x221/0x550 [ 23.950190] worker_thread+0x50/0x3b0 [ 23.950993] kthread+0x105/0x140 [ 23.951644] ? process_one_work+0x550/0x550 [ 23.952508] ? kthread_park+0x80/0x80 [ 23.953367] ret_from_fork+0x3a/0x50 [ 23.954025] Modules linked in: [ 23.954613] CR2: 000000000000353d [ 23.955248] ---[ end trace 93d982b1fb3e1a69 ]---Instead, shrink the zones when offlining memory or when onlining failed. Introduce and use remove_pfn_range_from_zone(() for that. We now properly shrink the zones, even if we have DIMMs whereby
- Some memory blocks fall into no zone (never onlined)
- Some memory blocks fall into multiple zones (offlined+re-onlined)
- Multiple memory blocks that fall into different zonesDrop the zone parameter (with a potential dubious value) from
__remove_pages()and__remove_section().Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Tony Luck <[email protected]>Cc Fenghua Yu <[email protected]>Cc Benjamin Herrenschmidt <[email protected]>Cc Paul Mackerras <[email protected]>Cc Michael Ellerman <[email protected]>Cc Heiko Carstens <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Yoshinori Sato <[email protected]>Cc Rich Felker <[email protected]>Cc Dave Hansen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Andrew Morton <[email protected]>Cc Mark Rutland <[email protected]>Cc Steve Capper <[email protected]>Cc Mike Rapoport <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Yu Zhao <[email protected]>Cc Jun Yao <[email protected]>Cc Robin Murphy <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc "Matthew Wilcox (Oracle)" <[email protected]>Cc Christophe Leroy <[email protected]>Cc "Aneesh Kumar K.V" <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Gerald Schaefer <[email protected]>Cc Halil Pasic <[email protected]>Cc Tom Lendacky <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Cc Qian Cai <[email protected]>Cc Jason Gunthorpe <[email protected]>
Quoted 8 lines
Cc Logan Gunthorpe <[email protected]>Cc Ira Weiny <[email protected]>Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")
@Andrew, can you convert that to
While adding cc'ing [email protected] # v4.13+ would be nice, I doubt it will be easily possible to backport, as we are missing some prereq patches (e.g., from Oscar like 2c2a5af6fed2 ("mm,memory_hotplug: add nid parameter to arch_remove_memory")). But, it could be done with some work.
I think "Cc: [email protected] # v5.0+" could be done more easily. Maybe it's okay to not cc:stable this one. We usually online all memory (except s390x), however, s390x does not remove that memory ever. Devmem with driver reserved memory would be, however, worth backporting this.
Thoughts?
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 5 lines · On Mon, 14 Oct 2019 11:39:13 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 1 line
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")@Andrew, can you convert that to
Fixes f1dd2cd13c4b ("mm, memory_hotplug: do not associate hotadded memory to zones until online") # visible after d0dc12e86b319
Done.
Quoted 11 lines
While adding cc'ing [email protected] # v4.13+ would be nice, I doubt it will be easily possible to backport, as we are missing some prereq patches (e.g., from Oscar like
2c2a5af6fed2("mm,memory_hotplug: add nid parameter toarch_remove_memory")). But, it could be done with some work.I think "Cc: [email protected] # v5.0+" could be done more easily. Maybe it's okay to not cc:stable this one. We usually online all memory (except s390x), however, s390x does not remove that memory ever. Devmem with driver reserved memory would be, however, worth backporting this.
I added
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
In reply to David HildenbrandQuoted 371 lines · David Hildenbrand
We currently try to shrink a single zone when removing memory. We use the zone of the first page of the memory we are removing. If that memmap was never initialized (e.g., memory was never onlined), we will read garbage and can trigger kernel BUGs (due to a stale pointer):
:/# [ 23.912993] BUG: unable to handle page fault for address: 000000000000353d [ 23.914219] #PF: supervisor write access in kernel mode [ 23.915199] #PF: error_code(0x0002) - not-present page [ 23.916160] PGD 0 P4D 0 [ 23.916627] Oops: 0002 [#1] SMP PTI [ 23.917256] CPU: 1 PID: 7 Comm: kworker/u8:0 Not tainted 5.3.0-rc5-next-20190820+ #317 [ 23.918900] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.4 [ 23.921194] Workqueue: kacpi_hotplug acpi_hotplug_work_fn [ 23.922249] RIP: 0010:clear_zone_contiguous+0x5/0x10 [ 23.923173] Code: 48 89 c6 48 89 c3 e8 2a fe ff ff 48 85 c0 75 cf 5b 5d c3 c6 85 fd 05 00 00 01 5b 5d c3 0f 1f 840 [ 23.926876] RSP: 0018:ffffad2400043c98 EFLAGS: 00010246 [ 23.927928] RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000000000 [ 23.929458] RDX: 0000000000200000 RSI: 0000000000140000 RDI: 0000000000002f40 [ 23.930899] RBP: 0000000140000000 R08: 0000000000000000 R09: 0000000000000001 [ 23.932362] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000140000 [ 23.933603] R13: 0000000000140000 R14: 0000000000002f40 R15: ffff9e3e7aff3680 [ 23.934913] FS: 0000000000000000(0000) GS:ffff9e3e7bb00000(0000) knlGS:0000000000000000 [ 23.936294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 23.937481] CR2: 000000000000353d CR3: 0000000058610000 CR4: 00000000000006e0 [ 23.938687] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 23.939889] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 23.941168] Call Trace: [ 23.941580] __remove_pages+0x4b/0x640 [ 23.942303] ? mark_held_locks+0x49/0x70 [ 23.943149] arch_remove_memory+0x63/0x8d [ 23.943921] try_remove_memory+0xdb/0x130 [ 23.944766] ? walk_memory_blocks+0x7f/0x9e [ 23.945616] __remove_memory+0xa/0x11 [ 23.946274] acpi_memory_device_remove+0x70/0x100 [ 23.947308] acpi_bus_trim+0x55/0x90 [ 23.947914] acpi_device_hotplug+0x227/0x3a0 [ 23.948714] acpi_hotplug_work_fn+0x1a/0x30 [ 23.949433] process_one_work+0x221/0x550 [ 23.950190] worker_thread+0x50/0x3b0 [ 23.950993] kthread+0x105/0x140 [ 23.951644] ? process_one_work+0x550/0x550 [ 23.952508] ? kthread_park+0x80/0x80 [ 23.953367] ret_from_fork+0x3a/0x50 [ 23.954025] Modules linked in: [ 23.954613] CR2: 000000000000353d [ 23.955248] ---[ end trace 93d982b1fb3e1a69 ]---Instead, shrink the zones when offlining memory or when onlining failed. Introduce and use remove_pfn_range_from_zone(() for that. We now properly shrink the zones, even if we have DIMMs whereby
- Some memory blocks fall into no zone (never onlined)
- Some memory blocks fall into multiple zones (offlined+re-onlined)
- Multiple memory blocks that fall into different zonesDrop the zone parameter (with a potential dubious value) from
__remove_pages()and__remove_section().Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Tony Luck <[email protected]>Cc Fenghua Yu <[email protected]>Cc Benjamin Herrenschmidt <[email protected]>Cc Paul Mackerras <[email protected]>Cc Michael Ellerman <[email protected]>Cc Heiko Carstens <[email protected]>Cc Vasily Gorbik <[email protected]>Cc Christian Borntraeger <[email protected]>Cc Yoshinori Sato <[email protected]>Cc Rich Felker <[email protected]>Cc Dave Hansen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Andrew Morton <[email protected]>Cc Mark Rutland <[email protected]>Cc Steve Capper <[email protected]>Cc Mike Rapoport <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Yu Zhao <[email protected]>Cc Jun Yao <[email protected]>Cc Robin Murphy <[email protected]>Cc Michal Hocko <[email protected]>Cc Oscar Salvador <[email protected]>Cc "Matthew Wilcox (Oracle)" <[email protected]>Cc Christophe Leroy <[email protected]>Cc "Aneesh Kumar K.V" <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Gerald Schaefer <[email protected]>Cc Halil Pasic <[email protected]>Cc Tom Lendacky <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Masahiro Yamada <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Cc Qian Cai <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Ira Weiny <[email protected]>Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")Signed-off-by David Hildenbrand <[email protected]>--- arch/arm64/mm/mmu.c | 4 +--- arch/ia64/mm/init.c | 4 +--- arch/powerpc/mm/mem.c | 3 +-- arch/s390/mm/init.c | 4 +--- arch/sh/mm/init.c | 4 +--- arch/x86/mm/init_32.c | 4 +--- arch/x86/mm/init_64.c | 4 +--- include/linux/memory_hotplug.h | 7 +++++-- mm/memory_hotplug.c | 31 ++++++++++++++++--------------- mm/memremap.c | 2 +- 10 files changed, 29 insertions(+), 38 deletions(-)arch/arm64/mm/mmu.c +1
-3@@ -1069,7 +1069,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone;/* * FIXME: Cleanup page tables (also in arch_add_memory() in case @@ -1078,7 +1077,6 @@ void arch_remove_memory(int nid, u64 start, u64 size,* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be * unlocked yet. */ - zone =page_zone (pfn_to_page (start_pfn)); - __remove_pages(zone, start_pfn, nr_pages, altmap); + __remove_pages(start_pfn, nr_pages, altmap); }#endif arch/ia64/mm/init.c +1
-3@@ -689,9 +689,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif arch/powerpc/mm/mem.c +1
-2@@ -130,10 +130,9 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap);int ret; -__remove_pages (page_zone (page), start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);/* Remove htab bolted mappings for this section of memory */ start = (unsigned long )__va (start);arch/s390/mm/init.c +1
-3@@ -291,10 +291,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);vmem_remove_mapping (start, size); }#endif /* CONFIG_MEMORY_HOTPLUG */ arch/sh/mm/init.c +1
-3@@ -434,9 +434,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn =PFN_DOWN (start);unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif /* CONFIG_MEMORY_HOTPLUG */ arch/x86/mm/init_32.c +1
-3@@ -865,10 +865,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif arch/x86/mm/init_64.c +1
-3@@ -1212,10 +1212,8 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap); -struct zone *zone =page_zone (page); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);kernel_physical_mapping_remove (start, start + size); }#endif /* CONFIG_MEMORY_HOTPLUG */ include/linux/memory_hotplug.h +5
-2@@ -126,8 +126,8 @@ static inline bool movable_node_is_enabled(void)extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap); -extern void __remove_pages (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages,struct vmem_altmap *altmap); +extern void __remove_pages (unsigned long start_pfn,unsigned long nr_pages, +struct vmem_altmap *altmap);/* reasonably generic interface to expand the physical pages */ extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,@@ -346,6 +346,9 @@ extern int add_memory(int nid, u64 start, u64 size);extern int add_memory_resource (int nid,struct resource *resource);extern void move_pfn_range_to_zone (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); +extern void remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages);extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_section (int nid,unsigned long pfn,unsigned long nr_pages,struct vmem_altmap *altmap);mm/memory_hotplug.c +16
-15@@ -457,8 +457,9 @@ static void update_pgdat_span(struct pglist_data *pgdat) pgdat->node_spanned_pages = node_end_pfn - node_start_pfn; } -static void __remove_zone (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages) +void __ref remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages) {struct pglist_data *pgdat = zone->zone_pgdat;unsigned long flags;@@ -473,28 +474,30 @@ static void __remove_zone(struct zone *zone, unsigned long start_pfn,return ;#endif +clear_zone_contiguous (zone); +pgdat_resize_lock (zone->zone_pgdat, &flags);shrink_zone_span (zone, start_pfn, start_pfn + nr_pages);update_pgdat_span (pgdat);pgdat_resize_unlock (zone->zone_pgdat, &flags); + +set_zone_contiguous (zone); } -static void __remove_section (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,unsigned long map_offset, -struct vmem_altmap *altmap) +static void __remove_section (unsigned long pfn,unsigned long nr_pages, +unsigned long map_offset, +struct vmem_altmap *altmap) {struct mem_section *ms =__nr_to_section (pfn_to_section_nr (pfn));if (WARN_ON_ONCE (!valid_section (ms)))return ; -__remove_zone (zone, pfn, nr_pages);sparse_remove_section (ms, pfn, nr_pages, map_offset, altmap); }/** -* __remove_pages() - remove sections of pages from a zone -* @zone: zone from which pages need to be removed +* __remove_pages() - remove sections of pages * @pfn: starting pageframe (must be aligned to start of a section) * @nr_pages: number of pages to remove (must be multiple of section size) * @altmap: alternative device page map or %NULL if default memmap is used @@ -504,16 +507,14 @@ static void __remove_section(struct zone *zone, unsigned long pfn,* sure that pages are marked reserved and zones are adjust properly by * calling offline_pages(). */ -void __remove_pages (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,struct vmem_altmap *altmap) +void __remove_pages (unsigned long pfn,unsigned long nr_pages, +struct vmem_altmap *altmap) {unsigned long map_offset =0 ;unsigned long nr, start_sec, end_sec; map_offset =vmem_altmap_offset (altmap); -clear_zone_contiguous (zone); -if (check_pfn_span (pfn, nr_pages,"remove" ))return ;@@ -525,13 +526,11 @@ void __remove_pages(struct zone *zone, unsigned long pfn,cond_resched (); pfns =min (nr_pages,PAGES_PER_SECTION - (pfn & ~PAGE_SECTION_MASK )); -__remove_section (zone, pfn, pfns, map_offset, altmap); +__remove_section (pfn, pfns, map_offset, altmap); pfn += pfns; nr_pages -= pfns; map_offset =0 ; } - -set_zone_contiguous (zone); }int set_online_page_callback (online_page_callback_t callback)@@ -859,6 +858,7 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ (unsigned long long ) pfn <<PAGE_SHIFT , (((unsigned long long) pfn + nr_pages) << PAGE_SHIFT) - 1);memory_notify (MEM_CANCEL_ONLINE , &arg); +remove_pfn_range_from_zone (zone, pfn, nr_pages);mem_hotplug_done ();return ret; }@@ -1605,6 +1605,7 @@ static int __ref __offline_pages(unsigned long start_pfn,writeback_set_ratelimit ();memory_notify (MEM_OFFLINE , &arg); +remove_pfn_range_from_zone (zone, start_pfn, nr_pages);mem_hotplug_done ();return 0 ;mm/memremap.c +0
-0
mm/memremap.c void memunmap_pages(struct dev_pagemap *pgmap) · line 146
Quoted 8 lines
mm/memremap.c +1
-1@@ -140,7 +140,7 @@ void memunmap_pages(struct dev_pagemap *pgmap)mem_hotplug_begin ();if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) { -__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +__remove_pages (PHYS_PFN (res->start),PHYS_PFN (resource_size (res)),NULL ); }else { arch_remove_memory(nid, res->start, resource_size(res),
I think I just found an issue with try_offline_node(). try_offline_node() is pretty much broken already (touches garbage memmaps and will not considers mixed NIDs within sections), however, relies on the node span to look for memory sections to probe. So it seems to rely on the nodes getting shrunk when removing memory, not when offlining.
As we shrink the node span when offlining now and not when removing, this can go wrong once we offline the last memory block of the node and offline the last CPU. We could still have memory around that we could re-online, however, the node would already be offline. Unlikely, but possible.
Note that the same is also broken without this patch in case memory is never onlined. The "pfn_to_nid(pfn) != nid" can easily succeed on the garbage memmap, resulting in no memory being detected as belonging to the node. Also, resize_pgdat_range() is called when onlining memory, not when adding it. :/ Oh this is so broken :)
The right fix is probably to walk over all memory blocks that could exist and test if they belong to the nid (if offline, check the block->nid, if online check all pageblocks). A fix we can then move in front of this patch.
Will look into this this week.
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 19 lines · On Sun, 27 Oct 2019 23:45:52 +0100 David Hildenbrand <[email protected]> wrote:
I think I just found an issue with
try_offline_node().try_offline_node()is pretty much broken already (touches garbage memmaps and will not considers mixed NIDs within sections), however, relies on the node span to look for memory sections to probe. So it seems to rely on the nodes getting shrunk when removing memory, not when offlining.As we shrink the node span when offlining now and not when removing, this can go wrong once we offline the last memory block of the node and offline the last CPU. We could still have memory around that we could re-online, however, the node would already be offline. Unlikely, but possible.
Note that the same is also broken without this patch in case memory is never onlined. The "
pfn_to_nid(pfn)!= nid" can easily succeed on the garbage memmap, resulting in no memory being detected as belonging to the node. Also,resize_pgdat_range()is called when onlining memory, not when adding it. :/ Oh this is so broken :)
Quoted 6 lines
The right fix is probably to walk over all memory blocks that could exist and test if they belong to the nid (if offline, check the
block->nid, if online check all pageblocks). A fix we can then move in front of this patch.Will look into this this week.
And this series shows almost no sign of having been reviewed. I'll hold it over for 5.6.
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 30 lines
Am 01.12.2019 um 00:22 schrieb Andrew Morton <[email protected]>:
Quoted 24 lines · On Sun, 27 Oct 2019 23:45:52 +0100 David Hildenbrand <[email protected]> wrote:
I think I just found an issue with
try_offline_node().try_offline_node()is pretty much broken already (touches garbage memmaps and will not considers mixed NIDs within sections), however, relies on the node span to look for memory sections to probe. So it seems to rely on the nodes getting shrunk when removing memory, not when offlining.As we shrink the node span when offlining now and not when removing, this can go wrong once we offline the last memory block of the node and offline the last CPU. We could still have memory around that we could re-online, however, the node would already be offline. Unlikely, but possible.
Note that the same is also broken without this patch in case memory is never onlined. The "
pfn_to_nid(pfn)!= nid" can easily succeed on the garbage memmap, resulting in no memory being detected as belonging to the node. Also,resize_pgdat_range()is called when onlining memory, not when adding it. :/ Oh this is so broken :)The right fix is probably to walk over all memory blocks that could exist and test if they belong to the nid (if offline, check the
block->nid, if online check all pageblocks). A fix we can then move in front of this patch.
Quoted 5 lines
Quoted 1 line
Will look into this this week.
And this series shows almost no sign of having been reviewed. I'll hold it over for 5.6.
Makes sense, can‘t do anything about it. Btw, this one is the last stable patch to fix access of uninitialized memmaps that is not upstream yet... so it has to remain broken for some longer.
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
In reply to Andrew MortonQuoted 26 lines · Andrew Morton
Quoted 24 lines · On Sun, 27 Oct 2019 23:45:52 +0100 David Hildenbrand <[email protected]> wrote:
I think I just found an issue with
try_offline_node().try_offline_node()is pretty much broken already (touches garbage memmaps and will not considers mixed NIDs within sections), however, relies on the node span to look for memory sections to probe. So it seems to rely on the nodes getting shrunk when removing memory, not when offlining.As we shrink the node span when offlining now and not when removing, this can go wrong once we offline the last memory block of the node and offline the last CPU. We could still have memory around that we could re-online, however, the node would already be offline. Unlikely, but possible.
Note that the same is also broken without this patch in case memory is never onlined. The "
pfn_to_nid(pfn)!= nid" can easily succeed on the garbage memmap, resulting in no memory being detected as belonging to the node. Also,resize_pgdat_range()is called when onlining memory, not when adding it. :/ Oh this is so broken :)The right fix is probably to walk over all memory blocks that could exist and test if they belong to the nid (if offline, check the
block->nid, if online check all pageblocks). A fix we can then move in front of this patch.
Quoted 5 lines
Quoted 1 line
Will look into this this week.
And this series shows almost no sign of having been reviewed. I'll hold it over for 5.6.
Hi Andrew, any chance we can get the (now at least reviewed - thx Oscar) fix in patch #5 into 5.5? (I want to do the final stable backports for the uninitialized memmap stuff)
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 29 lines · On Wed, 18 Dec 2019 18:08:04 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 28 lines · Andrew Morton
Quoted 25 lines · On Sun, 27 Oct 2019 23:45:52 +0100 David Hildenbrand <[email protected]> wrote:
I think I just found an issue with
try_offline_node().try_offline_node()is pretty much broken already (touches garbage memmaps and will not considers mixed NIDs within sections), however, relies on the node span to look for memory sections to probe. So it seems to rely on the nodes getting shrunk when removing memory, not when offlining.As we shrink the node span when offlining now and not when removing, this can go wrong once we offline the last memory block of the node and offline the last CPU. We could still have memory around that we could re-online, however, the node would already be offline. Unlikely, but possible.
Note that the same is also broken without this patch in case memory is never onlined. The "
pfn_to_nid(pfn)!= nid" can easily succeed on the garbage memmap, resulting in no memory being detected as belonging to the node. Also,resize_pgdat_range()is called when onlining memory, not when adding it. :/ Oh this is so broken :)The right fix is probably to walk over all memory blocks that could exist and test if they belong to the nid (if offline, check the
block->nid, if online check all pageblocks). A fix we can then move in front of this patch.Will look into this this week.
Quoted 7 lines
Quoted 3 lines
And this series shows almost no sign of having been reviewed. I'll hold it over for 5.6.
Hi Andrew, any chance we can get the (now at least reviewed - thx Oscar) fix in patch #5 into 5.5? (I want to do the final stable backports for the uninitialized memmap stuff)
Sure, I queued it for the next batch of 5.5 fixes.
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
In reply to David HildenbrandQuoted 2 lines · David Hildenbrand
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")Signed-off-by David Hildenbrand <[email protected]>
I did not see anything wrong with the taken approach, and makes sense to me. The only thing that puzzles me is we seem to not balance spanned_pages for ZONE_DEVICE anymore.memremap_pages() increments them via move_pfn_range_to_zone, but we skip ZONE_DEVICE in remove_pfn_range_from_zone.
That is not really related to this patch, so I might be missing something, but it caught my eye while reviewing this.
Anyway, for this one:
off-topic: I __think__ we really need to trim the CC list.
Quoted 272 lines
--- arch/arm64/mm/mmu.c | 4 +--- arch/ia64/mm/init.c | 4 +--- arch/powerpc/mm/mem.c | 3 +-- arch/s390/mm/init.c | 4 +--- arch/sh/mm/init.c | 4 +--- arch/x86/mm/init_32.c | 4 +--- arch/x86/mm/init_64.c | 4 +--- include/linux/memory_hotplug.h | 7 +++++-- mm/memory_hotplug.c | 31 ++++++++++++++++--------------- mm/memremap.c | 2 +- 10 files changed, 29 insertions(+), 38 deletions(-)arch/arm64/mm/mmu.c +1
-3@@ -1069,7 +1069,6 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone;/* * FIXME: Cleanup page tables (also in arch_add_memory() in case @@ -1078,7 +1077,6 @@ void arch_remove_memory(int nid, u64 start, u64 size,* unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be * unlocked yet. */ - zone =page_zone (pfn_to_page (start_pfn)); - __remove_pages(zone, start_pfn, nr_pages, altmap); + __remove_pages(start_pfn, nr_pages, altmap); }#endif arch/ia64/mm/init.c +1
-3@@ -689,9 +689,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif arch/powerpc/mm/mem.c +1
-2@@ -130,10 +130,9 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap);int ret; -__remove_pages (page_zone (page), start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);/* Remove htab bolted mappings for this section of memory */ start = (unsigned long )__va (start);arch/s390/mm/init.c +1
-3@@ -291,10 +291,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);vmem_remove_mapping (start, size); }#endif /* CONFIG_MEMORY_HOTPLUG */ arch/sh/mm/init.c +1
-3@@ -434,9 +434,7 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn =PFN_DOWN (start);unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif /* CONFIG_MEMORY_HOTPLUG */ arch/x86/mm/init_32.c +1
-3@@ -865,10 +865,8 @@ void arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct zone *zone; - zone =page_zone (pfn_to_page (start_pfn)); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap); }#endif arch/x86/mm/init_64.c +1
-3@@ -1212,10 +1212,8 @@ void __ref arch_remove_memory(int nid, u64 start, u64 size, {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -struct page *page =pfn_to_page (start_pfn) +vmem_altmap_offset (altmap); -struct zone *zone =page_zone (page); -__remove_pages (zone, start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap);kernel_physical_mapping_remove (start, start + size); }#endif /* CONFIG_MEMORY_HOTPLUG */ include/linux/memory_hotplug.h +5
-2@@ -126,8 +126,8 @@ static inline bool movable_node_is_enabled(void)extern void arch_remove_memory (int nid,u64 start,u64 size,struct vmem_altmap *altmap); -extern void __remove_pages (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages,struct vmem_altmap *altmap); +extern void __remove_pages (unsigned long start_pfn,unsigned long nr_pages, +struct vmem_altmap *altmap);/* reasonably generic interface to expand the physical pages */ extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,@@ -346,6 +346,9 @@ extern int add_memory(int nid, u64 start, u64 size);extern int add_memory_resource (int nid,struct resource *resource);extern void move_pfn_range_to_zone (struct zone *zone,unsigned long start_pfn,unsigned long nr_pages,struct vmem_altmap *altmap); +extern void remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages);extern bool is_memblock_offlined (struct memory_block *mem);extern int sparse_add_section (int nid,unsigned long pfn,unsigned long nr_pages,struct vmem_altmap *altmap);mm/memory_hotplug.c +16
-15@@ -457,8 +457,9 @@ static void update_pgdat_span(struct pglist_data *pgdat) pgdat->node_spanned_pages = node_end_pfn - node_start_pfn; } -static void __remove_zone (struct zone *zone,unsigned long start_pfn, -unsigned long nr_pages) +void __ref remove_pfn_range_from_zone (struct zone *zone, +unsigned long start_pfn, +unsigned long nr_pages) {struct pglist_data *pgdat = zone->zone_pgdat;unsigned long flags;@@ -473,28 +474,30 @@ static void __remove_zone(struct zone *zone, unsigned long start_pfn,return ;#endif +clear_zone_contiguous (zone); +pgdat_resize_lock (zone->zone_pgdat, &flags);shrink_zone_span (zone, start_pfn, start_pfn + nr_pages);update_pgdat_span (pgdat);pgdat_resize_unlock (zone->zone_pgdat, &flags); + +set_zone_contiguous (zone); } -static void __remove_section (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,unsigned long map_offset, -struct vmem_altmap *altmap) +static void __remove_section (unsigned long pfn,unsigned long nr_pages, +unsigned long map_offset, +struct vmem_altmap *altmap) {struct mem_section *ms =__nr_to_section (pfn_to_section_nr (pfn));if (WARN_ON_ONCE (!valid_section (ms)))return ; -__remove_zone (zone, pfn, nr_pages);sparse_remove_section (ms, pfn, nr_pages, map_offset, altmap); }/** -* __remove_pages() - remove sections of pages from a zone -* @zone: zone from which pages need to be removed +* __remove_pages() - remove sections of pages * @pfn: starting pageframe (must be aligned to start of a section) * @nr_pages: number of pages to remove (must be multiple of section size) * @altmap: alternative device page map or %NULL if default memmap is used @@ -504,16 +507,14 @@ static void __remove_section(struct zone *zone, unsigned long pfn,* sure that pages are marked reserved and zones are adjust properly by * calling offline_pages(). */ -void __remove_pages (struct zone *zone,unsigned long pfn, -unsigned long nr_pages,struct vmem_altmap *altmap) +void __remove_pages (unsigned long pfn,unsigned long nr_pages, +struct vmem_altmap *altmap) {unsigned long map_offset =0 ;unsigned long nr, start_sec, end_sec; map_offset =vmem_altmap_offset (altmap); -clear_zone_contiguous (zone); -if (check_pfn_span (pfn, nr_pages,"remove" ))return ;@@ -525,13 +526,11 @@ void __remove_pages(struct zone *zone, unsigned long pfn,cond_resched (); pfns =min (nr_pages,PAGES_PER_SECTION - (pfn & ~PAGE_SECTION_MASK )); -__remove_section (zone, pfn, pfns, map_offset, altmap); +__remove_section (pfn, pfns, map_offset, altmap); pfn += pfns; nr_pages -= pfns; map_offset =0 ; } - -set_zone_contiguous (zone); }int set_online_page_callback (online_page_callback_t callback)@@ -859,6 +858,7 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages, int online_typ (unsigned long long ) pfn <<PAGE_SHIFT , (((unsigned long long) pfn + nr_pages) << PAGE_SHIFT) - 1);memory_notify (MEM_CANCEL_ONLINE , &arg); +remove_pfn_range_from_zone (zone, pfn, nr_pages);mem_hotplug_done ();return ret; }@@ -1605,6 +1605,7 @@ static int __ref __offline_pages(unsigned long start_pfn,writeback_set_ratelimit ();memory_notify (MEM_OFFLINE , &arg); +remove_pfn_range_from_zone (zone, start_pfn, nr_pages);mem_hotplug_done ();return 0 ;mm/memremap.c +1
-1@@ -140,7 +140,7 @@ void memunmap_pages(struct dev_pagemap *pgmap)mem_hotplug_begin ();if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) { -__remove_pages (page_zone (first_page),PHYS_PFN (res->start), +__remove_pages (PHYS_PFN (res->start),PHYS_PFN (resource_size (res)),NULL ); }else { arch_remove_memory(nid, res->start, resource_size(res),Signature · 1 line
2.21.0
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v6 05/10] mm/memory_hotplug: Shrink zones when offlining memory
Quoted 9 lines · Oscar Salvador
Quoted 2 lines · David Hildenbrand
Fixes d0dc12e86b31 ("mm/memory_hotplug: optimize memory hotplug")Signed-off-by David Hildenbrand <[email protected]>I did not see anything wrong with the taken approach, and makes sense to me. The only thing that puzzles me is we seem to not balance
spanned_pagesforZONE_DEVICEanymore.memremap_pages()increments them viamove_pfn_range_to_zone, but we skipZONE_DEVICEinremove_pfn_range_from_zone.
Yes, documented e.g., in
commit 7ce700bf11b5e2cb84e4352bbdf2123a7a239c84
Author: David Hildenbrand <[email protected]> Date: Thu Nov 21 17:53:56 2019 -0800
mm/memory_hotplug: don't access uninitialized memmaps inshrink_zone_span()
Needs some more thought - but is definitely not urgent (well, now it's at least no longer completely broken).
Quoted 8 lines
That is not really related to this patch, so I might be missing something, but it caught my eye while reviewing this.
Anyway, for this one:
Reviewed-by Oscar Salvador <[email protected]>
Thanks!
Quoted 2 lines
off-topic: I
__think__we really need to trim the CC list.
Yes we should :) - done.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v6 6/10 mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()
In reply to David HildenbrandLet's poison the pages similar to when adding new memory in sparse_add_section(). Also call remove_pfn_range_from_zone() from memunmap_pages(), so we can poison the memmap from there as well.
While at it, calculate the pfn in memunmap_pages() only once.
--- mm/memory_hotplug.c | 3 +++ mm/memremap.c | 2 ++ 2 files changed, 5 insertions(+)
mm/memory_hotplug.c +3 -0
@@ -464,6 +464,9 @@ void __ref remove_pfn_range_from_zone(struct zone *zone,struct pglist_data *pgdat = zone->zone_pgdat;unsigned long flags; +/* Poison struct pages because they are now uninitialized again. */ +page_init_poison (pfn_to_page (start_pfn),sizeof (struct page ) * nr_pages); +#ifdef CONFIG_ZONE_DEVICE /* * Zone shrinking code cannot properly deal with ZONE_DEVICE. So
mm/memremap.c +2 -0
@@ -139,6 +139,8 @@ void memunmap_pages(struct dev_pagemap *pgmap) nid =page_to_nid (first_page);mem_hotplug_begin (); +remove_pfn_range_from_zone (page_zone (first_page),PHYS_PFN (res->start), +PHYS_PFN (resource_size (res)));if (pgmap->type =MEMORY_DEVICE_PRIVATE ) {__remove_pages (PHYS_PFN (res->start),PHYS_PFN (resource_size (res)),NULL );
Signature · 1 line
2.21.0
Re: [PATCH v6 06/10] mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()
Quoted 5 lines · David Hildenbrand
Let's poison the pages similar to when adding new memory in
sparse_add_section(). Also callremove_pfn_range_from_zone()frommemunmap_pages(), so we can poison the memmap from there as well.While at it, calculate the pfn in
memunmap_pages()only once.
FWIW, this comment is stale and could be dropped :)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 06/10] mm/memory_hotplug: Poison memmap in remove_pfn_range_from_zone()
In reply to David HildenbrandQuoted 6 lines · David Hildenbrand
Let's poison the pages similar to when adding new memory in
sparse_add_section(). Also callremove_pfn_range_from_zone()frommemunmap_pages(), so we can poison the memmap from there as well.While at it, calculate the pfn in
memunmap_pages()only once.
Quoted 7 lines
Cc Andrew Morton <[email protected]>Cc David Hildenbrand <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Looks good to me, it is fine as long as we do not access those pages later on, and if my eyes did not lie to me, we have to proper checks (pfn_to_online_page) in place to avoid that, so:
Signature · 2 lines
Oscar Salvador SUSE L3
PATCH v6 7/10 mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn
In reply to David HildenbrandWith shrink_pgdat_span() out of the way, we now always have a valid zone.
--- mm/memory_hotplug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +2 -2
@@ -337,7 +337,7 @@ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone,if (unlikely (pfn_to_nid (start_pfn) != nid))continue ; -if (zone && zone !=page_zone (pfn_to_page (start_pfn))) +if (zone !=page_zone (pfn_to_page (start_pfn)))continue ;return start_pfn;
@@ -362,7 +362,7 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,if (unlikely (pfn_to_nid (pfn) != nid))continue ; -if (zone && zone !=page_zone (pfn_to_page (pfn))) +if (zone !=page_zone (pfn_to_page (pfn)))continue ;return pfn;
Signature · 1 line
2.21.0
Re: [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn
Quoted 11 lines · David Hildenbrand
With
shrink_pgdat_span()out of the way, we now always have a valid zone.Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 29 lines
--- mm/memory_hotplug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)mm/memory_hotplug.c +2
-2@@ -337,7 +337,7 @@ static unsigned long find_smallest_section_pfn(int nid, struct zone *zone,if (unlikely (pfn_to_nid (start_pfn) != nid))continue ; -if (zone && zone !=page_zone (pfn_to_page (start_pfn))) +if (zone !=page_zone (pfn_to_page (start_pfn)))continue ;return start_pfn;@@ -362,7 +362,7 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,if (unlikely (pfn_to_nid (pfn) != nid))continue ; -if (zone && zone !=page_zone (pfn_to_page (pfn))) +if (zone !=page_zone (pfn_to_page (pfn)))continue ;return pfn;Signature · 1 line
2.21.0
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn
In reply to David HildenbrandQuoted 11 lines · David Hildenbrand
With
shrink_pgdat_span()out of the way, we now always have a valid zone.Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn
Quoted 13 lines · Wei Yang
Quoted 11 lines · David Hildenbrand
With
shrink_pgdat_span()out of the way, we now always have a valid zone.Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 1 line
Reviewed-by Wei Yang <[email protected]>
Just FYI, the patches are now upstream, so the rb's can no longer be applied. (but we can send fixes if we find that something is broken ;) ). Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 07/10] mm/memory_hotplug: We always have a zone in find_(smallest|biggest)_section_pfn
Quoted 14 lines · David Hildenbrand
Quoted 13 lines · Wei Yang
Quoted 11 lines · David Hildenbrand
With
shrink_pgdat_span()out of the way, we now always have a valid zone.Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 6 lines
Quoted 1 line
Reviewed-by Wei Yang <[email protected]>Just FYI, the patches are now upstream, so the rb's can no longer be applied. (but we can send fixes if we find that something is broken ;) ). Thanks!
Thanks for reminding. :-)
Quoted 4 lines
Signature · 3 lines
Thanks, David / dhildenb
Signature · 2 lines
Wei Yang Help you, Help me
PATCH v6 8/10 mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
In reply to David HildenbrandIf we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)
mm/memory_hotplug.c +7 -27
@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/*
@@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } } - -/* -* The section is not biggest or smallest mem_section in the zone, it -* only creates a hole in the zone. So in this case, we need not -* change the zone. But perhaps, the zone has only hole data. Thus -* it check the zone has only hole or not. -*/ - pfn = zone_start_pfn; -for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_to_online_page (pfn))) -continue ; - -if (page_zone (pfn_to_page (pfn)) != zone) -continue ; - -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; - -/* If we find valid section, we have nothing to do */ -zone_span_writeunlock (zone); -return ; - } - -/* The zone has no valid section */ - zone->zone_start_pfn =0 ; - zone->spanned_pages =0 ;zone_span_writeunlock (zone); }
Signature · 1 line
2.21.0
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 4 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Quoted 8 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Heh, I have been here before.
I have to confess that when I wrote my version of this I was not really 100% about removing it, because hotplug was a sort of a "catchall" for all sort of weird and corner-cases configurations, but thinking more about it, I cannot think of any situation that would make this blow up.
Quoted 60 lines
--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +7
-27@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } } - -/* -* The section is not biggest or smallest mem_section in the zone, it -* only creates a hole in the zone. So in this case, we need not -* change the zone. But perhaps, the zone has only hole data. Thus -* it check the zone has only hole or not. -*/ - pfn = zone_start_pfn; -for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_to_online_page (pfn))) -continue ; - -if (page_zone (pfn_to_page (pfn)) != zone) -continue ; - -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; - -/* If we find valid section, we have nothing to do */ -zone_span_writeunlock (zone); -return ; - } - -/* The zone has no valid section */ - zone->zone_start_pfn =0 ; - zone->spanned_pages =0 ;zone_span_writeunlock (zone); }Signature · 1 line
2.21.0
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 14 lines · Oscar Salvador
Quoted 4 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Quoted 8 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 7 lines
Heh, I have been here before.
I have to confess that when I wrote my version of this I was not really 100% about removing it, because hotplug was a sort of a "catchall" for all sort of weird and corner-cases configurations, but thinking more about it, I cannot think of any situation that would make this blow up.Reviewed-by Oscar Salvador <[email protected]>
Thanks for your review Oscar!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
In reply to David HildenbrandQuoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/*
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0@@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;
Thinking in which case (zone_start_pfn != start_pfn) and it comes here.
Quoted 36 lines
mm/memory_hotplug.c +1
-27+ } } - -/* -* The section is not biggest or smallest mem_section in the zone, it -* only creates a hole in the zone. So in this case, we need not -* change the zone. But perhaps, the zone has only hole data. Thus -* it check the zone has only hole or not. -*/ - pfn = zone_start_pfn; -for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_to_online_page (pfn))) -continue ; - -if (page_zone (pfn_to_page (pfn)) != zone) -continue ; - -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; - -/* If we find valid section, we have nothing to do */ -zone_span_writeunlock (zone); -return ; - } - -/* The zone has no valid section */ - zone->zone_start_pfn =0 ; - zone->spanned_pages =0 ;zone_span_writeunlock (zone); }Signature · 1 line
2.21.0
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,
Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.
Could only happen in case the zone_start_pfn would have been "out of the zone already". If you ask me: unlikely :)
This change at least maintains the same result as before (where the all-holes check would have caught it).
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 42 lines · David Hildenbrand
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.
Quoted 2 lines
Could only happen in case the
zone_start_pfnwould have been "out of the zone already". If you ask me: unlikely :)
Yeah, I also think it's unlikely to come here.
The 'if (zone_start_pfn = start_pfn)' checking also covers the case (zone_start_pfn = start_pfn && zone_end_pfn = end_pfn). So this zone_start_pfn/spanned_pages resetting can be removed to avoid confusion.
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 46 lines · Baoquan He
Quoted 42 lines · David Hildenbrand
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn == end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.Quoted 2 lines
Could only happen in case the
zone_start_pfnwould have been "out of the zone already". If you ask me: unlikely :)
Quoted 6 lines
Yeah, I also think it's unlikely to come here.
The 'if (
zone_start_pfn==start_pfn)' checking also covers the case (zone_start_pfn==start_pfn&&zone_end_pfn==end_pfn). So this zone_start_pfn/spanned_pages resetting can be removed to avoid confusion.
At least I would find it more confusing without it (or want a comment explaining why this does not have to be handled and why the !pfn case is not possible).
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 49 lines · David Hildenbrand
Quoted 48 lines · Baoquan He
Quoted 42 lines · David Hildenbrand
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.Quoted 2 lines
Could only happen in case the
zone_start_pfnwould have been "out of the zone already". If you ask me: unlikely :)Yeah, I also think it's unlikely to come here.
Quoted 8 lines
Quoted 4 lines
The 'if (
zone_start_pfn=start_pfn)' checking also covers the case (zone_start_pfn=start_pfn&&zone_end_pfn=end_pfn). So this zone_start_pfn/spanned_pages resetting can be removed to avoid confusion.At least I would find it more confusing without it (or want a comment explaining why this does not have to be handled and why the !pfn case is not possible).
I don't get why being w/o it will be more confusing, but it's OK since it doesn't impact anything.
Quoted 3 lines
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Yeah, noticed you told Wei the status in another patch thread, I am fine with it, just leave it to you to decide. Thanks.
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 55 lines · Baoquan He
Quoted 54 lines · David Hildenbrand
Quoted 52 lines · Baoquan He
Quoted 42 lines · David Hildenbrand
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.Quoted 2 lines
Could only happen in case the
zone_start_pfnwould have been "out of the zone already". If you ask me: unlikely :)Yeah, I also think it's unlikely to come here.
The 'if (
zone_start_pfn=start_pfn)' checking also covers the case (zone_start_pfn=start_pfn&&zone_end_pfn=end_pfn). So this zone_start_pfn/spanned_pages resetting can be removed to avoid confusion.
Quoted 6 lines
Quoted 3 lines
At least I would find it more confusing without it (or want a comment explaining why this does not have to be handled and why the !pfn case is not possible).
I don't get why being w/o it will be more confusing, but it's OK since it doesn't impact anything.
Because we could actually BUG_ON(!pfn) here, right? Only having a "if (pfn)" leaves the reader wondering "why is the other case not handled".
Quoted 7 lines
Quoted 4 lines
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Yeah, noticed you told Wei the status in another patch thread, I am fine with it, just leave it to you to decide. Thanks.
I am fairly busy right now. Can you send a patch (double-checking and making this eventually unconditional?). Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 68 lines · David Hildenbrand
Quoted 55 lines · Baoquan He
Quoted 54 lines · David Hildenbrand
Quoted 52 lines · Baoquan He
Quoted 42 lines · David Hildenbrand
Quoted 32 lines · Baoquan He
Quoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 34 +++++++--------------------------- 1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 413
Quoted 6 lines
mm/memory_hotplug.c +3
-0start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ;Thinking in which case (
zone_start_pfn!=start_pfn) and it comes here.Quoted 2 lines
Could only happen in case the
zone_start_pfnwould have been "out of the zone already". If you ask me: unlikely :)Yeah, I also think it's unlikely to come here.
The 'if (
zone_start_pfn=start_pfn)' checking also covers the case (zone_start_pfn=start_pfn&&zone_end_pfn=end_pfn). So this zone_start_pfn/spanned_pages resetting can be removed to avoid confusion.Quoted 6 lines
Quoted 3 lines
At least I would find it more confusing without it (or want a comment explaining why this does not have to be handled and why the !pfn case is not possible).
I don't get why being w/o it will be more confusing, but it's OK since it doesn't impact anything.
Because we could actually
BUG_ON(!pfn)here, right? Only having a "if (pfn)" leaves the reader wondering "why is the other case not handled".Quoted 2 lines
Quoted 2 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Yeah, noticed you told Wei the status in another patch thread, I am fine with it, just leave it to you to decide. Thanks.
I am fairly busy right now. Can you send a patch (double-checking and making this eventually unconditional?). Thanks!
Understood, sorry about the noise, David. I will think about this.
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Yeah, noticed you told Wei the status in another patch thread, I am fine with it, just leave it to you to decide. Thanks.
I am fairly busy right now. Can you send a patch (double-checking and making this eventually unconditional?). Thanks!
Understood, sorry about the noise, David. I will think about this.
No need to excuse, really, I'm very happy about review feedback!
The review of this series happened fairly late. Bad, because it's not perfect, but good, because no serious stuff was found (so far :) ). If you also don't have time to look into this, I can put it onto my todo list, just let me know.
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 6 lines · David Hildenbrand
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
Anyhow, that patch is already upstream and I don't consider this high priority. Thanks :)
Yeah, noticed you told Wei the status in another patch thread, I am fine with it, just leave it to you to decide. Thanks.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I am fairly busy right now. Can you send a patch (double-checking and making this eventually unconditional?). Thanks!
Understood, sorry about the noise, David. I will think about this.
No need to excuse, really, I'm very happy about review feedback!
Glad to hear it, thanks.
Quoted 4 lines
The review of this series happened fairly late. Bad, because it's not perfect, but good, because no serious stuff was found (so far :) ). If you also don't have time to look into this, I can put it onto my todo list, just let me know.
Both is OK to me, as long as thing is clear to us. I will discuss with Wei Yang for now. You can post patch anytime if you make one.
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
In reply to David HildenbrandQuoted 31 lines · David Hildenbrand
If we have holes, the holes will automatically get detected and removed once we remove the next bigger/smaller section. The extra checks can go.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc David Hildenbrand <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
mm/memory_hotplug.c| 34 +++++++---------------------------1 file changed, 7 insertions(+), 27 deletions(-)mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn == end_pfn) {/*
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 8 lines
mm/memory_hotplug.c +4
-0@@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn) zone->spanned_pages = pfn - zone_start_pfn +1 ; +else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }
If it is me, I would like to take out these two similar logic out.
For example:
if () {
} else if () {
} else {
goto out;
}
/* The zone has no valid section */
if (!pfn) {
zone->zone_start_pfn = 0;
zone->spanned_pages = 0;
}out:
zone_span_writeunlock(zone);
Well, this is just my personal taste :-)
Quoted 32 lines
mm/memory_hotplug.c +0
-27- -/* -* The section is not biggest or smallest mem_section in the zone, it -* only creates a hole in the zone. So in this case, we need not -* change the zone. But perhaps, the zone has only hole data. Thus -* it check the zone has only hole or not. -*/ - pfn = zone_start_pfn; -for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_to_online_page (pfn))) -continue ; - -if (page_zone (pfn_to_page (pfn)) != zone) -continue ; - -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; - -/* If we find valid section, we have nothing to do */ -zone_span_writeunlock (zone); -return ; - } - -/* The zone has no valid section */ - zone->zone_start_pfn =0 ; - zone->spanned_pages =0 ;zone_span_writeunlock (zone); }Signature · 1 line
2.21.0
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)
Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 7 lines
For example:
if () { } else if () { } else { goto out;
Here the last else is unnecessary, right?
Quoted 3 lines
}
Like this, I believe both David and I will be satisfactory. Even though I still think his 2nd resetting is not needed :-)
Quoted 48 lines
/* The zone has no valid section */ if (!pfn) { zone->zone_start_pfn = 0; zone->spanned_pages = 0; }out:
zone_span_writeunlock(zone);Well, this is just my personal taste :-)
Quoted 32 lines
mm/memory_hotplug.c +0
-27- -/* -* The section is not biggest or smallest mem_section in the zone, it -* only creates a hole in the zone. So in this case, we need not -* change the zone. But perhaps, the zone has only hole data. Thus -* it check the zone has only hole or not. -*/ - pfn = zone_start_pfn; -for (; pfn < zone_end_pfn; pfn +=PAGES_PER_SUBSECTION ) { -if (unlikely (!pfn_to_online_page (pfn))) -continue ; - -if (page_zone (pfn_to_page (pfn)) != zone) -continue ; - -/* Skip range to be removed */ -if (pfn >= start_pfn && pfn < end_pfn) -continue ; - -/* If we find valid section, we have nothing to do */ -zone_span_writeunlock (zone); -return ; - } - -/* The zone has no valid section */ - zone->zone_start_pfn =0 ; - zone->spanned_pages =0 ;zone_span_writeunlock (zone); }Signature · 1 line
2.21.0Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 31 lines · Baoquan He
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn == end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
For example:
if () { } else if () { } else { goto out;Here the last else is unnecessary, right?
I am afraid not.
If the range is not the first or last, we would leave pfn not initialized.
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 38 lines · Wei Yang
Quoted 37 lines · Baoquan He
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 7 lines
For example:
if () { } else if () { } else { goto out;
Quoted 6 lines
Quoted 2 lines
Here the last else is unnecessary, right?
I am afraid not.
If the range is not the first or last, we would leave pfn not initialized.
Ah, you are right. I forgot that one. Then pfn can be assigned the zone_start_pfn as the old code. Then the following logic is the same as the original code, find_smallest_section_pfn()/find_biggest_section_pfn() have done the iteration the old for loop was doing.
unsigned long pfn = zone_start_pfn;
if () {
} else if () {
}
/* The zone has no valid section */
if (!pfn) {
zone->zone_start_pfn = 0;
zone->spanned_pages = 0;
}Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 51 lines · Baoquan He
Quoted 38 lines · Wei Yang
Quoted 37 lines · Baoquan He
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn == end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 7 lines
For example:
if () { } else if () { } else { goto out;Quoted 6 lines
Quoted 2 lines
Here the last else is unnecessary, right?
I am afraid not.
If the range is not the first or last, we would leave pfn not initialized.
Ah, you are right. I forgot that one. Then pfn can be assigned the
zone_start_pfnas the old code. Then the following logic is the same as the original code, find_smallest_section_pfn()/find_biggest_section_pfn() have done the iteration the old for loop was doing.
Quoted 10 lines
unsigned long pfn = zone_start_pfn;
if () {
} else if () {
}
/* The zone has no valid section */
if (!pfn) {
zone->zone_start_pfn = 0;
zone->spanned_pages = 0;
}This one look better :-)
Thanks
Signature · 2 lines
Wei Yang Help you, Help me
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 57 lines · Wei Yang
Quoted 56 lines · Baoquan He
Quoted 38 lines · Wei Yang
Quoted 37 lines · Baoquan He
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn = end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 7 lines
For example:
if () { } else if () { } else { goto out;Quoted 6 lines
Quoted 2 lines
Here the last else is unnecessary, right?
I am afraid not.
If the range is not the first or last, we would leave pfn not initialized.
Ah, you are right. I forgot that one. Then pfn can be assigned the
zone_start_pfnas the old code. Then the following logic is the same as the original code, find_smallest_section_pfn()/find_biggest_section_pfn() have done the iteration the old for loop was doing.unsigned long pfn = zone_start_pfn; if () { } else if () { }
Quoted 7 lines
Quoted 5 lines
/* The zone has no valid section */ if (!pfn) { zone->zone_start_pfn = 0; zone->spanned_pages = 0; }This one look better :-)
Thanks for your confirmation, I will make one patch like this and post.
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 64 lines · Baoquan He
Quoted 63 lines · Wei Yang
Quoted 61 lines · Baoquan He
Quoted 38 lines · Wei Yang
Quoted 37 lines · Baoquan He
Hi Wei Yang,
Quoted 17 lines · Wei Yang
Quoted 17 lines
mm/memory_hotplug.c +3
-0@@ -393,6 +393,9 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,if (pfn) { zone->zone_start_pfn = pfn; zone->spanned_pages = zone_end_pfn - pfn; + }else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; } }else if (zone_end_pfn == end_pfn) {/* @@ -405,34 +408,11 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, start_pfn);if (pfn)Quoted 8 lines
mm/memory_hotplug.c static void shrink_zone_span(struct zone *zone, unsigned long start_pfn, · line 415
Quoted 6 lines
mm/memory_hotplug.c +4
-0zone->spanned_pages = pfn - zone_start_pfn +1 ; + else { + zone->zone_start_pfn =0 ; + zone->spanned_pages =0 ; + } }If it is me, I would like to take out these two similar logic out.
I also like this style.
Quoted 7 lines
For example:
if () { } else if () { } else { goto out;Quoted 6 lines
Quoted 2 lines
Here the last else is unnecessary, right?
I am afraid not.
If the range is not the first or last, we would leave pfn not initialized.
Ah, you are right. I forgot that one. Then pfn can be assigned the
zone_start_pfnas the old code. Then the following logic is the same as the original code, find_smallest_section_pfn()/find_biggest_section_pfn() have done the iteration the old for loop was doing.unsigned long pfn = zone_start_pfn; if () { } else if () { } /* The zone has no valid section */ if (!pfn) { zone->zone_start_pfn = 0; zone->spanned_pages = 0; }
Quoted 3 lines
Quoted 1 line
This one look better :-)
Thanks for your confirmation, I will make one patch like this and post.
Sure :-)
Signature · 2 lines
Wei Yang Help you, Help me
RE: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
In reply to Wei YangFrom: Wei Yang
Quoted 1 line
Sent: 05 February 2020 09:59
...
Quoted 9 lines
If it is me, I would like to take out these two similar logic out.
For example:
if () { } else if () { } else { goto out; }
I'm pretty sure the kernel layout rules disallow 'else if'. It is also pretty horrid unless the conditionals are all related (so it is almost a switch statement).
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Re: [PATCH v6 08/10] mm/memory_hotplug: Don't check for "all holes" in shrink_zone_span()
Quoted 7 lines · David Laight
From: Wei Yang
Quoted 1 line
Sent: 05 February 2020 09:59
...
Quoted 4 lines
If it is me, I would like to take out these two similar logic out.
For example:
Quoted 7 lines
Quoted 5 lines
if () { } else if () { } else { goto out; }I'm pretty sure the kernel layout rules disallow 'else if'.
Huh?
git grep "else if" | wc -l
49336
I'm afraid I don't get what you mean :)
Signature · 3 lines
Thanks, David / dhildenb
PATCH v6 9/10 mm/memory_hotplug: Drop local variables in shrink_zone_span()
In reply to David HildenbrandGet rid of the unnecessary local variables.
--- mm/memory_hotplug.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
mm/memory_hotplug.c +6 -9
@@ -374,14 +374,11 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,static void shrink_zone_span (struct zone *zone,unsigned long start_pfn,unsigned long end_pfn) { -unsigned long zone_start_pfn = zone->zone_start_pfn; -unsigned long z =zone_end_pfn (zone);/* zone_end_pfn namespace clash */ -unsigned long zone_end_pfn = z;unsigned long pfn;int nid =zone_to_nid (zone);zone_span_writelock (zone); -if (zone_start_pfn = start_pfn) { +if (zone->zone_start_pfn = start_pfn) {/* * If the section is smallest section in the zone, it need * shrink zone->zone_start_pfn and zone->zone_spanned_pages.
@@ -389,25 +386,25 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,* for shrinking zone. */ pfn =find_smallest_section_pfn (nid, zone, end_pfn, - zone_end_pfn); +zone_end_pfn (zone));if (pfn) { + zone->spanned_pages =zone_end_pfn (zone) - pfn; zone->zone_start_pfn = pfn; - zone->spanned_pages = zone_end_pfn - pfn; }else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ; } - }else if (zone_end_pfn = end_pfn) { + }else if (zone_end_pfn (zone) = end_pfn) {/* * If the section is biggest section in the zone, it need * shrink zone->spanned_pages. * In this case, we find second biggest valid mem_section for * shrinking zone. */ - pfn =find_biggest_section_pfn (nid, zone, zone_start_pfn, + pfn =find_biggest_section_pfn (nid, zone, zone->zone_start_pfn, start_pfn);if (pfn) - zone->spanned_pages = pfn - zone_start_pfn +1 ; + zone->spanned_pages = pfn - zone->zone_start_pfn +1 ;else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ;
Signature · 1 line
2.21.0
Re: [PATCH v6 09/10] mm/memory_hotplug: Drop local variables in shrink_zone_span()
Quoted 19 lines · David Hildenbrand
Get rid of the unnecessary local variables.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)mm/memory_hotplug.c +0
-0
mm/memory_hotplug.c static unsigned long find_biggest_section_pfn(int nid, struct zone *zone, · line 378
Quoted 8 lines
mm/memory_hotplug.c +0
-3@@ -374,14 +374,11 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,static void shrink_zone_span (struct zone *zone,unsigned long start_pfn,unsigned long end_pfn) { -unsigned long zone_start_pfn = zone->zone_start_pfn; -unsigned long z =zone_end_pfn (zone);/* zone_end_pfn namespace clash */ -unsigned long zone_end_pfn = z;unsigned long pfn;int nid =zone_to_nid (zone);
We could also remove the nid, right?
AFAICS, the nid is only used in find_{smallest/biggest}_section_pfn so we could place there as well.
Anyway, nothing to nit-pick about:
Quoted 41 lines
mm/memory_hotplug.c +6
-6zone_span_writelock (zone); -if (zone_start_pfn = start_pfn) { +if (zone->zone_start_pfn = start_pfn) {/* * If the section is smallest section in the zone, it need * shrink zone->zone_start_pfn and zone->zone_spanned_pages. @@ -389,25 +386,25 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,* for shrinking zone. */ pfn =find_smallest_section_pfn (nid, zone, end_pfn, - zone_end_pfn); +zone_end_pfn (zone));if (pfn) { + zone->spanned_pages =zone_end_pfn (zone) - pfn; zone->zone_start_pfn = pfn; - zone->spanned_pages = zone_end_pfn - pfn; }else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ; } - }else if (zone_end_pfn = end_pfn) { + }else if (zone_end_pfn (zone) = end_pfn) {/* * If the section is biggest section in the zone, it need * shrink zone->spanned_pages. * In this case, we find second biggest valid mem_section for * shrinking zone. */ - pfn =find_biggest_section_pfn (nid, zone, zone_start_pfn, + pfn =find_biggest_section_pfn (nid, zone, zone->zone_start_pfn, start_pfn);if (pfn) - zone->spanned_pages = pfn - zone_start_pfn +1 ; + zone->spanned_pages = pfn - zone->zone_start_pfn +1 ;else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ;Signature · 1 line
2.21.0
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v6 09/10] mm/memory_hotplug: Drop local variables in shrink_zone_span()
Quoted 24 lines · Oscar Salvador
Quoted 23 lines · David Hildenbrand
Get rid of the unnecessary local variables.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- mm/memory_hotplug.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)mm/memory_hotplug.c +0
-1@@ -374,14 +374,11 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,static void shrink_zone_span (struct zone *zone,unsigned long start_pfn,unsigned long end_pfn) { -unsigned long zone_start_pfn = zone->zone_start_pfn;
Quoted 8 lines
mm/memory_hotplug.c static unsigned long find_biggest_section_pfn(int nid, struct zone *zone, · line 378
Quoted 4 lines
mm/memory_hotplug.c +0
-2-unsigned long z =zone_end_pfn (zone);/* zone_end_pfn namespace clash */ -unsigned long zone_end_pfn = z;unsigned long pfn;int nid =zone_to_nid (zone);We could also remove the nid, right?
AFAICS, the nid is only used in find_{smallest/biggest}_section_pfn so we could place there as well.
I remember sending a patch on this (which was acked, but not picked up yet)...
oh, there it is :)
https://lore.kernel.org/linux-mm/[email protected]/
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 09/10] mm/memory_hotplug: Drop local variables in shrink_zone_span()
In reply to David HildenbrandQuoted 10 lines · David Hildenbrand
Get rid of the unnecessary local variables.
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc David Hildenbrand <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Looks reasonable.
Quoted 57 lines
---
mm/memory_hotplug.c| 15 ++++++---------1 file changed, 6 insertions(+), 9 deletions(-)mm/memory_hotplug.c +6
-9@@ -374,14 +374,11 @@ static unsigned long find_biggest_section_pfn(int nid, struct zone *zone,static void shrink_zone_span (struct zone *zone,unsigned long start_pfn,unsigned long end_pfn) { -unsigned long zone_start_pfn = zone->zone_start_pfn; -unsigned long z =zone_end_pfn (zone);/* zone_end_pfn namespace clash */ -unsigned long zone_end_pfn = z;unsigned long pfn;int nid =zone_to_nid (zone);zone_span_writelock (zone); -if (zone_start_pfn == start_pfn) { +if (zone->zone_start_pfn == start_pfn) {/* * If the section is smallest section in the zone, it need * shrink zone->zone_start_pfn and zone->zone_spanned_pages. @@ -389,25 +386,25 @@ static void shrink_zone_span(struct zone *zone, unsigned long start_pfn,* for shrinking zone. */ pfn =find_smallest_section_pfn (nid, zone, end_pfn, - zone_end_pfn); +zone_end_pfn (zone));if (pfn) { + zone->spanned_pages =zone_end_pfn (zone) - pfn; zone->zone_start_pfn = pfn; - zone->spanned_pages = zone_end_pfn - pfn; }else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ; } - }else if (zone_end_pfn == end_pfn) { + }else if (zone_end_pfn (zone) == end_pfn) {/* * If the section is biggest section in the zone, it need * shrink zone->spanned_pages. * In this case, we find second biggest valid mem_section for * shrinking zone. */ - pfn =find_biggest_section_pfn (nid, zone, zone_start_pfn, + pfn =find_biggest_section_pfn (nid, zone, zone->zone_start_pfn, start_pfn);if (pfn) - zone->spanned_pages = pfn - zone_start_pfn +1 ; + zone->spanned_pages = pfn - zone->zone_start_pfn +1 ;else { zone->zone_start_pfn =0 ; zone->spanned_pages =0 ;Signature · 1 line
2.21.0
Signature · 2 lines
Wei Yang Help you, Help me
PATCH v6 10/10 mm/memory_hotplug: Cleanup __remove_pages()
In reply to David HildenbrandLet's drop the basically unused section stuff and simplify.
Also, let's use a shorter variant to calculate the number of pages to the next section boundary.
--- mm/memory_hotplug.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-)
mm/memory_hotplug.c +6 -11
@@ -490,25 +490,20 @@ static void __remove_section(unsigned long pfn, unsigned long nr_pages,void __remove_pages (unsigned long pfn,unsigned long nr_pages,struct vmem_altmap *altmap) { +const unsigned long end_pfn = pfn + nr_pages; +unsigned long cur_nr_pages;unsigned long map_offset =0 ; -unsigned long nr, start_sec, end_sec; map_offset =vmem_altmap_offset (altmap);if (check_pfn_span (pfn, nr_pages,"remove" ))return ; - start_sec =pfn_to_section_nr (pfn); - end_sec =pfn_to_section_nr (pfn + nr_pages -1 ); -for (nr = start_sec; nr <= end_sec; nr++) { -unsigned long pfns; - +for (; pfn < end_pfn; pfn += cur_nr_pages) {cond_resched (); - pfns =min (nr_pages,PAGES_PER_SECTION - - (pfn & ~PAGE_SECTION_MASK )); -__remove_section (pfn, pfns, map_offset, altmap); - pfn += pfns; - nr_pages -= pfns; +/* Select all remaining pages up to the next section boundary */ + cur_nr_pages =min (end_pfn - pfn, -(pfn |PAGE_SECTION_MASK )); +__remove_section (pfn, cur_nr_pages, map_offset, altmap); map_offset =0 ; } }
Signature · 1 line
2.21.0
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 5 lines · David Hildenbrand
Let's drop the basically unused section stuff and simplify.
Also, let's use a shorter variant to calculate the number of pages to the next section boundary.
Quoted 7 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
I have to confess that it took me while to wrap around my head with the new min() change, but looks ok:
Quoted 44 lines
--- mm/memory_hotplug.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-)mm/memory_hotplug.c +6
-11@@ -490,25 +490,20 @@ static void __remove_section(unsigned long pfn, unsigned long nr_pages,void __remove_pages (unsigned long pfn,unsigned long nr_pages,struct vmem_altmap *altmap) { +const unsigned long end_pfn = pfn + nr_pages; +unsigned long cur_nr_pages;unsigned long map_offset =0 ; -unsigned long nr, start_sec, end_sec; map_offset =vmem_altmap_offset (altmap);if (check_pfn_span (pfn, nr_pages,"remove" ))return ; - start_sec =pfn_to_section_nr (pfn); - end_sec =pfn_to_section_nr (pfn + nr_pages -1 ); -for (nr = start_sec; nr <= end_sec; nr++) { -unsigned long pfns; - +for (; pfn < end_pfn; pfn += cur_nr_pages) {cond_resched (); - pfns =min (nr_pages,PAGES_PER_SECTION - - (pfn & ~PAGE_SECTION_MASK )); -__remove_section (pfn, pfns, map_offset, altmap); - pfn += pfns; - nr_pages -= pfns; +/* Select all remaining pages up to the next section boundary */ + cur_nr_pages =min (end_pfn - pfn, -(pfn |PAGE_SECTION_MASK )); +__remove_section (pfn, cur_nr_pages, map_offset, altmap); map_offset =0 ; } }Signature · 1 line
2.21.0
Signature · 2 lines
Oscar Salvador SUSE L3
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 14 lines · Oscar Salvador
Quoted 5 lines · David Hildenbrand
Let's drop the basically unused section stuff and simplify.
Also, let's use a shorter variant to calculate the number of pages to the next section boundary.
Quoted 7 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 2 lines
I have to confess that it took me while to wrap around my head with the new
min()change, but looks ok:
It's a pattern commonly used in compilers and emulators to calculate the number of bytes to the next block/alignment. (we're missing a macro (like we have ALIGN_UP/IS_ALIGNED) for that - but it's hard to come up with a good name (e.g., SIZE_TO_NEXT_ALIGN) .
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 8 lines · David Hildenbrand
Quoted 2 lines · Oscar Salvador
I have to confess that it took me while to wrap around my head with the new
min()change, but looks ok:It's a pattern commonly used in compilers and emulators to calculate the number of bytes to the next block/alignment. (we're missing a macro (like we have ALIGN_UP/IS_ALIGNED) for that - but it's hard to come up with a good name (e.g.,
SIZE_TO_NEXT_ALIGN) .
You can just write the easy to understand
... ALIGN_UP(x) - x ...
which is better *without* having a separate name. Does that not generate good machine code for you?
Segher
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 5 lines · Segher Boessenkool
Quoted 4 lines · David Hildenbrand
Quoted 2 lines · Oscar Salvador
I have to confess that it took me while to wrap around my head with the new
min()change, but looks ok:
Quoted 8 lines
Quoted 4 lines
It's a pattern commonly used in compilers and emulators to calculate the number of bytes to the next block/alignment. (we're missing a macro (like we have ALIGN_UP/IS_ALIGNED) for that - but it's hard to come up with a good name (e.g.,
SIZE_TO_NEXT_ALIGN) .You can just write the easy to understand
...
ALIGN_UP(x)- x ...
you mean
ALIGN_UP(x, PAGES_PER_SECTION) - x
but ...
Quoted 3 lines
which is better *without* having a separate name. Does that not generate good machine code for you?
1. There is no ALIGN_UP. "SECTION_ALIGN_UP(x) - x" would be possible
2. It would be wrong if x is already aligned.
e.g., let's use 4096 for simplicity as we all know that value by heart (for both x and the block size).
a) -(4096 | -4096) -> 4096
b) #define ALIGN_UP(x, a) ((x + a - 1) & -(a))
ALIGN_UP(4096, 4096) - 4096 -> 0
Not as easy as it seems ...
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 13 lines · David Hildenbrand
Quoted 9 lines · Segher Boessenkool
Quoted 4 lines · David Hildenbrand
It's a pattern commonly used in compilers and emulators to calculate the number of bytes to the next block/alignment. (we're missing a macro (like we have ALIGN_UP/IS_ALIGNED) for that - but it's hard to come up with a good name (e.g.,
SIZE_TO_NEXT_ALIGN) .You can just write the easy to understand
...
ALIGN_UP(x)- x ...you mean
Quoted 8 lines
ALIGN_UP(x, PAGES_PER_SECTION)- xbut ...
Quoted 2 lines
which is better *without* having a separate name. Does that not generate good machine code for you?
1. There is no
ALIGN_UP. "SECTION_ALIGN_UP(x)- x" would be possible
Erm, you started it ;-)
Quoted 5 lines
2. It would be wrong if x is already aligned.
e.g., let's use 4096 for simplicity as we all know that value by heart (for both x and the block size).
Quoted 7 lines
a) -(4096 | -4096) -> 4096
b) #define
ALIGN_UP(x, a)((x + a - 1) & -(a))
ALIGN_UP(4096, 4096)- 4096 -> 0Not as easy as it seems ...
If you always want to return a number >= 1, it it simply
ALIGN_UP(x + 1) - x
(and replace 1 by any other minimum size required). This *also* is easy to read, without having to have any details (and quirks :-/ ) of those utility functions memorised.
Segher
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 16 lines · Segher Boessenkool
Quoted 15 lines · David Hildenbrand
Quoted 9 lines · Segher Boessenkool
Quoted 4 lines · David Hildenbrand
It's a pattern commonly used in compilers and emulators to calculate the number of bytes to the next block/alignment. (we're missing a macro (like we have ALIGN_UP/IS_ALIGNED) for that - but it's hard to come up with a good name (e.g.,
SIZE_TO_NEXT_ALIGN) .You can just write the easy to understand
...
ALIGN_UP(x)- x ...you mean
ALIGN_UP(x, PAGES_PER_SECTION)- x
Quoted 8 lines
Quoted 6 lines
but ...
Quoted 2 lines
which is better *without* having a separate name. Does that not generate good machine code for you?
1. There is no
ALIGN_UP. "SECTION_ALIGN_UP(x)- x" would be possibleErm, you started it ;-)
Yeah, I was thinking in the wrong code base :)
Quoted 8 lines
Quoted 8 lines
2. It would be wrong if x is already aligned.
e.g., let's use 4096 for simplicity as we all know that value by heart (for both x and the block size).
a) -(4096 | -4096) -> 4096
Quoted 8 lines
Quoted 5 lines
b) #define
ALIGN_UP(x, a)((x + a - 1) & -(a))
ALIGN_UP(4096, 4096)- 4096 -> 0Not as easy as it seems ...
If you always want to return a number >= 1, it it simply
ALIGN_UP(x + 1)- x
I'm sorry to have to correct you again for some corner cases:
ALIGN_UP(1, 4096) - 4096 = 0
Again, not as easy as it seems ...
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 6 lines
I'm sorry to have to correct you again for some corner cases:
ALIGN_UP(1, 4096)- 4096 = 0Again, not as easy as it seems ...
Eh, wait, I'm messing up things. Will double check :)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
Quoted 9 lines · David Hildenbrand
Quoted 6 lines
I'm sorry to have to correct you again for some corner cases:
ALIGN_UP(1, 4096)- 4096 = 0Again, not as easy as it seems ...
Eh, wait, I'm messing up things. Will double check :)
Yes, makes sense, will send a patch and cc you. Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v6 10/10] mm/memory_hotplug: Cleanup __remove_pages()
In reply to David HildenbrandQuoted 5 lines · David Hildenbrand
Let's drop the basically unused section stuff and simplify.
Also, let's use a shorter variant to calculate the number of pages to the next section boundary.
Quoted 7 lines
Cc Andrew Morton <[email protected]>Cc Oscar Salvador <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Dan Williams <[email protected]>Cc Wei Yang <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Finally understand the code.
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 84 lines · David Hildenbrand
This series fixes the access of uninitialized memmaps when shrinking zones/nodes and when removing memory. Also, it contains all fixes for crashes that can be triggered when removing certain namespace using
memunmap_pages()-ZONE_DEVICE, reported by Aneesh.We stop trying to shrink
ZONE_DEVICE, as it's buggy, fixing it would be more involved (we don't haveSECTION_IS_ONLINEas an indicator), and shrinking is only of limited use (set_zone_contiguous()cannot detect theZONE_DEVICEas contiguous).We continue shrinking !ZONE_DEVICE zones, however, I reduced the amount of code to a minimum. Shrinking is especially necessary to keep
zone->contiguousset where possible, especially, on memory unplug of DIMMs at zone boundaries.--------------------------------------------------------------------------
Zones are now properly shrunk when offlining memory blocks or when onlining failed. This allows to properly shrink zones on memory unplug even if the separate memory blocks of a DIMM were onlined to different zones or re-onlined to a different zone after offlining.
Example:
:/# cat /proc/zoneinfo Node 1, zone Movable spanned 0 present 0 managed 0 :/# echo "online_movable" > /sys/devices/system/memory/memory41/state :/# echo "online_movable" > /sys/devices/system/memory/memory43/state :/# cat /proc/zoneinfo Node 1, zone Movable spanned 98304 present 65536 managed 65536 :/# echo 0 > /sys/devices/system/memory/memory43/online :/# cat /proc/zoneinfo Node 1, zone Movable spanned 32768 present 32768 managed 32768 :/# echo 0 > /sys/devices/system/memory/memory41/online :/# cat /proc/zoneinfo Node 1, zone Movable spanned 0 present 0 managed 0--------------------------------------------------------------------------
I tested this with DIMMs on x86. I didn't test the
ZONE_DEVICEpart.v4 -> v6:
- "mm/memunmap: Don't access uninitialized memmap inmemunmap_pages()" -- Minimize code changes, rephrase subject and description
- "mm/memory_hotplug: Don't access uninitialized memmaps inshrink_zone_span()" -- Add ifdef to make it compile withoutZONE_DEVICEv4 -> v5:
- "mm/memory_hotplug: Don't access uninitialized memmaps inshrink_zone_span()" -- Add more details whyZONE_DEVICEis special
- Include two patches from Aneesh
-- "mm/memunmap: Use the correct start and end pfn when removing pages
from zone"
-- "mm/memmap_init: Update variable name inmemmap_init_zone"v3 -> v4:
- Drop "mm/memremap: Get rid ofmemmap_init_zone_device()"
-- As Alexander noticed, it was messy either way
- Drop "mm/memory_hotplug: Exit early in__remove_pages()on BUGs"
- Drop "mm: Exit early inset_zone_contiguous()if already contiguous"
- Drop "mm/memory_hotplug: Optimize zone shrinking code when checking for holes"
- Merged "mm/memory_hotplug: Remove pages from a zone before removing memory" and "mm/memory_hotplug: Remove zone parameter from__remove_pages()" into "mm/memory_hotplug: Shrink zones when offlining memory"
- Added "mm/memory_hotplug: Poison memmap inremove_pfn_range_from_zone()"
- Stop shrinkingZONE_DEVICE
- Reshuffle patches, moving all fixes to the front. Add Fixes: tags.
- Change subject/description of various patches
- Minor changes (too many to mention)
Quoted 5 lines
Cc Aneesh Kumar K.V <[email protected]>Cc Andrew Morton <[email protected]>Cc Dan Williams <[email protected]>Cc Michal Hocko <[email protected]>
So, patch #1-#4 are already upstream. The other patches have been in -next for quite a long time, and I (+other RH people) ran excessive tests on them.
Especially patch #5 is a BUG fix I want to see upstream rather sooner than later (last know "uninitialized memmap" access).
Andrew decided not to send these for 5.5 due to lack of ack/review - which is unfortunate, but the right thing to do.
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Signature · 3 lines
Thanks, David / dhildenb
Oscar Salvador
maintainerQuoted 2 lines · David Hildenbrand
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Hi,
I will be having a look at patch#5 shortly.
Thanks for the reminder
Signature · 2 lines
Oscar Salvador SUSE L3
Andrew Morton
maintainerQuoted 9 lines · On Tue, 3 Dec 2019 14:36:38 +0100 Oscar Salvador <[email protected]> wrote:
Quoted 2 lines · David Hildenbrand
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Hi,
I will be having a look at patch#5 shortly.
Thanks for the reminder
Things haven't improved a lot :(
mm-memmap_init-update-variable-name-in-memmap_init_zone.patch mm-memory_hotplug-poison-memmap-in-remove_pfn_range_from_zone.patch mm-memory_hotplug-we-always-have-a-zone-in-find_smallestbiggest_section_pfn.patch mm-memory_hotplug-dont-check-for-all-holes-in-shrink_zone_span.patch mm-memory_hotplug-drop-local-variables-in-shrink_zone_span.patch mm-memory_hotplug-cleanup-__remove_pages.patch
The first patch has reviews, the remainder are unloved.
Quoted 14 lines · Andrew Morton
Quoted 9 lines · On Tue, 3 Dec 2019 14:36:38 +0100 Oscar Salvador <[email protected]> wrote:
Quoted 2 lines · David Hildenbrand
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Hi,
I will be having a look at patch#5 shortly.
Thanks for the reminder
Things haven't improved a lot :(
Quoted 8 lines
mm-memmap_init-update-variable-name-in-memmap_init_zone.patch mm-memory_hotplug-poison-memmap-in-remove_pfn_range_from_zone.patch mm-memory_hotplug-we-always-have-a-zone-in-find_smallestbiggest_section_pfn.patch mm-memory_hotplug-dont-check-for-all-holes-in-shrink_zone_span.patch mm-memory_hotplug-drop-local-variables-in-shrink_zone_span.patch mm-memory_hotplug-cleanup-__remove_pages.patch
The first patch has reviews, the remainder are unloved.
Trying hard not to rant about the review mentality on this list, but I'm afraid I can't totally bite my tongue ... :)
Now, this is an uncomfortable situation for you and me. You have to ping people about review and patches are stuck in your tree. I have a growing list of patches that are somewhat considered "done", but well, not-upstream-at-all. I have patches that are long in RHEL and were properly tested, but could get dropped any time because -ENOREVIEW.
Our process nowadays seems to be, to only upstream what has an ACK/RB (fixes/features/cleanups). I can understand this is desirable (yet, I am not sure if this makes sense with the current take-and-not-give-back review mentality on this list).
Although it will make upstreaming stuff *even harder* and *even slower*, maybe we should start to only queue patches that have an ACK/RB, so they won't get blocked by this later on? At least that makes your life easier and people won't have to eventually follow up on patches that have been in linux-next for months.
Note: the result will be that many of my patches will still not get reviewed, won't get queued/upstreamed, I will continuously ping and resend, I will lose interest because I have better things to do, I will lose interest in our code quality, I will lose interest to review.
(side note: some people might actually enjoy me sending less cleanup patches, so this approach might be desirable for some ;) )
One alternative is to send patches upstream once they have been lying around in linux-next for $RANDOM number of months, because they obviously saw some testing and nobody started to yell at them once stumbling over them on linux-mm.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 22 lines · David Hildenbrand
Quoted 21 lines · Andrew Morton
Quoted 9 lines · On Tue, 3 Dec 2019 14:36:38 +0100 Oscar Salvador <[email protected]> wrote:
Quoted 2 lines · David Hildenbrand
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Hi,
I will be having a look at patch#5 shortly.
Thanks for the reminder
Things haven't improved a lot :(
mm-memmap_init-update-variable-name-in-memmap_init_zone.patch mm-memory_hotplug-poison-memmap-in-remove_pfn_range_from_zone.patch mm-memory_hotplug-we-always-have-a-zone-in-find_smallestbiggest_section_pfn.patch mm-memory_hotplug-dont-check-for-all-holes-in-shrink_zone_span.patch mm-memory_hotplug-drop-local-variables-in-shrink_zone_span.patch mm-memory_hotplug-cleanup-__remove_pages.patch
Quoted 4 lines
Quoted 1 line
The first patch has reviews, the remainder are unloved.
Trying hard not to rant about the review mentality on this list, but I'm afraid I can't totally bite my tongue ... :)
I am afraid this is less about mentality than the lack of man power. This is not a new problem. We have much more code producers than reviewers.
In this particular case the review is expected from me and I am sorry that my bandwith doesn't scale with the email traffic in my inbox. I do very much appreciate the amount of work you are doing in the hotplug area but we need more reviewers here.
Quoted 11 lines
Now, this is an uncomfortable situation for you and me. You have to ping people about review and patches are stuck in your tree. I have a growing list of patches that are somewhat considered "done", but well, not-upstream-at-all. I have patches that are long in RHEL and were properly tested, but could get dropped any time because
-ENOREVIEW.Our process nowadays seems to be, to only upstream what has an ACK/RB (fixes/features/cleanups). I can understand this is desirable (yet, I am not sure if this makes sense with the current take-and-not-give-back review mentality on this list).
Quoted 5 lines
Although it will make upstreaming stuff *even harder* and *even slower*, maybe we should start to only queue patches that have an ACK/RB, so they won't get blocked by this later on? At least that makes your life easier and people won't have to eventually follow up on patches that have been in linux-next for months.
I wouldn't mind if patched got merged to mmotm less pro-actively at all. People tend to care less to follow up on patches that are in the queue already from my past experience. And also it encourages to generate more code than review.
This is certainly not a black or white of course. Some areas have barely anybody for a review except for the person actively writing code in that area so this really needs the case by case approach.
Anyway this is not a new discussion or a new problem we are facing. I believe that part of the problem is that the MM subsystem doesn't really have official maintainers so there is nobody really responsible for particular parts of the subsystem. Sure Andrew is merging patches based on the review feedback or his gut feeling but I am afraid this is not enough.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
The first patch has reviews, the remainder are unloved.
Trying hard not to rant about the review mentality on this list, but I'm afraid I can't totally bite my tongue ... :)
I am afraid this is less about mentality than the lack of man power. This is not a new problem. We have much more code producers than reviewers.
It's part of the "take-and-not-give-back review mentality" and I hope you "smelled" that the comment was not targeted at you.
[...]
Quoted 10 lines
Quoted 5 lines
Although it will make upstreaming stuff *even harder* and *even slower*, maybe we should start to only queue patches that have an ACK/RB, so they won't get blocked by this later on? At least that makes your life easier and people won't have to eventually follow up on patches that have been in linux-next for months.
I wouldn't mind if patched got merged to mmotm less pro-actively at all. People tend to care less to follow up on patches that are in the queue already from my past experience. And also it encourages to generate more code than review.
The current process will at least encourage me in the long term to generate less code (changes) as well. I consider patches that have not been merged upstream as on my TODO list - and it seems to keep on growing. (yes, there are people that fire-and-forget)
Then, I much rather prefer to just get no reply on my patches, ping two times, and eventually dump them into the trash. As explained, will WHP result in me generating less code changes overall. Problem partially solved (at least one producer less) :)
Quoted 4 lines
This is certainly not a black or white of course. Some areas have barely anybody for a review except for the person actively writing code in that area so this really needs the case by case approach.
Yes.
Quoted 7 lines
Anyway this is not a new discussion or a new problem we are facing. I believe that part of the problem is that the MM subsystem doesn't really have official maintainers so there is nobody really responsible for particular parts of the subsystem. Sure Andrew is merging patches based on the review feedback or his gut feeling but I am afraid this is not enough.
If we would have "official maintainers" would it really help (besides Andrew having less stuff to do of course)? E.g., if you would pick up the memory hotplug patches, you would still have to have a look at them
- it's still the producer-consumer imbalance (and you would have an even higher workload).
But yeah, we should most probably finally have official maintainers. For people sending patches, it's often not obvious whom to cc (and whom to ping).
Smells like a "LSF/MM/BPF TOPIC", but as you said, it's not a new problem/discussion. (I won't be around, so I can't bring that topic up)
Anyhow, my two cents.
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerQuoted 27 lines · On Fri, 31 Jan 2020 10:18:34 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 14 lines · Andrew Morton
Quoted 9 lines · On Tue, 3 Dec 2019 14:36:38 +0100 Oscar Salvador <[email protected]> wrote:
Quoted 2 lines · David Hildenbrand
@Michal, @Oscar, can some of you at least have a patch #5 now so we can proceed with that? (the other patches can stay in -next some time longer)
Hi,
I will be having a look at patch#5 shortly.
Thanks for the reminder
Things haven't improved a lot :(
Quoted 8 lines
mm-memmap_init-update-variable-name-in-memmap_init_zone.patch mm-memory_hotplug-poison-memmap-in-remove_pfn_range_from_zone.patch mm-memory_hotplug-we-always-have-a-zone-in-find_smallestbiggest_section_pfn.patch mm-memory_hotplug-dont-check-for-all-holes-in-shrink_zone_span.patch mm-memory_hotplug-drop-local-variables-in-shrink_zone_span.patch mm-memory_hotplug-cleanup-__remove_pages.patch
The first patch has reviews, the remainder are unloved.
Trying hard not to rant about the review mentality on this list, but I'm afraid I can't totally bite my tongue ... :)
Quoted 8 lines
Now, this is an uncomfortable situation for you and me. You have to ping people about review and patches are stuck in your tree. I have a growing list of patches that are somewhat considered "done", but well, not-upstream-at-all. I have patches that are long in RHEL and were properly tested, but could get dropped any time because
-ENOREVIEW.Our process nowadays seems to be, to only upstream what has an ACK/RB (fixes/features/cleanups).
Yes, we've been doing this for a couple of years now. I make an exception for Vitaly's zswap patches because he appears to be the only person who knows the code (since Harry's internship ended).
I think this is the first time we've hit a significant logjam. Presumably the holiday season contributed to this.
It isn't clear to me that we've gained much from this policy. But until this cycle I've seen little harm.
Quoted 9 lines
I can understand this is desirable (yet, I am
not sure if this makes sense with the current take-and-not-give-back review mentality on this list).Although it will make upstreaming stuff *even harder* and *even slower*, maybe we should start to only queue patches that have an ACK/RB, so they won't get blocked by this later on? At least that makes your life easier and people won't have to eventually follow up on patches that have been in linux-next for months.
The merge rate would still be the review rate, but the resulting merges would be of less tested code.
Quoted 5 lines
Note: the result will be that many of my patches will still not get reviewed, won't get queued/upstreamed, I will continuously ping and resend, I will lose interest because I have better things to do, I will lose interest in our code quality, I will lose interest to review.
Quoted 7 lines
(side note: some people might actually enjoy me sending less cleanup patches, so this approach might be desirable for some ;) )
One alternative is to send patches upstream once they have been lying around in linux-next for $RANDOM number of months, because they obviously saw some testing and nobody started to yell at them once stumbling over them on linux-mm.
Yes, I think that's the case with these patches and I've sent them to Linus. Hopefully Michel will be able to find time to look them over in the next month or so.
Quoted 4 lines
Quoted 4 lines
I can understand this is desirable (yet, I am
not sure if this makes sense with the current take-and-not-give-back review mentality on this list).
Quoted 8 lines
Quoted 5 lines
Although it will make upstreaming stuff *even harder* and *even slower*, maybe we should start to only queue patches that have an ACK/RB, so they won't get blocked by this later on? At least that makes your life easier and people won't have to eventually follow up on patches that have been in linux-next for months.
The merge rate would still be the review rate, but the resulting merges would be of less tested code.
That's a valid point.
Quoted 9 lines
Quoted 9 lines
Note: the result will be that many of my patches will still not get reviewed, won't get queued/upstreamed, I will continuously ping and resend, I will lose interest because I have better things to do, I will lose interest in our code quality, I will lose interest to review.
(side note: some people might actually enjoy me sending less cleanup patches, so this approach might be desirable for some ;) )
Quoted 8 lines
Quoted 4 lines
One alternative is to send patches upstream once they have been lying around in linux-next for $RANDOM number of months, because they obviously saw some testing and nobody started to yell at them once stumbling over them on linux-mm.
Yes, I think that's the case with these patches and I've sent them to Linus. Hopefully Michel will be able to find time to look them over in the next month or so.
I really hope we'll find more reviewers in general - I'm also not happy if my patches go upstream with little/no review. However, patches shouldn't be stuck for multiple merge windows in linux-next IMHO (excluding exceptions of course) - then they should either be sent upstream (and eventually fixed later) or dropped.
Thanks Andrew!
Signature · 3 lines
Thanks, David / dhildenb
Oscar Salvador
maintainerQuoted 5 lines · David Hildenbrand
I really hope we'll find more reviewers in general - I'm also not happy if my patches go upstream with little/no review. However, patches shouldn't be stuck for multiple merge windows in linux-next IMHO (excluding exceptions of course) - then they should either be sent upstream (and eventually fixed later) or dropped.
First of all sorry for my lack of review, as lately I have been a bit disconnected of the list because lack of time.
Lucky my I managed to find some time, so I went through the patches that did lack review (#6-#10).
I hope this helps in moving forward the series, although Michal's review would be great as well.
Signature · 2 lines
Oscar Salvador SUSE L3