Skip to content

mm/memory_hotplug: backport of pending stable fixes

David Hildenbrand linux-mm stable 3 versions – Applied

Status from this message: This is a note to let you know that I've just added the patch titled

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest24 patches 26 repliesApplied
  2. v224 patches 3 repliesSuperseded
  3. v125 patches 11 repliesSuperseded

Changes in v3 from the cover letter

- Fix inverted author information of two patches

Patches in v3 Applied

#Subject and filesSizeRepliesApplied
1/24mm/memory_hotplug: make remove_memory() take the device_hotplug_lockarch/powerpc/platforms/powernv/memtrace.c arch/powerpc/platforms/pseries/hotplug-memory.c drivers/acpi/acpi_memhotplug.c +2 more+15 -71 reply
2/24mm, sparse: drop pgdat_resize_lock in sparse_add/remove_one_section()include/linux/mmzone.h mm/sparse.c+2 -101 reply
3/24mm, sparse: pass nid instead of pgdat to sparse_add_one_section()include/linux/memory_hotplug.h mm/memory_hotplug.c mm/sparse.c+7 -71 reply
4/24drivers/base/memory.c: remove an unnecessary check on NR_MEM_SECTIONSdrivers/base/memory.c+1 -11 reply
5/24mm, memory_hotplug: add nid parameter to arch_remove_memoryarch/ia64/mm/init.c arch/powerpc/mm/mem.c arch/s390/mm/init.c +7 more+18 -111 reply
6/24mm/memory_hotplug: release memory resource after arch_remove_memory()mm/memory_hotplug.c+21 -141 reply
7/24drivers/base/memory.c: clean up relics in function parametersdrivers/base/memory.c+6 -61 reply
8/24mm, memory_hotplug: update a comment in unregister_memory()drivers/base/memory.c+1 -11 reply
9/24mm/memory_hotplug: make unregister_memory_section() never faildrivers/base/memory.c include/linux/memory.h mm/memory_hotplug.c+7 -151 reply
10/24mm/memory_hotplug: make __remove_section() never failmm/memory_hotplug.c+9 -131 reply
11/24powerpc/mm: Fix section mismatch warningarch/powerpc/mm/mem.c+4 -41 reply
12/24mm/memory_hotplug: make __remove_pages() and arch_remove_memory() never failarch/ia64/mm/init.c arch/powerpc/mm/mem.c arch/s390/mm/init.c +5 more+24 -401 reply
13/24s390x/mm: implement arch_remove_memory()arch/s390/mm/init.c+7 -61 reply
14/24mm/memory_hotplug: allow arch_remove_memory() without CONFIG_MEMORY_HOTREMOVEarch/ia64/mm/init.c arch/powerpc/mm/mem.c arch/s390/mm/init.c +8 more+0 -261 reply
15/24drivers/base/memory: pass a block_id to init_memory_block()drivers/base/memory.c+11 -161 reply
16/24mm/memory_hotplug: create memory block devices after arch_add_memory()drivers/base/memory.c include/linux/memory.h mm/memory_hotplug.c+63 -361 reply
17/24mm/memory_hotplug: remove memory block devices before arch_remove_memory()drivers/base/memory.c drivers/base/node.c include/linux/memory.h +2 more+30 -311 reply
18/24mm/memory_hotplug: make unregister_memory_block_under_nodes() never faildrivers/base/node.c include/linux/node.h+7 -161 reply
19/24mm/memory_hotplug: remove "zone" parameter from sparse_remove_one_sectioninclude/linux/memory_hotplug.h mm/memory_hotplug.c mm/sparse.c+4 -41 reply
20/24mm/hotplug: kill is_dev_zone() usage in __remove_pages()mm/memory_hotplug.c+2 -51 reply
21/24drivers/base/node.c: simplify unregister_memory_block_under_nodes()drivers/base/memory.c drivers/base/node.c+16 -241 reply
22/24mm/memunmap: don't access uninitialized memmap in memunmap_pages()kernel/memremap.c+7 -31 reply
23/24mm/memory_hotplug: fix try_offline_node()drivers/base/memory.c include/linux/memory.h mm/memory_hotplug.c+65 -161 reply
24/24mm/memory_hotplug: shrink zones when offlining memoryarch/ia64/mm/init.c arch/powerpc/mm/mem.c arch/s390/mm/init.c +7 more+27 -511 reply

Reviews trailers per patch

16 of 24 patches have a Reviewed-by or Acked-by.

PatchPavel TatashinRafael J. WysockiRashmica GuptaOscar SalvadorDavid HildenbrandMichal HockoAndrew MortonMukesh OjhaDan WilliamsWei YangAneesh Kumar K.VMasayoshi Mizuma
1/24R-b R-b R-b R-b R-b R-b R-b R-b
2/24R-b R-b A-b A-b
3/24R-b R-b A-b A-b
4/24R-b R-b
5/24R-b R-b R-b R-b
6/24R-b R-b
7/24R-b R-b R-b R-b A-b A-b R-b R-b
8/24
9/24
10/24R-b R-b
11/24
12/24
13/24
14/24R-b R-b
15/24A-b A-b
16/24R-b R-b A-b A-b
17/24A-b A-b R-b R-b
18/24R-b R-b A-b A-b R-b R-b
19/24R-b R-b A-b A-b R-b R-b R-b R-b
20/24R-b R-b R-b R-b T-b T-b
21/24
22/24
23/24T-b T-b
24/24R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 12 reviewers.

Read the v3 thread

Cover letter [PATCH for 4.19-stable v3 00/24] mm/memory_hotplug: backport of pending stable fixes

This is the backport of the following fixes for 4.19-stable:

- d84f2f5a7552 ("drivers/base/node.c: simplify unregister_memory_block_under_nodes()")
-- Turned out to not only be a cleanup but also a fix
- 2c91f8fc6c99 ("mm/memory_hotplug: fix try_offline_node()")
-- Automatic stable backport failed due to missing dependencies.
- feee6b298916 ("mm/memory_hotplug: shrink zones when offlining memory") -- Was marked as stable 5.0+ due to the backport complexity,, but it's also relevant for 4.19/4.14. As I have to backport quite some cleanups already ...

All tackle memory unplug issues, especially when memory was never onlined (or onlining failed), paired with memory unplug. When trying to access garbage memmaps we crash the kernel (e.g., because the derviced pgdat pointer is broken)

To minimize manual code changes, I decided to pull in quite some cleanups. Still some manual code changes are necessary (indicated in the individual patches). Especially missing arm64 hot(un)plug, missing sub-section hotadd support, and missing unification of mm/hmm.c and kernel/memremap.c requires care.

Due to:
- 4e0d2e7ef14d ("mm, sparse: pass nid instead of pgdat to sparse_add_one_section()")
I need:
- afe9b36ca890 ("mm/memunmap: don't access uninitialized memmap in memunmap_pages()")

Please note that:
- 4c4b7f9ba948 ("mm/memory_hotplug: remove memory block devices before arch_remove_memory()")
Makes big (e.g., 32TB) machines boot up slower (e.g., 2h vs 10m). There is a performance fix in linux-next, but it does not seem to classify as a fix for current RC / stable.

I did quite some testing with hot(un)plug, onlining/offlining of memory blocks and memory-less/CPU-less NUMA nodes under x86_64 - the same set of tests I run against upstream on a fairly regular basis. I compile-tested on PowerPC, arm64, s390x, i386 and sh. I did not test any ZONE_DEVICE/HMM thingies.

The 4.14 backport might take a bit - it would be quite a lot of patches to backport and it is not that severely broken, so I am thinking about simpler (less invasive) alternatives.

v2 -> v3:
- Fix inverted author information of two patches

v1 -> v2:
- Fix patch authors
- Dropped "mm/memory_hotplug: make __remove_pages() and arch_remove_memory() never fail"
-- Only creates a minor conflict in another patch
- "mm/memory_hotplug: make remove_memory() take the device_hotplug_lock" -- Fix wrong upstream commit id
- "mm/memory_hotplug: shrink zones when offlining memory"
- "mm/memunmap: don't access uninitialized memmap in memunmap_pages()" -- Fix usage of wrong pfn

CCing only some people to minimize noise.

Cc Michal Hocko <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Andrew Morton <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Baoquan He <[email protected]>
Cc Dan Williams <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Wei Yang <[email protected]>

Aneesh Kumar K.V (2):
powerpc/mm: Fix section mismatch warning
mm/memunmap: don't access uninitialized memmap in memunmap_pages()

Baoquan He (1):
drivers/base/memory.c: clean up relics in function parameters

Dan Carpenter (1):
mm, memory_hotplug: update a comment in unregister_memory()

Dan Williams (1):
mm/hotplug: kill is_dev_zone() usage in __remove_pages()

David Hildenbrand (15):
mm/memory_hotplug: make remove_memory() take the device_hotplug_lock mm/memory_hotplug: release memory resource after arch_remove_memory() mm/memory_hotplug: make unregister_memory_section() never fail mm/memory_hotplug: make __remove_section() never fail mm/memory_hotplug: make __remove_pages() and arch_remove_memory()
never fail
s390x/mm: implement arch_remove_memory() mm/memory_hotplug: allow arch_remove_memory() without
CONFIG_MEMORY_HOTREMOVE
drivers/base/memory: pass a block_id to init_memory_block() mm/memory_hotplug: create memory block devices after arch_add_memory() mm/memory_hotplug: remove memory block devices before
arch_remove_memory()
mm/memory_hotplug: make unregister_memory_block_under_nodes() never
fail
mm/memory_hotplug: remove "zone" parameter from
sparse_remove_one_section
drivers/base/node.c: simplify unregister_memory_block_under_nodes() mm/memory_hotplug: fix try_offline_node() mm/memory_hotplug: shrink zones when offlining memory

Oscar Salvador (1):
mm, memory_hotplug: add nid parameter to arch_remove_memory

Wei Yang (3):
mm, sparse: drop pgdat_resize_lock in sparse_add/remove_one_section() mm, sparse: pass nid instead of pgdat to sparse_add_one_section() drivers/base/memory.c: remove an unnecessary check on NR_MEM_SECTIONS

 arch/ia64/mm/init.c                           |  15 +-
 arch/powerpc/mm/mem.c                         |  25 +--
 arch/powerpc/platforms/powernv/memtrace.c     |   2 +-
 .../platforms/pseries/hotplug-memory.c        |   6 +-
 arch/s390/mm/init.c                           |  16 +-
 arch/sh/mm/init.c                             |  15 +-
 arch/x86/mm/init_32.c                         |   9 +-
 arch/x86/mm/init_64.c                         |  17 +-
 drivers/acpi/acpi_memhotplug.c                |   2 +-
 drivers/base/memory.c                         | 203 +++++++++++-------
 drivers/base/node.c                           |  52 ++---
 include/linux/memory.h                        |   8 +-
 include/linux/memory_hotplug.h                |  22 +-
 include/linux/mmzone.h                        |   3 +-
 include/linux/node.h                          |   7 +-
 kernel/memremap.c                             |  12 +-
 mm/hmm.c                                      |   8 +-
 mm/memory_hotplug.c                           | 166 +++++++-------
 mm/sparse.c                                   |  27 +--
 19 files changed, 303 insertions(+), 312 deletions(-)
Signature · 1 line
2.24.1