Skip to content

mm/memory_hotplug: Consider all zones when removing memory

David Hildenbrand linux-mm linuxppc-dev lkml 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest6 patches 23 repliesIn review
  2. v15 patches 1 replySuperseded

Changes in v2 from the cover letter

- Include "mm: Introduce for_each_zone_nid()"
- "mm/memory_hotplug: Pass nid instead of zone to __remove_pages()" -- Pass the nid instead of the zone and use it to reduce the number of zones to process

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6mm/memory_hotplug: Exit early in __remove_pages() on BUGsmm/memory_hotplug.c+3 -30 replies
2/6mm: Exit early in set_zone_contiguous() if already contiguousmm/page_alloc.c+3 -00 replies
3/6mm/memory_hotplug: Process all zones when removing memorymm/memory_hotplug.c+18 -79 replies
4/6mm/memory_hotplug: Cleanup __remove_pages()mm/memory_hotplug.c+6 -110 replies
5/6mm: Introduce for_each_zone_nid()include/linux/mmzone.h+5 -00 replies
6/6mm/memory_hotplug: Pass nid instead of zone to __remove_pages()arch/arm64/mm/mmu.c arch/ia64/mm/init.c arch/powerpc/mm/mem.c +7 more+18 -311 reply

Reviews trailers per patch

1 of 6 patches has a Reviewed-by or Acked-by.

PatchRobin Murphy
1/6
2/6
3/6
4/6
5/6
6/6A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/6] mm/memory_hotplug: Consider all zones when removing memory

Working on virtio-mem, I was able to trigger a kernel BUG (with debug options enabled) when removing memory that was never onlined. I was able to reproduce with DIMMs. As far as I can see the same can also happen without debug configs enabled, if we're unlucky and the uninitialized memmap contains selected garbage .

The root problem is that we should not try to derive the zone of memory we are removing from the first PFN. The individual memory blocks of a DIMM could be spanned by different ZONEs, multiple ZONES (after being offline and re-onlined) or no ZONE at all (never onlined).

Let's process all applicable zones when removing memory so we're on the safe side. In the long term, we want to resize the zones when offlining memory (and before removing ZONE_DEVICE memory), however, that will require more thought (and most probably a new SECTION_ACTIVE / pfn_active() thingy). More details about that in patch #3.

Along with the fix, some related cleanups.

v1 -> v2:
- Include "mm: Introduce for_each_zone_nid()"
- "mm/memory_hotplug: Pass nid instead of zone to __remove_pages()" -- Pass the nid instead of the zone and use it to reduce the number of zones to process

--- snip ---

I gave this a quick test with a DIMM on x86-64:

Start with a NUMA-less node 1. Hotplug a DIMM (512MB) to Node 1. 1st memory block is not onlined. 2nd and 4th is onlined MOVABLE. 3rd is onlined NORMAL.

:/# echo "online_movable" > /sys/devices/system/memory/memory41/state
[...]
:/# echo "online_movable" > /sys/devices/system/memory/memory43/state
:/# echo "online_kernel" > /sys/devices/system/memory/memory42/state
:/# cat /sys/devices/system/memory/memory40/state
offline
:/# cat /proc/zoneinfo
Node 1, zone   Normal
 [...]
        spanned  32768
        present  32768
        managed  32768
 [...]
Node 1, zone  Movable
 [...]
        spanned  98304
        present  65536
        managed  65536
 [...]

Trigger hotunplug. If it succeeds (block 42 can be offlined):

:/# cat /proc/zoneinfo

Node 1, zone Normal

  pages free     0
        min      0
        low      0
        high     0
        spanned  0
        present  0
        managed  0
        protection: (0, 0, 0, 0, 0)
Node 1, zone  Movable
  pages free     0
        min      0
        low      0
        high     0
        spanned  0
        present  0
        managed  0
        protection: (0, 0, 0, 0, 0)

So all zones were properly fixed up and we don't access the memmap of the first, never-onlined memory block (garbage). I am no longer able to trigger the BUG. I did a similar test with an already populated node.

David Hildenbrand (6):
mm/memory_hotplug: Exit early in __remove_pages() on BUGs
mm: Exit early in set_zone_contiguous() if already contiguous mm/memory_hotplug: Process all zones when removing memory mm/memory_hotplug: Cleanup __remove_pages()
mm: Introduce for_each_zone_nid()
mm/memory_hotplug: Pass nid instead of zone to __remove_pages()

 arch/arm64/mm/mmu.c            |  4 +--
 arch/ia64/mm/init.c            |  4 +--
 arch/powerpc/mm/mem.c          |  3 +-
 arch/s390/mm/init.c            |  4 +--
 arch/sh/mm/init.c              |  4 +--
 arch/x86/mm/init_32.c          |  4 +--
 arch/x86/mm/init_64.c          |  4 +--
 include/linux/memory_hotplug.h |  2 +-
 include/linux/mmzone.h         |  5 ++++
 mm/memory_hotplug.c            | 51 +++++++++++++++++++---------------
 mm/memremap.c                  |  3 +-
 mm/page_alloc.c                |  3 ++
 12 files changed, 46 insertions(+), 45 deletions(-)
Signature · 1 line
2.21.0