Skip to content

Make MIGRATE_ISOLATE a standalone bit

Zi Yan linux-mm lkml 10 versions – Applied

Status from this message: [merged mm-stable] mm-page_isolation-remove-migratetype-parameter-from-more-functions.patch removed from -mm tree

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v10 Latest6 patches 0 repliesApplied 6/6
  2. v96 patches 2 repliesSuperseded
  3. v86 patches 2 repliesSuperseded
Earlier versions · 7
VersionPostedPatchesRepliesStatus
v76 patches3 repliesSuperseded
v66 patches24 repliesSuperseded
v56 patches11 repliesSuperseded
v44 patches37 repliesSuperseded
v34 patches18 repliesSuperseded
v24 patches4 repliesSuperseded
v1 RFC4 patches10 repliesSuperseded

Patches in v10 All 6 applied

#Subject and filesSizeRepliesApplied
1/6mm/page_alloc: pageblock flags functions clean up.Documentation/mm/physical_memory.rst include/linux/mmzone.h include/linux/page-isolation.h +3 more+162 -670 repliesApplied
2/6mm/page_isolation: make page isolation a standalone bit.include/linux/mmzone.h include/linux/page-isolation.h include/linux/pageblock-flags.h +1 more+57 -30 repliesApplied
3/6mm/page_alloc: add support for initializing pageblock as isolated.include/linux/memory_hotplug.h include/linux/page-isolation.h kernel/kexec_handover.c +6 more+61 -200 repliesApplied
4/6mm/page_isolation: remove migratetype from move_freepages_block_isolate()include/linux/page-isolation.h mm/page_alloc.c mm/page_isolation.c+75 -310 repliesApplied
5/6mm/page_isolation: remove migratetype from undo_isolate_page_range()include/linux/page-isolation.h mm/memory_hotplug.c mm/page_alloc.c +1 more+7 -110 repliesApplied
6/6mm/page_isolation: remove migratetype parameter from more functions.drivers/virtio/virtio_mem.c include/linux/gfp.h include/linux/page-isolation.h +5 more+80 -680 repliesApplied

Reviews trailers per patch

All 6 patches have a Reviewed-by or Acked-by.

PatchVlastimil BabkaDavid Hildenbrand
1/6R-b A-b
2/6R-b A-b
3/6R-b A-b
4/6R-b A-b
5/6R-b A-b
6/6R-b A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v10 thread

Cover letter [PATCH v10 0/6] Make MIGRATE_ISOLATE a standalone bit

Hi all,

This patchset moves MIGRATE_ISOLATE to a standalone bit to avoid being overwritten during pageblock isolation process. Currently, MIGRATE_ISOLATE is part of enum migratetype (in include/linux/mmzone.h), thus, setting a pageblock to MIGRATE_ISOLATE overwrites its original migratetype. This causes pageblock migratetype loss during alloc_contig_range() and memory offline, especially when the process fails due to a failed pageblock isolation and the code tries to undo the finished pageblock isolations.

It is on top of mm-new.

In terms of performance for changing pageblock types, no performance change is observed:

1. I used perf to collect stats of offlining and onlining all memory of a 40GB VM 10 times and see that get_pfnblock_flags_mask() and set_pfnblock_flags_mask() take about 0.12% and 0.02% of the whole process respectively with and without this patchset across 3 runs.

2. I used perf to collect stats of dd from /dev/random to a 40GB tmpfs file and find get_pfnblock_flags_mask() takes about 0.05% of the process with and without this patchset across 3 runs.

Changelog
===
From V9[8]:
1. changed acr_flags_t item prefix from ACR_ to ACR_FLAGS_ to avoid conflicting with drm/msm hdmi_acr_cts enum.
2. converted kexec_handover to use init_pageblock_migratetype() too.

From V8[7]:
1. made init_pageblock_migratetype() set right migratetype, when page_group_by_mobility_disabled is 1.

From V7[6]:
1. restored acr_flags_t and renamed ACR_OTHER to ACR_NONE

From V6[5]:
1. used MIGRATETYPE_AND_ISO_MASK in init_pageblock_migratetype() too.
2. fixed an indentation issue in Patch 3.
3. removed acr_flags_t and used enum pb_isolate_mode instead in alloc_contig_range().
4. collected review tags.

From V5[4]:
1. used atomic version bitops for pageblock standalone bit operations.
2. added a helper function for standalone bit check.
3. renamed PB_migrate_skip to PB_compact_skip.
4. used #define MIGRATETYPE_AND_ISO_MASK MIGRATETYPE_MASK to simplify !CONFIG_MEMORY_ISOLATION code.
5. added __MIGRATE_TYPE_END to make sure migratetypes can be stored in PB_migratetype_bits.
6. used set and clear to implement toggle_pageblock_isolate() and added VM_WARN_ONCE in __move_freepages_block_isolate() to warn isolating a isolated pageblock and unisolating a not isolated pageblock.
7. dropped toggle_pfnblock_bit().
8. made acr_flags_t an enum and added ACR_OTHER for non CMA allocation.
9. renamed pb_isolate_mode items to have PB_ISOLATE_MODE prefix.
10. collected reviewed-by.

From v4[3]:
1. cleaned up existing pageblock flag functions:
a. added {get,set}_{pfnblock,pageblock}_migratetype() to change
pageblock migratetype
b. added {get,set,clear}_pfnblock_bit() to change pageblock
standalone bit, i.e., PB_migrate_skip and PB_migrate_isolate (added in this series).
c. removed {get,set}_pfnblock_flags_mask().
2. added __NR_PAGEBLOCK_BITS to present the number of pageblock flag bits and used roundup_pow_of_two(__NR_PAGEBLOCK_BITS) as NR_PAGEBLOCK_BITS.
3. moved {get,set,clear}_pageblock_isolate() to linux/page-isolation.h.
4. added init_pageblock_migratetype() to initialize a pageblock with a migratetype and isolated. It is used by memmap_init_range(), which is called by move_pfn_range_to_zone() in online_pages() from mm/memory_hotplug.c. Other set_pageblock_migratetype() users are changed too except the ones in mm/page_alloc.c.
5. toggle_pageblock_isolate() is reimplemented using __change_bit().
6. set_pageblock_migratetype() gives a warning if a pageblock is changed from MIGRATE_ISOLATE to other migratetype.
7. added pb_isolate_mode: MEMORY_OFFLINE, CMA_ALLOCATION, ISOLATE_MODE_OTHERS to replace isolate flags.
8. REPORT_FAILURE is removed, since it is only used by MEMORY_OFFLINE.

From v3[2]:
1. kept the original is_migrate_isolate_page()
2. moved {get,set,clear}_pageblock_isolate() to mm/page_isolation.c
3. used a single version for get_pageblock_migratetype() and get_pfnblock_migratetype().
4. replace get_pageblock_isolate() with get_pageblock_migratetype() == MIGRATE_ISOLATE, a get_pageblock_isolate() becomes private in mm/page_isolation.c
5. made set_pageblock_migratetype() not accept MIGRATE_ISOLATE, so that people need to use the dedicate {get,set,clear}_pageblock_isolate() APIs.
6. changed online_page() from mm/memory_hotplug.c to first set pageblock migratetype to MIGRATE_MOVABLE, then isolate pageblocks.
7. added __maybe_unused to get_pageblock_isolate(), since it is only used in VM_BUG_ON(), which could be not present when MM debug is off. It is reported by kernel test robot.
7. fixed test_pages_isolated() type issues reported by kernel test robot.

From v2[1]:
1. Moved MIGRATETYPE_NO_ISO_MASK to Patch 2, where it is used.
2. Removed spurious changes in Patch 1.
3. Refactored code so that migratetype mask is passed properly for all callers to {get,set}_pfnblock_flags_mask().
4. Added toggle_pageblock_isolate() for setting and clearing MIGRATE_ISOLATE.
5. Changed get_pageblock_migratetype() when CONFIG_MEMORY_ISOLATION to handle MIGRATE_ISOLATE case. It acts like a parsing layer for get_pfnblock_flags_mask().

Design
===

Pageblock flags are read in words to achieve good performance and existing pageblock flags take 4 bits per pageblock. To avoid a substantial change to the pageblock flag code, 8 pageblock flag bits are used.

It might look like the pageblock flags have doubled the overhead, but in reality, the overhead is only 1 byte per 2MB/4MB (based on pageblock config), or 0.0000476 %.

Any comment and/or suggestion is welcome. Thanks.

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/
[5] https://lore.kernel.org/linux-mm/[email protected]/
[6] https://lore.kernel.org/linux-mm/[email protected]/
[7] https://lore.kernel.org/linux-mm/[email protected]/

Zi Yan (6):
mm/page_alloc: pageblock flags functions clean up. mm/page_isolation: make page isolation a standalone bit. mm/page_alloc: add support for initializing pageblock as isolated. mm/page_isolation: remove migratetype from
move_freepages_block_isolate()
mm/page_isolation: remove migratetype from undo_isolate_page_range() mm/page_isolation: remove migratetype parameter from more functions.

 Documentation/mm/physical_memory.rst |   2 +-
 drivers/virtio/virtio_mem.c          |   2 +-
 include/linux/gfp.h                  |   7 +-
 include/linux/memory_hotplug.h       |   3 +-
 include/linux/mmzone.h               |  21 +-
 include/linux/page-isolation.h       |  47 +++-
 include/linux/pageblock-flags.h      |  48 ++--
 include/trace/events/kmem.h          |  14 +-
 kernel/kexec_handover.c              |   4 +-
 mm/cma.c                             |   2 +-
 mm/hugetlb.c                         |   4 +-
 mm/internal.h                        |   3 +-
 mm/memory_hotplug.c                  |  24 +-
 mm/memremap.c                        |   2 +-
 mm/mm_init.c                         |  24 +-
 mm/page_alloc.c                      | 321 +++++++++++++++++++++------
 mm/page_isolation.c                  | 100 ++++-----
 17 files changed, 435 insertions(+), 193 deletions(-)
Signature · 1 line
2.47.2