Skip to content

RFC Kernel thread based async batch migration

Bharata B Rao linux-mm lkml oe-kbuild-all 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest4 patches 8 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4mm: migrate: Allow misplaced migration without VMA toomm/migrate.c+3 -20 replies
2/4migrate: implement migrate_misplaced_folios_batchinclude/linux/migrate.h mm/migrate.c+37 -00 replies
3/4mm: kmigrated - Async kernel migration threadinclude/linux/mmzone.h include/linux/page_ext.h mm/Makefile +3 more+264 -15 replies
4/4mm: sched: Batch-migrate misplaced pagesmm/memory.c+5 -250 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH v1 0/4] Kernel thread based async batch migration

Hi,

This is a continuation of the earlier post[1] that attempted to convert migrations from NUMA Balancing to be async and batched. In this version, per-node kernel threads are created to handle migrations in an async manner.

This adds a few fields to the extended page flags that can be used both by the sub-systems that request migrations and kmigrated which migrates the pages. Some of the fields are potentially defined to be used by kpromoted-like subsystem to manage hot page metrics, but are unused right now.

Currently only NUMA Balancing is changed to make use of the async batched migration. It does so by recording the target NID and the readiness of the page to be migrated in the extended page flags fields.

Each kmigrated routinely scans its PFNs, identifies the pages marked for migration and batch-migrates them. Unlike the previous approach, the responsibility of isolating the pages is now with kmigrated.

The major difference between this approach and the way kpromoted[2] tracked hot pages is the elimination of heavy synchronization points between producers(sub-systems that request migrations or report a hot page) and the consumer (kmigrated or kpromoted). Instead of tracking only the list of hot pages in an orthogonal manner, this approach ties the hot page or migration infomation to the struct page.

TODOs:

- Very lightly tested(only with NUMAB=1) and posted to get some feedback on the overall approach.
- Currently uses the flags field from page extension sub-system. However need to check if it is preferrable to use/allocate a separate 32bit field exclusively for this purpose within page extension sub-system or outside of it.
- Benefit of async batch migration still needs to be measured.
- Need to really tune a few things like the number of pages to batch, the aggressiveness of kthread, the kthread sleep interval etc.
- The logic to skip scanning of zones that don't have any pages marked for migration needs to be added.
- No separate kernel config is defined currently and dependency on PAGE_EXTENSION isn't cleanly laid out. Some added definitions currently sit in page_ext.h which may not be an ideal location for them.

[1] v0 - https://lore.kernel.org/linux-mm/[email protected]/
[2] kpromoted patchset - https://lore.kernel.org/linux-mm/[email protected]/

Bharata B Rao (3):
mm: migrate: Allow misplaced migration without VMA too
mm: kmigrated - Async kernel migration thread
mm: sched: Batch-migrate misplaced pages

Gregory Price (1):
migrate: implement migrate_misplaced_folios_batch

 include/linux/migrate.h  |   6 ++
 include/linux/mmzone.h   |   5 +
 include/linux/page_ext.h |  17 +++
 mm/Makefile              |   3 +-
 mm/kmigrated.c           | 223 +++++++++++++++++++++++++++++++++++++++
 mm/memory.c              |  30 +-----
 mm/migrate.c             |  36 ++++++-
 mm/mm_init.c             |   6 ++
 mm/page_ext.c            |  11 ++
 9 files changed, 309 insertions(+), 28 deletions(-)
 create mode 100644 mm/kmigrated.c
Signature · 1 line
2.34.1