RFC PMEM NUMA node and hotness accounting/migration
Get series with b4
b4 am [email protected]Versions newest first
- v2 RFC Latest21 patches 41 repliesIn review
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/21 | e820: cheat PMEM as DRAMarch/ | +2 | 5 replies | |
| 2/21 | acpi/numa: memorize NUMA node type from SRAT tablearch/ arch/ drivers/ | +9 | 0 replies | |
| 3/21 | x86/numa_emulation: fix fake NUMA in uniform casearch/ | +15 | 0 replies | |
| 4/21 | x86/numa_emulation: pass numa node type to fake nodesarch/ | +14 | 0 replies | |
| 5/21 | mmzone: new pgdat flags for DRAM and PMEMinclude/ | +26 | 0 replies | |
| 6/21 | x86,numa: update numa node typearch/ | +1 | 0 replies | |
| 7/21 | mm: export node type {pmem|dram} under /sys/bus/nodedrivers/ | +10 | 0 replies | |
| 8/21 | mm: introduce and export pgdat peer_nodedrivers/ include/ mm/ | +52 | 2 replies | |
| 9/21 | mm: avoid duplicate peer target nodemm/ | +6 | 0 replies | |
| 10/21 | mm: build separate zonelist for PMEM and DRAM nodemm/ mm/ | +43 | 3 replies | |
| 11/21 | kvm: allocate page table pages from DRAMarch/ | +11 | 4 replies | |
| 12/21 | x86/pgtable: allocate page table pages from DRAMarch/ arch/ | +25 | 0 replies | |
| 13/21 | x86/pgtable: dont check PMD accessed bitarch/ | +2 | 0 replies | |
| 14/21 | kvm: register in mm_structinclude/ virt/ | +14 | 3 replies | |
| 15/21 | ept-idle: EPT walk for virtual machinearch/ arch/ | +753 | 0 replies | |
| 16/21 | mm-idle: mm_walk for normal taskarch/ mm/ | +205 | 0 replies | |
| 17/21 | proc: introduce /proc/PID/idle_pagesfs/ fs/ fs/ | +57 | 0 replies | |
| 18/21 | kvm-ept-idle: enable modulearch/ arch/ | +15 | 0 replies | |
| 19/21 | mm/migrate.c: add move_pages(MPOL_MF_SW_YOUNG) flagmm/ | +10 | 0 replies | |
| 20/21 | mm/vmscan.c: migrate anon DRAM pages to PMEM nodemm/ | +33 | 0 replies | |
| 21/21 | mm/vmscan.c: shrink anon list if can migrate to PMEMmm/ | +3 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [RFC][PATCH v2 00/21] PMEM NUMA node and hotness accounting/migration
This is an attempt to use NVDIMM/PMEM as volatile NUMA memory that's transparent to normal applications and virtual machines.
The code is still in active development. It's provided for early design review.
Key functionalities:
1) create and describe PMEM NUMA node for NVDIMM memory
2) dumb /proc/PID/idle_pages interface, for user space driven hot page accounting
3) passive kernel cold page migration in page reclaim path
4) improved move_pages() for active user space hot/cold page migration
(1) is foundation for transparent usage of NVDIMM for normal apps and virtual machines. (2-4) enable auto placing hot pages in DRAM for better performance. A user space migration daemon is being built based on this kernel patchset to make the full vertical solution.
Base kernel is v4.20 . The patches are not suitable for upstreaming in near future -- some are quick hacks, some others need more works. However they are complete enough to demo the necessary kernel changes for the proposed app&VM transparent NVDIMM volatile use model.
The interfaces are far from finalized. They kind of illustrate what would be necessary for creating a user space driven solution. The exact forms will ask for more thoughts and inputs. We may adopt HMAT based solution for NUMA node related interface when they are ready. The /proc/PID/idle_pages interface is standalone but non-trivial. Before upstreaming some day, it's expected to take long time to collect various real use cases and feedbacks, so as to refine and stabilize the format.
Create PMEM numa node
[PATCH 01/21] e820: cheat PMEM as DRAM
Mark numa node as DRAM/PMEM
[PATCH 02/21] acpi/numa: memorize NUMA node type from SRAT table [PATCH 03/21] x86/numa_emulation: fix fake NUMA in uniform case [PATCH 04/21] x86/numa_emulation: pass numa node type to fake nodes [PATCH 05/21] mmzone: new pgdat flags for DRAM and PMEM [PATCH 06/21] x86,numa: update numa node type
[PATCH 07/21] mm: export node type {pmem|dram} under /sys/bus/node
Point neighbor DRAM/PMEM to each other
[PATCH 08/21] mm: introduce and export pgdat peer_node [PATCH 09/21] mm: avoid duplicate peer target node
Standalone zonelist for DRAM and PMEM nodes
[PATCH 10/21] mm: build separate zonelist for PMEM and DRAM node
Keep page table pages in DRAM
[PATCH 11/21] kvm: allocate page table pages from DRAM [PATCH 12/21] x86/pgtable: allocate page table pages from DRAM
/proc/PID/idle_pages interface for virtual machine and normal tasks
[PATCH 13/21] x86/pgtable: dont check PMD accessed bit [PATCH 14/21] kvm: register in mm_struct [PATCH 15/21] ept-idle: EPT walk for virtual machine [PATCH 16/21] mm-idle: mm_walk for normal task [PATCH 17/21] proc: introduce /proc/PID/idle_pages [PATCH 18/21] kvm-ept-idle: enable module
Mark hot pages
[PATCH 19/21] mm/migrate.c: add move_pages(MPOL_MF_SW_YOUNG) flag
Kernel DRAM=>PMEM migration
[PATCH 20/21] mm/vmscan.c: migrate anon DRAM pages to PMEM node [PATCH 21/21] mm/vmscan.c: shrink anon list if can migrate to PMEM
arch/x86/include/asm/numa.h | 2 arch/x86/include/asm/pgalloc.h | 10 arch/x86/include/asm/pgtable.h | 3 arch/x86/kernel/e820.c | 3 arch/x86/kvm/Kconfig | 11 arch/x86/kvm/Makefile | 4 arch/x86/kvm/ept_idle.c | 841 +++++++++++++++++++++++++++++++ arch/x86/kvm/ept_idle.h | 116 ++++ arch/x86/kvm/mmu.c | 12 arch/x86/mm/numa.c | 3 arch/x86/mm/numa_emulation.c | 30 + arch/x86/mm/pgtable.c | 22 drivers/acpi/numa.c | 5 drivers/base/node.c | 21 fs/proc/base.c | 2 fs/proc/internal.h | 1 fs/proc/task_mmu.c | 54 + include/linux/mm_types.h | 11 include/linux/mmzone.h | 38 + mm/mempolicy.c | 14 mm/migrate.c | 13 mm/page_alloc.c | 77 ++ mm/pagewalk.c | 1 mm/vmscan.c | 38 + virt/kvm/kvm_main.c | 3 25 files changed, 1306 insertions(+), 29 deletions(-)
V1 patches: https://lkml.org/lkml/2018/9/2/13
Regards,
Fengguang