x86/mm: Tracking linear mapping split events
Subsystems (4)
PATCH v6 1/1 x86/mm: Tracking linear mapping split events
To help with debugging the sluggishness caused by TLB miss/reload, we introduce monotonic hugepage [direct mapped] split event counts since system state: SYSTEM_RUNNING to be displayed as part of /proc/vmstat in x86 servers
The lifetime split event information will be displayed at the bottom of /proc/vmstat
....swap_ra 0swap_ra_hit 0direct_map_level2_splits 94direct_map_level3_splits 4nr_unstable 0
....
One of the many lasting sources of direct hugepage splits is kernel tracing (kprobes, tracepoints).
Note that the kernel's code segment [512 MB] points to the same physical addresses that have been already mapped in the kernel's direct mapping range.
Source : Documentation/x86/x86_64/mm.rst
When we enable kernel tracing, the kernel has to modify attributes/permissions
of the text segment hugepages that are direct mapped causing them to split.
Kernel's direct mapped hugepages do not coalesce back after split and remain in place for the remainder of the lifetime.
An instance of direct page splits when we turn on dynamic kernel tracing
....
cat /proc/vmstat | grep -i direct_map_level direct_map_level2_splits 784direct_map_level3_splits 12
bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @ [pid, comm] = count(); }'
cat /proc/vmstat | grep -idirect_map_leveldirect_map_level2_splits 789direct_map_level3_splits 12
....
---
This patch has been acked and can be routed through either x86 or -mm Please let me know if there's anything needed. Thanks.
--- arch/x86/mm/pat/set_memory.c | 8 ++++++++ include/linux/vm_event_item.h | 4 ++++ mm/vmstat.c | 4 ++++ 3 files changed, 16 insertions(+)
arch/x86/mm/pat/set_memory.c +8 -0
@@ -16,6 +16,8 @@#include <linux/pci.h> #include <linux/vmalloc.h> #include <linux/libnvdimm.h> +#include <linux/vmstat.h> +#include <linux/kernel.h> #include <asm/e820/api.h> #include <asm/processor.h>
@@ -91,6 +93,12 @@ static void split_page_count(int level)return ; direct_pages_count[level]--; +if (system_state ==SYSTEM_RUNNING ) { +if (level ==PG_LEVEL_2M ) +count_vm_event (DIRECT_MAP_LEVEL2_SPLIT ); +else if (level ==PG_LEVEL_1G ) +count_vm_event (DIRECT_MAP_LEVEL3_SPLIT ); + } direct_pages_count[level -1 ] +=PTRS_PER_PTE ; }
include/linux/vm_event_item.h +4 -0
@@ -120,6 +120,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_SWAP SWAP_RA, SWAP_RA_HIT, +#endif +#ifdef CONFIG_X86 +DIRECT_MAP_LEVEL2_SPLIT , +DIRECT_MAP_LEVEL3_SPLIT , #endifNR_VM_EVENT_ITEMS };
mm/vmstat.c +4 -0
@@ -1350,6 +1350,10 @@ const char * const vmstat_text[] = { "swap_ra", "swap_ra_hit", #endif +#ifdef CONFIG_X86 + "direct_map_level2_splits", + "direct_map_level3_splits", +#endif #endif /* CONFIG_VM_EVENT_COUNTERS || CONFIG_MEMCG */ }; #endif /* CONFIG_PROC_FS || CONFIG_SYSFS || CONFIG_NUMA || CONFIG_MEMCG */
Signature · 1 line
2.24.1
Hello,
Quoted 4 lines · Saravanan D
To help with debugging the sluggishness caused by TLB miss/reload, we introduce monotonic hugepage [direct mapped] split event counts since system state:
SYSTEM_RUNNINGto be displayed as part of/proc/vmstatin x86 servers
...
Quoted 4 lines
Signed-off-by Saravanan D <[email protected]>Acked-by Tejun Heo <[email protected]>Acked-by Johannes Weiner <[email protected]>Acked-by Dave Hansen <[email protected]>
Andrew, do you mind picking this one up? It has enough acks and can go through either mm or x86 tree.
Thank you.
Signature · 1 line
tejun
Andrew Morton
maintainerQuoted 10 lines · On Thu, 18 Feb 2021 15:57:44 -0800 Saravanan D <[email protected]> wrote:
To help with debugging the sluggishness caused by TLB miss/reload, we introduce monotonic hugepage [direct mapped] split event counts since system state:
SYSTEM_RUNNINGto be displayed as part of/proc/vmstatin x86 servers...
arch/x86/mm/pat/set_memory.c enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, · line 129
Quoted 10 lines
arch/x86/mm/pat/set_memory.c +4
-0@@ -120,6 +120,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_SWAP SWAP_RA, SWAP_RA_HIT, +#endif +#ifdef CONFIG_X86 +DIRECT_MAP_LEVEL2_SPLIT , +DIRECT_MAP_LEVEL3_SPLIT , #endifNR_VM_EVENT_ITEMS };
This is the first appearance of arch-specific fields in /proc/vmstat.
I don't really see a problem with this - vmstat is basically a dumping ground of random developer stuff. But was this the best place in which to present this data?
Quoted 25 lines · Andrew Morton
Quoted 10 lines · On Thu, 18 Feb 2021 15:57:44 -0800 Saravanan D <[email protected]> wrote:
To help with debugging the sluggishness caused by TLB miss/reload, we introduce monotonic hugepage [direct mapped] split event counts since system state:
SYSTEM_RUNNINGto be displayed as part of/proc/vmstatin x86 servers...
arch/x86/mm/pat/set_memory.c enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, · line 129
Quoted 10 lines
arch/x86/mm/pat/set_memory.c +4
-0@@ -120,6 +120,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_SWAP SWAP_RA, SWAP_RA_HIT, +#endif +#ifdef CONFIG_X86 +DIRECT_MAP_LEVEL2_SPLIT , +DIRECT_MAP_LEVEL3_SPLIT , #endifNR_VM_EVENT_ITEMS };This is the first appearance of arch-specific fields in
/proc/vmstat.
Quoted 3 lines
I don't really see a problem with this - vmstat is basically a dumping ground of random developer stuff. But was this the best place in which to present this data?
IMO it's a big plus for discoverability.
One of the first things I tend to do when triaging mysterious memory issues is going to /proc/vmstat and seeing if anything looks abnormal. There is value in making that file comprehensive for all things that could indicate memory-related pathologies.
The impetus for adding these is a real-world tlb regression caused by kprobes chewing up the direct mapping that took longer to debug than necessary. We have the /proc/meminfo lines on the DirectMap, but those are more useful when you already have a theory - they simply don't make problems immediately stand out the same way.