RFC selftests: mm: introduce page allocation stall reproducer
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest3 patches 3 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | lib/test_mempress_timer: add module to generate kernel allocation pressurelib/ lib/ lib/ | +152 | 0 replies | |
| 2/3 | selftests: mm: add script to induce userspace memory contentiontools/ | +235 | 0 replies | |
| 3/3 | selftests: mm: add script for memory allocation stall testtools/ | +80 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/3] selftests: mm: introduce page allocation stall reproducer
Background
==========
Under severe system memory pressure, system unresponsiveness often occurs due to page allocation stalls. In commit 94e0bcde055e ("mm,page_alloc: reintroduce page allocation stall warning"), David Rientjes introduced a warning mechanism to emit a kernel log when a page allocation takes longer than 10 seconds. This log is used to correlate a frozen system with the system memory state at the time of failure.
To further debug and analyze these allocation stalls, we need a reproducible test case. This patch series introduces a new selftest designed to artificially mimic the severe memory pressure scenarios seen in production, allowing us to observe the resulting allocation stalls.
Patch Series Architecture
=========================
This selftest creates memory contention by saturating both userspace and kernelspace:
1. Kernelspace: A generic kernel module (test_mempress_timer.ko) binds timers to every online CPU, continuously executing atomic page allocations (GFP_ATOMIC | __GFP_NOWARN) from a timer. This behavior simulates a massive influx of networking allocations. However, such stalls can be triggered by any mechanism that allocates below the per-zone min watermarks, which forces the page allocator to continuously loop when servicing user allocations.
2. Userspace: A Python script creates a memory pressure scenario by spawning a primary memory hogger process to lock a percentage of system memory, driving the system to a low-watermark state. It simultaneously spawns concurrent worker processes that intentionally overcommit the remaining memory to churn anonymous pages.
3. Orchestration: A Bash script constructs a loopback zswap device and coordinates the execution of both workloads. The memory pressure situation is designed to last for 15 minutes by default.
Experimental Results
====================
We executed this test on a bare-metal node with the following specifications:
- CPU: 224-core Intel(R) Xeon(R) Platinum 8481C (2 Sockets, 2 NUMA)
- Memory: 503 GiB RAM
Test parameters used in the orchestrator script:
./page_alloc_stall.sh 80 64 2.5 50 memtoy/memtoy
- 80: Primary memory hogger limits memory by locking 80% of RAM.
- 64: Spawns 64 concurrent memory worker processes.
- 2.5: Workers overcommit the remaining freely available memory by 2.5x.
- 50: Allocates 50 GiB for the synthetic loopback zswap device.
- memtoy/memtoy: Path to the external binary used to allocate memory.
During execution, we mimicked the memory pressure scenario observed on production systems. Under these conditions, we observed page allocation stalls occurring across various processes, highlighting the resulting system unresponsiveness:
[ 7467.002149] watchdog: BUG: soft lockup - CPU#156 stuck for 21s! ... [ 7561.805279] cron: page allocation stall for 566 secs... [ 7668.275609] systemd: page allocation stall for 673 secs...
Test Execution
==============
This is submitted as an RFC. We are actively seeking feedback from the community on this testing methodology and integration.
The primary objective of this selftest is to establish a measurable baseline for page allocator responsiveness. This baseline will be used to validate upcoming improvements to the allocator, and will serve as a permanent framework to prevent future regressions.
To run the selftest:
1. Compile the kernel with CONFIG_TEST_MEMPRESS_TIMER=m, CONFIG_MEMCG=y, and CONFIG_ZSWAP=y.
2. Execute with default parameter values:
./tools/testing/selftests/mm/page_alloc_stall.sh
WARNING: This selftest is explicitly designed to exhaust system resources and heavily saturate the CPU. The machine will become highly unresponsive. If executed remotely, active SSH/network connections are expected to drop.
Credits and Dependencies
========================
This test depends on an external memory allocation tool, memtoy, written by KOSAKI Motohiro (https://github.com/kosaki/memtoy), which we utilize for handling the underlying anonymous mappings.
We would like to extend our thankfulness to Shakeel Butt <[email protected]> for graciously allowing us to reuse some of his original ideas and module implementations in this test.
---
Jason Miu (3):
lib/test_mempress_timer: add module to generate kernel allocation
pressure
selftests: mm: add script to induce userspace memory contention
selftests: mm: add script for memory allocation stall test
lib/Kconfig.debug | 11 + lib/Makefile | 1 + lib/test_mempress_timer.c | 140 +++++++++++ .../testing/selftests/mm/page_alloc_stall.sh | 80 ++++++ .../selftests/mm/page_alloc_stall_pressure.py | 235 ++++++++++++++++++ 5 files changed, 467 insertions(+) create mode 100644 lib/test_mempress_timer.c create mode 100644 tools/testing/selftests/mm/page_alloc_stall.sh create mode 100644 tools/testing/selftests/mm/page_alloc_stall_pressure.py
Signature · 1 line
2.55.0.691.gc56d675ccc-goog