Multigenerational LRU Framework
Subsystems (21)
v6 cover Multigenerational LRU Framework
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.
Design objectives
=================
The design objectives are:
1. Better representation of access recency
2. Try to profit from spatial locality
3. Clear fast path making obvious choices
4. Simple self-correcting heuristics
The representation of access recency is at the core of all LRU approximations. The multigenerational LRU (MGLRU) divides pages into multiple lists (generations), each having bounded access recency (a time interval). Generations establish a common frame of reference and help make better choices, e.g., between different memcgs on a computer or different computers in a data center (for cluster job scheduling).
Exploiting spatial locality improves the efficiency when gathering the accessed bit. A rmap walk targets a single page and doesn't try to profit from discovering an accessed PTE. A page table walk can sweep all hotspots in an address space, but its search space can be too large to make a profit. The key is to optimize both methods and use them in combination. (PMU is another option for further exploration.)
Fast path reduces code complexity and runtime overhead. Unmapped pages don't require TLB flushes; clean pages don't require writeback. These facts are only helpful when other conditions, e.g., access recency, are similar. With generations as a common frame of reference, additional factors stand out. But obvious choices might not be good choices; thus self-correction is required (the next objective).
The benefits of simple self-correcting heuristics are self-evident. Again with generations as a common frame of reference, this becomes attainable. Specifically, pages in the same generation are categorized based on additional factors, and a closed-loop control statistically compares the refault percentages across all categories and throttles the eviction of those that have higher percentages.
Patchset overview
=================
1. mm: x86, arm64: add arch_has_hw_pte_young()
2. mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG Materializing hardware optimizations when trying to clear the accessed bit in many PTEs. If hardware automatically sets the accessed bit in PTEs, there is no need to worry about bursty page faults (emulating the accessed bit). If it also sets the accessed bit in non-leaf PMD entries, there is no need to search the PTE table pointed to by a PMD entry that doesn't have the accessed bit set.
3. mm/vmscan.c: refactor shrink_node()
A minor refactor.
4. mm: multigenerational lru: groundwork
Adding the basic data structure and the functions to initialize it and insert/remove pages.
5. mm: multigenerational lru: mm_struct list
An infra keeps track of mm_struct's for page table walkers and provides them with optimizations, i.e., switch_mm() tracking and Bloom filters.
6. mm: multigenerational lru: aging
7. mm: multigenerational lru: eviction
"The page reclaim" is a producer/consumer model. "The aging" produces cold pages, whereas "the eviction " consumes them. Cold pages flow through generations. The aging uses the mm_struct list infra to sweep dense hotspots in page tables. During a page table walk, the aging clears the accessed bit and tags accessed pages with the youngest generation number. The eviction sorts those pages when it encounters them. For pages in the oldest generation, eviction walks the rmap to check the accessed bit one more time before evicting them. During an rmap walk, the eviction feeds dense hotspots back to the aging. Dense hotspots flow through the Bloom filters. For pages not mapped in page tables, the eviction uses the PID controller to statistically determine whether they have higher refaults. If so, the eviction throttles their eviction by moving them to the next generation (the second oldest).
8. mm: multigenerational lru: user interface
The knobs to turn on/off MGLRU and provide the userspace with thrashing prevention, working set estimation (the aging) and proactive reclaim (the eviction).
9. mm: multigenerational lru: Kconfig
The Kconfig options.
Benchmark results
=================
Independent lab results
-----------------------
Based on the popularity of searches [01] and the memory usage in Google's public cloud, the most popular open-source memory-hungry applications, in alphabetical order, are:
Apache Cassandra Memcached
Apache Hadoop MongoDB
Apache Spark PostgreSQL
MariaDB (MySQL) RedisAn independent lab evaluated MGLRU with the most widely used benchmark suites for the above applications. They posted 960 data points along with kernel metrics and perf profiles collected over more than 500 hours of total benchmark time. Their final reports show that, with 95% confidence intervals (CIs), the above applications all performed significantly better for at least part of their benchmark matrices.
On 5.14:
1. Apache Spark [02] took 95% CIs [9.28, 11.19]% and [12.20, 14.93]% less wall time to sort three billion random integers, respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in wall time for the rest of the benchmark matrix.
2. MariaDB [03] achieved 95% CIs [5.24, 10.71]% and [20.22, 25.97]% more transactions per minute (TPM), respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
3. Memcached [04] achieved 95% CIs [23.54, 32.25]%, [20.76, 41.61]% and [21.59, 30.02]% more operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [13.85, 15.97]% and [23.94, 29.92]% more OPS, respectively, for random access and Gaussian access, when THP=never. There were no statistically significant changes in OPS for the rest of the benchmark matrix.
4. MongoDB [05] achieved 95% CIs [2.23, 3.44]%, [6.97, 9.73]% and [2.16, 3.55]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when underutilizing memory; 95% CIs [8.83, 10.03]%, [21.12, 23.14]% and [5.53, 6.46]% more OPS, respectively, for exponential access, random access and Zipfian access, when overcommitting memory.
On 5.15:
5. Apache Cassandra [06] achieved 95% CIs [1.06, 4.10]%, [1.94, 5.43]% and [4.11, 7.50]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when swap was off; 95% CIs [0.50, 2.60]%, [6.51, 8.77]% and [3.29, 6.75]% more OPS, respectively, for exponential access, random access and Zipfian access, when swap was on.
6. Apache Hadoop [07] took 95% CIs [5.31, 9.69]% and [2.02, 7.86]% less average wall time to finish twelve parallel TeraSort jobs, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in average wall time for the rest of the benchmark matrix.
7. PostgreSQL [08] achieved 95% CI [1.75, 6.42]% more transactions per minute (TPM) under the high-concurrency condition, when swap was off; 95% CIs [12.82, 18.69]% and [22.70, 46.86]% more TPM, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
8. Redis [09] achieved 95% CIs [0.58, 5.94]%, [6.55, 14.58]% and [11.47, 19.36]% more total operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [1.27, 3.54]%, [10.11, 14.81]% and [8.75, 13.64]% more total OPS, respectively, for sequential access, random access and Gaussian access, when THP=never.
Our lab results
---------------
To supplement the above results, we ran the following benchmark suites on 5.16-rc7 and found no regressions [10]. (These synthetic benchmarks are popular among MM developers, but we prefer large-scale A/B experiments to validate improvements.)
fs_fio_bench_hdd_mq pft
fs_lmbench pgsql-hammerdb
fs_parallelio redis
fs_postmark stream
hackbench sysbenchthread
kernbench tpcc_spark
memcached unixbench
multichase vm-scalability
mutilate will-it-scalenginx
[01] https://trends.google.com
[02] https://lore.kernel.org/linux-mm/[email protected]/
[03] https://lore.kernel.org/linux-mm/[email protected]/
[04] https://lore.kernel.org/linux-mm/[email protected]/
[05] https://lore.kernel.org/linux-mm/[email protected]/
[06] https://lore.kernel.org/linux-mm/[email protected]/
[07] https://lore.kernel.org/linux-mm/[email protected]/
[08] https://lore.kernel.org/linux-mm/[email protected]/
[09] https://lore.kernel.org/linux-mm/[email protected]/
[10] https://lore.kernel.org/linux-mm/[email protected]/
Read-world applications
=======================
Third-party testimonials
------------------------
Konstantin wrote [11]:
I have Archlinux with 8G RAM + zswap + swap. While developing, I have lots of apps opened such as multiple LSP-servers for different langs, chats, two browsers, etc... Usually, my system gets quickly to a point of SWAP-storms, where I have to kill LSP-servers, restart browsers to free memory, etc, otherwise the system lags heavily and is barely usable.
1.5 day ago I migrated from 5.11.15 kernel to 5.12 + the LRU patchset, and I started up by opening lots of apps to create memory pressure, and worked for a day like this. Till now I had *not a single SWAP-storm*, and mind you I got 3.4G in SWAP. I was never getting to the point of 3G in SWAP before without a single SWAP-storm.
The Arch Linux Zen kernel [12] has been using MGLRU since 5.12. Many of its users reported their positive experiences to me, e.g., Shivodit wrote:
I've tried the latest Zen kernel (5.14.13-zen1-1-zen in the archlinux testing repos), everything's been smooth so far. I also decided to copy a large volume of files to check performance under I/O load, and everything went smoothly - no stuttering was present, everything was responsive.
Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition to improvements in other UX metrics, e.g., an 85% decrease in the number of low-memory kills at the 75th percentile and an 18% decrease in rendering latency at the 50th percentile.
[11] https://lore.kernel.org/linux-mm/[email protected]/
[12] https://github.com/zen-kernel/zen-kernel/
[13] https://research.google/pubs/pub44271/
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.
[14] https://lore.kernel.org/lkml/[email protected]/
[15] https://lore.kernel.org/lkml/[email protected]/
[16] https://lore.kernel.org/lkml/[email protected]/
Yu Zhao (9):
mm: x86, arm64: add arch_has_hw_pte_young()
mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
mm/vmscan.c: refactor shrink_node()
mm: multigenerational lru: groundwork
mm: multigenerational lru: mm_struct list
mm: multigenerational lru: aging
mm: multigenerational lru: eviction
mm: multigenerational lru: user interface
mm: multigenerational lru: Kconfig
Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 80 + arch/Kconfig | 9 + arch/arm64/include/asm/cpufeature.h | 5 + arch/arm64/include/asm/pgtable.h | 13 +- arch/arm64/kernel/cpufeature.c | 19 + arch/arm64/tools/cpucaps | 1 + arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 9 +- arch/x86/mm/pgtable.c | 5 +- fs/exec.c | 2 + fs/fuse/dev.c | 3 +- include/linux/cgroup.h | 15 +- include/linux/memcontrol.h | 11 + include/linux/mm.h | 42 + include/linux/mm_inline.h | 204 ++ include/linux/mm_types.h | 78 + include/linux/mmzone.h | 175 ++ include/linux/nodemask.h | 1 + include/linux/oom.h | 16 + include/linux/page-flags-layout.h | 19 +- include/linux/page-flags.h | 4 +- include/linux/pgtable.h | 17 +- include/linux/sched.h | 4 + include/linux/swap.h | 4 + kernel/bounds.c | 3 + kernel/cgroup/cgroup-internal.h | 1 - kernel/exit.c | 1 + kernel/fork.c | 9 + kernel/sched/core.c | 1 + mm/Kconfig | 48 + mm/huge_memory.c | 3 +- mm/memcontrol.c | 26 + mm/memory.c | 21 +- mm/mm_init.c | 6 +- mm/oom_kill.c | 4 +- mm/page_alloc.c | 1 + mm/rmap.c | 7 + mm/swap.c | 51 +- mm/vmscan.c | 2691 ++++++++++++++++++++++++++- mm/workingset.c | 119 +- 41 files changed, 3591 insertions(+), 139 deletions(-) create mode 100644 Documentation/vm/multigen_lru.rst
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
[*] BENCHMARK
Name: fio_bench_hdd_mq
Description: FIO benchmark running against HDD multi-queue
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 10 machine_total_logical_cores | 72 machine_total_physical_cores | 36 machine_total_ram_gib | 256
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
----------------------------------------------------------------+-------+------------------------+------------------------+------------------------+------------------------+------------------------+----------------
hdd_bfq_randread_sync1_8k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 433.3917662682603 | 557.8477202301904 | 489.18242841063557 | 490.6666666666667 | 19.097252390047217 |
(B) 84f99e060a66 | 481 | 426.431164231961 | 536.8888888888889 | 487.8554059336687 | 487.1111111111111 | 19.48371520112119 |
| | -1.61% | -3.76% | -0.27% | -0.72% | +2.02% | + is good
hdd_bfq_randread_sync1_8k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.4456787946614986e+07 | 1.9117502479907356e+07 | 1.645379955129552e+07 | 1.6380090804665854e+07 | 679101.4140279738 |
(B) 84f99e060a66 | 481 | 1.4906508581318656e+07 | 1.9300071993749753e+07 | 1.6499377634492567e+07 | 1.6433589051097086e+07 | 689703.6846144461 |
| | +3.11% | +0.95% | +0.28% | +0.33% | +1.56% | <not defined>
hdd_bfq_randread_sync1_8k_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 446.764 | 768 | 642.4415851769633 | 640 | 49.16317155463473 |
(B) 84f99e060a66 | 481 | 480 | 826.9960159362549 | 641.5146399847603 | 640 | 49.84433236262648 |
| | +7.44% | +7.68% | -0.14% | +0.00% | +1.39% | + is good
hdd_bfq_randread_sync1_8k_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.0446467379999999e+07 | 1.8519737817843866e+07 | 1.2503646313603505e+07 | 1.2448606008064516e+07 | 955886.7640578166 |
(B) 84f99e060a66 | 481 | 9.912718333333334e+06 | 1.5913304476987448e+07 | 1.252581114876277e+07 | 1.2491253064257028e+07 | 911555.2614030009 |
| | -5.11% | -14.07% | +0.18% | +0.34% | -4.64% | <not defined>
hdd_bfq_randread_sync1_8m_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 65405.593625498004 | 131072 | 102073.76626549932 | 98304 | 16325.722371561216 |
(B) 84f99e060a66 | 481 | 65405.593625498004 | 131072 | 101552.34142018206 | 98304 | 16719.246285256806 |
| | +0.00% | +0.00% | -0.51% | +0.00% | +2.41% | + is good
hdd_bfq_randread_sync1_8m_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 6.4083031916230366e+07 | 1.0661207040092166e+08 | 8.072525879536723e+07 | 7.972267310126582e+07 | 7.477349032091799e+06 |
(B) 84f99e060a66 | 481 | 6.2635378e+07 | 1.2953105246037737e+08 | 8.129236612411591e+07 | 8.028132555691057e+07 | 8.401679764088364e+06 |
| | -2.26% | +21.50% | +0.70% | +0.70% | +12.36% | <not defined>
hdd_bfq_randread_sync1_256k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 12281.12793271359 | 15132.444444444445 | 13802.442575918547 | 13767.111111111111 | 527.1238640248814 |
(B) 84f99e060a66 | 480 | 11031.917662682603 | 15246.222222222223 | 13792.122426958833 | 13816.67994687915 | 562.7906225959111 |
| | -10.17% | +0.75% | -0.07% | +0.36% | +6.77% | + is good
hdd_bfq_randread_sync1_256k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.7051231348338768e+07 | 2.150583715581118e+07 | 1.8638179679985933e+07 | 1.859534185701346e+07 | 717361.022579436 |
(B) 84f99e060a66 | 481 | 1.6743855827803267e+07 | 2.2890306188082602e+07 | 1.8663653487930812e+07 | 1.8575175956296563e+07 | 792153.6139743543 |
| | -1.80% | +6.44% | +0.14% | -0.11% | +10.43% | <not defined>
hdd_bfq_randread_sync1_prio_256k_antagonized_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 1251.4555112881806 | 5798.982735723771 | 3195.510963197429 | 3185.777777777778 | 656.6188770523272 |
(B) 84f99e060a66 | 480 | 1535.4466578131917 | 6026.538291279327 | 3072.9427623541496 | 3068.9012837538735 | 649.9058467436236 |
| | +22.69% | +3.92% | -3.84% | -3.67% | -1.02% | <not defined>
hdd_bfq_randread_sync1_prio_256k_antagonized_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 5.182094534557305e+07 | 1.8752826185425484e+08 | 8.760924727551427e+07 | 8.352359803888217e+07 | 1.977023914873974e+07 |
(B) 84f99e060a66 | 481 | 4.868626716238696e+07 | 2.3129365653821653e+08 | 9.622444278248079e+07 | 9.149304520876768e+07 | 2.5072713157997187e+07 |
| | -6.05% | +23.34% | +9.83% | +9.54% | +26.82% | - is good
hdd_bfq_read_antagonist_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 479 | 49141.06748148148 | 92770.28154050466 | 72409.54922214251 | 72788.78264718903 | 7631.977908722226 |
(B) 84f99e060a66 | 477 | 40042.492714937165 | 94590.7180168216 | 69249.11870520914 | 69176.88888888889 | 9833.606205014574 |
| | -18.52% | +1.96% | -4.36% | -4.96% | +28.85% | <not defined>
hdd_bfq_read_antagonist_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 9.835670096201618e+07 | 4.235589828743171e+08 | 1.5040176964768988e+08 | 1.383429005580708e+08 | 4.320986674287423e+07 |
(B) 84f99e060a66 | 481 | 9.417358957137848e+07 | 6.887692560481511e+08 | 1.8459262277609617e+08 | 1.657244056006056e+08 | 7.933471967203863e+07 |
| | -4.25% | +62.61% | +22.73% | +19.79% | +83.60% | <not defined>
hdd_mq_deadline_randread_sync1_8k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 437.3333333333333 | 561.4457724656927 | 495.29163403792217 | 493.95263390880916 | 19.203905318609316 |
(B) 84f99e060a66 | 480 | 433.2952633908809 | 550.3762726870295 | 494.36658941271946 | 494.22222222222223 | 20.681036661081237 |
| | -0.92% | -1.97% | -0.19% | +0.05% | +7.69% | + is good
hdd_mq_deadline_randread_sync1_8k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.4287019730821868e+07 | 1.854003493473249e+07 | 1.624403335564022e+07 | 1.6215042711709704e+07 | 660211.3697913568 |
(B) 84f99e060a66 | 481 | 1.457414547219332e+07 | 1.987693170638872e+07 | 1.6277774871744607e+07 | 1.6237274064249396e+07 | 697562.5497383556 |
| | +2.01% | +7.21% | +0.21% | +0.14% | +5.66% | <not defined>
hdd_mq_deadline_randread_sync1_8k_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 509.26693227091636 | 800 | 645.1839843619289 | 640 | 48.390152611566336 |
(B) 84f99e060a66 | 481 | 541.394422310757 | 800 | 642.3397967382033 | 640 | 46.36162021875187 |
| | +6.31% | +0.00% | -0.44% | +0.00% | -4.19% | + is good
hdd_mq_deadline_randread_sync1_8k_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 9.958389815261044e+06 | 1.5173071762295082e+07 | 1.2445965580579078e+07 | 1.2398376431451613e+07 | 854781.9443522815 |
(B) 84f99e060a66 | 481 | 1.004605020746888e+07 | 1.4730739472118959e+07 | 1.2498833967873506e+07 | 1.2471198e+07 | 835410.5859617554 |
| | +0.88% | -2.92% | +0.42% | +0.59% | -2.27% | <not defined>
hdd_mq_deadline_randread_sync1_8m_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 65404.552 | 131072 | 101739.09871906973 | 98304 | 17121.347317133757 |
(B) 84f99e060a66 | 481 | 32768 | 131072 | 101071.1305132899 | 98304 | 17759.554496706638 |
| | -49.90% | +0.00% | -0.66% | +0.00% | +3.73% | + is good
hdd_mq_deadline_randread_sync1_8m_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 6.42105164229249e+07 | 1.1846633073443983e+08 | 8.12111689093428e+07 | 7.994055613761468e+07 | 8.305693600908015e+06 |
(B) 84f99e060a66 | 481 | 6.3309874666666664e+07 | 1.8769406183846155e+08 | 8.191909383006239e+07 | 8.013581253846154e+07 | 1.035309668574437e+07 |
| | -1.40% | +58.44% | +0.87% | +0.24% | +24.65% | <not defined>
hdd_mq_deadline_randread_sync1_256k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 12288 | 15345.823815847722 | 13955.313638231552 | 13988.24789729969 | 531.5198182748803 |
(B) 84f99e060a66 | 481 | 12726.653386454183 | 15912.910137228862 | 13987.495069078263 | 13989.14386896857 | 502.3380051071074 |
| | +3.57% | +3.70% | +0.23% | +0.01% | -5.49% | + is good
hdd_mq_deadline_randread_sync1_256k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.6587447357276253e+07 | 2.1053193505200885e+07 | 1.843522900376671e+07 | 1.8378568200036444e+07 | 712649.1729512813 |
(B) 84f99e060a66 | 481 | 1.6634398856924305e+07 | 2.0707193470965516e+07 | 1.8385433302425563e+07 | 1.83216160420976e+07 | 652160.1536307137 |
| | +0.28% | -1.64% | -0.27% | -0.31% | -8.49% | <not defined>
hdd_mq_deadline_randread_sync1_prio_256k_antagonized_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 1251.3554670208057 | 4662.554227534307 | 2590.199788013022 | 2614.277556440903 | 467.11616745046547 |
(B) 84f99e060a66 | 480 | 1024 | 4206.53652058433 | 2563.1885636551333 | 2503.1111111111113 | 502.574450952279 |
| | -18.17% | -9.78% | -1.04% | -4.25% | +7.59% | <not defined>
hdd_mq_deadline_randread_sync1_prio_256k_antagonized_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 7.006917667153478e+07 | 2.0013257296266732e+08 | 1.0504890526408787e+08 | 1.0381339802055775e+08 | 1.6038319039515162e+07 |
(B) 84f99e060a66 | 481 | 6.771219982498592e+07 | 2.700827838838999e+08 | 1.0682645729081483e+08 | 1.0264751235651374e+08 | 2.0996399340127923e+07 |
| | -3.36% | +34.95% | +1.69% | -1.12% | +30.91% | - is good
hdd_mq_deadline_read_antagonist_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 54598.84373616645 | 101857.998229305 | 78569.87317444892 | 80041.66799468792 | 7010.904844771771 |
(B) 84f99e060a66 | 480 | 47317.0290530051 | 98304 | 76752.34087065593 | 76458.66666666667 | 9130.285131845274 |
| | -13.34% | -3.49% | -2.31% | -4.48% | +30.23% | <not defined>
hdd_mq_deadline_read_antagonist_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 9.07430549319759e+07 | 2.0507063259337398e+08 | 1.1080574314078984e+08 | 1.0644658997988726e+08 | 1.6111792088606713e+07 |
(B) 84f99e060a66 | 481 | 8.921996962268418e+07 | 2.059040225760676e+08 | 1.1478550705415063e+08 | 1.0669493362907396e+08 | 2.0720287473059524e+07 |
| | -1.68% | +0.41% | +3.59% | +0.23% | +28.60% | <not defined>
hdd_mq_none_randread_sync1_8k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 433.58477202301907 | 547.5555555555555 | 495.651889187694 | 495.44444444444446 | 18.920574696942346 |
(B) 84f99e060a66 | 480 | 408.8888888888889 | 568.5568835768039 | 495.5592125092227 | 495.5502434705622 | 19.850744247491154 |
| | -5.70% | +3.84% | -0.02% | +0.02% | +4.92% | + is good
hdd_mq_none_randread_sync1_8k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.4739365473777534e+07 | 1.8363069459155805e+07 | 1.6220706242792107e+07 | 1.6167072555843754e+07 | 624498.9705654499 |
(B) 84f99e060a66 | 481 | 1.4144933475971542e+07 | 1.999434545073922e+07 | 1.6228009886758411e+07 | 1.6184862094213549e+07 | 655002.1366949107 |
| | -4.03% | +8.88% | +0.05% | +0.11% | +4.88% | <not defined>
hdd_mq_none_randread_sync1_8k_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 480 | 768 | 642.5459059230853 | 640 | 48.349233074838665 |
(B) 84f99e060a66 | 481 | 512 | 864 | 643.8444091409829 | 640 | 48.39906828252791 |
| | +6.67% | +12.50% | +0.20% | +0.00% | +0.10% | + is good
hdd_mq_none_randread_sync1_8k_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.0255825841463415e+07 | 1.6170251333333334e+07 | 1.2493725390487112e+07 | 1.249267448e+07 | 873499.0003406552 |
(B) 84f99e060a66 | 481 | 9.744641596153846e+06 | 1.55867633187251e+07 | 1.2472582618731404e+07 | 1.239832636437247e+07 | 890636.2226532341 |
| | -4.98% | -3.61% | -0.17% | -0.76% | +1.96% | <not defined>
hdd_mq_none_randread_sync1_8m_onedisk_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 65405.593625498004 | 131072 | 101828.76370165072 | 98304 | 17081.367956301114 |
(B) 84f99e060a66 | 481 | 65404.552 | 131072 | 101917.20606140923 | 98304 | 15646.78555952052 |
| | -0.00% | +0.00% | +0.09% | +0.00% | -8.40% | + is good
hdd_mq_none_randread_sync1_8m_onedisk_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 6.4492800356862746e+07 | 1.1046080762445414e+08 | 8.107383330775987e+07 | 8.005707593333334e+07 | 7.835925234954876e+06 |
(B) 84f99e060a66 | 481 | 6.516480875897436e+07 | 1.0510505822274882e+08 | 8.078266977005598e+07 | 8.006642728971963e+07 | 7.525479082183226e+06 |
| | +1.04% | -4.85% | -0.36% | +0.01% | -3.96% | <not defined>
hdd_mq_none_randread_sync1_256k_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 12395.359008410802 | 15466.46303674192 | 13965.324254696627 | 13986.445329791944 | 540.5537898230401 |
(B) 84f99e060a66 | 480 | 12047.32802124834 | 15465.56706507304 | 14005.528092522502 | 13994.666666666666 | 533.422194380433 |
| | -2.81% | -0.01% | +0.29% | +0.06% | -1.32% | + is good
hdd_mq_none_randread_sync1_256k_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1.6759992100125207e+07 | 2.1025788906987265e+07 | 1.841854575650044e+07 | 1.8403686714389365e+07 | 707193.8393873073 |
(B) 84f99e060a66 | 481 | 1.6703674210050877e+07 | 2.1005133391747385e+07 | 1.8354179024851043e+07 | 1.8299694568854973e+07 | 691484.2208106014 |
| | -0.34% | -0.10% | -0.35% | -0.57% | -2.22% | <not defined>
hdd_mq_none_randread_sync1_prio_256k_antagonized_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 1193.9154316069057 | 3404.9694555112883 | 2470.783381806479 | 2501.119079238601 | 352.37169323696077 |
(B) 84f99e060a66 | 480 | 1251.5555555555557 | 3413.3333333333335 | 2462.8524977844877 | 2502.347056219566 | 353.6393090865356 |
| | +4.83% | +0.25% | -0.32% | +0.05% | +0.36% | <not defined>
hdd_mq_none_randread_sync1_prio_256k_antagonized_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 7.92758670781183e+07 | 3.6828972945182586e+08 | 1.2276833274554719e+08 | 1.1244387408005123e+08 | 3.580428866616608e+07 |
(B) 84f99e060a66 | 481 | 8.165940581725675e+07 | 5.55477427650102e+08 | 1.2173782155950747e+08 | 1.1184941037184313e+08 | 3.943540948359279e+07 |
| | +3.01% | +50.83% | -0.84% | -0.53% | +10.14% | - is good
hdd_mq_none_read_antagonist_bw_read | | | | | | |
(A) 4db0b9c9d0f3 | 480 | 50943.45728198318 | 98304 | 77545.46814265505 | 76458.67463479415 | 7768.638181488058 |
(B) 84f99e060a66 | 480 | 54584.33820274458 | 98304 | 78673.59394682551 | 80041.66799468792 | 7623.4260348939215 |
| | +7.15% | +0.00% | +1.45% | +4.69% | -1.87% | <not defined>
hdd_mq_none_read_antagonist_lat_read | | | | | | |
(A) 4db0b9c9d0f3 | 481 | 9.16133477798651e+07 | 3.2367121380585676e+08 | 1.1565995741373444e+08 | 1.0929578126101434e+08 | 2.4595102484695204e+07 |
(B) 84f99e060a66 | 481 | 9.223103197939356e+07 | 2.6360696231197134e+08 | 1.1001900110101567e+08 | 1.0668273801871714e+08 | 1.5779026561421016e+07 |
| | +0.67% | -18.56% | -4.88% | -2.39% | -35.84% | <not defined>[*] BENCHMARK
Name: lmbench
Description: lmbench is a suite of simple, portable, ANSI/C microbenchmarks for UNIX/POSIX. In general, it measures two key features (latency and bandwidth). lmbench is intended to give system developers insight into basic costs of key operations.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 43 machine_total_logical_cores | 12 machine_total_physical_cores | 6 machine_total_ram_gib | 32
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------------------------------------------------+-------+-----------+-----------+-----------+-----------+--------+----------------
development_1_BCOPY_50__BCOPY_libc_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 7805.47 | 7805.47 | 7805.47 | 7805.47 | 0 |
(B) 84f99e060a66 | 1 | 7742.96 | 7742.96 | 7742.96 | 7742.96 | 0 |
| | -0.80% | -0.80% | -0.80% | -0.80% | --- | + is good
development_1_BCOPY_50__BCOPY_memory_read_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 8073.16 | 8073.16 | 8073.16 | 8073.16 | 0 |
(B) 84f99e060a66 | 1 | 8256.44 | 8256.44 | 8256.44 | 8256.44 | 0 |
| | +2.27% | +2.27% | +2.27% | +2.27% | --- | + is good
development_1_BCOPY_50__BCOPY_memory_write_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 6493.75 | 6493.75 | 6493.75 | 6493.75 | 0 |
(B) 84f99e060a66 | 1 | 6516.47 | 6516.47 | 6516.47 | 6516.47 | 0 |
| | +0.35% | +0.35% | +0.35% | +0.35% | --- | + is good
development_1_BCOPY_50__BCOPY_unrolled_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4361.41 | 4361.41 | 4361.41 | 4361.41 | 0 |
(B) 84f99e060a66 | 1 | 4395.6 | 4395.6 | 4395.6 | 4395.6 | 0 |
| | +0.78% | +0.78% | +0.78% | +0.78% | --- | + is good
development_1_CONNECT_50__CONNECT_localhost_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 26.3817 | 26.3817 | 26.3817 | 26.3817 | 0 |
(B) 84f99e060a66 | 1 | 24.6721 | 24.6721 | 24.6721 | 24.6721 | 0 |
| | -6.48% | -6.48% | -6.48% | -6.48% | --- | - is good
development_1_CTX_50__CTX_96P_0K_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 3.18 | 3.18 | 3.18 | 3.18 | 0 |
(B) 84f99e060a66 | 1 | 3.26 | 3.26 | 3.26 | 3.26 | 0 |
| | +2.52% | +2.52% | +2.52% | +2.52% | --- | - is good
development_1_CTX_50__CTX_96P_16K_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 3.64 | 3.64 | 3.64 | 3.64 | 0 |
(B) 84f99e060a66 | 1 | 3.71 | 3.71 | 3.71 | 3.71 | 0 |
| | +1.92% | +1.92% | +1.92% | +1.92% | --- | - is good
development_1_CTX_50__CTX_96P_64K_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 6 | 6 | 6 | 6 | 0 |
(B) 84f99e060a66 | 1 | 6.49 | 6.49 | 6.49 | 6.49 | 0 |
| | +8.17% | +8.17% | +8.17% | +8.17% | --- | - is good
development_1_FILE_50__FILE_read_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4977.81 | 4977.81 | 4977.81 | 4977.81 | 0 |
(B) 84f99e060a66 | 1 | 4916.3 | 4916.3 | 4916.3 | 4916.3 | 0 |
| | -1.24% | -1.24% | -1.24% | -1.24% | --- | + is good
development_1_MMAP_50__MMAP_read_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 9510.52 | 9510.52 | 9510.52 | 9510.52 | 0 |
(B) 84f99e060a66 | 1 | 9538.52 | 9538.52 | 9538.52 | 9538.52 | 0 |
| | +0.29% | +0.29% | +0.29% | +0.29% | --- | + is good
development_1_MMAP_50__MMAP_read_open2close_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4870.85 | 4870.85 | 4870.85 | 4870.85 | 0 |
(B) 84f99e060a66 | 1 | 4841.19 | 4841.19 | 4841.19 | 4841.19 | 0 |
| | -0.61% | -0.61% | -0.61% | -0.61% | --- | + is good
development_1_OPS_50__OPS_double_add_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.4 | 1.4 | 1.4 | 1.4 | 0 |
(B) 84f99e060a66 | 1 | 1.39 | 1.39 | 1.39 | 1.39 | 0 |
| | -0.71% | -0.71% | -0.71% | -0.71% | --- | - is good
development_1_OPS_50__OPS_double_div_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 6.58 | 6.58 | 6.58 | 6.58 | 0 |
(B) 84f99e060a66 | 1 | 6.55 | 6.55 | 6.55 | 6.55 | 0 |
| | -0.46% | -0.46% | -0.46% | -0.46% | --- | - is good
development_1_OPS_50__OPS_double_mul_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.4 | 1.4 | 1.4 | 1.4 | 0 |
(B) 84f99e060a66 | 1 | 1.39 | 1.39 | 1.39 | 1.39 | 0 |
| | -0.71% | -0.71% | -0.71% | -0.71% | --- | - is good
development_1_OPS_50__OPS_float_add_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.4 | 1.4 | 1.4 | 1.4 | 0 |
(B) 84f99e060a66 | 1 | 1.39 | 1.39 | 1.39 | 1.39 | 0 |
| | -0.71% | -0.71% | -0.71% | -0.71% | --- | - is good
development_1_OPS_50__OPS_float_div_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 5.17 | 5.17 | 5.17 | 5.17 | 0 |
(B) 84f99e060a66 | 1 | 5.15 | 5.15 | 5.15 | 5.15 | 0 |
| | -0.39% | -0.39% | -0.39% | -0.39% | --- | - is good
development_1_OPS_50__OPS_float_mul_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.4 | 1.4 | 1.4 | 1.4 | 0 |
(B) 84f99e060a66 | 1 | 1.39 | 1.39 | 1.39 | 1.39 | 0 |
| | -0.71% | -0.71% | -0.71% | -0.71% | --- | - is good
development_1_OPS_50__OPS_int64_bit_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.32 | 0.32 | 0.32 | 0.32 | 0 |
(B) 84f99e060a66 | 1 | 0.31 | 0.31 | 0.31 | 0.31 | 0 |
| | -3.13% | -3.13% | -3.13% | -3.13% | --- | - is good
development_1_OPS_50__OPS_int64_div_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 19.65 | 19.65 | 19.65 | 19.65 | 0 |
(B) 84f99e060a66 | 1 | 19.54 | 19.54 | 19.54 | 19.54 | 0 |
| | -0.56% | -0.56% | -0.56% | -0.56% | --- | - is good
development_1_OPS_50__OPS_int64_mul_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.41 | 1.41 | 1.41 | 1.41 | 0 |
(B) 84f99e060a66 | 1 | 1.41 | 1.41 | 1.41 | 1.41 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | <not defined>
development_1_OPS_50__OPS_integer_bit_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.31 | 0.31 | 0.31 | 0.31 | 0 |
(B) 84f99e060a66 | 1 | 0.31 | 0.31 | 0.31 | 0.31 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | - is good
development_1_OPS_50__OPS_integer_div_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 10.97 | 10.97 | 10.97 | 10.97 | 0 |
(B) 84f99e060a66 | 1 | 10.92 | 10.92 | 10.92 | 10.92 | 0 |
| | -0.46% | -0.46% | -0.46% | -0.46% | --- | - is good
development_1_OPS_50__OPS_integer_mul_latency_ns | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.44 | 1.44 | 1.44 | 1.44 | 0 |
(B) 84f99e060a66 | 1 | 1.44 | 1.44 | 1.44 | 1.44 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | - is good
development_1_PAGEFAULT_50__Pagefaults_ms | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.4384 | 0.4384 | 0.4384 | 0.4384 | 0 |
(B) 84f99e060a66 | 1 | 0.4584 | 0.4584 | 0.4584 | 0.4584 | 0 |
| | +4.56% | +4.56% | +4.56% | +4.56% | --- | - is good
development_1_PIPE_50__PIPE_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2236.41 | 2236.41 | 2236.41 | 2236.41 | 0 |
(B) 84f99e060a66 | 1 | 2230.42 | 2230.42 | 2230.42 | 2230.42 | 0 |
| | -0.27% | -0.27% | -0.27% | -0.27% | --- | + is good
development_1_PIPE_50__PIPE_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 7.646 | 7.646 | 7.646 | 7.646 | 0 |
(B) 84f99e060a66 | 1 | 7.7923 | 7.7923 | 7.7923 | 7.7923 | 0 |
| | +1.91% | +1.91% | +1.91% | +1.91% | --- | - is good
development_1_PROC_50__Process_fork__bin_sh_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1403.875 | 1403.875 | 1403.875 | 1403.875 | 0 |
(B) 84f99e060a66 | 1 | 1408.8684 | 1408.8684 | 1408.8684 | 1408.8684 | 0 |
| | +0.36% | +0.36% | +0.36% | +0.36% | --- | - is good
development_1_PROC_50__Process_fork_execve_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 629.3372 | 629.3372 | 629.3372 | 629.3372 | 0 |
(B) 84f99e060a66 | 1 | 634.5765 | 634.5765 | 634.5765 | 634.5765 | 0 |
| | +0.83% | +0.83% | +0.83% | +0.83% | --- | - is good
development_1_PROC_50__Process_fork_exit_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 209.3123 | 209.3123 | 209.3123 | 209.3123 | 0 |
(B) 84f99e060a66 | 1 | 204.6007 | 204.6007 | 204.6007 | 204.6007 | 0 |
| | -2.25% | -2.25% | -2.25% | -2.25% | --- | - is good
development_1_SELECT_50__Select_100fd_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2.5596 | 2.5596 | 2.5596 | 2.5596 | 0 |
(B) 84f99e060a66 | 1 | 2.641 | 2.641 | 2.641 | 2.641 | 0 |
| | +3.18% | +3.18% | +3.18% | +3.18% | --- | - is good
development_1_SELECT_50__Select_100tcp_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.5837 | 4.5837 | 4.5837 | 4.5837 | 0 |
(B) 84f99e060a66 | 1 | 4.7101 | 4.7101 | 4.7101 | 4.7101 | 0 |
| | +2.76% | +2.76% | +2.76% | +2.76% | --- | - is good
development_1_SYSCALL_50__syscall_fstat_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.8384 | 0.8384 | 0.8384 | 0.8384 | 0 |
(B) 84f99e060a66 | 1 | 0.8248 | 0.8248 | 0.8248 | 0.8248 | 0 |
| | -1.62% | -1.62% | -1.62% | -1.62% | --- | - is good
development_1_SYSCALL_50__syscall_open_close_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 3.7714 | 3.7714 | 3.7714 | 3.7714 | 0 |
(B) 84f99e060a66 | 1 | 3.7521 | 3.7521 | 3.7521 | 3.7521 | 0 |
| | -0.51% | -0.51% | -0.51% | -0.51% | --- | - is good
development_1_SYSCALL_50__syscall_read_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.6679 | 0.6679 | 0.6679 | 0.6679 | 0 |
(B) 84f99e060a66 | 1 | 0.6671 | 0.6671 | 0.6671 | 0.6671 | 0 |
| | -0.12% | -0.12% | -0.12% | -0.12% | --- | - is good
development_1_SYSCALL_50__syscall_stat_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.5253 | 1.5253 | 1.5253 | 1.5253 | 0 |
(B) 84f99e060a66 | 1 | 1.5942 | 1.5942 | 1.5942 | 1.5942 | 0 |
| | +4.52% | +4.52% | +4.52% | +4.52% | --- | - is good
development_1_SYSCALL_50__syscall_syscall_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.6118 | 0.6118 | 0.6118 | 0.6118 | 0 |
(B) 84f99e060a66 | 1 | 0.6114 | 0.6114 | 0.6114 | 0.6114 | 0 |
| | -0.07% | -0.07% | -0.07% | -0.07% | --- | - is good
development_1_SYSCALL_50__syscall_write_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.6313 | 0.6313 | 0.6313 | 0.6313 | 0 |
(B) 84f99e060a66 | 1 | 0.6292 | 0.6292 | 0.6292 | 0.6292 | 0 |
| | -0.33% | -0.33% | -0.33% | -0.33% | --- | - is good
development_1_TCP_50__TCP_localhost_latency | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 14.7063 | 14.7063 | 14.7063 | 14.7063 | 0 |
(B) 84f99e060a66 | 1 | 15.0935 | 15.0935 | 15.0935 | 15.0935 | 0 |
| | +2.63% | +2.63% | +2.63% | +2.63% | --- | - is good
development_1_TCP_50__TCP_socket_bandwidth_10MB_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4387.19 | 4387.19 | 4387.19 | 4387.19 | 0 |
(B) 84f99e060a66 | 1 | 4435.23 | 4435.23 | 4435.23 | 4435.23 | 0 |
| | +1.10% | +1.10% | +1.10% | +1.10% | --- | + is good
development_1_TCP_50__TCP_socket_bandwidth_64B_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 63.26 | 63.26 | 63.26 | 63.26 | 0 |
(B) 84f99e060a66 | 1 | 66.65 | 66.65 | 66.65 | 66.65 | 0 |
| | +5.36% | +5.36% | +5.36% | +5.36% | --- | + is good
development_1_UDP_50__UDP_usinglocalhost_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 11.5949 | 11.5949 | 11.5949 | 11.5949 | 0 |
(B) 84f99e060a66 | 1 | 11.5451 | 11.5451 | 11.5451 | 11.5451 | 0 |
| | -0.43% | -0.43% | -0.43% | -0.43% | --- | - is good
development_1_UNIX_50__AF_UNIX_sock_stream_bandwidth_MB_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 5700.99 | 5700.99 | 5700.99 | 5700.99 | 0 |
(B) 84f99e060a66 | 1 | 5722.12 | 5722.12 | 5722.12 | 5722.12 | 0 |
| | +0.37% | +0.37% | +0.37% | +0.37% | --- | + is good
development_1_UNIX_50__AF_UNIX_sock_stream_latency_us | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 12.9419 | 12.9419 | 12.9419 | 12.9419 | 0 |
(B) 84f99e060a66 | 1 | 12.8052 | 12.8052 | 12.8052 | 12.8052 | 0 |
| | -1.06% | -1.06% | -1.06% | -1.06% | --- | - is good[*] BENCHMARK
Name: parallelio
Description: parallelio measures the impact of IO on the performance of an in-memory workload. The primary worklod in-memory workload. The primary workload is a memcached server with a memcachetest client tuned to use 80% of memory. The test runs multiple times starting with no parallel IO andthen doing IO with dd to a file sized to be 10% of physical memory and finishing with a file 80% of physical memory. The metrics of the test are how much performance is lost by running the IO in parallel and checking if it prematurely pushed to swap. A positive swapout figure is not necessarily bad but high swaptotals imply that the in-memory workload is thrashing.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 10 machine_total_logical_cores | 72 machine_total_physical_cores | 36 machine_total_ram_gib | 256
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------------------+-------+--------+--------+-------------------+--------+--------------------+----------------
parallelio_memcachetest_0M | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 405274 | 414589 | 409674 | 409159 | 3820.2290507245766 |
(B) 84f99e060a66 | 3 | 408722 | 410821 | 409731 | 409650 | 858.825166530806 |
| | +0.85% | -0.91% | +0.01% | +0.12% | -77.52% | <not defined>
parallelio_memcachetest_25784M | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 408843 | 414268 | 411967.6666666667 | 412792 | 2290.167436867645 |
(B) 84f99e060a66 | 1 | 410109 | 410109 | 410109 | 410109 | 0 |
| | +0.31% | -1.00% | -0.45% | -0.65% | -100.00% | <not defined>
parallelio_memcachetest_maxM | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 403039 | 407154 | 405639.3333333333 | 406725 | 1847.0355227288503 |
(B) 84f99e060a66 | 1 | 410109 | 410109 | 410109 | 410109 | 0 |
| | +1.75% | +0.73% | +1.10% | +0.83% | -100.00% | + is good
parallelio_memcachetest_minM | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 405274 | 414589 | 409674 | 409159 | 3820.2290507245766 |
(B) 84f99e060a66 | 3 | 408722 | 410821 | 409731 | 409650 | 858.825166530806 |
| | +0.85% | -0.91% | +0.01% | +0.12% | -77.52% | + is good[*] BENCHMARK
Name: postmark
Description: PostMark is a benchmark that demonstrates system performance for short-lived small files seen typically in Internet applications such as electronic mail, netnews and web-based commerce. The benchmark creates an initial pool of random text files of varying size. Once the pool has been created, the benchmark performs two types of transactions on the pool: (i) create or delete a file; (ii) read from or append to a file. The incidence of each transaction and its subtype are chosen randomly to eliminate the effect of caching and read-ahead.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 43 machine_total_logical_cores | 12 machine_total_physical_cores | 6 machine_total_ram_gib | 32
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
--------------------------+-------+---------+---------+---------+---------+--------+------------
postmark_CreateTransact | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 12 | 12 | 12 | 12 | 0 |
(B) 84f99e060a66 | 1 | 12 | 12 | 12 | 12 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | + is good
postmark_DataRead_MB | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 20.61 | 20.61 | 20.61 | 20.61 | 0 |
(B) 84f99e060a66 | 1 | 19.84 | 19.84 | 19.84 | 19.84 | 0 |
| | -3.74% | -3.74% | -3.74% | -3.74% | --- | + is good
postmark_DataWrite_MB | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 57.81 | 57.81 | 57.81 | 57.81 | 0 |
(B) 84f99e060a66 | 1 | 55.66 | 55.66 | 55.66 | 55.66 | 0 |
| | -3.72% | -3.72% | -3.72% | -3.72% | --- | + is good
postmark_DeleteTransact | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 12 | 12 | 12 | 12 | 0 |
(B) 84f99e060a66 | 1 | 12 | 12 | 12 | 12 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | + is good
postmark_FilesCreate | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 31 | 31 | 31 | 31 | 0 |
(B) 84f99e060a66 | 1 | 29 | 29 | 29 | 29 | 0 |
| | -6.45% | -6.45% | -6.45% | -6.45% | --- | + is good
postmark_FilesDeleted | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 3095 | 3095 | 3095 | 3095 | 0 |
(B) 84f99e060a66 | 1 | 4127 | 4127 | 4127 | 4127 | 0 |
| | +33.34% | +33.34% | +33.34% | +33.34% | --- | + is good
postmark_Transactions | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 24 | 24 | 24 | 24 | 0 |
(B) 84f99e060a66 | 1 | 24 | 24 | 24 | 24 | 0 |
| | +0.00% | +0.00% | +0.00% | +0.00% | --- | + is good[*] BENCHMARK
Name: hackbench
Description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. It's main job is to create a specified number of pairs of schedulable entities (either threads or traditional processes) which communicate via either sockets or pipes and time how long it takes for each pair to send data back and forth.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
--------------------------------+-------+---------+---------+--------------------+--------+---------------------+----------------
hackbench_process_pipes_234 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 3.052 | 3.773 | 3.3674285714285714 | 3.329 | 0.285106925878123 |
(B) 84f99e060a66 | 7 | 2.998 | 3.359 | 3.1277142857142857 | 3.074 | 0.11753262682058513 |
| | -1.77% | -10.97% | -7.12% | -7.66% | -58.78% | <not defined>
hackbench_process_pipes_max | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 3.052 | 3.773 | 3.3674285714285714 | 3.329 | 0.285106925878123 |
(B) 84f99e060a66 | 7 | 2.998 | 3.359 | 3.1277142857142857 | 3.074 | 0.11753262682058513 |
| | -1.77% | -10.97% | -7.12% | -7.66% | -58.78% | - is good
hackbench_process_sockets_234 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 6.156 | 6.36 | 6.284571428571429 | 6.285 | 0.06349771006637552 |
(B) 84f99e060a66 | 7 | 6.194 | 6.806 | 6.408428571428572 | 6.335 | 0.20896606501183543 |
| | +0.62% | +7.01% | +1.97% | +0.80% | +229.09% | <not defined>
hackbench_process_sockets_max | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 6.156 | 6.36 | 6.284571428571429 | 6.285 | 0.06349771006637552 |
(B) 84f99e060a66 | 7 | 6.194 | 6.806 | 6.408428571428572 | 6.335 | 0.20896606501183543 |
| | +0.62% | +7.01% | +1.97% | +0.80% | +229.09% | - is good
hackbench_thread_pipes_234 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 4.222 | 4.651 | 4.377857142857143 | 4.361 | 0.14447879584554812 |
(B) 84f99e060a66 | 7 | 3.71 | 4.328 | 4.072 | 4.105 | 0.20410991436688505 |
| | -12.13% | -6.94% | -6.99% | -5.87% | +41.27% | <not defined>
hackbench_thread_pipes_max | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 4.222 | 4.651 | 4.377857142857143 | 4.361 | 0.14447879584554812 |
(B) 84f99e060a66 | 7 | 3.71 | 4.328 | 4.072 | 4.105 | 0.20410991436688505 |
| | -12.13% | -6.94% | -6.99% | -5.87% | +41.27% | - is good
hackbench_thread_sockets_234 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 6.163 | 6.274 | 6.194857142857143 | 6.18 | 0.03647867702577801 |
(B) 84f99e060a66 | 7 | 5.993 | 6.286 | 6.166857142857143 | 6.235 | 0.10362766945922779 |
| | -2.76% | +0.19% | -0.45% | +0.89% | +184.08% | <not defined>
hackbench_thread_sockets_max | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 6.163 | 6.274 | 6.194857142857143 | 6.18 | 0.03647867702577801 |
(B) 84f99e060a66 | 7 | 5.993 | 6.286 | 6.166857142857143 | 6.235 | 0.10362766945922779 |
| | -2.76% | +0.19% | -0.45% | +0.89% | +184.08% | - is good[*] BENCHMARK
Name: kernbench
Description: kernbench is a simple average of five kernel compiles of the vmlinuxbinary image. kernbench checks out and builds v5.9.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------+-------+---------+---------+--------------------+---------+--------------------+----------------
kernbench_elsp_112 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 51.13 | 53.51 | 52.581999999999994 | 52.64 | 0.7946672259505867 |
(B) 84f99e060a66 | 5 | 51.52 | 53.22 | 52.128 | 52.08 | 0.6266865245080647 |
| | +0.76% | -0.54% | -0.86% | -1.06% | -21.14% | <not defined>
kernbench_elsp_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 51.13 | 53.51 | 52.581999999999994 | 52.64 | 0.7946672259505867 |
(B) 84f99e060a66 | 5 | 51.52 | 53.22 | 52.128 | 52.08 | 0.6266865245080647 |
| | +0.76% | -0.54% | -0.86% | -1.06% | -21.14% | - is good
kernbench_syst_112 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 478.47 | 498.33 | 493.546 | 496.52 | 7.582455011406262 |
(B) 84f99e060a66 | 5 | 479.37 | 496.39 | 491.84200000000004 | 494.62 | 6.357538517382332 |
| | +0.19% | -0.39% | -0.35% | -0.38% | -16.15% | <not defined>
kernbench_syst_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 478.47 | 498.33 | 493.546 | 496.52 | 7.582455011406262 |
(B) 84f99e060a66 | 5 | 479.37 | 496.39 | 491.84200000000004 | 494.62 | 6.357538517382332 |
| | +0.19% | -0.39% | -0.35% | -0.38% | -16.15% | - is good
kernbench_user_112 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 3086.46 | 3091.98 | 3088.51 | 3086.97 | 2.2000000000000863 |
(B) 84f99e060a66 | 5 | 3085.06 | 3092.73 | 3089.646 | 3090.99 | 2.9837466380374402 |
| | -0.05% | +0.02% | +0.04% | +0.13% | +35.62% | <not defined>
kernbench_user_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 3086.46 | 3091.98 | 3088.51 | 3086.97 | 2.2000000000000863 |
(B) 84f99e060a66 | 5 | 3085.06 | 3092.73 | 3089.646 | 3090.99 | 2.9837466380374402 |
| | -0.05% | +0.02% | +0.04% | +0.13% | +35.62% | - is good[*] BENCHMARK
Name: memcached
Description: memcached benchmark drived by mcperf.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
-------------------+-------+---------------+---------------+------------------------+---------------+-------------------+------------
QPS | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 1.0583485e+06 | 1.0695842e+06 | 1.0632277999999998e+06 | 1.0616673e+06 | 3820.724428691447 |
(B) 84f99e060a66 | 5 | 1.0654889e+06 | 1.0724847e+06 | 1.06819692e+06 | 1.0683428e+06 | 2454.762893152803 |
| | +0.67% | +0.27% | +0.47% | +0.63% | -35.75% | + is good[*] BENCHMARK
Name: multichase
Description: multichase is a pointer chaser microbenchmark.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
--------------------------------------------------+-------+--------+--------+--------------------+--------+---------------------+------------
best_ns_per_chase_4_thread_local_2m | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 100.9 | 101.2 | 101 | 100.9 | 0.12649110640673303 |
(B) 84f99e060a66 | 5 | 101 | 101.2 | 101.14 | 101.2 | 0.08000000000000318 |
| | +0.10% | +0.00% | +0.14% | +0.30% | -36.75% | - is good
best_ns_per_chase_4_thread_local_4k | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 103.3 | 103.5 | 103.4 | 103.4 | 0.06324555320336923 |
(B) 84f99e060a66 | 5 | 103.3 | 103.5 | 103.4 | 103.4 | 0.06324555320336793 |
| | +0.00% | +0.00% | +0.00% | +0.00% | -0.00% | - is good
best_ns_per_chase_all_cores | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 388.7 | 389.8 | 389.22 | 389.2 | 0.43081318457077317 |
(B) 84f99e060a66 | 5 | 388.6 | 389.4 | 388.98 | 389 | 0.2993325909419067 |
| | -0.03% | -0.10% | -0.06% | -0.05% | -30.52% | - is good
best_ns_per_chase_all_cores_interleaved | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 321.7 | 322.8 | 322.26 | 322.3 | 0.36110940170536426 |
(B) 84f99e060a66 | 5 | 322.2 | 323.3 | 322.61999999999995 | 322.5 | 0.3762977544445408 |
| | +0.16% | +0.15% | +0.11% | +0.06% | +4.21% | - is good
best_ns_per_chase_all_thread_local_2m | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 172.3 | 211 | 191.51999999999998 | 188.2 | 13.47136221768238 |
(B) 84f99e060a66 | 5 | 173.3 | 206.7 | 193.18 | 193.1 | 11.561729974359364 |
| | +0.58% | -2.04% | +0.87% | +2.60% | -14.18% | - is good
best_ns_per_chase_all_thread_local_4k | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 181 | 187.2 | 183.62 | 182.7 | 2.609521028848008 |
(B) 84f99e060a66 | 5 | 179 | 184 | 182.38 | 182.7 | 1.8148278155241067 |
| | -1.10% | -1.71% | -0.68% | +0.00% | -30.45% | - is good
best_ns_per_chase_one_socket_local | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 173.3 | 173.8 | 173.56 | 173.6 | 0.1854723699099109 |
(B) 84f99e060a66 | 5 | 173.6 | 173.7 | 173.64 | 173.6 | 0.04898979485566097 |
| | +0.17% | -0.06% | +0.05% | +0.00% | -73.59% | - is good
best_ns_per_chase_one_socket_remote | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 254.4 | 255 | 254.57999999999998 | 254.5 | 0.21354156504062177 |
(B) 84f99e060a66 | 5 | 254.2 | 254.9 | 254.48000000000002 | 254.4 | 0.27856776554368984 |
| | -0.08% | -0.04% | -0.04% | -0.04% | +30.45% | - is good
best_ns_per_chase_single_thread_local_0 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 142.8 | 144.8 | 144.20000000000002 | 144.6 | 0.7589466384404142 |
(B) 84f99e060a66 | 5 | 143.3 | 145.1 | 144.12 | 144 | 0.6554387843269566 |
| | +0.35% | +0.21% | -0.06% | -0.41% | -13.64% | - is good
best_ns_per_chase_single_thread_local_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 142.8 | 143.9 | 143.48 | 143.6 | 0.3762977544445269 |
(B) 84f99e060a66 | 5 | 141.7 | 144.4 | 142.84 | 142.3 | 1.098362417419681 |
| | -0.77% | +0.35% | -0.45% | -0.91% | +191.89% | - is good
best_ns_per_chase_single_thread_local_2m | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 69.006 | 69.578 | 69.399 | 69.466 | 0.2022117701816582 |
(B) 84f99e060a66 | 5 | 69.506 | 69.838 | 69.64559999999999 | 69.657 | 0.11971399249879164 |
| | +0.72% | +0.37% | +0.36% | +0.27% | -40.80% | - is good
best_ns_per_chase_single_thread_local_4k | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 69.949 | 70.246 | 70.0564 | 69.999 | 0.11009741141371474 |
(B) 84f99e060a66 | 5 | 69.954 | 70.332 | 70.104 | 70.103 | 0.127741927337898 |
| | +0.01% | +0.12% | +0.07% | +0.15% | +16.03% | - is good
best_ns_per_chase_single_thread_local_50g_2m | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 111.6 | 111.7 | 111.66000000000001 | 111.7 | 0.0489897948556674 |
(B) 84f99e060a66 | 5 | 111 | 111.6 | 111.42 | 111.5 | 0.22271057451320025 |
| | -0.54% | -0.09% | -0.21% | -0.18% | +354.61% | - is good
best_ns_per_chase_single_thread_local_50g_4k | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 141.9 | 144.2 | 142.9 | 142.9 | 0.8173126696681018 |
(B) 84f99e060a66 | 5 | 141.7 | 144.5 | 143.16 | 143.4 | 0.9134549797335416 |
| | -0.14% | +0.21% | +0.18% | +0.35% | +11.76% | - is good
best_ns_per_chase_single_thread_remote | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 248.8 | 256.6 | 252.98000000000002 | 254.4 | 3.3695103501844326 |
(B) 84f99e060a66 | 5 | 247.5 | 259.4 | 252.45999999999998 | 252.3 | 4.070675619599272 |
| | -0.52% | +1.09% | -0.21% | -0.83% | +20.81% | - is good
best_ns_per_chase_single_thread_remote_reversed | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 268.3 | 272.1 | 270.76000000000005 | 271 | 1.3124023773218374 |
(B) 84f99e060a66 | 5 | 251.7 | 270.8 | 265.72 | 268.4 | 7.135656942426554 |
| | -6.19% | -0.48% | -1.86% | -0.96% | +443.71% | - is good[*] BENCHMARK
Name: nginx
Description: nginx web server benchmark.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
-------------------+-------+-----------+-----------+-----------+-----------+--------+------------
requests_per_sec | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 239142.25 | 239142.25 | 239142.25 | 239142.25 | 0 |
(B) 84f99e060a66 | 1 | 240934.99 | 240934.99 | 240934.99 | 240934.99 | 0 |
| | +0.75% | +0.75% | +0.75% | +0.75% | --- | + is good[*] BENCHMARK
Name: pft
Description: pft is a page fault microbenchmark.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------+-------+-----------------+-----------------+------------------------+------------------------+--------------------+----------------
pft_faults_cpu_1 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 484154.455 | 498063.798 | 492351.76326666656 | 492720.2755 | 4126.981613976473 |
(B) 84f99e060a66 | 30 | 482742.695 | 494479.926 | 489330.9015999999 | 489143.3515 | 2618.891200946336 |
| | -0.29% | -0.72% | -0.61% | -0.73% | -36.54% | <not defined>
pft_faults_cpu_4 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 417255.026 | 436288.218 | 428640.1996333333 | 430395.16150000005 | 4746.028941246967 |
(B) 84f99e060a66 | 30 | 415617.241 | 437189.129 | 427759.93749999994 | 429933.131 | 5261.030587409123 |
| | -0.39% | +0.21% | -0.21% | -0.11% | +10.85% | <not defined>
pft_faults_cpu_7 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 372190.395 | 400357.397 | 387933.1743333333 | 387656.1515 | 6724.31140527816 |
(B) 84f99e060a66 | 30 | 369391.438 | 409099.695 | 388038.53069999994 | 387980.819 | 8197.679887071488 |
| | -0.75% | +2.18% | +0.03% | +0.08% | +21.91% | <not defined>
pft_faults_cpu_12 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 309306.251 | 368030.388 | 334522.85160000005 | 334821.131 | 11465.852240019683 |
(B) 84f99e060a66 | 30 | 318117.629 | 391477.132 | 350409.11253333336 | 342996.01749999996 | 19334.051623320604 |
| | +2.85% | +6.37% | +4.75% | +2.44% | +68.62% | <not defined>
pft_faults_cpu_21 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 212079.413 | 240668.262 | 223933.0883666667 | 223797.9645 | 6708.986752443093 |
(B) 84f99e060a66 | 30 | 209471.97 | 253796.094 | 231684.6007 | 232578.2415 | 8994.590384813879 |
| | -1.23% | +5.45% | +3.46% | +3.92% | +34.07% | <not defined>
pft_faults_cpu_30 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 149503.828 | 169490.187 | 157576.0666666667 | 156733.52000000002 | 4880.567067589405 |
(B) 84f99e060a66 | 30 | 146606.506 | 159328.049 | 153893.34433333337 | 153843.4095 | 3658.1395313921544 |
| | -1.94% | -6.00% | -2.34% | -1.84% | -25.05% | <not defined>
pft_faults_cpu_48 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 89186.867 | 105649.211 | 96896.45703333334 | 96477.243 | 3547.1707756648643 |
(B) 84f99e060a66 | 30 | 95000.878 | 104614.989 | 99554.7385 | 99217.7575 | 2777.940310004704 |
| | +6.52% | -0.98% | +2.74% | +2.84% | -21.69% | <not defined>
pft_faults_cpu_79 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 57662.997 | 65575.52 | 61964.21549999999 | 62236.382 | 2375.343252427598 |
(B) 84f99e060a66 | 30 | 59007.184 | 64639.405 | 62038.662000000004 | 62079.3465 | 1445.370658156032 |
| | +2.33% | -1.43% | +0.12% | -0.25% | -39.15% | <not defined>
pft_faults_cpu_110 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 38555.581 | 43263.373 | 41597.62980000001 | 41827.095 | 1083.5797160778825 |
(B) 84f99e060a66 | 30 | 38994.493 | 42748.637 | 40923.156866666664 | 40935.1565 | 803.2742110120612 |
| | +1.14% | -1.19% | -1.62% | -2.13% | -25.87% | <not defined>
pft_faults_cpu_112 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 37132.764 | 42444.162 | 39839.86603333334 | 39874.229 | 1333.3398674946425 |
(B) 84f99e060a66 | 30 | 38258.017 | 41967.179 | 40467.65890000001 | 40828.6995 | 1129.7326503081576 |
| | +3.03% | -1.12% | +1.58% | +2.39% | -15.27% | <not defined>
pft_faults_cpu_max | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 37132.764 | 42444.162 | 39839.86603333334 | 39874.229 | 1333.3398674946425 |
(B) 84f99e060a66 | 30 | 38258.017 | 41967.179 | 40467.65890000001 | 40828.6995 | 1129.7326503081576 |
| | +3.03% | -1.12% | +1.58% | +2.39% | -15.27% | + is good
pft_faults_cpu_min | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 484154.455 | 498063.798 | 492351.76326666656 | 492720.2755 | 4126.981613976473 |
(B) 84f99e060a66 | 30 | 482742.695 | 494479.926 | 489330.9015999999 | 489143.3515 | 2618.891200946336 |
| | -0.29% | -0.72% | -0.61% | -0.73% | -36.54% | <not defined>
pft_faults_sec_1 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 482655.212 | 496337.449 | 490741.9783666667 | 491119.00399999996 | 4087.2010942607367 |
(B) 84f99e060a66 | 30 | 481121.66 | 492991.898 | 487799.8169666667 | 487654.93200000003 | 2614.246674701353 |
| | -0.32% | -0.67% | -0.60% | -0.71% | -36.04% | <not defined>
pft_faults_sec_4 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 1.656675025e+06 | 1.734861822e+06 | 1.7028609316666666e+06 | 1.708249671e+06 | 18396.918176364477 |
(B) 84f99e060a66 | 30 | 1.65463698e+06 | 1.740377412e+06 | 1.6982376496e+06 | 1.7050573065e+06 | 21839.976016046272 |
| | -0.12% | +0.32% | -0.27% | -0.19% | +18.72% | <not defined>
pft_faults_sec_7 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 2.576784434e+06 | 2.781667761e+06 | 2.691555006733333e+06 | 2.688371363e+06 | 49862.74427563325 |
(B) 84f99e060a66 | 30 | 2.542283262e+06 | 2.842252505e+06 | 2.688004269666667e+06 | 2.6849550785e+06 | 60323.253971796265 |
| | -1.34% | +2.18% | -0.13% | -0.13% | +20.98% | <not defined>
pft_faults_sec_12 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.633401181e+06 | 4.374679466e+06 | 3.938532944666667e+06 | 3.939308249e+06 | 149662.241812175 |
(B) 84f99e060a66 | 30 | 3.725404786e+06 | 4.656017688e+06 | 4.1528862671666667e+06 | 4.0556863770000003e+06 | 246622.15667207883 |
| | +2.53% | +6.43% | +5.44% | +2.95% | +64.79% | <not defined>
pft_faults_sec_21 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 4.257593175e+06 | 4.97357222e+06 | 4.576851050233333e+06 | 4.593935882999999e+06 | 168307.08800032237 |
(B) 84f99e060a66 | 30 | 4.171296711e+06 | 5.280228369e+06 | 4.7697711534e+06 | 4.7752310005e+06 | 219077.6547498594 |
| | -2.03% | +6.17% | +4.22% | +3.95% | +30.17% | <not defined>
pft_faults_sec_30 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 4.1342909e+06 | 4.83459836e+06 | 4.389753519166667e+06 | 4.361737706499999e+06 | 178674.78729718164 |
(B) 84f99e060a66 | 30 | 3.876906006e+06 | 4.532188933e+06 | 4.238873766100001e+06 | 4.247939301e+06 | 133754.5309846267 |
| | -6.23% | -6.26% | -3.44% | -2.61% | -25.14% | <not defined>
pft_faults_sec_48 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.92643344e+06 | 4.513617248e+06 | 4.2207080896333335e+06 | 4.1957761735e+06 | 126421.31890762268 |
(B) 84f99e060a66 | 30 | 4.094672788e+06 | 4.456402988e+06 | 4.276243805466667e+06 | 4.27633757e+06 | 108063.75636518569 |
| | +4.28% | -1.27% | +1.32% | +1.92% | -14.52% | <not defined>
pft_faults_sec_79 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.973659764e+06 | 4.432766382e+06 | 4.249352675600002e+06 | 4.2673665505e+06 | 125455.19112125924 |
(B) 84f99e060a66 | 30 | 4.056523429e+06 | 4.393364151e+06 | 4.229541857999999e+06 | 4.223422776e+06 | 93487.38184650465 |
| | +2.09% | -0.89% | -0.47% | -1.03% | -25.48% | <not defined>
pft_faults_sec_110 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.924296032e+06 | 4.323501374e+06 | 4.198218468433334e+06 | 4.1928832964999997e+06 | 83650.67974854974 |
(B) 84f99e060a66 | 30 | 3.869292322e+06 | 4.273456407e+06 | 4.109747521433334e+06 | 4.1008859835e+06 | 85885.92884576393 |
| | -1.40% | -1.16% | -2.11% | -2.19% | +2.67% | <not defined>
pft_faults_sec_112 | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.883455039e+06 | 4.315943652e+06 | 4.1168769454e+06 | 4.14878067e+06 | 118389.0875606618 |
(B) 84f99e060a66 | 30 | 3.912952449e+06 | 4.28290919e+06 | 4.1253468313666666e+06 | 4.1466393205000004e+06 | 98247.69508444188 |
| | +0.76% | -0.77% | +0.21% | -0.05% | -17.01% | <not defined>
pft_faults_sec_max | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 3.883455039e+06 | 4.315943652e+06 | 4.1168769454e+06 | 4.14878067e+06 | 118389.0875606618 |
(B) 84f99e060a66 | 30 | 3.912952449e+06 | 4.28290919e+06 | 4.1253468313666666e+06 | 4.1466393205000004e+06 | 98247.69508444188 |
| | +0.76% | -0.77% | +0.21% | -0.05% | -17.01% | + is good
pft_faults_sec_min | | | | | | |
(A) 4db0b9c9d0f3 | 30 | 482655.212 | 496337.449 | 490741.9783666667 | 491119.00399999996 | 4087.2010942607367 |
(B) 84f99e060a66 | 30 | 481121.66 | 492991.898 | 487799.8169666667 | 487654.93200000003 | 2614.246674701353 |
| | -0.32% | -0.67% | -0.60% | -0.71% | -36.04% | <not defined>[*] BENCHMARK
Name: pgsql-hammerdb
Description: Postgres benchmark drived by hammerdb.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
------------------------+-------+--------+--------+--------+--------+--------+------------
new_orders_per_minute | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 337717 | 337717 | 337717 | 337717 | 0 |
(B) 84f99e060a66 | 1 | 341671 | 341671 | 341671 | 341671 | 0 |
| | +1.17% | +1.17% | +1.17% | +1.17% | --- | + is good[*] BENCHMARK
Name: redis
Description: redis is a single-threaded server in-memory data store. This is a basic configuration that uses one server even if running on a NUMA machine with clients unbound.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
-------------------------------+-------+-----------+-----------+--------------------+--------------------+--------------------+----------------
redis_large_112_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 86121.52 | 97641.95 | 89782.828 | 87288.98 | 4211.594426691152 |
(B) 84f99e060a66 | 5 | 81245.33 | 106947.3 | 99600.116 | 103938.22 | 9595.9089538086 |
| | -5.66% | +9.53% | +10.93% | +19.07% | +127.85% | <not defined>
redis_large_112_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 83982.8 | 99338.41 | 91855.92199999999 | 91971.79 | 4859.882740146716 |
(B) 84f99e060a66 | 5 | 94742.73 | 107902.8 | 102147.482 | 103681.74 | 5328.512361773783 |
| | +12.81% | +8.62% | +11.20% | +12.73% | +9.64% | <not defined>
redis_large_112_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 77050.51 | 103915.53 | 91193.33 | 90605.98 | 8760.399742863337 |
(B) 84f99e060a66 | 5 | 102777.04 | 106855.88 | 105447.25 | 105883.97 | 1508.874453186883 |
| | +33.39% | +2.83% | +15.63% | +16.86% | -82.78% | <not defined>
redis_large_112_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 76444.42 | 105362.97 | 91002.66000000002 | 91349.625 | 8440.749480048564 |
(B) 84f99e060a66 | 10 | 92986.93 | 108131.48 | 103406.56999999999 | 105310.51500000001 | 5454.677779080998 |
| | +21.64% | +2.63% | +13.63% | +15.28% | -35.38% | <not defined>
redis_large_112_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 63462.25 | 67180.82 | 65070.407999999996 | 64373.27 | 1328.4272375768319 |
(B) 84f99e060a66 | 5 | 66256.32 | 67993.45 | 67324.19 | 67227.34 | 637.2648417730207 |
| | +4.40% | +1.21% | +3.46% | +4.43% | -52.03% | <not defined>
redis_large_112_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 28794.13 | 30684.35 | 29872.71 | 29611.64 | 720.4084873181326 |
(B) 84f99e060a66 | 5 | 30024.98 | 30781.64 | 30501.838 | 30559.27 | 260.3957379374709 |
| | +4.27% | +0.32% | +2.11% | +3.20% | -63.85% | <not defined>
redis_large_112_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 20009.64 | 22382.88 | 21281.039999999997 | 21232.28 | 945.9369798881954 |
(B) 84f99e060a66 | 5 | 21256.83 | 22582.08 | 22141.614 | 22271.86 | 471.52600961558824 |
| | +6.23% | +0.89% | +4.04% | +4.90% | -50.15% | <not defined>
redis_large_112_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 15732.15 | 17724.53 | 16962.964 | 17026.55 | 738.9997225872282 |
(B) 84f99e060a66 | 5 | 17638.86 | 18001.54 | 17813.78 | 17760.41 | 132.1847308882529 |
| | +12.12% | +1.56% | +5.02% | +4.31% | -82.11% | <not defined>
redis_large_112_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 69913.45 | 83562.44 | 75068.09 | 73061.49 | 4699.897939789759 |
(B) 84f99e060a66 | 5 | 82621.41 | 86909.67 | 85500.666 | 86207.64 | 1513.6861191224532 |
| | +18.18% | +4.01% | +13.90% | +17.99% | -67.79% | <not defined>
redis_large_112_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 76210.8 | 91588.52 | 81885.012 | 76924.86 | 6672.886753669362 |
(B) 84f99e060a66 | 5 | 97981.58 | 104706.56 | 102659.61 | 103046.04 | 2454.010128585455 |
| | +28.57% | +14.32% | +25.37% | +33.96% | -63.22% | <not defined>
redis_large_112_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 85209.36 | 95981.27 | 88791.15199999999 | 87157.36 | 3883.6930501026986 |
(B) 84f99e060a66 | 5 | 84930.74 | 100740.45 | 91988.232 | 94507.24 | 6083.486538712485 |
| | -0.33% | +4.96% | +3.60% | +8.43% | +56.64% | <not defined>
redis_large_112_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 87508.2 | 100579.34 | 96678.14 | 98047.87 | 4697.426045455105 |
(B) 84f99e060a66 | 5 | 95320.71 | 109097.65 | 104946.528 | 107563.89 | 5187.759965481825 |
| | +8.93% | +8.47% | +8.55% | +9.71% | +10.44% | <not defined>
redis_large_112_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 83984.21 | 88511.24 | 87263.608 | 88101.07 | 1681.5648135162658 |
(B) 84f99e060a66 | 5 | 91495.49 | 108322.41 | 103931.492 | 107452.93 | 6323.713351904558 |
| | +8.94% | +22.38% | +19.10% | +21.97% | +276.06% | <not defined>
redis_large_112_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 75318.22 | 101266.85 | 90612.204 | 99986 | 12424.38900873697 |
(B) 84f99e060a66 | 5 | 91482.94 | 108124.48 | 104229.332 | 107524.57 | 6411.221696748284 |
| | +21.46% | +6.77% | +15.03% | +7.54% | -48.40% | <not defined>
redis_large_max_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 86121.52 | 97641.95 | 89782.828 | 87288.98 | 4211.594426691152 |
(B) 84f99e060a66 | 5 | 81245.33 | 106947.3 | 99600.116 | 103938.22 | 9595.9089538086 |
| | -5.66% | +9.53% | +10.93% | +19.07% | +127.85% | + is good
redis_large_max_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 83982.8 | 99338.41 | 91855.92199999999 | 91971.79 | 4859.882740146716 |
(B) 84f99e060a66 | 5 | 94742.73 | 107902.8 | 102147.482 | 103681.74 | 5328.512361773783 |
| | +12.81% | +8.62% | +11.20% | +12.73% | +9.64% | + is good
redis_large_max_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 77050.51 | 103915.53 | 91193.33 | 90605.98 | 8760.399742863337 |
(B) 84f99e060a66 | 5 | 102777.04 | 106855.88 | 105447.25 | 105883.97 | 1508.874453186883 |
| | +33.39% | +2.83% | +15.63% | +16.86% | -82.78% | + is good
redis_large_max_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 76444.42 | 105362.97 | 91002.66000000002 | 91349.625 | 8440.749480048564 |
(B) 84f99e060a66 | 10 | 92986.93 | 108131.48 | 103406.56999999999 | 105310.51500000001 | 5454.677779080998 |
| | +21.64% | +2.63% | +13.63% | +15.28% | -35.38% | + is good
redis_large_max_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 63462.25 | 67180.82 | 65070.407999999996 | 64373.27 | 1328.4272375768319 |
(B) 84f99e060a66 | 5 | 66256.32 | 67993.45 | 67324.19 | 67227.34 | 637.2648417730207 |
| | +4.40% | +1.21% | +3.46% | +4.43% | -52.03% | + is good
redis_large_max_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 28794.13 | 30684.35 | 29872.71 | 29611.64 | 720.4084873181326 |
(B) 84f99e060a66 | 5 | 30024.98 | 30781.64 | 30501.838 | 30559.27 | 260.3957379374709 |
| | +4.27% | +0.32% | +2.11% | +3.20% | -63.85% | + is good
redis_large_max_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 20009.64 | 22382.88 | 21281.039999999997 | 21232.28 | 945.9369798881954 |
(B) 84f99e060a66 | 5 | 21256.83 | 22582.08 | 22141.614 | 22271.86 | 471.52600961558824 |
| | +6.23% | +0.89% | +4.04% | +4.90% | -50.15% | + is good
redis_large_max_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 15732.15 | 17724.53 | 16962.964 | 17026.55 | 738.9997225872282 |
(B) 84f99e060a66 | 5 | 17638.86 | 18001.54 | 17813.78 | 17760.41 | 132.1847308882529 |
| | +12.12% | +1.56% | +5.02% | +4.31% | -82.11% | + is good
redis_large_max_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 69913.45 | 83562.44 | 75068.09 | 73061.49 | 4699.897939789759 |
(B) 84f99e060a66 | 5 | 82621.41 | 86909.67 | 85500.666 | 86207.64 | 1513.6861191224532 |
| | +18.18% | +4.01% | +13.90% | +17.99% | -67.79% | + is good
redis_large_max_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 76210.8 | 91588.52 | 81885.012 | 76924.86 | 6672.886753669362 |
(B) 84f99e060a66 | 5 | 97981.58 | 104706.56 | 102659.61 | 103046.04 | 2454.010128585455 |
| | +28.57% | +14.32% | +25.37% | +33.96% | -63.22% | + is good
redis_large_max_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 85209.36 | 95981.27 | 88791.15199999999 | 87157.36 | 3883.6930501026986 |
(B) 84f99e060a66 | 5 | 84930.74 | 100740.45 | 91988.232 | 94507.24 | 6083.486538712485 |
| | -0.33% | +4.96% | +3.60% | +8.43% | +56.64% | + is good
redis_large_max_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 87508.2 | 100579.34 | 96678.14 | 98047.87 | 4697.426045455105 |
(B) 84f99e060a66 | 5 | 95320.71 | 109097.65 | 104946.528 | 107563.89 | 5187.759965481825 |
| | +8.93% | +8.47% | +8.55% | +9.71% | +10.44% | + is good
redis_large_max_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 83984.21 | 88511.24 | 87263.608 | 88101.07 | 1681.5648135162658 |
(B) 84f99e060a66 | 5 | 91495.49 | 108322.41 | 103931.492 | 107452.93 | 6323.713351904558 |
| | +8.94% | +22.38% | +19.10% | +21.97% | +276.06% | + is good
redis_large_max_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 75318.22 | 101266.85 | 90612.204 | 99986 | 12424.38900873697 |
(B) 84f99e060a66 | 5 | 91482.94 | 108124.48 | 104229.332 | 107524.57 | 6411.221696748284 |
| | +21.46% | +6.77% | +15.03% | +7.54% | -48.40% | + is good
redis_medium_112_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71464.3 | 105296.41 | 98298.09 | 104887.77 | 13418.040732782114 |
(B) 84f99e060a66 | 5 | 71037.87 | 105764.15 | 88069.89799999999 | 86184.61 | 15326.191434544195 |
| | -0.60% | +0.44% | -10.41% | -17.83% | +14.22% | <not defined>
redis_medium_112_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 89952.33 | 106894.71 | 102742.21399999999 | 105374.08 | 6425.891039034509 |
(B) 84f99e060a66 | 5 | 72035.73 | 108530.5 | 91255.438 | 97238.42 | 16017.995235722103 |
| | -19.92% | +1.53% | -11.18% | -7.72% | +149.27% | <not defined>
redis_medium_112_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 70546.73 | 106202.2 | 98540.89600000001 | 105563.19 | 14009.409711759596 |
(B) 84f99e060a66 | 5 | 99800.4 | 106281.22 | 104831.92599999999 | 106112.05 | 2520.25111201722 |
| | +41.47% | +0.07% | +6.38% | +0.52% | -82.01% | <not defined>
redis_medium_112_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 70596.54 | 109253.8 | 102989.75099999999 | 106360.395 | 10868.068935072552 |
(B) 84f99e060a66 | 10 | 67317.41 | 107284.62 | 99283.43799999998 | 106264.28 | 14661.570625569964 |
| | -4.64% | -1.80% | -3.60% | -0.09% | +34.91% | <not defined>
redis_medium_112_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67344.6 | 69536.2 | 68463.45199999999 | 68535.4 | 852.3302381213465 |
(B) 84f99e060a66 | 5 | 66827.05 | 69237.7 | 68027.628 | 68157.03 | 811.0692465480314 |
| | -0.77% | -0.43% | -0.64% | -0.55% | -4.84% | <not defined>
redis_medium_112_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 30814.74 | 31701.75 | 31275.802 | 31251.95 | 349.8131587805122 |
(B) 84f99e060a66 | 5 | 30562.35 | 31373.53 | 31060.512000000002 | 31065.55 | 298.0781213306358 |
| | -0.82% | -1.04% | -0.69% | -0.60% | -14.79% | <not defined>
redis_medium_112_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 22397.92 | 23195.94 | 22846.742000000002 | 22862.89 | 264.14444346985744 |
(B) 84f99e060a66 | 5 | 22414.99 | 23173.9 | 22770.994 | 22789.43 | 259.2454181350172 |
| | +0.08% | -0.10% | -0.33% | -0.32% | -1.85% | <not defined>
redis_medium_112_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 17964.61 | 18386.75 | 18159.345999999998 | 18146.19 | 145.55850653259614 |
(B) 84f99e060a66 | 5 | 17892.93 | 18372.22 | 18151.72 | 18141.91 | 158.25971072891696 |
| | -0.40% | -0.08% | -0.04% | -0.02% | +8.73% | <not defined>
redis_medium_112_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 82891.24 | 93764.65 | 90304.758 | 91224.23 | 3859.0309716476713 |
(B) 84f99e060a66 | 5 | 82569.56 | 92498.38 | 87018.09199999999 | 88550.43 | 3799.8862641158103 |
| | -0.39% | -1.35% | -3.64% | -2.93% | -1.53% | <not defined>
redis_medium_112_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 70866.7 | 105719.42 | 88936.732 | 86625.09 | 14234.524281637798 |
(B) 84f99e060a66 | 5 | 70656.4 | 82149.02 | 75741.51999999999 | 76091.91 | 4234.288704918454 |
| | -0.30% | -22.30% | -14.84% | -12.16% | -70.25% | <not defined>
redis_medium_112_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 79164.03 | 101245.32 | 93598.43000000001 | 97924.02 | 8205.051511289863 |
(B) 84f99e060a66 | 5 | 82054.65 | 101936.8 | 91002.572 | 86782.96 | 8258.393592532146 |
| | +3.65% | +0.68% | -2.77% | -11.38% | +0.65% | <not defined>
redis_medium_112_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71204.79 | 108873.16 | 99893.516 | 106202.2 | 14385.239455086732 |
(B) 84f99e060a66 | 5 | 107066.38 | 108577.63 | 107833.04000000001 | 107805.09 | 506.20947272843495 |
| | +50.36% | -0.27% | +7.95% | +1.51% | -96.48% | <not defined>
redis_medium_112_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71078.26 | 107169.65 | 98957.382 | 106803.38 | 14032.888516120123 |
(B) 84f99e060a66 | 5 | 71128.81 | 106746.37 | 84412.32400000001 | 78814.63 | 13971.086249029599 |
| | +0.07% | -0.39% | -14.70% | -26.21% | -0.44% | <not defined>
redis_medium_112_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 90555.1 | 108108.11 | 103639.896 | 106849.02 | 6593.160307380974 |
(B) 84f99e060a66 | 5 | 79706.68 | 108601.21 | 100128.342 | 106168.39 | 10971.534908081734 |
| | -11.98% | +0.46% | -3.39% | -0.64% | +66.41% | <not defined>
redis_medium_max_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71464.3 | 105296.41 | 98298.09 | 104887.77 | 13418.040732782114 |
(B) 84f99e060a66 | 5 | 71037.87 | 105764.15 | 88069.89799999999 | 86184.61 | 15326.191434544195 |
| | -0.60% | +0.44% | -10.41% | -17.83% | +14.22% | + is good
redis_medium_max_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 89952.33 | 106894.71 | 102742.21399999999 | 105374.08 | 6425.891039034509 |
(B) 84f99e060a66 | 5 | 72035.73 | 108530.5 | 91255.438 | 97238.42 | 16017.995235722103 |
| | -19.92% | +1.53% | -11.18% | -7.72% | +149.27% | + is good
redis_medium_max_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 70546.73 | 106202.2 | 98540.89600000001 | 105563.19 | 14009.409711759596 |
(B) 84f99e060a66 | 5 | 99800.4 | 106281.22 | 104831.92599999999 | 106112.05 | 2520.25111201722 |
| | +41.47% | +0.07% | +6.38% | +0.52% | -82.01% | + is good
redis_medium_max_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 70596.54 | 109253.8 | 102989.75099999999 | 106360.395 | 10868.068935072552 |
(B) 84f99e060a66 | 10 | 67317.41 | 107284.62 | 99283.43799999998 | 106264.28 | 14661.570625569964 |
| | -4.64% | -1.80% | -3.60% | -0.09% | +34.91% | + is good
redis_medium_max_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67344.6 | 69536.2 | 68463.45199999999 | 68535.4 | 852.3302381213465 |
(B) 84f99e060a66 | 5 | 66827.05 | 69237.7 | 68027.628 | 68157.03 | 811.0692465480314 |
| | -0.77% | -0.43% | -0.64% | -0.55% | -4.84% | + is good
redis_medium_max_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 30814.74 | 31701.75 | 31275.802 | 31251.95 | 349.8131587805122 |
(B) 84f99e060a66 | 5 | 30562.35 | 31373.53 | 31060.512000000002 | 31065.55 | 298.0781213306358 |
| | -0.82% | -1.04% | -0.69% | -0.60% | -14.79% | + is good
redis_medium_max_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 22397.92 | 23195.94 | 22846.742000000002 | 22862.89 | 264.14444346985744 |
(B) 84f99e060a66 | 5 | 22414.99 | 23173.9 | 22770.994 | 22789.43 | 259.2454181350172 |
| | +0.08% | -0.10% | -0.33% | -0.32% | -1.85% | + is good
redis_medium_max_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 17964.61 | 18386.75 | 18159.345999999998 | 18146.19 | 145.55850653259614 |
(B) 84f99e060a66 | 5 | 17892.93 | 18372.22 | 18151.72 | 18141.91 | 158.25971072891696 |
| | -0.40% | -0.08% | -0.04% | -0.02% | +8.73% | + is good
redis_medium_max_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 82891.24 | 93764.65 | 90304.758 | 91224.23 | 3859.0309716476713 |
(B) 84f99e060a66 | 5 | 82569.56 | 92498.38 | 87018.09199999999 | 88550.43 | 3799.8862641158103 |
| | -0.39% | -1.35% | -3.64% | -2.93% | -1.53% | + is good
redis_medium_max_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 70866.7 | 105719.42 | 88936.732 | 86625.09 | 14234.524281637798 |
(B) 84f99e060a66 | 5 | 70656.4 | 82149.02 | 75741.51999999999 | 76091.91 | 4234.288704918454 |
| | -0.30% | -22.30% | -14.84% | -12.16% | -70.25% | + is good
redis_medium_max_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 79164.03 | 101245.32 | 93598.43000000001 | 97924.02 | 8205.051511289863 |
(B) 84f99e060a66 | 5 | 82054.65 | 101936.8 | 91002.572 | 86782.96 | 8258.393592532146 |
| | +3.65% | +0.68% | -2.77% | -11.38% | +0.65% | + is good
redis_medium_max_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71204.79 | 108873.16 | 99893.516 | 106202.2 | 14385.239455086732 |
(B) 84f99e060a66 | 5 | 107066.38 | 108577.63 | 107833.04000000001 | 107805.09 | 506.20947272843495 |
| | +50.36% | -0.27% | +7.95% | +1.51% | -96.48% | + is good
redis_medium_max_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71078.26 | 107169.65 | 98957.382 | 106803.38 | 14032.888516120123 |
(B) 84f99e060a66 | 5 | 71128.81 | 106746.37 | 84412.32400000001 | 78814.63 | 13971.086249029599 |
| | +0.07% | -0.39% | -14.70% | -26.21% | -0.44% | + is good
redis_medium_max_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 90555.1 | 108108.11 | 103639.896 | 106849.02 | 6593.160307380974 |
(B) 84f99e060a66 | 5 | 79706.68 | 108601.21 | 100128.342 | 106168.39 | 10971.534908081734 |
| | -11.98% | +0.46% | -3.39% | -0.64% | +66.41% | + is good
redis_small_112_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71942.45 | 99009.9 | 92823.784 | 97847.36 | 10459.059890851764 |
(B) 84f99e060a66 | 5 | 97656.24 | 103199.18 | 99421.75399999999 | 99009.9 | 1965.130453416262 |
| | +35.74% | +4.23% | +7.11% | +1.19% | -81.21% | <not defined>
redis_small_112_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71428.57 | 100502.52 | 94127.908 | 99403.58 | 11357.9112414764 |
(B) 84f99e060a66 | 5 | 99502.48 | 105263.16 | 101133.712 | 100300.91 | 2088.725085858836 |
| | +39.30% | +4.74% | +7.44% | +0.90% | -81.61% | <not defined>
redis_small_112_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67430.88 | 97465.88 | 80725.542 | 71377.59 | 13650.5478862138 |
(B) 84f99e060a66 | 5 | 97370.98 | 103519.66 | 99722.42199999999 | 99206.34 | 2051.336756560467 |
| | +44.40% | +6.21% | +23.53% | +38.99% | -84.97% | <not defined>
redis_small_112_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 71275.84 | 98039.22 | 84531.177 | 82620.555 | 11398.639723914474 |
(B) 84f99e060a66 | 10 | 71275.84 | 102880.66 | 95015.00399999999 | 98961.13 | 10393.818308459315 |
| | +0.00% | +4.94% | +12.40% | +19.78% | -8.82% | <not defined>
redis_small_112_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 52521.01 | 68166.33 | 60484.19 | 64516.13 | 6610.609484164073 |
(B) 84f99e060a66 | 5 | 52631.58 | 67796.61 | 60568.95 | 64808.82 | 6536.080283922469 |
| | +0.21% | -0.54% | +0.14% | +0.45% | -1.13% | <not defined>
redis_small_112_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 27693.16 | 31152.65 | 29220.967999999997 | 29568.3 | 1266.5903558356993 |
(B) 84f99e060a66 | 5 | 27964.21 | 30892.8 | 29287.316 | 29726.52 | 1136.400625212782 |
| | +0.98% | -0.83% | +0.23% | +0.54% | -10.28% | <not defined>
redis_small_112_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 20964.36 | 22872.83 | 21715.272 | 21588.95 | 642.6844450397102 |
(B) 84f99e060a66 | 5 | 21043.77 | 22670.6 | 21761.058 | 21881.84 | 561.3368975365854 |
| | +0.38% | -0.88% | +0.21% | +1.36% | -12.66% | <not defined>
redis_small_112_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 17424.64 | 18532.25 | 17707.992 | 17537.71 | 416.9294510777572 |
(B) 84f99e060a66 | 5 | 17325.02 | 18315.02 | 17634.145999999997 | 17488.63 | 350.3809420388055 |
| | -0.57% | -1.17% | -0.42% | -0.28% | -15.96% | <not defined>
redis_small_112_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71022.73 | 102564.1 | 89180.122 | 99403.58 | 14342.618805543705 |
(B) 84f99e060a66 | 5 | 71377.59 | 104166.67 | 90567.052 | 98039.22 | 12918.439933028136 |
| | +0.50% | +1.56% | +1.56% | -1.37% | -9.93% | <not defined>
redis_small_112_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 86058.52 | 98231.83 | 94980.186 | 96993.21 | 4506.456414144485 |
(B) 84f99e060a66 | 5 | 88417.33 | 97656.24 | 95486.45 | 97087.38 | 3546.8647583238903 |
| | +2.74% | -0.59% | +0.53% | +0.10% | -21.29% | <not defined>
redis_small_112_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 98039.22 | 100200.4 | 98800.73599999999 | 98716.68 | 774.7318891487538 |
(B) 84f99e060a66 | 5 | 91074.68 | 100603.62 | 97840.432 | 99108.03 | 3437.0745702553527 |
| | -7.10% | +0.40% | -0.97% | +0.40% | +343.65% | <not defined>
redis_small_112_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 66711.14 | 98425.2 | 81154.564 | 71326.68 | 14081.393480563775 |
(B) 84f99e060a66 | 5 | 99403.58 | 105042.02 | 101136.232 | 99900.09 | 2141.9583192153873 |
| | +49.01% | +6.72% | +24.62% | +40.06% | -84.79% | <not defined>
redis_small_112_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 72358.9 | 99800.4 | 93916.536 | 99009.9 | 10782.800109525542 |
(B) 84f99e060a66 | 5 | 99403.58 | 105485.23 | 101242.73 | 100200.4 | 2246.7532169955816 |
| | +37.38% | +5.70% | +7.80% | +1.20% | -79.16% | <not defined>
redis_small_112_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67385.45 | 99900.09 | 81554.86600000001 | 71581.96 | 14338.592272095051 |
(B) 84f99e060a66 | 5 | 71890.73 | 104275.29 | 94837.97600000001 | 99601.6 | 11640.699220503204 |
| | +6.69% | +4.38% | +16.29% | +39.14% | -18.82% | <not defined>
redis_small_max_GET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71942.45 | 99009.9 | 92823.784 | 97847.36 | 10459.059890851764 |
(B) 84f99e060a66 | 5 | 97656.24 | 103199.18 | 99421.75399999999 | 99009.9 | 1965.130453416262 |
| | +35.74% | +4.23% | +7.11% | +1.19% | -81.21% | + is good
redis_small_max_INCR | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71428.57 | 100502.52 | 94127.908 | 99403.58 | 11357.9112414764 |
(B) 84f99e060a66 | 5 | 99502.48 | 105263.16 | 101133.712 | 100300.91 | 2088.725085858836 |
| | +39.30% | +4.74% | +7.44% | +0.90% | -81.61% | + is good
redis_small_max_LPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67430.88 | 97465.88 | 80725.542 | 71377.59 | 13650.5478862138 |
(B) 84f99e060a66 | 5 | 97370.98 | 103519.66 | 99722.42199999999 | 99206.34 | 2051.336756560467 |
| | +44.40% | +6.21% | +23.53% | +38.99% | -84.97% | + is good
redis_small_max_LPUSH | | | | | | |
(A) 4db0b9c9d0f3 | 10 | 71275.84 | 98039.22 | 84531.177 | 82620.555 | 11398.639723914474 |
(B) 84f99e060a66 | 10 | 71275.84 | 102880.66 | 95015.00399999999 | 98961.13 | 10393.818308459315 |
| | +0.00% | +4.94% | +12.40% | +19.78% | -8.82% | + is good
redis_small_max_LRANGE_100 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 52521.01 | 68166.33 | 60484.19 | 64516.13 | 6610.609484164073 |
(B) 84f99e060a66 | 5 | 52631.58 | 67796.61 | 60568.95 | 64808.82 | 6536.080283922469 |
| | +0.21% | -0.54% | +0.14% | +0.45% | -1.13% | + is good
redis_small_max_LRANGE_300 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 27693.16 | 31152.65 | 29220.967999999997 | 29568.3 | 1266.5903558356993 |
(B) 84f99e060a66 | 5 | 27964.21 | 30892.8 | 29287.316 | 29726.52 | 1136.400625212782 |
| | +0.98% | -0.83% | +0.23% | +0.54% | -10.28% | + is good
redis_small_max_LRANGE_500 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 20964.36 | 22872.83 | 21715.272 | 21588.95 | 642.6844450397102 |
(B) 84f99e060a66 | 5 | 21043.77 | 22670.6 | 21761.058 | 21881.84 | 561.3368975365854 |
| | +0.38% | -0.88% | +0.21% | +1.36% | -12.66% | + is good
redis_small_max_LRANGE_600 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 17424.64 | 18532.25 | 17707.992 | 17537.71 | 416.9294510777572 |
(B) 84f99e060a66 | 5 | 17325.02 | 18315.02 | 17634.145999999997 | 17488.63 | 350.3809420388055 |
| | -0.57% | -1.17% | -0.42% | -0.28% | -15.96% | + is good
redis_small_max_MSET_10 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 71022.73 | 102564.1 | 89180.122 | 99403.58 | 14342.618805543705 |
(B) 84f99e060a66 | 5 | 71377.59 | 104166.67 | 90567.052 | 98039.22 | 12918.439933028136 |
| | +0.50% | +1.56% | +1.56% | -1.37% | -9.93% | + is good
redis_small_max_PING_BULK | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 86058.52 | 98231.83 | 94980.186 | 96993.21 | 4506.456414144485 |
(B) 84f99e060a66 | 5 | 88417.33 | 97656.24 | 95486.45 | 97087.38 | 3546.8647583238903 |
| | +2.74% | -0.59% | +0.53% | +0.10% | -21.29% | + is good
redis_small_max_PING_INLINE | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 98039.22 | 100200.4 | 98800.73599999999 | 98716.68 | 774.7318891487538 |
(B) 84f99e060a66 | 5 | 91074.68 | 100603.62 | 97840.432 | 99108.03 | 3437.0745702553527 |
| | -7.10% | +0.40% | -0.97% | +0.40% | +343.65% | + is good
redis_small_max_SADD | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 66711.14 | 98425.2 | 81154.564 | 71326.68 | 14081.393480563775 |
(B) 84f99e060a66 | 5 | 99403.58 | 105042.02 | 101136.232 | 99900.09 | 2141.9583192153873 |
| | +49.01% | +6.72% | +24.62% | +40.06% | -84.79% | + is good
redis_small_max_SET | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 72358.9 | 99800.4 | 93916.536 | 99009.9 | 10782.800109525542 |
(B) 84f99e060a66 | 5 | 99403.58 | 105485.23 | 101242.73 | 100200.4 | 2246.7532169955816 |
| | +37.38% | +5.70% | +7.80% | +1.20% | -79.16% | + is good
redis_small_max_SPOP | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 67385.45 | 99900.09 | 81554.86600000001 | 71581.96 | 14338.592272095051 |
(B) 84f99e060a66 | 5 | 71890.73 | 104275.29 | 94837.97600000001 | 99601.6 | 11640.699220503204 |
| | +6.69% | +4.38% | +16.29% | +39.14% | -18.82% | + is good[*] BENCHMARK
Name: stream
Description: The STREAM benchmark is a simple synthetic benchmark program that measures sustainable memory bandwidth (in MB/s) and the corresponding computation rate for simple vector kernels.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
--------------------------------+-------+----------+----------+--------------------+----------+--------------------+----------------
stream_omp_llcs_add | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 22535.34 | 22843.54 | 22692.140000000003 | 22697.54 | 125.88004872364232 |
(B) 84f99e060a66 | 3 | 22575.5 | 22817.18 | 22708.606666666667 | 22733.14 | 100.1789030795518 |
| | +0.18% | -0.12% | +0.07% | +0.16% | -20.42% | + is good
stream_omp_llcs_cale | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 21177.26 | 21481.86 | 21343.32 | 21370.84 | 125.86580896600515 |
(B) 84f99e060a66 | 3 | 21181.64 | 21482.28 | 21363.833333333332 | 21427.58 | 130.75124099687224 |
| | +0.02% | +0.00% | +0.10% | +0.27% | +3.88% | <not defined>
stream_omp_llcs_copy | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 21672.48 | 21970.32 | 21848.106666666667 | 21901.52 | 127.32348443586098 |
(B) 84f99e060a66 | 3 | 21729.34 | 21947.58 | 21859.36666666667 | 21901.18 | 93.87381790941001 |
| | +0.26% | -0.10% | +0.05% | -0.00% | -26.27% | + is good
stream_omp_llcs_spread_add | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 23957.28 | 24110.2 | 24008.460000000003 | 23957.9 | 71.94148918855313 |
(B) 84f99e060a66 | 3 | 24048.46 | 24176.1 | 24122.753333333334 | 24143.7 | 54.17296086507334 |
| | +0.38% | +0.27% | +0.48% | +0.78% | -24.70% | + is good
stream_omp_llcs_spread_cale | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 22520.3 | 22686.48 | 22588.213333333333 | 22557.86 | 71.15683023356871 |
(B) 84f99e060a66 | 3 | 22492.22 | 22590.84 | 22527.266666666666 | 22498.74 | 45.0318711827762 |
| | -0.12% | -0.42% | -0.27% | -0.26% | -36.71% | <not defined>
stream_omp_llcs_spread_copy | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 22939.62 | 23205.3 | 23077.006666666664 | 23086.1 | 108.65382991664622 |
(B) 84f99e060a66 | 3 | 22990.68 | 23027.42 | 23006.306666666667 | 23000.82 | 15.492676406037758 |
| | +0.22% | -0.77% | -0.31% | -0.37% | -85.74% | + is good
stream_omp_llcs_spread_triad | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 23893.94 | 24038.62 | 23981.6 | 24012.24 | 62.91360637148918 |
(B) 84f99e060a66 | 3 | 24101.42 | 24109.2 | 24105.993333333332 | 24107.36 | 3.3199330649230476 |
| | +0.87% | +0.29% | +0.52% | +0.40% | -94.72% | + is good
stream_omp_llcs_triad | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 22481.2 | 22847.08 | 22687.286666666667 | 22733.58 | 152.91467802521686 |
(B) 84f99e060a66 | 3 | 22552.6 | 22847.1 | 22726.29333333333 | 22779.18 | 125.91084711899201 |
| | +0.32% | +0.00% | +0.17% | +0.20% | -17.66% | + is good
stream_omp_nodes_add | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 43473.82 | 44035.92 | 43819.69333333333 | 43949.34 | 247.11036148966852 |
(B) 84f99e060a66 | 3 | 43966.06 | 44021.64 | 43988.86666666666 | 43978.9 | 23.75969883834887 |
| | +1.13% | -0.03% | +0.39% | +0.07% | -90.38% | + is good
stream_omp_nodes_cale | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 41266.66 | 41827.52 | 41541.420000000006 | 41530.08 | 229.11049968664912 |
(B) 84f99e060a66 | 3 | 41509.96 | 41809.12 | 41676.520000000004 | 41710.48 | 124.46990961674327 |
| | +0.59% | -0.04% | +0.33% | +0.43% | -45.67% | <not defined>
stream_omp_nodes_copy | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 42152.32 | 42453.44 | 42340.46 | 42415.62 | 133.92804734881528 |
(B) 84f99e060a66 | 3 | 42313.68 | 42651.36 | 42475.193333333336 | 42460.54 | 138.2461228710901 |
| | +0.38% | +0.47% | +0.32% | +0.11% | +3.22% | + is good
stream_omp_nodes_spread_add | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 46556.3 | 46572.58 | 46565.64 | 46568.04 | 6.8595237929947395 |
(B) 84f99e060a66 | 3 | 46590.1 | 46628.86 | 46613.94666666666 | 46622.88 | 17.037952406972313 |
| | +0.07% | +0.12% | +0.10% | +0.12% | +148.38% | + is good
stream_omp_nodes_spread_cale | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 43479.12 | 43572.42 | 43516.58666666666 | 43498.22 | 40.24279093480293 |
(B) 84f99e060a66 | 3 | 43421.88 | 43708.9 | 43519.96666666667 | 43429.12 | 133.628733769684 |
| | -0.13% | +0.31% | +0.01% | -0.16% | +232.06% | <not defined>
stream_omp_nodes_spread_copy | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 44680.66 | 44813.96 | 44766.71333333333 | 44805.52 | 60.94637278423656 |
(B) 84f99e060a66 | 3 | 44484.82 | 44774.08 | 44597.08666666667 | 44532.36 | 126.64910299283304 |
| | -0.44% | -0.09% | -0.38% | -0.61% | +107.80% | + is good
stream_omp_nodes_spread_triad | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 46425.72 | 46524.82 | 46476.62 | 46479.32 | 40.50242791076154 |
(B) 84f99e060a66 | 3 | 46525.12 | 46829.82 | 46693.26666666667 | 46724.86 | 126.38334946590818 |
| | +0.21% | +0.66% | +0.47% | +0.53% | +212.04% | + is good
stream_omp_nodes_triad | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 43550.7 | 43982.02 | 43810.44666666666 | 43898.62 | 186.79781963990425 |
(B) 84f99e060a66 | 3 | 43898.6 | 44159.54 | 44019.7 | 44000.96 | 107.34931019806498 |
| | +0.80% | +0.40% | +0.48% | +0.23% | -42.53% | + is good
stream_single_add | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 11777.74 | 11914.54 | 11832.186666666666 | 11804.28 | 59.232010114652255 |
(B) 84f99e060a66 | 3 | 11607.02 | 12019.86 | 11824.220000000001 | 11845.78 | 169.22931503337858 |
| | -1.45% | +0.88% | -0.07% | +0.35% | +185.71% | + is good
stream_single_cale | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 11108.2 | 11173.12 | 11133.539999999999 | 11119.3 | 28.3517759584829 |
(B) 84f99e060a66 | 3 | 10884.56 | 11234.86 | 11068.66 | 11086.56 | 143.56840413777255 |
| | -2.01% | +0.55% | -0.58% | -0.29% | +406.38% | <not defined>
stream_single_copy | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 11396.58 | 11485.98 | 11436.106666666667 | 11425.76 | 37.2234704931653 |
(B) 84f99e060a66 | 3 | 11125.68 | 11495.18 | 11329.753333333334 | 11368.4 | 153.30304657406094 |
| | -2.38% | +0.08% | -0.93% | -0.50% | +311.85% | + is good
stream_single_triad | | | | | | |
(A) 4db0b9c9d0f3 | 3 | 11780.48 | 11898.12 | 11827.6 | 11804.2 | 50.79673480320089 |
(B) 84f99e060a66 | 3 | 11586.2 | 12035.82 | 11835.193333333335 | 11883.56 | 186.71553574592673 |
| | -1.65% | +1.16% | +0.06% | +0.67% | +267.57% | + is good[*] BENCHMARK
Name: sysbenchthread
Description: Sysbench is a suite of microbenchmarks. This runs the thread microbench which has threads loop on a lock, yield, unlock. It is considered to be of questionable value given its vunerability to load balancing and sched_yield decisions.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------+-------+--------+--------+--------------------+--------+----------------------+----------------
sysbenchthread_1 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.71 | 2.76 | 2.742857142857143 | 2.75 | 0.014846149779161799 |
(B) 84f99e060a66 | 7 | 2.67 | 2.74 | 2.7128571428571426 | 2.71 | 0.022497165354319542 |
| | -1.48% | -0.72% | -1.09% | -1.45% | +51.54% | <not defined>
sysbenchthread_4 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.74 | 2.79 | 2.7699999999999996 | 2.77 | 0.015118578920369087 |
(B) 84f99e060a66 | 7 | 2.72 | 2.76 | 2.747142857142857 | 2.75 | 0.012777531299998568 |
| | -0.73% | -1.08% | -0.83% | -0.72% | -15.48% | <not defined>
sysbenchthread_7 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.72 | 2.75 | 2.738571428571429 | 2.74 | 0.011248582677159752 |
(B) 84f99e060a66 | 7 | 2.73 | 2.76 | 2.748571428571428 | 2.75 | 0.008329931278350345 |
| | +0.37% | +0.36% | +0.37% | +0.36% | -25.95% | <not defined>
sysbenchthread_12 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.71 | 2.76 | 2.744285714285714 | 2.75 | 0.016781914463529526 |
(B) 84f99e060a66 | 7 | 2.72 | 2.74 | 2.7271428571428573 | 2.73 | 0.006998542122237587 |
| | +0.37% | -0.72% | -0.62% | -0.73% | -58.30% | <not defined>
sysbenchthread_21 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.79 | 2.7585714285714285 | 2.77 | 0.020995626366712965 |
(B) 84f99e060a66 | 7 | 2.72 | 2.78 | 2.738571428571429 | 2.73 | 0.020995626366712813 |
| | -0.37% | -0.36% | -0.73% | -1.44% | -0.00% | <not defined>
sysbenchthread_30 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.72 | 2.78 | 2.74 | 2.73 | 0.020701966780270524 |
(B) 84f99e060a66 | 7 | 2.71 | 2.76 | 2.734285714285714 | 2.74 | 0.017612611437054292 |
| | -0.37% | -0.72% | -0.21% | +0.37% | -14.92% | <not defined>
sysbenchthread_48 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.77 | 2.7528571428571427 | 2.76 | 0.014846149779161651 |
(B) 84f99e060a66 | 7 | 2.73 | 2.76 | 2.742857142857143 | 2.74 | 0.010301575072754249 |
| | +0.00% | -0.36% | -0.36% | -0.72% | -30.61% | <not defined>
sysbenchthread_79 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.7 | 2.76 | 2.7300000000000004 | 2.73 | 0.019999999999999945 |
(B) 84f99e060a66 | 7 | 2.69 | 2.75 | 2.722857142857143 | 2.72 | 0.02249716535431943 |
| | -0.37% | -0.36% | -0.26% | -0.37% | +12.49% | <not defined>
sysbenchthread_110 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.72 | 2.76 | 2.742857142857143 | 2.74 | 0.013850513878332155 |
(B) 84f99e060a66 | 7 | 2.72 | 2.75 | 2.73 | 2.73 | 0.010690449676496893 |
| | +0.00% | -0.36% | -0.47% | -0.36% | -22.82% | <not defined>
sysbenchthread_141 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.77 | 2.75 | 2.75 | 0.013093073414159504 |
(B) 84f99e060a66 | 7 | 2.71 | 2.75 | 2.73 | 2.73 | 0.013093073414159578 |
| | -0.73% | -0.72% | -0.73% | -0.73% | +0.00% | <not defined>
sysbenchthread_172 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.76 | 2.7471428571428573 | 2.75 | 0.010301575072754037 |
(B) 84f99e060a66 | 7 | 2.7 | 2.76 | 2.73 | 2.73 | 0.021380899352993758 |
| | -1.10% | +0.00% | -0.62% | -0.73% | +107.55% | <not defined>
sysbenchthread_203 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.74 | 2.82 | 2.7671428571428573 | 2.76 | 0.024327694808466156 |
(B) 84f99e060a66 | 7 | 2.67 | 2.77 | 2.727142857142857 | 2.73 | 0.02864276807966222 |
| | -2.55% | -1.77% | -1.45% | -1.09% | +17.74% | <not defined>
sysbenchthread_234 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.74 | 2.78 | 2.7599999999999993 | 2.77 | 0.015118578920368945 |
(B) 84f99e060a66 | 7 | 2.72 | 2.77 | 2.745714285714286 | 2.74 | 0.016781914463529463 |
| | -0.73% | -0.36% | -0.52% | -1.08% | +11.00% | <not defined>
sysbenchthread_265 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.79 | 2.76 | 2.76 | 0.023299294900428702 |
(B) 84f99e060a66 | 7 | 2.71 | 2.77 | 2.737142857142857 | 2.73 | 0.019059520091608894 |
| | -0.73% | -0.72% | -0.83% | -1.09% | -18.20% | <not defined>
sysbenchthread_296 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.79 | 2.7514285714285713 | 2.74 | 0.01958758457257428 |
(B) 84f99e060a66 | 7 | 2.72 | 2.75 | 2.735714285714286 | 2.74 | 0.00903507902905251 |
| | -0.37% | -1.43% | -0.57% | +0.00% | -53.87% | <not defined>
sysbenchthread_327 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.72 | 2.75 | 2.738571428571429 | 2.74 | 0.009897433186107772 |
(B) 84f99e060a66 | 7 | 2.71 | 2.77 | 2.7357142857142858 | 2.74 | 0.017612611437054282 |
| | -0.37% | +0.73% | -0.10% | +0.00% | +77.95% | <not defined>
sysbenchthread_358 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.74 | 2.79 | 2.7614285714285716 | 2.76 | 0.016413036132965755 |
(B) 84f99e060a66 | 7 | 2.71 | 2.76 | 2.7399999999999998 | 2.74 | 0.015118578920369066 |
| | -1.09% | -1.08% | -0.78% | -0.72% | -7.89% | <not defined>
sysbenchthread_389 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.73 | 2.76 | 2.744285714285714 | 2.74 | 0.011780301787479 |
(B) 84f99e060a66 | 7 | 2.71 | 2.73 | 2.724285714285714 | 2.73 | 0.009035079029052493 |
| | -0.73% | -1.09% | -0.73% | -0.36% | -23.30% | <not defined>
sysbenchthread_420 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.74 | 2.79 | 2.76 | 2.76 | 0.017728105208558324 |
(B) 84f99e060a66 | 7 | 2.7 | 2.77 | 2.73 | 2.72 | 0.0239045721866878 |
| | -1.46% | -0.72% | -1.09% | -1.45% | +34.84% | <not defined>
sysbenchthread_451 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.72 | 2.77 | 2.7457142857142856 | 2.75 | 0.018405855323892807 |
(B) 84f99e060a66 | 7 | 2.7 | 2.74 | 2.72 | 2.72 | 0.014142135623731006 |
| | -0.74% | -1.08% | -0.94% | -1.09% | -23.17% | <not defined>
sysbenchthread_482 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.71 | 2.8 | 2.742857142857143 | 2.74 | 0.028139593719417412 |
(B) 84f99e060a66 | 7 | 2.69 | 2.75 | 2.727142857142857 | 2.73 | 0.02050385727772484 |
| | -0.74% | -1.79% | -0.57% | -0.36% | -27.14% | <not defined>
sysbenchthread_512 | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.7 | 2.75 | 2.7399999999999998 | 2.75 | 0.01690308509457031 |
(B) 84f99e060a66 | 7 | 2.71 | 2.75 | 2.7285714285714286 | 2.73 | 0.01245399698154476 |
| | +0.37% | +0.00% | -0.42% | -0.73% | -26.32% | <not defined>
sysbenchthread_max | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.7 | 2.75 | 2.7399999999999998 | 2.75 | 0.01690308509457031 |
(B) 84f99e060a66 | 7 | 2.71 | 2.75 | 2.7285714285714286 | 2.73 | 0.01245399698154476 |
| | +0.37% | +0.00% | -0.42% | -0.73% | -26.32% | - is good
sysbenchthread_min | | | | | | |
(A) 4db0b9c9d0f3 | 7 | 2.71 | 2.76 | 2.742857142857143 | 2.75 | 0.014846149779161799 |
(B) 84f99e060a66 | 7 | 2.67 | 2.74 | 2.7128571428571426 | 2.71 | 0.022497165354319542 |
| | -1.48% | -0.72% | -1.09% | -1.45% | +51.54% | <not defined>[*] BENCHMARK
Name: tpcc_spark
Description: spark sql drived by tpcc.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 256 machine_total_physical_cores | 128 machine_total_ram_gib | 1024
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------------+-------+--------+--------+--------+--------+--------+------------
geo_mean_of_running_time | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 196.95 | 196.95 | 196.95 | 196.95 | 0 |
(B) 84f99e060a66 | 1 | 194.48 | 194.48 | 194.48 | 194.48 | 0 |
| | -1.25% | -1.25% | -1.25% | -1.25% | --- | - is good[*] BENCHMARK
Name: unixbench
Description: UnixBench is the original BYTE UNIX benchmark suite, updated and revised by many people over the years. The purpose of UnixBench is to provide a basic indicator of the performance of a Unix-like system; hence, multiple tests are used to test various aspects of the system's performance. These test results are then compared to the scores from a baseline system to produce an index value, which is generally easier to handle than the raw scores. The entire set of index values is then combined to make an overall index for the system.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
-----------------------------+-------+----------------+----------------+------------------------+----------------+--------------------+----------------
unixbench_dhry2reg_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4.34865736e+07 | 4.39381302e+07 | 4.360061844e+07 | 4.35072825e+07 | 170784.08300336136 |
(B) 84f99e060a66 | 5 | 4.30655821e+07 | 4.41854107e+07 | 4.3621910260000005e+07 | 4.34371903e+07 | 417733.3283639879 |
| | -0.97% | +0.56% | +0.05% | -0.16% | +144.60% | <not defined>
unixbench_dhry2reg_lbr_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4.27938746e+07 | 4.36690393e+07 | 4.316068858e+07 | 4.30693911e+07 | 331395.00245936995 |
(B) 84f99e060a66 | 5 | 4.27925274e+07 | 4.35598498e+07 | 4.3150110199999996e+07 | 4.31818175e+07 | 285228.20294770785 |
| | -0.00% | -0.25% | -0.02% | +0.26% | -13.93% | <not defined>
unixbench_dhry2reg_lbr_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4.27938746e+07 | 4.36690393e+07 | 4.316068858e+07 | 4.30693911e+07 | 331395.00245936995 |
(B) 84f99e060a66 | 5 | 4.27925274e+07 | 4.35598498e+07 | 4.3150110199999996e+07 | 4.31818175e+07 | 285228.20294770785 |
| | -0.00% | -0.25% | -0.02% | +0.26% | -13.93% | + is good
unixbench_dhry2reg_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4.34865736e+07 | 4.39381302e+07 | 4.360061844e+07 | 4.35072825e+07 | 170784.08300336136 |
(B) 84f99e060a66 | 5 | 4.30655821e+07 | 4.41854107e+07 | 4.3621910260000005e+07 | 4.34371903e+07 | 417733.3283639879 |
| | -0.97% | +0.56% | +0.05% | -0.16% | +144.60% | + is good
unixbench_execl_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4727.9 | 4792.1 | 4765.64 | 4768 | 21.35093440578216 |
(B) 84f99e060a66 | 5 | 4723.7 | 4759.4 | 4742.96 | 4741.8 | 11.76343487251919 |
| | -0.09% | -0.68% | -0.48% | -0.55% | -44.90% | <not defined>
unixbench_execl_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4727.9 | 4792.1 | 4765.64 | 4768 | 21.35093440578216 |
(B) 84f99e060a66 | 5 | 4723.7 | 4759.4 | 4742.96 | 4741.8 | 11.76343487251919 |
| | -0.09% | -0.68% | -0.48% | -0.55% | -44.90% | + is good
unixbench_pipe_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 920680.7 | 937292 | 928936.9800000001 | 929920.6 | 6293.150348243733 |
(B) 84f99e060a66 | 5 | 933339.5 | 945690.7 | 938268.84 | 937086.6 | 4363.416421383563 |
| | +1.37% | +0.90% | +1.00% | +0.77% | -30.66% | <not defined>
unixbench_pipe_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 920680.7 | 937292 | 928936.9800000001 | 929920.6 | 6293.150348243733 |
(B) 84f99e060a66 | 5 | 933339.5 | 945690.7 | 938268.84 | 937086.6 | 4363.416421383563 |
| | +1.37% | +0.90% | +1.00% | +0.77% | -30.66% | + is good
unixbench_spawn_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4387.2 | 4430.4 | 4409.04 | 4402.9 | 15.872315521057407 |
(B) 84f99e060a66 | 5 | 4320 | 4426.7 | 4377.78 | 4398.3 | 43.46213984607764 |
| | -1.53% | -0.08% | -0.71% | -0.10% | +173.82% | <not defined>
unixbench_spawn_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 4387.2 | 4430.4 | 4409.04 | 4402.9 | 15.872315521057407 |
(B) 84f99e060a66 | 5 | 4320 | 4426.7 | 4377.78 | 4398.3 | 43.46213984607764 |
| | -1.53% | -0.08% | -0.71% | -0.10% | +173.82% | + is good
unixbench_syscall_1 | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 478285 | 486881.9 | 484686.64 | 485948.1 | 3236.679989495396 |
(B) 84f99e060a66 | 5 | 487976 | 495269.6 | 491429.3 | 491264.2 | 2527.8799022105263 |
| | +2.03% | +1.72% | +1.39% | +1.09% | -21.90% | <not defined>
unixbench_syscall_max | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 478285 | 486881.9 | 484686.64 | 485948.1 | 3236.679989495396 |
(B) 84f99e060a66 | 5 | 487976 | 495269.6 | 491429.3 | 491264.2 | 2527.8799022105263 |
| | +2.03% | +1.72% | +1.39% | +1.09% | -21.90% | + is good[*] BENCHMARK
Name: vm-scalability
Description: The motivation behind this suite is to exercise functions and regions of Linux kernel mm/ which are of interest to us.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
---------------------------------------+-------+------------------+------------------+-------------------+------------------+------------------------+------------
300s_128G_truncate_throughput | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 3.1402485727e+10 | 3.3524240955e+10 | 3.23467699884e+10 | 3.2213360935e+10 | 6.990084677110682e+08 |
(B) 84f99e060a66 | 5 | 3.2437265981e+10 | 3.3540529674e+10 | 3.2928619191e+10 | 3.2833916458e+10 | 3.9777278144953775e+08 |
| | +3.30% | +0.05% | +1.80% | +1.93% | -43.09% | + is good
300s_512G_anon_wx_rand_mt_throughput | | | | | | |
(A) 4db0b9c9d0f3 | 5 | 7.913872e+06 | 8.193942e+06 | 8.019662e+06 | 7.956032e+06 | 105183.50364196849 |
(B) 84f99e060a66 | 5 | 8.110757e+06 | 8.295155e+06 | 8.1804372e+06 | 8.156313e+06 | 62091.47716844879 |
| | +2.49% | +1.24% | +2.00% | +2.52% | -40.97% | + is good[*] BENCHMARK
Name: will-it-scale
Description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
[*] KERNELS
+-----------------+ | BASE KERNEL (A) | +-----------------+
CommitId: 4db0b9c9d0f3ee84d954c94d7f48aeccc348b354
Describe: v5.16-rc7-12-g4db0b9c9d0f3
+-----------------+ | TEST KERNEL (B) | +-----------------+
CommitId: 84f99e060a66b8ae5f1867e60f73e47a77a567aa
Describe: v5.16-rc7-22-g84f99e060a66
[*] TAGS
LABEL | VALUE
-------------------------------+-------------------------------------------
machine_platform_arch | X86_64 machine_total_disks | 0 machine_total_logical_cores | 112 machine_total_physical_cores | 56 machine_total_ram_gib | 384
[*] METRICS
LABEL | COUNT | MIN | MAX | MEAN | MEDIAN | STDDEV | DIRECTION
-----------------------------------+-------+---------------+---------------+---------------+---------------+--------+------------
brk1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.182196e+06 | 1.182196e+06 | 1.182196e+06 | 1.182196e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.180751e+06 | 1.180751e+06 | 1.180751e+06 | 1.180751e+06 | 0 |
| | -0.12% | -0.12% | -0.12% | -0.12% | --- | + is good
brk1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 262491 | 262491 | 262491 | 262491 | 0 |
(B) 84f99e060a66 | 1 | 271219 | 271219 | 271219 | 271219 | 0 |
| | +3.33% | +3.33% | +3.33% | +3.33% | --- | + is good
brk1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.110562 | 1.110562 | 1.110562 | 1.110562 | 0 |
(B) 84f99e060a66 | 1 | 1.118674 | 1.118674 | 1.118674 | 1.118674 | 0 |
| | +0.73% | +0.73% | +0.73% | +0.73% | --- | + is good
context_switch1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 329041 | 329041 | 329041 | 329041 | 0 |
(B) 84f99e060a66 | 1 | 335598 | 335598 | 335598 | 335598 | 0 |
| | +1.99% | +1.99% | +1.99% | +1.99% | --- | + is good
context_switch1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 297899 | 297899 | 297899 | 297899 | 0 |
(B) 84f99e060a66 | 1 | 304690 | 304690 | 304690 | 304690 | 0 |
| | +2.28% | +2.28% | +2.28% | +2.28% | --- | + is good
context_switch1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.093104 | 1.093104 | 1.093104 | 1.093104 | 0 |
(B) 84f99e060a66 | 1 | 1.166822 | 1.166822 | 1.166822 | 1.166822 | 0 |
| | +6.74% | +6.74% | +6.74% | +6.74% | --- | + is good
dup1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 914239 | 914239 | 914239 | 914239 | 0 |
(B) 84f99e060a66 | 1 | 916208 | 916208 | 916208 | 916208 | 0 |
| | +0.22% | +0.22% | +0.22% | +0.22% | --- | + is good
dup1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 319906 | 319906 | 319906 | 319906 | 0 |
(B) 84f99e060a66 | 1 | 313047 | 313047 | 313047 | 313047 | 0 |
| | -2.14% | -2.14% | -2.14% | -2.14% | --- | + is good
dup1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.988697 | 0.988697 | 0.988697 | 0.988697 | 0 |
(B) 84f99e060a66 | 1 | 0.993808 | 0.993808 | 0.993808 | 0.993808 | 0 |
| | +0.52% | +0.52% | +0.52% | +0.52% | --- | + is good
eventfd1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 883174 | 883174 | 883174 | 883174 | 0 |
(B) 84f99e060a66 | 1 | 884553 | 884553 | 884553 | 884553 | 0 |
| | +0.16% | +0.16% | +0.16% | +0.16% | --- | + is good
eventfd1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 840497 | 840497 | 840497 | 840497 | 0 |
(B) 84f99e060a66 | 1 | 839392 | 839392 | 839392 | 839392 | 0 |
| | -0.13% | -0.13% | -0.13% | -0.13% | --- | + is good
eventfd1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.984995 | 0.984995 | 0.984995 | 0.984995 | 0 |
(B) 84f99e060a66 | 1 | 0.992564 | 0.992564 | 0.992564 | 0.992564 | 0 |
| | +0.77% | +0.77% | +0.77% | +0.77% | --- | + is good
futex1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.685172e+06 | 1.685172e+06 | 1.685172e+06 | 1.685172e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.690119e+06 | 1.690119e+06 | 1.690119e+06 | 1.690119e+06 | 0 |
| | +0.29% | +0.29% | +0.29% | +0.29% | --- | + is good
futex1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.682975e+06 | 1.682975e+06 | 1.682975e+06 | 1.682975e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.687864e+06 | 1.687864e+06 | 1.687864e+06 | 1.687864e+06 | 0 |
| | +0.29% | +0.29% | +0.29% | +0.29% | --- | + is good
futex1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.977692 | 0.977692 | 0.977692 | 0.977692 | 0 |
(B) 84f99e060a66 | 1 | 0.992208 | 0.992208 | 0.992208 | 0.992208 | 0 |
| | +1.48% | +1.48% | +1.48% | +1.48% | --- | + is good
futex2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.477896e+06 | 1.477896e+06 | 1.477896e+06 | 1.477896e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.477849e+06 | 1.477849e+06 | 1.477849e+06 | 1.477849e+06 | 0 |
| | -0.00% | -0.00% | -0.00% | -0.00% | --- | + is good
futex2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.475865e+06 | 1.475865e+06 | 1.475865e+06 | 1.475865e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.472128e+06 | 1.472128e+06 | 1.472128e+06 | 1.472128e+06 | 0 |
| | -0.25% | -0.25% | -0.25% | -0.25% | --- | + is good
futex2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.987299 | 0.987299 | 0.987299 | 0.987299 | 0 |
(B) 84f99e060a66 | 1 | 0.998003 | 0.998003 | 0.998003 | 0.998003 | 0 |
| | +1.08% | +1.08% | +1.08% | +1.08% | --- | + is good
futex3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.947335e+06 | 1.947335e+06 | 1.947335e+06 | 1.947335e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.947543e+06 | 1.947543e+06 | 1.947543e+06 | 1.947543e+06 | 0 |
| | +0.01% | +0.01% | +0.01% | +0.01% | --- | + is good
futex3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.94926e+06 | 1.94926e+06 | 1.94926e+06 | 1.94926e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.943119e+06 | 1.943119e+06 | 1.943119e+06 | 1.943119e+06 | 0 |
| | -0.32% | -0.32% | -0.32% | -0.32% | --- | + is good
futex3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.98336 | 0.98336 | 0.98336 | 0.98336 | 0 |
(B) 84f99e060a66 | 1 | 0.993609 | 0.993609 | 0.993609 | 0.993609 | 0 |
| | +1.04% | +1.04% | +1.04% | +1.04% | --- | + is good
futex4_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.663509e+06 | 1.663509e+06 | 1.663509e+06 | 1.663509e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.661453e+06 | 1.661453e+06 | 1.661453e+06 | 1.661453e+06 | 0 |
| | -0.12% | -0.12% | -0.12% | -0.12% | --- | + is good
futex4_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.663546e+06 | 1.663546e+06 | 1.663546e+06 | 1.663546e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.663659e+06 | 1.663659e+06 | 1.663659e+06 | 1.663659e+06 | 0 |
| | +0.01% | +0.01% | +0.01% | +0.01% | --- | + is good
futex4_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.983372 | 0.983372 | 0.983372 | 0.983372 | 0 |
(B) 84f99e060a66 | 1 | 0.994158 | 0.994158 | 0.994158 | 0.994158 | 0 |
| | +1.10% | +1.10% | +1.10% | +1.10% | --- | + is good
getppid1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2.040868e+06 | 2.040868e+06 | 2.040868e+06 | 2.040868e+06 | 0 |
(B) 84f99e060a66 | 1 | 2.036659e+06 | 2.036659e+06 | 2.036659e+06 | 2.036659e+06 | 0 |
| | -0.21% | -0.21% | -0.21% | -0.21% | --- | + is good
getppid1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2.037917e+06 | 2.037917e+06 | 2.037917e+06 | 2.037917e+06 | 0 |
(B) 84f99e060a66 | 1 | 2.035906e+06 | 2.035906e+06 | 2.035906e+06 | 2.035906e+06 | 0 |
| | -0.10% | -0.10% | -0.10% | -0.10% | --- | + is good
getppid1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.983373 | 0.983373 | 0.983373 | 0.983373 | 0 |
(B) 84f99e060a66 | 1 | 0.993657 | 0.993657 | 0.993657 | 0.993657 | 0 |
| | +1.05% | +1.05% | +1.05% | +1.05% | --- | + is good
lock1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.110729e+06 | 1.110729e+06 | 1.110729e+06 | 1.110729e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.10133e+06 | 1.10133e+06 | 1.10133e+06 | 1.10133e+06 | 0 |
| | -0.85% | -0.85% | -0.85% | -0.85% | --- | + is good
lock1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 741007 | 741007 | 741007 | 741007 | 0 |
(B) 84f99e060a66 | 1 | 751044 | 751044 | 751044 | 751044 | 0 |
| | +1.35% | +1.35% | +1.35% | +1.35% | --- | + is good
lock1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.945011 | 0.945011 | 0.945011 | 0.945011 | 0 |
(B) 84f99e060a66 | 1 | 0.968271 | 0.968271 | 0.968271 | 0.968271 | 0 |
| | +2.46% | +2.46% | +2.46% | +2.46% | --- | + is good
lseek1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2.029997e+06 | 2.029997e+06 | 2.029997e+06 | 2.029997e+06 | 0 |
(B) 84f99e060a66 | 1 | 2.029771e+06 | 2.029771e+06 | 2.029771e+06 | 2.029771e+06 | 0 |
| | -0.01% | -0.01% | -0.01% | -0.01% | --- | + is good
lseek1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.874244e+06 | 1.874244e+06 | 1.874244e+06 | 1.874244e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.877163e+06 | 1.877163e+06 | 1.877163e+06 | 1.877163e+06 | 0 |
| | +0.16% | +0.16% | +0.16% | +0.16% | --- | + is good
lseek1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.985154 | 0.985154 | 0.985154 | 0.985154 | 0 |
(B) 84f99e060a66 | 1 | 0.993361 | 0.993361 | 0.993361 | 0.993361 | 0 |
| | +0.83% | +0.83% | +0.83% | +0.83% | --- | + is good
lseek2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 2.026038e+06 | 2.026038e+06 | 2.026038e+06 | 2.026038e+06 | 0 |
(B) 84f99e060a66 | 1 | 2.022826e+06 | 2.022826e+06 | 2.022826e+06 | 2.022826e+06 | 0 |
| | -0.16% | -0.16% | -0.16% | -0.16% | --- | + is good
lseek2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.873717e+06 | 1.873717e+06 | 1.873717e+06 | 1.873717e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.872512e+06 | 1.872512e+06 | 1.872512e+06 | 1.872512e+06 | 0 |
| | -0.06% | -0.06% | -0.06% | -0.06% | --- | + is good
lseek2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.986396 | 0.986396 | 0.986396 | 0.986396 | 0 |
(B) 84f99e060a66 | 1 | 0.993642 | 0.993642 | 0.993642 | 0.993642 | 0 |
| | +0.73% | +0.73% | +0.73% | +0.73% | --- | + is good
malloc1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 97835 | 97835 | 97835 | 97835 | 0 |
(B) 84f99e060a66 | 1 | 98645 | 98645 | 98645 | 98645 | 0 |
| | +0.83% | +0.83% | +0.83% | +0.83% | --- | + is good
malloc1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 41808 | 41808 | 41808 | 41808 | 0 |
(B) 84f99e060a66 | 1 | 42333 | 42333 | 42333 | 42333 | 0 |
| | +1.26% | +1.26% | +1.26% | +1.26% | --- | + is good
malloc1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.306071 | 0.306071 | 0.306071 | 0.306071 | 0 |
(B) 84f99e060a66 | 1 | 0.311208 | 0.311208 | 0.311208 | 0.311208 | 0 |
| | +1.68% | +1.68% | +1.68% | +1.68% | --- | + is good
malloc2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 8.5964713e+07 | 8.5964713e+07 | 8.5964713e+07 | 8.5964713e+07 | 0 |
(B) 84f99e060a66 | 1 | 8.5729456e+07 | 8.5729456e+07 | 8.5729456e+07 | 8.5729456e+07 | 0 |
| | -0.27% | -0.27% | -0.27% | -0.27% | --- | + is good
malloc2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 8.5909382e+07 | 8.5909382e+07 | 8.5909382e+07 | 8.5909382e+07 | 0 |
(B) 84f99e060a66 | 1 | 8.5643483e+07 | 8.5643483e+07 | 8.5643483e+07 | 8.5643483e+07 | 0 |
| | -0.31% | -0.31% | -0.31% | -0.31% | --- | + is good
malloc2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.990744 | 0.990744 | 0.990744 | 0.990744 | 0 |
(B) 84f99e060a66 | 1 | 0.999912 | 0.999912 | 0.999912 | 0.999912 | 0 |
| | +0.93% | +0.93% | +0.93% | +0.93% | --- | + is good
mmap1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 405846 | 405846 | 405846 | 405846 | 0 |
(B) 84f99e060a66 | 1 | 411130 | 411130 | 411130 | 411130 | 0 |
| | +1.30% | +1.30% | +1.30% | +1.30% | --- | + is good
mmap1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 123503 | 123503 | 123503 | 123503 | 0 |
(B) 84f99e060a66 | 1 | 127042 | 127042 | 127042 | 127042 | 0 |
| | +2.87% | +2.87% | +2.87% | +2.87% | --- | + is good
mmap1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.975408 | 0.975408 | 0.975408 | 0.975408 | 0 |
(B) 84f99e060a66 | 1 | 0.978192 | 0.978192 | 0.978192 | 0.978192 | 0 |
| | +0.29% | +0.29% | +0.29% | +0.29% | --- | + is good
mmap2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 358352 | 358352 | 358352 | 358352 | 0 |
(B) 84f99e060a66 | 1 | 363506 | 363506 | 363506 | 363506 | 0 |
| | +1.44% | +1.44% | +1.44% | +1.44% | --- | + is good
mmap2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 110050 | 110050 | 110050 | 110050 | 0 |
(B) 84f99e060a66 | 1 | 113049 | 113049 | 113049 | 113049 | 0 |
| | +2.73% | +2.73% | +2.73% | +2.73% | --- | + is good
mmap2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.973879 | 0.973879 | 0.973879 | 0.973879 | 0 |
(B) 84f99e060a66 | 1 | 0.969964 | 0.969964 | 0.969964 | 0.969964 | 0 |
| | -0.40% | -0.40% | -0.40% | -0.40% | --- | + is good
open1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 204637 | 204637 | 204637 | 204637 | 0 |
(B) 84f99e060a66 | 1 | 206390 | 206390 | 206390 | 206390 | 0 |
| | +0.86% | +0.86% | +0.86% | +0.86% | --- | + is good
open1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 182428 | 182428 | 182428 | 182428 | 0 |
(B) 84f99e060a66 | 1 | 184978 | 184978 | 184978 | 184978 | 0 |
| | +1.40% | +1.40% | +1.40% | +1.40% | --- | + is good
open1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.270073 | 0.270073 | 0.270073 | 0.270073 | 0 |
(B) 84f99e060a66 | 1 | 0.280171 | 0.280171 | 0.280171 | 0.280171 | 0 |
| | +3.74% | +3.74% | +3.74% | +3.74% | --- | + is good
open2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 187624 | 187624 | 187624 | 187624 | 0 |
(B) 84f99e060a66 | 1 | 188959 | 188959 | 188959 | 188959 | 0 |
| | +0.71% | +0.71% | +0.71% | +0.71% | --- | + is good
open2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 175546 | 175546 | 175546 | 175546 | 0 |
(B) 84f99e060a66 | 1 | 178140 | 178140 | 178140 | 178140 | 0 |
| | +1.48% | +1.48% | +1.48% | +1.48% | --- | + is good
open2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.240907 | 0.240907 | 0.240907 | 0.240907 | 0 |
(B) 84f99e060a66 | 1 | 0.246053 | 0.246053 | 0.246053 | 0.246053 | 0 |
| | +2.14% | +2.14% | +2.14% | +2.14% | --- | + is good
page_fault1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 296104 | 296104 | 296104 | 296104 | 0 |
(B) 84f99e060a66 | 1 | 293518 | 293518 | 293518 | 293518 | 0 |
| | -0.87% | -0.87% | -0.87% | -0.87% | --- | + is good
page_fault1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 245302 | 245302 | 245302 | 245302 | 0 |
(B) 84f99e060a66 | 1 | 239501 | 239501 | 239501 | 239501 | 0 |
| | -2.36% | -2.36% | -2.36% | -2.36% | --- | + is good
page_fault1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.591157 | 0.591157 | 0.591157 | 0.591157 | 0 |
(B) 84f99e060a66 | 1 | 0.557408 | 0.557408 | 0.557408 | 0.557408 | 0 |
| | -5.71% | -5.71% | -5.71% | -5.71% | --- | + is good
page_fault2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 263242 | 263242 | 263242 | 263242 | 0 |
(B) 84f99e060a66 | 1 | 265093 | 265093 | 265093 | 265093 | 0 |
| | +0.70% | +0.70% | +0.70% | +0.70% | --- | + is good
page_fault2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 209335 | 209335 | 209335 | 209335 | 0 |
(B) 84f99e060a66 | 1 | 204871 | 204871 | 204871 | 204871 | 0 |
| | -2.13% | -2.13% | -2.13% | -2.13% | --- | + is good
page_fault2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.583137 | 0.583137 | 0.583137 | 0.583137 | 0 |
(B) 84f99e060a66 | 1 | 0.588389 | 0.588389 | 0.588389 | 0.588389 | 0 |
| | +0.90% | +0.90% | +0.90% | +0.90% | --- | + is good
page_fault3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 649712 | 649712 | 649712 | 649712 | 0 |
(B) 84f99e060a66 | 1 | 652223 | 652223 | 652223 | 652223 | 0 |
| | +0.39% | +0.39% | +0.39% | +0.39% | --- | + is good
page_fault3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 490220 | 490220 | 490220 | 490220 | 0 |
(B) 84f99e060a66 | 1 | 497988 | 497988 | 497988 | 497988 | 0 |
| | +1.58% | +1.58% | +1.58% | +1.58% | --- | + is good
page_fault3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.985479 | 0.985479 | 0.985479 | 0.985479 | 0 |
(B) 84f99e060a66 | 1 | 0.998238 | 0.998238 | 0.998238 | 0.998238 | 0 |
| | +1.29% | +1.29% | +1.29% | +1.29% | --- | + is good
pipe1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 768258 | 768258 | 768258 | 768258 | 0 |
(B) 84f99e060a66 | 1 | 767503 | 767503 | 767503 | 767503 | 0 |
| | -0.10% | -0.10% | -0.10% | -0.10% | --- | + is good
pipe1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 735575 | 735575 | 735575 | 735575 | 0 |
(B) 84f99e060a66 | 1 | 734427 | 734427 | 734427 | 734427 | 0 |
| | -0.16% | -0.16% | -0.16% | -0.16% | --- | + is good
pipe1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.992941 | 0.992941 | 0.992941 | 0.992941 | 0 |
(B) 84f99e060a66 | 1 | 0.99995 | 0.99995 | 0.99995 | 0.99995 | 0 |
| | +0.71% | +0.71% | +0.71% | +0.71% | --- | + is good
poll1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.704129e+06 | 1.704129e+06 | 1.704129e+06 | 1.704129e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.701779e+06 | 1.701779e+06 | 1.701779e+06 | 1.701779e+06 | 0 |
| | -0.14% | -0.14% | -0.14% | -0.14% | --- | + is good
poll1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.626484e+06 | 1.626484e+06 | 1.626484e+06 | 1.626484e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.625619e+06 | 1.625619e+06 | 1.625619e+06 | 1.625619e+06 | 0 |
| | -0.05% | -0.05% | -0.05% | -0.05% | --- | + is good
poll1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.98568 | 0.98568 | 0.98568 | 0.98568 | 0 |
(B) 84f99e060a66 | 1 | 0.993592 | 0.993592 | 0.993592 | 0.993592 | 0 |
| | +0.80% | +0.80% | +0.80% | +0.80% | --- | + is good
poll2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 520641 | 520641 | 520641 | 520641 | 0 |
(B) 84f99e060a66 | 1 | 515893 | 515893 | 515893 | 515893 | 0 |
| | -0.91% | -0.91% | -0.91% | -0.91% | --- | + is good
poll2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 201450 | 201450 | 201450 | 201450 | 0 |
(B) 84f99e060a66 | 1 | 201523 | 201523 | 201523 | 201523 | 0 |
| | +0.04% | +0.04% | +0.04% | +0.04% | --- | + is good
poll2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.996518 | 0.996518 | 0.996518 | 0.996518 | 0 |
(B) 84f99e060a66 | 1 | 1.018321 | 1.018321 | 1.018321 | 1.018321 | 0 |
| | +2.19% | +2.19% | +2.19% | +2.19% | --- | + is good
posix_semaphore1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.16846e+07 | 4.16846e+07 | 4.16846e+07 | 4.16846e+07 | 0 |
(B) 84f99e060a66 | 1 | 4.153911e+07 | 4.153911e+07 | 4.153911e+07 | 4.153911e+07 | 0 |
| | -0.35% | -0.35% | -0.35% | -0.35% | --- | + is good
posix_semaphore1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.1673991e+07 | 4.1673991e+07 | 4.1673991e+07 | 4.1673991e+07 | 0 |
(B) 84f99e060a66 | 1 | 4.1539286e+07 | 4.1539286e+07 | 4.1539286e+07 | 4.1539286e+07 | 0 |
| | -0.32% | -0.32% | -0.32% | -0.32% | --- | + is good
posix_semaphore1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.990928 | 0.990928 | 0.990928 | 0.990928 | 0 |
(B) 84f99e060a66 | 1 | 0.998181 | 0.998181 | 0.998181 | 0.998181 | 0 |
| | +0.73% | +0.73% | +0.73% | +0.73% | --- | + is good
pread1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 964152 | 964152 | 964152 | 964152 | 0 |
(B) 84f99e060a66 | 1 | 961951 | 961951 | 961951 | 961951 | 0 |
| | -0.23% | -0.23% | -0.23% | -0.23% | --- | + is good
pread1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 942991 | 942991 | 942991 | 942991 | 0 |
(B) 84f99e060a66 | 1 | 941389 | 941389 | 941389 | 941389 | 0 |
| | -0.17% | -0.17% | -0.17% | -0.17% | --- | + is good
pread1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.430538 | 0.430538 | 0.430538 | 0.430538 | 0 |
(B) 84f99e060a66 | 1 | 0.442705 | 0.442705 | 0.442705 | 0.442705 | 0 |
| | +2.83% | +2.83% | +2.83% | +2.83% | --- | + is good
pread2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 461091 | 461091 | 461091 | 461091 | 0 |
(B) 84f99e060a66 | 1 | 469057 | 469057 | 469057 | 469057 | 0 |
| | +1.73% | +1.73% | +1.73% | +1.73% | --- | + is good
pread2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 458218 | 458218 | 458218 | 458218 | 0 |
(B) 84f99e060a66 | 1 | 462848 | 462848 | 462848 | 462848 | 0 |
| | +1.01% | +1.01% | +1.01% | +1.01% | --- | + is good
pread2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.072925 | 0.072925 | 0.072925 | 0.072925 | 0 |
(B) 84f99e060a66 | 1 | 0.074731 | 0.074731 | 0.074731 | 0.074731 | 0 |
| | +2.48% | +2.48% | +2.48% | +2.48% | --- | + is good
pread3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 957692 | 957692 | 957692 | 957692 | 0 |
(B) 84f99e060a66 | 1 | 952552 | 952552 | 952552 | 952552 | 0 |
| | -0.54% | -0.54% | -0.54% | -0.54% | --- | + is good
pread3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 938537 | 938537 | 938537 | 938537 | 0 |
(B) 84f99e060a66 | 1 | 928731 | 928731 | 928731 | 928731 | 0 |
| | -1.04% | -1.04% | -1.04% | -1.04% | --- | + is good
pread3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.431635 | 0.431635 | 0.431635 | 0.431635 | 0 |
(B) 84f99e060a66 | 1 | 0.437535 | 0.437535 | 0.437535 | 0.437535 | 0 |
| | +1.37% | +1.37% | +1.37% | +1.37% | --- | + is good
pthread_mutex1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.4554995e+07 | 4.4554995e+07 | 4.4554995e+07 | 4.4554995e+07 | 0 |
(B) 84f99e060a66 | 1 | 4.4471991e+07 | 4.4471991e+07 | 4.4471991e+07 | 4.4471991e+07 | 0 |
| | -0.19% | -0.19% | -0.19% | -0.19% | --- | + is good
pthread_mutex1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.0863168e+07 | 1.0863168e+07 | 1.0863168e+07 | 1.0863168e+07 | 0 |
(B) 84f99e060a66 | 1 | 1.0721714e+07 | 1.0721714e+07 | 1.0721714e+07 | 1.0721714e+07 | 0 |
| | -1.30% | -1.30% | -1.30% | -1.30% | --- | + is good
pthread_mutex1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.957534 | 0.957534 | 0.957534 | 0.957534 | 0 |
(B) 84f99e060a66 | 1 | 0.971139 | 0.971139 | 0.971139 | 0.971139 | 0 |
| | +1.42% | +1.42% | +1.42% | +1.42% | --- | + is good
pthread_mutex2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.4283917e+07 | 4.4283917e+07 | 4.4283917e+07 | 4.4283917e+07 | 0 |
(B) 84f99e060a66 | 1 | 4.4068084e+07 | 4.4068084e+07 | 4.4068084e+07 | 4.4068084e+07 | 0 |
| | -0.49% | -0.49% | -0.49% | -0.49% | --- | + is good
pthread_mutex2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 4.5017533e+07 | 4.5017533e+07 | 4.5017533e+07 | 4.5017533e+07 | 0 |
(B) 84f99e060a66 | 1 | 4.4966205e+07 | 4.4966205e+07 | 4.4966205e+07 | 4.4966205e+07 | 0 |
| | -0.11% | -0.11% | -0.11% | -0.11% | --- | + is good
pthread_mutex2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.976762 | 0.976762 | 0.976762 | 0.976762 | 0 |
(B) 84f99e060a66 | 1 | 0.979919 | 0.979919 | 0.979919 | 0.979919 | 0 |
| | +0.32% | +0.32% | +0.32% | +0.32% | --- | + is good
pwrite1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 842395 | 842395 | 842395 | 842395 | 0 |
(B) 84f99e060a66 | 1 | 858472 | 858472 | 858472 | 858472 | 0 |
| | +1.91% | +1.91% | +1.91% | +1.91% | --- | + is good
pwrite1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 793710 | 793710 | 793710 | 793710 | 0 |
(B) 84f99e060a66 | 1 | 803656 | 803656 | 803656 | 803656 | 0 |
| | +1.25% | +1.25% | +1.25% | +1.25% | --- | + is good
pwrite1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.52403 | 0.52403 | 0.52403 | 0.52403 | 0 |
(B) 84f99e060a66 | 1 | 0.548598 | 0.548598 | 0.548598 | 0.548598 | 0 |
| | +4.69% | +4.69% | +4.69% | +4.69% | --- | + is good
pwrite2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 282885 | 282885 | 282885 | 282885 | 0 |
(B) 84f99e060a66 | 1 | 280819 | 280819 | 280819 | 280819 | 0 |
| | -0.73% | -0.73% | -0.73% | -0.73% | --- | + is good
pwrite2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 264000 | 264000 | 264000 | 264000 | 0 |
(B) 84f99e060a66 | 1 | 259404 | 259404 | 259404 | 259404 | 0 |
| | -1.74% | -1.74% | -1.74% | -1.74% | --- | + is good
pwrite2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.038412 | 0.038412 | 0.038412 | 0.038412 | 0 |
(B) 84f99e060a66 | 1 | 0.038888 | 0.038888 | 0.038888 | 0.038888 | 0 |
| | +1.24% | +1.24% | +1.24% | +1.24% | --- | + is good
pwrite3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 323618 | 323618 | 323618 | 323618 | 0 |
(B) 84f99e060a66 | 1 | 330871 | 330871 | 330871 | 330871 | 0 |
| | +2.24% | +2.24% | +2.24% | +2.24% | --- | + is good
pwrite3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 300116 | 300116 | 300116 | 300116 | 0 |
(B) 84f99e060a66 | 1 | 305806 | 305806 | 305806 | 305806 | 0 |
| | +1.90% | +1.90% | +1.90% | +1.90% | --- | + is good
pwrite3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.062845 | 0.062845 | 0.062845 | 0.062845 | 0 |
(B) 84f99e060a66 | 1 | 0.068264 | 0.068264 | 0.068264 | 0.068264 | 0 |
| | +8.62% | +8.62% | +8.62% | +8.62% | --- | + is good
read1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 849497 | 849497 | 849497 | 849497 | 0 |
(B) 84f99e060a66 | 1 | 851557 | 851557 | 851557 | 851557 | 0 |
| | +0.24% | +0.24% | +0.24% | +0.24% | --- | + is good
read1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 825148 | 825148 | 825148 | 825148 | 0 |
(B) 84f99e060a66 | 1 | 825696 | 825696 | 825696 | 825696 | 0 |
| | +0.07% | +0.07% | +0.07% | +0.07% | --- | + is good
read1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.492216 | 0.492216 | 0.492216 | 0.492216 | 0 |
(B) 84f99e060a66 | 1 | 0.504087 | 0.504087 | 0.504087 | 0.504087 | 0 |
| | +2.41% | +2.41% | +2.41% | +2.41% | --- | + is good
read2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 804636 | 804636 | 804636 | 804636 | 0 |
(B) 84f99e060a66 | 1 | 835800 | 835800 | 835800 | 835800 | 0 |
| | +3.87% | +3.87% | +3.87% | +3.87% | --- | + is good
read2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 782155 | 782155 | 782155 | 782155 | 0 |
(B) 84f99e060a66 | 1 | 813638 | 813638 | 813638 | 813638 | 0 |
| | +4.03% | +4.03% | +4.03% | +4.03% | --- | + is good
read2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.474276 | 0.474276 | 0.474276 | 0.474276 | 0 |
(B) 84f99e060a66 | 1 | 0.509206 | 0.509206 | 0.509206 | 0.509206 | 0 |
| | +7.36% | +7.36% | +7.36% | +7.36% | --- | + is good
readseek1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 700211 | 700211 | 700211 | 700211 | 0 |
(B) 84f99e060a66 | 1 | 702424 | 702424 | 702424 | 702424 | 0 |
| | +0.32% | +0.32% | +0.32% | +0.32% | --- | + is good
readseek1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 666974 | 666974 | 666974 | 666974 | 0 |
(B) 84f99e060a66 | 1 | 669790 | 669790 | 669790 | 669790 | 0 |
| | +0.42% | +0.42% | +0.42% | +0.42% | --- | + is good
readseek1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.709055 | 0.709055 | 0.709055 | 0.709055 | 0 |
(B) 84f99e060a66 | 1 | 0.723066 | 0.723066 | 0.723066 | 0.723066 | 0 |
| | +1.98% | +1.98% | +1.98% | +1.98% | --- | + is good
readseek2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 359075 | 359075 | 359075 | 359075 | 0 |
(B) 84f99e060a66 | 1 | 362790 | 362790 | 362790 | 362790 | 0 |
| | +1.03% | +1.03% | +1.03% | +1.03% | --- | + is good
readseek2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 353384 | 353384 | 353384 | 353384 | 0 |
(B) 84f99e060a66 | 1 | 358400 | 358400 | 358400 | 358400 | 0 |
| | +1.42% | +1.42% | +1.42% | +1.42% | --- | + is good
readseek2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.121419 | 0.121419 | 0.121419 | 0.121419 | 0 |
(B) 84f99e060a66 | 1 | 0.124333 | 0.124333 | 0.124333 | 0.124333 | 0 |
| | +2.40% | +2.40% | +2.40% | +2.40% | --- | + is good
readseek3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 421886 | 421886 | 421886 | 421886 | 0 |
(B) 84f99e060a66 | 1 | 414691 | 414691 | 414691 | 414691 | 0 |
| | -1.71% | -1.71% | -1.71% | -1.71% | --- | + is good
readseek3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 400726 | 400726 | 400726 | 400726 | 0 |
(B) 84f99e060a66 | 1 | 397327 | 397327 | 397327 | 397327 | 0 |
| | -0.85% | -0.85% | -0.85% | -0.85% | --- | + is good
readseek3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.195407 | 0.195407 | 0.195407 | 0.195407 | 0 |
(B) 84f99e060a66 | 1 | 0.20046 | 0.20046 | 0.20046 | 0.20046 | 0 |
| | +2.59% | +2.59% | +2.59% | +2.59% | --- | + is good
sched_yield_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.426126e+06 | 1.426126e+06 | 1.426126e+06 | 1.426126e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.428767e+06 | 1.428767e+06 | 1.428767e+06 | 1.428767e+06 | 0 |
| | +0.19% | +0.19% | +0.19% | +0.19% | --- | + is good
sched_yield_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.356631e+06 | 1.356631e+06 | 1.356631e+06 | 1.356631e+06 | 0 |
(B) 84f99e060a66 | 1 | 1.357941e+06 | 1.357941e+06 | 1.357941e+06 | 1.357941e+06 | 0 |
| | +0.10% | +0.10% | +0.10% | +0.10% | --- | + is good
sched_yield_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 1.007993 | 1.007993 | 1.007993 | 1.007993 | 0 |
(B) 84f99e060a66 | 1 | 1.014633 | 1.014633 | 1.014633 | 1.014633 | 0 |
| | +0.66% | +0.66% | +0.66% | +0.66% | --- | + is good
signal1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 226669 | 226669 | 226669 | 226669 | 0 |
(B) 84f99e060a66 | 1 | 227201 | 227201 | 227201 | 227201 | 0 |
| | +0.23% | +0.23% | +0.23% | +0.23% | --- | + is good
signal1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 93562 | 93562 | 93562 | 93562 | 0 |
(B) 84f99e060a66 | 1 | 95743 | 95743 | 95743 | 95743 | 0 |
| | +2.33% | +2.33% | +2.33% | +2.33% | --- | + is good
signal1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.957449 | 0.957449 | 0.957449 | 0.957449 | 0 |
(B) 84f99e060a66 | 1 | 0.963292 | 0.963292 | 0.963292 | 0.963292 | 0 |
| | +0.61% | +0.61% | +0.61% | +0.61% | --- | + is good
unix1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 522468 | 522468 | 522468 | 522468 | 0 |
(B) 84f99e060a66 | 1 | 530562 | 530562 | 530562 | 530562 | 0 |
| | +1.55% | +1.55% | +1.55% | +1.55% | --- | + is good
unix1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 510443 | 510443 | 510443 | 510443 | 0 |
(B) 84f99e060a66 | 1 | 518060 | 518060 | 518060 | 518060 | 0 |
| | +1.49% | +1.49% | +1.49% | +1.49% | --- | + is good
unix1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.979953 | 0.979953 | 0.979953 | 0.979953 | 0 |
(B) 84f99e060a66 | 1 | 0.959629 | 0.959629 | 0.959629 | 0.959629 | 0 |
| | -2.07% | -2.07% | -2.07% | -2.07% | --- | + is good
unlink1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 25502 | 25502 | 25502 | 25502 | 0 |
(B) 84f99e060a66 | 1 | 25657 | 25657 | 25657 | 25657 | 0 |
| | +0.61% | +0.61% | +0.61% | +0.61% | --- | + is good
unlink1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 25408 | 25408 | 25408 | 25408 | 0 |
(B) 84f99e060a66 | 1 | 25585 | 25585 | 25585 | 25585 | 0 |
| | +0.70% | +0.70% | +0.70% | +0.70% | --- | + is good
unlink1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.070072 | 0.070072 | 0.070072 | 0.070072 | 0 |
(B) 84f99e060a66 | 1 | 0.070865 | 0.070865 | 0.070865 | 0.070865 | 0 |
| | +1.13% | +1.13% | +1.13% | +1.13% | --- | + is good
unlink2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 52050 | 52050 | 52050 | 52050 | 0 |
(B) 84f99e060a66 | 1 | 51677 | 51677 | 51677 | 51677 | 0 |
| | -0.72% | -0.72% | -0.72% | -0.72% | --- | + is good
unlink2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 51363 | 51363 | 51363 | 51363 | 0 |
(B) 84f99e060a66 | 1 | 51159 | 51159 | 51159 | 51159 | 0 |
| | -0.40% | -0.40% | -0.40% | -0.40% | --- | + is good
unlink2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.302468 | 0.302468 | 0.302468 | 0.302468 | 0 |
(B) 84f99e060a66 | 1 | 0.29703 | 0.29703 | 0.29703 | 0.29703 | 0 |
| | -1.80% | -1.80% | -1.80% | -1.80% | --- | + is good
write1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 770496 | 770496 | 770496 | 770496 | 0 |
(B) 84f99e060a66 | 1 | 762628 | 762628 | 762628 | 762628 | 0 |
| | -1.02% | -1.02% | -1.02% | -1.02% | --- | + is good
write1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 757006 | 757006 | 757006 | 757006 | 0 |
(B) 84f99e060a66 | 1 | 746414 | 746414 | 746414 | 746414 | 0 |
| | -1.40% | -1.40% | -1.40% | -1.40% | --- | + is good
write1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.602205 | 0.602205 | 0.602205 | 0.602205 | 0 |
(B) 84f99e060a66 | 1 | 0.596005 | 0.596005 | 0.596005 | 0.596005 | 0 |
| | -1.03% | -1.03% | -1.03% | -1.03% | --- | + is good
writeseek1_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 629967 | 629967 | 629967 | 629967 | 0 |
(B) 84f99e060a66 | 1 | 626663 | 626663 | 626663 | 626663 | 0 |
| | -0.52% | -0.52% | -0.52% | -0.52% | --- | + is good
writeseek1_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 582490 | 582490 | 582490 | 582490 | 0 |
(B) 84f99e060a66 | 1 | 579400 | 579400 | 579400 | 579400 | 0 |
| | -0.53% | -0.53% | -0.53% | -0.53% | --- | + is good
writeseek1_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.785506 | 0.785506 | 0.785506 | 0.785506 | 0 |
(B) 84f99e060a66 | 1 | 0.800547 | 0.800547 | 0.800547 | 0.800547 | 0 |
| | +1.91% | +1.91% | +1.91% | +1.91% | --- | + is good
writeseek2_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 206887 | 206887 | 206887 | 206887 | 0 |
(B) 84f99e060a66 | 1 | 207179 | 207179 | 207179 | 207179 | 0 |
| | +0.14% | +0.14% | +0.14% | +0.14% | --- | + is good
writeseek2_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 195575 | 195575 | 195575 | 195575 | 0 |
(B) 84f99e060a66 | 1 | 193851 | 193851 | 193851 | 193851 | 0 |
| | -0.88% | -0.88% | -0.88% | -0.88% | --- | + is good
writeseek2_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.057881 | 0.057881 | 0.057881 | 0.057881 | 0 |
(B) 84f99e060a66 | 1 | 0.060053 | 0.060053 | 0.060053 | 0.060053 | 0 |
| | +3.75% | +3.75% | +3.75% | +3.75% | --- | + is good
writeseek3_per_process_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 252163 | 252163 | 252163 | 252163 | 0 |
(B) 84f99e060a66 | 1 | 251815 | 251815 | 251815 | 251815 | 0 |
| | -0.14% | -0.14% | -0.14% | -0.14% | --- | + is good
writeseek3_per_thread_ops | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 236912 | 236912 | 236912 | 236912 | 0 |
(B) 84f99e060a66 | 1 | 236544 | 236544 | 236544 | 236544 | 0 |
| | -0.16% | -0.16% | -0.16% | -0.16% | --- | + is good
writeseek3_scalability | | | | | | |
(A) 4db0b9c9d0f3 | 1 | 0.098322 | 0.098322 | 0.098322 | 0.098322 | 0 |
(B) 84f99e060a66 | 1 | 0.090771 | 0.090771 | 0.090771 | 0.090771 | 0 |
| | -7.68% | -7.68% | -7.68% | -7.68% | --- | + is goodPATCH v6 1/9 mm: x86, arm64: add arch_has_hw_pte_young()
In reply to Yu ZhaoSome architectures automatically set the accessed bit in PTEs, e.g., x86 and arm64 v8.2. On architectures that don't have this capability, clearing the accessed bit in a PTE usually triggers a page fault following the TLB miss of this PTE.
Being aware of this capability can help make better decisions, e.g., whether to spread the work out over a period of time to avoid bursty page faults when trying to clear the accessed bit in a large number of PTEs.
--- arch/arm64/include/asm/cpufeature.h | 5 +++++ arch/arm64/include/asm/pgtable.h | 13 ++++++++----- arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + arch/x86/include/asm/pgtable.h | 6 +++--- include/linux/pgtable.h | 13 +++++++++++++ mm/memory.c | 14 +------------- 7 files changed, 50 insertions(+), 21 deletions(-)
arch/arm64/include/asm/cpufeature.h +5 -0
@@ -779,6 +779,11 @@ static inline bool system_supports_tlb_range(void)cpus_have_const_cap (ARM64_HAS_TLB_RANGE ); } +static inline bool system_has_hw_af (void ) +{ +return IS_ENABLED (CONFIG_ARM64_HW_AFDBM ) &&cpus_have_const_cap (ARM64_HW_AF ); +} +extern int do_emulate_mrs (struct pt_regs *regs,u32 sys_reg,u32 rt);static inline u32 id_aa64mmfr0_parange_to_phys_shift (int parange)
arch/arm64/include/asm/pgtable.h +8 -5
@@ -999,13 +999,16 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,* page after fork() + CoW for pfn mappings. We don't always have a * hardware-managed access flag on arm64. */ -static inline bool arch_faults_on_old_pte (void ) +static inline bool arch_has_hw_pte_young (bool local) { -WARN_ON (preemptible ()); +if (local) { +WARN_ON (preemptible ()); +return cpu_has_hw_af (); + } -return !cpu_has_hw_af (); +return system_has_hw_af (); } -#define arch_faults_on_old_pte arch_faults_on_old_pte +#define arch_has_hw_pte_young arch_has_hw_pte_young/* * Experimentally, it's cheap to set the access flag in hardware and we
@@ -1013,7 +1016,7 @@ static inline bool arch_faults_on_old_pte(void)*/ static inline bool arch_wants_old_prefaulted_pte (void ) { -return !arch_faults_on_old_pte (); +return arch_has_hw_pte_young (true); }#define arch_wants_old_prefaulted_pte arch_wants_old_prefaulted_pte
arch/arm64/kernel/cpufeature.c +19 -0
@@ -2171,6 +2171,25 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_hw_dbm, .cpu_enable = cpu_enable_hw_dbm, }, + { +/* +* __cpu_setup always enables this capability. But if the boot +* CPU has it and a late CPU doesn't, the absent +* ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU will prevent this late CPU +* from going online. There is neither known hardware does that +* nor obvious reasons to design hardware works that way, hence +* no point leaving the door open here. If the need arises, a +* new weak system feature flag should do the trick. +*/ + .desc = "Hardware update of the Access flag", + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .capability = ARM64_HW_AF, + .sys_reg = SYS_ID_AA64MMFR1_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR1_HADBS_SHIFT, + .min_field_value = 1, + .matches = has_cpuid_feature, + }, #endif { .desc = "CRC32 instructions",
arch/arm64/tools/cpucaps +1 -0
@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPE
arch/x86/include/asm/pgtable.h +3 -3
@@ -1397,10 +1397,10 @@ static inline bool arch_has_pfn_modify_check(void) return boot_cpu_has_bug(X86_BUG_L1TF); } -#define arch_faults_on_old_pte arch_faults_on_old_pte -static inline bool arch_faults_on_old_pte (void ) +#define arch_has_hw_pte_young arch_has_hw_pte_young +static inline bool arch_has_hw_pte_young (bool local) { -return false; +return true; }#endif /* __ASSEMBLY__ */
include/linux/pgtable.h +13 -0
@@ -258,6 +258,19 @@ static inline int pmdp_clear_flush_young(struct vm_area_struct *vma,#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #endif +#ifndef arch_has_hw_pte_young +/* +* Return whether the accessed bit is supported by the local CPU or system-wide. +* +* This stub assumes accessing thru an old PTE triggers a page fault. +* Architectures that automatically set the access bit should overwrite it. +*/ +static inline bool arch_has_hw_pte_young (bool local) +{ +return false; +} +#endif +#ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long address,
mm/memory.c +1 -13
@@ -121,18 +121,6 @@ int randomize_va_space __read_mostly =2 ;#endif -#ifndef arch_faults_on_old_pte -static inline bool arch_faults_on_old_pte (void ) -{ -/* -* Those arches which don't have hw access flag feature need to -* implement their own helper. By default, "true" means pagefault -* will be hit on old pte. -*/ -return true; -} -#endif -#ifndef arch_wants_old_prefaulted_ptestatic inline bool arch_wants_old_prefaulted_pte (void ) {
@@ -2755,7 +2743,7 @@ static inline bool cow_user_page(struct page *dst, struct page *src,* On architectures with software "accessed" bits, we would * take a double page fault, so mark it accessed here. */ -if (arch_faults_on_old_pte () && !pte_young (vmf->orig_pte)) { +if (!arch_has_hw_pte_young (true) && !pte_young (vmf->orig_pte)) {pte_t entry; vmf->pte =pte_offset_map_lock (mm, vmf->pmd, addr, &vmf->ptl);
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Will Deacon
maintainerRe: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 107 lines · Yu Zhao
Some architectures automatically set the accessed bit in PTEs, e.g., x86 and arm64 v8.2. On architectures that don't have this capability, clearing the accessed bit in a PTE usually triggers a page fault following the TLB miss of this PTE.
Being aware of this capability can help make better decisions, e.g., whether to spread the work out over a period of time to avoid bursty page faults when trying to clear the accessed bit in a large number of PTEs.
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- arch/arm64/include/asm/cpufeature.h | 5 +++++ arch/arm64/include/asm/pgtable.h | 13 ++++++++----- arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + arch/x86/include/asm/pgtable.h | 6 +++--- include/linux/pgtable.h | 13 +++++++++++++ mm/memory.c | 14 +------------- 7 files changed, 50 insertions(+), 21 deletions(-)arch/arm64/include/asm/cpufeature.h +5
-0@@ -779,6 +779,11 @@ static inline bool system_supports_tlb_range(void)cpus_have_const_cap (ARM64_HAS_TLB_RANGE ); } +static inline bool system_has_hw_af (void ) +{ +return IS_ENABLED (CONFIG_ARM64_HW_AFDBM ) &&cpus_have_const_cap (ARM64_HW_AF ); +} +extern int do_emulate_mrs (struct pt_regs *regs,u32 sys_reg,u32 rt);static inline u32 id_aa64mmfr0_parange_to_phys_shift (int parange)arch/arm64/include/asm/pgtable.h +8
-5@@ -999,13 +999,16 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,* page after fork() + CoW for pfn mappings. We don't always have a * hardware-managed access flag on arm64. */ -static inline bool arch_faults_on_old_pte (void ) +static inline bool arch_has_hw_pte_young (bool local) { -WARN_ON (preemptible ()); +if (local) { +WARN_ON (preemptible ()); +return cpu_has_hw_af (); + } -return !cpu_has_hw_af (); +return system_has_hw_af (); } -#define arch_faults_on_old_pte arch_faults_on_old_pte +#define arch_has_hw_pte_young arch_has_hw_pte_young/* * Experimentally, it's cheap to set the access flag in hardware and we @@ -1013,7 +1016,7 @@ static inline bool arch_faults_on_old_pte(void)*/ static inline bool arch_wants_old_prefaulted_pte (void ) { -return !arch_faults_on_old_pte (); +return arch_has_hw_pte_young (true); }#define arch_wants_old_prefaulted_pte arch_wants_old_prefaulted_ptearch/arm64/kernel/cpufeature.c +19
-0@@ -2171,6 +2171,25 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_hw_dbm, .cpu_enable = cpu_enable_hw_dbm, }, + { +/* +* __cpu_setup always enables this capability. But if the boot +* CPU has it and a late CPU doesn't, the absent +* ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU will prevent this late CPU +* from going online. There is neither known hardware does that +* nor obvious reasons to design hardware works that way, hence +* no point leaving the door open here. If the need arises, a +* new weak system feature flag should do the trick. +*/ + .desc = "Hardware update of the Access flag", + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .capability = ARM64_HW_AF, + .sys_reg = SYS_ID_AA64MMFR1_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR1_HADBS_SHIFT, + .min_field_value = 1, + .matches = has_cpuid_feature, + }, #endif { .desc = "CRC32 instructions",arch/arm64/tools/cpucaps +0
-0
arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 7 lines
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPE
As discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
That way, this patch just ends up being a renaming exercise and we're all good.
Thanks,
Will
Re: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 121 lines · Will Deacon
Quoted 107 lines · Yu Zhao
Some architectures automatically set the accessed bit in PTEs, e.g., x86 and arm64 v8.2. On architectures that don't have this capability, clearing the accessed bit in a PTE usually triggers a page fault following the TLB miss of this PTE.
Being aware of this capability can help make better decisions, e.g., whether to spread the work out over a period of time to avoid bursty page faults when trying to clear the accessed bit in a large number of PTEs.
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- arch/arm64/include/asm/cpufeature.h | 5 +++++ arch/arm64/include/asm/pgtable.h | 13 ++++++++----- arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + arch/x86/include/asm/pgtable.h | 6 +++--- include/linux/pgtable.h | 13 +++++++++++++ mm/memory.c | 14 +------------- 7 files changed, 50 insertions(+), 21 deletions(-)arch/arm64/include/asm/cpufeature.h +5
-0@@ -779,6 +779,11 @@ static inline bool system_supports_tlb_range(void)cpus_have_const_cap (ARM64_HAS_TLB_RANGE ); } +static inline bool system_has_hw_af (void ) +{ +return IS_ENABLED (CONFIG_ARM64_HW_AFDBM ) &&cpus_have_const_cap (ARM64_HW_AF ); +} +extern int do_emulate_mrs (struct pt_regs *regs,u32 sys_reg,u32 rt);static inline u32 id_aa64mmfr0_parange_to_phys_shift (int parange)arch/arm64/include/asm/pgtable.h +8
-5@@ -999,13 +999,16 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,* page after fork() + CoW for pfn mappings. We don't always have a * hardware-managed access flag on arm64. */ -static inline bool arch_faults_on_old_pte (void ) +static inline bool arch_has_hw_pte_young (bool local) { -WARN_ON (preemptible ()); +if (local) { +WARN_ON (preemptible ()); +return cpu_has_hw_af (); + } -return !cpu_has_hw_af (); +return system_has_hw_af (); } -#define arch_faults_on_old_pte arch_faults_on_old_pte +#define arch_has_hw_pte_young arch_has_hw_pte_young/* * Experimentally, it's cheap to set the access flag in hardware and we @@ -1013,7 +1016,7 @@ static inline bool arch_faults_on_old_pte(void)*/ static inline bool arch_wants_old_prefaulted_pte (void ) { -return !arch_faults_on_old_pte (); +return arch_has_hw_pte_young (true); }#define arch_wants_old_prefaulted_pte arch_wants_old_prefaulted_ptearch/arm64/kernel/cpufeature.c +19
-0@@ -2171,6 +2171,25 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_hw_dbm, .cpu_enable = cpu_enable_hw_dbm, }, + { +/* +* __cpu_setup always enables this capability. But if the boot +* CPU has it and a late CPU doesn't, the absent +* ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU will prevent this late CPU +* from going online. There is neither known hardware does that +* nor obvious reasons to design hardware works that way, hence +* no point leaving the door open here. If the need arises, a +* new weak system feature flag should do the trick. +*/ + .desc = "Hardware update of the Access flag", + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .capability = ARM64_HW_AF, + .sys_reg = SYS_ID_AA64MMFR1_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR1_HADBS_SHIFT, + .min_field_value = 1, + .matches = has_cpuid_feature, + }, #endif { .desc = "CRC32 instructions",arch/arm64/tools/cpucaps +0
-0arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 7 lines
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPEAs discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
Quoted 2 lines
That way, this patch just ends up being a renaming exercise and we're all good.
No, renaming alone isn't enough. A caller needs to disable preemption before calling system_has_hw_af(), and I don't think it's reasonable to ask this caller to do it on x86 as well.
It seems you really prefer not to have HW_AF. So the best I can accommodate, considering other potential archs, e.g., risc-v (I do plan to provide benchmark results on risc-v, btw), is:
static inline bool arch_has_hw_pte_young(bool local)
{
bool hw_af; if (local) {
WARN_ON(preemptible());
return cpu_has_hw_af();
}preempt_disable(); hw_af = system_has_hw_af(); preempt_enable(); return hw_af;
}
Or please give me something else I can call without disabling preemption, sounds good?
Will Deacon
maintainerRe: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 45 lines · Yu Zhao
Quoted 19 lines · Will Deacon
arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 12 lines · Yu Zhao
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPEAs discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
Quoted 2 lines
That way, this patch just ends up being a renaming exercise and we're all good.
No, renaming alone isn't enough. A caller needs to disable preemption before calling
system_has_hw_af(), and I don't think it's reasonable to ask this caller to do it on x86 as well.It seems you really prefer not to have
HW_AF. So the best I can accommodate, considering other potential archs, e.g., risc-v (I do plan to provide benchmark results on risc-v, btw), is:static inline bool
arch_has_hw_pte_young(bool local){ bool hw_af;if (local) { WARN_ON(preemptible()); return cpu_has_hw_af(); }preempt_disable(); hw_af = system_has_hw_af(); preempt_enable(); return hw_af;
Quoted 4 lines
}
Or please give me something else I can call without disabling preemption, sounds good?
Sure thing, let me take a look. Do you have your series on a public git tree someplace?
Cheers,
Will
Re: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 45 lines · Will Deacon
Quoted 44 lines · Yu Zhao
Quoted 19 lines · Will Deacon
arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 12 lines · Yu Zhao
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPEAs discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
Quoted 2 lines
That way, this patch just ends up being a renaming exercise and we're all good.
No, renaming alone isn't enough. A caller needs to disable preemption before calling
system_has_hw_af(), and I don't think it's reasonable to ask this caller to do it on x86 as well.It seems you really prefer not to have
HW_AF. So the best I can accommodate, considering other potential archs, e.g., risc-v (I do plan to provide benchmark results on risc-v, btw), is:static inline bool
arch_has_hw_pte_young(bool local){ bool hw_af;if (local) { WARN_ON(preemptible()); return cpu_has_hw_af(); }preempt_disable(); hw_af = system_has_hw_af(); preempt_enable();
Quoted 8 lines
Quoted 5 lines
return hw_af;}
Or please give me something else I can call without disabling preemption, sounds good?
Sure thing, let me take a look. Do you have your series on a public git tree someplace?
Thanks!
This patch (updated) on Gerrit:
https://linux-mm-review.googlesource.com/c/page-reclaim/+/1500/1
And the entire series:
git fetch https://linux-mm.googlesource.com/page-reclaim refs/changes/08/1508/1
Will Deacon
maintainerRe: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 52 lines · Yu Zhao
Quoted 51 lines · Will Deacon
Quoted 49 lines · Yu Zhao
Quoted 19 lines · Will Deacon
arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 12 lines · Yu Zhao
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPEAs discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
Quoted 2 lines
That way, this patch just ends up being a renaming exercise and we're all good.
No, renaming alone isn't enough. A caller needs to disable preemption before calling
system_has_hw_af(), and I don't think it's reasonable to ask this caller to do it on x86 as well.It seems you really prefer not to have
HW_AF. So the best I can accommodate, considering other potential archs, e.g., risc-v (I do plan to provide benchmark results on risc-v, btw), is:static inline bool
arch_has_hw_pte_young(bool local){ bool hw_af;if (local) { WARN_ON(preemptible()); return cpu_has_hw_af(); }preempt_disable(); hw_af = system_has_hw_af(); preempt_enable(); return hw_af;}
Or please give me something else I can call without disabling preemption, sounds good?
Quoted 7 lines
Quoted 2 lines
Sure thing, let me take a look. Do you have your series on a public git tree someplace?
Thanks!
This patch (updated) on Gerrit:
https://linux-mm-review.googlesource.com/c/page-reclaim/+/1500/1
How about folding in something like the diff below? I've basically removed that 'bool local' argument and dropped the preemptible() check from the arm64 code.
Will
--->8
arch/arm64/include/asm/pgtable.h +2 -15
@@ -998,27 +998,14 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,* the pte is old and cannot be marked young. So we always end up with zeroed * page after fork() + CoW for pfn mappings. We don't always have a * hardware-managed access flag on arm64. -* -* The system-wide support isn't used when involving correctness and therefore -* is allowed to be flaky. */ -static inline bool arch_has_hw_pte_young (bool local) -{ -WARN_ON (local &&preemptible ()); - -return cpu_has_hw_af (); -} -#define arch_has_hw_pte_young arch_has_hw_pte_young +#define arch_has_hw_pte_young cpu_has_hw_af/* * Experimentally, it's cheap to set the access flag in hardware and we * benefit from prefaulting mappings as 'old' to start with. */ -static inline bool arch_wants_old_prefaulted_pte (void ) -{ -return arch_has_hw_pte_young (true); -} -#define arch_wants_old_prefaulted_pte arch_wants_old_prefaulted_pte +#define arch_wants_old_prefaulted_pte cpu_has_hw_afstatic inline pgprot_t arch_filter_pgprot (pgprot_t prot) {
arch/x86/include/asm/pgtable.h +1 -1
@@ -1398,7 +1398,7 @@ static inline bool arch_has_pfn_modify_check(void) }#define arch_has_hw_pte_young arch_has_hw_pte_young -static inline bool arch_has_hw_pte_young (bool local) +static inline bool arch_has_hw_pte_young (void ) {return true; }
include/linux/pgtable.h +3 -6
@@ -260,15 +260,12 @@ static inline int pmdp_clear_flush_young(struct vm_area_struct *vma,#ifndef arch_has_hw_pte_young/* -* Return whether the accessed bit is supported by the local CPU or system-wide. +* Return whether the accessed bit is supported by the local CPU. * -* This stub assumes accessing thru an old PTE triggers a page fault. +* This stub assumes accessing through an old PTE triggers a page fault. * Architectures that automatically set the access bit should overwrite it. -* -* Note that the system-wide support can be flaky and therefore shouldn't be -* used when involving correctness. */ -static inline bool arch_has_hw_pte_young (bool local) +static inline bool arch_has_hw_pte_young (void ) {return false; }
mm/memory.c +1 -1
@@ -2743,7 +2743,7 @@ static inline bool cow_user_page(struct page *dst, struct page *src,* On architectures with software "accessed" bits, we would * take a double page fault, so mark it accessed here. */ -if (!arch_has_hw_pte_young (true) && !pte_young (vmf->orig_pte)) { +if (!arch_has_hw_pte_young () && !pte_young (vmf->orig_pte)) {pte_t entry; vmf->pte =pte_offset_map_lock (mm, vmf->pmd, addr, &vmf->ptl);
Re: [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young()
Quoted 56 lines · Will Deacon
Quoted 55 lines · Yu Zhao
Quoted 53 lines · Will Deacon
Quoted 45 lines · Yu Zhao
Quoted 19 lines · Will Deacon
arch/arm64/tools/cpucaps HAS_STAGE2_FWB · line 42
Quoted 12 lines · Yu Zhao
arch/arm64/tools/cpucaps +1
-0@@ -36,6 +36,7 @@ HAS_STAGE2_FWB HAS_SYSREG_GIC_CPUIF HAS_TLB_RANGE HAS_VIRT_HOST_EXTN +HW_AF HW_DBM KVM_PROTECTED_MODE MISMATCHED_CACHE_TYPEAs discussed in the previous threads, we really don't need the complexity of the additional cap for the arm64 part. Please can you just use the existing code instead? It's both simpler and, as you say, it's equivalent for existing hardware.
Quoted 2 lines
That way, this patch just ends up being a renaming exercise and we're all good.
No, renaming alone isn't enough. A caller needs to disable preemption before calling
system_has_hw_af(), and I don't think it's reasonable to ask this caller to do it on x86 as well.It seems you really prefer not to have
HW_AF. So the best I can accommodate, considering other potential archs, e.g., risc-v (I do plan to provide benchmark results on risc-v, btw), is:static inline bool
arch_has_hw_pte_young(bool local){ bool hw_af;if (local) { WARN_ON(preemptible()); return cpu_has_hw_af(); }preempt_disable(); hw_af = system_has_hw_af(); preempt_enable(); return hw_af;Quoted 4 lines
}
Or please give me something else I can call without disabling preemption, sounds good?
Sure thing, let me take a look. Do you have your series on a public git tree someplace?
Quoted 8 lines
Quoted 4 lines
Thanks!
This patch (updated) on Gerrit:
https://linux-mm-review.googlesource.com/c/page-reclaim/+/1500/1How about folding in something like the diff below? I've basically removed that 'bool local' argument and dropped the
preemptible()check from the arm64 code.
This looks great, thanks.
PATCH v6 2/9 mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
In reply to Yu ZhaoSome architectures support the accessed bit in non-leaf PMD entries, e.g., x86_64 sets the accessed bit in a non-leaf PMD entry when using it as part of linear address translation [1]. Page table walkers that clear the accessed bit may use this feature to reduce their search space.
Although an inline function is preferable, this capability is added as a configuration option for the consistency with the existing macros.
[1]: Intel 64 and IA-32 Architectures Software Developer's Manual
Volume 3 (June 2021), section 4.8
--- arch/Kconfig | 9 +++++++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 3 ++- arch/x86/mm/pgtable.c | 5 ++++- include/linux/pgtable.h | 4 ++-- 5 files changed, 18 insertions(+), 4 deletions(-)
arch/Kconfig +9 -0
@@ -1312,6 +1312,15 @@ config ARCH_HAS_PARANOID_L1D_FLUSH config DYNAMIC_SIGFRAME bool +config ARCH_HAS_NONLEAF_PMD_YOUNG + bool + depends on PGTABLE_LEVELS > 2 + help + Architectures that select this option are capable of setting the + accessed bit in non-leaf PMD entries when using them as part of linear + address translations. Page table walkers that clear the accessed bit + may use this feature to reduce their search space. + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig"
arch/x86/Kconfig +1 -0
@@ -85,6 +85,7 @@ config X86 select ARCH_HAS_PMEM_API if X86_64 select ARCH_HAS_PTE_DEVMAP if X86_64 select ARCH_HAS_PTE_SPECIAL + select ARCH_HAS_NONLEAF_PMD_YOUNG if X86_64 select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64 select ARCH_HAS_COPY_MC if X86_64 select ARCH_HAS_SET_MEMORY
arch/x86/include/asm/pgtable.h +2 -1
@@ -817,7 +817,8 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)static inline int pmd_bad (pmd_t pmd) { -return (pmd_flags (pmd) & ~_PAGE_USER) != _KERNPG_TABLE; +return (pmd_flags (pmd) & ~(_PAGE_USER | _PAGE_ACCESSED)) != + (_KERNPG_TABLE & ~_PAGE_ACCESSED); }static inline unsigned long pages_to_mb (unsigned long npg)
arch/x86/mm/pgtable.c +4 -1
@@ -550,7 +550,7 @@ int ptep_test_and_clear_young(struct vm_area_struct *vma,return ret; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#if defined(CONFIG_TRANSPARENT_HUGEPAGE ) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG )int pmdp_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pmd_t *pmdp) {
@@ -562,6 +562,9 @@ int pmdp_test_and_clear_young(struct vm_area_struct *vma,return ret; } +#endif + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE int pudp_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pud_t *pudp) {
include/linux/pgtable.h +2 -2
@@ -211,7 +211,7 @@ static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,#endif #ifndef __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG -#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#if defined(CONFIG_TRANSPARENT_HUGEPAGE ) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG )static inline int pmdp_test_and_clear_young (struct vm_area_struct *vma,unsigned long address,pmd_t *pmdp)
@@ -232,7 +232,7 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,BUILD_BUG ();return 0 ; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG */ #endif #ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Re: [PATCH v6 2/9] mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
arch/x86/Kconfig config X86 · line 85
Quoted 7 lines · Yu Zhao
arch/x86/Kconfig +1
-0@@ -85,6 +85,7 @@ config X86 + select ARCH_HAS_NONLEAF_PMD_YOUNG if X86_64
Why is this limited to 64-bit?
I'm ok with that - maybe it's a simple case of "this is not worth doing on 32-bit", but I'd like the explanation to be written out.
Right now the commit message literally points the architecture manual that sio relevant for both 32-bit and 64-bit - and then the patch itself makes it 64-bit only.
Linus
PATCH v6 3/9 mm/vmscan.c: refactor shrink_node()
In reply to Yu ZhaoThis patch refactors shrink_node() to improve readability for the upcoming changes to mm/vmscan.c.
--- mm/vmscan.c | 198 +++++++++++++++++++++++++++------------------------- 1 file changed, 104 insertions(+), 94 deletions(-)
mm/vmscan.c +104 -94
@@ -2716,6 +2716,109 @@ enum scan_balance { SCAN_FILE, }; +static void prepare_scan_count (pg_data_t *pgdat,struct scan_control *sc) +{ +unsigned long file; +struct lruvec *target_lruvec; + + target_lruvec =mem_cgroup_lruvec (sc->target_mem_cgroup, pgdat); + +/* +* Flush the memory cgroup stats, so that we read accurate per-memcg +* lruvec stats for heuristics. +*/ +mem_cgroup_flush_stats (); + +/* +* Determine the scan balance between anon and file LRUs. +*/ +spin_lock_irq (&target_lruvec->lru_lock); + sc->anon_cost = target_lruvec->anon_cost; + sc->file_cost = target_lruvec->file_cost; +spin_unlock_irq (&target_lruvec->lru_lock); + +/* +* Target desirable inactive:active list ratios for the anon +* and file LRU lists. +*/ +if (!sc->force_deactivate) { +unsigned long refaults; + + refaults =lruvec_page_state (target_lruvec, +WORKINGSET_ACTIVATE_ANON ); +if (refaults != target_lruvec->refaults[0 ] || +inactive_is_low (target_lruvec,LRU_INACTIVE_ANON )) + sc->may_deactivate |=DEACTIVATE_ANON ; +else + sc->may_deactivate &= ~DEACTIVATE_ANON ; + +/* +* When refaults are being observed, it means a new +* workingset is being established. Deactivate to get +* rid of any stale active pages quickly. +*/ + refaults =lruvec_page_state (target_lruvec, +WORKINGSET_ACTIVATE_FILE ); +if (refaults != target_lruvec->refaults[1 ] || +inactive_is_low (target_lruvec,LRU_INACTIVE_FILE )) + sc->may_deactivate |=DEACTIVATE_FILE ; +else + sc->may_deactivate &= ~DEACTIVATE_FILE ; + }else + sc->may_deactivate =DEACTIVATE_ANON |DEACTIVATE_FILE ; + +/* +* If we have plenty of inactive file pages that aren't +* thrashing, try to reclaim those first before touching +* anonymous pages. +*/ + file =lruvec_page_state (target_lruvec,NR_INACTIVE_FILE ); +if (file >> sc->priority && !(sc->may_deactivate &DEACTIVATE_FILE )) + sc->cache_trim_mode =1 ; +else + sc->cache_trim_mode =0 ; + +/* +* Prevent the reclaimer from falling into the cache trap: as +* cache pages start out inactive, every cache fault will tip +* the scan balance towards the file LRU. And as the file LRU +* shrinks, so does the window for rotation from references. +* This means we have a runaway feedback loop where a tiny +* thrashing file LRU becomes infinitely more attractive than +* anon pages. Try to detect this based on file LRU size. +*/ +if (!cgroup_reclaim (sc)) { +unsigned long total_high_wmark =0 ; +unsigned long free, anon; +int z; + + free =sum_zone_node_page_state (pgdat->node_id,NR_FREE_PAGES ); + file =node_page_state (pgdat,NR_ACTIVE_FILE ) + +node_page_state (pgdat,NR_INACTIVE_FILE ); + +for (z =0 ; z <MAX_NR_ZONES ; z++) { +struct zone *zone = &pgdat->node_zones[z]; + +if (!managed_zone (zone)) +continue ; + + total_high_wmark +=high_wmark_pages (zone); + } + +/* +* Consider anon: if that's low too, this isn't a +* runaway file reclaim problem, but rather just +* extreme pressure. Reclaim as per usual then. +*/ + anon =node_page_state (pgdat,NR_INACTIVE_ANON ); + + sc->file_is_tiny = + file + free <= total_high_wmark && + !(sc->may_deactivate &DEACTIVATE_ANON ) && + anon >> sc->priority; + } +} +/* * Determine how aggressively the anon and file LRU lists should be * scanned. The relative value of each set of LRU lists is determined
@@ -3186,109 +3289,16 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc)unsigned long nr_reclaimed, nr_scanned;struct lruvec *target_lruvec;bool reclaimable = false; -unsigned long file; target_lruvec =mem_cgroup_lruvec (sc->target_mem_cgroup, pgdat); again: -/* -* Flush the memory cgroup stats, so that we read accurate per-memcg -* lruvec stats for heuristics. -*/ -mem_cgroup_flush_stats (); -memset (&sc->nr,0 ,sizeof (sc->nr)); nr_reclaimed = sc->nr_reclaimed; nr_scanned = sc->nr_scanned; -/* -* Determine the scan balance between anon and file LRUs. -*/ -spin_lock_irq (&target_lruvec->lru_lock); - sc->anon_cost = target_lruvec->anon_cost; - sc->file_cost = target_lruvec->file_cost; -spin_unlock_irq (&target_lruvec->lru_lock); - -/* -* Target desirable inactive:active list ratios for the anon -* and file LRU lists. -*/ -if (!sc->force_deactivate) { -unsigned long refaults; - - refaults =lruvec_page_state (target_lruvec, -WORKINGSET_ACTIVATE_ANON ); -if (refaults != target_lruvec->refaults[0 ] || -inactive_is_low (target_lruvec,LRU_INACTIVE_ANON )) - sc->may_deactivate |=DEACTIVATE_ANON ; -else - sc->may_deactivate &= ~DEACTIVATE_ANON ; - -/* -* When refaults are being observed, it means a new -* workingset is being established. Deactivate to get -* rid of any stale active pages quickly. -*/ - refaults =lruvec_page_state (target_lruvec, -WORKINGSET_ACTIVATE_FILE ); -if (refaults != target_lruvec->refaults[1 ] || -inactive_is_low (target_lruvec,LRU_INACTIVE_FILE )) - sc->may_deactivate |=DEACTIVATE_FILE ; -else - sc->may_deactivate &= ~DEACTIVATE_FILE ; - }else - sc->may_deactivate =DEACTIVATE_ANON |DEACTIVATE_FILE ; - -/* -* If we have plenty of inactive file pages that aren't -* thrashing, try to reclaim those first before touching -* anonymous pages. -*/ - file =lruvec_page_state (target_lruvec,NR_INACTIVE_FILE ); -if (file >> sc->priority && !(sc->may_deactivate &DEACTIVATE_FILE )) - sc->cache_trim_mode =1 ; -else - sc->cache_trim_mode =0 ; - -/* -* Prevent the reclaimer from falling into the cache trap: as -* cache pages start out inactive, every cache fault will tip -* the scan balance towards the file LRU. And as the file LRU -* shrinks, so does the window for rotation from references. -* This means we have a runaway feedback loop where a tiny -* thrashing file LRU becomes infinitely more attractive than -* anon pages. Try to detect this based on file LRU size. -*/ -if (!cgroup_reclaim (sc)) { -unsigned long total_high_wmark =0 ; -unsigned long free, anon; -int z; - - free =sum_zone_node_page_state (pgdat->node_id,NR_FREE_PAGES ); - file =node_page_state (pgdat,NR_ACTIVE_FILE ) + -node_page_state (pgdat,NR_INACTIVE_FILE ); - -for (z =0 ; z <MAX_NR_ZONES ; z++) { -struct zone *zone = &pgdat->node_zones[z]; -if (!managed_zone (zone)) -continue ; - - total_high_wmark +=high_wmark_pages (zone); - } - -/* -* Consider anon: if that's low too, this isn't a -* runaway file reclaim problem, but rather just -* extreme pressure. Reclaim as per usual then. -*/ - anon =node_page_state (pgdat,NR_INACTIVE_ANON ); - - sc->file_is_tiny = - file + free <= total_high_wmark && - !(sc->may_deactivate &DEACTIVATE_ANON ) && - anon >> sc->priority; - } +prepare_scan_count (pgdat, sc);shrink_node_memcgs (pgdat, sc);
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
PATCH v6 4/9 mm: multigenerational lru: groundwork
In reply to Yu ZhaoEvictable pages are divided into multiple generations for each lruvec. The youngest generation number is stored in lrugen->max_seq for both anon and file types as they're aged on an equal footing. The oldest generation numbers are stored in lrugen->min_seq[] separately for anon and file types as clean file pages may be evicted regardless of swap constraints. These three variables are monotonically increasing.
Generation numbers are truncated into order_base_2(MAX_NR_GENS+1) bits in order to fit into the gen counter in folio->flags. Each truncated generation number is an index to lrugen->lists[]. The sliding window technique is used to track at least MIN_NR_GENS and at most MAX_NR_GENS generations.
There are two conceptually independent processes (as in manufacturing process): "the aging", which produces young generations, and "the eviction", which consumes old generations. They form a closed-loop system, i.e., "the page reclaim". Both processes can be triggered separately from userspace for the purposes of working set estimation and proactive reclaim. These features are required to optimize job scheduling in data centers. The variable size of the sliding window is designed for such use cases.
To avoid confusions, the terms "hot" and "cold" will be applied to the multigenerational lru, as a new convention; the terms "active" and "inactive" will be applied to the active/inactive lru, as usual.
The protection of hot pages and the selection of cold pages are based on page access channels and patterns. There are two access channels:
one thru page tables and the other thru file descriptors. The protection of the former channel is by design stronger because:
1) The uncertainty in determining the access patterns of the former channel is higher due to the approximation of the accessed bit.
2) The cost of evicting the former channel is higher due to the TLB flushes required and the likelihood of encountering the dirty bit.
3) The penalty of underprotecting the former channel is higher because applications usually don't prepare themselves for major faults like they do for blocked I/O. For example, GUI applications commonly use dedicated I/O threads to avoid blocking the rendering threads.
There are also two access patterns: one with temporal locality and the other without. For the reasons listed above, the former channel is assumed to follow the former pattern unless VM_SEQ_READ or VM_RAND_READ is present and the latter channel is assumed to follow the latter pattern unless outlying refaults have been observed.
The "outlying refaults" will be addressed in [PATCH 07/10]. A few macros, i.e., LRU_REFS_*, used in that patch are added in this one to make the patchset less diffy.
A page is added to the youngest generation on faulting. The aging needs to check the accessed bit at least twice before handing this page over to the eviction. The first check takes care of the accessed bit set on the initial fault; the second check makes sure this page hasn't been used since then. This process, AKA second chance, requires a minimum of two generations, hence MIN_NR_GENS.
--- fs/fuse/dev.c | 3 +- include/linux/cgroup.h | 15 +- include/linux/mm.h | 37 +++++ include/linux/mm_inline.h | 189 ++++++++++++++++++++++ include/linux/mmzone.h | 76 +++++++++ include/linux/page-flags-layout.h | 19 ++- include/linux/page-flags.h | 4 +- include/linux/sched.h | 4 + kernel/bounds.c | 3 + kernel/cgroup/cgroup-internal.h | 1 - mm/huge_memory.c | 3 +- mm/memcontrol.c | 2 + mm/memory.c | 7 + mm/mm_init.c | 6 +- mm/page_alloc.c | 1 + mm/swap.c | 9 +- mm/vmscan.c | 259 ++++++++++++++++++++++++++++++ 17 files changed, 623 insertions(+), 15 deletions(-)
fs/fuse/dev.c +2 -1
@@ -785,7 +785,8 @@ static int fuse_check_page(struct page *page) 1 << PG_active | 1 << PG_workingset | 1 << PG_reclaim | - 1 << PG_waiters))) { + 1 << PG_waiters | + LRU_GEN_MASK | LRU_REFS_MASK))) {dump_page (page,"fuse: trying to steal weird page" );return 1 ; }
include/linux/cgroup.h +14 -1
@@ -432,6 +432,18 @@ static inline void cgroup_put(struct cgroup *cgrp)css_put (&cgrp->self); } +extern struct mutex cgroup_mutex; + +static inline void cgroup_lock (void ) +{ +mutex_lock (&cgroup_mutex); +} + +static inline void cgroup_unlock (void ) +{ +mutex_unlock (&cgroup_mutex); +} +/** * task_css_set_check - obtain a task's css_set with extra access conditions * @task: the task to obtain css_set for
@@ -446,7 +458,6 @@ static inline void cgroup_put(struct cgroup *cgrp)* as locks used during the cgroup_subsys::attach() methods. */ #ifdef CONFIG_PROVE_RCU -extern struct mutex cgroup_mutex;extern spinlock_t css_set_lock;#define task_css_set_check (task, __c) \ rcu_dereference_check((task)->cgroups, \
@@ -707,6 +718,8 @@ struct cgroup;static inline u64 cgroup_id (const struct cgroup *cgrp) {return 1 ; }static inline void css_get (struct cgroup_subsys_state *css) {}static inline void css_put (struct cgroup_subsys_state *css) {} +static inline void cgroup_lock (void ) {} +static inline void cgroup_unlock (void ) {}static inline int cgroup_attach_task_all (struct task_struct *from,struct task_struct *t) {return 0 ; } static inline int cgroupstats_build(struct cgroupstats *stats,
include/linux/mm.h +37 -0
@@ -227,6 +227,7 @@ int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *,#define PAGE_ALIGNED (addr) IS_ALIGNED((unsigned long)(addr), PAGE_SIZE)#define lru_to_page (head) (list_entry((head)->prev, struct page, lru)) +#define lru_to_folio (head) (list_entry((head)->prev, struct folio, lru))void setup_initial_init_mm (void *start_code,void *end_code,void *end_data,void *brk);
@@ -1070,6 +1071,8 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf);#define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH)#define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH)#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) +#define LRU_GEN_PGOFF (KASAN_TAG_PGOFF - LRU_GEN_WIDTH) +#define LRU_REFS_PGOFF (LRU_GEN_PGOFF - LRU_REFS_WIDTH)/* * Define the bit shifts to access each section. For non-existent
@@ -1920,6 +1923,40 @@ static inline void unmap_mapping_range(struct address_space *mapping, loff_t const holebegin, loff_t const holelen, int even_cows) { } #endif +#ifdef CONFIG_LRU_GEN +static inline void task_enter_lru_fault(void) +{ + WARN_ON_ONCE(current->in_lru_fault); + + current->in_lru_fault =1 ; +} + +static inline void task_exit_lru_fault (void ) +{ +WARN_ON_ONCE (!current->in_lru_fault); + + current->in_lru_fault =0 ; +} + +static inline bool task_in_lru_fault (void ) +{ +return current->in_lru_fault; +} +#else +static inline void task_enter_lru_fault (void ) +{ +} + +static inline void task_exit_lru_fault (void ) +{ +} + +static inline bool task_in_lru_fault (void ) +{ +return false; +} +#endif /* CONFIG_LRU_GEN */ +static inline void unmap_shared_mapping_range (struct address_space *mapping,loff_t const holebegin,loff_t const holelen) {
include/linux/mm_inline.h +189 -0
212 lines not on this page · raw message
include/linux/mmzone.h +76 -0
88 lines not on this page · raw message
include/linux/page-flags-layout.h +14 -5
@@ -26,6 +26,14 @@#define ZONES_WIDTH ZONES_SHIFT +#ifdef CONFIG_LRU_GEN +/* LRU_GEN_WIDTH is generated from order_base_2(CONFIG_NR_LRU_GENS + 1). */ +#define LRU_REFS_WIDTH (CONFIG_TIERS_PER_GEN - 2) +#else +#define LRU_GEN_WIDTH 0 +#define LRU_REFS_WIDTH 0 +#endif /* CONFIG_LRU_GEN */ +#ifdef CONFIG_SPARSEMEM #include <asm/sparsemem.h> #define SECTIONS_SHIFT (MAX_PHYSMEM_BITS - SECTION_SIZE_BITS)
@@ -55,7 +63,8 @@#define SECTIONS_WIDTH 0#endif -#if ZONES_WIDTH +SECTIONS_WIDTH +NODES_SHIFT <=BITS_PER_LONG -NR_PAGEFLAGS +#if ZONES_WIDTH +LRU_GEN_WIDTH +LRU_REFS_WIDTH +SECTIONS_WIDTH +NODES_SHIFT \ + <=BITS_PER_LONG -NR_PAGEFLAGS #define NODES_WIDTH NODES_SHIFT#elif defined(CONFIG_SPARSEMEM_VMEMMAP )#error "Vmemmap: No space for nodes field in page flags"
@@ -89,8 +98,8 @@#define LAST_CPUPID_SHIFT 0#endif -#if ZONES_WIDTH +SECTIONS_WIDTH +NODES_WIDTH +KASAN_TAG_WIDTH +LAST_CPUPID_SHIFT \ - <=BITS_PER_LONG -NR_PAGEFLAGS +#if ZONES_WIDTH +LRU_GEN_WIDTH +LRU_REFS_WIDTH +SECTIONS_WIDTH +NODES_WIDTH + \ +KASAN_TAG_WIDTH +LAST_CPUPID_SHIFT <=BITS_PER_LONG -NR_PAGEFLAGS #define LAST_CPUPID_WIDTH LAST_CPUPID_SHIFT#else #define LAST_CPUPID_WIDTH 0
@@ -100,8 +109,8 @@#define LAST_CPUPID_NOT_IN_PAGE_FLAGS #endif -#if ZONES_WIDTH +SECTIONS_WIDTH +NODES_WIDTH +KASAN_TAG_WIDTH +LAST_CPUPID_WIDTH \ - >BITS_PER_LONG -NR_PAGEFLAGS +#if ZONES_WIDTH +LRU_GEN_WIDTH +LRU_REFS_WIDTH +SECTIONS_WIDTH +NODES_WIDTH + \ +KASAN_TAG_WIDTH +LAST_CPUPID_WIDTH >BITS_PER_LONG -NR_PAGEFLAGS #error "Not enough bits in page flags"#endif
include/linux/page-flags.h +2 -2
@@ -961,7 +961,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page)1UL << PG_private |1UL << PG_private_2 | \1UL << PG_writeback |1UL << PG_reserved | \1UL << PG_slab |1UL << PG_active | \ - 1UL << PG_unevictable | __PG_MLOCKED) + 1UL << PG_unevictable | __PG_MLOCKED | LRU_GEN_MASK)/* * Flags checked when a page is prepped for return by the page allocator.
@@ -972,7 +972,7 @@ static inline void ClearPageSlabPfmemalloc(struct page *page)* alloc-free cycle to prevent from reusing the page. */ #define PAGE_FLAGS_CHECK_AT_PREP \ - (PAGEFLAGS_MASK & ~__PG_HWPOISON) + ((PAGEFLAGS_MASK & ~__PG_HWPOISON) | LRU_GEN_MASK | LRU_REFS_MASK)#define PAGE_FLAGS_PRIVATE \ (1UL << PG_private | 1UL << PG_private_2)
include/linux/sched.h +4 -0
@@ -909,6 +909,10 @@ struct task_struct {#ifdef CONFIG_MEMCG unsigned in_user_fault:1;#endif +#ifdef CONFIG_LRU_GEN +/* whether the lru algorithm may apply for this access */ + unsigned in_lru_fault:1; +#endif #ifdef CONFIG_COMPAT_BRK unsigned brk_randomized:1;#endif
kernel/bounds.c +3 -0
@@ -22,6 +22,9 @@ int main(void)DEFINE (NR_CPUS_BITS ,ilog2 (CONFIG_NR_CPUS ));#endif DEFINE (SPINLOCK_SIZE ,sizeof (spinlock_t)); +#ifdef CONFIG_LRU_GEN +DEFINE (LRU_GEN_WIDTH ,order_base_2 (CONFIG_NR_LRU_GENS +1 )); +#endif /* End of constants */ return 0 ;
kernel/cgroup/cgroup-internal.h +0 -1
@@ -146,7 +146,6 @@ struct cgroup_mgctx {#define DEFINE_CGROUP_MGCTX (name) \ struct cgroup_mgctx name = CGROUP_MGCTX_INIT(name) -extern struct mutex cgroup_mutex;extern spinlock_t css_set_lock;extern struct cgroup_subsys *cgroup_subsys[];extern struct list_head cgroup_roots;
mm/huge_memory.c +2 -1
@@ -2364,7 +2364,8 @@ static void __split_huge_page_tail(struct page *head, int tail,#ifdef CONFIG_64BIT (1L << PG_arch_2) | #endif - (1L << PG_dirty))); + (1L << PG_dirty) | + LRU_GEN_MASK | LRU_REFS_MASK));/* ->mapping in first tail page is compound_mapcount */ VM_BUG_ON_PAGE(tail > 2 && page_tail->mapping != TAIL_MAPPING,
mm/memcontrol.c +2 -0
@@ -5098,6 +5098,7 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg)static void mem_cgroup_free (struct mem_cgroup *memcg) { +lru_gen_free_memcg (memcg);memcg_wb_domain_exit (memcg);__mem_cgroup_free (memcg); }
@@ -5161,6 +5162,7 @@ static struct mem_cgroup *mem_cgroup_alloc(void) memcg->deferred_split_queue.split_queue_len =0 ;#endif idr_replace (&mem_cgroup_idr, memcg, memcg->id.id); +lru_gen_init_memcg (memcg);return memcg; fail:mem_cgroup_id_remove (memcg);
mm/memory.c +7 -0
@@ -4745,6 +4745,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags, struct pt_regs *regs) { vm_fault_t ret; +bool lru_fault = !(vma->vm_flags & (VM_SEQ_READ |VM_RAND_READ ));__set_current_state (TASK_RUNNING );
@@ -4766,11 +4767,17 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,if (flags &FAULT_FLAG_USER )mem_cgroup_enter_user_fault (); +if (lru_fault) +task_enter_lru_fault (); +if (unlikely (is_vm_hugetlb_page (vma))) ret =hugetlb_fault (vma->vm_mm, vma, address, flags);else ret =__handle_mm_fault (vma, address, flags); +if (lru_fault) +task_exit_lru_fault (); +if (flags &FAULT_FLAG_USER ) {mem_cgroup_exit_user_fault ();/*
mm/mm_init.c +4 -2
@@ -65,14 +65,16 @@ void __init mminit_verify_pageflags_layout(void) shift =8 *sizeof (unsigned long ); width = shift -SECTIONS_WIDTH -NODES_WIDTH -ZONES_WIDTH - -LAST_CPUPID_SHIFT -KASAN_TAG_WIDTH ; + -LAST_CPUPID_SHIFT -KASAN_TAG_WIDTH -LRU_GEN_WIDTH -LRU_REFS_WIDTH ;mminit_dprintk (MMINIT_TRACE ,"pageflags_layout_widths" , -"Section %d Node %d Zone %d Lastcpupid %d Kasantag %d Flags %d\n" , +"Section %d Node %d Zone %d Lastcpupid %d Kasantag %d Gen %d Tier %d Flags %d\n" ,SECTIONS_WIDTH ,NODES_WIDTH ,ZONES_WIDTH ,LAST_CPUPID_WIDTH ,KASAN_TAG_WIDTH , +LRU_GEN_WIDTH , +LRU_REFS_WIDTH ,NR_PAGEFLAGS ); mminit_dprintk(MMINIT_TRACE, "pageflags_layout_shifts", "Section %d Node %d Zone %d Lastcpupid %d Kasantag %d\n",
mm/page_alloc.c +1 -0
@@ -7416,6 +7416,7 @@ static void __meminit pgdat_init_internals(struct pglist_data *pgdat)pgdat_page_ext_init (pgdat);lruvec_init (&pgdat->__lruvec); +lru_gen_init_state (NULL , &pgdat->__lruvec); } static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx, int nid,
mm/swap.c +7 -2
@@ -462,6 +462,11 @@ void folio_add_lru(struct folio *folio)VM_BUG_ON_FOLIO (folio_test_active (folio) &&folio_test_unevictable (folio), folio);VM_BUG_ON_FOLIO (folio_test_lru (folio), folio); +/* see the comment in lru_gen_add_folio() */ +if (lru_gen_enabled () && !folio_test_unevictable (folio) && +task_in_lru_fault () && !(current->flags &PF_MEMALLOC )) +folio_set_active (folio); +folio_get (folio);local_lock (&lru_pvecs.lock); pvec =this_cpu_ptr (&lru_pvecs.lru_add);
@@ -563,7 +568,7 @@ static void lru_deactivate_file_fn(struct page *page, struct lruvec *lruvec)static void lru_deactivate_fn (struct page *page,struct lruvec *lruvec) { -if (PageActive (page) && !PageUnevictable (page)) { +if (!PageUnevictable (page) && (PageActive (page) ||lru_gen_enabled ())) {int nr_pages =thp_nr_pages (page);del_page_from_lru_list (page, lruvec);
@@ -677,7 +682,7 @@ void deactivate_file_page(struct page *page)*/ void deactivate_page (struct page *page) { -if (PageLRU (page) &&PageActive (page) && !PageUnevictable (page)) { +if (PageLRU (page) && !PageUnevictable (page) && (PageActive (page) ||lru_gen_enabled ())) {struct pagevec *pvec;local_lock (&lru_pvecs.lock);
mm/vmscan.c +259 -0
271 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Re: [PATCH v6 4/9] mm: multigenerational lru: groundwork
include/linux/mm.h static inline void unmap_mapping_range(struct address_space *mapping, · line 1929
Quoted 10 lines · Yu Zhao
index
a7e4a9e7d807..fadbf8e6abcd100644include/linux/mm.h +4
-0+#ifdef CONFIG_LRU_GEN +static inline void task_enter_lru_fault (void ) +{ +WARN_ON_ONCE (current->in_lru_fault);
...
Why are these in this very core header file?
They are used in one single file - mm/memory.c.
They should be just static functions there.
I'm also not sure why the calling convention is
if (lru_fault)
task_enter_lru_fault();instead of doing just
task_enter_lru_fault(vma);
and having that function do
/* Don't do LRU fault accounting for SEQ/RAND files */
if (unlikely(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ)))
return;which would seem to be a lot more legible and straightforward.
In fact, you could do it without any conditionals at all, if you just remove the WARN_ON_ONCE() from the exit path, turning it into just
current->in_lru_fault = !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ));
for 'enter' and just
current->in_lru_fault = 0;
for exit.
It seems pointless to have that extra variable, and the extra conditionals, for a case that is probably very unusual indeed.
Linus
Re: [PATCH v6 4/9] mm: multigenerational lru: groundwork
In reply to Yu ZhaoYu Zhao <[email protected]> writes:
.....
+
Quoted 11 lines
+/*
+ * Evictable pages are divided into multiple generations. The youngest and the
+ * oldest generation numbers,max_seqandmin_seq, are monotonically increasing.
+ * They form a sliding window of a variable size [MIN_NR_GENS,MAX_NR_GENS]. An
+ * offset withinMAX_NR_GENS, gen, indexes the lru list of the corresponding
+ * generation. The gen counter infolio->flagsstores gen+1 while a page is on
+ * lrugen->lists[]. Otherwise, it stores 0.
+ *
+ * A page is added to the youngest generation on faulting. The aging needs to
+ * check the accessed bit at least twice before handing this page over to the
+ * eviction. The first check takes care of the accessed bit set on the initial
Quoted 8 lines
+ * fault; the second check makes sure this page hasn't been used since then.
+ * This process, AKA second chance, requires a minimum of two generations,
+ * henceMIN_NR_GENS. And to be compatible with the active/inactive lru, these
+ * two generations are mapped to the active; the rest of generations, if they
+ * exist, are mapped to the inactive.PG_activeis always cleared while a page
+ * is on lrugen->lists[] so that demotion, which happens consequently when the
+ * aging creates a new generation, needs not to worry about it.
+ */
Where do we clear PG_active in the code? Is this the reason we endup with
void deactivate_page(struct page *page)
{
- if (PageLRU(page) && PageActive(page) && !PageUnevictable(page)) {
+ if (PageLRU(page) && !PageUnevictable(page) && (PageActive(page) || lru_gen_enabled())) {
Quoted 18 lines
include/linux/mmzone.h +18
-0+#define MIN_NR_GENS 2U +#define MAX_NR_GENS ((unsigned int)CONFIG_NR_LRU_GENS) + +struct lru_gen_struct { +/* the aging increments the youngest generation number */ +unsigned long max_seq; +/* the eviction increments the oldest generation numbers */ +unsigned long min_seq[ANON_AND_FILE ]; +/* the birth time of each generation in jiffies */ +unsigned long timestamps[MAX_NR_GENS ]; +/* the multigenerational lru lists */ +struct list_head lists[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ]; +/* the sizes of the above lists */ +unsigned long nr_pages[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ]; +/* whether the multigenerational lru is enabled */ +bool enabled; +}; +
....
Quoted 6 lines
static void
__meminitzone_init_internals(struct zone *zone, enumzone_typeidx, int nid,mm/swap.c +0
-0
mm/swap.c void folio_add_lru(struct folio *folio) · line 469
Quoted 8 lines
mm/swap.c +5
-0@@ -462,6 +462,11 @@ void folio_add_lru(struct folio *folio)VM_BUG_ON_FOLIO (folio_test_active (folio) &&folio_test_unevictable (folio), folio);VM_BUG_ON_FOLIO (folio_test_lru (folio), folio); +/* see the comment in lru_gen_add_folio() */ +if (lru_gen_enabled () && !folio_test_unevictable (folio) && +task_in_lru_fault () && !(current->flags &PF_MEMALLOC )) +folio_set_active (folio); +
Can you explain this better? What is the significance of marking the folio active here. Do we need to differentiate parallel page faults (across different vmas) w.r.t task_in_lru_fault()?
Quoted 5 lines
mm/swap.c +0
-0folio_get (folio);local_lock (&lru_pvecs.lock); pvec =this_cpu_ptr (&lru_pvecs.lru_add);@@ -563,7 +568,7 @@ static void lru_deactivate_file_fn(struct page *page, struct lruvec *lruvec)
Re: [PATCH v6 4/9] mm: multigenerational lru: groundwork
Quoted 25 lines · Aneesh Kumar K.V
Yu Zhao <[email protected]> writes:
.....
+
Quoted 11 lines
+/*
+ * Evictable pages are divided into multiple generations. The youngest and the
+ * oldest generation numbers,max_seqandmin_seq, are monotonically increasing.
+ * They form a sliding window of a variable size [MIN_NR_GENS,MAX_NR_GENS]. An
+ * offset withinMAX_NR_GENS, gen, indexes the lru list of the corresponding
+ * generation. The gen counter infolio->flagsstores gen+1 while a page is on
+ * lrugen->lists[]. Otherwise, it stores 0.
+ *
+ * A page is added to the youngest generation on faulting. The aging needs to
+ * check the accessed bit at least twice before handing this page over to the
+ * eviction. The first check takes care of the accessed bit set on the initialQuoted 8 lines
+ * fault; the second check makes sure this page hasn't been used since then.
+ * This process, AKA second chance, requires a minimum of two generations,
+ * henceMIN_NR_GENS. And to be compatible with the active/inactive lru, these
+ * two generations are mapped to the active; the rest of generations, if they
+ * exist, are mapped to the inactive.PG_activeis always cleared while a page
+ * is on lrugen->lists[] so that demotion, which happens consequently when the
+ * aging creates a new generation, needs not to worry about it.
+ */
Quoted 2 lines
Where do we clear
PG_activein the code? Is this the reason we endup with
We clear PG_active when we add a page (folio) to MGLRU lists:
include/linux/mm_inline.h
lru_gen_add_folio()
do {
new_flags = old_flags = READ_ONCE(folio->flags);
...
new_flags &= ~(LRU_GEN_MASK | BIT(PG_active));
^^^^^^^^^
...
} while (cmpxchg(&folio->flags, old_flags, new_flags) != old_flags);We also set it when we isolate a page (for page migration):
include/linux/mm_inline.h
lru_gen_del_folio()
do {
new_flags = old_flags = READ_ONCE(folio->flags);
...
else if (lru_gen_is_active(lruvec, gen))
new_flags |= BIT(PG_active);
^^^^^^^^^
} while (cmpxchg(&folio->flags, old_flags, new_flags) != old_flags);Quoted 5 lines
void
deactivate_page(struct page *page)
{
- if (PageLRU(page) && PageActive(page) && !PageUnevictable(page)) {
+ if (PageLRU(page) && !PageUnevictable(page) && (PageActive(page) ||lru_gen_enabled())) {
That's correct.
Quoted 30 lines
Quoted 18 lines
include/linux/mmzone.h +18
-0+#define MIN_NR_GENS 2U +#define MAX_NR_GENS ((unsigned int)CONFIG_NR_LRU_GENS) + +struct lru_gen_struct { +/* the aging increments the youngest generation number */ +unsigned long max_seq; +/* the eviction increments the oldest generation numbers */ +unsigned long min_seq[ANON_AND_FILE ]; +/* the birth time of each generation in jiffies */ +unsigned long timestamps[MAX_NR_GENS ]; +/* the multigenerational lru lists */ +struct list_head lists[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ]; +/* the sizes of the above lists */ +unsigned long nr_pages[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ]; +/* whether the multigenerational lru is enabled */ +bool enabled; +}; +....
Quoted 9 lines
static void
__meminitzone_init_internals(struct zone *zone, enumzone_typeidx, int nid,mm/swap.c +0
-0@@ -462,6 +462,11 @@ void folio_add_lru(struct folio *folio)VM_BUG_ON_FOLIO (folio_test_active (folio) &&folio_test_unevictable (folio), folio);VM_BUG_ON_FOLIO (folio_test_lru (folio), folio);
Quoted 10 lines
mm/swap.c void folio_add_lru(struct folio *folio) · line 469
Quoted 5 lines
mm/swap.c +5
-0+/* see the comment in lru_gen_add_folio() */ +if (lru_gen_enabled () && !folio_test_unevictable (folio) && +task_in_lru_fault () && !(current->flags &PF_MEMALLOC )) +folio_set_active (folio); +Can you explain this better? What is the significance of marking the folio active here. Do we need to differentiate parallel page faults (across different vmas) w.r.t
task_in_lru_fault()?
All pages faulted in need to be added to the youngest generation. But without PG_active, lru_gen_add_folio() doesn't know whether a page was faulted in, or something else, e.g., page cache readahead. This is because pages aren't immediately sent to lru_gen_add_folio(). They are batched by lru_pvecs:
/**
* folio_add_lru - Add a folio to an LRU list.
* @folio: The folio to be added to the LRU.
*
* Queue the folio for addition to the LRU. The decision on whether
* to add the page to the [in]active [file|anon] list is deferred until the
* pagevec is drained. This gives a chance for the caller of folio_add_lru()
* have the folio added to the active list using folio_mark_accessed(). */
void folio_add_lru(struct folio *folio)
{
struct pagevec *pvec;
VM_BUG_ON_FOLIO(folio_test_active(folio) && folio_test_unevictable(folio), folio);
VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
/* see the comment in lru_gen_add_folio() */
if (lru_gen_enabled() && !folio_test_unevictable(folio) &&
lru_gen_in_pgfault() && !(current->flags & PF_MEMALLOC))
folio_set_active(folio);
folio_get(folio);
local_lock(&lru_pvecs.lock);
pvec = this_cpu_ptr(&lru_pvecs.lru_add);
if (pagevec_add_and_need_flush(pvec, &folio->page))
__pagevec_lru_add(pvec);
local_unlock(&lru_pvecs.lock);
}PATCH v6 5/9 mm: multigenerational lru: mm_struct list
In reply to Yu ZhaoTo exploit spatial locality, the aging prefers to walk page tables to search for young PTEs. And this patch paves the way for that.
An mm_struct list is maintained for each memcg, and an mm_struct follows its owner task to the new memcg when this task is migrated.
To avoid confusions, the term "iteration" specifically means the traversal of an entire mm_struct list; the term "walk" will be applied to page tables and the rmap, as usual.
A page table walker, i.e., a thread in the aging path, iterates an mm_struct list and calls walk_page_range() with each mm_struct on this list. The iteration finishes when it reaches the end of this list. When multiple page table walkers iterate the same list, each of them gets a unique mm_struct; therefore the aging can run concurrently.
This infra also provides the following optimizations:
1) It tracks the usage of mm_struct's between context switches so that page table walkers may skip processes that have been sleeping since the last iteration.
2) It provides generational Bloom filters to record populated branches so that page table walkers may reduce their search space based on the query results.
--- fs/exec.c | 2 + include/linux/memcontrol.h | 5 + include/linux/mm_inline.h | 5 + include/linux/mm_types.h | 78 ++++++++ include/linux/mmzone.h | 61 +++++++ kernel/exit.c | 1 + kernel/fork.c | 9 + kernel/sched/core.c | 1 + mm/memcontrol.c | 24 +++ mm/vmscan.c | 352 +++++++++++++++++++++++++++++++++++++ 10 files changed, 538 insertions(+)
fs/exec.c +2 -0
@@ -1005,6 +1005,7 @@ static int exec_mmap(struct mm_struct *mm) active_mm = tsk->active_mm; tsk->active_mm = mm; tsk->mm = mm; +lru_gen_add_mm (mm);/* * This prevents preemption while active_mm is being loaded and * it and mm are being updated, which could cause problems for
@@ -1015,6 +1016,7 @@ static int exec_mmap(struct mm_struct *mm)if (!IS_ENABLED (CONFIG_ARCH_WANT_IRQS_OFF_ACTIVATE_MM ))local_irq_enable ();activate_mm (active_mm, mm); +lru_gen_use_mm (mm);if (IS_ENABLED (CONFIG_ARCH_WANT_IRQS_OFF_ACTIVATE_MM ))local_irq_enable (); tsk->mm->vmacache_seqnum =0 ;
include/linux/memcontrol.h +5 -0
@@ -340,6 +340,11 @@ struct mem_cgroup {struct deferred_split deferred_split_queue;#endif +#ifdef CONFIG_LRU_GEN +/* per-memcg mm_struct list */ +struct lru_gen_mm_list mm_list; +#endif +struct mem_cgroup_per_node *nodeinfo[]; };
include/linux/mm_inline.h +5 -0
@@ -110,6 +110,11 @@ static inline int lru_gen_from_seq(unsigned long seq)return seq %MAX_NR_GENS ; } +static inline int lru_hist_from_seq (unsigned long seq) +{ +return seq %NR_HIST_GENS ; +} +static inline bool lru_gen_is_active (struct lruvec *lruvec,int gen) {unsigned long max_seq = lruvec->lrugen.max_seq;
include/linux/mm_types.h +78 -0
102 lines not on this page · raw message
include/linux/mmzone.h +61 -0
79 lines not on this page · raw message
kernel/exit.c +1 -0
@@ -463,6 +463,7 @@ void mm_update_next_owner(struct mm_struct *mm) goto retry; }WRITE_ONCE (mm->owner, c); +lru_gen_migrate_mm (mm);task_unlock (c);put_task_struct (c); }
kernel/fork.c +9 -0
@@ -1078,6 +1078,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, goto fail_nocontext; mm->user_ns =get_user_ns (user_ns); +lru_gen_init_mm (mm);return mm; fail_nocontext:
@@ -1120,6 +1121,7 @@ static inline void __mmput(struct mm_struct *mm) }if (mm->binfmt)module_put (mm->binfmt->module); +lru_gen_del_mm (mm);mmdrop (mm); }
@@ -2603,6 +2605,13 @@ pid_t kernel_clone(struct kernel_clone_args *args)get_task_struct (p); } +if (IS_ENABLED (CONFIG_LRU_GEN ) && !(clone_flags &CLONE_VM )) { +/* lock the task to synchronize with memcg migration */ +task_lock (p); +lru_gen_add_mm (p->mm); +task_unlock (p); + } +wake_up_new_task (p);/* forking complete and child started to run, tell ptracer */
kernel/sched/core.c +1 -0
@@ -4956,6 +4956,7 @@ context_switch(struct rq *rq, struct task_struct *prev,* finish_task_switch()'s mmdrop(). */ switch_mm_irqs_off (prev->active_mm, next->mm, next); +lru_gen_use_mm (next->mm);if (!prev->mm) {// from kernel /* will mmdrop() in finish_task_switch(). */
mm/memcontrol.c +24 -0
@@ -6135,6 +6135,29 @@ static void mem_cgroup_move_task(void) }#endif +#ifdef CONFIG_LRU_GEN +static void mem_cgroup_attach (struct cgroup_taskset *tset) +{ +struct cgroup_subsys_state *css; +struct task_struct *task =NULL ; + +cgroup_taskset_for_each_leader (task, css, tset) +break ; + +if (!task) +return ; + +task_lock (task); +if (task->mm && task->mm->owner == task) +lru_gen_migrate_mm (task->mm); +task_unlock (task); +} +#else +static void mem_cgroup_attach (struct cgroup_taskset *tset) +{ +} +#endif /* CONFIG_LRU_GEN */ +static int seq_puts_memcg_tunable (struct seq_file *m,unsigned long value) {if (value ==PAGE_COUNTER_MAX )
@@ -6478,6 +6501,7 @@ struct cgroup_subsys memory_cgrp_subsys = { .css_reset = mem_cgroup_css_reset, .css_rstat_flush = mem_cgroup_css_rstat_flush, .can_attach = mem_cgroup_can_attach, + .attach = mem_cgroup_attach, .cancel_attach = mem_cgroup_cancel_attach, .post_attach = mem_cgroup_move_task, .dfl_cftypes = memory_files,
mm/vmscan.c +352 -0
392 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Michal Hocko
maintainerRe: [PATCH v6 5/9] mm: multigenerational lru: mm_struct list
Quoted 5 lines · Yu Zhao
To exploit spatial locality, the aging prefers to walk page tables to search for young PTEs. And this patch paves the way for that.
An
mm_structlist is maintained for each memcg, and anmm_structfollows its owner task to the new memcg when this task is migrated.
How does this work actually for the memcg reclaim? I can see you lru_gen_migrate_mm on the task migration. My concern is, though, that such a task leaves all the memory behind in the previous memcg (in cgroup v2, in v1 you can opt in for charge migration). If you move the mm to a new memcg then you age it somewhere where the memory is not really consumed.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 5/9] mm: multigenerational lru: mm_struct list
Quoted 7 lines · Michal Hocko
Quoted 5 lines · Yu Zhao
To exploit spatial locality, the aging prefers to walk page tables to search for young PTEs. And this patch paves the way for that.
An
mm_structlist is maintained for each memcg, and anmm_structfollows its owner task to the new memcg when this task is migrated.
Quoted 6 lines
How does this work actually for the memcg reclaim? I can see you
lru_gen_migrate_mmon the task migration. My concern is, though, that such a task leaves all the memory behind in the previous memcg (in cgroup v2, in v1 you can opt in for charge migration). If you move the mm to a new memcg then you age it somewhere where the memory is not really consumed.
There are two options to gather the accessed bit: page table walks and rmap walks. Page table walks sweep dense hotspots that are NOT misplaced in terms of reclaim scope (lruvec); rmap walks cover what page table walks miss, e.g., misplaced dense hotspots or sparse ones.
Dense hotspots are stored in Bloom filters for each lruvec.
If an mm leaves everything in the old memcg, page table walks in the new memcg reclaim path basically ignore this mm after the first scan, because everything is misplaced.
In the old memcg reclaim path, page table walks won't see this mm at all. But rmap walks will catch everything later in the eviction path, i.e., lru_gen_look_around(). This function is less efficient compared with page table walks because, for each rmap walk of a non-shared page, it only can gather the accessed bit from 64 PTEs at most. But it's still a lot faster than the original rmap, which only gathers the accessed bit from a single PTE, for each walk of a non-shared page.
Michal Hocko
maintainerRe: [PATCH v6 5/9] mm: multigenerational lru: mm_struct list
Quoted 20 lines · Yu Zhao
Quoted 7 lines · Michal Hocko
Quoted 5 lines · Yu Zhao
To exploit spatial locality, the aging prefers to walk page tables to search for young PTEs. And this patch paves the way for that.
An
mm_structlist is maintained for each memcg, and anmm_structfollows its owner task to the new memcg when this task is migrated.Quoted 6 lines
How does this work actually for the memcg reclaim? I can see you
lru_gen_migrate_mmon the task migration. My concern is, though, that such a task leaves all the memory behind in the previous memcg (in cgroup v2, in v1 you can opt in for charge migration). If you move the mm to a new memcg then you age it somewhere where the memory is not really consumed.There are two options to gather the accessed bit: page table walks and rmap walks. Page table walks sweep dense hotspots that are NOT misplaced in terms of reclaim scope (lruvec); rmap walks cover what page table walks miss, e.g., misplaced dense hotspots or sparse ones.
Quoted 5 lines
Dense hotspots are stored in Bloom filters for each lruvec.
If an mm leaves everything in the old memcg, page table walks in the new memcg reclaim path basically ignore this mm after the first scan, because everything is misplaced.
OK, so do I get it right that pages mapped from a different memcg than the reclaimed one are considered effectivelly non-present from the the reclaim logic POV? This would be worth mentioning in the migration callback because it is not really that straightforward to put those two together.
Quoted 8 lines
In the old memcg reclaim path, page table walks won't see this mm at all. But rmap walks will catch everything later in the eviction path, i.e.,
lru_gen_look_around(). This function is less efficient compared with page table walks because, for each rmap walk of a non-shared page, it only can gather the accessed bit from 64 PTEs at most. But it's still a lot faster than the original rmap, which only gathers the accessed bit from a single PTE, for each walk of a non-shared page.
Again, something that should be really documented.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 5/9] mm: multigenerational lru: mm_struct list
Quoted 27 lines · Michal Hocko
Quoted 20 lines · Yu Zhao
Quoted 7 lines · Michal Hocko
Quoted 5 lines · Yu Zhao
To exploit spatial locality, the aging prefers to walk page tables to search for young PTEs. And this patch paves the way for that.
An
mm_structlist is maintained for each memcg, and anmm_structfollows its owner task to the new memcg when this task is migrated.Quoted 6 lines
How does this work actually for the memcg reclaim? I can see you
lru_gen_migrate_mmon the task migration. My concern is, though, that such a task leaves all the memory behind in the previous memcg (in cgroup v2, in v1 you can opt in for charge migration). If you move the mm to a new memcg then you age it somewhere where the memory is not really consumed.There are two options to gather the accessed bit: page table walks and rmap walks. Page table walks sweep dense hotspots that are NOT misplaced in terms of reclaim scope (lruvec); rmap walks cover what page table walks miss, e.g., misplaced dense hotspots or sparse ones.
Quoted 5 lines
Dense hotspots are stored in Bloom filters for each lruvec.
If an mm leaves everything in the old memcg, page table walks in the new memcg reclaim path basically ignore this mm after the first scan, because everything is misplaced.
Quoted 5 lines
OK, so do I get it right that pages mapped from a different memcg than the reclaimed one are considered effectivelly non-present from the the reclaim logic POV? This would be worth mentioning in the migration callback because it is not really that straightforward to put those two together.
That's correct. Will document this in detail.
Quoted 10 lines
Quoted 8 lines
In the old memcg reclaim path, page table walks won't see this mm at all. But rmap walks will catch everything later in the eviction path, i.e.,
lru_gen_look_around(). This function is less efficient compared with page table walks because, for each rmap walk of a non-shared page, it only can gather the accessed bit from 64 PTEs at most. But it's still a lot faster than the original rmap, which only gathers the accessed bit from a single PTE, for each walk of a non-shared page.Again, something that should be really documented.
Noted.
PATCH v6 6/9 mm: multigenerational lru: aging
In reply to Yu ZhaoTo avoid confusions, the term "scan" will be applied to PTEs in a page table and pages on an lru list. It emphasizes on consecutive elements in a set rather than the data structure holding this set together.
The aging produces young generations. Given an lruvec, it iterates lruvec_memcg()->mm_list and calls walk_page_range() with each mm_struct on this list to scan PTEs for accessed pages. On finding a young PTE, it clears the accessed bit and updates the gen counter of the page mapped by this PTE to (max_seq%MAX_NR_GENS)+1. After each iteration of this list, it increments max_seq. The aging is needed before the eviction can continue when max_seq-min_seq+1 reaches MIN_NR_GENS.
To avoid confusions, the terms "promotion" and "demotion" will be applied to the multigenerational lru, as a new convention; the terms "activation" and "deactivation" will be applied to the active/inactive lru, as usual.
IOW, the aging promotes a page to the youngest generation when it finds this page accessed thru page tables; demotion happens consequently when it creates a new generation. Note that promotion doesn't require any lru list operations in the aging path, only the update of the gen counter and the lru sizes; demotion, unless as the result of the creation of a new generation, requires lru list operations, e.g., lru_deactivate_fn().
The aging uses the following optimizations when walking page tables:
1) It uses the accessed bit in non-leaf PMD entries, the hint from the CPU scheduler and the Bloom filters to reduce its search space.
2) It doesn't zigzag between a PGD table and the same PMD or PTE table spanning multiple VMAs. In other words, it finishes all the VMAs within the range of the same PMD or PTE table before it returns to a PGD table. This improves the cache performance for workloads that have large numbers of tiny VMAs, especially when CONFIG_PGTABLE_LEVELS=5.
The aging is only interested in accessed pages and therefore has the complexity of O(nr_hot_evictable_pages). The worst case scenario is the aging fails to exploit any spatial locality and the eviction has to promote all accessed pages when walking the rmap, which is similar to the active/inactive lru. However, generations still can provide better temporal locality.
--- include/linux/memcontrol.h | 6 + include/linux/mm.h | 5 + include/linux/mmzone.h | 10 + include/linux/oom.h | 16 + include/linux/swap.h | 4 + mm/oom_kill.c | 4 +- mm/rmap.c | 7 + mm/vmscan.c | 896 +++++++++++++++++++++++++++++++++++++ 8 files changed, 946 insertions(+), 2 deletions(-)
include/linux/memcontrol.h +6 -0
@@ -1393,18 +1393,24 @@ mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)static inline void lock_page_memcg (struct page *page) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }static inline void unlock_page_memcg (struct page *page) { +rcu_read_unlock (); }static inline void folio_memcg_lock (struct folio *folio) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }static inline void folio_memcg_unlock (struct folio *folio) { +rcu_read_unlock (); }static inline void mem_cgroup_handle_over_high (void )
include/linux/mm.h +5 -0
@@ -1599,6 +1599,11 @@ static inline unsigned long folio_pfn(struct folio *folio)return page_to_pfn (&folio->page); } +static inline struct folio *pfn_folio (unsigned long pfn) +{ +return page_folio (pfn_to_page (pfn)); +} +/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin pages */ #ifdef CONFIG_MIGRATION static inline bool is_pinnable_page (struct page *page)
include/linux/mmzone.h +10 -0
@@ -304,6 +304,7 @@ enum lruvec_flags { };struct lruvec ; +struct page_vma_mapped_walk ;#define LRU_GEN_MASK ((BIT(LRU_GEN_WIDTH) - 1) << LRU_GEN_PGOFF)#define LRU_REFS_MASK ((BIT(LRU_REFS_WIDTH) - 1) << LRU_REFS_PGOFF)
@@ -410,6 +411,7 @@ struct lru_gen_mm_walk { };void lru_gen_init_state (struct mem_cgroup *memcg,struct lruvec *lruvec); +void lru_gen_look_around (struct page_vma_mapped_walk *pvmw);#ifdef CONFIG_MEMCG void lru_gen_init_memcg (struct mem_cgroup *memcg);
@@ -422,6 +424,10 @@ static inline void lru_gen_init_state(struct mem_cgroup *memcg, struct lruvec *l { } +static inline void lru_gen_look_around (struct page_vma_mapped_walk *pvmw) +{ +} +#ifdef CONFIG_MEMCG static inline void lru_gen_init_memcg (struct mem_cgroup *memcg) {
@@ -1048,6 +1054,10 @@ typedef struct pglist_data {unsigned long flags; +#ifdef CONFIG_LRU_GEN +/* kswap mm walk data */ +struct lru_gen_mm_walk mm_walk; +#endif ZONE_PADDING (_pad2_)/* Per-node vmstats */
include/linux/oom.h +16 -0
@@ -57,6 +57,22 @@ struct oom_control {extern struct mutex oom_lock;extern struct mutex oom_adj_mutex; +#ifdef CONFIG_MMU +extern struct task_struct *oom_reaper_list; +extern struct wait_queue_head oom_reaper_wait; + +static inline bool oom_reaping_in_progress (void ) +{ +/* a racy check can be used to reduce the chance of overkilling */ +return READ_ONCE (oom_reaper_list) || !waitqueue_active (&oom_reaper_wait); +} +#else +static inline bool oom_reaping_in_progress (void ) +{ +return false; +} +#endif +static inline void set_current_oom_origin (void ) { current->signal->oom_flag_origin = true;
include/linux/swap.h +4 -0
@@ -137,6 +137,10 @@ union swap_header {*/ struct reclaim_state {unsigned long reclaimed_slab; +#ifdef CONFIG_LRU_GEN +/* per-thread mm walk data */ +struct lru_gen_mm_walk *mm_walk; +#endif };#ifdef __KERNEL__
mm/oom_kill.c +2 -2
@@ -508,8 +508,8 @@ bool process_shares_mm(struct task_struct *p, struct mm_struct *mm)* victim (if that is possible) to help the OOM killer to move on. */ static struct task_struct *oom_reaper_th; -static DECLARE_WAIT_QUEUE_HEAD (oom_reaper_wait); -static struct task_struct *oom_reaper_list; +DECLARE_WAIT_QUEUE_HEAD (oom_reaper_wait); +struct task_struct *oom_reaper_list;static DEFINE_SPINLOCK (oom_reaper_lock);bool __oom_reap_task_mm (struct mm_struct *mm)
mm/rmap.c +7 -0
@@ -73,6 +73,7 @@#include <linux/page_idle.h> #include <linux/memremap.h> #include <linux/userfaultfd_k.h> +#include <linux/mm_inline.h> #include <asm/tlbflush.h>
@@ -790,6 +791,12 @@ static bool page_referenced_one(struct page *page, struct vm_area_struct *vma, }if (pvmw.pte) { +if (lru_gen_enabled () &&pte_young (*pvmw.pte) && + !(vma->vm_flags & (VM_SEQ_READ |VM_RAND_READ ))) { +lru_gen_look_around (&pvmw); + referenced++; + } +if (ptep_clear_flush_young_notify (vma, address, pvmw.pte)) {/*
mm/vmscan.c +896 -0
938 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
I am still reading through the series. It is a lot of code and quite hard to wrap ones head around so these are mostly random things I have run into. More will likely follow up.
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]
Quoted 17 lines
include/linux/memcontrol.h +3
-0@@ -1393,18 +1393,24 @@ mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)static inline void lock_page_memcg (struct page *page) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }static inline void unlock_page_memcg (struct page *page) { +rcu_read_unlock (); }
include/linux/memcontrol.h mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) · line 1414
Quoted 10 lines
include/linux/memcontrol.h +3
-0static inline void folio_memcg_lock (struct folio *folio) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }static inline void folio_memcg_unlock (struct folio *folio) { +rcu_read_unlock (); }
This should go into a separate patch and merge it independently. I haven't really realized that !MEMCG configuration has a different locking scopes.
[...]
Quoted 13 lines
include/linux/oom.h +5
-0@@ -57,6 +57,22 @@ struct oom_control {extern struct mutex oom_lock;extern struct mutex oom_adj_mutex; +#ifdef CONFIG_MMU +extern struct task_struct *oom_reaper_list; +extern struct wait_queue_head oom_reaper_wait; + +static inline bool oom_reaping_in_progress (void )
include/linux/oom.h struct oom_control { · line 74
Quoted 10 lines
include/linux/oom.h +10
-0+{ +/* a racy check can be used to reduce the chance of overkilling */ +return READ_ONCE (oom_reaper_list) || !waitqueue_active (&oom_reaper_wait); +} +#else +static inline bool oom_reaping_in_progress (void ) +{ +return false; +} +#endif
I do not like this. These are internal oom reaper's and no code should really make any decisions based on that. oom_reaping_in_progress is not telling much anyway. This is a global queue for oom reaper that can contain oom victims from different oom scopes (e.g. global OOM, memcg OOM or memory policy OOM).
Your lru_gen_age_node uses this to decide whether to trigger out_of_memory and that is clearly wrong for the above reasons. out_of_memory is designed to skip over any action if there is an oom victim pending from the oom domain (have a look at oom_evaluate_task).
[...]
include/linux/oom.h struct oom_control {
Quoted 12 lines
include/linux/oom.h +12
-0+static bool age_lruvec (struct lruvec *lruvec,struct scan_control *sc, +unsigned long min_ttl) +{ +bool need_aging; +long nr_to_scan; +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +int swappiness =get_swappiness (memcg); +DEFINE_MAX_SEQ (lruvec); +DEFINE_MIN_SEQ (lruvec); + +if (mem_cgroup_below_min (memcg)) +return false;
mem_cgroup_below_min requires effective values to be calculated for the reclaimed hierarchy. Have a look at mem_cgroup_calculate_protection
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 29 lines · Michal Hocko
I am still reading through the series. It is a lot of code and quite hard to wrap ones head around so these are mostly random things I have run into. More will likely follow up.
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 23 lines
include/linux/memcontrol.h +5
-0@@ -1393,18 +1393,24 @@ mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg)static inline void lock_page_memcg (struct page *page) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }static inline void unlock_page_memcg (struct page *page) { +rcu_read_unlock (); }static inline void folio_memcg_lock (struct folio *folio) { +/* to match folio_memcg_rcu() */ +rcu_read_lock (); }
Quoted 8 lines
include/linux/memcontrol.h mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) · line 1414
Quoted 4 lines
include/linux/memcontrol.h +1
-0static inline void folio_memcg_unlock (struct folio *folio) { +rcu_read_unlock (); }This should go into a separate patch and merge it independently. I haven't really realized that !MEMCG configuration has a different locking scopes.
Considered it done.
Quoted 19 lines
Quoted 19 lines
include/linux/oom.h +11
-0@@ -57,6 +57,22 @@ struct oom_control {extern struct mutex oom_lock;extern struct mutex oom_adj_mutex; +#ifdef CONFIG_MMU +extern struct task_struct *oom_reaper_list; +extern struct wait_queue_head oom_reaper_wait; + +static inline bool oom_reaping_in_progress (void ) +{ +/* a racy check can be used to reduce the chance of overkilling */ +return READ_ONCE (oom_reaper_list) || !waitqueue_active (&oom_reaper_wait); +} +#else +static inline bool oom_reaping_in_progress (void )
Quoted 8 lines
include/linux/oom.h struct oom_control { · line 74
Quoted 4 lines
include/linux/oom.h +4
-0+{ +return false; +} +#endif I do not like this. These are internal oom reaper's and no code should really make any decisions based on that.
oom_reaping_in_progressis not telling much anyway.
There is a perfectly legitimate reason for this.
If there is already a oom kill victim and the oom reaper is making progress, the system may still be under memory pressure until the oom reaping is done. The page reclaim has two choices in this transient
state: kill more processes or keep reclaiming (a few more) hot pages.
The first choice, AKA overkilling, is generally a bad one. The oom reaper is single threaded and it can't go faster with additional victims. Additional processes are sacrificed for nothing -- this is an overcorrection of a system that tries to strike a balance between the tendencies to release memory pressure and to improve memory utilization.
Quoted 3 lines
This is a global queue for oom reaper that can
contain oom victims from different oom scopes (e.g. global OOM, memcg OOM or memory policy OOM).
True, but this is a wrong reason to make the conclusion below. Oom kill scopes do NOT matter; only the pool the freed memory goes into does. And there is only one global pool free pages.
Quoted 2 lines
Your
lru_gen_age_nodeuses this to decide whether to triggerout_of_memoryand that is clearly wrong for the above reasons.
I hope my explanation above is clear enough. There is nothing wrong with the purpose and the usage of oom_reaping_in_progress(), and it has been well tested in the Arch Linux Zen kernel.
Without it, overkills can be easily reproduced by the following simple script. That is additional oom kills happen to processes other than "tail".
# enable zram
while true;
do
tail /dev/zero
doneQuoted 2 lines
out_of_memoryis designed to skip over any action if there is an oom victim pending from the oom domain (have a look atoom_evaluate_task).
Where exactly? Point me to the code please.
I don't see such a logic inside out_of_memory() or oom_evaluate_task(). Currently the only thing that could remotely prevent overkills is oom_lock. But it's inadequate.
This is the entire pipeline:
low on memory -> out_of_memory() -> oom_reaper() -> free memory
To avoid overkills, we need to consider the later half of it too. oom_reaping_in_progress() is exactly for this purpose.
Quoted 7 lines
Quoted 7 lines
include/linux/oom.h +7
-0+static bool age_lruvec (struct lruvec *lruvec,struct scan_control *sc, +unsigned long min_ttl) +{ +bool need_aging; +long nr_to_scan; +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +int swappiness =get_swappiness (memcg);
Quoted 8 lines
include/linux/oom.h
Quoted 5 lines
include/linux/oom.h +5
-0+DEFINE_MAX_SEQ (lruvec); +DEFINE_MIN_SEQ (lruvec); + +if (mem_cgroup_below_min (memcg)) +return false;
mem_cgroup_below_minrequires effective values to be calculated for the reclaimed hierarchy. Have a look atmem_cgroup_calculate_protection
I always keep that in mind, and age_lruvec() is called *after* mem_cgroup_calculate_protection():
balance_pgdat()
memcgs_need_aging = 0
do {
lru_gen_age_node()
if (!memcgs_need_aging) {
memcgs_need_aging = 1
return
}
age_lruvec() shrink_node_memcgs()
mem_cgroup_calculate_protection()
lru_gen_shrink_lruvec()
if ...
memcgs_need_aging = 0
} while ...Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 1 line · Yu Zhao
On Thu, Jan 06, 2022 at 05:06:42PM +0100, Michal Hocko wrote:
[...]
Quoted 50 lines
Quoted 19 lines
Quoted 19 lines
include/linux/oom.h +11
-0@@ -57,6 +57,22 @@ struct oom_control {extern struct mutex oom_lock;extern struct mutex oom_adj_mutex; +#ifdef CONFIG_MMU +extern struct task_struct *oom_reaper_list; +extern struct wait_queue_head oom_reaper_wait; + +static inline bool oom_reaping_in_progress (void ) +{ +/* a racy check can be used to reduce the chance of overkilling */ +return READ_ONCE (oom_reaper_list) || !waitqueue_active (&oom_reaper_wait); +} +#else +static inline bool oom_reaping_in_progress (void )Quoted 8 lines
include/linux/oom.h struct oom_control { · line 74
Quoted 4 lines
include/linux/oom.h +4
-0+{ +return false; +} +#endif I do not like this. These are internal oom reaper's and no code should really make any decisions based on that.
oom_reaping_in_progressis not telling much anyway.There is a perfectly legitimate reason for this.
If there is already a oom kill victim and the oom reaper is making progress, the system may still be under memory pressure until the oom reaping is done. The page reclaim has two choices in this transient
state: kill more processes or keep reclaiming (a few more) hot pages.The first choice, AKA overkilling, is generally a bad one. The oom reaper is single threaded and it can't go faster with additional victims. Additional processes are sacrificed for nothing -- this is an overcorrection of a system that tries to strike a balance between the tendencies to release memory pressure and to improve memory utilization.
Quoted 3 lines
This is a global queue for oom reaper that can
contain oom victims from different oom scopes (e.g. global OOM, memcg OOM or memory policy OOM).True, but this is a wrong reason to make the conclusion below. Oom kill scopes do NOT matter; only the pool the freed memory goes into does. And there is only one global pool free pages.
Quoted 6 lines
Quoted 2 lines
Your
lru_gen_age_nodeuses this to decide whether to triggerout_of_memoryand that is clearly wrong for the above reasons.I hope my explanation above is clear enough. There is nothing wrong with the purpose and the usage of
oom_reaping_in_progress(), and it has been well tested in the Arch Linux Zen kernel.
I disagree. An ongoing oom kill in one domain (say memcg A) shouldn't be any base for any decisions in reclaim in other domain (say memcg B or even a global reclaim). Those are fundamentally different conditions.
Quoted 9 lines
Without it, overkills can be easily reproduced by the following simple script. That is additional oom kills happen to processes other than "tail".
# enable zram while true; do tail /dev/zero done
I would be interested to hear more (care to send oom reports?).
Quoted 8 lines
Quoted 2 lines
out_of_memoryis designed to skip over any action if there is an oom victim pending from the oom domain (have a look atoom_evaluate_task).Where exactly? Point me to the code please.
I don't see such a logic inside
out_of_memory()oroom_evaluate_task(). Currently the only thing that could remotely prevent overkills isoom_lock. But it's inadequate.
OK, let me try to exaplain. The protocol is rather convoluted. Once the oom killer is invoked it choses a victim to kill. oom_evaluate_task will evaluate _all_ tasks from the oom respective domain (select_bad_process which distinguishes memcg vs global oom kill and oom_cpuset_eligible for the cpuset domains). If there is any pre-existing oom victim (tsk_is_oom_victim) then the scan is aborted and the oom killer bails out. OOM victim stops being considered as relevant once the oom reaper manages to release its address space (or give up on the mmap_sem contention) and sets MMF_OOM_SKIP flag for the mm.
That being said the out_of_memory automatically backs off and relies on the oom reaper to process its queue.
Does it make more clear for you now?
Quoted 35 lines
This is the entire pipeline:
low on memory ->out_of_memory()->oom_reaper()-> free memoryTo avoid overkills, we need to consider the later half of it too.
oom_reaping_in_progress()is exactly for this purpose.Quoted 7 lines
Quoted 7 lines
include/linux/oom.h +7
-0+static bool age_lruvec (struct lruvec *lruvec,struct scan_control *sc, +unsigned long min_ttl) +{ +bool need_aging; +long nr_to_scan; +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +int swappiness =get_swappiness (memcg);Quoted 8 lines
include/linux/oom.h
Quoted 5 lines
include/linux/oom.h +5
-0+DEFINE_MAX_SEQ (lruvec); +DEFINE_MIN_SEQ (lruvec); + +if (mem_cgroup_below_min (memcg)) +return false;
mem_cgroup_below_minrequires effective values to be calculated for the reclaimed hierarchy. Have a look atmem_cgroup_calculate_protectionI always keep that in mind, and
age_lruvec()is called *after*mem_cgroup_calculate_protection():balance_pgdat() memcgs_need_aging = 0 do { lru_gen_age_node() if (!memcgs_need_aging) { memcgs_need_aging = 1 return } age_lruvec()
Quoted 6 lines
shrink_node_memcgs()
mem_cgroup_calculate_protection()
lru_gen_shrink_lruvec()
if ...
memcgs_need_aging = 0
} while ...Uff, this is really subtle. I really think you should be following the existing pattern when the effective values are calculated right in the same context as they are evaluated.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 56 lines · Michal Hocko
Quoted 1 line · Yu Zhao
On Thu, Jan 06, 2022 at 05:06:42PM +0100, Michal Hocko wrote:
[...]
Quoted 53 lines
Quoted 19 lines
Quoted 19 lines
include/linux/oom.h +11
-0@@ -57,6 +57,22 @@ struct oom_control {extern struct mutex oom_lock;extern struct mutex oom_adj_mutex; +#ifdef CONFIG_MMU +extern struct task_struct *oom_reaper_list; +extern struct wait_queue_head oom_reaper_wait; + +static inline bool oom_reaping_in_progress (void ) +{ +/* a racy check can be used to reduce the chance of overkilling */ +return READ_ONCE (oom_reaper_list) || !waitqueue_active (&oom_reaper_wait); +} +#else +static inline bool oom_reaping_in_progress (void )Quoted 8 lines
include/linux/oom.h struct oom_control { · line 74
Quoted 4 lines
include/linux/oom.h +4
-0+{ +return false; +} +#endif I do not like this. These are internal oom reaper's and no code should really make any decisions based on that.
oom_reaping_in_progressis not telling much anyway.There is a perfectly legitimate reason for this.
If there is already a oom kill victim and the oom reaper is making progress, the system may still be under memory pressure until the oom reaping is done. The page reclaim has two choices in this transient
state: kill more processes or keep reclaiming (a few more) hot pages.The first choice, AKA overkilling, is generally a bad one. The oom reaper is single threaded and it can't go faster with additional victims. Additional processes are sacrificed for nothing -- this is an overcorrection of a system that tries to strike a balance between the tendencies to release memory pressure and to improve memory utilization.
Quoted 3 lines
This is a global queue for oom reaper that can
contain oom victims from different oom scopes (e.g. global OOM, memcg OOM or memory policy OOM).True, but this is a wrong reason to make the conclusion below. Oom kill scopes do NOT matter; only the pool the freed memory goes into does. And there is only one global pool free pages.
Quoted 2 lines
Your
lru_gen_age_nodeuses this to decide whether to triggerout_of_memoryand that is clearly wrong for the above reasons.
Quoted 7 lines
Quoted 3 lines
I hope my explanation above is clear enough. There is nothing wrong with the purpose and the usage of
oom_reaping_in_progress(), and it has been well tested in the Arch Linux Zen kernel.I disagree. An ongoing oom kill in one domain (say memcg A) shouldn't be any base for any decisions in reclaim in other domain (say memcg B or even a global reclaim). Those are fundamentally different conditions.
I agree for the memcg A oom and memcg B reclaim case, because memory freed from A doesn't go to B.
I still think for the memcg A and the global reclaim case, memory freed from A can be considered when deciding whether to make more kills during global reclaim.
But this is something really minor, and I'll go with your suggestion, i.e., getting rid of oom_reaping_in_progress().
Quoted 11 lines
Quoted 9 lines
Without it, overkills can be easily reproduced by the following simple script. That is additional oom kills happen to processes other than "tail".
# enable zram while true; do tail /dev/zero doneI would be interested to hear more (care to send oom reports?).
I agree with what said below. I think those additional ooms might have been from different oom domains. I plan to leave this for now and go with your suggestion as mentioned above.
Quoted 19 lines
Quoted 8 lines
Quoted 2 lines
out_of_memoryis designed to skip over any action if there is an oom victim pending from the oom domain (have a look atoom_evaluate_task).Where exactly? Point me to the code please.
I don't see such a logic inside
out_of_memory()oroom_evaluate_task(). Currently the only thing that could remotely prevent overkills isoom_lock. But it's inadequate.OK, let me try to exaplain. The protocol is rather convoluted. Once the oom killer is invoked it choses a victim to kill.
oom_evaluate_taskwill evaluate _all_ tasks from the oom respective domain (select_bad_processwhich distinguishes memcg vs global oom kill andoom_cpuset_eligiblefor the cpuset domains). If there is any pre-existing oom victim (tsk_is_oom_victim) then the scan is aborted and the oom killer bails out. OOM victim stops being considered as relevant once the oom reaper manages to release its address space (or give up on themmap_semcontention) and setsMMF_OOM_SKIPflag for the mm.
Quoted 4 lines
That being said the
out_of_memoryautomatically backs off and relies on the oom reaper to process its queue.Does it make more clear for you now?
Yes, you are right, thanks.
Quoted 42 lines
Quoted 35 lines
This is the entire pipeline:
low on memory ->out_of_memory()->oom_reaper()-> free memoryTo avoid overkills, we need to consider the later half of it too.
oom_reaping_in_progress()is exactly for this purpose.Quoted 7 lines
Quoted 7 lines
include/linux/oom.h +7
-0+static bool age_lruvec (struct lruvec *lruvec,struct scan_control *sc, +unsigned long min_ttl) +{ +bool need_aging; +long nr_to_scan; +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +int swappiness =get_swappiness (memcg);Quoted 8 lines
include/linux/oom.h
Quoted 5 lines
include/linux/oom.h +5
-0+DEFINE_MAX_SEQ (lruvec); +DEFINE_MIN_SEQ (lruvec); + +if (mem_cgroup_below_min (memcg)) +return false;
mem_cgroup_below_minrequires effective values to be calculated for the reclaimed hierarchy. Have a look atmem_cgroup_calculate_protectionI always keep that in mind, and
age_lruvec()is called *after*mem_cgroup_calculate_protection():balance_pgdat() memcgs_need_aging = 0 do { lru_gen_age_node() if (!memcgs_need_aging) { memcgs_need_aging = 1 return } age_lruvec()Quoted 6 lines
shrink_node_memcgs() mem_cgroup_calculate_protection() lru_gen_shrink_lruvec() if ... memcgs_need_aging = 0 } while ...
Quoted 3 lines
Uff, this is really subtle. I really think you should be following the existing pattern when the effective values are calculated right in the same context as they are evaluated.
Consider it done.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Yu Zhaomm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3951
Quoted 13 lines · Yu Zhao
mm/vmscan.c +13
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL ); + +return current->reclaim_state->mm_walk; +} + +static void free_mm_walk (struct lru_gen_mm_walk *walk) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +kvfree (walk); +}
Do I get it right that you are allocating from the reclaim context? What prevents this to completely deplete the memory as the reclaim context is PF_MEMALLOC?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Yu Zhao
mm/vmscan.c +9
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL ); + +return current->reclaim_state->mm_walk; +} + +static void free_mm_walk (struct lru_gen_mm_walk *walk)
Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3951
Quoted 4 lines
mm/vmscan.c +4
-0+{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +kvfree (walk); +}Do I get it right that you are allocating from the reclaim context? What prevents this to completely deplete the memory as the reclaim context is
PF_MEMALLOC?
Yes, and in general the same reason zram/zswap/etc. allocate memory in the reclaim context: to make more free memory.
In this case, lru_gen_mm_walk is small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 20 lines · Yu Zhao
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Yu Zhao
mm/vmscan.c +9
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL ); + +return current->reclaim_state->mm_walk; +} + +static void free_mm_walk (struct lru_gen_mm_walk *walk)Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3951
Quoted 4 lines
mm/vmscan.c +4
-0+{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +kvfree (walk); +}Do I get it right that you are allocating from the reclaim context? What prevents this to completely deplete the memory as the reclaim context is
PF_MEMALLOC?
Quoted 2 lines
Yes, and in general the same reason zram/zswap/etc. allocate memory in the reclaim context: to make more free memory.
I have to admit that I am not really familiar with zram/zswap but I find the concept of requiring memory to do the reclaim really problematic.
Quoted 5 lines
In this case,
lru_gen_mm_walkis small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.
Well, this is not a strong argument to be honest. Kswapd being stuck and the majority of the reclaim being done in the direct reclaim context is a situation I have seen many many times. We used to have problems with direct reclaimers throttling to prevent an over eager OOM situations.
Have you considered using a pool of preallocated objects instead?
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
On Fri 07-01-22 09:55:09, Michal Hocko wrote:
[...]
Quoted 9 lines
Quoted 5 lines
In this case,
lru_gen_mm_walkis small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.Well, this is not a strong argument to be honest. Kswapd being stuck and the majority of the reclaim being done in the direct reclaim context is a situation I have seen many many times.
Also do not forget that memcg reclaim is effectivelly only direct reclaim. Not that the memcg reclaim indicates a global memory shortage but it can add up and race with the global reclaim as well.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 8 lines · Michal Hocko
On Fri 07-01-22 09:55:09, Michal Hocko wrote:
[...]Quoted 6 lines
Quoted 5 lines
In this case,
lru_gen_mm_walkis small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.
Quoted 7 lines
Quoted 3 lines
Well, this is not a strong argument to be honest. Kswapd being stuck and the majority of the reclaim being done in the direct reclaim context is a situation I have seen many many times.
Also do not forget that memcg reclaim is effectivelly only direct reclaim. Not that the memcg reclaim indicates a global memory shortage but it can add up and race with the global reclaim as well.
I don't dispute any of the above, and I probably don't like this code more than you do.
But let's not forget the purposes of PF_MEMALLOC, besides preventing recursive reclaims, include letting reclaim dip into reserves so that it can make more free memory. So I think it's acceptable if the following conditions are met:
1. The allocation size is small.
2. The number of allocations is bounded.
3. Its failure doesn't stall reclaim.
And it'd be nice if
4. The allocation happens rarely, e.g., slow path only.
The code in question meets all of them.
1. This allocation is 160 bytes.
2. It's bounded by the number of page table walkers which, in the worst, is same as the number of mm_struct's.
3. Most importantly, its failure doesn't stall the aging. The aging will fallback to the rmap-based function lru_gen_look_around(). But this function only gathers the accessed bit from at most 64 PTEs, meaning it's less efficient (retains ~80% performance gains).
4. This allocation is rare, i.e., only when the aging is required, which is similar to the low inactive case for the active/inactive lru.
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 20 lines · Yu Zhao
Quoted 8 lines · Michal Hocko
On Fri 07-01-22 09:55:09, Michal Hocko wrote:
[...]Quoted 6 lines
Quoted 5 lines
In this case,
lru_gen_mm_walkis small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.Quoted 7 lines
Quoted 3 lines
Well, this is not a strong argument to be honest. Kswapd being stuck and the majority of the reclaim being done in the direct reclaim context is a situation I have seen many many times.
Also do not forget that memcg reclaim is effectivelly only direct reclaim. Not that the memcg reclaim indicates a global memory shortage but it can add up and race with the global reclaim as well.
I don't dispute any of the above, and I probably don't like this code more than you do.
Quoted 9 lines
But let's not forget the purposes of
PF_MEMALLOC, besides preventing recursive reclaims, include letting reclaim dip into reserves so that it can make more free memory. So I think it's acceptable if the following conditions are met:
1. The allocation size is small.
2. The number of allocations is bounded.
3. Its failure doesn't stall reclaim.
And it'd be nice if
4. The allocation happens rarely, e.g., slow path only.
I would add
0. The allocation should be done only if absolutely _necessary_.
Please keep in mind that whatever you allocate from that context will be consuming a very precious memory reserves which are shared with other components of the system. Even worse these can go all the way to depleting memory completely where other things can fall apart.
Quoted 12 lines
The code in question meets all of them.
1. This allocation is 160 bytes.
2. It's bounded by the number of page table walkers which, in the worst, is same as the number of mm_struct's.
3. Most importantly, its failure doesn't stall the aging. The aging will fallback to the rmap-based functionlru_gen_look_around(). But this function only gathers the accessed bit from at most 64 PTEs, meaning it's less efficient (retains ~80% performance gains).
4. This allocation is rare, i.e., only when the aging is required, which is similar to the low inactive case for the active/inactive lru.
I think this fallback behavior deserves much more detailed explanation in changelogs.
Quoted 4 lines
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
I would disagree here. This is not an optimization. You should be avoiding allocations from the memory reclaim because any allocation just add a runtime behavior complexity and potential corner cases.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 31 lines · Michal Hocko
Quoted 20 lines · Yu Zhao
Quoted 8 lines · Michal Hocko
On Fri 07-01-22 09:55:09, Michal Hocko wrote:
[...]Quoted 6 lines
Quoted 5 lines
In this case,
lru_gen_mm_walkis small (160 bytes); it's per direct reclaimer; and direct reclaimers rarely come here, i.e., only when kswapd can't keep up in terms of the aging, which is similar to the condition where the inactive list is empty for the active/inactive lru.Quoted 7 lines
Quoted 3 lines
Well, this is not a strong argument to be honest. Kswapd being stuck and the majority of the reclaim being done in the direct reclaim context is a situation I have seen many many times.
Also do not forget that memcg reclaim is effectivelly only direct reclaim. Not that the memcg reclaim indicates a global memory shortage but it can add up and race with the global reclaim as well.
I don't dispute any of the above, and I probably don't like this code more than you do.
Quoted 9 lines
But let's not forget the purposes of
PF_MEMALLOC, besides preventing recursive reclaims, include letting reclaim dip into reserves so that it can make more free memory. So I think it's acceptable if the following conditions are met:
1. The allocation size is small.
2. The number of allocations is bounded.
3. Its failure doesn't stall reclaim.
And it'd be nice if
4. The allocation happens rarely, e.g., slow path only.
Quoted 7 lines
I would add
0. The allocation should be done only if absolutely _necessary_.Please keep in mind that whatever you allocate from that context will be consuming a very precious memory reserves which are shared with other components of the system. Even worse these can go all the way to depleting memory completely where other things can fall apart.
I agree but I also see a distinction:
1,2,3 are objective;
0,4 are subjective.
For some users, page reclaim itself could be not absolutely necessary because they are okay with OOM kills. But for others, the situation could be reversed.
Quoted 13 lines
Quoted 12 lines
The code in question meets all of them.
1. This allocation is 160 bytes.
2. It's bounded by the number of page table walkers which, in the worst, is same as the number of mm_struct's.
3. Most importantly, its failure doesn't stall the aging. The aging will fallback to the rmap-based functionlru_gen_look_around(). But this function only gathers the accessed bit from at most 64 PTEs, meaning it's less efficient (retains ~80% performance gains).
4. This allocation is rare, i.e., only when the aging is required, which is similar to the low inactive case for the active/inactive lru.
Quoted 2 lines
I think this fallback behavior deserves much more detailed explanation in changelogs.
Will do.
Quoted 8 lines
Quoted 4 lines
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
I would disagree here. This is not an optimization. You should be avoiding allocations from the memory reclaim because any allocation just add a runtime behavior complexity and potential corner cases.
Would __GFP_NOMEMALLOC address your concern? It prevents allocations from accessing the reserves even under PF_MEMALLOC.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
On Thu 13-01-22 02:43:38, Yu Zhao wrote:
[...]
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
I would disagree here. This is not an optimization. You should be avoiding allocations from the memory reclaim because any allocation just add a runtime behavior complexity and potential corner cases.
Would
__GFP_NOMEMALLOCaddress your concern? It prevents allocations from accessing the reserves even underPF_MEMALLOC.
__GFP_NOMEMALLOC would deal with the complete memory depletion concern for sure but I am not sure how any of these allocations would succeed when called from the direct reclaim. Some access to memory reserves is necessary if you insist on allocating from the reclaim process.
You can have a look at the limited memory reserves access by oom victims for an example of how this can be done.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 14 lines · Michal Hocko
On Thu 13-01-22 02:43:38, Yu Zhao wrote:
[...]Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
I would disagree here. This is not an optimization. You should be avoiding allocations from the memory reclaim because any allocation just add a runtime behavior complexity and potential corner cases.
Would
__GFP_NOMEMALLOCaddress your concern? It prevents allocations from accessing the reserves even underPF_MEMALLOC.
Quoted 7 lines
__GFP_NOMEMALLOCwould deal with the complete memory depletion concern for sure but I am not sure how any of these allocations would succeed when called from the direct reclaim. Some access to memory reserves is necessary if you insist on allocating from the reclaim process.You can have a look at the limited memory reserves access by oom victims for an example of how this can be done.
Thanks. I'll change GFP_KERNEL to __GFP_HIGH | __GFP_NOMEMALLOC. __GFP_HIGH allows some access to memory reserves and __GFP_NOMEMALLOC prevents the complete depletion. Basically the combination lower the min watermark by 1/2, and we have been using them for add_to_swap_cache().
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 20 lines · Yu Zhao
Quoted 19 lines · Michal Hocko
On Thu 13-01-22 02:43:38, Yu Zhao wrote:
[...]Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
The bottom line is I can try various optimizations, e.g., preallocate a few buffers for a limited number of page walkers and if this number has been reached, fallback to the rmap-based function. But I have yet to see evidence that calls for additional complexity.
I would disagree here. This is not an optimization. You should be avoiding allocations from the memory reclaim because any allocation just add a runtime behavior complexity and potential corner cases.
Would
__GFP_NOMEMALLOCaddress your concern? It prevents allocations from accessing the reserves even underPF_MEMALLOC.
__GFP_NOMEMALLOCwould deal with the complete memory depletion concern for sure but I am not sure how any of these allocations would succeed when called from the direct reclaim. Some access to memory reserves is necessary if you insist on allocating from the reclaim process.
Quoted 8 lines
Quoted 2 lines
You can have a look at the limited memory reserves access by oom victims for an example of how this can be done.
Thanks. I'll change
GFP_KERNELto__GFP_HIGH|__GFP_NOMEMALLOC.__GFP_HIGHallows some access to memory reserves and__GFP_NOMEMALLOCprevents the complete depletion. Basically the combination lower the min watermark by 1/2, and we have been using them foradd_to_swap_cache().
Yes this will prevent the complete memory depletion. There are other users of this portion of memory reserves so the reclaim might be out of luck. How this turns out in practice remains to be seen though but it certainly is an opportunity for corner cases and hard to test behavior.
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Michal HockoQuoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );
One thing I have overlooked completely. You cannot really use GFP_KERNEL allocation here because the reclaim context can be constrained (e.g. GFP_NOFS). This allocation will not do any reclaim as it is PF_MEMALLOC but I suspect that the lockdep will complain anyway.
Also kvmalloc is not really great here. a) vmalloc path is never executed for small objects and b) we do not really want to make a dependency between vmalloc and the reclaim (by vmalloc -> reclaim -> vmalloc).
Even if we rule out vmalloc and look at kmalloc alone. Is this really safe? I do not see any recursion prevention in the SL.B code. Maybe this just happens to work but the dependency should be really documented so that future SL.B changes won't break the whole scheme.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 17 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );One thing I have overlooked completely. You cannot really use
GFP_KERNELallocation here because the reclaim context can be constrained (e.g.GFP_NOFS). This allocation will not do any reclaim as it isPF_MEMALLOCbut I suspect that the lockdep will complain anyway.Also kvmalloc is not really great here. a) vmalloc path is never executed for small objects and b) we do not really want to make a dependency between vmalloc and the reclaim (by vmalloc -> reclaim -> vmalloc).
Quoted 4 lines
Even if we rule out vmalloc and look at kmalloc alone. Is this really safe? I do not see any recursion prevention in the SL.B code. Maybe this just happens to work but the dependency should be really documented so that future SL.B changes won't break the whole scheme.
Slab implementations drop all locks before calling into page allocator (thus possibly reclaim) so slab itself should be fine and I don't expect it to change. But we could eventually reach the page allocator recursively again, that's true and not great.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 23 lines · Vlastimil Babka
Quoted 17 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );One thing I have overlooked completely. You cannot really use
GFP_KERNELallocation here because the reclaim context can be constrained (e.g.GFP_NOFS). This allocation will not do any reclaim as it isPF_MEMALLOCbut I suspect that the lockdep will complain anyway.Also kvmalloc is not really great here. a) vmalloc path is never executed for small objects and b) we do not really want to make a dependency between vmalloc and the reclaim (by vmalloc -> reclaim -> vmalloc).
Quoted 4 lines
Even if we rule out vmalloc and look at kmalloc alone. Is this really safe? I do not see any recursion prevention in the SL.B code. Maybe this just happens to work but the dependency should be really documented so that future SL.B changes won't break the whole scheme.
Quoted 4 lines
Slab implementations drop all locks before calling into page allocator (thus possibly reclaim) so slab itself should be fine and I don't expect it to change. But we could eventually reach the page allocator recursively again, that's true and not great.
Thanks for double checking. If recursion is really intended and something SL.B allocators should support then this is definitely worth documenting so that a subtle change won't break in the future.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Michal HockoQuoted 8 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );One thing I have overlooked completely.
I appreciate your attention to details but GFP_KERNEL is legit in the reclaim path. It's been used many years in our production, e.g.,
page reclaim
swap_writepage()
frontswap_store()
zswap_frontswap_store()
zswap_entry_cache_alloc(GFP_KERNEL)
(And I always test my changes with lockdep, kasan, DEBUG_VM, etc., no
warnings ever seen from using GFP_KERNEL in the reclaim path.)
Quoted 10 lines
You cannot really use
GFP_KERNEL
allocation here because the reclaim context can be constrained (e.g.GFP_NOFS). This allocation will not do any reclaim as it isPF_MEMALLOCbut I suspect that the lockdep will complain anyway.Also kvmalloc is not really great here. a) vmalloc path is never executed for small objects and b) we do not really want to make a dependency between vmalloc and the reclaim (by vmalloc -> reclaim -> vmalloc).
Quoted 4 lines
Even if we rule out vmalloc and look at kmalloc alone. Is this really safe? I do not see any recursion prevention in the SL.B code. Maybe this just happens to work but the dependency should be really documented so that future SL.B changes won't break the whole scheme.
Affirmative, as Vlastimil has clarified.
Thanks!
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 18 lines · Yu Zhao
Quoted 8 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );One thing I have overlooked completely.
I appreciate your attention to details but
GFP_KERNELis legit in the reclaim path. It's been used many years in our production, e.g.,
page reclaim
swap_writepage()
frontswap_store()
zswap_frontswap_store()
zswap_entry_cache_alloc(GFP_KERNEL)
Quoted 2 lines
(And I always test my changes with lockdep, kasan,
DEBUG_VM, etc., no
warnings ever seen from usingGFP_KERNELin the reclaim path.)
OK, I can see it now. __need_reclaim will check for PF_MEMALLOC and skip the fs_reclaim tracking.
I still maintain I am not really happy about (nor in the zswap example) allocations from the direct reclaim context. I would really recommend using a pre-allocated pool of objects.
If there are strong reasons for not doing so then at lease change that to kzalloc.
Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 22 lines · Michal Hocko
Quoted 18 lines · Yu Zhao
Quoted 8 lines · Michal Hocko
Quoted 5 lines · Michal Hocko
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3942
Quoted 4 lines · Yu Zhao
mm/vmscan.c +4
-0+static struct lru_gen_mm_walk *alloc_mm_walk (void ) +{ +if (!current->reclaim_state || !current->reclaim_state->mm_walk) +return kvzalloc (sizeof (struct lru_gen_mm_walk ),GFP_KERNEL );One thing I have overlooked completely.
I appreciate your attention to details but
GFP_KERNELis legit in the reclaim path. It's been used many years in our production, e.g.,
page reclaim
swap_writepage()
frontswap_store()
zswap_frontswap_store()
zswap_entry_cache_alloc(GFP_KERNEL)Quoted 2 lines
(And I always test my changes with lockdep, kasan,
DEBUG_VM, etc., no
warnings ever seen from usingGFP_KERNELin the reclaim path.)
Quoted 6 lines
OK, I can see it now.
__need_reclaimwill check forPF_MEMALLOCand skip thefs_reclaimtracking.I still maintain I am not really happy about (nor in the zswap example) allocations from the direct reclaim context. I would really recommend using a pre-allocated pool of objects.
Not trying to argue anything -- there are many other places in the reclaim path that must allocate memory to make progress, e.g.,
add_to_swap_cache()
xas_nomem()
__swap_writepage()
bio_alloc()
The only way to not allocate memory is drop clean pages. Writing dirty pages (not swap) might require allocations as well. (But we only write dirty pages in kswapd, not in the direct reclaim path.)
Quoted 2 lines
If there are strong reasons for not doing so then at lease change that to kzalloc.
Consider it done.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Yu ZhaoOn Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]
Quoted 21 lines
mm/vmscan.c +21
-0+static void lru_gen_age_node (struct pglist_data *pgdat,struct scan_control *sc) +{ +struct mem_cgroup *memcg; +bool success = false; +unsigned long min_ttl =READ_ONCE (lru_gen_min_ttl); + +VM_BUG_ON (!current_is_kswapd ()); + + current->reclaim_state->mm_walk = &pgdat->mm_walk; + + memcg =mem_cgroup_iter (NULL ,NULL ,NULL ); +do { +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat); + +if (age_lruvec (lruvec, sc, min_ttl)) + success = true; + +cond_resched (); + }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL ))); + +if (!success &&mutex_trylock (&oom_lock)) {
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 4214
Quoted 10 lines
mm/vmscan.c +10
-0+struct oom_control oc = { + .gfp_mask = sc->gfp_mask, + .order = sc->order, + }; + +if (!oom_reaping_in_progress ()) +out_of_memory (&oc); + +mutex_unlock (&oom_lock); + }
Why do you need to trigger oom killer from this path? Why cannot you rely on the page allocator to do that like we do now?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void lru_gen_age_node (struct pglist_data *pgdat,struct scan_control *sc) +{ +struct mem_cgroup *memcg; +bool success = false; +unsigned long min_ttl =READ_ONCE (lru_gen_min_ttl); + +VM_BUG_ON (!current_is_kswapd ()); + + current->reclaim_state->mm_walk = &pgdat->mm_walk; + + memcg =mem_cgroup_iter (NULL ,NULL ,NULL ); +do { +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat); + +if (age_lruvec (lruvec, sc, min_ttl)) + success = true; + +cond_resched (); + }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL ))); + +if (!success &&mutex_trylock (&oom_lock)) { +struct oom_control oc = { + .gfp_mask = sc->gfp_mask, + .order = sc->order, + }; +
Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 4214
Quoted 5 lines
mm/vmscan.c +5
-0+if (!oom_reaping_in_progress ()) +out_of_memory (&oc); + +mutex_unlock (&oom_lock); + }Why do you need to trigger oom killer from this path? Why cannot you rely on the page allocator to do that like we do now?
This is per desktop users' (repeated) requests. The can't tolerate thrashing as servers do because of UI lags; and they usually don't have fancy tools like oomd.
Related discussions I saw:
https://github.com/zen-kernel/zen-kernel/issues/218
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
From patch 8:
Personal computers
------------------
:Thrashing prevention: Write ``N`` to
``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of ``N`` milliseconds from getting evicted. The OOM killer is invoked if this working set can't be kept in memory. Based on the average human detectable lag (~100ms), ``N=1000`` usually eliminates intolerable lags due to thrashing. Larger values like ``N=3000`` make lags less noticeable at the cost of more OOM kills.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 42 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void lru_gen_age_node (struct pglist_data *pgdat,struct scan_control *sc) +{ +struct mem_cgroup *memcg; +bool success = false; +unsigned long min_ttl =READ_ONCE (lru_gen_min_ttl); + +VM_BUG_ON (!current_is_kswapd ()); + + current->reclaim_state->mm_walk = &pgdat->mm_walk; + + memcg =mem_cgroup_iter (NULL ,NULL ,NULL ); +do { +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat); + +if (age_lruvec (lruvec, sc, min_ttl)) + success = true; + +cond_resched (); + }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL ))); + +if (!success &&mutex_trylock (&oom_lock)) { +struct oom_control oc = { + .gfp_mask = sc->gfp_mask, + .order = sc->order, + }; +Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 4214
Quoted 5 lines
mm/vmscan.c +5
-0+if (!oom_reaping_in_progress ()) +out_of_memory (&oc); + +mutex_unlock (&oom_lock); + }Why do you need to trigger oom killer from this path? Why cannot you rely on the page allocator to do that like we do now?
This is per desktop users' (repeated) requests. The can't tolerate thrashing as servers do because of UI lags; and they usually don't have fancy tools like oomd.
Quoted 6 lines
Related discussions I saw:
https://github.com/zen-kernel/zen-kernel/issues/218
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
I do not really see any arguments why an userspace based trashing detection cannot be used for those. Could you clarify?
Also my question was pointing to why out_of_memory is called from the reclaim rather than the allocator (memcg charging path). It is the caller of the reclaim to control different reclaim strategies and tell when all the hopes are lost and the oom killer should be invoked. This allows for a different policies at the allocator level and this change will break that AFAICS. E.g. what if the underlying allocation context is __GFP_NORETRY?
Quoted 10 lines
Quoted 1 line
From patch 8:
Personal computers
------------------
:Thrashing prevention: Write ``N`` to
``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of ``N`` milliseconds from getting evicted. The OOM killer is invoked if this working set can't be kept in memory. Based on the average human detectable lag (~100ms), ``N=1000`` usually eliminates intolerable lags due to thrashing. Larger values like ``N=3000`` make lags less noticeable at the cost of more OOM kills.
This is a very good example of something that should be a self contained patch with its own justification. TBH it is really not all that clear to me that we want to provide any user visible knob to control OOM behavior based on a time based QoS.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 50 lines · Michal Hocko
Quoted 42 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void lru_gen_age_node (struct pglist_data *pgdat,struct scan_control *sc) +{ +struct mem_cgroup *memcg; +bool success = false; +unsigned long min_ttl =READ_ONCE (lru_gen_min_ttl); + +VM_BUG_ON (!current_is_kswapd ()); + + current->reclaim_state->mm_walk = &pgdat->mm_walk; + + memcg =mem_cgroup_iter (NULL ,NULL ,NULL ); +do { +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat); + +if (age_lruvec (lruvec, sc, min_ttl)) + success = true; + +cond_resched (); + }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL ))); + +if (!success &&mutex_trylock (&oom_lock)) { +struct oom_control oc = { + .gfp_mask = sc->gfp_mask, + .order = sc->order, + }; +Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 4214
Quoted 5 lines
mm/vmscan.c +5
-0+if (!oom_reaping_in_progress ()) +out_of_memory (&oc); + +mutex_unlock (&oom_lock); + }Why do you need to trigger oom killer from this path? Why cannot you rely on the page allocator to do that like we do now?
This is per desktop users' (repeated) requests. The can't tolerate thrashing as servers do because of UI lags; and they usually don't have fancy tools like oomd.
Quoted 2 lines
I do not really see any arguments why an userspace based trashing detection cannot be used for those. Could you clarify?
It definitely can be done. But who is going to do it for every distro and all individual users? AFAIK, not a single distro provides such a solution for desktop/laptop/phone users.
And also there is the theoretical question how reliable a userspace solution can be. What if this usespace solution itself gets stuck in the direct reclaim path. I'm sure if nobody has done some search to prove or debunk it.
In addition, what exactly PSI values should be used on different models of consumer electronics? Nobody knows. We have a team working on this and we haven't figured it out for all our Chromebook models.
As Andrew said, "a blunt instrument like this would be useful".
https://lore.kernel.org/lkml/[email protected]/
I'd like to have less code in kernel too, but I've learned never to walk over users. If I remove this and they come after me asking why, I'd have a hard time convincing them.
Quoted 7 lines
Also my question was pointing to why
out_of_memoryis called from the reclaim rather than the allocator (memcg charging path). It is the caller of the reclaim to control different reclaim strategies and tell when all the hopes are lost and the oom killer should be invoked. This allows for a different policies at the allocator level and this change will break that AFAICS. E.g. what if the underlying allocation context is__GFP_NORETRY?
This is called in kswapd only, and by default (min_ttl=0) it doesn't do anything. So __GFP_NORETRY doesn't apply. The question would be more along the lines of long-term ABI support.
And I'll add the following comments, if you think we can keep this logic:
OOM kill if every generation from all memcgs is younger than min_ttl. Another theoretical possibility is all memcgs are either below min or ineligible at priority 0, but this isn't the main goal.
(Please read my reply at the bottom to decide whether we should keep
it or not. Thanks.)
Quoted 11 lines
Quoted 10 lines
Quoted 1 line
From patch 8:
Personal computers
------------------
:Thrashing prevention: Write ``N`` to
``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of ``N`` milliseconds from getting evicted. The OOM killer is invoked if this working set can't be kept in memory. Based on the average human detectable lag (~100ms), ``N=1000`` usually eliminates intolerable lags due to thrashing. Larger values like ``N=3000`` make lags less noticeable at the cost of more OOM kills.
Quoted 2 lines
This is a very good example of something that should be a self contained patch with its own justification.
Consider it done.
Quoted 3 lines
TBH it is really not all that clear to
me that we want to provide any user visible knob to control OOM behavior based on a time based QoS.
Agreed, and it didn't exist until v4, i.e., after I was demanded to provide it for several times.
For example:
https://github.com/zen-kernel/zen-kernel/issues/223
And another example:
Your Multigenerational LRU patchset is pretty complex and effective, but does not eliminate thrashing condition fully on an old PCs with slow HDD.
I'm kindly asking you to cooperate with hakavlad if it's possible and maybe re-implement parts of le9 patch in your patchset wherever acceptable, as they are quite similar in the core concept.
This is excerpt of an email from [email protected], and he has posted demo videos in this discussion:
https://lore.kernel.org/lkml/[email protected]/
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 51 lines · Yu Zhao
Quoted 50 lines · Michal Hocko
Quoted 48 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void lru_gen_age_node (struct pglist_data *pgdat,struct scan_control *sc) +{ +struct mem_cgroup *memcg; +bool success = false; +unsigned long min_ttl =READ_ONCE (lru_gen_min_ttl); + +VM_BUG_ON (!current_is_kswapd ()); + + current->reclaim_state->mm_walk = &pgdat->mm_walk; + + memcg =mem_cgroup_iter (NULL ,NULL ,NULL ); +do { +struct lruvec *lruvec =mem_cgroup_lruvec (memcg, pgdat); + +if (age_lruvec (lruvec, sc, min_ttl)) + success = true; + +cond_resched (); + }while ((memcg =mem_cgroup_iter (NULL , memcg,NULL ))); + +if (!success &&mutex_trylock (&oom_lock)) { +struct oom_control oc = { + .gfp_mask = sc->gfp_mask, + .order = sc->order, + }; +Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 4214
Quoted 5 lines
mm/vmscan.c +5
-0+if (!oom_reaping_in_progress ()) +out_of_memory (&oc); + +mutex_unlock (&oom_lock); + }Why do you need to trigger oom killer from this path? Why cannot you rely on the page allocator to do that like we do now?
This is per desktop users' (repeated) requests. The can't tolerate thrashing as servers do because of UI lags; and they usually don't have fancy tools like oomd.
Related discussions I saw:
https://github.com/zen-kernel/zen-kernel/issues/218
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
Quoted 6 lines
Quoted 2 lines
I do not really see any arguments why an userspace based trashing detection cannot be used for those. Could you clarify?
It definitely can be done. But who is going to do it for every distro and all individual users? AFAIK, not a single distro provides such a solution for desktop/laptop/phone users.
If existing interfaces provides sufficient information to make those calls then I would definitely prefer a userspace solution.
Quoted 4 lines
And also there is the theoretical question how reliable a userspace solution can be. What if this usespace solution itself gets stuck in the direct reclaim path. I'm sure if nobody has done some search to prove or debunk it.
I have to confess I haven't checked oomd or other solutions but with a sufficient care (all the code mlocked in + no allocations done while collecting data) I believe this should be achieveable.
Quoted 3 lines
In addition, what exactly PSI values should be used on different models of consumer electronics? Nobody knows. We have a team working on this and we haven't figured it out for all our Chromebook models.
I believe this is a matter of tuning for a specific deployment. We do not have only psi but also refault counters that can be used.
Quoted 15 lines
As Andrew said, "a blunt instrument like this would be useful".
https://lore.kernel.org/lkml/[email protected]/I'd like to have less code in kernel too, but I've learned never to walk over users. If I remove this and they come after me asking why, I'd have a hard time convincing them.
Quoted 7 lines
Also my question was pointing to why
out_of_memoryis called from the reclaim rather than the allocator (memcg charging path). It is the caller of the reclaim to control different reclaim strategies and tell when all the hopes are lost and the oom killer should be invoked. This allows for a different policies at the allocator level and this change will break that AFAICS. E.g. what if the underlying allocation context is__GFP_NORETRY?
Quoted 2 lines
This is called in kswapd only, and by default (min_ttl=0) it doesn't do anything. So
__GFP_NORETRYdoesn't apply.
My bad. I must have got lost when traversing the code but I can see you are enforcing that by a VM_BUG_ON. So the limited scope reclaim is not a problem indeed.
Quoted 43 lines
The question would be
more along the lines of long-term ABI support.And I'll add the following comments, if you think we can keep this logic:
OOM kill if every generation from all memcgs is younger thanmin_ttl. Another theoretical possibility is all memcgs are either below min or ineligible at priority 0, but this isn't the main goal.(Please read my reply at the bottom to decide whether we should keep
it or not. Thanks.)Quoted 11 lines
Quoted 10 lines
Quoted 1 line
From patch 8:
Personal computers
------------------
:Thrashing prevention: Write ``N`` to
``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of ``N`` milliseconds from getting evicted. The OOM killer is invoked if this working set can't be kept in memory. Based on the average human detectable lag (~100ms), ``N=1000`` usually eliminates intolerable lags due to thrashing. Larger values like ``N=3000`` make lags less noticeable at the cost of more OOM kills.Quoted 2 lines
This is a very good example of something that should be a self contained patch with its own justification.
Consider it done.
Quoted 3 lines
TBH it is really not all that clear to
me that we want to provide any user visible knob to control OOM behavior based on a time based QoS.Agreed, and it didn't exist until v4, i.e., after I was demanded to provide it for several times.
For example:
https://github.com/zen-kernel/zen-kernel/issues/223And another example:
Your Multigenerational LRU patchset is pretty complex and effective, but does not eliminate thrashing condition fully on an old PCs with slow HDD.
Quoted 7 lines
I'm kindly asking you to cooperate with hakavlad if it's possible and maybe re-implement parts of le9 patch in your patchset wherever acceptable, as they are quite similar in the core concept.
This is excerpt of an email from [email protected], and he has posted demo videos in this discussion:
https://lore.kernel.org/lkml/[email protected]/
That is all an interesting feedback but we should be really craful about ABI constrains and future maintainability of the knob. I still stand behind my statement that kernel should implement such features only if it is clear that we cannot really implement a similar logic in the userspace.
Signature · 2 lines
Michal Hocko SUSE Labs
A message that is not in the archive (sent off-list or not yet received): [email protected]
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 13 lines · Alexey Avramov
Quoted 2 lines
I do not really see any arguments why an userspace based trashing detection cannot be used for those.
Firsly,
because this is the task of the kernel, not the user space. Memory is managed by the kernel, not by the user space.
The absence of such a mechanism in the kernel is a fundamental problem. The userspace tools are ugly hacks:
some of them consume a lot of CPU [1],
some of them consume a lot of memory [2],
some of them cannot intoprocess_mrelease()(earlyoom, nohang), some of them kill only the whole cgroup (systemd-oomd, oomd) [3] and depends on systemd andcgroup_v2(oomd, systemd-oomd).
Thanks for those links. Read through them and my understanding is that most of those are very specific to the tool used and nothing really fundamental because of lack of kernel support.
Quoted 3 lines
One of the biggest challenges for userspace oom-killers is to potentially function under intense memory pressure and are prone to getting stuck in memory reclaim themselves [4].
This one is more interesting and the truth is that handling the complete OOM situation from the userspace is really tricky. Especially when with a more complex oom decision policy. In the past we have discussed potential ways to implement a oom kill policy be kernel modules or eBPF. Without anybody following up on that.
But I suspect you are mixing up two things here. One of them is out of memory situation where no memory can be reclaimed and allocated.
The other is one where the memory can be reclaimed, a progress is made, but that leads to a trashing when the most of the time is spent on refaulting a memory reclaimed shortly before.
The first one is addressed by the global oom killer and it tries to be really conservative as much as possible because this is a very disruptive operation. But the later one is more complex and a proper handling really depends on the particular workload to be handled properly because it is more of a QoS than an emergency action to keep the system alive.
There are workloads which prefer a temporary trashing over its working set during a peak memory demand rather than an OOM kill because way too much work would be thrown away. On the other side workloads that are latency sensitive can see even the direct reclaim as a runtime visible problem.
I hope you can imagine there is a really large gap between those two cases and no simple solution can be applied to the whole range. Therefore we have PSI and refault stats exported to the userspace so that a workload specific policy can be implemented there.
If userspace has hard time to use that data and action upon then let's talk about specifics. For the most steady trashing situations I have seen the userspace with mlocked memory and the code can make a forward progress and mediate the situation.
[...]
Quoted 7 lines
[1] https://github.com/facebookincubator/oomd/issues/79
[2] https://github.com/hakavlad/nohang#memory-and-cpu-usage
[3] https://github.com/facebookincubator/oomd/issues/125
[4] https://lore.kernel.org/all/CALvZod7vtDxJZtNhn81V=oE-EPOf=4KZB2Bv6Giz+u3bFFyOLg@mail.gmail.com/
[5] https://github.com/zen-kernel/zen-kernel/issues/223
[6] https://raw.githubusercontent.com/hakavlad/cache-tests/main/mg-LRU-v3_vs_classic-LRU/3-firefox-tail-OOM/mg-LRU-1/psi2
[7] https://lore.kernel.org/linux-mm/[email protected]/
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 2 lines
But the later one is more complex and a proper handling really depends on the particular workload
That is why I advocate the introduction of new tunables.
Quoted 2 lines
There are workloads which prefer a temporary trashing over its working set during a peak memory demand rather than an OOM kill
OK, for such cases, the OOM handles can be set to 0. It can even be the default value.
Quoted 2 lines
On the other side workloads that are
latency sensitive
I daresay that this is the case with most workloads. An internet server that falls into thrashing is a dead server.
Quoted 1 line
no simple solution can be applied to the whole
There are several solutions and they can be taken into the kernel at the same time, they all work:
- min_ttl_ms + MGLRU
- vm.min_filelist_kbytes-like knobs
- PSI-based solutions.
Quoted 3 lines
For the most steady trashing situations I have
seen the userspace with mlocked memory and the code can make a forward progress and mediate the situation.
I still don't see a problem in making all the kernel-space solutions in the kernel.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Michal HockoQuoted 2 lines
I do not really see any arguments why an userspace based trashing detection cannot be used for those.
Firsly,
because this is the task of the kernel, not the user space. Memory is managed by the kernel, not by the user space.
The absence of such a mechanism in the kernel is a fundamental problem. The userspace tools are ugly hacks:
some of them consume a lot of CPU [1],
some of them consume a lot of memory [2],
some of them cannot into process_mrelease() (earlyoom, nohang), some of them kill only the whole cgroup (systemd-oomd, oomd) [3] and depends on systemd and cgroup_v2 (oomd, systemd-oomd). One of the biggest challenges for userspace oom-killers is to potentially function under intense memory pressure and are prone to getting stuck in memory reclaim themselves [4].
It is strange that after decades of user complaints about thrashing and not-working OOM killer I have to explain the obvious things. The basic mechanism must be implemented in the kernel.
Stop shifting responsibility to the user space!
Secondly,
the real reason for the min_ttl_ms mechanism is that without it, multi-minute stalls are possible [5] even when the killer is expected to arrive, and memory pressure is closed to 100 at this period [6]. This fixes a bug that does not exist in the mainline LRU (this is MGLRU-specific bug). BTW, the similar symptoms were recently fixed in the mainline [7].
[1] https://github.com/facebookincubator/oomd/issues/79
[2] https://github.com/hakavlad/nohang#memory-and-cpu-usage
[3] https://github.com/facebookincubator/oomd/issues/125
[4] https://lore.kernel.org/all/CALvZod7vtDxJZtNhn81V=oE-EPOf=4KZB2Bv6Giz+u3bFFyOLg@mail.gmail.com/
[5] https://github.com/zen-kernel/zen-kernel/issues/223
[6] https://raw.githubusercontent.com/hakavlad/cache-tests/main/mg-LRU-v3_vs_classic-LRU/3-firefox-tail-OOM/mg-LRU-1/psi2
[7] https://lore.kernel.org/linux-mm/[email protected]/
[I am duplicating a previous message here - it was not delivered to mailing lists]
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Yu ZhaoOn Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]
Quoted 17 lines
mm/vmscan.c +17
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end;
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3921
Quoted 10 lines
mm/vmscan.c +10
-0+ + err = -EBUSY ; + +rcu_read_lock (); +#ifdef CONFIG_MEMCG +if (memcg &&atomic_read (&memcg->moving_account)) + goto contended; +#endif +if (!mmap_read_trylock (mm)) + goto contended;
Have you evaluated the behavior under mmap_sem contention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusive mmap_sem IIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.
Or is this considered statistically insignificant thus a theoretical problem?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))
Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.
You are right. Here it must be a trylock; otherwise it can deadlock.
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e., lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.
Quoted 2 lines
Or is this considered statistically insignificant thus a theoretical problem?
Yes. People who work on the maple tree and SPF at Google expressed the same concern during the design review meeting (all stakeholders on the mailing list were also invited). So we had a counter to monitor the contention in previous versions, i.e., MM_LOCK_CONTENTION in v4 here:
https://lore.kernel.org/lkml/[email protected]/
And we also combined this patchset with the SPF patchset to see if the latter makes any difference. Our conclusion was the contention is statistically insignificant to the performance under memory pressure.
This can be explained by how often we create a new generation. (We only walk page tables when we create a new generation. And it's similar to the low inactive condition for the active/inactive lru.)
Usually we only do so every few seconds. We'd run into problems with other parts of the kernel, e.g., lru lock contention, i/o congestion, etc. if we create more than a few generation every second.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 38 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.
Quoted 1 line
You are right. Here it must be a trylock; otherwise it can deadlock.
Yeah, this is clear.
Quoted 6 lines
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e.,
lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.
I have to say that I really have hard time to understand the runtime behavior depending on that interaction. How does the reclaim behave when the virtual scan is enabled, partially enabled and almost completely disabled due to different constrains? I do not see any such an evaluation described in changelogs and I consider this to be a rather important information to judge the overall behavior.
Quoted 13 lines
Quoted 2 lines
Or is this considered statistically insignificant thus a theoretical problem?
Yes. People who work on the maple tree and SPF at Google expressed the same concern during the design review meeting (all stakeholders on the mailing list were also invited). So we had a counter to monitor the contention in previous versions, i.e.,
MM_LOCK_CONTENTIONin v4 here:
https://lore.kernel.org/lkml/[email protected]/And we also combined this patchset with the SPF patchset to see if the latter makes any difference. Our conclusion was the contention is statistically insignificant to the performance under memory pressure.
Quoted 7 lines
This can be explained by how often we create a new generation. (We only walk page tables when we create a new generation. And it's similar to the low inactive condition for the active/inactive lru.)
Usually we only do so every few seconds. We'd run into problems with other parts of the kernel, e.g., lru lock contention, i/o congestion, etc. if we create more than a few generation every second.
This would be a very good information to have in changelogs. Ideally with some numbers and analysis.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 50 lines · Michal Hocko
Quoted 38 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.Quoted 1 line
You are right. Here it must be a trylock; otherwise it can deadlock.
Yeah, this is clear.
Quoted 6 lines
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e.,
lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.
Quoted 6 lines
I have to say that I really have hard time to understand the runtime behavior depending on that interaction. How does the reclaim behave when the virtual scan is enabled, partially enabled and almost completely disabled due to different constrains? I do not see any such an evaluation described in changelogs and I consider this to be a rather important information to judge the overall behavior.
It doesn't have (partially) enabled/disabled states nor does its behavior change with different reclaim constraints. Having either would make its design too complex to implement or benchmark.
There is feedback loop connecting page table walks and rmap walks by Bloom filters. The Bloom filters hold dense hot areas. Page table walks test whether virtual areas are in the Bloom filters and scan those that were tested positive. Anything they miss will be caught by rmap walks later (shrink_page_list()). And when rmap walks find new dense hot areas, they add those area to the Bloom filters.
A dense hot area means it has many accessed pages belonging to the reclaim domain, and clearing the accessed bit in all PTEs within this area by one page table walk is more efficient than doing it one by one by many rmap walks, in terms of cacheline utilization.
Quoted 17 lines
Quoted 17 lines
Quoted 2 lines
Or is this considered statistically insignificant thus a theoretical problem?
Yes. People who work on the maple tree and SPF at Google expressed the same concern during the design review meeting (all stakeholders on the mailing list were also invited). So we had a counter to monitor the contention in previous versions, i.e.,
MM_LOCK_CONTENTIONin v4 here:
https://lore.kernel.org/lkml/[email protected]/And we also combined this patchset with the SPF patchset to see if the latter makes any difference. Our conclusion was the contention is statistically insignificant to the performance under memory pressure.
This can be explained by how often we create a new generation. (We only walk page tables when we create a new generation. And it's similar to the low inactive condition for the active/inactive lru.)
Quoted 6 lines
Quoted 3 lines
Usually we only do so every few seconds. We'd run into problems with other parts of the kernel, e.g., lru lock contention, i/o congestion, etc. if we create more than a few generation every second.
This would be a very good information to have in changelogs. Ideally with some numbers and analysis.
Will do. Thanks.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 58 lines · Yu Zhao
Quoted 50 lines · Michal Hocko
Quoted 38 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.Quoted 1 line
You are right. Here it must be a trylock; otherwise it can deadlock.
Yeah, this is clear.
Quoted 6 lines
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e.,
lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.Quoted 6 lines
I have to say that I really have hard time to understand the runtime behavior depending on that interaction. How does the reclaim behave when the virtual scan is enabled, partially enabled and almost completely disabled due to different constrains? I do not see any such an evaluation described in changelogs and I consider this to be a rather important information to judge the overall behavior.
Quoted 3 lines
It doesn't have (partially) enabled/disabled states nor does its behavior change with different reclaim constraints. Having either would make its design too complex to implement or benchmark.
Let me clarify. By "partially enabled" I really meant behavior depedning on runtime conditions. Say mmap_sem cannot be locked for half of scanned tasks and/or allocation for the mm walker fails due to lack of memory. How does this going to affect reclaim efficiency. How does a user/admin know that the memory reclaim is in a "degraded" mode because of the contention?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 59 lines · Michal Hocko
Quoted 58 lines · Yu Zhao
Quoted 56 lines · Michal Hocko
Quoted 38 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
mm/vmscan.c +26
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.Quoted 1 line
You are right. Here it must be a trylock; otherwise it can deadlock.
Yeah, this is clear.
Quoted 6 lines
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e.,
lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.I have to say that I really have hard time to understand the runtime behavior depending on that interaction. How does the reclaim behave when the virtual scan is enabled, partially enabled and almost completely disabled due to different constrains? I do not see any such an evaluation described in changelogs and I consider this to be a rather important information to judge the overall behavior.
Quoted 8 lines
Quoted 3 lines
It doesn't have (partially) enabled/disabled states nor does its behavior change with different reclaim constraints. Having either would make its design too complex to implement or benchmark.
Let me clarify. By "partially enabled" I really meant behavior depedning on runtime conditions. Say
mmap_semcannot be locked for half of scanned tasks and/or allocation for the mm walker fails due to lack of memory. How does this going to affect reclaim efficiency.
Understood. This is not only possible -- it's the default for our ARM hardware that doesn't support the accessed bit, i.e., CPUs that don't automatically set the accessed bit.
In try_to_inc_max_seq(), we have:
/*
* If the hardware doesn't automatically set the accessed bit, fallback
* to lru_gen_look_around(), which only clears the accessed bit in a
* handful of PTEs. Spreading the work out over a period of time usually
* is less efficient, but it avoids bursty page faults.
*/
if the accessed bit is not supported
return
if alloc_mm_walk() fails
return
walk_mm()
if mmap_sem contented
return
scan page tables
We have a microbenchmark that specifically measures this worst case scenario by entirely disabling page table scanning. Its results showed that this still retains more than 90% of the optimal performance. I'll share this microbenchmark in another email when answering Barry's questions regarding the accessed bit.
Our profiling infra also indirectly confirms this: it collects data from real users running on hardware with and without the accessed bit. Users running on hardware without the accessed bit indeed suffer a small performance degradation, compared with users running on hardware with it. But they still benefit almost as much, compared with users running on the same hardware but without MGLRU.
Quoted 3 lines
How does a user/admin
know that the memory reclaim is in a "degraded" mode because of the contention?
As we previously discussed here:
https://lore.kernel.org/linux-mm/[email protected]/
there used to be a counter measuring the contention, and it was deemed unnecessary and removed in v4. But I don't have a problem if we want to revive it.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 98 lines · Yu Zhao
Quoted 59 lines · Michal Hocko
Quoted 58 lines · Yu Zhao
Quoted 56 lines · Michal Hocko
Quoted 38 lines · Yu Zhao
Quoted 28 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 26 lines
(quoted diff) +26
-0+static void walk_mm(struct lruvec *lruvec, struct mm_struct *mm, struct lru_gen_mm_walk *walk) +{ + static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + + int err; +#ifdef CONFIG_MEMCG + struct mem_cgroup *memcg = lruvec_memcg(lruvec); +#endif + + walk->next_addr = FIRST_USER_ADDRESS; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY; + + rcu_read_lock(); +#ifdef CONFIG_MEMCG + if (memcg && atomic_read(&memcg->moving_account)) + goto contended; +#endif + if (!mmap_read_trylock(mm))Quoted 8 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ goto contended;Have you evaluated the behavior under
mmap_semcontention? I mean what would be an effect of some mms being excluded from the walk? This path is called from direct reclaim and we do allocate with exclusivemmap_semIIRC and the trylock can fail in a presence of pending writer if I am not mistaken so even the read lock holder (e.g. an allocation from the #PF) can bypass the walk.Quoted 1 line
You are right. Here it must be a trylock; otherwise it can deadlock.
Yeah, this is clear.
Quoted 6 lines
I think there might be a misunderstanding: the aging doesn't exclusively rely on page table walks to gather the accessed bit. It prefers page table walks but it can also fallback to the rmap-based function, i.e.,
lru_gen_look_around(), which only gathers the accessed bit from at most 64 PTEs and therefore is less efficient. But it still retains about 80% of the performance gains.I have to say that I really have hard time to understand the runtime behavior depending on that interaction. How does the reclaim behave when the virtual scan is enabled, partially enabled and almost completely disabled due to different constrains? I do not see any such an evaluation described in changelogs and I consider this to be a rather important information to judge the overall behavior.
Quoted 8 lines
Quoted 3 lines
It doesn't have (partially) enabled/disabled states nor does its behavior change with different reclaim constraints. Having either would make its design too complex to implement or benchmark.
Let me clarify. By "partially enabled" I really meant behavior depedning on runtime conditions. Say
mmap_semcannot be locked for half of scanned tasks and/or allocation for the mm walker fails due to lack of memory. How does this going to affect reclaim efficiency.Understood. This is not only possible -- it's the default for our ARM hardware that doesn't support the accessed bit, i.e., CPUs that don't automatically set the accessed bit.
In
try_to_inc_max_seq(), we have:
/*
* If the hardware doesn't automatically set the accessed bit, fallback
* tolru_gen_look_around(), which only clears the accessed bit in a
* handful of PTEs. Spreading the work out over a period of time usually
* is less efficient, but it avoids bursty page faults.
*/
if the accessed bit is not supported
returnif
alloc_mm_walk()fails
return
walk_mm()
ifmmap_semcontented
returnscan page tables
We have a microbenchmark that specifically measures this worst case scenario by entirely disabling page table scanning. Its results showed that this still retains more than 90% of the optimal performance. I'll share this microbenchmark in another email when answering Barry's questions regarding the accessed bit.
Quoted 6 lines
Our profiling infra also indirectly confirms this: it collects data from real users running on hardware with and without the accessed bit. Users running on hardware without the accessed bit indeed suffer a small performance degradation, compared with users running on hardware with it. But they still benefit almost as much, compared with users running on the same hardware but without MGLRU.
This definitely a good information to have in the cover letter.
Quoted 9 lines
Quoted 3 lines
How does a user/admin
know that the memory reclaim is in a "degraded" mode because of the contention?As we previously discussed here:
https://lore.kernel.org/linux-mm/[email protected]/
there used to be a counter measuring the contention, and it was deemed unnecessary and removed in v4. But I don't have a problem if we want to revive it.
Well, counter might be rather tricky but few trace points would make some sense to me.
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
In reply to Yu ZhaoOn Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]
Quoted 15 lines
mm/vmscan.c +15
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do {
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3919
Quoted 10 lines
mm/vmscan.c +10
-0+unsigned long start = walk->next_addr; +unsigned long end = mm->highest_vm_end; + + err = -EBUSY ; + +rcu_read_lock (); +#ifdef CONFIG_MEMCG +if (memcg &&atomic_read (&memcg->moving_account)) + goto contended; +#endif
Why do you need to check for moving_account?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 21 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 19 lines
mm/vmscan.c +19
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY;
Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3919
Quoted 6 lines
mm/vmscan.c +6
-0+ +rcu_read_lock (); +#ifdef CONFIG_MEMCG +if (memcg &&atomic_read (&memcg->moving_account)) + goto contended; +#endif Why do you need to check for
moving_account?
This check, if succeeds, blocks memcg migration.
Our goal is to move pages between different generations of the same lruvec (the first arg). Meanwhile, pages can also be migrated between different memcgs (different lruvecs).
The active/inactive lru uses isolation to block memcg migration.
Generations account pages similarly to the active/inactive lru, i.e., each generation has nr_pages counter. However, unlike the active/ inactive lru, a page can be moved to a different generation without getting isolated or even without being under the lru lock, as long as the delta is eventually accounted for (which does require the lru lock when it happens).
The generation counter in page->flags (folio->flags to be precise) stores 0 when a page is isolated, to synchronize with isolation.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 31 lines · Yu Zhao
Quoted 21 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 19 lines
mm/vmscan.c +19
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do { + unsigned long start = walk->next_addr; + unsigned long end = mm->highest_vm_end; + + err = -EBUSY;Quoted 8 lines
mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3919
Quoted 6 lines
mm/vmscan.c +6
-0+ +rcu_read_lock (); +#ifdef CONFIG_MEMCG +if (memcg &&atomic_read (&memcg->moving_account)) + goto contended; +#endif Why do you need to check for
moving_account?
Quoted 1 line
This check, if succeeds, blocks memcg migration.
OK, I can see that you rely on the RCU here for the synchronization. A comment which mentions mem_cgroup_move_charge would be helpful for clarity. Is there any reason you are not using folio_memcg_lock in the pte walk instead?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 32 lines · Michal Hocko
Quoted 31 lines · Yu Zhao
Quoted 29 lines · Michal Hocko
On Tue 04-01-22 13:22:25, Yu Zhao wrote:
[...]Quoted 15 lines
mm/vmscan.c +15
-0+static void walk_mm (struct lruvec *lruvec,struct mm_struct *mm,struct lru_gen_mm_walk *walk) +{ +static const struct mm_walk_ops mm_walk_ops = { + .test_walk = should_skip_vma, + .p4d_entry = walk_pud_range, + }; + +int err; +#ifdef CONFIG_MEMCG +struct mem_cgroup *memcg =lruvec_memcg (lruvec); +#endif + + walk->next_addr =FIRST_USER_ADDRESS ; + + do {mm/vmscan.c static bool get_next_mm(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, · line 3919
Quoted 10 lines
mm/vmscan.c +10
-0+unsigned long start = walk->next_addr; +unsigned long end = mm->highest_vm_end; + + err = -EBUSY ; + +rcu_read_lock (); +#ifdef CONFIG_MEMCG +if (memcg &&atomic_read (&memcg->moving_account)) + goto contended; +#endif Why do you need to check for
moving_account?
Quoted 5 lines
Quoted 1 line
This check, if succeeds, blocks memcg migration.
OK, I can see that you rely on the RCU here for the synchronization. A comment which mentions
mem_cgroup_move_chargewould be helpful for clarity.
Will do
Quoted 2 lines
Is there any reason you are not using
folio_memcg_lockin the pte walk instead?
We have a particular lruvec (the first arg), hence a particular memcg to lock. But we don't have a particular page to lock.
Michal Hocko
maintainerRe: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 1 line · Yu Zhao
On Wed, Jan 12, 2022 at 11:17:53AM +0100, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 2 lines
Is there any reason you are not using
folio_memcg_lockin the pte walk instead?We have a particular lruvec (the first arg), hence a particular memcg to lock. But we don't have a particular page to lock.
That is certainly true at this layer but the locking should be needed only for specific pages, no? So you can move the lock down to the callback which examines respective pages. Or is there anything preventing that?
To be honest, and that is the reason I am asking, I really do not like to open code the migration synchronization outside of the memcg proper. Code paths which need a stable memcg are supposed to be using folio_memcg_lock for the specific examination time. If you prefer a trylock approach for this usecase then we can add one.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/9] mm: multigenerational lru: aging
Quoted 11 lines · Michal Hocko
Quoted 1 line · Yu Zhao
On Wed, Jan 12, 2022 at 11:17:53AM +0100, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 2 lines
Is there any reason you are not using
folio_memcg_lockin the pte walk instead?We have a particular lruvec (the first arg), hence a particular memcg to lock. But we don't have a particular page to lock.
That is certainly true at this layer but the locking should be needed only for specific pages, no?
Yes.
Quoted 3 lines
So you can move the lock down to the
callback which examines respective pages. Or is there anything preventing that?
No.
Quoted 2 lines
To be honest, and that is the reason I am asking, I really do not like to open code the migration synchronization outside of the memcg proper.
Agreed.
Quoted 2 lines
Code paths which need a stable memcg are supposed to be using
folio_memcg_lockfor the specific examination time.
No argument here, just a clarification: when possible I prefer to lock a batch of pages rather than individual ones.
Quoted 2 lines
If you prefer a
trylock approach for this usecase then we can add one.
Done. Thanks.
PATCH v6 7/9 mm: multigenerational lru: eviction
In reply to Yu ZhaoThe eviction consumes old generations. Given an lruvec, it scans pages on lrugen->lists[] indexed by min_seq%MAX_NR_GENS. A feedback loop modeled after the PID controller monitors refaults over anon and file types and decides which type to evict when both are available from the same generation.
Each generation is divided into multiple tiers. Tiers represent different ranges of numbers of accesses thru file descriptors. A page accessed N times thru file descriptors is in tier order_base_2(N). The feedback loop also monitors refaults over all tiers and decides when to promote pages in which tiers (N>1), using the first tier (N=0,1) as a baseline.
The eviction sorts a page according to the gen counter if the aging has found this page accessed thru page tables, which completes the promotion of this page. The eviction also promotes a page to the next generation (min_seq+1 rather than max_seq) if this page was accessed multiple times thru file descriptors and the feedback loop has detected higher refaults from the tier this page is in. This approach has the following advantages:
1) It removes the cost of activation (recall the terms) in the buffered access path by inferring whether pages accessed multiple times thru file descriptors are statistically hot and thus worth promoting in the eviction path.
2) It takes pages accessed thru page tables into account and avoids overprotecting pages accessed multiple times thru file descriptors.
3) More tiers, which require additional bits in folio->flags, provide better protection for pages accessed more than twice thru file descriptors, when under heavy buffered I/O workloads.
The eviction increments min_seq when lrugen->lists[] indexed by min_seq%MAX_NR_GENS is empty.
--- include/linux/mm_inline.h | 10 + include/linux/mmzone.h | 28 ++ mm/swap.c | 42 +++ mm/vmscan.c | 571 +++++++++++++++++++++++++++++++++++++- mm/workingset.c | 119 +++++++- 5 files changed, 767 insertions(+), 3 deletions(-)
include/linux/mm_inline.h +10 -0
@@ -115,6 +115,14 @@ static inline int lru_hist_from_seq(unsigned long seq)return seq %NR_HIST_GENS ; } +static inline int lru_tier_from_refs (int refs) +{ +VM_BUG_ON (refs >BIT (LRU_REFS_WIDTH )); + +/* see the comment on MAX_NR_TIERS */ +return order_base_2 (refs +1 ); +} +static inline bool lru_gen_is_active (struct lruvec *lruvec,int gen) {unsigned long max_seq = lruvec->lrugen.max_seq;
@@ -243,6 +251,8 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, gen = ((new_flags &LRU_GEN_MASK ) >>LRU_GEN_PGOFF ) -1 ; new_flags &= ~LRU_GEN_MASK ; +if ((new_flags &LRU_REFS_FLAGS ) !=LRU_REFS_FLAGS ) + new_flags &= ~(LRU_REFS_MASK |LRU_REFS_FLAGS );/* for shrink_page_list() */ if (reclaiming) new_flags &= ~(BIT (PG_referenced) |BIT (PG_reclaim));
include/linux/mmzone.h +28 -0
@@ -336,6 +336,25 @@ struct page_vma_mapped_walk;#define MIN_NR_GENS 2U#define MAX_NR_GENS ((unsigned int)CONFIG_NR_LRU_GENS) +/* +* Each generation is divided into multiple tiers. Tiers represent different +* ranges of numbers of accesses thru file descriptors. A page accessed N times +* thru file descriptors is in tier order_base_2(N). A page in the first tier +* (N=0,1) is marked by PG_referenced unless it was faulted in thru page tables +* or read ahead. A page in any other tier (N>1) is marked by PG_referenced and +* PG_workingset. Additional bits in folio->flags are required to support more +* than two tiers. +* +* In contrast to moving across generations (promotion), moving across tiers +* only requires operations on folio->flags and therefore has a negligible cost +* in the buffered access path. In the eviction path, comparisons of +* refaulted/(evicted+promoted) from the first tier and the rest infer whether +* pages accessed multiple times thru file descriptors are statistically hot +* and thus worth promoting. +*/ +#define MAX_NR_TIERS ((unsigned int)CONFIG_TIERS_PER_GEN) +#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) +/* whether to keep historical stats for evicted generations */ #ifdef CONFIG_LRU_GEN_STATS #define NR_HIST_GENS ((unsigned int)CONFIG_NR_LRU_GENS)
@@ -354,6 +373,15 @@ struct lru_gen_struct {struct list_head lists[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ];/* the sizes of the above lists */ unsigned long nr_pages[MAX_NR_GENS ][ANON_AND_FILE ][MAX_NR_ZONES ]; +/* the exponential moving average of refaulted */ +unsigned long avg_refaulted[ANON_AND_FILE ][MAX_NR_TIERS ]; +/* the exponential moving average of evicted+promoted */ +unsigned long avg_total[ANON_AND_FILE ][MAX_NR_TIERS ]; +/* the first tier doesn't need promotion, hence the minus one */ +unsigned long promoted[NR_HIST_GENS ][ANON_AND_FILE ][MAX_NR_TIERS -1 ]; +/* can be modified without holding the lru lock */ +atomic_long_t evicted[NR_HIST_GENS ][ANON_AND_FILE ][MAX_NR_TIERS ]; +atomic_long_t refaulted[NR_HIST_GENS ][ANON_AND_FILE ][MAX_NR_TIERS ];/* whether the multigenerational lru is enabled */ bool enabled; };
mm/swap.c +42 -0
54 lines not on this page · raw message
mm/vmscan.c +570 -1
663 lines not on this page · raw message
mm/workingset.c +117 -2
151 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Re: [PATCH v6 7/9] mm: multigenerational lru: eviction
...
+static int isolate_folios(struct lruvec *lruvec, struct scan_control *sc, int swappiness,
Quoted 16 lines
mm/vmscan.c +16
-0+ int *type_scanned, struct list_head *list) +{ + int i; +int type; +int scanned; +int tier =-1 ; +DEFINE_MIN_SEQ (lruvec); + +VM_BUG_ON (!seq_is_valid (lruvec)); + +/* +* Try to make the obvious choice first. When anon and file are both +* available from the same generation, interpret swappiness 1 as file +* first and 200 as anon first. +*/ +if (!swappiness)
mm/vmscan.c void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) · line 4687
Quoted 10 lines
mm/vmscan.c +10
-0+ type =1 ; + else if (min_seq[0] < min_seq[1]) + type =0 ; + else if (swappiness == 1) + type =1 ; + else if (swappiness == 200) + type =0 ; + else + type =get_type_to_scan (lruvec, swappiness, &tier); +
Wondering wether it will make it simpler to use
#define ANON 0
#define FILE 1
and then
else if (min_seq[ANON] < min_seq[FILE]) type = ANON;
The usage of 0/1 across code do confuse
-aneesh
Re: [PATCH v6 7/9] mm: multigenerational lru: eviction
Quoted 34 lines · Aneesh Kumar K.V
...
+static int isolate_folios(struct lruvec *lruvec, struct
scan_control*sc, int swappiness,Quoted 16 lines
mm/vmscan.c +16
-0+ int *type_scanned, struct list_head *list) +{ + int i; +int type; +int scanned; +int tier =-1 ; +DEFINE_MIN_SEQ (lruvec); + +VM_BUG_ON (!seq_is_valid (lruvec)); + +/* +* Try to make the obvious choice first. When anon and file are both +* available from the same generation, interpret swappiness 1 as file +* first and 200 as anon first. +*/ +if (!swappiness)mm/vmscan.c void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) · line 4687
Quoted 10 lines
mm/vmscan.c +10
-0+ type =1 ; + else if (min_seq[0] < min_seq[1]) + type =0 ; + else if (swappiness == 1) + type =1 ; + else if (swappiness == 200) + type =0 ; + else + type =get_type_to_scan (lruvec, swappiness, &tier); +Wondering wether it will make it simpler to use
#define ANON 0
#define FILE 1
Quoted 5 lines
and then
else if (min_seq[ANON] < min_seq[FILE]) type = ANON;The usage of 0/1 across code do confuse
I agree, and I plan to do this later because the existing code uses this convention and needs renaming too.
PATCH v6 8/9 mm: multigenerational lru: user interface
In reply to Yu ZhaoAdd /sys/kernel/mm/lru_gen/enabled as a runtime kill switch.
Add /sys/kernel/mm/lru_gen/min_ttl_ms for thrashing prevention. Compared with the size-based approach, e.g., [1], this time-based approach has the following advantages:
1) It's easier to configure because it's agnostic to applications and memory sizes.
2) It's more reliable because it's directly wired to the OOM killer.
Add /sys/kernel/debug/lru_gen for working set estimation and proactive reclaim. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages:
1) It offers better choices because it's aware of memcgs, NUMA nodes, shared mappings and unmapped page cache.
2) It's more scalable because it's O(nr_hot_evictable_pages), whereas the PFN-based approach is O(nr_total_pages).
Add /sys/kernel/debug/lru_gen_full for debugging.
[1] https://lore.kernel.org/lkml/[email protected]/
--- Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 62 +++++ include/linux/nodemask.h | 1 + mm/vmscan.c | 415 ++++++++++++++++++++++++++++++ 4 files changed, 479 insertions(+) create mode 100644 Documentation/vm/multigen_lru.rst
Documentation/vm/index.rst +1 -0
@@ -38,3 +38,4 @@ algorithms. If you are looking for advice on simply allocating memory, see the unevictable-lru z3fold zsmalloc + multigen_lru
Documentation/vm/multigen_lru.rst new file +62 -0
62 lines not on this page · raw message
include/linux/nodemask.h +1 -0
@@ -486,6 +486,7 @@ static inline int num_node_state(enum node_states state)#define first_online_node 0#define first_memory_node 0#define next_online_node (nid) (MAX_NUMNODES) +#define next_memory_node (nid) (MAX_NUMNODES)#define nr_node_ids 1U#define nr_online_nodes 1U
mm/vmscan.c +415 -0
433 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Mike Rapoport
maintainerRe: [PATCH v6 8/9] mm: multigenerational lru: user interface
Hi,
Quoted 20 lines · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Add
/sys/kernel/mm/lru_gen/min_ttl_msfor thrashing prevention. Compared with the size-based approach, e.g., [1], this time-based approach has the following advantages:
1) It's easier to configure because it's agnostic to applications and memory sizes.
2) It's more reliable because it's directly wired to the OOM killer.Add
/sys/kernel/debug/lru_genfor working set estimation and proactive reclaim. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages:
1) It offers better choices because it's aware of memcgs, NUMA nodes, shared mappings and unmapped page cache.
2) It's more scalable because it'sO(nr_hot_evictable_pages), whereas the PFN-based approach isO(nr_total_pages).Add
/sys/kernel/debug/lru_gen_fullfor debugging.
Quoted 7 lines
[1] https://lore.kernel.org/lkml/[email protected]/
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 62 +++++
The description of user visible interfaces should go to Documentation/admin-guide/mm
Documentation/vm/multigen_lru.rst should have contained design description and the implementation details and it would be great to actually have such document.
Quoted 22 lines
include/linux/nodemask.h | 1 + mm/vmscan.c | 415 ++++++++++++++++++++++++++++++ 4 files changed, 479 insertions(+) create mode 100644 Documentation/vm/multigen_lru.rstDocumentation/vm/index.rst +1
-0@@ -38,3 +38,4 @@ algorithms. If you are looking for advice on simply allocating memory, see the unevictable-lru z3fold zsmalloc + multigen_lruDocumentation/vm/multigen_lru.rst new file +2
-0@@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: GPL-2.0 +
Documentation/vm/multigen_lru.rst line 12
Quoted 10 lines
Documentation/vm/multigen_lru.rst new file +10
-0+===================== +Multigenerational LRU +===================== + +Quick start +=========== +Runtime configurations +---------------------- +:Required: Write ``1`` to ``/sys/kernel/mm/lru_gen/enable`` if the + feature wasn't enabled by default.
Required for what? This sentence seem to lack context. Maybe add an overview what is Multigenerational LRU so that users will have an idea what these knobs control.
Documentation/vm/multigen_lru.rst line 15
Quoted 3 lines
Documentation/vm/multigen_lru.rst new file +3
-0+ +Recipes +=======
Some more context here will be also helpful.
Quoted 15 lines
Documentation/vm/multigen_lru.rst new file +15
-0+Personal computers +------------------ +:Thrashing prevention: Write ``N`` to + ``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of + ``N`` milliseconds from getting evicted. The OOM killer is invoked if + this working set can't be kept in memory. Based on the average human + detectable lag (~100ms), ``N=1000`` usually eliminates intolerable + lags due to thrashing. Larger values like ``N=3000`` make lags less + noticeable at the cost of more OOM kills. + +Data centers +------------ +:Debugfs interface: ``/sys/kernel/debug/lru_gen`` has the following + format: + ::
Documentation/vm/multigen_lru.rst line 40
Quoted 10 lines
Documentation/vm/multigen_lru.rst new file +10
-0+ + memcg memcg_id memcg_path + node node_id + min_gen birth_time anon_size file_size + ... + max_gen birth_time anon_size file_size + + ``min_gen`` is the oldest generation number and ``max_gen`` is the + youngest generation number. ``birth_time`` is in milliseconds. + ``anon_size`` and ``file_size`` are in pages.
And what does oldest and youngest generations mean from the user perspective?
Quoted 22 lines
Documentation/vm/multigen_lru.rst new file +22
-0+ + This file also accepts commands in the following subsections. + Multiple command lines are supported, so does concatenation with + delimiters ``,`` and ``;``. + + ``/sys/kernel/debug/lru_gen_full`` contains additional stats for + debugging. + +:Working set estimation: Write ``+ memcg_id node_id max_gen + [can_swap [full_scan]]`` to ``/sys/kernel/debug/lru_gen`` to trigger + the aging. It scans PTEs for accessed pages and promotes them to the + youngest generation ``max_gen``. Then it creates a new generation + ``max_gen+1``. Set ``can_swap`` to 1 to scan for accessed anon pages + when swap is off. Set ``full_scan`` to 0 to reduce the overhead as + well as the coverage when scanning PTEs. + +:Proactive reclaim: Write ``- memcg_id node_id min_gen [swappiness + [nr_to_reclaim]]`` to ``/sys/kernel/debug/lru_gen`` to trigger the + eviction. It evicts generations less than or equal to ``min_gen``. + ``min_gen`` should be less than ``max_gen-1`` as ``max_gen`` and + ``max_gen-1`` aren't fully aged and therefore can't be evicted. Use + ``nr_to_reclaim`` to limit the number of pages to evict.
...
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Quoted 31 lines · Mike Rapoport
Hi,
Quoted 20 lines · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Add
/sys/kernel/mm/lru_gen/min_ttl_msfor thrashing prevention. Compared with the size-based approach, e.g., [1], this time-based approach has the following advantages:
1) It's easier to configure because it's agnostic to applications and memory sizes.
2) It's more reliable because it's directly wired to the OOM killer.Add
/sys/kernel/debug/lru_genfor working set estimation and proactive reclaim. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages:
1) It offers better choices because it's aware of memcgs, NUMA nodes, shared mappings and unmapped page cache.
2) It's more scalable because it'sO(nr_hot_evictable_pages), whereas the PFN-based approach isO(nr_total_pages).Add
/sys/kernel/debug/lru_gen_fullfor debugging.Quoted 7 lines
[1] https://lore.kernel.org/lkml/[email protected]/
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 62 +++++
Quoted 6 lines
The description of user visible interfaces should go to
Documentation/admin-guide/mm
Documentation/vm/multigen_lru.rstshould have contained design description and the implementation details and it would be great to actually have such document.
Will do, thanks.
Quoted 28 lines
Quoted 28 lines
include/linux/nodemask.h | 1 + mm/vmscan.c | 415 ++++++++++++++++++++++++++++++ 4 files changed, 479 insertions(+) create mode 100644 Documentation/vm/multigen_lru.rstDocumentation/vm/index.rst +1
-0@@ -38,3 +38,4 @@ algorithms. If you are looking for advice on simply allocating memory, see the unevictable-lru z3fold zsmalloc + multigen_lruDocumentation/vm/multigen_lru.rst new file +8
-0@@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================== +Multigenerational LRU +===================== + +Quick start +===========
Quoted 8 lines
Documentation/vm/multigen_lru.rst line 12
Quoted 4 lines
Documentation/vm/multigen_lru.rst new file +4
-0+Runtime configurations +---------------------- +:Required: Write ``1`` to ``/sys/kernel/mm/lru_gen/enable`` if the + feature wasn't enabled by default.Required for what? This sentence seem to lack context. Maybe add an overview what is Multigenerational LRU so that users will have an idea what these knobs control.
Apparently I left an important part of this quick start in the next patch, where Kconfig options are added. I'm wonder whether I should squash the next patch into this one.
I always separate Kconfig changes and leave them in the last patch because it gives me peace of mind knowing it'll never give any auto bisectors a hard time.
But I saw people not following this practice, and I'm also tempted to do so. Can anybody remind me whether it's considered a bad practice to have code changes and Kconfig changes in the same patch?
Quoted 5 lines
Documentation/vm/multigen_lru.rst line 15
Quoted 3 lines
Documentation/vm/multigen_lru.rst new file +3
-0+ +Recipes +=======Some more context here will be also helpful.
Will do.
Quoted 20 lines
Quoted 20 lines
Documentation/vm/multigen_lru.rst new file +20
-0+Personal computers +------------------ +:Thrashing prevention: Write ``N`` to + ``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of + ``N`` milliseconds from getting evicted. The OOM killer is invoked if + this working set can't be kept in memory. Based on the average human + detectable lag (~100ms), ``N=1000`` usually eliminates intolerable + lags due to thrashing. Larger values like ``N=3000`` make lags less + noticeable at the cost of more OOM kills. + +Data centers +------------ +:Debugfs interface: ``/sys/kernel/debug/lru_gen`` has the following + format: + :: + + memcg memcg_id memcg_path + node node_id + min_gen birth_time anon_size file_size + ...
Quoted 8 lines
Documentation/vm/multigen_lru.rst line 40
Quoted 5 lines
Documentation/vm/multigen_lru.rst new file +5
-0+ max_gen birth_time anon_size file_size + + ``min_gen`` is the oldest generation number and ``max_gen`` is the + youngest generation number. ``birth_time`` is in milliseconds. + ``anon_size`` and ``file_size`` are in pages.And what does oldest and youngest generations mean from the user perspective?
Good question. Will add more details in the next spin.
Michal Hocko
maintainerRe: [PATCH v6 8/9] mm: multigenerational lru: user interface
On Wed 12-01-22 01:35:52, Yu Zhao wrote:
[...]
Quoted 3 lines
But I saw people not following this practice, and I'm also tempted to do so. Can anybody remind me whether it's considered a bad practice to have code changes and
Kconfigchanges in the same patch?
If you want to have the patch series bisectable then it is preferable to add kconfig options early so that the code is enabled in the respective steps. Sometimes that can be impractical though (e.g. when the feature is incomplete at that stage).
Signature · 2 lines
Michal Hocko SUSE Labs
Mike Rapoport
maintainerRe: [PATCH v6 8/9] mm: multigenerational lru: user interface
In reply to Yu ZhaoQuoted 78 lines · Yu Zhao
Quoted 31 lines · Mike Rapoport
Hi,
Quoted 20 lines · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Add
/sys/kernel/mm/lru_gen/min_ttl_msfor thrashing prevention. Compared with the size-based approach, e.g., [1], this time-based approach has the following advantages:
1) It's easier to configure because it's agnostic to applications and memory sizes.
2) It's more reliable because it's directly wired to the OOM killer.Add
/sys/kernel/debug/lru_genfor working set estimation and proactive reclaim. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages:
1) It offers better choices because it's aware of memcgs, NUMA nodes, shared mappings and unmapped page cache.
2) It's more scalable because it'sO(nr_hot_evictable_pages), whereas the PFN-based approach isO(nr_total_pages).Add
/sys/kernel/debug/lru_gen_fullfor debugging.Quoted 7 lines
[1] https://lore.kernel.org/lkml/[email protected]/
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 62 +++++Quoted 6 lines
The description of user visible interfaces should go to
Documentation/admin-guide/mm
Documentation/vm/multigen_lru.rstshould have contained design description and the implementation details and it would be great to actually have such document.Will do, thanks.
Quoted 28 lines
Quoted 28 lines
include/linux/nodemask.h | 1 + mm/vmscan.c | 415 ++++++++++++++++++++++++++++++ 4 files changed, 479 insertions(+) create mode 100644 Documentation/vm/multigen_lru.rstDocumentation/vm/index.rst +1
-0@@ -38,3 +38,4 @@ algorithms. If you are looking for advice on simply allocating memory, see the unevictable-lru z3fold zsmalloc + multigen_lruDocumentation/vm/multigen_lru.rst new file +8
-0@@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================== +Multigenerational LRU +===================== + +Quick start +===========Quoted 8 lines
Documentation/vm/multigen_lru.rst line 12
Quoted 4 lines
Documentation/vm/multigen_lru.rst new file +4
-0+Runtime configurations +---------------------- +:Required: Write ``1`` to ``/sys/kernel/mm/lru_gen/enable`` if the + feature wasn't enabled by default.Required for what? This sentence seem to lack context. Maybe add an overview what is Multigenerational LRU so that users will have an idea what these knobs control.
Quoted 3 lines
Apparently I left an important part of this quick start in the next patch, where
Kconfigoptions are added. I'm wonder whether I should squash the next patch into this one.
I think documentation deserves a separate patch.
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Quoted 79 lines · Mike Rapoport
Quoted 78 lines · Yu Zhao
Quoted 31 lines · Mike Rapoport
Hi,
Quoted 20 lines · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Add
/sys/kernel/mm/lru_gen/min_ttl_msfor thrashing prevention. Compared with the size-based approach, e.g., [1], this time-based approach has the following advantages:
1) It's easier to configure because it's agnostic to applications and memory sizes.
2) It's more reliable because it's directly wired to the OOM killer.Add
/sys/kernel/debug/lru_genfor working set estimation and proactive reclaim. Compared with the page table-based approach and the PFN-based approach, e.g., mm/damon/[vp]addr.c, this lruvec-based approach has the following advantages:
1) It offers better choices because it's aware of memcgs, NUMA nodes, shared mappings and unmapped page cache.
2) It's more scalable because it'sO(nr_hot_evictable_pages), whereas the PFN-based approach isO(nr_total_pages).Add
/sys/kernel/debug/lru_gen_fullfor debugging.Quoted 7 lines
[1] https://lore.kernel.org/lkml/[email protected]/
Signed-off-by Yu Zhao <[email protected]>Tested-by Konstantin Kharlamov <[email protected]>--- Documentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 62 +++++Quoted 6 lines
The description of user visible interfaces should go to
Documentation/admin-guide/mm
Documentation/vm/multigen_lru.rstshould have contained design description and the implementation details and it would be great to actually have such document.Will do, thanks.
Quoted 36 lines
Quoted 22 lines
include/linux/nodemask.h | 1 + mm/vmscan.c | 415 ++++++++++++++++++++++++++++++ 4 files changed, 479 insertions(+) create mode 100644 Documentation/vm/multigen_lru.rstDocumentation/vm/index.rst +1
-0@@ -38,3 +38,4 @@ algorithms. If you are looking for advice on simply allocating memory, see the unevictable-lru z3fold zsmalloc + multigen_lruDocumentation/vm/multigen_lru.rst new file +2
-0@@ -0,0 +1,62 @@ +.. SPDX-License-Identifier: GPL-2.0 +Documentation/vm/multigen_lru.rst line 12
Quoted 10 lines
Documentation/vm/multigen_lru.rst new file +10
-0+===================== +Multigenerational LRU +===================== + +Quick start +=========== +Runtime configurations +---------------------- +:Required: Write ``1`` to ``/sys/kernel/mm/lru_gen/enable`` if the + feature wasn't enabled by default.Required for what? This sentence seem to lack context. Maybe add an overview what is Multigenerational LRU so that users will have an idea what these knobs control.
Quoted 5 lines
Quoted 3 lines
Apparently I left an important part of this quick start in the next patch, where
Kconfigoptions are added. I'm wonder whether I should squash the next patch into this one.I think documentation deserves a separate patch.
Will do.
Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
In reply to Yu ZhaoQuoted 1 line · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.
Got the below lockdep warning while using the above kill/enable switch
[ 84.252952] ======================================================
[ 84.253012] WARNING: possible circular locking dependency detected
[ 84.253074] 5.16.0-rc8-16204-g1cdcf1120b31 #511 Not tainted
[ 84.253135] ------------------------------------------------------
[ 84.253194] bash/2862 is trying to acquire lock:
[ 84.253243] c0000000021ff740 (cgroup_mutex){+.+.}-{3:3}, at: store_enable+0x80/0x1510
[ 84.253340]
but task is already holding lock:
[ 84.253410] c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50
[ 84.253503]
which lock already depends on the new lock.[ 84.253608]
the existing dependency chain (in reverse order) is:
[ 84.253693]
-> #2 (mem_hotplug_lock){++++}-{0:0}:
[ 84.253768] lock_acquire+0x134/0x4a0
[ 84.253821] percpu_down_write+0x80/0x1c0
[ 84.253872] try_online_node+0x40/0x90
[ 84.253924] cpu_up+0x7c/0x160
[ 84.253976] bringup_nonboot_cpus+0xc4/0x120
[ 84.254027] smp_init+0x48/0xd4
[ 84.254079] kernel_init_freeable+0x274/0x45c
[ 84.254134] kernel_init+0x44/0x194
[ 84.254188] ret_from_kernel_thread+0x5c/0x64
[ 84.254241]
-> #1 (cpu_hotplug_lock){++++}-{0:0}:
[ 84.254321] lock_acquire+0x134/0x4a0
[ 84.254373] cpus_read_lock+0x6c/0x180
[ 84.254426] static_key_disable+0x24/0x50
[ 84.254477] rebind_subsystems+0x3b0/0x5a0
[ 84.254528] cgroup_setup_root+0x24c/0x530
[ 84.254581] cgroup1_get_tree+0x7d8/0xb80
[ 84.254638] vfs_get_tree+0x48/0x150
[ 84.254695] path_mount+0x8b8/0xd20
[ 84.254752] do_mount+0xb8/0xe0
[ 84.254808] sys_mount+0x250/0x390
[ 84.254863] system_call_exception+0x15c/0x2b0
[ 84.254932] system_call_common+0xec/0x250
[ 84.254989]
-> #0 (cgroup_mutex){+.+.}-{3:3}:
[ 84.255072] check_prev_add+0x180/0x1050
[ 84.255129] __lock_acquire+0x17b8/0x25c0
[ 84.255186] lock_acquire+0x134/0x4a0
[ 84.255243] __mutex_lock+0xdc/0xa90
[ 84.255300] store_enable+0x80/0x1510
[ 84.255356] kobj_attr_store+0x2c/0x50
[ 84.255413] sysfs_kf_write+0x6c/0xb0
[ 84.255471] kernfs_fop_write_iter+0x1bc/0x2b0
[ 84.255539] new_sync_write+0x130/0x1d0
[ 84.255594] vfs_write+0x2cc/0x4c0
[ 84.255645] ksys_write+0x84/0x140
[ 84.255699] system_call_exception+0x15c/0x2b0
[ 84.255771] system_call_common+0xec/0x250
[ 84.255829]
other info that might help us debug this:[ 84.255933] Chain exists of:
cgroup_mutex --> cpu_hotplug_lock --> mem_hotplug_lock[ 84.256070] Possible unsafe locking scenario:
[ 84.256149] CPU0 CPU1
[ 84.256201] ---- ----
[ 84.256255] lock(mem_hotplug_lock);
[ 84.256311] lock(cpu_hotplug_lock);
[ 84.256380] lock(mem_hotplug_lock);
[ 84.256448] lock(cgroup_mutex);
[ 84.256491]
*** DEADLOCK ***[ 84.256571] 5 locks held by bash/2862:
[ 84.256626] #0: c00000002043d460 (sb_writers#5){.+.+}-{0:0}, at: ksys_write+0x84/0x140
[ 84.256728] #1: c00000004bafc888 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x178/0x2b0
[ 84.256830] #2: c000000020b993b8 (kn->active#207){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x184/0x2b0
[ 84.256942] #3: c0000000020e5cd0 (cpu_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x20/0x50
[ 84.257045] #4: c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50
[ 84.257152]
stack backtrace:
[ 84.257220] CPU: 107 PID: 2862 Comm: bash Not tainted 5.16.0-rc8-16204-g1cdcf1120b31 #511
[ 84.257309] Call Trace:
[ 84.257346] [c000000040d5b4a0] [c000000000a89f94] dump_stack_lvl+0x98/0xe0 (unreliable)
[ 84.257438] [c000000040d5b4e0] [c000000000267244] print_circular_bug.isra.0+0x3b4/0x3e0
[ 84.257528] [c000000040d5b580] [c0000000002673e0] check_noncircular+0x170/0x1a0
[ 84.257605] [c000000040d5b650] [c000000000268be0] check_prev_add+0x180/0x1050
[ 84.257683] [c000000040d5b710] [c00000000026ca48] __lock_acquire+0x17b8/0x25c0
[ 84.257760] [c000000040d5b840] [c00000000026e4c4] lock_acquire+0x134/0x4a0
[ 84.257837] [c000000040d5b940] [c00000000148a53c] __mutex_lock+0xdc/0xa90
[ 84.257914] [c000000040d5ba60] [c0000000004d5080] store_enable+0x80/0x1510
[ 84.257989] [c000000040d5bbc0] [c000000000a9286c] kobj_attr_store+0x2c/0x50
[ 84.258066] [c000000040d5bbe0] [c000000000752c4c] sysfs_kf_write+0x6c/0xb0
[ 84.258143] [c000000040d5bc20] [c000000000750fcc] kernfs_fop_write_iter+0x1bc/0x2b0
[ 84.258219] [c000000040d5bc70] [c000000000615df0] new_sync_write+0x130/0x1d0
[ 84.258295] [c000000040d5bd10] [c00000000061997c] vfs_write+0x2cc/0x4c0
[ 84.258373] [c000000040d5bd60] [c000000000619d54] ksys_write+0x84/0x140
[ 84.258450] [c000000040d5bdb0] [c00000000002c91c] system_call_exception+0x15c/0x2b0
[ 84.258528] [c000000040d5be10] [c00000000000c64c] system_call_common+0xec/0x250
[ 84.258604] --- interrupt: c00 at 0x79c551e76554
[ 84.258658] NIP: 000079c551e76554 LR: 000079c551de2674 CTR: 0000000000000000
[ 84.258732] REGS: c000000040d5be80 TRAP: 0c00 Not tainted (5.16.0-rc8-16204-g1cdcf1120b31)
[ 84.258817] MSR: 800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE> CR: 28422428 XER: 00000000
[ 84.258931] IRQMASK: 0
GPR00: 0000000000000004 00007fffc8e9a320 000079c551f77100 0000000000000001
GPR04: 0000017190973cc0 0000000000000002 0000000000000010 0000017190973cc0
GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR12: 0000000000000000 000079c5520ab1d0 0000017190943590 000001718749b738
GPR16: 00000171873b0ae0 0000000000000000 0000000020000000 0000017190973a60
GPR20: 0000000000000000 0000000000000001 0000017187443ca0 00007fffc8e9a514
GPR24: 00007fffc8e9a510 000001718749b0d0 000079c551f719d8 000079c551f72308
GPR28: 0000000000000002 000079c551f717e8 0000017190973cc0 0000000000000002
[ 84.259600] NIP [000079c551e76554] 0x79c551e76554
[ 84.259651] LR [000079c551de2674] 0x79c551de2674
[ 84.259701] --- interrupt: c00Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Quoted 20 lines · Aneesh Kumar K.V
Quoted 1 line · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Got the below lockdep warning while using the above kill/enable switch
[ 84.252952] ====================================================== [ 84.253012] WARNING: possible circular locking dependency detected [ 84.253074] 5.16.0-rc8-16204-g1cdcf1120b31 #511 Not tainted [ 84.253135] ------------------------------------------------------ [ 84.253194] bash/2862 is trying to acquire lock: [ 84.253243] c0000000021ff740 (cgroup_mutex){+.+.}-{3:3}, at: store_enable+0x80/0x1510 [ 84.253340] but task is already holding lock: [ 84.253410] c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50 [ 84.253503] which lock already depends on the new lock.
Quoted 2 lines
[ 84.255933] Chain exists of:
cgroup_mutex --> cpu_hotplug_lock --> mem_hotplug_lockThanks. Will reverse the order between mem_hotplug_lock and cgroup_mutex in the next spin.
Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Quoted 21 lines · Yu Zhao
Quoted 20 lines · Aneesh Kumar K.V
Quoted 1 line · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Got the below lockdep warning while using the above kill/enable switch
[ 84.252952] ====================================================== [ 84.253012] WARNING: possible circular locking dependency detected [ 84.253074] 5.16.0-rc8-16204-g1cdcf1120b31 #511 Not tainted [ 84.253135] ------------------------------------------------------ [ 84.253194] bash/2862 is trying to acquire lock: [ 84.253243] c0000000021ff740 (cgroup_mutex){+.+.}-{3:3}, at: store_enable+0x80/0x1510 [ 84.253340] but task is already holding lock: [ 84.253410] c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50 [ 84.253503] which lock already depends on the new lock.
Quoted 5 lines
Quoted 2 lines
[ 84.255933] Chain exists of: cgroup_mutex --> cpu_hotplug_lock --> mem_hotplug_lockThanks. Will reverse the order between
mem_hotplug_lockandcgroup_mutexin the next spin.
It also needs the unlocked variant of static_key_enable/disable.
[ 71.204397][ T2819] bash/2819 is trying to acquire lock:
[ 71.204446][ T2819] c0000000020e5cd0 (cpu_hotplug_lock){++++}-{0:0}, at: static_key_disable+0x24/0x50
[ 71.204542][ T2819]
[ 71.204542][ T2819] but task is already holding lock:
[ 71.204613][ T2819] c0000000020e5cd0 (cpu_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x20/0x50
[ 71.204710][ T2819]
[ 71.204710][ T2819] other info that might help us debug this:
[ 71.204787][ T2819] Possible unsafe locking scenario:
[ 71.204787][ T2819]
[ 71.204860][ T2819] CPU0
[ 71.204901][ T2819] ----
[ 71.204941][ T2819] lock(cpu_hotplug_lock);
[ 71.204998][ T2819] lock(cpu_hotplug_lock);
[ 71.205053][ T2819]
[ 71.205053][ T2819] *** DEADLOCK ***-aneesh
Re: [PATCH v6 8/9] mm: multigenerational lru: user interface
Quoted 21 lines · Aneesh Kumar K.V
Quoted 20 lines · Yu Zhao
Quoted 19 lines · Aneesh Kumar K.V
Quoted 1 line · Yu Zhao
Add
/sys/kernel/mm/lru_gen/enabledas a runtime kill switch.Got the below lockdep warning while using the above kill/enable switch
[ 84.252952] ====================================================== [ 84.253012] WARNING: possible circular locking dependency detected [ 84.253074] 5.16.0-rc8-16204-g1cdcf1120b31 #511 Not tainted [ 84.253135] ------------------------------------------------------ [ 84.253194] bash/2862 is trying to acquire lock: [ 84.253243] c0000000021ff740 (cgroup_mutex){+.+.}-{3:3}, at: store_enable+0x80/0x1510 [ 84.253340] but task is already holding lock: [ 84.253410] c000000002221348 (mem_hotplug_lock){++++}-{0:0}, at: mem_hotplug_begin+0x30/0x50 [ 84.253503] which lock already depends on the new lock.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
[ 84.255933] Chain exists of: cgroup_mutex --> cpu_hotplug_lock --> mem_hotplug_lockThanks. Will reverse the order between
mem_hotplug_lockandcgroup_mutexin the next spin.It also needs the unlocked variant of static_key_enable/disable.
Right. This is what I have at the moment. Tested with QEMU memory hotplug. Can you please give it try too? Thanks.
cgroup_lock()
cpus_read_lock()
get_online_mems()
if (enable)
static_branch_enable_cpuslocked()
else
static_branch_disable_cpuslocked()
put_online_mems()
cpus_read_unlock()
cgroup_unlock()
PATCH v6 9/9 mm: multigenerational lru: Kconfig
In reply to Yu ZhaoAdd configuration options for the multigenerational lru.
--- Documentation/vm/multigen_lru.rst | 18 ++++++++++++ mm/Kconfig | 48 +++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+)
Documentation/vm/multigen_lru.rst +18 -0
@@ -6,6 +6,13 @@ Multigenerational LRU Quick start =========== +Build configurations +-------------------- +:Required: Set ``CONFIG_LRU_GEN=y``. + +:Optional: Set ``CONFIG_LRU_GEN_ENABLED=y`` to enable this feature by + default. + Runtime configurations ---------------------- :Required: Write ``1`` to ``/sys/kernel/mm/lru_gen/enable`` if the
@@ -25,6 +32,17 @@ Personal computers Data centers ------------ +:Optional: Change ``CONFIG_NR_LRU_GENS`` to a larger value to support + more generations for ``Working set estimation`` and + ``Proactive reclaim``. + +:Optional: Change ``CONFIG_TIERS_PER_GEN`` to a larger value to + support more tiers, which generally provide better protection for + page cache when under heavy buffered I/O workloads. + +:Optional: Set ``CONFIG_LRU_GEN_STATS=y`` to enable full stats for + debugging. See ``Debugfs interface``. + :Debugfs interface: ``/sys/kernel/debug/lru_gen`` has the following format: ::
mm/Kconfig +48 -0
54 lines not on this page · raw message
Signature · 1 line
2.34.1.448.ga2b2bfdf31-goog
Re: [PATCH v6 9/9] mm: multigenerational lru: Kconfig
Quoted 15 lines · Yu Zhao
Add configuration options for the multigenerational lru.
def_bool ARCH_HAS_SET_DIRECT_MAP && !EMBEDDEDmm/Kconfig +11
-0+config NR_LRU_GENS + int "Max number of generations" + depends on LRU_GEN + range 4 31 + default 4 + help + This option uses order_base_2(N+1) bits in page flags. + + Do not configure more generations than you plan to use. They have a + per-memcg and per-node memory overhead. +
mm/Kconfig config IO_MAPPING · line 939
Quoted 10 lines
mm/Kconfig +10
-0+config TIERS_PER_GEN + int "Number of tiers per generation" + depends on LRU_GEN + range 2 5 + default 4 + help + This option uses N-2 bits in page flags. + + Larger values generally provide better protection for page cache when + under heavy buffered I/O workloads.
These are not appropriate questions to ask users.
No user has any idea what the answer should be. And no, we don't add "benchmark tuning Kconfig questions" to the kernel. We leave those kinds of games to companies that need to fake their benchmark numbers.
If *you* can't give a good number for these config options, then no user or distro can either.
So just pick a number, and stand by it.
Don't do this kind of "I don't know what the right number is, so I'll just push the blame on the user".
Linus
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.
<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.
Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.
Hi Andrew, Linus,
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Thanks!
Quoted 5 lines · Yu Zhao
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
I'll leave this to Andrew. I had some stylistic nits, but all the actual complexity is in that aging and eviction, and while I looked at the patches, I certainly couldn't make much of a judgement on them.
The proof is in the numbers, and they look fine, but who knows what happens when others test it. I don't see anything that looks worrisome per se, I just see the silly small things that made me go "Eww".
Linus
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Yu Zhao
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Quoted 7 lines
I'll leave this to Andrew. I had some stylistic nits, but all the actual complexity is in that aging and eviction, and while I looked at the patches, I certainly couldn't make much of a judgement on them.
The proof is in the numbers, and they look fine, but who knows what happens when others test it. I don't see anything that looks worrisome per se, I just see the silly small things that made me go "Eww".
I appreciate your time, I'll address all your comments togather with others' in the next spin, after I hear from Andrew. (I'm assuming he will have comments too.)
Michal Hocko
maintainerOn Tue 04-01-22 13:30:00, Yu Zhao wrote:
[...]
Quoted 4 lines
Hi Andrew, Linus,
Can you please take a look at this patchset and let me know if it's 5.17 material?
I am still not done with the review and have seen at least few problems that would need to be addressed.
But more fundamentally I believe there are really some important questions to be answered. First and foremost this is a major addition to the memory reclaim and there should be a wider consensus that we really want to go that way. The patchset doesn't have a single ack nor reviewed-by AFAICS. I haven't seen a lot of discussion since v2
(http://lkml.kernel.org/r/[email protected])
nor do I see any clarification on how concerns raised there have been addressed or at least how they are planned to be addressed.
Johannes has made some excellent points
http://lkml.kernel.org/r/[email protected]. Let me quote for reference part of it I find the most important:
: Realistically, I think incremental changes are unavoidable to get this : merged upstream.
:
: Not just in the sense that they need to be smaller changes, but also : in the sense that they need to replace old code. It would be : impossible to maintain both, focus development and testing resources, : and provide a reasonably stable experience with both systems tugging : at a complicated shared code base.
:
: On the other hand, the existing code also has billions of hours of : production testing and tuning. We can't throw this all out overnight - : it needs to be surgical and the broader consequences of each step need : to be well understood.
:
: We also have millions of servers relying on being able to do upgrades : for drivers and fixes in other subsystems that we can't put on hold : until we stabilized a new reclaim implementation from scratch.
Fully agreed on all points here.
I do appreciate there is a lot of work behind this patchset and I also do understand it has gained a considerable amount of testing as well. Your numbers are impressive but my experience tells me that it is equally important to understand the worst case behavior and there is not really much mentioned about those in changelogs.
We also shouldn't ignore costs the code is adding. One of them would be a further page flags depletion. We have been hitting problems on that front for years and many features had to be reworked to bypass a lack of space in page->flags.
I will be looking more into the code (especially the memcg side of it) but I really believe that a consensus on above Johannes' points need to be found first before this work can move forward.
Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 48 lines · Michal Hocko
On Tue 04-01-22 13:30:00, Yu Zhao wrote:
[...]Quoted 4 lines
Hi Andrew, Linus,
Can you please take a look at this patchset and let me know if it's 5.17 material?
I am still not done with the review and have seen at least few problems that would need to be addressed.
But more fundamentally I believe there are really some important questions to be answered. First and foremost this is a major addition to the memory reclaim and there should be a wider consensus that we really want to go that way. The patchset doesn't have a single ack nor reviewed-by AFAICS. I haven't seen a lot of discussion since v2
(http://lkml.kernel.org/r/[email protected])
nor do I see any clarification on how concerns raised there have been addressed or at least how they are planned to be addressed.Johannes has made some excellent points
http://lkml.kernel.org/r/[email protected]. Let me quote for reference part of it I find the most important:
: Realistically, I think incremental changes are unavoidable to get this : merged upstream.
:
: Not just in the sense that they need to be smaller changes, but also : in the sense that they need to replace old code. It would be : impossible to maintain both, focus development and testing resources, : and provide a reasonably stable experience with both systems tugging : at a complicated shared code base.
:
: On the other hand, the existing code also has billions of hours of : production testing and tuning. We can't throw this all out overnight - : it needs to be surgical and the broader consequences of each step need : to be well understood.
:
: We also have millions of servers relying on being able to do upgrades : for drivers and fixes in other subsystems that we can't put on hold : until we stabilized a new reclaim implementation from scratch.Fully agreed on all points here.
I do appreciate there is a lot of work behind this patchset and I also do understand it has gained a considerable amount of testing as well. Your numbers are impressive but my experience tells me that it is equally important to understand the worst case behavior and there is not really much mentioned about those in changelogs.
Quoted 8 lines
We also shouldn't ignore costs the code is adding. One of them would be a further page flags depletion. We have been hitting problems on that front for years and many features had to be reworked to bypass a lack of space in
page->flags.I will be looking more into the code (especially the memcg side of it) but I really believe that a consensus on above Johannes' points need to be found first before this work can move forward.
Thanks for the summary. I appreciate your time and I agree your assessment is fair.
So I've acknowledged your concerns, and you've acknowledged my numbers (the performance improvements) are impressive.
Now we are in agreement, cheers.
Next, I argue that the benefits of this patchset outweigh its risks, because, drawing from my past experience,
1. There have been many larger and/or riskier patchsets taken; I'll assemble a list if you disagree. And this patchset is fully guarded by #ifdef; Linus has also assessed on this point.
2. There have been none that came with the testing/benchmarking coverage as this one did. Please point me to some if I'm mistaken, and I'll gladly match them.
The numbers might not materialize in the real world; the code is not perfect; and many other risks... But all the top eight open source memory hogs were covered, which is unprecedented; memcached and fio showed significant improvements and it only takes a few commands to see for yourselves.
Regarding the acks and the reviewed-bys, I certainly can ask people who have reaped the benefits of this patchset to do them, if it's required. But I see less fun in that. I prefer to provide empirical evidence and convince people who are on the other side of the aisle.
Michal Hocko
maintainerOn Fri 07-01-22 11:45:40, Yu Zhao wrote:
[...]
Quoted 4 lines
Next, I argue that the benefits of this patchset outweigh its risks, because, drawing from my past experience,
1. There have been many larger and/or riskier patchsets taken; I'll assemble a list if you disagree.
No question about that. Changes in the reclaim path are paved with failures and reverts and fine tuning on top of existing fine tuning. The difference from your patchset is that they tend to be much much smaller and go incremental and therefore easier to review.
Quoted 2 lines
And this patchset is fully guarded
by #ifdef; Linus has also assessed on this point.
I appreciate you made the new behavior an opt-in and therefore existing workloads are less likely to regress. I do not think ifdefs help all that much, though, because a) realistically the config will likely be enabled for most distribution kernels and b) the parallel reclaim implementation adds a maintenance overhead regardless of those ifdef. The later point is especially worrying because the memory reclaim is a complex and hard to review beast already. Any future changes would need to consider both reclaim algorithms of course.
Hence I argue we really need a wider consensus this is the right direction we want to pursue.
Quoted 3 lines
2. There have been none that came with the testing/benchmarking coverage as this one did. Please point me to some if I'm mistaken, and I'll gladly match them.
I do appreciate your numbers but you should realize that this is an area that is really hard to get any conclusive testing for. We keep learning about fallouts on workloads we haven't really anticipated or where the runtime effects happen to disagree with our intuition. So while those numbers are nice there are other important aspects to consider like the maintenance cost for example.
Quoted 10 lines
The numbers might not materialize in the real world; the code is not perfect; and many other risks... But all the top eight open source memory hogs were covered, which is unprecedented; memcached and fio showed significant improvements and it only takes a few commands to see for yourselves.
Regarding the acks and the reviewed-bys, I certainly can ask people who have reaped the benefits of this patchset to do them, if it's required. But I see less fun in that. I prefer to provide empirical evidence and convince people who are on the other side of the aisle.
I like to hear from users who benefit from your work and that certainly gives more credit to it. But it will be the MM community to maintain the code and address future issues.
We do not have a dedicated maintainer for the memory reclaim but certainly there are people who have helped shaping the existing code and have learned a lot from the past issues - like Johannes, Rik, Mel just to name few. If I were you I would be really looking into finding an agreement with them. I myself can help you with memcg and oom side of the things (we already have discussions about those).
Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 11 lines · Michal Hocko
On Fri 07-01-22 11:45:40, Yu Zhao wrote:
[...]Quoted 4 lines
Next, I argue that the benefits of this patchset outweigh its risks, because, drawing from my past experience,
1. There have been many larger and/or riskier patchsets taken; I'll assemble a list if you disagree.No question about that. Changes in the reclaim path are paved with failures and reverts and fine tuning on top of existing fine tuning. The difference from your patchset is that they tend to be much much smaller and go incremental and therefore easier to review.
No argument here.
Quoted 7 lines
Quoted 2 lines
And this patchset is fully guarded
by #ifdef; Linus has also assessed on this point.I appreciate you made the new behavior an opt-in and therefore existing workloads are less likely to regress. I do not think ifdefs help all that much, though, because a) realistically the config will likely be enabled for most distribution kernels
There is also a runtime kill switch.
Quoted 5 lines
b) the parallel
reclaim implementation adds a maintenance overhead regardless of those ifdef. The later point is especially worrying because the memory reclaim is a complex and hard to review beast already. Any future changes would need to consider both reclaim algorithms of course.
A perfectly legitimate concern.
If this patchset is taken:
1. There will be refactoring that makes the long-term maintenance as affordable as possible, i.e., similar to the SL.B model, but can also make runtime switch.
2. There will also be optimizations for mmu notifier (KVM), THP, etc.
3. Most importantly, Google will be committing more resource on this. And that's why we need to hear a decision -- our resource planning depends on it.
Quoted 2 lines
Hence I argue we really need a wider consensus this is the right direction we want to pursue.
We've been doing our best to get this consensus -- we invited all the stakeholders to meetings a long time ago -- but unfortunately we couldn't move the needle.
I agree consensus is important. But, IMO, progress is even more important. And personally, I'd rather try something wrong than do nothing.
Quoted 6 lines
Quoted 3 lines
2. There have been none that came with the testing/benchmarking coverage as this one did. Please point me to some if I'm mistaken, and I'll gladly match them.
I do appreciate your numbers but you should realize that this is an area that is really hard to get any conclusive testing for.
Fully agreed. That's why we started a new initiative, and we hope more people will following these practices:
1. All results in this area should be reported with at least standard deviations, or preferably confidence intervals.
2. Real applications should be benchmarked (with synthetic load generator), not just synthetic benchmarks (not real applications).
3. A wide range of devices should be covered, i.e., servers, desktops, laptops and phones.
I'm very confident to say our benchmark reports were hold to the highest standards. We have worked with MariaDB (company), EnterpriseDB (Postgres), Redis (company), etc. on these reports. They have copies of these reports (PDF version):
https://linux-mm.googlesource.com/benchmarks/
We welcome any expert in those applications to examine our reports, and we'll be happy to run any other benchmarks or same benchmarks with different configurations that anybody thinks it's important and we've missed.
Quoted 5 lines
We keep learning
about fallouts on workloads we haven't really anticipated or where the runtime effects happen to disagree with our intuition. So while those numbers are nice there are other important aspects to consider like the maintenance cost for example.
I assume we agree this is not an easy decision. Can I also assume we agree that this decision should be make within a reasonable time frame?
Quoted 6 lines
Quoted 6 lines
The numbers might not materialize in the real world; the code is not perfect; and many other risks... But all the top eight open source memory hogs were covered, which is unprecedented; memcached and fio showed significant improvements and it only takes a few commands to see for yourselves.
Quoted 8 lines
Quoted 4 lines
Regarding the acks and the reviewed-bys, I certainly can ask people who have reaped the benefits of this patchset to do them, if it's required. But I see less fun in that. I prefer to provide empirical evidence and convince people who are on the other side of the aisle.
I like to hear from users who benefit from your work and that certainly gives more credit to it. But it will be the MM community to maintain the code and address future issues.
I'll ask downstream kernel maintainers (from different distros) that have taken this patchset to ACK.
I'll ask credible testers who are professionals, researchers, contributors to other subsystems to provide Test-by's. There are many other individual testers I may not be able to acknowledge their efforts, e.g., my coworker just sent this to me:
Using that v5 for some time and confirm that difference under heavy load and memory pressure is significant."
https://www.phoronix.com/forums/forum/software/general-linux-open-source/1301258-mglru-is-a-very-enticing-enhancement-for-linux-in-2022#post1301275
I'll leave the reviews in your capable hands. As I said, I prefer to convince people with empirical evidence.
Quoted 6 lines
We do not have a dedicated maintainer for the memory reclaim but certainly there are people who have helped shaping the existing code and have learned a lot from the past issues - like Johannes, Rik, Mel just to name few. If I were you I would be really looking into finding an agreement with them. I myself can help you with memcg and oom side of the things (we already have discussions about those).
Unfortunately people have different priorities. As I said, we tried to get all the stakeholders in the same (conference) room so that we can make some good progress. But we failed.
Rest assured, we'll keep trying. But please understand we need to do cost control and therefore we can't keep investing in this effort forever. So I think it's not unreasonable, after I've addressed all pending comments, to ask for some clear instructions from the leadership:
Yes
No
Or something specific
Thanks!
Quoted 22 lines
Quoted 6 lines
Quoted 3 lines
2. There have been none that came with the testing/benchmarking coverage as this one did. Please point me to some if I'm mistaken, and I'll gladly match them.
I do appreciate your numbers but you should realize that this is an area that is really hard to get any conclusive testing for.
Fully agreed. That's why we started a new initiative, and we hope more people will following these practices:
1. All results in this area should be reported with at least standard deviations, or preferably confidence intervals.
2. Real applications should be benchmarked (with synthetic load generator), not just synthetic benchmarks (not real applications).
3. A wide range of devices should be covered, i.e., servers, desktops, laptops and phones.I'm very confident to say our benchmark reports were hold to the highest standards. We have worked with MariaDB (company), EnterpriseDB (Postgres), Redis (company), etc. on these reports. They have copies of these reports (PDF version):
https://linux-mm.googlesource.com/benchmarks/
Quoted 4 lines
We welcome any expert in those applications to examine our reports, and we'll be happy to run any other benchmarks or same benchmarks with different configurations that anybody thinks it's important and we've missed.
I really think this gets at the heart of the issue with mm development, and is one of the reasons it's been extra frustrating to not have an MM conf for the past couple of years; I think sorting out how we measure & proceed on changes would be easier done f2f. E.g. concluding with a consensus that if something doesn't regress on X, Y, and Z, and has reasonably maintainable and readable code, we should merge it and try it out.
But since f2f isn't an option until 2052 at the earliest...
I understand the desire for an "incremental approach that gets us from A->B". In the abstract it sounds great. However, with a change like this one, I think it's highly likely that such a path would be littered with regressions both large and small, and would probably be more difficult to reason about than the relatively clean design of MGLRU. On top of that, I don't think we'll get the kind of user feedback we need for something like this *without* merging it. Yu has done a tremendous job collecting data here (and the results are really incredible), but I think we can all agree that without extensive testing in the field with all sorts of weird codes, we're not going to find the problematic behaviors we're concerned about.
So unless we want to eschew big mm changes entirely (we shouldn't! look at net or scheduling for how important big rewrites are to progress), I think we should be open to experimenting with new stuff. We can always revert if things get too unwieldy.
None of this is to say that there may not be lots more comments on the code or potential fixes/changes to incorporate before merging; I'm mainly arguing about the mindset we should have to changes like this, not all the stuff the community is already really good at (i.e. testing and reviewing code on a nuts & bolts level).
Thanks,
Jesse
Quoted 4 lines · Jesse Barnes
So unless we want to eschew big mm changes entirely (we shouldn't! look at net or scheduling for how important big rewrites are to progress), I think we should be open to experimenting with new stuff.
So I personally think this is worth going with, partly simply due to the reported improvements that have been measured.
But also to a large extent because the whole notion of doing multi-generational LRU isn't exactly some wackadoodle crazy thing. We already do active vs inactive, the whole multi-generational thing just doesn't seem to be so "far out".
But yes, numbers talk, and I get the feeling that we just need to try
it. Maybe not 5.17, but..
Linus
Michal Hocko
maintainerQuoted 27 lines · Jesse Barnes
Quoted 22 lines
Quoted 6 lines
Quoted 3 lines
2. There have been none that came with the testing/benchmarking coverage as this one did. Please point me to some if I'm mistaken, and I'll gladly match them.
I do appreciate your numbers but you should realize that this is an area that is really hard to get any conclusive testing for.
Fully agreed. That's why we started a new initiative, and we hope more people will following these practices:
1. All results in this area should be reported with at least standard deviations, or preferably confidence intervals.
2. Real applications should be benchmarked (with synthetic load generator), not just synthetic benchmarks (not real applications).
3. A wide range of devices should be covered, i.e., servers, desktops, laptops and phones.I'm very confident to say our benchmark reports were hold to the highest standards. We have worked with MariaDB (company), EnterpriseDB (Postgres), Redis (company), etc. on these reports. They have copies of these reports (PDF version):
https://linux-mm.googlesource.com/benchmarks/Quoted 4 lines
We welcome any expert in those applications to examine our reports, and we'll be happy to run any other benchmarks or same benchmarks with different configurations that anybody thinks it's important and we've missed.
Quoted 7 lines
I really think this gets at the heart of the issue with mm development, and is one of the reasons it's been extra frustrating to not have an MM conf for the past couple of years; I think sorting out how we measure & proceed on changes would be easier done f2f. E.g. concluding with a consensus that if something doesn't regress on X, Y, and Z, and has reasonably maintainable and readable code, we should merge it and try it out.
I am fully with you on that! I hope we can have LSFMM this year finally.
Quoted 1 line
But since f2f isn't an option until 2052 at the earliest...
Let's be more optimistic than that ;)
Quoted 6 lines
I understand the desire for an "incremental approach that gets us from
A->B". In the abstract it sounds great. However, with a change like this one, I think it's highly likely that such a path would be littered with regressions both large and small, and would probably be more difficult to reason about than the relatively clean design of MGLRU.
There are certainly things that do not make much sense to split up of course. On the other hand the patchset is making a lot of decisions and assumptions that are neither documented in the code nor in the changelog. From my past experience these are really problematic from a long term maintenance POV. We are struggling with those already because changelog tend to be much more coarse in the past yet the code stays with us and we have been really "great" at not touching many of those because "something might break". This results in the complexity grow and further maintenance burden.
Quoted 6 lines
On top of that, I don't think we'll get the kind of user feedback we need for something like this *without* merging it. Yu has done a tremendous job collecting data here (and the results are really incredible), but I think we can all agree that without extensive testing in the field with all sorts of weird codes, we're not going to find the problematic behaviors we're concerned about.
This is understood.
Quoted 4 lines
So unless we want to eschew big mm changes entirely (we shouldn't! look at net or scheduling for how important big rewrites are to progress), I think we should be open to experimenting with new stuff. We can always revert if things get too unwieldy.
As long as the patchset doesn't include new user visible interfaces which have proven to be really hard to revert.
Quoted 5 lines
None of this is to say that there may not be lots more comments on the code or potential fixes/changes to incorporate before merging; I'm mainly arguing about the mindset we should have to changes like this, not all the stuff the community is already really good at (i.e. testing and reviewing code on a nuts & bolts level).
From my reading of this and previous discussions I have gathered that there was no opposition just for the sake of it. There have been very specific questions regarding the implementation and/or future plans to address issues expressed in the past.
So far I have only managed to check the memcg and oom integration finding some issues there. All of them should be fixable reasonably easily but it also points that a deep dive into this is really necessary.
I have also raised questions about future maintainability of the resulting code. As you could have noticed the review power in the MM community is lacking behind and we tend to have more code producers than reviewers and maintainers.
Not to mention other things like page flags depletion which is something we have been struggling for quite some time already.
All that being said there is a lot of work for such a large change to be merged.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.org
Brian - the Chrome OS kernel memory maintainer
https://www.chromium.org
Jan - the Arch Linux Zen kernel maintainer
https://archlinux.org
Steven - the Liquorix kernel maintainer
https://liquorix.net
Suleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernel
Also my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcached
Holger - who has been testing/patching/contributing to various
subsystems since ~2008
Shuang - researcher at University of Rochester
benchmarked fio and provided a report
Sofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reports
Can you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Holger Hoffstätte
Tested-byQuoted 73 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernelAlso my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedHolger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a report
Quoted 8 lines
Sofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsCan you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Have been pounding on this "in production" on several different machines (server, desktop, laptop) and 5.15.x without any issues, so:
Looking forward to seeing this in mainline!
cheers,
Holger
Jan Alexander Steffens (heftig)
Acked-byQuoted 74 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernel
Also my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedHolger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a report
Quoted 8 lines
Sofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsCan you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Sure. Thanks for your work! Once it hits mainline, MGLRU will also be enabled in the vanilla Arch kernel.
Shuang Zhai
Tested-byQuoted 73 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernelAlso my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedHolger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a report
Quoted 7 lines
Sofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsCan you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
I have tested MGLRU using fio [1]. The performance improvement is fabulous. I hope this patchset could eventually get merged to enable large scale test and let more users talk about their experience.
Suleiman Souhlal
Acked-byQuoted 54 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.org
Quoted 8 lines
Jan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernel
Android on ChromeOS has been using MGLRU for a while now, with great results. It would be great for more people to more easily be able to benefit from it.
-- Suleiman
Sofia Trinh
Tested-byQuoted 68 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernelAlso my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcached
Quoted 8 lines
Holger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a reportSofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reports
Daniel Byrne
Tested-byof byte-addressable Optante, performance improvement in terms of throughput (measured as queries per second) was about 10% for a series of workloads.
Quoted 84 lines · Sofia Trinh
Quoted 6 lines · Yu Zhao
Quoted 4 lines · Yu Zhao
Quoted 3 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage andit
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.
<snipped>
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smallerchanges
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
with similar effects.
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements willlikely
Quoted 30 lines
Quoted 11 lines
Quoted 8 lines
materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgQuoted 8 lines
Jan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernel
Quoted 6 lines
Also my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedQuoted 8 lines
Holger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a reportSofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsTested-by Sofia Trinh <[email protected]>
Signature · 1 line
Daniel Byrne
Quoted 34 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
My gratitude to Donald who has been helping test MGLRU since v2:
Donald Carr ([email protected])
Founder of Chaos Reins (http://chaos-reins.com), an SF based consultancy company specializing in designing/creating embedded Linux appliances.
Can you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Donald Carr
Tested-byQuoted 44 lines · Yu Zhao
Quoted 34 lines · Yu Zhao
On Tue, Jan 04, 2022 at 01:30:00PM -0700, Yu Zhao wrote:
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Quoted 7 lines
My gratitude to Donald who has been helping test MGLRU since v2:
Donald Carr ([email protected])
Founder of Chaos Reins (http://chaos-reins.com), an SF based consultancy company specializing in designing/creating embedded Linux appliances.
Quoted 4 lines
Can you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Steven Barrett
Acked-byQuoted 73 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernelAlso my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedHolger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a report
Quoted 7 lines
Sofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsCan you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
This feature has been a huge improvement for desktop linux, system responsiveness has hit a new level high memory pressure. Thanks Yu!
Brian Geffon
Acked-byQuoted 46 lines · Yu Zhao
Quoted 33 lines · Yu Zhao
Quoted 6 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.<snipped>
Quoted 10 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Quoted 10 lines
Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.Hi Andrew, Linus,
Quoted 7 lines
Can you please take a look at this patchset and let me know if it's 5.17 material?
My goal is to get it merged asap so that users can reap the benefits and I can push the sequels. Please examine the data provided -- I think the unprecedented coverage and the magnitude of the improvements warrant a green light.
Downstream kernel maintainers who have been carrying MGLRU for more than 3 versions, can you please provide your Acked-by tags?
Quoted 7 lines
Having this patchset in the mainline will make your job easier :)
Alexandre - the XanMod Kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.org
MGLRU has been maturing in ChromeOS for quite some time, we've maintained it in a number of different kernels between 4.14 and 5.15, and it's become the default
for tens of millions of users. We've seen substantial improvements in terms of CPU utilization and memory pressure resulting in fewer OOM kills and reduced UI latency. I would love to see this make it upstream so more desktop users can benefit.
Quoted 28 lines
Jan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernelAlso my gratitude to those who have helped test MGLRU:
Daniel - researcher at Michigan Tech
benchmarked memcachedHolger - who has been testing/patching/contributing to various
subsystems since ~2008Shuang - researcher at University of Rochester
benchmarked fio and provided a reportSofia - EDI https://www.edi.works
benchmarked the top eight memory hogs and provided reportsCan you please provide your Tested-by tags? This will ensure the credit for your contributions.
Thanks!
Fio / pmem benchmark with MGLRU
TLDR
====
With the MGLRU, fio achieved 95% CIs [38.95, 40.26]%, [4.12, 6.64]% and [9.26, 10.36]% higher throughput, respectively, for random access, Zipfian (distribution) access and Gaussian (distribution) access, when the average number of jobs per CPU is 1; 95% CIs [42.32, 49.15]%, [9.44, 9.89]% and [20.99, 22.86]% higher throughput, respectively, for random access, Zipfian access and Gaussian access, when the average number of jobs per CPU is 2.
Background
==========
Many applications running on warehouse-scale computers heavily use POSIX read(2)/write(2) and page cache, e.g., Apache Kafka, a distributed streaming application used by "more than 80% of all Fortune 100 companies" [1] and PostgreSQL, "the world's most advanced open source relational database" [2].
Intel DC Persistent Memory, as an affordable alternative to DRAM, can deliver large capacity and data persistence. Specifically, the device used in this benchmark can achieve up to 36 GiB/s and 15 GiB/s throughput, respectively, for sequential and random read access.
Our research group at the University of Rochester focuses on the intersection of computer architecture and system software. My current research interest is memory management on tiered memory systems.
Matrix
======
Kernels: version [+ patchset]
* Baseline: 5.15
* Patched: 5.15 + MGLRU
Access patterns (4KB read):
* Random (uniform)
* Zipfian (theta 0.8; the recommended range is 0-2)
* Gaussian (deviation 40; the possible range is 0-100)
Concurrency conditions (the average number of jobs per CPU):
* 1
* 2
Total file size (GB): 400 (~2x memory capacity)
Total configurations: 12
Data points per configuration: 10
Total run duration (minutes) per data point: ~30
Notes
-----
1. All files were stored on pmem. Each job had the exclusive access to a single file.
2. Due to the hardware limitation when accessing remote pmem [3], numactl was used to bind the fio processes to the local pmem. Only one of the two NUMA nodes was used during the benchmark.
3. During dry runs, we observed that the throughput doesn't improve beyond 2 jobs per CPU for random access. Moreover, the patched kernel showed consistent improvements over the baseline kernel when using 3 or 4 jobs per CPU.
4. We wanted to simulate the real-world scenarios and therefore used default swap configuration (on). Moreover, we didn't observe any negative impact on performance with dry runs that disabled swap.
Procedure
=========
<for each kernel>
grub2-reboot <baseline, patched>
<for each concurrency condition>
<generate test files>
<for each access pattern>
<for each data point>
<reboot>
<run fio>
Hardware
--------
Memory (GiB per socket): 192
CPU (# per socket): 40
Pmem (GiB per socket): 768
Fio
---
$ fio -version fio-3.28
$ numactl --cpubind=0 --membind=0 fio --name=randread \
--directory=/mnt/pmem/ --size={10G, 5G} --io_size=1000TB \
--time_based --numjobs={40, 80} --ioengine=io_uring \
--ramp_time=20m --runtime=10m --iodepth=128 \
--iodepth_batch_submit=32 --iodepth_batch_complete=32 \
--rw=randread --random_distribution={random, zipf:0.8, normal:40} \
--direct=0 --norandommap --group_reportingResults
=======
Throughput
----------
The patched kernel achieved substantially higher throughput for all three access patterns and two concurrency conditions. Specifically, comparing the patched with the baseline kernel, fio achieved 95% CIs [38.95, 40.26]%, [4.12, 6.64]% and [9.26, 10.36]% higher throughput, respectively, for random access, Zipfian access, and Gaussian access, when the average number of jobs per CPU is 1; 95% CIs [42.32, 49.15]%, [9.44, 9.89]% and [20.99, 22.86]% higher throughput, respectively, for random access, Zipfian access and Gaussian access, when the average number of jobs per CPU is 2.
+---------------------+---------------+---------------+ | Mean MiB/s [95% CI] | 1 job / CPU | 2 jobs / CPU | +---------------------+---------------+---------------+ | Random access | 8411 / 11742 | 8417 / 12267 | | | [3275, 3387] | [3562, 4137] | +---------------------+---------------+---------------+ | Zipfian access | 14576 / 15360 | 12932 / 14181 | | | [600, 967] | [1220, 1279] | +---------------------+---------------+---------------+ | Gaussian access | 14564 / 15993 | 11513 / 14037 | | | [1348, 1508] | [2417, 2631] | +---------------------+---------------+---------------+
Table 1. Throughput comparison between the baseline and the patched
kernels
The patched kernel exhibited less degradation in throughput when running more concurrent jobs. Comparing 2 jobs per CPU with 1 job per CPU, fio achieved 95% CIs [-11.54, -11.02]%, [-16.91, -12.01]% and [-21.61, -20.30]% higher throughput, respectively, for random access, Zipfian access and Gaussian access, when using the baseline kernel; 95% CIs [2.04, 6.92]%, [-8.86, -6.48]% and [-12.83, -11.64]% higher throughput, respectively, for random access, Zipfian access and Gaussian access, when using the patched kernel. There were no statistically significant changes in throughput for the rest of the test matrix.
+---------------------+-----------------+----------------+ | Mean MiB/s [95% CI] | Baseline kernel | Patched kernel | +---------------------+-----------------+----------------+ | Random access | 8411 / 8417 | 11741 / 12267 | | | [-55, 69] | [239, 812] | +---------------------+-----------------+----------------+ | Zipfian access | 14576 / 12932 | 15360/ 14181 | | | [-1682, -1607] | [-1361, -996] | +---------------------+-----------------+----------------+ | Gaussian access | 14565 / 11513 | 15993 / 14037 | | | [-3147, -2957] | [-2051, -1861] | +---------------------+-----------------+----------------+
Table 2. Throughput comparison between 1 job per CPU and 2 jobs per
CPU
Tail Latency
------------
Comparing the patched with the baseline kernel, fio experienced 95% CIs [-41.77, -40.35]% and [6.64, 13.95]% higher latency at the 99th percentile, respectively, for random access and Gaussian access, when the average number of jobs per CPU is 1; 95% CIs [-41.97, -40.59]%, [-47.74, -47.04]% and [-51.32, -50.27]% higher latency at the 99th percentile, respectively, for random access, Zipfian access and Gaussian access, when the average number of jobs per CPU is 2. There were no statistically significant changes in latency at the 99th percentile for the rest of the test matrix.
+------------------------------+----------------+------------------+ | 99th percentile latency (us) | 1 job / CPU | 2 jobs / CPU | +------------------------------+----------------+------------------+ | Random access | 12466 / 7347 | 25560 / 15008 | | | [-5207, -5030] | [-10729, -10375] | +------------------------------+----------------+------------------+ | Zipfian access | 3395 / 3382 | 14563 / 7661 | | | [-131, 105] | [-6953,-6850] | +------------------------------+----------------+------------------+ | Gaussian access | 3280 / 3618 | 15611 / 7681 | | | [217, 457] | [-8012, -7848] | +------------------------------+----------------+------------------+
Table 3. Comparison of the 99th percentile latency between the
baseline and the patched kernels (lower is better)
Metrics collected during each run are available at:
https://github.com/zhaishuang1/MglruPerf/tree/master
A peek at 5.16-rc6
------------------
We also ran the benchmark on 5.16-rc6 with swap off. However, we haven't collected enough data points to establish a 95% CI. Here are a few numbers we've collected:
+----------------+------------+----------+----------------+----------+ | Access pattern | Jobs / CPU | 5.16-rc6 | 5.16-rc6-mglru | % change | +----------------+------------+----------+----------------+----------+ | Random access | 1 | 7467 | 10440 | 39.8% | +----------------+------------+----------+----------------+----------+ | Random access | 2 | 7504 | 13417 | 78.8% | +----------------+------------+----------+----------------+----------+ | Random access | 3 | 7511 | 13954 | 85.8% | +----------------+------------+----------+----------------+----------+ | Random access | 4 | 7542 | 13925 | 84.6% | +----------------+------------+----------+----------------+----------+
Reference
=========
[1] https://kafka.apache.org/documentation/#design_filesystem
[2] https://www.postgresql.org/docs/11/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-MEMORY
[3] System Evaluation of the Intel Optane byte-addressable NVM, MEMSYS 2019.
Appendix
========
Throughput
----------
$ cat raw_data_fio.r
v <- c(
# baseline 40 procs random
8467.89, 8428.34, 8383.32, 8253.12, 8464.65, 8307.42, 8424.78, 8434.44, 8474.88, 8468.26,
# baseline 40 procs zipf
14570.44, 14598.03, 14550.74, 14640.29, 14591.4, 14573.35, 14503.18, 14613.39, 14598.61, 14522.27,
# baseline 40 procs gaussian
14504.95, 14427.23, 14652.19, 14519.47, 14557.97, 14617.92, 14555.87, 14446.94, 14678.12, 14688.33,
# baseline 80 procs random
8427.51, 8267.23, 8437.48, 8432.37, 8441.4, 8454.26, 8413.13, 8412.44, 8444.36, 8444.32,
# baseline 80 procs zipf
12980.12, 12946.43, 12911.95, 12925.83, 12952.75, 12841.44, 12920.35, 12924.19, 12944.38, 12967.72,
# baseline 80 procs gaussian
11666.29, 11624.72, 11454.82, 11482.36, 11462.24, 11379.46, 11691.5, 11471.19, 11402.08, 11494.13,
# patched 40 procs random
11706.69, 11778.1, 11774.07, 11750.07, 11744.97, 11766.65, 11727.79, 11708.41, 11745.3, 11716.45,
# patched 40 procs zipf
15498.31, 14647.94, 15423.35, 15467.32, 15467.05, 15342.49, 15511.34, 15414.06, 15401.1, 15431.57,
# patched 40 procs gaussian
15957.86, 15957.13, 16022.69, 16035.85, 16150.2, 15904.5, 15943.36, 16036.78, 16025.95, 15900.56,
# patched 80 procs random
12568.51, 11772.25, 11622.15, 12057.66, 11971.72, 12693.36, 12399.71, 12553.23, 12242.74, 12793.34,
# patched 80 procs zipf
14194.78, 14213.61, 14148.66, 14182.35, 14183.91, 14192.23, 14163.2, 14179.7, 14162.12, 14196.34,
# patched 80 procs gaussian
14084.86, 13706.34, 14089.42, 14058.4, 14096.74, 14108.06, 14043.41, 14072.15, 14088.44, 14024.51
)a <- array(v, dim = c(10, 3, 2, 2))
# baseline vs patched
for (concurr in 1:2) {
for (dist in 1:3) {
r <- t.test(a[, dist, concurr, 1], a[, dist, concurr, 2])
print(r) p <- r$conf.int * 100 / r$estimate[1]
if ((p[1] > 0 && p[2] < 0) || (p[1] < 0 && p[2] > 0)) {
s <- sprintf("concurr%d dist%d: no significance", concurr, dist)
} else {
s <- sprintf("concurr%d dist%d: [%.2f, %.2f]%%", concurr, dist, -p[2], -p[1])
}
print(s)
}
}# low concurr vs high concurr
for (kern in 1:2) {
for (dist in 1:3) {
r <- t.test(a[, dist, 1, kern], a[, dist, 2, kern])
print(r) p <- r$conf.int * 100 / r$estimate[1]
if ((p[1] > 0 && p[2] < 0) || (p[1] < 0 && p[2] > 0)) {
s <- sprintf("kern%d dist%d: no significance", kern, dist)
} else {
s <- sprintf("kern%d dist%d: [%.2f, %.2f]%%", kern, dist, -p[2], -p[1])
}
print(s)
}
}$ R -q -s -f raw_data_fio.r
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -132.15, df = 11.177, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-3386.514 -3275.766
sample estimates:
mean of x mean of y
8410.71 11741.85
[1] "concurr1 dist1: [38.95, 40.26]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -9.5917, df = 9.4797, p-value = 3.463e-06 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-967.8353 -600.7307
sample estimates:
mean of x mean of y
14576.17 15360.45
[1] "concurr1 dist2: [4.12, 6.64]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -37.744, df = 17.33, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-1508.328 -1348.850
sample estimates:
mean of x mean of y
14564.90 15993.49
[1] "concurr1 dist3: [9.26, 10.36]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -30.144, df = 9.3334, p-value = 1.281e-10 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-4137.381 -3562.653
sample estimates:
mean of x mean of y
8417.45 12267.47
[1] "concurr2 dist1: [42.32, 49.15]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -92.164, df = 13.276, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-1279.417 -1220.931
sample estimates:
mean of x mean of y
12931.52 14181.69
[1] "concurr2 dist2: [9.44, 9.89]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -49.453, df = 17.863, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-2631.656 -2417.052
sample estimates:
mean of x mean of y
11512.88 14037.23
[1] "concurr2 dist3: [20.99, 22.86]%"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = -0.22947, df = 16.403, p-value = 0.8213
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-68.88155 55.40155
sample estimates:
mean of x mean of y
8410.71 8417.45
[1] "kern1 dist1: no significance"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = 91.86, df = 17.875, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
1607.021 1682.287
sample estimates:
mean of x mean of y
14576.17 12931.52
[1] "kern1 dist2: [-11.54, -11.02]%"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = 67.477, df = 17.539, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
2956.815 3147.225
sample estimates:
mean of x mean of y
14564.90 11512.88
[1] "kern1 dist3: [-21.61, -20.30]%"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = -4.1443, df = 9.0781, p-value = 0.002459 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-812.1507 -239.0833
sample estimates:
mean of x mean of y
11741.85 12267.47
[1] "kern2 dist1: [2.04, 6.92]%"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = 14.566, df = 9.1026, p-value = 1.291e-07 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
996.0064 1361.5196
sample estimates:
mean of x mean of y
15360.45 14181.69
[1] "kern2 dist2: [-8.86, -6.48]%"
Welch Two Sample t-test
data: a[, dist, 1, kern] and a[, dist, 2, kern]
t = 43.826, df = 15.275, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
1861.263 2051.247
sample estimates:
mean of x mean of y
15993.49 14037.23
[1] "kern2 dist3: [-12.83, -11.64]%"
99th Percentile Latency
-----------------------
$ cat raw_data_fio_lat.r
v <- c(
# baseline 40 procs random
12649, 12387, 12518, 12518, 12518, 12387, 12518, 12518, 12387, 12256,
# baseline 40 procs zipf
3458, 3294, 3425, 3294, 3294, 3359, 3752, 3326, 3294, 3458,
# baseline 40 procs gaussian
3326, 3458, 3195, 3392, 3326, 3228, 3228, 3326, 3130, 3195,
# baseline 80 procs random
25560, 26084, 25560, 25560, 25297, 25297, 25822, 25560, 25560, 25297,
# baseline 80 procs zipf
14484, 14615, 14615, 14484, 14484, 14615, 14615, 14615, 14615, 14484,
# baseline 80 procs gaussian
15664, 15664, 15533, 15533, 15533, 15664, 15795, 15533, 15664, 15533,
# patched 40 procs random
7439, 7242, 7373, 7373, 7373, 7439, 7242, 7308, 7308, 7373,
# patched 40 procs zipf
3261, 3425, 3392, 3294, 3359, 3556, 3228, 3490, 3458, 3359,
# patched 40 procs gaussian
3687, 3523, 3556, 3523, 3752, 3654, 3884, 3490, 3392, 3720,
# patched 80 procs random
15008, 15008, 15008, 15008, 15008, 15008, 15008, 15008, 15008, 15008,
# patched 80 procs zipf
7701, 7635, 7701, 7701, 7635, 7635, 7701, 7635, 7635, 7635,
# patched 80 procs gaussian
7635, 7898, 7701, 7635, 7635, 7635, 7635, 7635, 7701, 7701
)a <- array(v, dim = c(10, 3, 2, 2))
# baseline vs patched
for (concurr in 1:2) {
for (dist in 1:3) {
r <- t.test(a[, dist, concurr, 1], a[, dist, concurr, 2])
print(r) p <- r$conf.int * 100 / r$estimate[1]
if ((p[1] > 0 && p[2] < 0) || (p[1] < 0 && p[2] > 0)) {
s <- sprintf("concurr%d dist%d: no significance", concurr, dist)
} else {
s <- sprintf("concurr%d dist%d: [%.2f, %.2f]%%", concurr, dist, -p[2], -p[1])
}
print(s)
}
}$ R -q -s -f raw_data_fio_lat.r
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = 123.52, df = 15.287, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
5030.417 5206.783
sample estimates:
mean of x mean of y
12465.6 7347.0
[1] "concurr1 dist1: [-41.77, -40.35]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = 0.23667, df = 16.437, p-value = 0.8158
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-104.7812 131.1812
sample estimates:
mean of x mean of y
3395.4 3382.2
[1] "concurr1 dist2: no significance"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = -5.9754, df = 16.001, p-value = 1.94e-05 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
-457.5065 -217.8935
sample estimates:
mean of x mean of y
3280.4 3618.1
[1] "concurr1 dist3: [6.64, 13.95]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = 134.89, df = 9, p-value = 3.437e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
10374.74 10728.66
sample estimates:
mean of x mean of y
25559.7 15008.0
[1] "concurr2 dist1: [-41.97, -40.59]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = 288.1, df = 13.292, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
6849.566 6952.834
sample estimates:
mean of x mean of y
14562.6 7661.4
[1] "concurr2 dist2: [-47.74, -47.04]%"
Welch Two Sample t-test
data: a[, dist, concurr, 1] and a[, dist, concurr, 2] t = 203.64, df = 17.798, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval:
7848.616 8012.384
sample estimates:
mean of x mean of y
15611.6 7681.1
[1] "concurr2 dist3: [-51.32, -50.27]%"
Hi Yu,
Quoted 7 lines · On Tue, 4 Jan 2022 13:22:19 -0700 Yu Zhao <[email protected]> wrote:
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.
[...]
Quoted 5 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
So impressive results!
Quoted 2 lines
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
I think similar works are already available out of the box with the latest mainline tree, though it might be suboptimal in some cases.
First, you can do thrashing prevention using DAMON-based Operation Scheme (DAMOS)[1] with MADV_COLD action. Second, for working set estimation, you can either use the DAMOS again with statistics action, or the damon_aggregated tracepoint[2]. The DAMON user space tool[3] helps the tracepoint analysis and visualization. Finally, for the proactive reclaim, you can again use the DAMOS with MADV_PAGEOUT action, or simply the DAMON-based proactive reclaim module (DAMON_RECLAIM)[4].
Nevertheless, as noted above, current DAMON based solutions might be suboptimal for some cases. First of all, DAMON currently doesn't provide page granularity monitoring. Though its monitoring results were useful for our users' production usages, there could be different requirements and situations. Secondly, the DAMON-based thrashing prevention wouldn't reduce the CPU usage of the reclamation logic's access scanning.
So, to me, MGLRU patchset looks providing something that DAMON doesn't provide, but also something that DAMON is already providing. Specifically, the efficient page granularity access scanning is what DAMON doesn't provide for now. However, the utilization of the access information for LRU list manipulation (thrashing prevention) and proactive reclamation is similar to what DAMON (specifically, DAMOS) provides. Also, this patchset is reducing the reclamation logic's CPU usage using the efficient page granularity access scanning.
IMHO, we might be able to reduce the duplicates by integrating MGLRU in DAMON. What I'm saying is, we could 1) introduce the efficient page granularity access scanning, 2) reduce the reclamation logic's CPU usage by making it to use the efficient page granularity access scanning, and 3) extend DAMON for page granularity monitoring with the efficient access sacanning[5]. Then, users could get the benefit of MGLRU by using DAMOS but setting it to use your efficient page granularity access scanning. To make it more simple, we can extend existing kernel logics to use DAMON in the way, or implement a new kernel module. Additional advantages of this approach would be 1) reducing the changes to the existing code, and 2) making the efficient page granularity access information be utilized for more general cases.
Of course, the integration might not be so simple as seems to me now. We could put DAMON and MGLRU together as those are for now, and let users select what they really want. I think it's up to you.
I didn't read this patchset thoroughly yet, so I might missing many things. If so, please feel free to let me know.
[1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#schemes
[2] https://docs.kernel.org/admin-guide/mm/damon/usage.html#tracepoint-for-monitoring-results
[3] https://github.com/awslabs/damo
[4] https://docs.kernel.org/admin-guide/mm/damon/reclaim.html
[5] https://docs.kernel.org/vm/damon/design.html#configurable-layers
Thanks,
SJ
[...]
Quoted 18 lines · SeongJae Park
Hi Yu,
Quoted 7 lines · On Tue, 4 Jan 2022 13:22:19 -0700 Yu Zhao <[email protected]> wrote:
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.[...]
Quoted 5 lines
Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
Quoted 7 lines
So impressive results!
Quoted 2 lines
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
I think similar works are already available out of the box with the latest mainline tree, though it might be suboptimal in some cases.
Ok, I will sound harsh because I hate it when people challenge facts while having no idea what they are talking about.
Our jobs are help the leadership make best decisions by providing them with facts, not feeding them crap.
Don't get me wrong -- you are welcome to start another thread and have a casual discussion with me. But this thread is not for that; it's for the leadership and stakeholder to make a decision. Check who are in "To" and "Cc" and what my request is.
Quoted 2 lines
I didn't read this patchset thoroughly yet, so I might missing many things. If so, please feel free to let me know.
Yes, apparently you didn't read this patchset thoroughly, and you have missed all things that matter to this thread.
Quoted 2 lines
First, you can do thrashing prevention using DAMON-based Operation Scheme (DAMOS)[1] with
MADV_COLDaction.
Here is thrashing prevention really means, from patch 8:
+Personal computers
+------------------
+:Thrashing prevention: Write ``N`` to
+ ``/sys/kernel/mm/lru_gen/min_ttl_ms`` to prevent the working set of
+ ``N`` milliseconds from getting evicted. The OOM killer is invoked if
+ this working set can't be kept in memory. Based on the average human
+ detectable lag (~100ms), ``N=1000`` usually eliminates intolerable
+ lags due to thrashing. Larger values like ``N=3000`` make lags less
+ noticeable at the cost of more OOM kills.
It's about when to trigger OOM kills. Got it? Or probably you don't understand what MADV_COLD is either?
Quoted 2 lines
Second, for working set estimation, you can either use the DAMOS again with statistics action, or the
damon_aggregatedtracepoint[2].
This is you are suggesting:
TRACE_EVENT(damon_aggregated,
TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u",
__entry->target_id, __entry->nr_regions,
__entry->start, __entry->end, __entry->nr_accesses)Now read my doc again:
+Data centers
+------------
+:Debugfs interface: ``/sys/kernel/debug/lru_gen`` has the following
+ format:
+ memcg memcg_id memcg_path
+ node node_id
Have you heard of something called memcg? And NUMA node? How exactly can this tracepoint provide information about different memcgs and NUMA node?
Quoted 2 lines
The DAMON user space tool[3] helps the tracepoint analysis and visualization.
What does "work out of box" mean? Should every Linux desktop, laptop and phone user install this tool?
Quoted 2 lines
Finally, for the proactive reclaim, you can again use the DAMOS with
MADV_PAGEOUTaction
How exactly does MADV_PAGEOUT find pages that are NOT mapped in page tables? Let me tell you another fact: they are usually the cheapest to reclaim.
Quoted 2 lines
or simply the DAMON-based proactive reclaim module (DAMON_RECLAIM)[4].
[4] https://docs.kernel.org/admin-guide/mm/damon/reclaim.html
How many knob does DAMON_RECLAIM have? 14? I lost count.
Quoted 1 line
Of course, the integration might not be so simple as seems to me now.
Look, I'm open to your suggestion. I probably should have been nicer. So I'm sorry. I just don't appreciate alternative facts.
Borislav Petkov
maintainerQuoted 2 lines · Yu Zhao
Look, I'm open to your suggestion. I probably should have been nicer. So I'm sorry. I just don't appreciate alternative facts.
Yes, you should've been *much* nicer. I'm reading lkml for pretty much 20 years now and you just made my eyebrows go up - something which pretty much never happens these days.
So you need to check yourself before replying. Looking at git history, you're not a newbie so you've probably picked up - at least from the sidelines - all those code of conduct discussions. And I'm not going to point you to it - I'm sure you can find it yourself and peruse it at your own convenience.
Long story short: we all try to be civil to each other now, even if it is hard sometimes.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 4 lines · On Wed, 5 Jan 2022 03:53:07 -0700 Yu Zhao <[email protected]> wrote:
Quoted 3 lines · SeongJae Park
Hi Yu,
On Tue, 4 Jan 2022 13:22:19 -0700 Yu Zhao <[email protected]> wrote:
[...]
Quoted 8 lines
Quoted 2 lines
I think similar works are already available out of the box with the latest mainline tree, though it might be suboptimal in some cases.
Ok, I will sound harsh because I hate it when people challenge facts while having no idea what they are talking about.
Our jobs are help the leadership make best decisions by providing them with facts, not feeding them crap.
I was using the word "similar", to represent this is only for a rough concept level similarity, rather than detailed facts. But, seems it was not enough, sorry. Anyway, I will not talk more and thus disturb you having the important discussion with leaders here, as you are asking.
Quoted 5 lines
Don't get me wrong -- you are welcome to start another thread and have a casual discussion with me. But this thread is not for that; it's for the leadership and stakeholder to make a decision. Check who are in "To" and "Cc" and what my request is.
Haha. Ok, good luck for you.
Thanks,
SJ
[...]
Quoted 13 lines · SeongJae Park
Quoted 4 lines · On Wed, 5 Jan 2022 03:53:07 -0700 Yu Zhao <[email protected]> wrote:
Quoted 3 lines · SeongJae Park
Hi Yu,
On Tue, 4 Jan 2022 13:22:19 -0700 Yu Zhao <[email protected]> wrote:
[...]
Quoted 6 lines
Quoted 2 lines
I think similar works are already available out of the box with the latest mainline tree, though it might be suboptimal in some cases.
Ok, I will sound harsh because I hate it when people challenge facts while having no idea what they are talking about.
Quoted 7 lines
Quoted 2 lines
Our jobs are help the leadership make best decisions by providing them with facts, not feeding them crap.
I was using the word "similar", to represent this is only for a rough concept level similarity, rather than detailed facts. But, seems it was not enough, sorry. Anyway, I will not talk more and thus disturb you having the important discussion with leaders here, as you are asking.
First of all, I want to apologize.
I detested what I read, and I still don't like "a rough concept level similarity" sitting next to a factual statement. But as Borislav has reminded me, my tone did cross the line. I should have had used an objective approach to express my (very) different views.
I hope that's all water under the bridge now. And I do plan to carry on with what I should have had done.
Note that with vm.swappiness=0, the vm.watermark_scale_factor value does not affect the swap possibility: MGLRU ignores sc->file_is_tiny.
With the classic 2-gen LRU swapping goes well at swappiness=0 and high vm.watermark_scale_factor, which is expected according to the documentation:
"At 0, the kernel will not initiate swap until the amount of free and file-backed pages is less than the high watermark in a zone." [1]
With vm.swappiness=0, no swap occurs with any vm.watermark_scale_factor value with MGLRU.
I used to see in practice (with MGLRU v3) the impossibility of swapping when vm.swappiness=0 and vm.watermark_scale_factor=1000.
At a minimum, this will require updating the documentation for vm.swappiness.
BTW, why MGLRU doesn't use something like sc->file_is_tiny?
[1] https://github.com/torvalds/linux/blob/v5.16/Documentation/admin-guide/sysctl/vm.rst#swappiness
In some of my benchmarks MGLRU really gave unrivaled performance. I assume the adoption of MGLRU into the kernel would save billions of dollars and greatly reduce carbon dioxide emissions.
However, there are also cases where MGLRU loses.
There are cases where MGLRU does not achieve the performance that the classic LRU gives (at least I got such results when testing MGLRU before[1], but I did not report them here).
As a Linux user, I would like to see both variants of LRU in the kernel, so that it is possible to switch to the suitable variant when needed: none of the LRU variants allowed me to squeeze the maximum for all cases.
I hope to test MGLRU v6 later and show you some of its weaknesses and anomalies with specific logs and benchmarks.
[1] I didn't have enough time and energy to decipher the results at that time:
https://github.com/hakavlad/cache-tests/tree/main/mg-LRU-v3_vs_classic-LRU
(but you can try to guess what it all means)
Hello.
Quoted 304 lines · On úterý 4. ledna 2022 21:22:19 CET Yu Zhao wrote:
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.Design objectives
=================
The design objectives are:
1. Better representation of access recency
2. Try to profit from spatial locality
3. Clear fast path making obvious choices
4. Simple self-correcting heuristicsThe representation of access recency is at the core of all LRU approximations. The multigenerational LRU (MGLRU) divides pages into multiple lists (generations), each having bounded access recency (a time interval). Generations establish a common frame of reference and help make better choices, e.g., between different memcgs on a computer or different computers in a data center (for cluster job scheduling).
Exploiting spatial locality improves the efficiency when gathering the accessed bit. A rmap walk targets a single page and doesn't try to profit from discovering an accessed PTE. A page table walk can sweep all hotspots in an address space, but its search space can be too large to make a profit. The key is to optimize both methods and use them in combination. (PMU is another option for further exploration.)
Fast path reduces code complexity and runtime overhead. Unmapped pages don't require TLB flushes; clean pages don't require writeback. These facts are only helpful when other conditions, e.g., access recency, are similar. With generations as a common frame of reference, additional factors stand out. But obvious choices might not be good choices; thus self-correction is required (the next objective).
The benefits of simple self-correcting heuristics are self-evident. Again with generations as a common frame of reference, this becomes attainable. Specifically, pages in the same generation are categorized based on additional factors, and a closed-loop control statistically compares the refault percentages across all categories and throttles the eviction of those that have higher percentages.
Patchset overview
=================
1. mm: x86, arm64: addarch_has_hw_pte_young()
2. mm: x86: addCONFIG_ARCH_HAS_NONLEAF_PMD_YOUNGMaterializing hardware optimizations when trying to clear the accessed bit in many PTEs. If hardware automatically sets the accessed bit in PTEs, there is no need to worry about bursty page faults (emulating the accessed bit). If it also sets the accessed bit in non-leaf PMD entries, there is no need to search the PTE table pointed to by a PMD entry that doesn't have the accessed bit set.3.
mm/vmscan.c: refactorshrink_node()
A minor refactor.4. mm: multigenerational lru: groundwork
Adding the basic data structure and the functions to initialize it and insert/remove pages.5. mm: multigenerational lru:
mm_structlist
An infra keeps track of mm_struct's for page table walkers and provides them with optimizations, i.e.,switch_mm()tracking and Bloom filters.6. mm: multigenerational lru: aging
7. mm: multigenerational lru: eviction
"The page reclaim" is a producer/consumer model. "The aging" produces cold pages, whereas "the eviction " consumes them. Cold pages flow through generations. The aging uses themm_structlist infra to sweep dense hotspots in page tables. During a page table walk, the aging clears the accessed bit and tags accessed pages with the youngest generation number. The eviction sorts those pages when it encounters them. For pages in the oldest generation, eviction walks the rmap to check the accessed bit one more time before evicting them. During an rmap walk, the eviction feeds dense hotspots back to the aging. Dense hotspots flow through the Bloom filters. For pages not mapped in page tables, the eviction uses the PID controller to statistically determine whether they have higher refaults. If so, the eviction throttles their eviction by moving them to the next generation (the second oldest).8. mm: multigenerational lru: user interface
The knobs to turn on/off MGLRU and provide the userspace with thrashing prevention, working set estimation (the aging) and proactive reclaim (the eviction).9. mm: multigenerational lru:
Kconfig
TheKconfigoptions.Benchmark results
=================
Independent lab results
-----------------------
Based on the popularity of searches [01] and the memory usage in Google's public cloud, the most popular open-source memory-hungry applications, in alphabetical order, are:Apache Cassandra Memcached Apache Hadoop MongoDB Apache Spark PostgreSQL MariaDB (MySQL) RedisAn independent lab evaluated MGLRU with the most widely used benchmark suites for the above applications. They posted 960 data points along with kernel metrics and perf profiles collected over more than 500 hours of total benchmark time. Their final reports show that, with 95% confidence intervals (CIs), the above applications all performed significantly better for at least part of their benchmark matrices.
On 5.14:
1. Apache Spark [02] took 95% CIs [9.28, 11.19]% and [12.20, 14.93]% less wall time to sort three billion random integers, respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in wall time for the rest of the benchmark matrix.
2. MariaDB [03] achieved 95% CIs [5.24, 10.71]% and [20.22, 25.97]% more transactions per minute (TPM), respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
3. Memcached [04] achieved 95% CIs [23.54, 32.25]%, [20.76, 41.61]% and [21.59, 30.02]% more operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [13.85, 15.97]% and [23.94, 29.92]% more OPS, respectively, for random access and Gaussian access, when THP=never. There were no statistically significant changes in OPS for the rest of the benchmark matrix.
4. MongoDB [05] achieved 95% CIs [2.23, 3.44]%, [6.97, 9.73]% and [2.16, 3.55]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when underutilizing memory; 95% CIs [8.83, 10.03]%, [21.12, 23.14]% and [5.53, 6.46]% more OPS, respectively, for exponential access, random access and Zipfian access, when overcommitting memory.On 5.15:
5. Apache Cassandra [06] achieved 95% CIs [1.06, 4.10]%, [1.94, 5.43]% and [4.11, 7.50]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when swap was off; 95% CIs [0.50, 2.60]%, [6.51, 8.77]% and [3.29, 6.75]% more OPS, respectively, for exponential access, random access and Zipfian access, when swap was on.
6. Apache Hadoop [07] took 95% CIs [5.31, 9.69]% and [2.02, 7.86]% less average wall time to finish twelve parallel TeraSort jobs, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in average wall time for the rest of the benchmark matrix.
7. PostgreSQL [08] achieved 95% CI [1.75, 6.42]% more transactions per minute (TPM) under the high-concurrency condition, when swap was off; 95% CIs [12.82, 18.69]% and [22.70, 46.86]% more TPM, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
8. Redis [09] achieved 95% CIs [0.58, 5.94]%, [6.55, 14.58]% and [11.47, 19.36]% more total operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [1.27, 3.54]%, [10.11, 14.81]% and [8.75, 13.64]% more total OPS, respectively, for sequential access, random access and Gaussian access, when THP=never.Our lab results
---------------
To supplement the above results, we ran the following benchmark suites on 5.16-rc7 and found no regressions [10]. (These synthetic benchmarks are popular among MM developers, but we prefer large-scale A/B experiments to validate improvements.)fs_fio_bench_hdd_mq pft fs_lmbench pgsql-hammerdb fs_parallelio redis fs_postmark stream hackbench sysbenchthread kernbench tpcc_spark memcached unixbench multichase vm-scalability mutilate will-it-scalenginx
[01] https://trends.google.com
[02] https://lore.kernel.org/linux-mm/[email protected]/
[03] https://lore.kernel.org/linux-mm/[email protected]/
[04] https://lore.kernel.org/linux-mm/[email protected]/
[05] https://lore.kernel.org/linux-mm/[email protected]/
[06] https://lore.kernel.org/linux-mm/[email protected]/
[07] https://lore.kernel.org/linux-mm/[email protected]/
[08] https://lore.kernel.org/linux-mm/[email protected]/
[09] https://lore.kernel.org/linux-mm/[email protected]/
[10] https://lore.kernel.org/linux-mm/[email protected]/Read-world applications
=======================
Third-party testimonials
------------------------
Konstantin wrote [11]:
I have Archlinux with 8G RAM + zswap + swap. While developing, I have lots of apps opened such as multiple LSP-servers for different langs, chats, two browsers, etc... Usually, my system gets quickly to a point of SWAP-storms, where I have to kill LSP-servers, restart browsers to free memory, etc, otherwise the system lags heavily and is barely usable.1.5 day ago I migrated from 5.11.15 kernel to 5.12 + the LRU patchset, and I started up by opening lots of apps to create memory pressure, and worked for a day like this. Till now I had *not a single SWAP-storm*, and mind you I got 3.4G in SWAP. I was never getting to the point of 3G in SWAP before without a single SWAP-storm.
The Arch Linux Zen kernel [12] has been using MGLRU since 5.12. Many of its users reported their positive experiences to me, e.g., Shivodit wrote:
I've tried the latest Zen kernel (5.14.13-zen1-1-zen in the archlinux testing repos), everything's been smooth so far. I also decided to copy a large volume of files to check performance under I/O load, and everything went smoothly - no stuttering was present, everything was responsive.Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition to improvements in other UX metrics, e.g., an 85% decrease in the number of low-memory kills at the 75th percentile and an 18% decrease in rendering latency at the 50th percentile.[11] https://lore.kernel.org/linux-mm/[email protected]/
[12] https://github.com/zen-kernel/zen-kernel/
[13] https://research.google/pubs/pub44271/Summery
=======
The facts are:
1. The independent lab results and the real-world applications indicate substantial improvements; there are no known regressions.
2. Thrashing prevention, working set estimation and proactive reclaim work out of the box; there are no equivalent solutions.
3. There is a lot of new code; nobody has demonstrated smaller changes with similar effects.Our options, accordingly, are:
1. Given the amount of evidence, the reported improvements will likely materialize for a wide range of workloads.
2. Gauging the interest from the past discussions [14][15][16], the new features will likely be put to use for both personal computers and data centers.
3. Based on Google's track record, the new code will likely be well maintained in the long term. It'd be more difficult if not impossible to achieve similar effects on top of the existing design.[14] https://lore.kernel.org/lkml/[email protected]/
[15] https://lore.kernel.org/lkml/[email protected]/
[16] https://lore.kernel.org/lkml/[email protected]/Yu Zhao (9):
mm: x86, arm64: addarch_has_hw_pte_young()
mm: x86: addCONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
mm/vmscan.c: refactorshrink_node()
mm: multigenerational lru: groundwork
mm: multigenerational lru:mm_structlist
mm: multigenerational lru: aging
mm: multigenerational lru: eviction
mm: multigenerational lru: user interface
mm: multigenerational lru:KconfigDocumentation/vm/index.rst | 1 + Documentation/vm/multigen_lru.rst | 80 + arch/Kconfig | 9 + arch/arm64/include/asm/cpufeature.h | 5 + arch/arm64/include/asm/pgtable.h | 13 +- arch/arm64/kernel/cpufeature.c | 19 + arch/arm64/tools/cpucaps | 1 + arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 9 +- arch/x86/mm/pgtable.c | 5 +- fs/exec.c | 2 + fs/fuse/dev.c | 3 +- include/linux/cgroup.h | 15 +- include/linux/memcontrol.h | 11 + include/linux/mm.h | 42 + include/linux/mm_inline.h | 204 ++ include/linux/mm_types.h | 78 + include/linux/mmzone.h | 175 ++ include/linux/nodemask.h | 1 + include/linux/oom.h | 16 + include/linux/page-flags-layout.h | 19 +- include/linux/page-flags.h | 4 +- include/linux/pgtable.h | 17 +- include/linux/sched.h | 4 + include/linux/swap.h | 4 + kernel/bounds.c | 3 + kernel/cgroup/cgroup-internal.h | 1 - kernel/exit.c | 1 + kernel/fork.c | 9 + kernel/sched/core.c | 1 + mm/Kconfig | 48 + mm/huge_memory.c | 3 +- mm/memcontrol.c | 26 + mm/memory.c | 21 +- mm/mm_init.c | 6 +-
Quoted 8 lines
mm/oom_kill.c | 4 +- mm/page_alloc.c | 1 + mm/rmap.c | 7 + mm/swap.c | 51 +- mm/vmscan.c | 2691 ++++++++++++++++++++++++++- mm/workingset.c | 119 +- 41 files changed, 3591 insertions(+), 139 deletions(-) create mode 100644 Documentation/vm/multigen_lru.rst
For the series:
I run this (and one of the previous spins) on nine machines (physical, virtual, workstations, servers) for quite some time with no hassle.
Thanks for your job, and please keep me in Cc once you post new spins. I'm more than happy to deploy those across the fleet.
Signature · 1 line
Oleksandr Natalenko (post-factum)
Quoted 9 lines · Oleksandr Natalenko
Hello.
Quoted 6 lines · On úterý 4. ledna 2022 21:22:19 CET Yu Zhao wrote:
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.
<snipped>
Quoted 7 lines
For the series:
Tested-by Oleksandr Natalenko <[email protected]>I run this (and one of the previous spins) on nine machines (physical, virtual, workstations, servers) for quite some time with no hassle.
Thanks for your job, and please keep me in Cc once you post new spins. I'm more than happy to deploy those across the fleet.
Thanks, Oleksandr. And if I may take the liberty of introducing you as:
Oleksandr - the post-factum kernel maintainer
https://gitlab.com/post-factum/pf-kernel
in addition to other downstream kernel maintainers I've introduced:
Quoted 14 lines
Alexandre - the XanMod kernel maintainer
https://xanmod.orgBrian - the Chrome OS kernel memory maintainer
https://www.chromium.orgJan - the Arch Linux Zen kernel maintainer
https://archlinux.orgSteven - the Liquorix kernel maintainer
https://liquorix.netSuleiman - the ARCVM (Android downstream) kernel memory maintainer
https://chromium.googlesource.com/chromiumos/third_party/kernel
Quoted 221 lines · Yu Zhao
TLDR
====
The current page reclaim is too expensive in terms of CPU usage and it often makes poor choices about what to evict. This patchset offers an alternative solution that is performant, versatile and straightforward.Design objectives
=================
The design objectives are:
1. Better representation of access recency
2. Try to profit from spatial locality
3. Clear fast path making obvious choices
4. Simple self-correcting heuristicsThe representation of access recency is at the core of all LRU approximations. The multigenerational LRU (MGLRU) divides pages into multiple lists (generations), each having bounded access recency (a time interval). Generations establish a common frame of reference and help make better choices, e.g., between different memcgs on a computer or different computers in a data center (for cluster job scheduling).
Exploiting spatial locality improves the efficiency when gathering the accessed bit. A rmap walk targets a single page and doesn't try to profit from discovering an accessed PTE. A page table walk can sweep all hotspots in an address space, but its search space can be too large to make a profit. The key is to optimize both methods and use them in combination. (PMU is another option for further exploration.)
Fast path reduces code complexity and runtime overhead. Unmapped pages don't require TLB flushes; clean pages don't require writeback. These facts are only helpful when other conditions, e.g., access recency, are similar. With generations as a common frame of reference, additional factors stand out. But obvious choices might not be good choices; thus self-correction is required (the next objective).
The benefits of simple self-correcting heuristics are self-evident. Again with generations as a common frame of reference, this becomes attainable. Specifically, pages in the same generation are categorized based on additional factors, and a closed-loop control statistically compares the refault percentages across all categories and throttles the eviction of those that have higher percentages.
Patchset overview
=================
1. mm: x86, arm64: addarch_has_hw_pte_young()
2. mm: x86: addCONFIG_ARCH_HAS_NONLEAF_PMD_YOUNGMaterializing hardware optimizations when trying to clear the accessed bit in many PTEs. If hardware automatically sets the accessed bit in PTEs, there is no need to worry about bursty page faults (emulating the accessed bit). If it also sets the accessed bit in non-leaf PMD entries, there is no need to search the PTE table pointed to by a PMD entry that doesn't have the accessed bit set.3.
mm/vmscan.c: refactorshrink_node()
A minor refactor.4. mm: multigenerational lru: groundwork
Adding the basic data structure and the functions to initialize it and insert/remove pages.5. mm: multigenerational lru:
mm_structlist
An infra keeps track of mm_struct's for page table walkers and provides them with optimizations, i.e.,switch_mm()tracking and Bloom filters.6. mm: multigenerational lru: aging
7. mm: multigenerational lru: eviction
"The page reclaim" is a producer/consumer model. "The aging" produces cold pages, whereas "the eviction " consumes them. Cold pages flow through generations. The aging uses themm_structlist infra to sweep dense hotspots in page tables. During a page table walk, the aging clears the accessed bit and tags accessed pages with the youngest generation number. The eviction sorts those pages when it encounters them. For pages in the oldest generation, eviction walks the rmap to check the accessed bit one more time before evicting them. During an rmap walk, the eviction feeds dense hotspots back to the aging. Dense hotspots flow through the Bloom filters. For pages not mapped in page tables, the eviction uses the PID controller to statistically determine whether they have higher refaults. If so, the eviction throttles their eviction by moving them to the next generation (the second oldest).8. mm: multigenerational lru: user interface
The knobs to turn on/off MGLRU and provide the userspace with thrashing prevention, working set estimation (the aging) and proactive reclaim (the eviction).9. mm: multigenerational lru:
Kconfig
TheKconfigoptions.Benchmark results
=================
Independent lab results
-----------------------
Based on the popularity of searches [01] and the memory usage in Google's public cloud, the most popular open-source memory-hungry applications, in alphabetical order, are:Apache Cassandra Memcached Apache Hadoop MongoDB Apache Spark PostgreSQL MariaDB (MySQL) RedisAn independent lab evaluated MGLRU with the most widely used benchmark suites for the above applications. They posted 960 data points along with kernel metrics and perf profiles collected over more than 500 hours of total benchmark time. Their final reports show that, with 95% confidence intervals (CIs), the above applications all performed significantly better for at least part of their benchmark matrices.
On 5.14:
1. Apache Spark [02] took 95% CIs [9.28, 11.19]% and [12.20, 14.93]% less wall time to sort three billion random integers, respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in wall time for the rest of the benchmark matrix.
2. MariaDB [03] achieved 95% CIs [5.24, 10.71]% and [20.22, 25.97]% more transactions per minute (TPM), respectively, under the medium- and the high-concurrency conditions, when overcommitting memory. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
3. Memcached [04] achieved 95% CIs [23.54, 32.25]%, [20.76, 41.61]% and [21.59, 30.02]% more operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [13.85, 15.97]% and [23.94, 29.92]% more OPS, respectively, for random access and Gaussian access, when THP=never. There were no statistically significant changes in OPS for the rest of the benchmark matrix.
4. MongoDB [05] achieved 95% CIs [2.23, 3.44]%, [6.97, 9.73]% and [2.16, 3.55]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when underutilizing memory; 95% CIs [8.83, 10.03]%, [21.12, 23.14]% and [5.53, 6.46]% more OPS, respectively, for exponential access, random access and Zipfian access, when overcommitting memory.On 5.15:
5. Apache Cassandra [06] achieved 95% CIs [1.06, 4.10]%, [1.94, 5.43]% and [4.11, 7.50]% more operations per second (OPS), respectively, for exponential (distribution) access, random access and Zipfian (distribution) access, when swap was off; 95% CIs [0.50, 2.60]%, [6.51, 8.77]% and [3.29, 6.75]% more OPS, respectively, for exponential access, random access and Zipfian access, when swap was on.
6. Apache Hadoop [07] took 95% CIs [5.31, 9.69]% and [2.02, 7.86]% less average wall time to finish twelve parallel TeraSort jobs, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in average wall time for the rest of the benchmark matrix.
7. PostgreSQL [08] achieved 95% CI [1.75, 6.42]% more transactions per minute (TPM) under the high-concurrency condition, when swap was off; 95% CIs [12.82, 18.69]% and [22.70, 46.86]% more TPM, respectively, under the medium- and the high-concurrency conditions, when swap was on. There were no statistically significant changes in TPM for the rest of the benchmark matrix.
8. Redis [09] achieved 95% CIs [0.58, 5.94]%, [6.55, 14.58]% and [11.47, 19.36]% more total operations per second (OPS), respectively, for sequential access, random access and Gaussian (distribution) access, when THP=always; 95% CIs [1.27, 3.54]%, [10.11, 14.81]% and [8.75, 13.64]% more total OPS, respectively, for sequential access, random access and Gaussian access, when THP=never.Our lab results
---------------
To supplement the above results, we ran the following benchmark suites on 5.16-rc7 and found no regressions [10]. (These synthetic benchmarks are popular among MM developers, but we prefer large-scale A/B experiments to validate improvements.)fs_fio_bench_hdd_mq pft fs_lmbench pgsql-hammerdb fs_parallelio redis fs_postmark stream hackbench sysbenchthread kernbench tpcc_spark memcached unixbench multichase vm-scalability mutilate will-it-scalenginx
[01] https://trends.google.com
[02] https://lore.kernel.org/linux-mm/[email protected]/
[03] https://lore.kernel.org/linux-mm/[email protected]/
[04] https://lore.kernel.org/linux-mm/[email protected]/
[05] https://lore.kernel.org/linux-mm/[email protected]/
[06] https://lore.kernel.org/linux-mm/[email protected]/
[07] https://lore.kernel.org/linux-mm/[email protected]/
[08] https://lore.kernel.org/linux-mm/[email protected]/
[09] https://lore.kernel.org/linux-mm/[email protected]/
[10] https://lore.kernel.org/linux-mm/[email protected]/Read-world applications
=======================
Third-party testimonials
------------------------
Konstantin wrote [11]:
I have Archlinux with 8G RAM + zswap + swap. While developing, I have lots of apps opened such as multiple LSP-servers for different langs, chats, two browsers, etc... Usually, my system gets quickly to a point of SWAP-storms, where I have to kill LSP-servers, restart browsers to free memory, etc, otherwise the system lags heavily and is barely usable.1.5 day ago I migrated from 5.11.15 kernel to 5.12 + the LRU patchset, and I started up by opening lots of apps to create memory pressure, and worked for a day like this. Till now I had *not a single SWAP-storm*, and mind you I got 3.4G in SWAP. I was never getting to the point of 3G in SWAP before without a single SWAP-storm.
The Arch Linux Zen kernel [12] has been using MGLRU since 5.12. Many of its users reported their positive experiences to me, e.g., Shivodit wrote:
I've tried the latest Zen kernel (5.14.13-zen1-1-zen in the archlinux testing repos), everything's been smooth so far. I also decided to copy a large volume of files to check performance under I/O load, and everything went smoothly - no stuttering was present, everything was responsive.
Quoted 5 lines
Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition to
Hi Yu,
Was the overall 40% decrease of kswap CPU usgae seen on x86 or arm64? And I am curious how much we are taking advantage of NONLEAF_PMD_YOUNG. Does it help a lot in decreasing the cpu usage? If so, this might be a good proof that arm64 also needs this hardware feature?
In short, I am curious how much the improvement in this patchset depends on the hardware ability of NONLEAF_PMD_YOUNG.
Quoted 8 lines
improvements in other UX metrics, e.g., an 85% decrease in the number of low-memory kills at the 75th percentile and an 18% decrease in rendering latency at the 50th percentile.
[11] https://lore.kernel.org/linux-mm/[email protected]/
[12] https://github.com/zen-kernel/zen-kernel/
[13] https://research.google/pubs/pub44271/
Thanks
Barry
Quoted 1 line · Barry Song
On Wed, Jan 5, 2022 at 7:17 PM Yu Zhao <[email protected]> wrote:
<snipped>
Quoted 11 lines
Quoted 5 lines
Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition toHi Yu,
Was the overall 40% decrease of kswap CPU usgae seen on x86 or arm64? And I am curious how much we are taking advantage of
NONLEAF_PMD_YOUNG. Does it help a lot in decreasing the cpu usage?
Hi Barry,
The fleet-wide profiling data I shared was from x86. For arm64, I only have data from synthetic benchmarks at the moment, and it also shows similar improvements.
For Chrome OS (individual users), walk_pte_range(), the function that would benefit from ARCH_HAS_NONLEAF_PMD_YOUNG, only uses a small portion (<4%) of kswapd CPU time. So ARCH_HAS_NONLEAF_PMD_YOUNG isn't that helpful.
Quoted 4 lines
If so, this might be
a good proof that arm64 also needs this hardware feature?
In short, I am curious how much the improvement in this patchset depends on the hardware ability ofNONLEAF_PMD_YOUNG.
For data centers, I do think ARCH_HAS_NONLEAF_PMD_YOUNG has some value. In addition to cold/hot memory scanning, there are other use cases like dirty tracking, which can benefit from the accessed bit on non-leaf entries. I know some proprietary software uses this capability on x86 for different purposes than this patchset does. And AFAIK, x86 is the only arch that supports this capability, e.g., risc-v and ppc can only set the accessed bit in PTEs.
In fact, I've discussed this with one of the arm maintainers Will. So please check with him too if you are interested in moving forward with the idea. I might be able to provide with additional data if you need it to make a decision.
Thanks.
Quoted 20 lines · Yu Zhao
Quoted 1 line · Barry Song
On Wed, Jan 5, 2022 at 7:17 PM Yu Zhao <[email protected]> wrote:
<snipped>
Quoted 11 lines
Quoted 5 lines
Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition toHi Yu,
Was the overall 40% decrease of kswap CPU usgae seen on x86 or arm64? And I am curious how much we are taking advantage of
NONLEAF_PMD_YOUNG. Does it help a lot in decreasing the cpu usage?Hi Barry,
Quoted 8 lines
The fleet-wide profiling data I shared was from x86. For arm64, I only have data from synthetic benchmarks at the moment, and it also shows similar improvements.
For Chrome OS (individual users),
walk_pte_range(), the function that would benefit fromARCH_HAS_NONLEAF_PMD_YOUNG, only uses a small portion (<4%) of kswapd CPU time. SoARCH_HAS_NONLEAF_PMD_YOUNGisn't that helpful.
Hi Yu,
Thanks!
In the current kernel, depending on reverse mapping, while memory is under pressure,
the cpu usage of kswapd can be very very high especially while a lot of pages have large mapcount, thus a huge reverse mapping cost.
Regarding <4%, I guess the figure came from machines with NONLEAF_PMD_YOUNG? In this case, we can skip many PTE scans while PMD has no accessed bit set. But for
a machine without NONLEAF, will the figure of cpu usage be much larger?
Quoted 6 lines
Quoted 5 lines
If so, this might be
a good proof that arm64 also needs this hardware feature?
In short, I am curious how much the improvement in this patchset depends on the hardware ability ofNONLEAF_PMD_YOUNG.
Quoted 7 lines
For data centers, I do think
ARCH_HAS_NONLEAF_PMD_YOUNGhas some value. In addition to cold/hot memory scanning, there are other use cases like dirty tracking, which can benefit from the accessed bit on non-leaf entries. I know some proprietary software uses this capability on x86 for different purposes than this patchset does. And AFAIK, x86 is the only arch that supports this capability, e.g., risc-v and ppc can only set the accessed bit in PTEs.
Yep. NONLEAF is a nice feature.
btw, page table should have a separate DIRTY bit, right? wouldn't dirty page tracking depend on the DIRTY bit rather than the accessed bit? so x86 also has NONLEAF dirty bit? Or they are scanning accessed bit of PMD before scanning DIRTY bits of PTEs?
Quoted 5 lines
In fact, I've discussed this with one of the arm maintainers Will. So please check with him too if you are interested in moving forward with the idea. I might be able to provide with additional data if you need it to make a decision.
I am interested in running it and have some data without NONLEAF especially while free memory is very limited and the system has memory thrashing.
Quoted 2 lines
Thanks.
Thanks
Barry
Quoted 30 lines · Barry Song
Quoted 20 lines · Yu Zhao
Quoted 1 line · Barry Song
On Wed, Jan 5, 2022 at 7:17 PM Yu Zhao <[email protected]> wrote:
<snipped>
Quoted 11 lines
Quoted 5 lines
Large-scale deployments
-----------------------
We've rolled out MGLRU to tens of millions of Chrome OS users and about a million Android users. Google's fleetwide profiling [13] shows an overall 40% decrease in kswapd CPU usage, in addition toHi Yu,
Was the overall 40% decrease of kswap CPU usgae seen on x86 or arm64? And I am curious how much we are taking advantage of
NONLEAF_PMD_YOUNG. Does it help a lot in decreasing the cpu usage?Hi Barry,
Quoted 8 lines
The fleet-wide profiling data I shared was from x86. For arm64, I only have data from synthetic benchmarks at the moment, and it also shows similar improvements.
For Chrome OS (individual users),
walk_pte_range(), the function that would benefit fromARCH_HAS_NONLEAF_PMD_YOUNG, only uses a small portion (<4%) of kswapd CPU time. SoARCH_HAS_NONLEAF_PMD_YOUNGisn't that helpful.
Quoted 7 lines
Hi Yu,
Thanks!In the current kernel, depending on reverse mapping, while memory is under pressure,
the cpu usage of kswapd can be very very high especially while a lot of pages have large mapcount, thus a huge reverse mapping cost.
Agreed. I've posted v7 which includes kswapd profiles collected from an arm64 v8.2 laptop under memory pressure.
Quoted 1 line
Regarding <4%, I guess the figure came from machines with NONLEAF_PMD_YOUNG?
No, it's from Snapdragon 7c. Please see the kswapd profiles in v7.
Quoted 3 lines
In this case, we can skip many PTE scans while PMD has no accessed bit set. But for
a machine without NONLEAF, will the figure of cpu usage be much larger?
So NONLEAF_PMD_YOUNG at most can save 4% CPU usage from kswapd. But this definitely can vary, depending on the workloads.
Quoted 13 lines
Quoted 5 lines
Quoted 4 lines
If so, this might be
a good proof that arm64 also needs this hardware feature?
In short, I am curious how much the improvement in this patchset depends on the hardware ability ofNONLEAF_PMD_YOUNG.Quoted 7 lines
For data centers, I do think
ARCH_HAS_NONLEAF_PMD_YOUNGhas some value. In addition to cold/hot memory scanning, there are other use cases like dirty tracking, which can benefit from the accessed bit on non-leaf entries. I know some proprietary software uses this capability on x86 for different purposes than this patchset does. And AFAIK, x86 is the only arch that supports this capability, e.g., risc-v and ppc can only set the accessed bit in PTEs.
Quoted 3 lines
Yep. NONLEAF is a nice feature.
btw, page table should have a separate DIRTY bit, right?
Yes.
Quoted 2 lines
wouldn't dirty page
tracking depend on the DIRTY bit rather than the accessed bit?
It depends on the goal.
Quoted 2 lines
so x86 also has
NONLEAF dirty bit?
No.
Quoted 2 lines
Or they are scanning accessed bit of PMD before
scanning DIRTY bits of PTEs?
A mandatory sync to disk must use the dirty bit to ensure data integrity. But for a voluntary sync to disk, it can use the accessed bit to narrow the search of dirty pages.
A mandatory sync is used to free specific dirty pages. A voluntary sync is used to keep the number of dirty pages low in general and it doesn't target any specific dirty pages.
Quoted 8 lines
Quoted 4 lines
In fact, I've discussed this with one of the arm maintainers Will. So please check with him too if you are interested in moving forward with the idea. I might be able to provide with additional data if you need it to make a decision.
I am interested in running it and have some data without NONLEAF especially while free memory is very limited and the system has memory thrashing.
The v7 has a switch to disable this feature on x86. If you can run your workloads on x86, then it might be able to help you measure the difference.