mm, swap: remove swap slot cache
Status from this message: [merged mm-stable] mm-swap-simplify-folio-swap-allocation.patch removed from -mm tree
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest7 patches 4 repliesApplied 7/7
- v27 patches 6 repliesSuperseded
- v17 patches 19 repliesSuperseded
Patches in v3 All 7 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/7 | mm, swap: avoid reclaiming irrelevant swap cachemm/ | +10 | 0 replies | Applied |
| 2/7 | mm, swap: drop the flag TTRS_DIRECTmm/ | +3 | 0 replies | Applied |
| 3/7 | mm, swap: avoid redundant swap device pinningmm/ mm/ | +14 | 0 replies | Applied |
| 4/7 | mm, swap: don't update the counter up-frontmm/ | +2 | 0 replies | Applied |
| 5/7 | mm, swap: use percpu cluster as allocation fast pathinclude/ mm/ | +121 | 0 replies | Applied |
| 6/7 | mm, swap: remove swap slot cacheinclude/ include/ mm/ +3 more | +67 | 4 replies | Applied |
| 7/7 | mm, swap: simplify folio swap allocationinclude/ mm/ mm/ +3 more | +95 | 0 replies | Applied |
Reviews trailers per patch
5 of 7 patches have a Reviewed-by or Acked-by.
| Patch | Baoquan He |
|---|---|
| 1/7 | R-b |
| 2/7 | R-b |
| 3/7 | R-b |
| 4/7 | R-b |
| 5/7 | |
| 6/7 | R-b |
| 7/7 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v3 0/7] mm, swap: remove swap slot cache
From: Kairui Song <[email protected]>
Slot cache was initially introduced by commit 67afa38e012e ("mm/swap:
add cache for swap slots allocation") to reduce the lock contention of si->lock.
Previous series "mm, swap: rework of swap allocator locks" [1] removed swap slot cache for freeing path as freeing path no longer touches si->lock in most cased. Allocation path also have slight to none contention on si->lock since that series, but slot cache still helps to reduce other overheads, like counters and the plist.
This series removes the slot cache from allocation path too, by using the cluster as allocation fast path and also reduce other overheads.
Now slot cache is completely gone, the code is much simplified without obvious feature or performance change, also clean up related workaround. Also this should avoid other potential issues, e.g. the long pinning of swap slots: swap slot cache pins swap slots with HAS_CACHE, causing reclaim or allocation fail to use these slots on scanning.
The only behavior change is the swap device allocation rotation mechanism, as explained in the patch "mm, swap: use percpu cluster as allocation fast path".
Test results are looking good after deleting the swap slot cache:
- vm-scalability with: usemem --init-time -O -y -x -R -31 1G, 12G memory cgroup using simulated pmem as SWAP (32G pmem, 32 CPUs), 16 test runs for each case, measuring the total throughput:
Before (KB/s) (stdev) After (KB/s) (stdev) Random (4K): 424907.60 (24410.78) 414745.92 (34554.78) Random (64K): 163308.82 (11635.72) 167314.50 (18434.99)
Sequential (4K, !-R): 6150056.79 (103205.90) 6321469.06 (115878.16)
- Build linux kernel with make -j96, using 4K folio with 1.5G memory cgroup limit and 64K folio with 2G memory cgroup limit, on top of tmpfs, 12 test runs, measuring the system time:
Before (s) (stdev) After (s) (stdev) make -j96 (4K): 6445.69 (61.95) 6408.80 (69.46) make -j96 (64K): 6841.71 (409.04) 6437.99 (435.55)
The performance is unchanged, slightly better in some cases.
[1] https://lore.kernel.org/linux-mm/[email protected]/
---
V2: https://lore.kernel.org/linux-mm/[email protected]/
Updates from V2:
- Make folio_alloc_swap() inline to fix build error [Stephen Rothwell]
- Flush the global percpu cluster cache on swapoff to prevent new swapon devices using the old invalid values. Based on:
https://lore.kernel.org/linux-mm/CAMgjq7AkRmb5ote-VZErM_2UdEC575j9WcrstcQOypEb+T-DLA@mail.gmail.com/
- Minor update for patch 5/7: in slow path also try the local cluster first to avoid fragmentation. It's a intermediate patch change for easier testing if someone run into a bisect, the final code after the whole series applies is not changed.
- Need to call mem_cgroup_try_charge_swap even if swap allocation failed for cgroup events.
- Collect reviews and minor improvements [Baoquan He].
V1: https://lore.kernel.org/linux-mm/[email protected]/
Updates from V1:
- Check the cluster with cluster_is_usable and cluster_is_empty in fast path too, improve performance and avoid fragmentation.
- Fix a build warning and error for !SWAP build reported by test bot.
- Global cluster array also record device for each order [Baoquan He]
- Adjust of comments and function name [Baoquan He]
- Collect Review-by [Baoquan He]
- Minor function style improvement [Matthew Wilcox]
Kairui Song (7):
mm, swap: avoid reclaiming irrelevant swap cache mm, swap: drop the flag TTRS_DIRECT
mm, swap: avoid redundant swap device pinning mm, swap: don't update the counter up-front mm, swap: use percpu cluster as allocation fast path mm, swap: remove swap slot cache
mm, swap: simplify folio swap allocation
include/linux/swap.h | 22 +-- include/linux/swap_slots.h | 28 --- mm/Makefile | 2 +- mm/shmem.c | 21 +-- mm/swap.h | 6 - mm/swap_slots.c | 295 ------------------------------ mm/swap_state.c | 79 +-------- mm/swapfile.c | 355 ++++++++++++++++++++----------------- mm/vmscan.c | 16 +- mm/zswap.c | 6 + 10 files changed, 232 insertions(+), 598 deletions(-) delete mode 100644 include/linux/swap_slots.h delete mode 100644 mm/swap_slots.c
Signature · 1 line
2.48.1