mm/swap, memcg: Introduce swap tiers for cgroup based swap control
Subsystems (7)
Youngjun Park
reviewerv8 cover mm/swap, memcg: Introduce swap tiers for cgroup based swap control
This is the v8 series of the swap tier patchset.
Great thanks to Shakeel Butt and Yosry for the reviews and discussions [1]. The main change in this version is the interface change to use memory.swap.tiers.max with '0' (disable) and 'max' (enable) values. This mechanism was suggested by Shakeel and Yosry.
This change allows for future extensions to control swap between tiers and aligns better with existing memcg interfaces. Even with this memcg interface change, only patch #3 needed updates. Internally, patch #3 still uses the existing mask processing method (which is implementation-efficient), so only the user-facing interface was modified.
We also discussed tier extensions. Thanks to Yosry, Nhat and Shakeel for their valuable feedback.
Here is a brief summary of our tentative conclusions. Please correct me if anything is misrepresented (details in references):
* Zswap tiering [2]:
Tiering applies only to the vswap + zswap combo. Zswap itself will not be tiered, as the current architecture requires a physical device for zswap allocation.
* Vswap tiering [3]:
Vswap should be handled transparently to the user. Vswap itself will not be tiered. But, someday supported if there is strong and real usecase.
* Relationship with zswap.writeback [4]:
If zswap tiering is introduced, it could replace the zswap-only tier. However, since zswap cannot be tiered independently, it is still needed for non-vswap cases. Separately, the internal logic could potentially be integrated into the tiering logic.
* Tier demotion [5]:
A separate interface like memory.swap.tiers.demotion might be needed. For now, we only support 0/max to enable/disable tiers. In the future, we could introduce an "auto" mode to automatically scale the limit based on swapfile size and memory.swap.max, similar to the direction memory tiering is heading in.
I plan to apply the swap tier infrastructure and the first use case (cgroup-based swap control) first, and continue following up on the discussions above.
Overview
========
Swap Tiers group swap devices into performance classes (e.g. NVMe, HDD, Network) and allow per-memcg selection of which tiers to use. This mechanism was suggested by Chris Li.
Design Rationale
================
Swap tier selection is attached to memcg. A child cgroup may select a subset of the parent's allowed tiers.
This
- Preserves cgroup inheritance semantics (boundary at parent, refinement at child).
- Reuses memcg, which already groups processes and enforces hierarchical memory limits.
- Aligns with existing memcg swap controls (e.g. swap.max, zswap.writeback)
- Avoids introducing a parallel swap control hierarchy.
Placing tier control outside memcg (e.g., via BPF, syscalls, or madvise) would allow swap preference to diverge from the memcg hierarchy. Integrating it into memcg keeps the swap policy consistent with existing memory ownership semantics. There are also real use cases built around memcg.
In the future, this can be extended to other interfaces to cover additional use cases.
I believe a memcg-based swap control is a good starting point before such extensions.
Use Cases
=========
#1: Latency separation (our primary deployment scenario)
[ / ]
|
+-- latency-sensitive workload (fast tier)
+-- background workload (slow tier)
The parent defines the memory boundary.
Each workload selects a swap tier via memory.swap.tiers.max according to latency requirements.
This prevents latency-sensitive workloads from being swapped to slow devices used by background workloads.
#2: Per-VM swap selection (Chris Li's deployment scenario)
[ / ]
|
+-- [ Job on VM ] (tiers: zswap, SSD)
|
+-- [ VMM guest memory ] (tiers: SSD)
The parent (job) has access to both zswap and SSD tiers. The child (VMM guest memory) selects SSD as its swap tier via memory.swap.tiers.max. In this deployment, swap device selection happens at the child level from the parent's available set.
#3: Tier isolation for reduced contention (hypothetical)
[ / ] (tiers: A, B)
|
+-- workload X (tiers: A)
+-- workload Y (tiers: B)
Each child uses a different tier. Since swap paths are separated per tier, synchronization overhead between the two workloads is reduced.
Future extension
================
#1: Intra-tier distribution policy:
Currently, swap devices with the same priority are allocated in a round-robin fashion. Per-tier policy files under /sys/kernel/mm/swap/tiers/ can control how devices within a tier are selected (e.g. round-robin, weighted).
#2: Inter-tier promotion and demotion:
Promotion and demotion apply between tiers, not within a single tier. The current interface defines only tier assignment; it does not yet define when or how pages move between tiers. Two triggering models are possible:
(a) User-triggered: userspace explicitly initiates migration between tiers (e.g. via a new interface or existing move_pages semantics).
(b) Kernel-triggered: the kernel moves pages between tiers at appropriate points such as reclaim or refault.
#3: Per-VMA, per-process swap and BPF:
Not just for memcg based swap, possible to extend Per-VMA or per-process swap. Or we can use it as BPF program.
#4: Zswap and vswap tiering:
Tiering applies to the vswap + zswap combination.
#5: Vswap on/off control:
Currently not supported. If a strong use case arises where vswap needs to be controlled by memcg, the tier interface could be used for it.
Experimentation
===============
Tested on our internal platform using NBD as a separate swap tier. Our first production's simple usecase.
Without tiers:
- No selective control over flash wear
- Cannot selectively assign NBD to specific applications
Cold launch improvement (preloaded vs. baseline):
- App A: 13.17s -> 4.18s (68%)
- App B: 5.60s -> 1.12s (80%)
- App C: 10.25s -> 2.00s (80%)
Performance impact with no tiers configured:
<1% regression in kernel build and vm-scalability benchmarks
Change log
===========
v8
- Changed the memcg interface to memory.swap.tiers.max.
Values are '0' (disable) and 'max' (enable). Default is 'max'.
- Addressed Sashiko's review: Update the mask value atomically at once and read the mask value while grabbing lock.
- Collected review tags from Kairui and Nhat.
- Rebase on recent mm-new
- v7 link: https://lore.kernel.org/linux-mm/[email protected]/
v7
- Collect Baoquan's review tag
- Baoquan's feedback on fixing improper comment
- Minor code adjustments per Baoquan's feedback.
- Rebase on recent mm-new
- v6 link: https://lore.kernel.org/linux-mm/[email protected]/
v6
- Sashiko AI review fixes
- Fix batch parsing error path to restore snapshot before exit
- Reject overlong tier names to prevent truncated duplicates
- Avoid restoring raw list_head via memcpy (stale pointer risk)
- Ensure early parse errors do not skip DEF_SWAP_PRIO validation
- Use (1U << TIER_DEFAULT_IDX) to avoid signed shift UB
- Defer tier mask inheritance to css_online() to close race window
- Add READ_ONCE()/WRITE_ONCE() for tier mask accesses
- Other fixes
- Fix build error reintroduced due to missing v5 change
- Fix WARNING in folio_tier_effective_mask by adding rcu_read_lock()
- default number of swap tier max (change to 32->31, for reserving last bit)
- commit message refinement.
- rebased on recently mm-new
- v5 link: https://lore.kernel.org/linux-mm/[email protected]/
v5
- Fixed build errors reported in v4
- rebased on up to date mm-new
- Minor cleanups
- Design docs with validation (by Shakeel Butt discussion)
- v4 link : https://lore.kernel.org/linux-mm/[email protected]/
v4
- Simplified control flow and indentation
- Added CONFIG option for MAX_SWAPTIER (default: 4)
- Added memory.swap.tiers.effective interface
- Reworked save/restore logic into snapshot/rollback model
- Removed tier priority modification support (deferred)
- Improved validation and fixed edge cases
- Rebased onto latest mm-new
- RFC v3 link: https://lore.kernel.org/linux-mm/[email protected]/
RFC v1 ~ v3
- Change the direction after discussion with Chris-Li
- apply some LPC feedback.
- RFC v2 - https://lore.kernel.org/linux-mm/[email protected]/
- RFC v1 - https://lore.kernel.org/linux-mm/[email protected]/
Earlier Approach (per cgroup swap priority)
- v1: https://lore.kernel.org/linux-mm/[email protected]/
- RFC: https://lore.kernel.org/linux-mm/aEvLjEInMQC7hEyh@yjaykim-PowerEdge-T330/T/#mbbb6a5e9e30843097e1f5f65fb98f31d582b973d
Reference
=========
[1] https://lore.kernel.org/linux-doc/[email protected]/
[2] https://lore.kernel.org/linux-mm/CAKEwX=Nz9SWcEVQGQjHN8P8OANJY4BG0w+iQOzoNOWuteoVjAg@mail.gmail.com/
[3] https://lore.kernel.org/cgroups/CAKEwX=O23a4iWBZoewKVb8QqODte6r3Xijckw3_oCJNoiO9M5A@mail.gmail.com/
[4] https://lore.kernel.org/linux-mm/CAO9r8zOg0OP1Ak1v7CRzSfQq0D8b4Dw+_T0Jui6YTM_KwQQNOA@mail.gmail.com/
[5] https://lore.kernel.org/linux-mm/CAO9r8zNi4-QC4sUi=xXWHt9WMeG39mbyoSf8kON9vLOZ=cbCmw@mail.gmail.com/
Youngjun Park (4):
mm: swap: introduce swap tier infrastructure
mm: swap: associate swap devices with tiers
mm: memcontrol: add interface for swap tier selection
mm: swap: filter swap allocation by memcg tier mask
Documentation/admin-guide/cgroup-v2.rst | 20 + Documentation/mm/index.rst | 1 + Documentation/mm/swap-tier.rst | 159 ++++++++ MAINTAINERS | 3 + include/linux/memcontrol.h | 5 + include/linux/swap.h | 1 + mm/Kconfig | 12 + mm/Makefile | 2 +- mm/memcontrol.c | 67 ++++ mm/swap.h | 4 + mm/swap_state.c | 75 ++++ mm/swap_tier.c | 483 ++++++++++++++++++++++++ mm/swap_tier.h | 76 ++++ mm/swapfile.c | 20 +- 14 files changed, 923 insertions(+), 5 deletions(-) create mode 100644 Documentation/mm/swap-tier.rst create mode 100644 mm/swap_tier.c create mode 100644 mm/swap_tier.h
base-commit: 9d335aed8840f6bf83ba93309ae5e185de829c21
Signature · 1 line
2.34.1
Youngjun Park
reviewerPATCH v8 1/4 mm: swap: introduce swap tier infrastructure
This patch introduces the "Swap tier" concept, which serves as an abstraction layer for managing swap devices based on their performance characteristics (e.g., NVMe, HDD, Network swap).
Swap tiers are user-named groups representing priority ranges. Tier names must consist of alphanumeric characters and underscores. These tiers collectively cover the entire priority space from -1 (DEF_SWAP_PRIO) to SHRT_MAX.
To configure tiers, a new sysfs interface is exposed at /sys/kernel/mm/swap/tiers. The input parser evaluates commands from left to right and supports batch input, allowing users to add or remove multiple tiers in a single write operation.
Tier management enforces continuous priority ranges anchored by start priorities. Operations trigger range splitting or merging, but overwriting start priorities is forbidden. Merging expands lower tiers upwards to preserve configured start priorities, except when removing DEF_SWAP_PRIO, which merges downwards.
MAINTAINERS +2 -0
@@ -17060,6 +17060,8 @@ F: mm/swap.c F: mm/swap.h F: mm/swap_table.h F: mm/swap_state.c +F: mm/swap_tier.c +F: mm/swap_tier.h F: mm/swapfile.c MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE)
mm/Kconfig +12 -0
@@ -19,6 +19,18 @@ menuconfig SWAP used to provide more virtual memory than the actual RAM present in your computer. If unsure say Y. +config NR_SWAP_TIERS + int "Number of swap device tiers" + depends on SWAP + default 4 + range 1 31 + help + Sets the number of swap device tiers. Swap devices are + grouped into tiers based on their priority, allowing the + system to prefer faster devices over slower ones. + + If unsure, say 4. + config ZSWAP bool "Compressed cache for swap pages" depends on SWAP
mm/Makefile +1 -1
@@ -75,7 +75,7 @@ ifdef CONFIG_MMU obj-$(CONFIG_ADVISE_SYSCALLS) += madvise.o endif -obj-$(CONFIG_SWAP) += page_io.o swap_state.o swapfile.o +obj-$(CONFIG_SWAP) += page_io.o swap_state.o swapfile.o swap_tier.o obj-$(CONFIG_ZSWAP) += zswap.o obj-$(CONFIG_HAS_DMA) += dmapool.o obj-$(CONFIG_HUGETLBFS) += hugetlb.o hugetlb_sysfs.o hugetlb_sysctl.o
mm/swap.h +4 -0
@@ -34,6 +34,10 @@ extern int page_cluster;#define swap_entry_order (order) 0#endif +#define DEF_SWAP_PRIO -1 + +extern spinlock_t swap_lock; +extern struct plist_head swap_active_head;extern struct swap_info_struct *swap_info[];/*
mm/swap_state.c +74 -0
88 lines not on this page · raw message
mm/swap_tier.c new file +302 -0
302 lines not on this page · raw message
mm/swap_tier.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _SWAP_TIER_H +#define _SWAP_TIER_H + +#include <linux/types.h> +#include <linux/spinlock.h> + +extern spinlock_t swap_tier_lock; + +/* Initialization and application */ +void swap_tiers_init (void ); +ssize_t swap_tiers_sysfs_show (char *buf); + +int swap_tiers_add (const char *name,int prio); +int swap_tiers_remove (const char *name); + +void swap_tiers_snapshot (void ); +void swap_tiers_snapshot_restore (void ); +bool swap_tiers_validate (void ); +#endif /* _SWAP_TIER_H */
mm/swapfile.c +5 -3
@@ -48,6 +48,7 @@#include "swap_table.h" #include "internal.h" #include "swap.h" +#include "swap_tier.h" static void swap_range_alloc (struct swap_info_struct *si,unsigned int nr_entries);
@@ -63,7 +64,8 @@ static void move_cluster(struct swap_info_struct *si,* * Also protects swap_active_head total_swap_pages, and the SWP_WRITEOK flag. */ -static DEFINE_SPINLOCK (swap_lock); +DEFINE_SPINLOCK (swap_lock); +static unsigned int nr_swapfiles;atomic_long_t nr_swap_pages;/*
@@ -74,7 +76,6 @@ atomic_long_t nr_swap_pages;EXPORT_SYMBOL_GPL (nr_swap_pages);/* protected with swap_lock. reading in vm_swap_full() doesn't need lock */ long total_swap_pages; -#define DEF_SWAP_PRIO -1unsigned long swapfile_maximum_size;#ifdef CONFIG_MIGRATION bool swap_migration_ad_supported;
@@ -87,7 +88,7 @@ static const char Bad_offset[] = "Bad swap offset entry ";* all active swap_info_structs * protected with swap_lock, and ordered by priority. */ -static PLIST_HEAD (swap_active_head); +PLIST_HEAD (swap_active_head);/* * all available (active, not full) swap_info_structs
@@ -3988,6 +3989,7 @@ static int __init swapfile_init(void) swap_migration_ad_supported = true;#endif /* CONFIG_MIGRATION */ +swap_tiers_init ();return 0 ; }subsys_initcall (swapfile_init);
Signature · 1 line
2.34.1
Youngjun Park
reviewerPATCH v8 2/4 mm: swap: associate swap devices with tiers
In reply to Youngjun ParkThis patch connects swap devices to the swap tier infrastructure, ensuring that devices are correctly assigned to tiers based on their priority.
A tier_mask is added to identify the tier membership of swap devices. Although tier-based allocation logic is not yet implemented, this mapping is necessary to track which tier a device belongs to. Upon activation, the device is assigned to a tier by matching its priority against the configured tier ranges.
The infrastructure allows dynamic modification of tiers, such as splitting or merging ranges. These operations are permitted provided that the tier assignment of already configured swap devices remains unchanged.
This patch also adds the documentation for the swap tier feature, covering the core concepts, sysfs interface usage, and configuration details.
Documentation/mm/index.rst +1 -0
@@ -21,6 +21,7 @@ see the :doc:`admin guide <../admin-guide/mm/index>`. page_reclaim swap swap-table + swap-tier page_cache shmfs oom
Documentation/mm/swap-tier.rst new file +150 -0
150 lines not on this page · raw message
MAINTAINERS +1 -0
@@ -17052,6 +17052,7 @@ L: [email protected] S: Maintained F: Documentation/ABI/testing/sysfs-kernel-mm-swap F: Documentation/mm/swap-table.rst +F: Documentation/mm/swap-tier.rst F: include/linux/swap.h F: include/linux/swapfile.h F: include/linux/swapops.h
include/linux/swap.h +1 -0
@@ -250,6 +250,7 @@ struct swap_info_struct {struct percpu_ref users;/* indicate and keep swap device valid. */ unsigned long flags;/* SWP_USED etc: see above */ signed short prio;/* swap priority of this type */ +int tier_mask;/* swap tier mask */ struct plist_node list;/* entry in swap_active_head */ signed char type;/* strange name for an index */ unsigned int max;/* size of this swap device */
mm/swap_state.c +1 -1
@@ -1063,7 +1063,7 @@ static ssize_t tiers_store(struct kobject *kobj, } } -if (!swap_tiers_validate ()) { +if (!swap_tiers_update ()) { ret = -EINVAL ; goto restore; }
mm/swap_tier.c +89 -12
165 lines not on this page · raw message
mm/swap_tier.h +12 -1
@@ -5,8 +5,15 @@#include <linux/types.h> #include <linux/spinlock.h> +/* Forward declarations */ +struct swap_info_struct ; +extern spinlock_t swap_tier_lock; +#define TIER_ALL_MASK (~0) +#define TIER_DEFAULT_IDX (31) +#define TIER_DEFAULT_MASK (1U << TIER_DEFAULT_IDX) +/* Initialization and application */ void swap_tiers_init (void );ssize_t swap_tiers_sysfs_show (char *buf);
@@ -16,5 +23,9 @@ int swap_tiers_remove(const char *name);void swap_tiers_snapshot (void );void swap_tiers_snapshot_restore (void ); -bool swap_tiers_validate (void ); +bool swap_tiers_update (void ); + +/* Tier assignment */ +void swap_tiers_assign_dev (struct swap_info_struct *swp); +#endif /* _SWAP_TIER_H */
mm/swapfile.c +2 -0
@@ -3036,6 +3036,8 @@ static void _enable_swap_info(struct swap_info_struct *si)/* Add back to available list */ add_to_avail_list (si, true); + +swap_tiers_assign_dev (si); }/*
Signature · 1 line
2.34.1
Youngjun Park
reviewerPATCH v8 3/4 mm: memcontrol: add interface for swap tier selection
In reply to Youngjun ParkIntroduce memory.swap.tiers.max, a flat-keyed file listing each tier defined in /sys/kernel/mm/swap/tiers with its state, "max" (allowed, the default) or "0" (disabled). A tier is one bit in the cgroup's tier mask, so writing "<tier> max" or "<tier> 0" sets or clears that bit.
Since the current use case lacks amount control, it only supports "max" (on) and "0" (off). Therefore, it does not track per-tier swap usage, relying instead on a fast runtime bitmask check.
We maintain both mask and effective_mask. The effective_mask is strictly bounded by the parent (e.g., if a parent is "0", the child's effective state is "0" even if its mask is "max"). Maintaining this separately avoids costly cgroup tree traversals to check ancestors at runtime.
Documentation/admin-guide/cgroup-v2.rst +20 -0
@@ -1850,6 +1850,26 @@ The following nested keys are defined. Swap usage hard limit. If a cgroup's swap usage reaches this limit, anonymous memory of the cgroup will not be swapped out. + memory.swap.tiers.max + A read-write flat-keyed file which exists on non-root + cgroups. The default is "max" for every tier. + + Limits the swap tiers this cgroup may swap to. Tiers are + defined globally in /sys/kernel/mm/swap/tiers and listed here, + one per line. When read, the values are displayed in descending + order of the tiers (highest tier first):: + + <tier_1> max + <tier_2> 0 + ... + + Currently, only "max" and "0" are supported. "max" allows the + tier, "0" disables it. Each write sets a single "<tier> max" + or "<tier> 0" pair. + + A child may only narrow what its parent allows. A tier an + ancestor disabled stays disabled regardless of the value here. + memory.swap.events A read-only flat-keyed file which exists on non-root cgroups. The following entries are defined. Unless specified
Documentation/mm/swap-tier.rst +9 -0
@@ -15,6 +15,15 @@ speed to fully utilize this feature. While the current implementation is integrated with cgroups, the concept is designed to be extensible for other subsystems in the future. +Use case +--------- + +Users can perform selective swapping by choosing a swap tier assigned according +to speed within a cgroup. + +For more information on cgroup v2, please refer to +``Documentation/admin-guide/cgroup-v2.rst``. + Priority Range --------------
include/linux/memcontrol.h +5 -0
@@ -283,6 +283,11 @@ struct mem_cgroup {struct lru_gen_mm_list mm_list;#endif +#ifdef CONFIG_SWAP +int tier_mask; +int tier_effective_mask; +#endif +#ifdef CONFIG_MEMCG_V1 /* Legacy consumer-oriented counters */ struct page_counter kmem;/* v1 only */
mm/memcontrol.c +67 -0
91 lines not on this page · raw message
mm/swap_state.c +3 -2
@@ -1021,6 +1021,7 @@ static ssize_t tiers_store(struct kobject *kobj,char *p, *token, *name, *tmp;int ret =0 ;short prio; +int mask =0 ; tmp =kstrdup (buf,GFP_KERNEL );if (!tmp)
@@ -1053,7 +1054,7 @@ static ssize_t tiers_store(struct kobject *kobj, goto restore;break ;case '-' : - ret =swap_tiers_remove (token +1 ); + ret =swap_tiers_remove (token +1 , &mask);if (ret) goto restore;break ;
@@ -1063,7 +1064,7 @@ static ssize_t tiers_store(struct kobject *kobj, } } -if (!swap_tiers_update ()) { +if (!swap_tiers_update (mask)) { ret = -EINVAL ; goto restore; }
mm/swap_tier.c +106 -2
131 lines not on this page · raw message
mm/swap_tier.h +51 -6
73 lines not on this page · raw message
Signature · 1 line
2.34.1
YoungJun Park
reviewerRe: [PATCH v8 3/4] mm: memcontrol: add interface for swap tier selection
Sahiko review
https://sashiko.dev/#/message/20260617053447.2831896-4-youngjun.park%40lge.com
I think this might be an overconcern. In normal
situations, cgroup delegation is strictly under the administrator's control.
Furthermore, if a user already has a delegated root, there are numerous other ways they could potentially cause a DoS attack. Therefore, I believe this doesn't need to be modified.
Youngjun Park
reviewerPATCH v8 4/4 mm: swap: filter swap allocation by memcg tier mask
In reply to Youngjun ParkApply memcg tier effective mask during swap slot allocation to enforce per-cgroup swap tier restrictions.
In the fast path, check the percpu cached swap_info's tier_mask against the folio's effective mask. If it does not match, fall through to the slow path. In the slow path, skip swap devices whose tier_mask is not covered by the folio's effective mask.
This works correctly when there is only one non-rotational device in the system and no devices share the same priority. However, there are known limitations:
- When non-rotational devices are distributed across multiple tiers, and different memcgs are configured to use those distinct tiers, they may constantly overwrite the shared percpu swap cache. This cache thrashing leads to frequent fast path misses.
- Combined with the above issue, if same-priority devices exist among them, a percpu cache miss (overwritten by another memcg) forces the allocator to round-robin to the next device prematurely, even if the current cluster is not fully exhausted.
These edge cases do not affect the primary use case of directing swap traffic per cgroup. Further optimization is planned for future work.
mm/swapfile.c +9 -1
@@ -1365,14 +1365,18 @@ static bool swap_alloc_fast(struct folio *folio)struct swap_cluster_info *ci;struct swap_info_struct *si;unsigned int offset; +int mask =folio_tier_effective_mask (folio);/* * Once allocated, swap_info_struct will never be completely freed, * so checking it's liveness by get_swap_device_info is enough. */ si =this_cpu_read (percpu_swap_cluster.si[order]); +if (!si || !swap_tiers_mask_test (si->tier_mask, mask)) +return false; + offset =this_cpu_read (percpu_swap_cluster.offset[order]); -if (!si || !offset || !get_swap_device_info (si)) +if (!offset || !get_swap_device_info (si))return false; ci =swap_cluster_lock (si, offset);
@@ -1392,10 +1396,14 @@ static bool swap_alloc_fast(struct folio *folio)static void swap_alloc_slow (struct folio *folio) {struct swap_info_struct *si, *next; +int mask =folio_tier_effective_mask (folio);spin_lock (&swap_avail_lock); start_over:plist_for_each_entry_safe (si, next, &swap_avail_head, avail_list) { +if (!swap_tiers_mask_test (si->tier_mask, mask)) +continue ; +/* Rotate the device and switch to a new cluster */ plist_requeue (&si->avail_list, &swap_avail_head);spin_unlock (&swap_avail_lock);
Signature · 1 line
2.34.1
YoungJun Park
reviewerRe: [PATCH v8 4/4] mm: swap: filter swap allocation by memcg tier mask
https://sashiko.dev/#/patchset/[email protected]?part=4
Regarding the review from Sachiko.
The AI review pointed out two things. a pre-existing issue and a valid issue regarding swap_sync_discard().
For the pre-existing issue, I believe it is out of scope to address it together in this patchset.
However, the second point is a valid issue. For the case where swap_sync_discard() discards a non-selected tier, succeeds, and then retries swap allocation. I will fix this in the next version by checking the memcg mask within swap_sync_discard().
Nhat Pham
reviewerQuoted 7 lines · Youngjun Park
This is the v8 series of the swap tier patchset.
Great thanks to Shakeel Butt and Yosry for the reviews and discussions [1]. The main change in this version is the interface change to use memory.swap.tiers.max with '0' (disable) and 'max' (enable) values. This mechanism was suggested by Shakeel and Yosry
I like this interface too :)
Quoted 11 lines
This change allows for future extensions to control swap between tiers and aligns better with existing memcg interfaces. Even with this memcg interface change, only patch #3 needed updates. Internally, patch #3 still uses the existing mask processing method (which is implementation-efficient), so only the user-facing interface was modified.
We also discussed tier extensions. Thanks to Yosry, Nhat and Shakeel for their valuable feedback.
Quoted 7 lines
Here is a brief summary of our tentative conclusions. Please correct me if anything is misrepresented (details in references):
* Zswap tiering [2]:
Tiering applies only to the vswap + zswap combo. Zswap itself will not be tiered, as the current architecture requires a physical device for zswap allocation.
I think Yosry wants zswap as a tier, right?
Just that without vswap, maybe don't allow it to be an tier of itself?
Quoted 33 lines · Vswap tiering
Vswap should be handled transparently to the user. Vswap itself will not be tiered. But, someday supported if there is strong and real usecase.
* Relationship with zswap.writeback [4]:
If zswap tiering is introduced, it could replace the zswap-only tier. However, since zswap cannot be tiered independently, it is still needed for non-vswap cases. Separately, the internal logic could potentially be integrated into the tiering logic.
* Tier demotion [5]:
A separate interface like memory.swap.tiers.demotion might be needed. For now, we only support 0/max to enable/disable tiers. In the future, we could introduce an "auto" mode to automatically scale the limit based on swapfile size and memory.swap.max, similar to the direction memory tiering is heading in.I plan to apply the swap tier infrastructure and the first use case (cgroup-based swap control) first, and continue following up on the discussions above.
Overview
========Swap Tiers group swap devices into performance classes (e.g. NVMe, HDD, Network) and allow per-memcg selection of which tiers to use. This mechanism was suggested by Chris Li.
#2: Inter-tier promotion and demotion:
Promotion and demotion apply between tiers, not within a single tier. The current interface defines only tier assignment; it does not yet define when or how pages move between tiers. Two triggering models are possible:
Quoted 4 lines
(a) User-triggered: userspace explicitly initiates migration between tiers (e.g. via a new interface or existing
move_pagessemantics).
(b) Kernel-triggered: the kernel moves pages between tiers at appropriate points such as reclaim or refault.
We'll likely need some kernel-triggered mechanism, or we'd have LRU inversion :)
Cold pages will fill up fast tiers first, and more recent/warm pages will land on slow tiers...
We'll also need to enforce isolation/fairness to make sure no wordload hoard the fast tiers too (but that probably requires demotion support).
Quoted 11 lines
#3: Per-VMA, per-process swap and BPF:
Not just for memcg based swap, possible to extend Per-VMA or per-process swap. Or we can use it as BPF program.#4: Zswap and vswap tiering:
Tiering applies to the vswap + zswap combination.#5: Vswap on/off control:
Currently not supported. If a strong use case arises where vswap needs to be controlled by memcg, the tier interface could be used for it.
+1.
Also, per-si/per-tier per-CPU allocation caching? :) Kairui already has a patch for it, IIUC, but if not it's pretty critical I'd say.
BTW, can we add some selftests, to make sure the new interface works as expected, and to have example programs for new users to model their scripts after? :)
YoungJun Park
reviewerQuoted 10 lines · Nhat Pham
Quoted 7 lines · Youngjun Park
This is the v8 series of the swap tier patchset.
Great thanks to Shakeel Butt and Yosry for the reviews and discussions [1]. The main change in this version is the interface change to use memory.swap.tiers.max with '0' (disable) and 'max' (enable) values. This mechanism was suggested by Shakeel and Yosry
I like this interface too :)
Good to hear. Now it looks like we have found a memcg interface that aligns well with the existing memcg model.
I like this idea as well. Thanks again to Shakeel Butt and Yosry.
Quoted 11 lines
Quoted 7 lines
Here is a brief summary of our tentative conclusions. Please correct me if anything is misrepresented (details in references):
* Zswap tiering [2]:
Tiering applies only to the vswap + zswap combo. Zswap itself will not be tiered, as the current architecture requires a physical device for zswap allocation.I think Yosry wants zswap as a tier, right?
Just that without vswap, maybe don't allow it to be an tier of itself?
With the current architecture, users cannot dynamically specify zswap as a tier, and zswap is a separate layer, so it is not tiered by itself.
Once your vswap work lands, I think we can make the zswap become the default, top-level tier.
After that, we can also look into cleaning up the zswap.writeback interface together.
Quoted 11 lines
#2: Inter-tier promotion and demotion:
Promotion and demotion apply between tiers, not within a single tier. The current interface defines only tier assignment; it does not yet define when or how pages move between tiers. Two triggering models are possible:Quoted 4 lines
(a) User-triggered: userspace explicitly initiates migration between tiers (e.g. via a new interface or existing
move_pagessemantics).
(b) Kernel-triggered: the kernel moves pages between tiers at appropriate points such as reclaim or refault.
Quoted 4 lines
We'll likely need some kernel-triggered mechanism, or we'd have LRU inversion :)
Cold pages will fill up fast tiers first, and more recent/warm pages will land on slow tiers...
Yeah, good point!
Quoted 3 lines
We'll also need to enforce isolation/fairness to make sure no wordload hoard the fast tiers too (but that probably requires demotion support).
Right, that makes sense.
BTW, One thing I am curious about, though, is whether there are strong real-world use cases that require demotion/promotion. Theoretically, this looks useful but it would be helpful to better understand the requirements from such deployments.
Quoted 8 lines
Quoted 8 lines
#3: Per-VMA, per-process swap and BPF:
Not just for memcg based swap, possible to extend Per-VMA or per-process swap. Or we can use it as BPF program.#4: Zswap and vswap tiering:
Tiering applies to the vswap + zswap combination.
Quoted 8 lines
Quoted 3 lines
#5: Vswap on/off control:
Currently not supported. If a strong use case arises where vswap needs to be controlled by memcg, the tier interface could be used for it.+1.
Also, per-si/per-tier per-CPU allocation caching? :) Kairui already has a patch for it, IIUC, but if not it's pretty critical I'd say.
Yes, I missed it. Thank you for addressing it.
we need an implementation that integrates this with the per-CPU allocation currently implemented on the vswap side.
If Kairui's patch lands, my patch #4 also can be optimized based on that.
Quoted 3 lines
BTW, can we add some selftests, to make sure the new interface works as expected, and to have example programs for new users to model their scripts after? :)
Yes, I agree. I think selftests are necessary.
Do you want them to be introduced in this patchset, or would it be okay to add them separately as follow-up work?
Nhat Pham
reviewerQuoted 18 lines · YoungJun Park
Quoted 9 lines · Nhat Pham
Quoted 7 lines · Youngjun Park
This is the v8 series of the swap tier patchset.
Great thanks to Shakeel Butt and Yosry for the reviews and discussions [1]. The main change in this version is the interface change to use memory.swap.tiers.max with '0' (disable) and 'max' (enable) values. This mechanism was suggested by Shakeel and Yosry
Quoted 5 lines
I like this interface too :)
I think Yosry wants zswap as a tier, right?
Just that without vswap, maybe don't allow it to be an tier of itself?
Quoted 8 lines
With the current architecture, users cannot dynamically specify zswap as a tier, and zswap is a separate layer, so it is not tiered by itself.
Once your vswap work lands, I think we can make the zswap become the default, top-level tier.
After that, we can also look into cleaning up the zswap.writeback interface together.
SGTM if Yosry is happy with it :) FWIW, zswap is a conceptual tier, whether we want it to express with your interface or not. This is just interface clean-up work.
Quoted 23 lines
Quoted 12 lines
#2: Inter-tier promotion and demotion:
Promotion and demotion apply between tiers, not within a single tier. The current interface defines only tier assignment; it does not yet define when or how pages move between tiers. Two triggering models are possible:Quoted 4 lines
(a) User-triggered: userspace explicitly initiates migration between tiers (e.g. via a new interface or existing
move_pagessemantics).
(b) Kernel-triggered: the kernel moves pages between tiers at appropriate points such as reclaim or refault.Quoted 4 lines
We'll likely need some kernel-triggered mechanism, or we'd have LRU inversion :)
Cold pages will fill up fast tiers first, and more recent/warm pages will land on slow tiers...
Yeah, good point!
Quoted 3 lines
We'll also need to enforce isolation/fairness to make sure no wordload hoard the fast tiers too (but that probably requires demotion support).
Quoted 6 lines
Right, that makes sense.
BTW, One thing I am curious about, though, is whether there are strong real-world use cases that require demotion/promotion. Theoretically, this looks useful but it would be helpful to better understand the requirements from such deployments.
I think so, yeah. The LRU inversion problem above is one :) Hard to make proper tiering without demotion.
Say I have a workload that have a SLO - for example a PSI target - but don't particularly care about exact memory placement. To optimize resource, we want to place the warmer stuff in fast tier, and the coldest stuff in slow tier, etc. Having the ability to do demotion derisk the initial placement - we can place things in the fast tier initially (and rather aggressively), then as pages age and prove their coldness, we can move them to slower and slower tier, etc.
Otherwise, what we end up with is really a placement preference interface more than true tiering. Which is still useful especially when co-tenant workloads have strict latency requirements, but perhaps we don't need a full hierarchy-style interface for it? :)
The other use case is for fairness enforcement. We can (and probably should) start with strict limits, but setting memory.swap.tier.max for each cgroup is a bit of a drag, and it might leave stranded capacity in cgroups that are allocated but not utilized their fast swap tier capacity. If demotion is possible, we can let workloads use more than what is fair, but then demote swap pages from swap tier to enforce fairness when necessary...
Obviously, it's a moot point if there is no good mechanism to transfer data one tier to another. The data might also be so cold that all of this has diminishing returns, and moving things around cost more than it's worth :) So I'm happy to start with something simple, then we can figure out the next steps.
Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
#3: Per-VMA, per-process swap and BPF:
Not just for memcg based swap, possible to extend Per-VMA or per-process swap. Or we can use it as BPF program.Quoted 6 lines
#4: Zswap and vswap tiering:
Tiering applies to the vswap + zswap combination.#5: Vswap on/off control:
Currently not supported. If a strong use case arises where vswap needs to be controlled by memcg, the tier interface could be used for it.+1.
Quoted 8 lines
Quoted 2 lines
Also, per-si/per-tier per-CPU allocation caching? :) Kairui already has a patch for it, IIUC, but if not it's pretty critical I'd say.
Yes, I missed it. Thank you for addressing it.
we need an implementation that integrates this with the per-CPU allocation currently implemented on the vswap side.If Kairui's patch lands, my patch #4 also can be optimized based on that.
Yup!!
Quoted 9 lines
Quoted 4 lines
BTW, can we add some selftests, to make sure the new interface works as expected, and to have example programs for new users to model their scripts after? :)
Yes, I agree. I think selftests are necessary.
Do you want them to be introduced in this patchset, or would it be okay to add them separately as follow-up work?
If you have to send another version, might as well include them :)
Otherwise a follow-up is good. Thanks in advance for keeping our codebase tested!
I'll take a look at the exact implementation on the swap side later, but I suspect nothing much will have changed :)