Skip to contentSkip to conversation

mm/swap, memcg: Introduce swap tiers for cgroup based swap control

PATCH Superseded Youngjun Park 2 participants 19 messages Apr 21 – May 27

Subsystems (7)

v6 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Youngjun Park

author reviewer
#0 lore raw

v6 cover mm/swap, memcg: Introduce swap tiers for cgroup based swap control

This version mainly addresses meaningful issues reported by Sashiko AI review (https://sashiko.dev/#/patchset/20260325175453.2523280-1-youngjun.park%40lge.com)
False positives were excluded,
and some items were determined not to require changes at this time.

For clarity, this cover letter is structured in two parts:

Part 1 describes the patch series itself (what is implemented in v5). Part 2 consolidates the design rationale and use case discussion, including clarification around the memcg-integrated model and comparison with BPF-based approaches.

This separation is intentional so reviewers can clearly distinguish between patch introduction and design discussion (for Shakeel's ongoing feedback).

v5:
https://lore.kernel.org/linux-mm/[email protected]/

v4:
https://lore.kernel.org/linux-mm/[email protected]/

Earlier RFC versions:
v3: https://lore.kernel.org/linux-mm/[email protected]/
v2: https://lore.kernel.org/linux-mm/[email protected]/
v1: https://lore.kernel.org/linux-mm/[email protected]/

Earlier Approach (per cgroup swap priority)
RFC: https://lore.kernel.org/linux-mm/aEvLjEInMQC7hEyh@yjaykim-PowerEdge-T330/T/#mbbb6a5e9e30843097e1f5f65fb98f31d582b973d
v1: https://lore.kernel.org/linux-mm/[email protected]/
======================================================================
Part 1: Patch Series Summary
======================================================================

Overview
========
Swap Tiers group swap devices into performance classes (e.g. NVMe, HDD, Network) and allow per-memcg selection of which tiers to use. This mechanism was suggested by Chris Li.

This series introduces:

- Core tier infrastructure
- Per-memcg tier assignment (subset of parent)
- memory.swap.tiers and memory.swap.tiers.effective interfaces

Changed in v6
=============

Sashiko AI review fixes:
------------------------
- Fix batch parsing error path to restore snapshot before exit
- Reject overlong tier names to prevent truncated duplicates
- Avoid restoring raw list_head via memcpy (stale pointer risk)
- Ensure early parse errors do not skip DEF_SWAP_PRIO validation
- Use (1U << TIER_DEFAULT_IDX) to avoid signed shift UB
- Defer tier mask inheritance to css_online() to close race window
- Add READ_ONCE()/WRITE_ONCE() for tier mask accesses

FYI. there are items. reviewed but no code change
- Legacy default swap priorities (-2, -3, etc.) are no longer used. (False positive)
- Large tree traversal under spin_lock not observed as a practical issue.
- Lowest tier (DEF_SWAP_PRIO) replacement in a single batch update
remains a functional limitation. (for interface usage enhancement. next feature update target.)

Other fixes:
------------
- Fix build error reintroduced due to missing v5 change (sorry for that..)
- Fix WARNING in folio_tier_effective_mask by adding rcu_read_lock() (syzbot CI fix)
- default number of swap tier max (change to 32->31, for reserving last bit)
- commit message refinement.
- rebased on recently mm-new

Changes in v5
=============
- Fixed build errors reported in v4
- rebased on up to date mm-new
- Minor cleanups
- Design docs with validation (by Shakeel Butt discussion)

Changes in v4 (summary)
=======================
- Simplified control flow and indentation
- Added CONFIG option for MAX_SWAPTIER (default: 4)
- Added memory.swap.tiers.effective interface
- Reworked save/restore logic into snapshot/rollback model
- Removed tier priority modification support (deferred)
- Improved validation and fixed edge cases
- Rebased onto latest mm-new

Deferred / Future Work
======================
- Per-tier swap_active_head to reduce contention (Suggested by Chris Li)
- Fast path and slow path allocation improvement
(this will be introduced after Kairui's work)

Real-world Results
==================
Tested on our internal platform using NBD as a separate swap tier. Our first production's simple usecase.

Without tiers:
- No selective control over flash wear
- Cannot selectively assign NBD to specific applications

Cold launch improvement (preloaded vs. baseline):
- App A: 13.17s -> 4.18s (68%)
- App B: 5.60s -> 1.12s (80%)
- App C: 10.25s -> 2.00s (80%)

Performance impact with no tiers configured:
<1% regression in kernel build and vm-scalability benchmarks (measured in RFC v2).

======================================================================
Part 2: Design Rationale and Use Cases
======================================================================

Design Rationale
================
Swap tier selection is attached to memcg. A child cgroup may select a subset of the parent's allowed tiers.

This:
- Preserves cgroup inheritance semantics (boundary at parent, refinement at child).
- Reuses memcg, which already groups processes and enforces hierarchical memory limits.
- Aligns with existing memcg swap controls (e.g. swap.max, zswap.writeback)
- Avoids introducing a parallel swap control hierarchy.

Placing tier control outside memcg (e.g. bpf, syscall, madvise etc..) would allow swap preference to diverge from the memcg hierarchy. Integrating it into memcg keeps swap policy consistent with existing memory ownership semantics.

Use case #1: Latency separation (our primary deployment scenario)
=================================================================
[ / ]
|
+-- latency-sensitive workload (fast tier)
+-- background workload (slow tier)

The parent defines the memory boundary.
Each workload selects a swap tier via memory.swap.tiers according to latency requirements.

This prevents latency-sensitive workloads from being swapped to slow devices used by background workloads.

Use case #2: Per-VM swap selection (Chris Li's deployment scenario)
==================================================================
[ / ]
|
+-- [ Job on VM ] (tiers: zswap, SSD)
|
+-- [ VMM guest memory ] (tiers: SSD)

The parent (job) has access to both zswap and SSD tiers. The child (VMM guest memory) selects SSD as its swap tier via memory.swap.tiers. In this deployment, swap device selection happens at the child level from the parent's available set.

Use case #3: Tier isolation for reduced contention (hypothetical)
=================================================================
[ / ] (tiers: A, B)
|
+-- workload X (tiers: A)
+-- workload Y (tiers: B)

Each child uses a different tier. Since swap paths are separated per tier, synchronization overhead between the two workloads is reduced.

How the Current Interface Supports Future Extensions
====================================================

- Intra-tier distribution policy:
Currently, swap devices with the same priority are allocated in a round-robin fashion. Per-tier policy files under /sys/kernel/mm/swap/tiers/ can control how devices within a tier are selected (e.g. round-robin, weighted).

- Inter-tier promotion and demotion:
Promotion and demotion apply between tiers, not within a single tier. The current interface defines only tier assignment; it does not yet define when or how pages move between tiers. Two triggering models are possible:

(a) User-triggered: userspace explicitly initiates migration between tiers (e.g. via a new interface or existing move_pages semantics).
(b) Kernel-triggered: the kernel moves pages between tiers at appropriate points such as reclaim or refault.

From the memcg perspective, inter-tier movement is bounded by memory.swap.tiers.effective -- pages can only be promoted or demoted to tiers within the memcg's effective set. The specific policy and triggering mechanism require further discussion and are not part of this series.

- Per-VMA or per-process swap hints:
A future madvise-style hint (e.g. MADV_SWAP_TIER) could reference the tier indices in /sys/kernel/mm/swap/tiers/. At reclaim time, the kernel would check the VMA hint against the memcg's effective tier set to pick the swap-out target.

BPF Comparison
==============
The use cases described above already rely on memcg for swap tier control, and real deployments are built around this model. A BPF-based approach has additional considerations:

- Hierarchy consistency: BPF programs operate outside the memcg tree. Without explicit constraints, a BPF selector could contradict parent tier restrictions. Edge cases such as zombie memcgs make the resolution less clear.
- Deployment scope: requiring BPF for core swap behavior may not be suitable for constrained or embedded configurations.

BPF could still work as an extension on top of the tier model in the future.

Youngjun Park (4):
mm: swap: introduce swap tier infrastructure
mm: swap: associate swap devices with tiers
mm: memcontrol: add interfaces for swap tier selection
mm: swap: filter swap allocation by memcg tier mask

 Documentation/admin-guide/cgroup-v2.rst |  29 ++
 Documentation/mm/index.rst              |   1 +
 Documentation/mm/swap-tier.rst          | 159 ++++++++
 MAINTAINERS                             |   3 +
 include/linux/memcontrol.h              |   5 +
 include/linux/swap.h                    |   1 +
 mm/Kconfig                              |  12 +
 mm/Makefile                             |   2 +-
 mm/memcontrol.c                         |  96 +++++
 mm/swap.h                               |   4 +
 mm/swap_state.c                         |  75 ++++
 mm/swap_tier.c                          | 483 ++++++++++++++++++++++++
 mm/swap_tier.h                          |  75 ++++
 mm/swapfile.c                           |  23 +-
 14 files changed, 963 insertions(+), 5 deletions(-)
 create mode 100644 Documentation/mm/swap-tier.rst
 create mode 100644 mm/swap_tier.c
 create mode 100644 mm/swap_tier.h

base-commit: 90f88d13d60cb842cfb6290b6e80a1cd86f37915

Signature · 1 line
2.34.1
author reviewer
#1 lore raw

PATCH v6 1/4 mm: swap: introduce swap tier infrastructure

This patch introduces the "Swap tier" concept, which serves as an abstraction layer for managing swap devices based on their performance characteristics (e.g., NVMe, HDD, Network swap).

Swap tiers are user-named groups representing priority ranges. Tier names must consist of alphanumeric characters and underscores. These tiers collectively cover the entire priority space from -1 (DEF_SWAP_PRIO) to SHRT_MAX.

To configure tiers, a new sysfs interface is exposed at /sys/kernel/mm/swap/tiers. The input parser evaluates commands from left to right and supports batch input, allowing users to add or remove multiple tiers in a single write operation.

Tier management enforces continuous priority ranges anchored by start priorities. Operations trigger range splitting or merging, but overwriting start priorities is forbidden. Merging expands lower tiers upwards to preserve configured start priorities, except when removing DEF_SWAP_PRIO, which merges downwards.

---
 MAINTAINERS     |   2 +
 mm/Kconfig      |  12 ++
 mm/Makefile     |   2 +-
 mm/swap.h       |   4 +
 mm/swap_state.c |  74 ++++++++++++
 mm/swap_tier.c  | 302 ++++++++++++++++++++++++++++++++++++++++++++++++
 mm/swap_tier.h  |  20 ++++
 mm/swapfile.c   |   8 +-
 8 files changed, 420 insertions(+), 4 deletions(-)
 create mode 100644 mm/swap_tier.c
 create mode 100644 mm/swap_tier.h
MAINTAINERS +2 -0
index 76d8291237be..83a181acd50b 100644
@@ -17033,6 +17033,8 @@ F:	mm/swap.c
 F:	mm/swap.h
 F:	mm/swap_table.h
 F:	mm/swap_state.c
+F:	mm/swap_tier.c
+F:	mm/swap_tier.h
 F:	mm/swapfile.c
 
 MEMORY MANAGEMENT - THP (TRANSPARENT HUGE PAGE)
mm/Kconfig +12 -0
index 0a43bb80df4f..d8242ceee7dd 100644
@@ -19,6 +19,18 @@ menuconfig SWAP
 	  used to provide more virtual memory than the actual RAM present
 	  in your computer.  If unsure say Y.
 
+config NR_SWAP_TIERS
+        int "Number of swap device tiers"
+        depends on SWAP
+        default 4
+        range 1 31
+        help
+          Sets the number of swap device tiers. Swap devices are
+          grouped into tiers based on their priority, allowing the
+          system to prefer faster devices over slower ones.
+
+          If unsure, say 4.
+
 config ZSWAP
 	bool "Compressed cache for swap pages"
 	depends on SWAP
mm/Makefile +1 -1
index 8ad2ab08244e..db6449f84991 100644
@@ -75,7 +75,7 @@ ifdef CONFIG_MMU
 	obj-$(CONFIG_ADVISE_SYSCALLS)	+= madvise.o
 endif
 
-obj-$(CONFIG_SWAP)	+= page_io.o swap_state.o swapfile.o
+obj-$(CONFIG_SWAP)	+= page_io.o swap_state.o swapfile.o swap_tier.o
 obj-$(CONFIG_ZSWAP)	+= zswap.o
 obj-$(CONFIG_HAS_DMA)	+= dmapool.o
 obj-$(CONFIG_HUGETLBFS)	+= hugetlb.o hugetlb_sysfs.o hugetlb_sysctl.o
mm/swap.h +4 -0
index a77016f2423b..fda8363bee73 100644
@@ -16,6 +16,10 @@ extern int page_cluster;
 #define swap_entry_order(order)	0
 #endif
 
+#define DEF_SWAP_PRIO  -1
+
+extern spinlock_t swap_lock;
+extern struct plist_head swap_active_head;
 extern struct swap_info_struct *swap_info[];
 
 /*
mm/swap_state.c +74 -0

88 lines not on this page · raw message

mm/swap_tier.c new file +302 -0

302 lines not on this page · raw message

mm/swap_tier.h new file +20 -0
new file mode 100644
index 000000000000..a1395ec02c24
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _SWAP_TIER_H
+#define _SWAP_TIER_H
+
+#include <linux/types.h>
+#include <linux/spinlock.h>
+
+extern spinlock_t swap_tier_lock;
+
+/* Initialization and application */
+void swap_tiers_init(void);
+ssize_t swap_tiers_sysfs_show(char *buf);
+
+int swap_tiers_add(const char *name, int prio);
+int swap_tiers_remove(const char *name);
+
+void swap_tiers_snapshot(void);
+void swap_tiers_snapshot_restore(void);
+bool swap_tiers_validate(void);
+#endif /* _SWAP_TIER_H */
mm/swapfile.c +5 -3
index c7e173b93e11..114e5d943de8 100644
@@ -49,6 +49,7 @@
 #include "swap_table.h"
 #include "internal.h"
 #include "swap.h"
+#include "swap_tier.h"
 
 static void swap_range_alloc(struct swap_info_struct *si,
 			     unsigned int nr_entries);
@@ -64,7 +65,8 @@ static void move_cluster(struct swap_info_struct *si,
  *
  * Also protects swap_active_head total_swap_pages, and the SWP_WRITEOK flag.
  */
-static DEFINE_SPINLOCK(swap_lock);
+DEFINE_SPINLOCK(swap_lock);
+
 static unsigned int nr_swapfiles;
 atomic_long_t nr_swap_pages;
 /*
@@ -75,7 +77,6 @@ atomic_long_t nr_swap_pages;
 EXPORT_SYMBOL_GPL(nr_swap_pages);
 /* protected with swap_lock. reading in vm_swap_full() doesn't need lock */
 long total_swap_pages;
-#define DEF_SWAP_PRIO  -1
 unsigned long swapfile_maximum_size;
 #ifdef CONFIG_MIGRATION
 bool swap_migration_ad_supported;
@@ -88,7 +89,7 @@ static const char Bad_offset[] = "Bad swap offset entry ";
  * all active swap_info_structs
  * protected with swap_lock, and ordered by priority.
  */
-static PLIST_HEAD(swap_active_head);
+PLIST_HEAD(swap_active_head);
 
 /*
  * all available (active, not full) swap_info_structs
@@ -3890,6 +3891,7 @@ static int __init swapfile_init(void)
 		swap_migration_ad_supported = true;
 #endif	/* CONFIG_MIGRATION */
 
+	swap_tiers_init();
 	return 0;
 }
 subsys_initcall(swapfile_init);
Signature · 1 line
2.34.1
reviewer Reviewed-by on v6 1/4
#2 lore raw

Re: [PATCH v6 1/4] mm: swap: introduce swap tier infrastructure

Quoted 14 lines · Youngjun Park

This patch introduces the "Swap tier" concept, which serves as an abstraction layer for managing swap devices based on their performance characteristics (e.g., NVMe, HDD, Network swap).

Swap tiers are user-named groups representing priority ranges. Tier names must consist of alphanumeric characters and underscores. These tiers collectively cover the entire priority space from -1 (DEF_SWAP_PRIO) to SHRT_MAX.

To configure tiers, a new sysfs interface is exposed at /sys/kernel/mm/swap/tiers. The input parser evaluates commands from left to right and supports batch input, allowing users to add or remove multiple tiers in a single write operation.

Quoted 8 lines

Tier management enforces continuous priority ranges anchored by start priorities. Operations trigger range splitting or merging, but overwriting start priorities is forbidden. Merging expands lower tiers upwards to preserve configured start priorities, except when removing DEF_SWAP_PRIO, which merges downwards.

This looks good to me.

While there's only one tiny concern, please see the inline comment.

Quoted 6 lines
mm/swap_tier.c new file +0 -0
new file mode 100644
index 000000000000..9490e891c5fe
@@ -0,0 +1,302 @@

......

mm/swap_tier.c line 59

Quoted 9 lines
mm/swap_tier.c new file +9 -0
+/*
+ * Naming Convention:
+ *   swap_tiers_*() - Public/exported functions
+ *   swap_tier_*()  - Private/internal functions
+ */
+
+static bool swap_tier_is_active(void)
+{
+	return !list_empty(&swap_tier_active_list) ? true : false;

The above line seems like generated by AI. Whatever else I have seen is "return !list_empty(&swap_tier_active_list);" which is enough.

Quoted 2 lines
mm/swap_tier.c new file +2 -0
+}
+

...snip...

author reviewer
#3 lore raw

Re: [PATCH v6 1/4] mm: swap: introduce swap tier infrastructure

Quoted 37 lines · Baoquan He
Quoted 14 lines · Youngjun Park

This patch introduces the "Swap tier" concept, which serves as an abstraction layer for managing swap devices based on their performance characteristics (e.g., NVMe, HDD, Network swap).

Swap tiers are user-named groups representing priority ranges. Tier names must consist of alphanumeric characters and underscores. These tiers collectively cover the entire priority space from -1 (DEF_SWAP_PRIO) to SHRT_MAX.

To configure tiers, a new sysfs interface is exposed at /sys/kernel/mm/swap/tiers. The input parser evaluates commands from left to right and supports batch input, allowing users to add or remove multiple tiers in a single write operation.

Quoted 8 lines

Tier management enforces continuous priority ranges anchored by start priorities. Operations trigger range splitting or merging, but overwriting start priorities is forbidden. Merging expands lower tiers upwards to preserve configured start priorities, except when removing DEF_SWAP_PRIO, which merges downwards.

This looks good to me.

While there's only one tiny concern, please see the inline comment.

Quoted 6 lines
mm/swap_tier.c new file +0 -0
new file mode 100644
index 000000000000..9490e891c5fe
@@ -0,0 +1,302 @@

......

Quoted 12 lines

mm/swap_tier.c line 59

Quoted 9 lines
mm/swap_tier.c new file +9 -0
+/*
+ * Naming Convention:
+ *   swap_tiers_*() - Public/exported functions
+ *   swap_tier_*()  - Private/internal functions
+ */
+
+static bool swap_tier_is_active(void)
+{
+	return !list_empty(&swap_tier_active_list) ? true : false;

The above line seems like generated by AI. Whatever else I have seen is "return !list_empty(&swap_tier_active_list);" which is enough.

Oops, it is generated by me. My coding style.

You are right. I will remove the ternary operator.

Best regards,
Youngjun Park

reviewer
#4 lore raw

Re: [PATCH v6 1/4] mm: swap: introduce swap tier infrastructure

In reply to Youngjun Park

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

mm/swap_tier.c line 292

Quoted 8 lines
mm/swap_tier.c new file +8 -0
+bool swap_tiers_validate(void)
+{
+	struct swap_tier *tier;
+
+	/*
+	 * Initial setting might not cover DEF_SWAP_PRIO.
+	 * Swap tier must cover the full range (DEF_SWAP_PRIO to SHRT_MAX).
+	 */

If so, do we need check if the upmost boundary SHRT_MAX is covered?

Quoted 10 lines
mm/swap_tier.c new file +10 -0
+	if (swap_tier_is_active()) {
+		tier = list_last_entry(&swap_tier_active_list,
+			struct swap_tier, list);
+
+		if (tier->prio != DEF_SWAP_PRIO)
+			return false;
+	}
+
+	return true;
+}

...snip...

author reviewer
#5 lore raw

Re: [PATCH v6 1/4] mm: swap: introduce swap tier infrastructure

Quoted 12 lines · Baoquan He

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

mm/swap_tier.c line 292

Quoted 8 lines
mm/swap_tier.c new file +8 -0
+bool swap_tiers_validate(void)
+{
+	struct swap_tier *tier;
+
+	/*
+	 * Initial setting might not cover DEF_SWAP_PRIO.
+	 * Swap tier must cover the full range (DEF_SWAP_PRIO to SHRT_MAX).
+	 */

If so, do we need check if the upmost boundary SHRT_MAX is covered?

Hello Baoquan

It naturally covers SHRT_MAX.
Because all swap_tier object prio represents start of priority and the prio value is assured on the range of DEF_SWAP_PRIO ~ SHAR_MAX on the privious routine.

swap_tier_validate function is for checking the first tier cover DEF_SWAP_PRIO. if it is not, it breaks the assumtion "cover DEF_SWAP_PRIO to SHRT_MAX"

Thanks for the review.
Youngjun Park

reviewer
#6 lore raw

Re: [PATCH v6 1/4] mm: swap: introduce swap tier infrastructure

Quoted 16 lines · YoungJun Park
Quoted 12 lines · Baoquan He

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

mm/swap_tier.c line 292

Quoted 8 lines
mm/swap_tier.c new file +8 -0
+bool swap_tiers_validate(void)
+{
+	struct swap_tier *tier;
+
+	/*
+	 * Initial setting might not cover DEF_SWAP_PRIO.
+	 * Swap tier must cover the full range (DEF_SWAP_PRIO to SHRT_MAX).
+	 */

If so, do we need check if the upmost boundary SHRT_MAX is covered?

Hello Baoquan

Quoted 7 lines

It naturally covers SHRT_MAX.
Because all swap_tier object prio represents start of priority and the prio value is assured on the range of DEF_SWAP_PRIO ~ SHAR_MAX on the privious routine.

swap_tier_validate function is for checking the first tier cover DEF_SWAP_PRIO. if it is not, it breaks the assumtion "cover DEF_SWAP_PRIO to SHRT_MAX"

Thanks, I got it now. We only track the beginning of prio range via tier->prio, while deduce the end of prio range from configured tiers. That checking is reasonable. Sorry for the noise.

author reviewer
#7 lore raw

PATCH v6 2/4 mm: swap: associate swap devices with tiers

In reply to Youngjun Park

This patch connects swap devices to the swap tier infrastructure, ensuring that devices are correctly assigned to tiers based on their priority.

A tier_mask is added to identify the tier membership of swap devices. Although tier-based allocation logic is not yet implemented, this mapping is necessary to track which tier a device belongs to. Upon activation, the device is assigned to a tier by matching its priority against the configured tier ranges.

The infrastructure allows dynamic modification of tiers, such as splitting or merging ranges. These operations are permitted provided that the tier assignment of already configured swap devices remains unchanged.

This patch also adds the documentation for the swap tier feature, covering the core concepts, sysfs interface usage, and configuration details.

---
 Documentation/mm/index.rst     |   1 +
 Documentation/mm/swap-tier.rst | 159 +++++++++++++++++++++++++++++++++
 MAINTAINERS                    |   1 +
 include/linux/swap.h           |   1 +
 mm/swap_state.c                |   2 +-
 mm/swap_tier.c                 | 101 ++++++++++++++++++---
 mm/swap_tier.h                 |  13 ++-
 mm/swapfile.c                  |   2 +
 8 files changed, 266 insertions(+), 14 deletions(-)
 create mode 100644 Documentation/mm/swap-tier.rst
Documentation/mm/index.rst +1 -0
index 7aa2a8886908..a0d1447c5569 100644
@@ -21,6 +21,7 @@ see the :doc:`admin guide <../admin-guide/mm/index>`.
    page_reclaim
    swap
    swap-table
+   swap-tier
    page_cache
    shmfs
    oom
Documentation/mm/swap-tier.rst new file +159 -0

159 lines not on this page · raw message

MAINTAINERS +1 -0
index 83a181acd50b..3987aa71681d 100644
@@ -17025,6 +17025,7 @@ R:	Youngjun Park <[email protected]>
 L:	[email protected]
 S:	Maintained
 F:	Documentation/mm/swap-table.rst
+F:	Documentation/mm/swap-tier.rst
 F:	include/linux/swap.h
 F:	include/linux/swapfile.h
 F:	include/linux/swapops.h
include/linux/swap.h +1 -0
index 1930f81e6be4..3bc06a1a4a17 100644
@@ -250,6 +250,7 @@ struct swap_info_struct {
 	struct percpu_ref users;	/* indicate and keep swap device valid. */
 	unsigned long	flags;		/* SWP_USED etc: see above */
 	signed short	prio;		/* swap priority of this type */
+	int tier_mask;			/* swap tier mask */
 	struct plist_node list;		/* entry in swap_active_head */
 	signed char	type;		/* strange name for an index */
 	unsigned int	max;		/* size of this swap device */
mm/swap_state.c +1 -1
index 04a6f1766b36..182e6d6c9c89 100644
@@ -980,7 +980,7 @@ static ssize_t tiers_store(struct kobject *kobj,
 		}
 	}
 
-	if (!swap_tiers_validate()) {
+	if (!swap_tiers_update()) {
 		ret = -EINVAL;
 		goto restore;
 	}
mm/swap_tier.c +89 -12

165 lines not on this page · raw message

mm/swap_tier.h +12 -1
index a1395ec02c24..3e355f857363 100644
@@ -5,8 +5,15 @@
 #include <linux/types.h>
 #include <linux/spinlock.h>
 
+/* Forward declarations */
+struct swap_info_struct;
+
 extern spinlock_t swap_tier_lock;
 
+#define TIER_ALL_MASK		(~0)
+#define TIER_DEFAULT_IDX	(31)
+#define TIER_DEFAULT_MASK	(1U << TIER_DEFAULT_IDX)
+
 /* Initialization and application */
 void swap_tiers_init(void);
 ssize_t swap_tiers_sysfs_show(char *buf);
@@ -16,5 +23,9 @@ int swap_tiers_remove(const char *name);
 
 void swap_tiers_snapshot(void);
 void swap_tiers_snapshot_restore(void);
-bool swap_tiers_validate(void);
+bool swap_tiers_update(void);
+
+/* Tier assignment */
+void swap_tiers_assign_dev(struct swap_info_struct *swp);
+
 #endif /* _SWAP_TIER_H */
mm/swapfile.c +2 -0
index 114e5d943de8..d5abc831cde7 100644
@@ -2914,6 +2914,8 @@ static void _enable_swap_info(struct swap_info_struct *si)
 
 	/* Add back to available list */
 	add_to_avail_list(si, true);
+
+	swap_tiers_assign_dev(si);
 }
 
 /*
Signature · 1 line
2.34.1
reviewer Reviewed-by on v6 2/4
#8 lore raw

Re: [PATCH v6 2/4] mm: swap: associate swap devices with tiers

Quoted 20 lines · Youngjun Park

This patch connects swap devices to the swap tier infrastructure, ensuring that devices are correctly assigned to tiers based on their priority.

A tier_mask is added to identify the tier membership of swap devices. Although tier-based allocation logic is not yet implemented, this mapping is necessary to track which tier a device belongs to. Upon activation, the device is assigned to a tier by matching its priority against the configured tier ranges.

The infrastructure allows dynamic modification of tiers, such as splitting or merging ranges. These operations are permitted provided that the tier assignment of already configured swap devices remains unchanged.

This patch also adds the documentation for the swap tier feature, covering the core concepts, sysfs interface usage, and configuration details.

Quoted 11 lines
---
 Documentation/mm/index.rst     |   1 +
 Documentation/mm/swap-tier.rst | 159 +++++++++++++++++++++++++++++++++
 MAINTAINERS                    |   1 +
 include/linux/swap.h           |   1 +
 mm/swap_state.c                |   2 +-
 mm/swap_tier.c                 | 101 ++++++++++++++++++---
 mm/swap_tier.h                 |  13 ++-
 mm/swapfile.c                  |   2 +
 8 files changed, 266 insertions(+), 14 deletions(-)
 create mode 100644 Documentation/mm/swap-tier.rst

LGTM,

author reviewer
#9 lore raw

PATCH v6 3/4 mm: memcontrol: add interfaces for swap tier selection

In reply to Youngjun Park

Integrate swap tier infrastructure with cgroup to allow selecting specific swap devices per cgroup.

Introduce memory.swap.tiers for configuring allowed tiers, and memory.swap.tiers.effective for exposing the effective tiers. The effective tiers are the intersection of the configured tiers and the parent's effective tiers.

Note that cgroups do not pin swap tiers, similar to cpuset and CPU hotplug, allowing configuration changes regardless of usage.

---
 Documentation/admin-guide/cgroup-v2.rst |  29 +++++++
 include/linux/memcontrol.h              |   5 ++
 mm/memcontrol.c                         |  96 +++++++++++++++++++++
 mm/swap_state.c                         |   5 +-
 mm/swap_tier.c                          | 108 +++++++++++++++++++++++-
 mm/swap_tier.h                          |  56 ++++++++++--
 6 files changed, 289 insertions(+), 10 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +29 -0
index 8ad0b2781317..823247f5c72c 100644
@@ -1850,6 +1850,35 @@ The following nested keys are defined.
 	Swap usage hard limit.  If a cgroup's swap usage reaches this
 	limit, anonymous memory of the cgroup will not be swapped out.
 
+  memory.swap.tiers
+        A read-write file which exists on non-root cgroups.
+        Format is similar to cgroup.subtree_control.
+
+        Controls which swap tiers this cgroup is allowed to swap
+        out to. All tiers are enabled by default.
+
+        ::
+
+            (-|+)TIER [(-|+)TIER ...]
+
+        "-" disables a tier, "+" re-enables it.
+        Entries are whitespace-delimited.
+
+        Changes here are combined with parent restrictions to
+        compute memory.swap.tiers.effective.
+
+        If a tier is removed from /sys/kernel/mm/swap/tiers,
+        any prior disable for that tier is invalidated.
+
+  memory.swap.tiers.effective
+        A read-only file which exists on non-root cgroups.
+
+        Shows the tiers this cgroup can actually swap out to.
+        This is the intersection of the parent's effective tiers
+        and this cgroup's own memory.swap.tiers configuration.
+        A child cannot enable a tier that is disabled in its
+        parent.
+
   memory.swap.events
 	A read-only flat-keyed file which exists on non-root cgroups.
 	The following entries are defined.  Unless specified
include/linux/memcontrol.h +5 -0
index dc3fa687759b..4ce1de14a026 100644
@@ -286,6 +286,11 @@ struct mem_cgroup {
 	struct lru_gen_mm_list mm_list;
 #endif
 
+#ifdef CONFIG_SWAP
+	int tier_mask;
+	int tier_effective_mask;
+#endif
+
 #ifdef CONFIG_MEMCG_V1
 	/* Legacy consumer-oriented counters */
 	struct page_counter kmem;		/* v1 only */
mm/memcontrol.c +96 -0

120 lines not on this page · raw message

mm/swap_state.c +3 -2
index 182e6d6c9c89..cc31c59473f7 100644
@@ -938,6 +938,7 @@ static ssize_t tiers_store(struct kobject *kobj,
 	char *p, *token, *name, *tmp;
 	int ret = 0;
 	short prio;
+	int mask = 0;
 
 	tmp = kstrdup(buf, GFP_KERNEL);
 	if (!tmp)
@@ -970,7 +971,7 @@ static ssize_t tiers_store(struct kobject *kobj,
 				goto restore;
 			break;
 		case '-':
-			ret = swap_tiers_remove(token + 1);
+			ret = swap_tiers_remove(token + 1, &mask);
 			if (ret)
 				goto restore;
 			break;
@@ -980,7 +981,7 @@ static ssize_t tiers_store(struct kobject *kobj,
 		}
 	}
 
-	if (!swap_tiers_update()) {
+	if (!swap_tiers_update(mask)) {
 		ret = -EINVAL;
 		goto restore;
 	}
mm/swap_tier.c +106 -2

132 lines not on this page · raw message

mm/swap_tier.h +50 -6

72 lines not on this page · raw message

Signature · 1 line
2.34.1
author reviewer
#10 lore raw

Re: [PATCH v6 3/4] mm: memcontrol: add interfaces for swap tier selection

On Tue, Apr 21, 2026 at 02:53:22PM +0900, Youngjun Park wrote:

There is one review from sashiko on this 3rd patch.
https://sashiko.dev/#/patchset/[email protected]?part=3

The concern about delegation and the global lock is valid in theory. However, I do not think a change is necessary.

memory.swap.tiers is writable only in delegated subtrees, and delegation itself is an administrative decision.

memory.swap.tiers is a configuration-time operation and not expected to be updated frequently. Contention on swap_tier_lock would happen in a configuration context (tier setup/update), not in a critical runtime path. Therefore, it is unlikely to have a significant impact on normal system operation.

Avoiding this would require computing the mask in the runtime path, potentially on every I/O .Keeping the cost in the rare setup path seems preferable.

Best regards,
Youngjun Park

reviewer
#11 lore raw

Re: [PATCH v6 3/4] mm: memcontrol: add interfaces for swap tier selection

In reply to Youngjun Park

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

Quoted 62 lines
mm/memcontrol.c +40 -0
index c3d98ab41f1f..0f67572e5e3e 100644
@@ -68,6 +68,7 @@
 #include <net/ip.h>
 #include "slab.h"
 #include "memcontrol-v1.h"
+#include "swap_tier.h"
 
 #include <linux/uaccess.h>
 
@@ -4130,6 +4131,8 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
 	refcount_set(&memcg->id.ref, 1);
 	css_get(css);
 
+	swap_tiers_memcg_inherit_mask(memcg);
+
 	/*
 	 * Ensure mem_cgroup_from_private_id() works once we're fully online.
 	 *
@@ -5667,6 +5670,88 @@ static int swap_events_show(struct seq_file *m, void *v)
 	return 0;
 }
 
+static int swap_tier_show(struct seq_file *m, void *v)
+{
+	struct mem_cgroup *memcg = mem_cgroup_from_seq(m);
+
+	swap_tiers_mask_show(m, READ_ONCE(memcg->tier_mask));
+	return 0;
+}
+
+static ssize_t swap_tier_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off)
+{
+	struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+	char *pos, *token;
+	int ret = 0;
+	int original_mask = 0;
+
+	pos = strstrip(buf);
+
+	spin_lock(&swap_tier_lock);
+	if (!*pos) {
+		WRITE_ONCE(memcg->tier_mask, TIER_ALL_MASK);
+		goto sync;
+	}
+
+	original_mask = memcg->tier_mask;
+
+	while ((token = strsep(&pos, " \t\n")) != NULL) {
+		int mask;
+
+		if (!*token)
+			continue;
+
+		if (token[0] != '-' && token[0] != '+') {
+			ret = -EINVAL;
+			goto err;
+		}
+

mm/memcontrol.c static int swap_events_show(struct seq_file *m, void *v) · line 5719

Quoted 10 lines
mm/memcontrol.c +10 -0
+		mask = swap_tiers_mask_lookup(token+1);
+		if (!mask) {
+			ret = -EINVAL;
+			goto err;
+		}
+
+		/*
+		 * if child already set, cannot add that tiers for hierarch mismatching.
+		 * parent compatible, child must respect parent selected swap device.
+		 */

This paragraph of code comment sounds a little unnatural. We are writing it into memcg, the child memcg is handled in swap_tiers_memcg_sync_mask(), isn't it? I don't get the 2nd sentence. Could you help explain?

Quoted 27 lines
mm/memcontrol.c +27 -0
+		switch (token[0]) {
+		case '-':
+			WRITE_ONCE(memcg->tier_mask,
+				   memcg->tier_mask & ~mask);
+			break;
+		case '+':
+			WRITE_ONCE(memcg->tier_mask,
+				   memcg->tier_mask | mask);
+			break;
+		default:
+			ret = -EINVAL;
+			break;
+		}
+
+		if (ret)
+			goto err;
+	}
+
+sync:
+	swap_tiers_memcg_sync_mask(memcg);
+err:
+	if (ret)
+		WRITE_ONCE(memcg->tier_mask, original_mask);
+	spin_unlock(&swap_tier_lock);
+	return ret ? ret : nbytes;
+}
+

...snip...

author reviewer
#12 lore raw

Re: [PATCH v6 3/4] mm: memcontrol: add interfaces for swap tier selection

Quoted 71 lines · Baoquan He

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

Quoted 69 lines
mm/memcontrol.c +47 -0
index c3d98ab41f1f..0f67572e5e3e 100644
@@ -68,6 +68,7 @@
 #include <net/ip.h>
 #include "slab.h"
 #include "memcontrol-v1.h"
+#include "swap_tier.h"
 
 #include <linux/uaccess.h>
 
@@ -4130,6 +4131,8 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css)
 	refcount_set(&memcg->id.ref, 1);
 	css_get(css);
 
+	swap_tiers_memcg_inherit_mask(memcg);
+
 	/*
 	 * Ensure mem_cgroup_from_private_id() works once we're fully online.
 	 *
@@ -5667,6 +5670,88 @@ static int swap_events_show(struct seq_file *m, void *v)
 	return 0;
 }
 
+static int swap_tier_show(struct seq_file *m, void *v)
+{
+	struct mem_cgroup *memcg = mem_cgroup_from_seq(m);
+
+	swap_tiers_mask_show(m, READ_ONCE(memcg->tier_mask));
+	return 0;
+}
+
+static ssize_t swap_tier_write(struct kernfs_open_file *of,
+				char *buf, size_t nbytes, loff_t off)
+{
+	struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+	char *pos, *token;
+	int ret = 0;
+	int original_mask = 0;
+
+	pos = strstrip(buf);
+
+	spin_lock(&swap_tier_lock);
+	if (!*pos) {
+		WRITE_ONCE(memcg->tier_mask, TIER_ALL_MASK);
+		goto sync;
+	}
+
+	original_mask = memcg->tier_mask;
+
+	while ((token = strsep(&pos, " \t\n")) != NULL) {
+		int mask;
+
+		if (!*token)
+			continue;
+
+		if (token[0] != '-' && token[0] != '+') {
+			ret = -EINVAL;
+			goto err;
+		}
+
+		mask = swap_tiers_mask_lookup(token+1);
+		if (!mask) {
+			ret = -EINVAL;
+			goto err;
+		}
+
+		/*
Quoted 8 lines

mm/memcontrol.c static int swap_events_show(struct seq_file *m, void *v) · line 5719

Quoted 3 lines
mm/memcontrol.c +3 -0
+		 * if child already set, cannot add that tiers for hierarch mismatching.
+		 * parent compatible, child must respect parent selected swap device.
+		 */

This paragraph of code comment sounds a little unnatural. We are writing it into memcg, the child memcg is handled in swap_tiers_memcg_sync_mask(), isn't it? I don't get the 2nd sentence. Could you help explain?

Thanks for the review.

I think this sentence is not appropriate.
Maybe I forgot to remove this line after modifying the code many times.

This sentence is not correct.

I explain one by one for clarification.

"if child already set, cannot add that tiers for hierarch mismatching."

mask can be changed (can change the tier_mask).
But the effective one (effective_mask) is recomputed with parent's effective_mask.

"parent compatible, child must respect parent selected swap device"

effective_mask must be restricted, which means child effective_mask must be subset of parent's effective_mask.

reviewer
#13 lore raw

Re: [PATCH v6 3/4] mm: memcontrol: add interfaces for swap tier selection

In reply to Youngjun Park

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

Quoted 4 lines
mm/swap_tier.c +0 -0
index 9019b8978770..b53d6cc67d1e 100644

...snip...

mm/swap_tier.c bool swap_tiers_update(void) · line 413

Quoted 11 lines
mm/swap_tier.c +7 -0
@@ -374,6 +404,80 @@ bool swap_tiers_update(void)
 			break;
 		swap_tiers_assign_dev(swp);
 	}
+	/*
+	 * XXX: Unused tiers default to ON, disabled after next tier added.
+	 * Use removed tier mask to clear settings for removed/re-added tiers.
+	 * (Could hold tier refs, but better to keep cgroup config independent)
+	 */
+	if (mask)
+		swap_tier_memcg_propagate(mask);

Code is neat, and high efficiency, while it's not easy to understand. I just sat in front of my computer the whole day yesterday to recall and understand why it's done like this, even though I made it clear to my self in the past. I think more words would be helpful to decrease the difficulty for people to undersntand it. E.g below two paragraphes. Just a suggestion and for your reference.

mm/swap_tier.c +11 -3
index b53d6cc67d1e..0a6adf14ab91 100644
@@ -405,9 +405,17 @@ bool swap_tiers_update(int mask)
 		swap_tiers_assign_dev(swp);
 	}
 	/*
-	 * XXX: Unused tiers default to ON, disabled after next tier added.
-	 * Use removed tier mask to clear settings for removed/re-added tiers.
-	 * (Could hold tier refs, but better to keep cgroup config independent)
+	 * When a tier is removed, its index (bit position in the mask) becomes
+	 * free for reassignment to a future tier.  If a memcg had previously
+	 * disabled this tier (cleared the bit in its swap.tiers file), the
+	 * effective mask would keep that bit clear -- meaning the new tier at
+	 * the same index would be silently unavailable, an invisible cgroup
+	 * constraint left behind by a tier that no longer exists.
+	 *
+	 * To prevent this, OR the removed tier's mask bit into every memcg's
+	 * tier_mask and tier_effective_mask.  This resets the bit so the new
+	 * tier is accessible by default; users who want to restrict it must
+	 * explicitly disable it after the tier is re-created.
 	 */
 	if (mask)
 		swap_tier_memcg_propagate(mask);
Quoted 4 lines
mm/swap_tier.c +1 -0
 	return true;
 }
+

...snip...

author reviewer
#14 lore raw

Re: [PATCH v6 3/4] mm: memcontrol: add interfaces for swap tier selection

Quoted 17 lines · Baoquan He

On 04/21/26 at 02:53pm, Youngjun Park wrote:
...snip...

Quoted 4 lines
mm/swap_tier.c +0 -0
index 9019b8978770..b53d6cc67d1e 100644

...snip...

Quoted 10 lines
mm/swap_tier.c +6 -0
@@ -374,6 +404,80 @@ bool swap_tiers_update(void)
 			break;
 		swap_tiers_assign_dev(swp);
 	}
+	/*
+	 * XXX: Unused tiers default to ON, disabled after next tier added.
+	 * Use removed tier mask to clear settings for removed/re-added tiers.
+	 * (Could hold tier refs, but better to keep cgroup config independent)
+	 */
+	if (mask)
Quoted 8 lines

mm/swap_tier.c bool swap_tiers_update(void) · line 413

Quoted 1 line
mm/swap_tier.c +1 -0
+		swap_tier_memcg_propagate(mask);

Code is neat, and high efficiency, while it's not easy to understand. I just sat in front of my computer the whole day yesterday to recall and understand why it's done like this, even though I made it clear to my self in the past. I think more words would be helpful to decrease the difficulty for people to undersntand it. E.g below two paragraphes. Just a suggestion and for your reference.

Thanks for the review again :)

Okay I see.
This explanation seems heavily abbreviated.

Quoted 13 lines
mm/swap_tier.c +2 -3
index b53d6cc67d1e..0a6adf14ab91 100644
@@ -405,9 +405,17 @@ bool swap_tiers_update(int mask)
 		swap_tiers_assign_dev(swp);
 	}
 	/*
-	 * XXX: Unused tiers default to ON, disabled after next tier added.
-	 * Use removed tier mask to clear settings for removed/re-added tiers.
-	 * (Could hold tier refs, but better to keep cgroup config independent)
+	 * When a tier is removed, its index (bit position in the mask) becomes
+	 * free for reassignment to a future tier.  If a memcg had previously

mm/swap_tier.c bool swap_tiers_update(int mask) · line 419

Quoted 10 lines
mm/swap_tier.c +9 -0
+	 * disabled this tier (cleared the bit in its swap.tiers file), the
+	 * effective mask would keep that bit clear -- meaning the new tier at
+	 * the same index would be silently unavailable, an invisible cgroup
+	 * constraint left behind by a tier that no longer exists.
+	 *
+	 * To prevent this, OR the removed tier's mask bit into every memcg's
+	 * tier_mask and tier_effective_mask.  This resets the bit so the new
+	 * tier is accessible by default; users who want to restrict it must
+	 * explicitly disable it after the tier is re-created.
 	 */

The explanation seems good.

it explains why we must clear the "disabled bit", when a new tier comes and we don't automatically diable, it can implicitly disable "new tier" which user might not want.

This comes from that fact we don't have a cgroup refecounting on the tier land. it is the corner case, we must sync for correct behavior.

I will change the comments as like it is.

author reviewer
#15 lore raw

PATCH v6 4/4 mm: swap: filter swap allocation by memcg tier mask

In reply to Youngjun Park

Apply memcg tier effective mask during swap slot allocation to enforce per-cgroup swap tier restrictions.

In the fast path, check the percpu cached swap_info's tier_mask against the folio's effective mask. If it does not match, fall through to the slow path. In the slow path, skip swap devices whose tier_mask is not covered by the folio's effective mask.

This works correctly when there is only one non-rotational device in the system and no devices share the same priority. However, there are known limitations:

- When non-rotational devices are distributed across multiple tiers, and different memcgs are configured to use those distinct tiers, they may constantly overwrite the shared percpu swap cache. This cache thrashing leads to frequent fast path misses.

- Combined with the above issue, if same-priority devices exist among them, a percpu cache miss (overwritten by another memcg) forces the allocator to round-robin to the next device prematurely, even if the current cluster is not fully exhausted.

These edge cases do not affect the primary use case of directing swap traffic per cgroup. Further optimization is planned for future work.

---
 mm/swapfile.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
mm/swapfile.c +12 -1
index d5abc831cde7..8734e5d26b08 100644
@@ -1352,15 +1352,22 @@ static bool swap_alloc_fast(struct folio *folio)
 	struct swap_cluster_info *ci;
 	struct swap_info_struct *si;
 	unsigned int offset;
+	int mask = folio_tier_effective_mask(folio);
 
 	/*
 	 * Once allocated, swap_info_struct will never be completely freed,
 	 * so checking it's liveness by get_swap_device_info is enough.
 	 */
 	si = this_cpu_read(percpu_swap_cluster.si[order]);
+	if (!si || !swap_tiers_mask_test(si->tier_mask, mask) ||
+		!get_swap_device_info(si))
+		return false;
+
 	offset = this_cpu_read(percpu_swap_cluster.offset[order]);
-	if (!si || !offset || !get_swap_device_info(si))
+	if (!offset) {
+		put_swap_device(si);
 		return false;
+	}
 
 	ci = swap_cluster_lock(si, offset);
 	if (cluster_is_usable(ci, order)) {
@@ -1379,10 +1386,14 @@ static bool swap_alloc_fast(struct folio *folio)
 static void swap_alloc_slow(struct folio *folio)
 {
 	struct swap_info_struct *si, *next;
+	int mask = folio_tier_effective_mask(folio);
 
 	spin_lock(&swap_avail_lock);
 start_over:
 	plist_for_each_entry_safe(si, next, &swap_avail_head, avail_list) {
+		if (!swap_tiers_mask_test(si->tier_mask, mask))
+			continue;
+
 		/* Rotate the device and switch to a new cluster */
 		plist_requeue(&si->avail_list, &swap_avail_head);
 		spin_unlock(&swap_avail_lock);
Signature · 1 line
2.34.1
author reviewer
#16 lore raw

Re: [PATCH v6 4/4] mm: swap: filter swap allocation by memcg tier mask

On Tue, Apr 21, 2026 at 02:53:23PM +0900, Youngjun Park wrote:

Another low impact review from sashiko
https://sashiko.dev/#/patchset/[email protected]?part=4

Proper concern,
But .. it is also not needed to modify. just small optimization of code flow. If somebody thinks differently, happy to apply it. :)

Best regards,
Youngjun Park

reviewer
#17 lore raw

Re: [PATCH v6 4/4] mm: swap: filter swap allocation by memcg tier mask

In reply to Youngjun Park
Quoted 48 lines · Youngjun Park

Apply memcg tier effective mask during swap slot allocation to enforce per-cgroup swap tier restrictions.

In the fast path, check the percpu cached swap_info's tier_mask against the folio's effective mask. If it does not match, fall through to the slow path. In the slow path, skip swap devices whose tier_mask is not covered by the folio's effective mask.

This works correctly when there is only one non-rotational device in the system and no devices share the same priority. However, there are known limitations:

- When non-rotational devices are distributed across multiple tiers, and different memcgs are configured to use those distinct tiers, they may constantly overwrite the shared percpu swap cache. This cache thrashing leads to frequent fast path misses.

- Combined with the above issue, if same-priority devices exist among them, a percpu cache miss (overwritten by another memcg) forces the allocator to round-robin to the next device prematurely, even if the current cluster is not fully exhausted.

These edge cases do not affect the primary use case of directing swap traffic per cgroup. Further optimization is planned for future work.

---
 mm/swapfile.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
mm/swapfile.c +1 -0
index d5abc831cde7..8734e5d26b08 100644
@@ -1352,15 +1352,22 @@ static bool swap_alloc_fast(struct folio *folio)
 	struct swap_cluster_info *ci;
 	struct swap_info_struct *si;
 	unsigned int offset;
+	int mask = folio_tier_effective_mask(folio);
 
 	/*
 	 * Once allocated, swap_info_struct will never be completely freed,
 	 * so checking it's liveness by get_swap_device_info is enough.
 	 */
 	si = this_cpu_read(percpu_swap_cluster.si[order]);

mm/swapfile.c static bool swap_alloc_fast(struct folio *folio) · line 1370

Quoted 10 lines
mm/swapfile.c +7 -1
+	if (!si || !swap_tiers_mask_test(si->tier_mask, mask) ||
+		!get_swap_device_info(si))
+		return false;
+
 	offset = this_cpu_read(percpu_swap_cluster.offset[order]);
-	if (!si || !offset || !get_swap_device_info(si))
+	if (!offset) {
+		put_swap_device(si);
 		return false;
+	}

The whole patch looks good to me except of one nitpick. Is it a lille cleaner with below tiny adjustment?

mm/swapfile.c +2 -5
index 2864cd8c2da9..cdf453bf6b80 100644
@@ -1359,15 +1359,12 @@ static bool swap_alloc_fast(struct folio *folio)
 	 * so checking it's liveness by get_swap_device_info is enough.
 	 */
 	si = this_cpu_read(percpu_swap_cluster.si[order]);
-	if (!si || !swap_tiers_mask_test(si->tier_mask, mask) ||
-		!get_swap_device_info(si))
+	if (!si || !swap_tiers_mask_test(si->tier_mask, mask))
 		return false;
 
 	offset = this_cpu_read(percpu_swap_cluster.offset[order]);
-	if (!offset) {
-		put_swap_device(si);
+	if (!offset || !get_swap_device_info(si))
 		return false;
-	}
 
 	ci = swap_cluster_lock(si, offset);
 	if (cluster_is_usable(ci, order)) {
Quoted 21 lines
mm/swapfile.c +4 -0
 	ci = swap_cluster_lock(si, offset);
 	if (cluster_is_usable(ci, order)) {
@@ -1379,10 +1386,14 @@ static bool swap_alloc_fast(struct folio *folio)
 static void swap_alloc_slow(struct folio *folio)
 {
 	struct swap_info_struct *si, *next;
+	int mask = folio_tier_effective_mask(folio);
 
 	spin_lock(&swap_avail_lock);
 start_over:
 	plist_for_each_entry_safe(si, next, &swap_avail_head, avail_list) {
+		if (!swap_tiers_mask_test(si->tier_mask, mask))
+			continue;
+
 		/* Rotate the device and switch to a new cluster */
 		plist_requeue(&si->avail_list, &swap_avail_head);
 		spin_unlock(&swap_avail_lock);
Signature · 1 line
2.34.1
author reviewer
#18 lore raw

Re: [PATCH v6 4/4] mm: swap: filter swap allocation by memcg tier mask

Quoted 72 lines · Baoquan He
Quoted 48 lines · Youngjun Park

Apply memcg tier effective mask during swap slot allocation to enforce per-cgroup swap tier restrictions.

In the fast path, check the percpu cached swap_info's tier_mask against the folio's effective mask. If it does not match, fall through to the slow path. In the slow path, skip swap devices whose tier_mask is not covered by the folio's effective mask.

This works correctly when there is only one non-rotational device in the system and no devices share the same priority. However, there are known limitations:

- When non-rotational devices are distributed across multiple tiers, and different memcgs are configured to use those distinct tiers, they may constantly overwrite the shared percpu swap cache. This cache thrashing leads to frequent fast path misses.

- Combined with the above issue, if same-priority devices exist among them, a percpu cache miss (overwritten by another memcg) forces the allocator to round-robin to the next device prematurely, even if the current cluster is not fully exhausted.

These edge cases do not affect the primary use case of directing swap traffic per cgroup. Further optimization is planned for future work.

---
 mm/swapfile.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
mm/swapfile.c +1 -0
index d5abc831cde7..8734e5d26b08 100644
@@ -1352,15 +1352,22 @@ static bool swap_alloc_fast(struct folio *folio)
 	struct swap_cluster_info *ci;
 	struct swap_info_struct *si;
 	unsigned int offset;
+	int mask = folio_tier_effective_mask(folio);
 
 	/*
 	 * Once allocated, swap_info_struct will never be completely freed,
 	 * so checking it's liveness by get_swap_device_info is enough.
 	 */
 	si = this_cpu_read(percpu_swap_cluster.si[order]);

mm/swapfile.c static bool swap_alloc_fast(struct folio *folio) · line 1370

Quoted 10 lines
mm/swapfile.c +7 -1
+	if (!si || !swap_tiers_mask_test(si->tier_mask, mask) ||
+		!get_swap_device_info(si))
+		return false;
+
 	offset = this_cpu_read(percpu_swap_cluster.offset[order]);
-	if (!si || !offset || !get_swap_device_info(si))
+	if (!offset) {
+		put_swap_device(si);
 		return false;
+	}

The whole patch looks good to me except of one nitpick. Is it a lille cleaner with below tiny adjustment?

mm/swapfile.c +0 -1
index 2864cd8c2da9..cdf453bf6b80 100644
@@ -1359,15 +1359,12 @@ static bool swap_alloc_fast(struct folio *folio)
 	 * so checking it's liveness by get_swap_device_info is enough.
 	 */
 	si = this_cpu_read(percpu_swap_cluster.si[order]);
-	if (!si || !swap_tiers_mask_test(si->tier_mask, mask) ||

mm/swapfile.c static bool swap_alloc_fast(struct folio *folio) · line 1370

Quoted 10 lines
mm/swapfile.c +2 -4
-		!get_swap_device_info(si))
+	if (!si || !swap_tiers_mask_test(si->tier_mask, mask))
 		return false;
 
 	offset = this_cpu_read(percpu_swap_cluster.offset[order]);
-	if (!offset) {
-		put_swap_device(si);
+	if (!offset || !get_swap_device_info(si))
 		return false;
-	}

Thanks!

Your suggested version of code is simpler than privious one. I will apply it.