mm/zswap: Implement per-cgroup proactive writeback

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

Quoted 15 lines · Nhat Pham

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 4 lines

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Quoted 4 lines

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

+1, why do we need to specifically proactively reclaim the compressed memory?

Also, if we do need to minimize the compressed memory and force higher writeback rates, we can do so with memory.zswap.max, right?

Quoted 16 lines · Yosry Ahmed

Quoted 15 lines · Nhat Pham

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 4 lines

Quoted 4 lines

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Thanks Nhat, Yosry — let me address both comments together.

Quoted 5 lines

Quoted 4 lines

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

Apologies for the lack of detailed explanation in the patch description, which led to the confusion.

While we are already utilizing memory.reclaim, it does not fully address our requirements.

Our deployment runs a userspace proactive reclaimer that drives memory.reclaim based on the system's runtime state (memory/CPU/IO pressure, refault rate, ...) and workload-specific
policy. That first stage compresses cold anon pages into zswap. Entries that then remain in zswap past a policy-defined age threshold are considered "twice cold", and the reclaimer wants
to write them back to the backing swap device at a moment of its own choosing, to further reclaim the DRAM still held by the compressed data.

This is the "second-level offloading" pattern described in Meta's TMO paper [1]. zswap proactive writeback is what this series introduces to address that second-level offloading stage.

[1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf

Quoted 6 lines

+1, why do we need to specifically proactively reclaim the compressed memory?

Also, if we do need to minimize the compressed memory and force higher writeback rates, we can do so with memory.zswap.max, right?

Here are a few reasons why memory.zswap.max is not enough:

1. Writing memory.zswap.max itself does not trigger any writeback immediately. For a memcg that has reached steady state (on which the userspace reclaimer is no longer invoking
memory.reclaim), after enough time has passed, the reclaimer has no good way to trigger proactive writeback for second-level offloading by lowering memory.zswap.max, because in steady
state nothing drives the zswap_store() -> shrink_memcg() path. The userspace reclaimer still has no control over when proactive writeback happens.

2. memory.zswap.max currently triggers zswap writeback via zswap_store() -> shrink_memcg(), and each over-limit event can write back at most NR_NODES entries. If zswap residency is far
above memory.zswap.max, converging to the target size requires at least O(over-limit pages / NR_NODES) zswap_store() events, with no batching — proactive writeback therefore has
significant latency.

3. memory.zswap.max is a stateful interface. If the userspace reclaimer crashes for any reason mid-operation, it may leave memory.zswap.max at some set value, putting the application in a
persistently throttled bad state.

4. Once the userspace reclaimer has lowered memory.zswap.max, if the workload is rapidly expanding and triggers memory reclaim via memory.high / kswapd / etc., the actual amount written back can exceed what was intended.

Thanks,
Hao

Quoted 48 lines · Hao Jia

Quoted 16 lines · Yosry Ahmed

Quoted 15 lines · Nhat Pham

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 4 lines

Quoted 4 lines

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Thanks Nhat, Yosry — let me address both comments together.

Quoted 5 lines

Quoted 4 lines

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

Apologies for the lack of detailed explanation in the patch description, which led to the confusion.

While we are already utilizing memory.reclaim, it does not fully address our requirements.

Our deployment runs a userspace proactive reclaimer that drives memory.reclaim based on the system's runtime state (memory/CPU/IO pressure, refault rate, ...) and workload-specific
policy. That first stage compresses cold anon pages into zswap. Entries that then remain in zswap past a policy-defined age threshold are considered "twice cold", and the reclaimer wants
to write them back to the backing swap device at a moment of its own choosing, to further reclaim the DRAM still held by the compressed data.

Quoted 5 lines

This is the "second-level offloading" pattern described in Meta's TMO paper [1]. zswap proactive writeback is what this series introduces to address that second-level offloading stage.

[1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf

Yeah that's what we've been trying to work on as well :) We are working on a couple of improvements to the mechanism side of this path (cc Alex) - hopefully it will help your use case too!

Anyway, back to my original inquiry: I understand your use case. It's pretty similar to our goal. What I'm not getting is why is memory.reclaim (which you already use) not sufficient for zswap -> disk swap offloading too?

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Furthermore, we already have a way to detect the "twice cold" entries you mentioned: the referenced bit. This is analogous to the way we treat uncompressed pages.

Quoted 32 lines

Quoted 6 lines

+1, why do we need to specifically proactively reclaim the compressed memory?

Also, if we do need to minimize the compressed memory and force higher writeback rates, we can do so with memory.zswap.max, right?

Here are a few reasons why memory.zswap.max is not enough:

1. Writing memory.zswap.max itself does not trigger any writeback immediately. For a memcg that has reached steady state (on which the userspace reclaimer is no longer invoking
memory.reclaim), after enough time has passed, the reclaimer has no good way to trigger proactive writeback for second-level offloading by lowering memory.zswap.max, because in steady
state nothing drives the zswap_store() -> shrink_memcg() path. The userspace reclaimer still has no control over when proactive writeback happens.

2. memory.zswap.max currently triggers zswap writeback via zswap_store() -> shrink_memcg(), and each over-limit event can write back at most NR_NODES entries. If zswap residency is far
above memory.zswap.max, converging to the target size requires at least O(over-limit pages / NR_NODES) zswap_store() events, with no batching — proactive writeback therefore has
significant latency.

3. memory.zswap.max is a stateful interface. If the userspace reclaimer crashes for any reason mid-operation, it may leave memory.zswap.max at some set value, putting the application in a
persistently throttled bad state.

Quoted 4 lines

4. Once the userspace reclaimer has lowered memory.zswap.max, if the workload is rapidly expanding and triggers memory reclaim via memory.high / kswapd / etc., the actual amount written back can exceed what was intended.

One more reason: IIRC, when you set memory.zswap.max to a value other than 0 max, every zswap store incurs a pretty expensive check (obj_cgroup_may_zswap), which does a force flush (__mem_cgroup_flush_stats). That was pretty expensive last time some of our internal services played with it. So yeah, it's not ideal...

(if you're using this, might wanna profile this as well).

Quoted 3 lines

Thanks,
Hao

Quoted 5 lines

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Yes, the memory.reclaim path does drive some zswap writeback, but it is not enough for our case.

1. For a memcg that has reached steady state (a common case being when memory.current is below the policy target), the userspace reclaimer may not invoke memory.reclaim on it for a long time, and so no second-level offloading happens through memory.reclaim. In this state we want memory.zswap.proactive_writeback to write back entries that have sat in zswap past an age threshold, to further reclaim the DRAM still held by the compressed data.

2. Even when memory.reclaim is running, the fraction of zswap residency that ends up reaching the backing swap device is still very small for many of our workloads, and the userspace reclaimer has no way to participate in or control the granularity of zswap writeback. So in our deployment we prefer to leave the zswap shrinker disabled, decouple LRU -> zswap from zswap -> swap, and use a dedicated proactive-writeback interface that lifts the writeback policy into userspace where it can evolve independently of the kernel.

Quoted 19 lines

Quoted 6 lines

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Yes, the memory.reclaim path does drive some zswap writeback, but it is not enough for our case.

1. For a memcg that has reached steady state (a common case being when memory.current is below the policy target), the userspace reclaimer may not invoke memory.reclaim on it for a long time, and so no second-level offloading happens through memory.reclaim. In this state we want memory.zswap.proactive_writeback to write back entries that have sat in zswap past an age threshold, to further reclaim the DRAM still held by the compressed data.

Quoted 9 lines

2. Even when memory.reclaim is running, the fraction of zswap residency that ends up reaching the backing swap device is still very small for many of our workloads, and the userspace reclaimer has no way to participate in or control the granularity of zswap writeback. So in our deployment we prefer to leave the zswap shrinker disabled, decouple LRU -> zswap from zswap -> swap, and use a dedicated proactive-writeback interface that lifts the writeback policy into userspace where it can evolve independently of the kernel.

To be honest I see the point of proactively reclaiming compressed memory in zswap. If you use memory.reclaim, you are also reclaiming hotter memory in the process, and you are not necessarily getting as much writeback as you want. The memory in zswap is a more conservative choice for proactive reclaim because it's memory that's guaranteed to be cold(ish) and not being accessed.

That being said, the interface is not great any way you cut it :/

I don't like the 'memory.zswap.proactive_writeback' name, maybe we can stay consistent by doing 'memory.zswap.reclaim', but that just as easily reads as "reclaim using zswap". Maybe 'memory.zswap.do_writeback' or something, idk.

I also don't like having two proactive reclaim interfaces, so a voice in my head wants to tie this into 'memory.reclaim' somehow, but that includes adding a pretty specific argument (e.g. 'memory.reclaim zswap_writeback_only=1'.

I don't like any of these options, and we also need to consider what the memcg maintainers think. I see the use case of proactive writeback but I am struggling to come up with a clean interface.

I also think we should take the 'age' aspect out of the conversation for now, it can be a separate discussion. Well, unless we decide to tie it to memory.reclaim. If memory.reclaim broadly supports age-based reclaim then zswap writeback can be a natural part of that without requiring a specific interface.

Quoted 52 lines · Yosry Ahmed

Quoted 19 lines

Quoted 6 lines

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Yes, the memory.reclaim path does drive some zswap writeback, but it is not enough for our case.

1. For a memcg that has reached steady state (a common case being when memory.current is below the policy target), the userspace reclaimer may not invoke memory.reclaim on it for a long time, and so no second-level offloading happens through memory.reclaim. In this state we want memory.zswap.proactive_writeback to write back entries that have sat in zswap past an age threshold, to further reclaim the DRAM still held by the compressed data.

Quoted 9 lines

2. Even when memory.reclaim is running, the fraction of zswap residency that ends up reaching the backing swap device is still very small for many of our workloads, and the userspace reclaimer has no way to participate in or control the granularity of zswap writeback. So in our deployment we prefer to leave the zswap shrinker disabled, decouple LRU -> zswap from zswap -> swap, and use a dedicated proactive-writeback interface that lifts the writeback policy into userspace where it can evolve independently of the kernel.

To be honest I see the point of proactively reclaiming compressed memory in zswap. If you use memory.reclaim, you are also reclaiming hotter memory in the process, and you are not necessarily getting as much writeback as you want. The memory in zswap is a more conservative choice for proactive reclaim because it's memory that's guaranteed to be cold(ish) and not being accessed.

That being said, the interface is not great any way you cut it :/

I don't like the 'memory.zswap.proactive_writeback' name, maybe we can stay consistent by doing 'memory.zswap.reclaim', but that just as easily reads as "reclaim using zswap". Maybe 'memory.zswap.do_writeback' or something, idk.

I also don't like having two proactive reclaim interfaces, so a voice in my head wants to tie this into 'memory.reclaim' somehow, but that includes adding a pretty specific argument (e.g. 'memory.reclaim zswap_writeback_only=1'.

I don't like any of these options, and we also need to consider what the memcg maintainers think. I see the use case of proactive writeback but I am struggling to come up with a clean interface.

Quoted 5 lines

I also think we should take the 'age' aspect out of the conversation for now, it can be a separate discussion. Well, unless we decide to tie it to memory.reclaim. If memory.reclaim broadly supports age-based reclaim then zswap writeback can be a natural part of that without requiring a specific interface.

Yeah perhaps extending memory.reclaim is best... Sort of analogous to the way we have swappiness to balance file v.s anon....

Quoted 64 lines · Nhat Pham

Quoted 48 lines · Hao Jia

Quoted 16 lines · Yosry Ahmed

Quoted 15 lines · Nhat Pham

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 4 lines

Quoted 4 lines

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Thanks Nhat, Yosry — let me address both comments together.

Quoted 5 lines

Quoted 4 lines

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

Apologies for the lack of detailed explanation in the patch description, which led to the confusion.

While we are already utilizing memory.reclaim, it does not fully address our requirements.

Our deployment runs a userspace proactive reclaimer that drives memory.reclaim based on the system's runtime state (memory/CPU/IO pressure, refault rate, ...) and workload-specific
policy. That first stage compresses cold anon pages into zswap. Entries that then remain in zswap past a policy-defined age threshold are considered "twice cold", and the reclaimer wants
to write them back to the backing swap device at a moment of its own choosing, to further reclaim the DRAM still held by the compressed data.

Quoted 5 lines

This is the "second-level offloading" pattern described in Meta's TMO paper [1]. zswap proactive writeback is what this series introduces to address that second-level offloading stage.

[1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf

Yeah that's what we've been trying to work on as well :) We are working on a couple of improvements to the mechanism side of this path (cc Alex) - hopefully it will help your use case too!

Anyway, back to my original inquiry: I understand your use case. It's pretty similar to our goal. What I'm not getting is why is memory.reclaim (which you already use) not sufficient for zswap -> disk swap offloading too?

Quoted 5 lines

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Yes, the memory.reclaim path does drive some zswap writeback, but it is not enough for our case.

1. For a memcg that has reached steady state (a common case being when memory.current is below the policy target), the userspace reclaimer may not invoke memory.reclaim on it for a long time, and so no second-level offloading happens through memory.reclaim. In this state we want memory.zswap.proactive_writeback to write back entries that have sat in zswap past an age threshold, to further reclaim the DRAM still held by the compressed data.

2. Even when memory.reclaim is running, the fraction of zswap residency that ends up reaching the backing swap device is still very small for many of our workloads, and the userspace reclaimer has no way to participate in or control the granularity of zswap writeback. So in our deployment we prefer to leave the zswap shrinker disabled, decouple LRU -> zswap from zswap -> swap, and use a dedicated proactive-writeback interface that lifts the writeback policy into userspace where it can evolve independently of the kernel.

Quoted 86 lines · Hao Jia

Quoted 64 lines · Nhat Pham

Quoted 48 lines · Hao Jia

Quoted 16 lines · Yosry Ahmed

Quoted 15 lines · Nhat Pham

Quoted 13 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Quoted 4 lines

Quoted 4 lines

Quoted 3 lines

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Thanks Nhat, Yosry — let me address both comments together.

Quoted 5 lines

Quoted 4 lines

We already have memory.reclaim, no? Would that not work to create headroom generally for your use case? Is there a reason why we are treating zswap memory as special here?

Apologies for the lack of detailed explanation in the patch description, which led to the confusion.

While we are already utilizing memory.reclaim, it does not fully address our requirements.

Our deployment runs a userspace proactive reclaimer that drives memory.reclaim based on the system's runtime state (memory/CPU/IO pressure, refault rate, ...) and workload-specific
policy. That first stage compresses cold anon pages into zswap. Entries that then remain in zswap past a policy-defined age threshold are considered "twice cold", and the reclaimer wants
to write them back to the backing swap device at a moment of its own choosing, to further reclaim the DRAM still held by the compressed data.

Quoted 5 lines

This is the "second-level offloading" pattern described in Meta's TMO paper [1]. zswap proactive writeback is what this series introduces to address that second-level offloading stage.

[1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf

Yeah that's what we've been trying to work on as well :) We are working on a couple of improvements to the mechanism side of this path (cc Alex) - hopefully it will help your use case too!

Anyway, back to my original inquiry: I understand your use case. It's pretty similar to our goal. What I'm not getting is why is memory.reclaim (which you already use) not sufficient for zswap -> disk swap offloading too?

Quoted 5 lines

Zswap objects are organized into LRU and exposed to the shrinker interface. Echo-ing to memory.reclaim should also offload some zswap entries, correct? Are there still cold zswap entries that escape this, somehow?

Yes, the memory.reclaim path does drive some zswap writeback, but it is not enough for our case.

1. For a memcg that has reached steady state (a common case being when memory.current is below the policy target), the userspace reclaimer may not invoke memory.reclaim on it for a long time, and so no second-level offloading happens through memory.reclaim. In this state we want memory.zswap.proactive_writeback to write back entries that have sat in zswap past an age threshold, to further reclaim the DRAM still held by the compressed data.

Quoted 9 lines

2. Even when memory.reclaim is running, the fraction of zswap residency that ends up reaching the backing swap device is still very small for many of our workloads, and the userspace reclaimer has no way to participate in or control the granularity of zswap writeback. So in our deployment we prefer to leave the zswap shrinker disabled, decouple LRU -> zswap from zswap -> swap, and use a dedicated proactive-writeback interface that lifts the writeback policy into userspace where it can evolve independently of the kernel.

I see. It's interesting - we've been dealing with the opposite problems (reclaiming too much from zswap) that it's refreshing to see the other end of the spectrum :) We should invest more into this to see why we are not reclaiming enough, but I see the value of adding a knob to hit zswap exclusively.

Regarding age-based reclaim, I agree with Yosry here. Let us try to land an interface to do targeted reclaim on compressed memory first. I do see the value of age information: with it, you can track zswap entries ages and the distribution of refault ages, and only reclaim the tail. However, I wonder if you can just build a system that adapt the reclaim request size based on PSI, refault rate etc. similar to how you're adjusting memory.reclaim on uncompressed memories with a senpai-like system. Something along the line of - if we are swapping in too much from disk (or if IO pressure is high), back off, and if not, stealing a bit more from zswap pool (perhaps with a bigger step size), etc. Is there a reason why zswap cannot adopt a similar strategy?

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. This reactive approach offers no precise control over when writeback happens, which can disturb latency-sensitive workloads, and it cannot direct writeback at a specific memory cgroup. However, there are scenarios where users might want to proactively write back cold pages from zswap to the backing swap device, for example, to free up memory for other applications or to prepare for upcoming memory-intensive workloads.

Therefore, implement a proactive writeback mechanism for zswap by adding a new cgroup interface file memory.zswap.proactive_writeback within the memory controller.

Users can trigger writeback by writing to this file with the following parameters:
- max=<bytes>: The maximum amount of memory to write back (optional,
default: unlimited).
- <age>: The minimum age of the pages to write back. Only pages that have been in zswap for at least this duration will be written back.

Example usage:
# Write back pages older than 1 hour (3600 seconds), max 10MB echo "max=10M 3600" > memory.zswap.proactive_writeback

The implementation consists of:
1. Add store_time to struct zswap_entry to record when each entry was inserted into zswap, used for proactive writeback age comparison.
2. Introduce struct zswap_shrink_walk_arg, passed as the cb_arg to list_lru_walk_one() in both the shrinker and proactive paths. It carries the per-invocation cutoff_time and proactive flag down to shrink_memcg_cb(), and propagates the encountered_page_in_swapcache out-signal from the callback back to the caller.
3. Modify the callback function shrink_memcg_cb() to proactively writeback zswap_entries that meet the time threshold.
4. Add zswap_proactive_writeback() as the proactive writeback driver:
a per-node batched list_lru_walk_one() loop bounded by the writeback budget.

Signed-off-by Hao Jia <[email protected]>

---
 Documentation/admin-guide/cgroup-v2.rst |  24 ++++
 include/linux/zswap.h                   |   8 ++
 mm/memcontrol.c                         |  76 ++++++++++
 mm/zswap.c                              | 176 ++++++++++++++++++++++--
 4 files changed, 276 insertions(+), 8 deletions(-)

Documentation/admin-guide/cgroup-v2.rst +24 -0

index 6efd0095ed99..05b664b3b3e8 100644
@@ -1908,6 +1908,30 @@ The following nested keys are defined.
        This setting has no effect if zswap is disabled, and swapping
        is allowed unless memory.swap.max is set to 0.
 
+  memory.zswap.proactive_writeback
+       A write-only nested-keyed file which exists in non-root cgroups.
+
+       This interface allows proactive writeback of pages from the zswap
+       pool to the backing swap device. This is useful to offload cold
+       pages from the zswap pool to the slower swap device. It is only
+       available if zswap writeback is enabled.
+
+       Users can trigger writeback by writing to this file with the following
+       parameters:
+
+       - "max=<bytes>" : Optional. The maximum amount of data to write back.
+         (default: unlimited). Please note that the kernel can over or under
+         writeback this value.
+
+       - "<age>" : Required. The minimum age of the pages to write back
+         (in seconds). Only pages that have been in the zswap pool for at
+         least this amount of time will be written back.
+
+       Example::
+
+         # Write back pages older than 1 hour (3600 seconds), max 10MB
+         echo "max=10M 3600" > memory.zswap.proactive_writeback
+
   memory.pressure
        A read-only nested-keyed file.
 

include/linux/zswap.h +8 -0

index efa6b551217e..7a51b4f95017 100644
@@ -44,6 +44,8 @@ void zswap_lruvec_state_init(struct lruvec *lruvec);
 void zswap_folio_swapin(struct folio *folio);
 bool zswap_is_enabled(void);
 bool zswap_never_enabled(void);
+int zswap_proactive_writeback(struct mem_cgroup *root, unsigned long nr_max_writeback,
+                             ktime_t cutoff);
 #else
 
 struct zswap_lruvec_state {};

@@ -78,6 +80,12 @@ static inline bool zswap_never_enabled(void)
        return true;
 }
 
+static inline int zswap_proactive_writeback(struct mem_cgroup *root,
+                                           unsigned long nr_max_writeback, ktime_t cutoff)
+{
+       return 0;
+}
+
 #endif
 
 #endif /* _LINUX_ZSWAP_H */

mm/memcontrol.c +76 -0

index 409c41359dc8..ba7f7b1954a8 100644
@@ -70,6 +70,7 @@
 #include "memcontrol-v1.h"
 
 #include <linux/uaccess.h>
+#include <linux/parser.h>
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/memcg.h>

@@ -5891,6 +5892,76 @@ static ssize_t zswap_writeback_write(struct kernfs_open_file *of,
        return nbytes;
 }
 
+enum {
+       ZSWAP_WRITEBACK_MAX,
+       ZSWAP_WRITEBACK_AGE,
+       ZSWAP_WRITEBACK_ERR,
+};
+
+static const match_table_t zswap_writeback_tokens = {
+       { ZSWAP_WRITEBACK_MAX, "max=%s" },
+       { ZSWAP_WRITEBACK_AGE, "%u" },
+       { ZSWAP_WRITEBACK_ERR, NULL },
+};
+
+static ssize_t zswap_proactive_writeback_write(struct kernfs_open_file *of,
+                                              char *buf, size_t nbytes,
+                                              loff_t off)
+{
+       struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+       unsigned long nr_max_writeback = ULONG_MAX;
+       substring_t args[MAX_OPT_ARGS];
+       unsigned int age_sec;
+       bool age_set = false;
+       ktime_t cutoff_time;
+       char *token, *end;
+       int err;
+
+       if (!mem_cgroup_zswap_writeback_enabled(memcg))
+               return -EINVAL;
+
+       buf = strstrip(buf);
+
+       while ((token = strsep(&buf, " ")) != NULL) {
+               if (!strlen(token))
+                       continue;
+
+               switch (match_token(token, zswap_writeback_tokens, args)) {
+               case ZSWAP_WRITEBACK_MAX:
+                       nr_max_writeback = memparse(args[0].from, &end);
+                       if (*end != '\0')
+                               return -EINVAL;
+                       nr_max_writeback >>= PAGE_SHIFT;
+                       break;
+               case ZSWAP_WRITEBACK_AGE:
+                       if (age_set)
+                               return -EINVAL;
+
+                       if (match_uint(&args[0], &age_sec))
+                               return -EINVAL;
+                       age_set = true;
+                       break;
+               default:
+                       return -EINVAL;
+               }
+       }
+
+       if (!age_set || !age_sec || !nr_max_writeback)
+               return -EINVAL;
+
+       cutoff_time = ktime_sub(ktime_get_boottime(),
+                               ns_to_ktime((u64)age_sec * NSEC_PER_SEC));
+       /* age_sec >= uptime: no entry can be that old, skip the walk. */
+       if (ktime_to_ns(cutoff_time) <= 0)
+               return nbytes;
+
+       err = zswap_proactive_writeback(memcg, nr_max_writeback, cutoff_time);
+       if (err)
+               return err;
+
+       return nbytes;
+}
+
 static struct cftype zswap_files[] = {
        {
                .name = "zswap.current",

@@ -5908,6 +5979,11 @@ static struct cftype zswap_files[] = {
                .seq_show = zswap_writeback_show,
                .write = zswap_writeback_write,
        },
+       {
+               .name = "zswap.proactive_writeback",
+               .flags = CFTYPE_NOT_ON_ROOT,
+               .write = zswap_proactive_writeback_write,
+       },
        { }     /* terminate */
 };
 #endif /* CONFIG_ZSWAP */

mm/zswap.c +8 -0

index 19538d6f169a..1173ac6836fa 100644
@@ -36,6 +36,7 @@
 #include <linux/workqueue.h>
 #include <linux/list_lru.h>
 #include <linux/zsmalloc.h>
+#include <linux/timekeeping.h>
 
 #include "swap.h"
 #include "internal.h"

@@ -160,6 +161,12 @@ struct zswap_pool {
        char tfm_name[CRYPTO_MAX_ALG_NAME];
 };
 
+struct zswap_shrink_walk_arg {
+       ktime_t cutoff_time;
+       bool proactive;
+       bool encountered_page_in_swapcache;
+};
+
 /* Global LRU lists shared by all zswap pools. */
 static struct list_lru zswap_list_lru;
 

@@ -183,6 +190,7 @@ static struct shrinker *zswap_shrinker;
  * handle - zsmalloc allocation handle that stores the compressed page data
  * objcg - the obj_cgroup that the compressed memory is charged to
  * lru - handle to the pool's lru used to evict pages.
+ * store_time - Time when the entry was stored, for proactive writeback.
  */
 struct zswap_entry {
        swp_entry_t swpentry;

Quoted 34 lines · Hao Jia

mm/zswap.c +8 -0

index 19538d6f169a..1173ac6836fa 100644
@@ -36,6 +36,7 @@
  #include <linux/workqueue.h>
  #include <linux/list_lru.h>
  #include <linux/zsmalloc.h>
+#include <linux/timekeeping.h>
 
  #include "swap.h"
  #include "internal.h"

@@ -160,6 +161,12 @@ struct zswap_pool {
         char tfm_name[CRYPTO_MAX_ALG_NAME];
  };
 
+struct zswap_shrink_walk_arg {
+       ktime_t cutoff_time;
+       bool proactive;
+       bool encountered_page_in_swapcache;
+};
+
  /* Global LRU lists shared by all zswap pools. */
  static struct list_lru zswap_list_lru;
 

@@ -183,6 +190,7 @@ static struct shrinker *zswap_shrinker;
   * handle - zsmalloc allocation handle that stores the compressed page data
   * objcg - the obj_cgroup that the compressed memory is charged to
   * lru - handle to the pool's lru used to evict pages.
+ * store_time - Time when the entry was stored, for proactive writeback.
   */
  struct zswap_entry {
         swp_entry_t swpentry;

@@ -192,6 +200,7 @@ struct zswap_entry {

mm/zswap.c struct zswap_entry { · line 203

Quoted 4 lines

mm/zswap.c +1 -0

         unsigned long handle;
         struct obj_cgroup *objcg;
         struct list_head lru;
+       ktime_t store_time;

On the implementation side - will this blow up struct zswap_entry memory footprint? If so, can you guard this behind a CONFIG option, if we are to go this route?

From: Hao Jia <[email protected]>

Currently, zswap writeback can be triggered by either the pool limit being hit or by the proactive writeback mechanism. However, the existing 'zswpwb' metric in memory.stat and /proc/vmstat counts all written back pages, making it difficult to distinguish between pages written back due to the pool limit and those written back proactively.

Add a new statistic 'zswpwb_proactive' to memory.stat and /proc/vmstat. This counter tracks the number of pages written back due to proactive writeback. This allows users to better monitor and tune the proactive writeback mechanism.

Signed-off-by Hao Jia <[email protected]>

---
 Documentation/admin-guide/cgroup-v2.rst |  4 ++++
 include/linux/vm_event_item.h           |  1 +
 mm/memcontrol.c                         |  1 +
 mm/vmstat.c                             |  1 +
 mm/zswap.c                              | 11 +++++++++--
 5 files changed, 16 insertions(+), 2 deletions(-)

Documentation/admin-guide/cgroup-v2.rst +0 -0

index 05b664b3b3e8..29a189b18efc 100644

Quoted 28 lines · Hao Jia

From: Hao Jia <[email protected]>

Currently, zswap writeback can be triggered by either the pool limit being hit or by the proactive writeback mechanism. However, the existing 'zswpwb' metric in memory.stat and /proc/vmstat counts all written back pages, making it difficult to distinguish between pages written back due to the pool limit and those written back proactively.

Add a new statistic 'zswpwb_proactive' to memory.stat and /proc/vmstat. This counter tracks the number of pages written back due to proactive writeback. This allows users to better monitor and tune the proactive writeback mechanism.

Signed-off-by Hao Jia <[email protected]>

---
  Documentation/admin-guide/cgroup-v2.rst |  4 ++++
  include/linux/vm_event_item.h           |  1 +
  mm/memcontrol.c                         |  1 +
  mm/vmstat.c                             |  1 +
  mm/zswap.c                              | 11 +++++++++--
  5 files changed, 16 insertions(+), 2 deletions(-)

Documentation/admin-guide/cgroup-v2.rst +0 -0

index 05b664b3b3e8..29a189b18efc 100644

Documentation/admin-guide/cgroup-v2.rst The following nested keys are defined. · line 1743

Quoted 10 lines

Documentation/admin-guide/cgroup-v2.rst +4 -0

@@ -1734,6 +1734,10 @@ The following nested keys are defined.
           zswpwb
                 Number of pages written from zswap to swap.
 
+         zswpwb_proactive
+               Number of pages written from zswap to swap by proactive
+               writeback. This is a subset of zswpwb.
+
           zswap_incomp
                 Number of incompressible pages currently stored in zswap
                 without compression. These pages could not be compressed to

nit: once we have reached consensus on an interface, can you add documentation for the new knob in cgroup v2 doc and zswap doc too, and how it interacts with the other interface (memory.zswap.writeback, shrinker_enabled sysfs knob).

A kselftest would be very much appreciated too :)

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. However, this reactive approach makes writeback timing indeterminate and can disrupt latency-sensitive workloads when eviction happens to coincide with a critical execution window.

Furthermore, in certain scenarios, it is desirable to trigger writeback in advance to free up memory. For example, users may want to prepare for an upcoming memory-intensive workload by flushing cold memory to the backing storage when the system is relatively idle.

To address these issues, this patch series introduces a per-cgroup interface that allows users to proactively write back cold compressed pages from zswap to the backing swap device.

Quoted 8 lines · Hao Jia

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. However, this reactive approach makes writeback timing indeterminate and can disrupt latency-sensitive workloads when eviction happens to coincide with a critical execution window.

Quoted 4 lines

Furthermore, in certain scenarios, it is desirable to trigger writeback in advance to free up memory. For example, users may want to prepare for an upcoming memory-intensive workload by flushing cold memory to the backing storage when the system is relatively idle.

I can imagine the zswap writeout can come at the least possible moment...

Quoted 3 lines

To address these issues, this patch series introduces a per-cgroup interface that allows users to proactively write back cold compressed pages from zswap to the backing swap device.

...but I see this series is not only per-cgroup proactive reclaim but it's also age-based reclaim.

The per-cg consumption and limits (and regular memory reclaim) are all measured in sizes. This age-based invocations don't seem commensurable (e.g. how would users in practice determine what is the desired input to here).

Could you explain more reasoning behind this design?

From: Hao Jia <[email protected]>

Zswap currently writes back pages to backing swap devices reactively, triggered either by memory pressure via the shrinker or by the pool reaching its size limit. However, this reactive approach makes writeback timing indeterminate and can disrupt latency-sensitive workloads when eviction happens to coincide with a critical execution window.

Furthermore, in certain scenarios, it is desirable to trigger writeback in advance to free up memory. For example, users may want to prepare for an upcoming memory-intensive workload by flushing cold memory to the backing storage when the system is relatively idle.