mm/damon: Helping with alternative for watermarks
Subsystems (2)
Hi SJ,
Regarding the alternative superior feature to replace watermarks [1], I am very interested in helping you design and/or implement it.
To share my context: I am primarily using DAMON_RECLAIM on my personal laptop. During my usage, I found that using the free memory rate (MemFree) as the default watermark metric is highly problematic for modern Linux environments, because the kernel aggressively uses free memory for page cache. Therefore, I compiled a custom kernel that supports monitoring the available memory rate (MemAvailable). Since I understand that the current watermark mechanism is not planned for long-term extension, I refrained from proposing improvements to it, and instead focused on understanding why that direction is limited.
I am very interested in understanding the design constraints behind deprecating watermarks. From my current perspective, watermarks have these drawbacks as a run/stop mechanism:
- Mismatch between local monitoring and global triggers: When monitoring a specific target (e.g., via VADDR), the scheme's run/stop decision is currently tied to global system watermarks. It is impossible to determine whether to run/stop the scheme based solely on that specific target's memory usage. Therefore, the new mechanism might need to allow user-space to pause/resume schemes based on custom, target-aware metrics.
- Lack of diverse metrics: Watermarks currently lack support for metrics like PSI or inactive/active memory ratio, even though such metrics might be better suited for DAMOS auto-tuning goals.
However, I realize these features could probably be achieved by simply improving the current watermarks. Most of the drawbacks I can think of are basically about "not enough metrics/customizability".
I assume there is a deeper architectural reason why you consider this direction unsuitable for long-term evolution. If you could share a bit about those constraints, I would be able to think about the new mechanism in the right direction, rather than approaching it from my own assumptions.
My need for a more accurate trigger metric (like MemAvailable) and always-on monitoring is exactly why I want to help build the new mechanism. I prefer to invest my time in the future architecture rather than adding features to a mechanism that is about to be deprecated.
If you are open to this, I would be happy to start by reviewing the design direction you have in mind, and then help with implementation or testing where it is feasible for me to contribute meaningfully.
[1] https://lore.kernel.org/damon/[email protected]
Best regards,
Rui Yan
SJ Park
maintainerHi Liew,
Quoted 4 lines · On Fri, 14 Aug 2026 16:47:10 +0800 Liew Rui Yan <[email protected]>…
Hi SJ,
Regarding the alternative superior feature to replace watermarks [1], I am very interested in helping you design and/or implement it.
Thank you for starting this discussion.
Quoted 6 lines
To share my context: I am primarily using
DAMON_RECLAIMon my personal laptop. During my usage, I found that using the free memory rate (MemFree) as the default watermark metric is highly problematic for modern Linux environments, because the kernel aggressively uses free memory for page cache.
Thank you for sharing your use case!
Quoted 32 lines
Therefore, I compiled a custom kernel that
supports monitoring the available memory rate (MemAvailable). Since I understand that the current watermark mechanism is not planned for long-term extension, I refrained from proposing improvements to it, and instead focused on understanding why that direction is limited.I am very interested in understanding the design constraints behind deprecating watermarks. From my current perspective, watermarks have these drawbacks as a run/stop mechanism:
- Mismatch between local monitoring and global triggers: When monitoring a specific target (e.g., via VADDR), the scheme's run/stop decision is currently tied to global system watermarks. It is impossible to determine whether to run/stop the scheme based solely on that specific target's memory usage. Therefore, the new mechanism might need to allow user-space to pause/resume schemes based on custom, target-aware metrics.
- Lack of diverse metrics: Watermarks currently lack support for metrics like PSI or inactive/active memory ratio, even though such metrics might be better suited for DAMOS auto-tuning goals.
However, I realize these features could probably be achieved by simply improving the current watermarks. Most of the drawbacks I can think of are basically about "not enough metrics/customizability".
I assume there is a deeper architectural reason why you consider this direction unsuitable for long-term evolution. If you could share a bit about those constraints, I would be able to think about the new mechanism in the right direction, rather than approaching it from my own assumptions.
Quoted 8 lines
My need for a more accurate trigger metric (like MemAvailable) and always-on monitoring is exactly why I want to help build the new mechanism. I prefer to invest my time in the future architecture rather than adding features to a mechanism that is about to be deprecated.
If you are open to this, I would be happy to start by reviewing the design direction you have in mind, and then help with implementation or testing where it is feasible for me to contribute meaningfully.
I think watermarks is bettert to be deprecated due to following reasons. First of all, it stops monitoring. It means the converged region age and addresses information is lost. This is the biggest problem. Secondly, we already have similar and better features including DAMOS quotas. For your DAMON_RECLAIM use case, I'd suggest using it without watermarks but quota_mem_pressure_us.
Hence I want to deprecate watermarks rather than improving it. If we need a better watermarks, I think we should improve/extend alternatives like DAMOS quotas, or implement a new one. But, we should first understand why we need a better watermarks. You mentioned you want to use DAMOS watermarks with DAMON_RECLAIM on your laptop. Could you further share why you want to use watermarks, and what was the problem? You mentioned it was problematic due to page cache, and you implemented memAvailable based watermarks. Why you use wateramrks? How exactly the page cache usage becomes a problem for you? Has the memAvaialble based one solved your problem?
Quoted 2 lines
Thanks,
SJ
[...]
Quoted 5 lines · On Fri, 14 Aug 2026 07:54:13 -0700 SJ Park <[email protected]> wrote:
I think watermarks is bettert to be deprecated due to following reasons. First of all, it stops monitoring. It means the converged region age and addresses information is lost. This is the biggest problem. Secondly, we already have similar and better features including DAMOS quotas. For your
DAMON_RECLAIMuse case, I'd suggest using it without watermarks butquota_mem_pressure_us.
While quota_mem_pressure_us may not be suitable for my use case, I realize that DAMOS Quota does meet my needs - only proactively reclaims memory when 'MemAvaialble <= 30%'. I used it on my machine by implementing DAMOS_QUOTA_AVAILABLE_MEM_BP, which can also be achieved through the user_input of DAMOS Quota.
I think DAMOS Quota is indeed similar to and better than Watermarks. The main drawback of DAMOS Quota at present is probably the lack of wmarks.low like Watermarks, but this can be solved by adding a parameter, which might be called fallback_value or some other better name.
Therefore, I agree that we should improve DAMOS Quota instead of Watermarks, as it is indeed much better than Watermarks.
Quoted 10 lines
Hence I want to deprecate watermarks rather than improving it. If we need a better watermarks, I think we should improve/extend alternatives like DAMOS quotas, or implement a new one. But, we should first understand why we need a better watermarks. You mentioned you want to use DAMOS watermarks with
DAMON_RECLAIMon your laptop. Could you further share why you want to use watermarks, and what was the problem? You mentioned it was problematic due to page cache, and you implemented memAvailable based watermarks. Why you use wateramrks? How exactly the page cache usage becomes a problem for you? Has the memAvaialble based one solved your problem?
1. Why use Watermarks? And what was the problem?
I now understand why we do not need better Watermarks, so my answer is based on my previous thought.
I used Watermarks primarily because it is the only mechanism I know for stopping/running a Scheme. The problem is it can only monitor MemFree.
2. How exactly the page cache becomes a problem for you?
After prolonged computer use, the page cache can become full. DAMON_RECLAIM might then mistakenly assume there is slight memory pressure and reclaim those cached pages. Unnecessary page cache reclamation can incur unnecessary overhead.
3. Has the MemAvailable based one solved your problem?
Yes, using MemAvailable as a watermarks/DAMOS Quota metric can avoid a lot of unnecessary memory reclamation.
Best regards,
Rui Yan
SJ Park
maintainerQuoted 33 lines · On Mon, 17 Aug 2026 16:39:23 +0800 Liew Rui Yan <[email protected]>…
Quoted 5 lines · On Fri, 14 Aug 2026 07:54:13 -0700 SJ Park <[email protected]> wrote:
I think watermarks is bettert to be deprecated due to following reasons. First of all, it stops monitoring. It means the converged region age and addresses information is lost. This is the biggest problem. Secondly, we already have similar and better features including DAMOS quotas. For your
DAMON_RECLAIMuse case, I'd suggest using it without watermarks butquota_mem_pressure_us.While
quota_mem_pressure_usmay not be suitable for my use case, I realize that DAMOS Quota does meet my needs - only proactively reclaims memory when 'MemAvaialble <= 30%'. I used it on my machine by implementingDAMOS_QUOTA_AVAILABLE_MEM_BP, which can also be achieved through theuser_inputof DAMOS Quota.I think DAMOS Quota is indeed similar to and better than Watermarks. The main drawback of DAMOS Quota at present is probably the lack of wmarks.low like Watermarks, but this can be solved by adding a parameter, which might be called
fallback_valueor some other better name.Therefore, I agree that we should improve DAMOS Quota instead of Watermarks, as it is indeed much better than Watermarks.
Quoted 10 lines
Hence I want to deprecate watermarks rather than improving it. If we need a better watermarks, I think we should improve/extend alternatives like DAMOS quotas, or implement a new one. But, we should first understand why we need a better watermarks. You mentioned you want to use DAMOS watermarks with
DAMON_RECLAIMon your laptop. Could you further share why you want to use watermarks, and what was the problem? You mentioned it was problematic due to page cache, and you implemented memAvailable based watermarks. Why you use wateramrks? How exactly the page cache usage becomes a problem for you? Has the memAvaialble based one solved your problem?
Quoted 7 lines
1. Why use Watermarks? And what was the problem?
I now understand why we do not need better Watermarks, so my answer is based on my previous thought.
I used Watermarks primarily because it is the only mechanism I know for stopping/running a Scheme. The problem is it can only monitor MemFree.
Thank you for clarifying. I'd like to go one step further. What is the initial problem? Why you need the automatic proactive reclamation? Do you have some data showing the problem?
Quoted 7 lines
2. How exactly the page cache becomes a problem for you?
After prolonged computer use, the page cache can become full.
DAMON_RECLAIMmight then mistakenly assume there is slight memory pressure and reclaim those cached pages. Unnecessary page cache reclamation can incur unnecessary overhead.
Thank you for sharing this. I can theoretically understand the issue. Maybe you are concerned if page cache is too aggressively reclaimed and therefore if file io becomes slow. But I'm not really sure if it is only theoretical or real. Since the proactive reclamation reclaims coldest page first, and because it works for not only page cache, the performance impact might not that big in real. Do you have some data showing the problem?
Quoted 5 lines
3. Has the MemAvailable based one solved your problem?
Yes, using MemAvailable as a watermarks/DAMOS Quota metric can avoid a lot of unnecessary memory reclamation.
Great to hear that. I'm curious if you have some data showing the improvement.
Thanks,
SJ
[...]
Quoted 7 lines · On Mon, 17 Aug 2026 06:57:42 -0700 SJ Park <[email protected]> wrote:
Quoted 5 lines · On Mon, 17 Aug 2026 16:39:23 +0800 Liew Rui Yan <[email protected]>…
1. Why use Watermarks? And what was the problem?
I now understand why we do not need better Watermarks, so my answer is based on my previous thought.
Quoted 6 lines
Quoted 2 lines
I used Watermarks primarily because it is the only mechanism I know for stopping/running a Scheme. The problem is it can only monitor MemFree.
Thank you for clarifying. I'd like to go one step further. What is the initial problem? Why you need the automatic proactive reclamation? Do you have some data showing the problem?
To be honest, I have not encountered any problems, and I do not have any rigorous data. I simply using DAMON to try proactive reclamation, and see if it would cause any serious issues. So far, it has not; it has almost no noticeable negative impact on daily use.
Quoted 9 lines
Quoted 8 lines
2. How exactly the page cache becomes a problem for you?
After prolonged computer use, the page cache can become full.
DAMON_RECLAIMmight then mistakenly assume there is slight memory pressure and reclaim those cached pages. Unnecessary page cache reclamation can incur unnecessary overhead.
Quoted 6 lines
Thank you for sharing this. I can theoretically understand the issue. Maybe you are concerned if page cache is too aggressively reclaimed and therefore if file io becomes slow. But I'm not really sure if it is only theoretical or real. Since the proactive reclamation reclaims coldest page first, and because it works for not only page cache, the performance impact might not that big in real. Do you have some data showing the problem?
This is currently only theoretical. I also think that the performance impact will likely be minimal. I think adding a MemAvailable option provides certainty, if users know for sure that their devices will not lag when MemFree is low, but will when MemAvailable is low, then they can choose the option that best suits their device based on their own situation.
However, since DAMOS Quota already supports checking current_value from user input, and I cannot yet prove whether this option can bring improvement in any workload, I think we can disregard whether to add this new option for now.
Quoted 8 lines
Quoted 6 lines
3. Has the MemAvailable based one solved your problem?
Yes, using MemAvailable as a watermarks/DAMOS Quota metric can avoid a lot of unnecessary memory reclamation.
Great to hear that. I'm curious if you have some data showing the improvement.
While I do not have rigorous data to demonstrate any improvement yet, I appreciate your guidance on this.
I will continue exploring DAMON on my end, and I might send out some patches later regarding a few logical issues I noticed in the code. Thank you again for your time and answers.
Best regards,
Rui Yan
SJ Park
maintainerQuoted 12 lines · On Fri, 21 Aug 2026 22:16:54 +0800 Liew Rui Yan <[email protected]>…
Quoted 10 lines · On Mon, 17 Aug 2026 06:57:42 -0700 SJ Park <[email protected]> wrote:
Quoted 7 lines · On Mon, 17 Aug 2026 16:39:23 +0800 Liew Rui Yan <[email protected]>…
1. Why use Watermarks? And what was the problem?
I now understand why we do not need better Watermarks, so my answer is based on my previous thought.
I used Watermarks primarily because it is the only mechanism I know for stopping/running a Scheme. The problem is it can only monitor MemFree.
Quoted 8 lines
Quoted 3 lines
Thank you for clarifying. I'd like to go one step further. What is the initial problem? Why you need the automatic proactive reclamation? Do you have some data showing the problem?
To be honest, I have not encountered any problems, and I do not have any rigorous data. I simply using DAMON to try proactive reclamation, and see if it would cause any serious issues. So far, it has not; it has almost no noticeable negative impact on daily use.
Thank you for clarifying. Also appreciate your test of DAMON. I'm curious if you noticed any positive impact, too.
Quoted 17 lines
Quoted 10 lines
Quoted 9 lines
2. How exactly the page cache becomes a problem for you?
After prolonged computer use, the page cache can become full.
DAMON_RECLAIMmight then mistakenly assume there is slight memory pressure and reclaim those cached pages. Unnecessary page cache reclamation can incur unnecessary overhead.Quoted 6 lines
Thank you for sharing this. I can theoretically understand the issue. Maybe you are concerned if page cache is too aggressively reclaimed and therefore if file io becomes slow. But I'm not really sure if it is only theoretical or real. Since the proactive reclamation reclaims coldest page first, and because it works for not only page cache, the performance impact might not that big in real. Do you have some data showing the problem?
Quoted 6 lines
This is currently only theoretical. I also think that the performance impact will likely be minimal. I think adding a MemAvailable option provides certainty, if users know for sure that their devices will not lag when MemFree is low, but will when MemAvailable is low, then they can choose the option that best suits their device based on their own situation.
I understand it will allow users to tune the system for target MemAvaialble value. But, I fail at understanding what is the advantage of it, even in theory. It would be nice if we can have a clear example scenario that shows advantage of MemAvailable based watermarks compared to MemFree based one.
Quoted 10 lines
However, since DAMOS Quota already supports checking
current_valuefrom user input, and I cannot yet prove whether this option can bring improvement in any workload, I think we can disregard whether to add this new option for now.Quoted 4 lines
Quoted 4 lines
3. Has the MemAvailable based one solved your problem?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Yes, using MemAvailable as a watermarks/DAMOS Quota metric can avoid a lot of unnecessary memory reclamation.
Great to hear that. I'm curious if you have some data showing the improvement.
While I do not have rigorous data to demonstrate any improvement yet, I appreciate your guidance on this.
I wouldn't say we can never add MemAvaialble based watermarks until we have proving data. But, I think we definitely need a clear theory to start a discussion.
Quoted 3 lines
I will continue exploring DAMON on my end, and I might send out some patches later regarding a few logical issues I noticed in the code.
I'm looking forward to the patches!
Quoted 1 line
Thank you again for your time and answers.
Thank you, too!
Quoted 3 lines
Best regards,
Rui Yan
Thanks,
SJ