mm: skip folio_activate() for mlocked folios
PATCH v2 1/1 mm: skip folio_activate() for mlocked folios
__mlock_folio() does not move folio to unevicable LRU, when folio_activate() removes folio from LRU.
To prevent this case also check for folio_test_mlocked() in folio_mark_accessed(). If folio is not yet marked as unevictable, but already marked as mlocked, then skip folio_activate() call to allow __mlock_folio() to make all necessary updates. It should be safe to skip folio_activate() here, because mlocked folio should end up in unevictable LRU eventually anyway.
To observe the problem mmap() and mlock() big file and check Unevictable and Mlocked values from /proc/meminfo. On freshly booted system without any other mlocked memory we expect them to match or be quite close.
See below for more detailed reproduction steps. Source code of stat.c is available at [1].
$ head -c 8G < /dev/urandom > /tmp/random.bin
$ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat $ /tmp/stat Unevictable: 8389668 kB Mlocked: 8389700 kB
Need to run binary twice. Problem does not reproduce on the first run, but always reproduces on the second run.
$ /tmp/stat Unevictable: 5374676 kB Mlocked: 8389332 kB
[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.
mm/swap.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
mm/swap.c +10 -0
@@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)* this list is never rotated or maintained, so marking an * unevictable page accessed has no effect. */ + }else if (folio_test_mlocked (folio)) { +/* +* Pages that are mlocked, but not yet on unevictable LRU. +* They might be still in mlock_fbatch waiting to be processed +* and activating it here might interfere with +* mlock_folio_batch(). __mlock_folio() will fail +* folio_test_clear_lru() check and give up. It happens because +* __folio_batch_add_and_move() clears LRU flag, when adding +* folio to activate batch. +*/ }else if (!folio_test_active (folio)) {/* * If the folio is on the LRU, queue it for activation via
Signature · 1 line
2.47.3
Quoted 32 lines · Dmitry Ilvokhin
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.To prevent this case also check for
folio_test_mlocked()infolio_mark_accessed(). If folio is not yet marked as unevictable, but already marked as mlocked, then skipfolio_activate()call to allow__mlock_folio()to make all necessary updates. It should be safe to skipfolio_activate()here, because mlocked folio should end up in unevictable LRU eventually anyway.To observe the problem
mmap()andmlock()big file and check Unevictable and Mlocked values from/proc/meminfo. On freshly booted system without any other mlocked memory we expect them to match or be quite close.See below for more detailed reproduction steps. Source code of
stat.cis available at [1].$ head -c 8G < /dev/urandom > /tmp/random.bin$ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat $ /tmp/stat Unevictable: 8389668 kB Mlocked: 8389700 kBNeed to run binary twice. Problem does not reproduce on the first run, but always reproduces on the second run.
$ /tmp/stat Unevictable: 5374676 kB Mlocked: 8389332 kB
Quoted 6 lines
[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
Co-developed-by Kiryl Shutsemau <[email protected]>Signed-off-by Kiryl Shutsemau <[email protected]>Signed-off-by Dmitry Ilvokhin <[email protected]>Acked-by Usama Arif <[email protected]>
Quoted 2 lines · On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <[email protected]> wrote:
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.
A trivial opinion. So the user-visible issue is the incorrect meminfo, right?
I read your changelog below saying you changed this message from v1 to frame on unevictable LRU rather than stat accounting, and I think that's nice to understand the detail. But I think further describing the resulting user-visible issue can be helpful at better understanding the motivation of this nice patch.
Quoted 29 lines
To prevent this case also check for
folio_test_mlocked()infolio_mark_accessed(). If folio is not yet marked as unevictable, but already marked as mlocked, then skipfolio_activate()call to allow__mlock_folio()to make all necessary updates. It should be safe to skipfolio_activate()here, because mlocked folio should end up in unevictable LRU eventually anyway.To observe the problem
mmap()andmlock()big file and check Unevictable and Mlocked values from/proc/meminfo. On freshly booted system without any other mlocked memory we expect them to match or be quite close.See below for more detailed reproduction steps. Source code of
stat.cis available at [1].$ head -c 8G < /dev/urandom > /tmp/random.bin$ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat $ /tmp/stat Unevictable: 8389668 kB Mlocked: 8389700 kBNeed to run binary twice. Problem does not reproduce on the first run, but always reproduces on the second run.
$ /tmp/stat Unevictable: 5374676 kB Mlocked: 8389332 kB
Quoted 6 lines
[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
Co-developed-by Kiryl Shutsemau <[email protected]>Signed-off-by Kiryl Shutsemau <[email protected]>Signed-off-by Dmitry Ilvokhin <[email protected]>Acked-by Usama Arif <[email protected]>
Because this is a fix of a user-visible issue, I'm wondering if this deserves
Fixes: and Cc: stable@.
Anyway my comments are only trivial ones, and I think the change is good.
Quoted 4 lines
---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.
Yet another trivial and personal opinion. Adding a link to the previous version could be helpful for reviewers like me.
Thanks,
SJ
[...]
Quoted 6 lines · SeongJae Park
Quoted 2 lines · On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <[email protected]> wrote:
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.A trivial opinion. So the user-visible issue is the incorrect meminfo, right?
The user-visible effect is that we unnecessary postpone moving pages to unevictable LRU that lead to unexpected stats: Mlocked > Unevictable.
Quoted 7 lines
Quoted 4 lines
---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.Yet another trivial and personal opinion. Adding a link to the previous version could be helpful for reviewers like me.
You probably missed recent Linus rant on Link: tags :P
Signature · 1 line
Kiryl Shutsemau / Kirill A. Shutemov
[Somehow I messed up the subject, so resending]
Cc Hugh, yangge, David
Quoted 53 lines · Dmitry Ilvokhin
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.To prevent this case also check for
folio_test_mlocked()infolio_mark_accessed(). If folio is not yet marked as unevictable, but already marked as mlocked, then skipfolio_activate()call to allow__mlock_folio()to make all necessary updates. It should be safe to skipfolio_activate()here, because mlocked folio should end up in unevictable LRU eventually anyway.To observe the problem
mmap()andmlock()big file and check Unevictable and Mlocked values from/proc/meminfo. On freshly booted system without any other mlocked memory we expect them to match or be quite close.See below for more detailed reproduction steps. Source code of
stat.cis available at [1].$ head -c 8G < /dev/urandom > /tmp/random.bin$ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat $ /tmp/stat Unevictable: 8389668 kB Mlocked: 8389700 kBNeed to run binary twice. Problem does not reproduce on the first run, but always reproduces on the second run.
$ /tmp/stat Unevictable: 5374676 kB Mlocked: 8389332 kB[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
Co-developed-by Kiryl Shutsemau <[email protected]>Signed-off-by Kiryl Shutsemau <[email protected]>Signed-off-by Dmitry Ilvokhin <[email protected]>Acked-by Usama Arif <[email protected]>---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.mm/swap.c | 10 ++++++++++ 1 file changed, 10 insertions(+)mm/swap.c +0
-0@@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)* this list is never rotated or maintained, so marking an * unevictable page accessed has no effect. */
mm/swap.c void folio_mark_accessed(struct folio *folio) · line 481
Quoted 10 lines
mm/swap.c +10
-0+ }else if (folio_test_mlocked (folio)) { +/* +* Pages that are mlocked, but not yet on unevictable LRU. +* They might be still in mlock_fbatch waiting to be processed +* and activating it here might interfere with +* mlock_folio_batch(). __mlock_folio() will fail +* folio_test_clear_lru() check and give up. It happens because +* __folio_batch_add_and_move() clears LRU flag, when adding +* folio to activate batch. +*/
This makes sense as activating an mlocked folio should be a noop but I am wondering why we are seeing this now. By this, I mean mlock()ed memory being delayed to get to unevictable LRU. Also I remember Hugh recently [1] removed the difference betwen mlock percpu cache and other percpu caches of clearing LRU bit on entry. Does you repro work even with Hugh's changes or without it?
Quoted 10 lines · On Wed, 8 Oct 2025 11:33:01 +0100 Kiryl Shutsemau <[email protected]> wrote:
Quoted 6 lines · SeongJae Park
Quoted 2 lines · On Mon, 6 Oct 2025 13:25:26 +0000 Dmitry Ilvokhin <[email protected]> wrote:
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.A trivial opinion. So the user-visible issue is the incorrect meminfo, right?
The user-visible effect is that we unnecessary postpone moving pages to unevictable LRU that lead to unexpected stats: Mlocked > Unevictable.
Thank you for clarifying!
Quoted 10 lines
Quoted 8 lines
Quoted 5 lines
---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.Yet another trivial and personal opinion. Adding a link to the previous version could be helpful for reviewers like me.
You probably missed recent Linus rant on Link: tags :P
I didn't miss it [1] :) And I pretty sure Linus wouldn't be angry for this, since what he dislikes is meaningless Link: tags on commit messages. We are discussing something about changelog after the '---' line, which will anyway not added to the commit message. Also I'm saying a link in a general form, not specifically Link: tag.
[1] https://lwn.net/Articles/1037069/
Thanks,
SJ
[...]
Quoted 67 lines · Shakeel Butt
[Somehow I messed up the subject, so resending]
Cc Hugh, yangge, David
Quoted 62 lines · Dmitry Ilvokhin
__mlock_folio()does not move folio to unevicable LRU, whenfolio_activate()removes folio from LRU.To prevent this case also check for
folio_test_mlocked()infolio_mark_accessed(). If folio is not yet marked as unevictable, but already marked as mlocked, then skipfolio_activate()call to allow__mlock_folio()to make all necessary updates. It should be safe to skipfolio_activate()here, because mlocked folio should end up in unevictable LRU eventually anyway.To observe the problem
mmap()andmlock()big file and check Unevictable and Mlocked values from/proc/meminfo. On freshly booted system without any other mlocked memory we expect them to match or be quite close.See below for more detailed reproduction steps. Source code of
stat.cis available at [1].$ head -c 8G < /dev/urandom > /tmp/random.bin$ cc -pedantic -Wall -std=c99 stat.c -O3 -o /tmp/stat $ /tmp/stat Unevictable: 8389668 kB Mlocked: 8389700 kBNeed to run binary twice. Problem does not reproduce on the first run, but always reproduces on the second run.
$ /tmp/stat Unevictable: 5374676 kB Mlocked: 8389332 kB[1]: https://gist.github.com/ilvokhin/e50c3d2ff5d9f70dcbb378c6695386dd
Co-developed-by Kiryl Shutsemau <[email protected]>Signed-off-by Kiryl Shutsemau <[email protected]>Signed-off-by Dmitry Ilvokhin <[email protected]>Acked-by Usama Arif <[email protected]>---
Changes in v2:
- Rephrase commit message: frame it in terms of unevicable LRU, not stat accounting.mm/swap.c | 10 ++++++++++ 1 file changed, 10 insertions(+)mm/swap.c +9
-0@@ -469,6 +469,16 @@ void folio_mark_accessed(struct folio *folio)* this list is never rotated or maintained, so marking an * unevictable page accessed has no effect. */ + }else if (folio_test_mlocked (folio)) { +/* +* Pages that are mlocked, but not yet on unevictable LRU. +* They might be still in mlock_fbatch waiting to be processed +* and activating it here might interfere with +* mlock_folio_batch(). __mlock_folio() will fail +* folio_test_clear_lru() check and give up. It happens because +* __folio_batch_add_and_move() clears LRU flag, when adding +* folio to activate batch.
Quoted 9 lines
mm/swap.c void folio_mark_accessed(struct folio *folio) · line 481
Quoted 1 line
mm/swap.c +1
-0+*/ This makes sense as activating an mlocked folio should be a noop but I am wondering why we are seeing this now. By this, I mean mlock()ed memory being delayed to get to unevictable LRU. Also I remember Hugh recently [1] removed the difference betwen mlock percpu cache and other percpu caches of clearing LRU bit on entry. Does you repro work even with Hugh's changes or without it?
Thanks Shakeel for mentioning Hugh's patch, I was not aware of it. Indeed, I could not reproduce problem on top of Hugh's patch anymore, which totally make sense, because folio_test_clear_lru() is gone from __folio_batch_add_and_move().
Now I wonder does folio_test_mlocked() check still make sense in the current codebase?
Quoted 1 line
Andrew Morton
maintainerQuoted 16 lines · On Wed, 8 Oct 2025 18:06:07 +0000 Dmitry Ilvokhin <[email protected]> wrote:
Quoted 6 lines
Quoted 6 lines
mm/swap.c +6
-0+* They might be still in mlock_fbatch waiting to be processed +* and activating it here might interfere with +* mlock_folio_batch(). __mlock_folio() will fail +* folio_test_clear_lru() check and give up. It happens because +* __folio_batch_add_and_move() clears LRU flag, when adding +* folio to activate batch. Quoted 9 lines
mm/swap.c void folio_mark_accessed(struct folio *folio) · line 481
Quoted 1 line
mm/swap.c +1
-0+*/ This makes sense as activating an mlocked folio should be a noop but I am wondering why we are seeing this now. By this, I mean mlock()ed memory being delayed to get to unevictable LRU. Also I remember Hugh recently [1] removed the difference betwen mlock percpu cache and other percpu caches of clearing LRU bit on entry. Does you repro work even with Hugh's changes or without it?
Quoted 9 lines
Thanks Shakeel for mentioning Hugh's patch, I was not aware of it. Indeed, I could not reproduce problem on top of Hugh's patch anymore, which totally make sense, because
folio_test_clear_lru()is gone from__folio_batch_add_and_move().Now I wonder does
folio_test_mlocked()check still make sense in the current codebase?Quoted 1 line
So I take it that this patch ("mm: skip folio_activate() for mlocked folios") is no longer needed?
Quoted 23 lines · Andrew Morton
Quoted 21 lines · On Wed, 8 Oct 2025 18:06:07 +0000 Dmitry Ilvokhin <[email protected]> wrote:
Quoted 6 lines
Quoted 6 lines
mm/swap.c +6
-0+* They might be still in mlock_fbatch waiting to be processed +* and activating it here might interfere with +* mlock_folio_batch(). __mlock_folio() will fail +* folio_test_clear_lru() check and give up. It happens because +* __folio_batch_add_and_move() clears LRU flag, when adding +* folio to activate batch. Quoted 9 lines
mm/swap.c void folio_mark_accessed(struct folio *folio) · line 481
Quoted 1 line
mm/swap.c +1
-0+*/ This makes sense as activating an mlocked folio should be a noop but I am wondering why we are seeing this now. By this, I mean mlock()ed memory being delayed to get to unevictable LRU. Also I remember Hugh recently [1] removed the difference betwen mlock percpu cache and other percpu caches of clearing LRU bit on entry. Does you repro work even with Hugh's changes or without it?
Thanks Shakeel for mentioning Hugh's patch, I was not aware of it. Indeed, I could not reproduce problem on top of Hugh's patch anymore, which totally make sense, because
folio_test_clear_lru()is gone from__folio_batch_add_and_move().
Quoted 7 lines
Quoted 4 lines
Now I wonder does
folio_test_mlocked()check still make sense in the current codebase?Quoted 1 line
So I take it that this patch ("mm: skip
folio_activate()for mlocked folios") is no longer needed?
Yes, this is my understanding as well. Hugh's patch addressed initial problem and this patch is no longer needed.