mm/vmscan: prevent MGLRU reclaim from pinning address space
Subsystems (3)
PATCH 1/1 mm/vmscan: prevent MGLRU reclaim from pinning address space
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable mm_struct to be able to take and release mmap_read_lock and a stable mm->mm_mt tree to walk. This address space pinning results in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace process_mrelease()) from doing their job during MGLRU scan because they check task_will_free_mem() which will yield negative result due to the elevated mm->mm_users.
Replace unnecessary address space pinning with mm_struct pinning by replacing mmget/mmput with mmgrab/mmdrop calls. mm_mt is contained within mm_struct itself, therefore it won't be freed as long as mm_struct is stable and it won't change during the walk because mmap_read_lock is being held.
--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/vmscan.c +3 -2
@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm); -return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)
@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );if (*iter) -mmput_async (*iter); +mmdrop (*iter); *iter = mm;
base-commit: 8c65073d94c8b7cc3170de31af38edc9f5d96f0e
Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
Lorenzo Stoakes (Oracle)
reviewerQuoted 4 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results
Hmm, I guess exit_mmap() calls __mt_destroy(), but that'll just destroy allocated state and leave the tree empty right, so traversal of that tree at that point would just do nothing?
Quoted 23 lines
in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace
process_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/vmscan.c +0
-0
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2863
Quoted 1 line
mm/vmscan.c +0
-0@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;
Not related to this series, but I really don't like how coupled MGLRU is to the rest of the 'classic' reclaim code.
Just in the middle of vmscan you walk into generic mm walker logic and the only hint it's MGLRU is you see lru_gen_xxx stuff (I'm also annoyed that we call it MGLRU but it's called lru_gen_xxx in the kernel :)
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2866
Quoted 3 lines
mm/vmscan.c +1
-0clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm);
Is the mm somehow pinned here or, on destruction, would move it from the mm list meaning that we can safely assume we have something sane in mm-> to grab? I guess this must have already been the case for mmget_not_zero() to have been used before though.
mm/vmscan.c static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite · line 3068
Quoted 12 lines
mm/vmscan.c +2
-2-return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );if (*iter) -mmput_async (*iter); +mmdrop (*iter);
This will now be a blocking call that could free the mm (via __mmdrop()), so could take a while, is that ok?
If before the code was intentionally deferring work here, doesn't that imply that being slow here might be an issue, somehow? Or was it just because they could? :)
Quoted 8 lines
mm/vmscan.c +0
-0*iter = mm;base-commit:
8c65073d94c8b7cc3170de31af38edc9f5d96f0e--
2.53.0.1018.g2bb0e51243-goog
Thanks, Lorenzo
Quoted 10 lines · Lorenzo Stoakes (Oracle)
Quoted 4 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning resultsHmm, I guess
exit_mmap()calls__mt_destroy(), but that'll just destroy allocated state and leave the tree empty right, so traversal of that tree at that point would just do nothing?
Correct. And __mt_destroy() happens under mmap_write_lock while traversal under mmap_read_lock, so they should not race.
Quoted 29 lines
Quoted 24 lines
in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace
process_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/vmscan.c +0
-0mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2863
Quoted 1 line
mm/vmscan.c +0
-0@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;Not related to this series, but I really don't like how coupled MGLRU is to the rest of the 'classic' reclaim code.
Quoted 3 lines
Just in the middle of vmscan you walk into generic mm walker logic and the only hint it's MGLRU is you see
lru_gen_xxxstuff (I'm also annoyed that we call it MGLRU but it's calledlru_gen_xxxin the kernel :)
I don't have a strong opinion on this. Perhaps the naming can be changed outside of this series.
Quoted 9 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2866
Quoted 4 lines
mm/vmscan.c +1
-0clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm);Is the mm somehow pinned here or, on destruction, would move it from the mm list meaning that we can safely assume we have something sane in mm-> to grab? I guess this must have already been the case for
mmget_not_zero()to have been used before though.
Yes, mm is stable because it's fetched from mm_list. When mm is added to this list via lru_gen_add_mm(mm) it is referenced and that reference is dropped only after lru_gen_del_mm(mm) removes the mm from this list (see https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1185
and https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1187).
Addition, removal and retrieval from that list happen under mm_list->lock which prevents races.
Quoted 10 lines
Quoted 10 lines
mm/vmscan.c +1
-1-return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );
Quoted 6 lines
mm/vmscan.c static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite · line 3068
Quoted 3 lines
mm/vmscan.c +1
-1if (*iter) -mmput_async (*iter); +mmdrop (*iter);This will now be a blocking call that could free the mm (via
__mmdrop()), so could take a while, is that ok?
mmdrop() should not be a heavy-weight operation. It simply destroys the metadata associated with mm_struct. mmput() OTOH will call exit_mmap() if it drops the last reference and that can take a while because that's when we free the memory of the process. I believe that's why mmput_async() was used here.
Quoted 4 lines
If before the code was intentionally deferring work here, doesn't that imply that being slow here might be an issue, somehow? Or was it just because they could? :)
I think the reason was the possibility of calling mmput() -> __mmput() -> exit_mmap(mm) which could indeed block us for a while.
Quoted 11 lines
Quoted 9 lines
mm/vmscan.c +0
-0*iter = mm;base-commit:
8c65073d94c8b7cc3170de31af38edc9f5d96f0e--
2.53.0.1018.g2bb0e51243-googThanks, Lorenzo
Quoted 8 lines · Suren Baghdasaryan
Quoted 7 lines · Lorenzo Stoakes (Oracle)
Quoted 4 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results
Quoted 6 lines
Quoted 3 lines
Hmm, I guess
exit_mmap()calls__mt_destroy(), but that'll just destroy allocated state and leave the tree empty right, so traversal of that tree at that point would just do nothing?Correct. And
__mt_destroy()happens undermmap_write_lockwhile traversal undermmap_read_lock, so they should not race.
Yeah that's fair.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace
process_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>
Given you have cleared up my concerns, this LGTM, so:
Quoted 14 lines
Quoted 14 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2863
Quoted 10 lines
--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/vmscan.c +0
-0@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;Not related to this series, but I really don't like how coupled MGLRU is to the rest of the 'classic' reclaim code.
Quoted 6 lines
Quoted 3 lines
Just in the middle of vmscan you walk into generic mm walker logic and the only hint it's MGLRU is you see
lru_gen_xxxstuff (I'm also annoyed that we call it MGLRU but it's calledlru_gen_xxxin the kernel :)I don't have a strong opinion on this. Perhaps the naming can be changed outside of this series.
I was thinking more of a new file for mglru :>)
I believe we also need some more active maintainership also... but that's another issue ;)
Quoted 11 lines
Quoted 10 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2866
Quoted 5 lines
mm/vmscan.c +1
-0clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm);Is the mm somehow pinned here or, on destruction, would move it from the mm list meaning that we can safely assume we have something sane in mm-> to grab? I guess this must have already been the case for
mmget_not_zero()to have been used before though.
Quoted 7 lines
Yes, mm is stable because it's fetched from
mm_list. When mm is added to this list vialru_gen_add_mm(mm)it is referenced and that reference is dropped only afterlru_gen_del_mm(mm)removes the mm from this list (see https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1185
and https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1187).
Addition, removal and retrieval from that list happen undermm_list->lockwhich prevents races.
Ack, thanks!
Quoted 18 lines
Quoted 11 lines
Quoted 11 lines
mm/vmscan.c +1
-1-return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );Quoted 6 lines
mm/vmscan.c static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite · line 3068
Quoted 3 lines
mm/vmscan.c +1
-1if (*iter) -mmput_async (*iter); +mmdrop (*iter);This will now be a blocking call that could free the mm (via
__mmdrop()), so could take a while, is that ok?
Quoted 5 lines
mmdrop()should not be a heavy-weight operation. It simply destroys the metadata associated withmm_struct.mmput()OTOH will callexit_mmap()if it drops the last reference and that can take a while because that's when we free the memory of the process. I believe that's whymmput_async()was used here.
Yeah that's fair enough! Thanks.
Quoted 8 lines
Quoted 5 lines
If before the code was intentionally deferring work here, doesn't that imply that being slow here might be an issue, somehow? Or was it just because they could? :)
I think the reason was the possibility of calling
mmput()->__mmput()->exit_mmap(mm)which could indeed block us for a while.
Yeah fair :)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
mm/vmscan.c +0
-0*iter = mm;
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
base-commit:
8c65073d94c8b7cc3170de31af38edc9f5d96f0e--
2.53.0.1018.g2bb0e51243-googThanks, Lorenzo
Cheers, Lorenzo
Quoted 33 lines · Lorenzo Stoakes (Oracle)
Quoted 8 lines · Suren Baghdasaryan
Quoted 7 lines · Lorenzo Stoakes (Oracle)
Quoted 4 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning resultsQuoted 6 lines
Quoted 3 lines
Hmm, I guess
exit_mmap()calls__mt_destroy(), but that'll just destroy allocated state and leave the tree empty right, so traversal of that tree at that point would just do nothing?Correct. And
__mt_destroy()happens undermmap_write_lockwhile traversal undermmap_read_lock, so they should not race.Yeah that's fair.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace
process_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>Given you have cleared up my concerns, this LGTM, so:
Reviewed-by Lorenzo Stoakes (Oracle) <[email protected]>
Thanks!
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2863
Quoted 11 lines
--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/vmscan.c +0
-0@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;Not related to this series, but I really don't like how coupled MGLRU is to the rest of the 'classic' reclaim code.
Just in the middle of vmscan you walk into generic mm walker logic and the only hint it's MGLRU is you see
lru_gen_xxxstuff (I'm also annoyed that we call it MGLRU but it's calledlru_gen_xxxin the kernel :)
Quoted 7 lines
Quoted 2 lines
I don't have a strong opinion on this. Perhaps the naming can be changed outside of this series.
I was thinking more of a new file for mglru :>)
I believe we also need some more active maintainership also... but that's another issue ;)
Yes, in my team I volunteered one developer to actively review and support MGLRU. Over time I hope he will be in a position to help with maintenance.
Quoted 73 lines
Quoted 12 lines
Quoted 11 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2866
Quoted 6 lines
mm/vmscan.c +1
-0clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm);Is the mm somehow pinned here or, on destruction, would move it from the mm list meaning that we can safely assume we have something sane in mm-> to grab? I guess this must have already been the case for
mmget_not_zero()to have been used before though.Quoted 7 lines
Yes, mm is stable because it's fetched from
mm_list. When mm is added to this list vialru_gen_add_mm(mm)it is referenced and that reference is dropped only afterlru_gen_del_mm(mm)removes the mm from this list (see https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1185
and https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1187).
Addition, removal and retrieval from that list happen undermm_list->lockwhich prevents races.Ack, thanks!
Quoted 18 lines
Quoted 11 lines
Quoted 11 lines
mm/vmscan.c +1
-1-return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );Quoted 6 lines
mm/vmscan.c static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite · line 3068
Quoted 3 lines
mm/vmscan.c +1
-1if (*iter) -mmput_async (*iter); +mmdrop (*iter);This will now be a blocking call that could free the mm (via
__mmdrop()), so could take a while, is that ok?Quoted 5 lines
mmdrop()should not be a heavy-weight operation. It simply destroys the metadata associated withmm_struct.mmput()OTOH will callexit_mmap()if it drops the last reference and that can take a while because that's when we free the memory of the process. I believe that's whymmput_async()was used here.Yeah that's fair enough! Thanks.
Quoted 8 lines
Quoted 5 lines
If before the code was intentionally deferring work here, doesn't that imply that being slow here might be an issue, somehow? Or was it just because they could? :)
I think the reason was the possibility of calling
mmput()->__mmput()->exit_mmap(mm)which could indeed block us for a while.Yeah fair :)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
mm/vmscan.c +0
-0*iter = mm;Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
base-commit:
8c65073d94c8b7cc3170de31af38edc9f5d96f0e--
2.53.0.1018.g2bb0e51243-googThanks, Lorenzo
Cheers, Lorenzo
Quoted 33 lines · Lorenzo Stoakes (Oracle)
Quoted 8 lines · Suren Baghdasaryan
Quoted 7 lines · Lorenzo Stoakes (Oracle)
Quoted 4 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning resultsQuoted 6 lines
Quoted 3 lines
Hmm, I guess
exit_mmap()calls__mt_destroy(), but that'll just destroy allocated state and leave the tree empty right, so traversal of that tree at that point would just do nothing?Correct. And
__mt_destroy()happens undermmap_write_lockwhile traversal undermmap_read_lock, so they should not race.Yeah that's fair.
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspace
process_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>Given you have cleared up my concerns, this LGTM, so:
Reviewed-by Lorenzo Stoakes (Oracle) <[email protected]>
Hi Andrew,
Any concerns about this patch or do you want someone maintaining MGLRU to Ack it before pulling into your tree?
The change is quite simple and I explain in my reply why it's safe, Sashiko seems to like it and I extensively tested it on Android. Please let me know if anything else is needed.
Thanks,
Suren.
Quoted 99 lines
Quoted 15 lines
Quoted 15 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2863
Quoted 11 lines
--- mm/vmscan.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)mm/vmscan.c +0
-0@@ -2863,8 +2863,9 @@ static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk)return NULL ;Not related to this series, but I really don't like how coupled MGLRU is to the rest of the 'classic' reclaim code.
Quoted 6 lines
Quoted 3 lines
Just in the middle of vmscan you walk into generic mm walker logic and the only hint it's MGLRU is you see
lru_gen_xxxstuff (I'm also annoyed that we call it MGLRU but it's calledlru_gen_xxxin the kernel :)I don't have a strong opinion on this. Perhaps the naming can be changed outside of this series.
I was thinking more of a new file for mglru :>)
I believe we also need some more active maintainership also... but that's another issue ;)
Quoted 11 lines
Quoted 10 lines
mm/vmscan.c static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) · line 2866
Quoted 5 lines
mm/vmscan.c +1
-0clear_bit (key, &mm->lru_gen.bitmap); +mmgrab (mm);Is the mm somehow pinned here or, on destruction, would move it from the mm list meaning that we can safely assume we have something sane in mm-> to grab? I guess this must have already been the case for
mmget_not_zero()to have been used before though.Quoted 7 lines
Yes, mm is stable because it's fetched from
mm_list. When mm is added to this list vialru_gen_add_mm(mm)it is referenced and that reference is dropped only afterlru_gen_del_mm(mm)removes the mm from this list (see https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1185
and https://elixir.bootlin.com/linux/v7.0-rc4/source/kernel/fork.c#L1187).
Addition, removal and retrieval from that list happen undermm_list->lockwhich prevents races.Ack, thanks!
Quoted 18 lines
Quoted 11 lines
Quoted 11 lines
mm/vmscan.c +1
-1-return mmget_not_zero (mm) ? mm :NULL ; +return mm; }void lru_gen_add_mm (struct mm_struct *mm)@@ -3064,7 +3065,7 @@ static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **itereset_bloom_filter (mm_state, walk->seq +1 );Quoted 6 lines
mm/vmscan.c static bool iterate_mm_list(struct lru_gen_mm_walk *walk, struct mm_struct **ite · line 3068
Quoted 3 lines
mm/vmscan.c +1
-1if (*iter) -mmput_async (*iter); +mmdrop (*iter);This will now be a blocking call that could free the mm (via
__mmdrop()), so could take a while, is that ok?Quoted 5 lines
mmdrop()should not be a heavy-weight operation. It simply destroys the metadata associated withmm_struct.mmput()OTOH will callexit_mmap()if it drops the last reference and that can take a while because that's when we free the memory of the process. I believe that's whymmput_async()was used here.Yeah that's fair enough! Thanks.
Quoted 8 lines
Quoted 5 lines
If before the code was intentionally deferring work here, doesn't that imply that being slow here might be an issue, somehow? Or was it just because they could? :)
I think the reason was the possibility of calling
mmput()->__mmput()->exit_mmap(mm)which could indeed block us for a while.Yeah fair :)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
mm/vmscan.c +0
-0*iter = mm;Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
base-commit:
8c65073d94c8b7cc3170de31af38edc9f5d96f0e--
2.53.0.1018.g2bb0e51243-googThanks, Lorenzo
Cheers, Lorenzo
Andrew Morton
maintainerQuoted 7 lines · On Fri, 27 Mar 2026 08:20:06 -0700 Suren Baghdasaryan <[email protected]> wrote:
Quoted 3 lines
Given you have cleared up my concerns, this LGTM, so:
Reviewed-by Lorenzo Stoakes (Oracle) <[email protected]>Hi Andrew,
Any concerns about this patch or do you want someone maintaining MGLRU to Ack it before pulling into your tree?
That would of course be welcome. But Suren+Lorenzo is more than enough for me.
Quoted 3 lines
The change is quite simple and I explain in my reply why it's safe, Sashiko seems to like it and I extensively tested it on Android. Please let me know if anything else is needed.
I think that if the fix is valuable enough for Android to backport then we at least should have the Fixes: there to help others who are using MGLRU on older kernels. (We don't have an "Improves:"!) And if it's a really big improvement then we could even pretend it's a bugfix and sneak it into -stable ;)
But the changelog doesn't say much at all about how this change affects users. Just "delays". So can you please expand on this (numbers are always nice) and I can paste that into the changelog.
Thanks.
Quoted 24 lines · Andrew Morton
Quoted 7 lines · On Fri, 27 Mar 2026 08:20:06 -0700 Suren Baghdasaryan <[email protected]> wrote:
Quoted 3 lines
Given you have cleared up my concerns, this LGTM, so:
Reviewed-by Lorenzo Stoakes (Oracle) <[email protected]>Hi Andrew,
Any concerns about this patch or do you want someone maintaining MGLRU to Ack it before pulling into your tree?That would of course be welcome. But Suren+Lorenzo is more than enough for me.
Quoted 3 lines
The change is quite simple and I explain in my reply why it's safe, Sashiko seems to like it and I extensively tested it on Android. Please let me know if anything else is needed.
I think that if the fix is valuable enough for Android to backport then we at least should have the Fixes: there to help others who are using MGLRU on older kernels. (We don't have an "Improves:"!) And if it's a really big improvement then we could even pretend it's a bugfix and sneak it into -stable ;)
Quoted 3 lines
But the changelog doesn't say much at all about how this change affects users. Just "delays". So can you please expand on this (numbers are always nice) and I can paste that into the changelog.
It affects the system in the sense that if the MM of the killed process is being reclaimed by kswapd then reapers won't be able to reap it. Even the process itself (which might have higher-priority than kswapd) will not free its memory until kswapd drops the last reference. IOW, we delay freeing the memory because kswapd is reclaiming it. In Android the visible result for us is that process_mrelease() (userspace reaper) skips MM in such cases and we see process memory not released for an unusually long time (secs).
Quoted 2 lines
Thanks.
Lorenzo Stoakes (Oracle)
reviewerQuoted 10 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspaceprocess_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.
Quoted 8 lines
Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>
Oh and sorry, missed this - do we need a 'Fixed' for something that's just a perf improvement?
This doesn't seem buggy?
Thanks, Lorenzo
Quoted 18 lines · Lorenzo Stoakes (Oracle)
Quoted 16 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspaceprocess_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.
Quoted 7 lines
Quoted 2 lines
Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>Oh and sorry, missed this - do we need a 'Fixed' for something that's just a perf improvement?
This doesn't seem buggy?
Well, if we are pinning something we should not is that a bug? I didn't CC stable, so I don't expect it to be backported (though if it's accepted, we will backport it in Android as it causes real problems with reclaim), however if someone is looking for MGLRU fixes I would like this patch to be discoverable. If you feel strongly about it, I'll remove the tag.
Thanks for the review!
Quoted 2 lines
Thanks, Lorenzo
Lorenzo Stoakes (Oracle)
reviewerQuoted 25 lines · Suren Baghdasaryan
Quoted 24 lines · Lorenzo Stoakes (Oracle)
Quoted 10 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspaceprocess_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Quoted 8 lines
Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>Oh and sorry, missed this - do we need a 'Fixed' for something that's just a perf improvement?
Quoted 8 lines
Quoted 1 line
This doesn't seem buggy?
Well, if we are pinning something we should not is that a bug? I didn't CC stable, so I don't expect it to be backported (though if it's accepted, we will backport it in Android as it causes real problems with reclaim), however if someone is looking for MGLRU fixes I would like this patch to be discoverable. If you feel strongly about it, I'll remove the tag.
I don't feel that strongly about it :>)
Quoted 2 lines
Thanks for the review!
No problem! Working through your vma killable series atm also :)
Quoted 3 lines
Quoted 3 lines
Thanks, Lorenzo
Cheers, Lorenzo
Quoted 36 lines · Lorenzo Stoakes (Oracle)
Quoted 25 lines · Suren Baghdasaryan
Quoted 24 lines · Lorenzo Stoakes (Oracle)
Quoted 10 lines · Suren Baghdasaryan
When shrinking lruvec, MGLRU pins address space before walking it. This is excessive since all it needs for walking the page range is a stable
mm_structto be able to take and releasemmap_read_lockand a stablemm->mm_mttree to walk. This address space pinning results in delays when releasing the memory of a dying process. This also prevents mm reapers (both in-kernel oom-reaper and userspaceprocess_mrelease()) from doing their job during MGLRU scan because they checktask_will_free_mem()which will yield negative result due to the elevatedmm->mm_users.Quoted 8 lines
Replace unnecessary address space pinning with
mm_structpinning by replacing mmget/mmput with mmgrab/mmdrop calls.mm_mtis contained withinmm_structitself, therefore it won't be freed as long asmm_structis stable and it won't change during the walk becausemmap_read_lockis being held.Fixes bd74fdaea146 ("mm: multi-gen LRU: support page table walks")Signed-off-by Suren Baghdasaryan <[email protected]>Oh and sorry, missed this - do we need a 'Fixed' for something that's just a perf improvement?
Quoted 8 lines
Quoted 1 line
This doesn't seem buggy?
Well, if we are pinning something we should not is that a bug? I didn't CC stable, so I don't expect it to be backported (though if it's accepted, we will backport it in Android as it causes real problems with reclaim), however if someone is looking for MGLRU fixes I would like this patch to be discoverable. If you feel strongly about it, I'll remove the tag.
Quoted 6 lines
I don't feel that strongly about it :>)
Quoted 2 lines
Thanks for the review!
No problem! Working through your vma killable series atm also :)
Thanks!
I will need to respin v5 after Sashiko found a couple of legit issues at [1] but I'll wait for you to post your feedback before sending it.
[1] https://sashiko.dev/#/patchset/[email protected]
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Thanks, Lorenzo
Cheers, Lorenzo