Skip to contentSkip to conversation

selftests: vm: add process_mrelease tests

PATCH Superseded Suren Baghdasaryan 3 participants 17 messages May 10, 2022 – May 16, 2022

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 selftests: vm: add process_mrelease tests

Introduce process_mrelease syscall sanity tests. They include tests of invalid pidfd and flags inputs, attempting to call process_mrelease with a live process and a valid usage of process_mrelease. Because process_mrelease has to be used against a process with a pending SIGKILL, it's possible that the process exits before process_mrelease gets called. In such cases we retry the test with a victim that allocates twice more memory up to 1GB. This would require the victim process to spend more time during exit and process_mrelease has a better chance of catching the process before it exits.

---
 tools/testing/selftests/vm/Makefile        |   1 +
 tools/testing/selftests/vm/mrelease_test.c | 176 +++++++++++++++++++++
 tools/testing/selftests/vm/run_vmtests.sh  |  16 ++
 3 files changed, 193 insertions(+)
 create mode 100644 tools/testing/selftests/vm/mrelease_test.c
tools/testing/selftests/vm/Makefile +1 -0
index 04a49e876a46..733fccbff0ef 100644
@@ -43,6 +43,7 @@ TEST_GEN_FILES += map_populate
 TEST_GEN_FILES += memfd_secret
 TEST_GEN_FILES += mlock-random-test
 TEST_GEN_FILES += mlock2-tests
+TEST_GEN_FILES += mrelease_test
 TEST_GEN_FILES += mremap_dontunmap
 TEST_GEN_FILES += mremap_test
 TEST_GEN_FILES += on-fault-limit
tools/testing/selftests/vm/mrelease_test.c new file +176 -0

176 lines not on this page · raw message

tools/testing/selftests/vm/run_vmtests.sh +16 -0
index 352ba00cf26b..1986162fea39 100755
@@ -287,6 +287,22 @@ else
 	echo "[PASS]"
 fi
 
+echo "---------------------"
+echo "running mrelease_test"
+echo "---------------------"
+./mrelease_test
+ret_val=$?
+
+if [ $ret_val -eq 0 ]; then
+	echo "[PASS]"
+elif [ $ret_val -eq $ksft_skip ]; then
+	 echo "[SKIP]"
+	 exitcode=$ksft_skip
+else
+	echo "[FAIL]"
+	exitcode=1
+fi
+
 echo "-------------------"
 echo "running mremap_test"
 echo "-------------------"
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
#1 lore raw

PATCH 2/3 mm: drop oom code from exit_mmap

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

---
 include/linux/oom.h |  2 --
 mm/mmap.c           | 25 ++++++-------------------
 mm/oom_kill.c       |  2 +-
 3 files changed, 7 insertions(+), 22 deletions(-)
include/linux/oom.h +0 -2
index 2db9a1432511..6cdf0772dbae 100644
@@ -106,8 +106,6 @@ static inline vm_fault_t check_stable_address_space(struct mm_struct *mm)
 	return 0;
 }
 
-bool __oom_reap_task_mm(struct mm_struct *mm);
-
 long oom_badness(struct task_struct *p,
 		unsigned long totalpages);
 
mm/mmap.c +6 -19
index 313b57d55a63..feaa840fb95d 100644
@@ -3105,30 +3105,13 @@ void exit_mmap(struct mm_struct *mm)
 	/* mm's last user has gone, and its about to be pulled down */
 	mmu_notifier_release(mm);
 
-	if (unlikely(mm_is_oom_victim(mm))) {
-		/*
-		 * Manually reap the mm to free as much memory as possible.
-		 * Then, as the oom reaper does, set MMF_OOM_SKIP to disregard
-		 * this mm from further consideration.  Taking mm->mmap_lock for
-		 * write after setting MMF_OOM_SKIP will guarantee that the oom
-		 * reaper will not run on this mm again after mmap_lock is
-		 * dropped.
-		 *
-		 * Nothing can be holding mm->mmap_lock here and the above call
-		 * to mmu_notifier_release(mm) ensures mmu notifier callbacks in
-		 * __oom_reap_task_mm() will not block.
-		 */
-		(void)__oom_reap_task_mm(mm);
-		set_bit(MMF_OOM_SKIP, &mm->flags);
-	}
-
-	mmap_write_lock(mm);
+	mmap_read_lock(mm);
 	arch_exit_mmap(mm);
 
 	vma = mm->mmap;
 	if (!vma) {
 		/* Can happen if dup_mmap() received an OOM */
-		mmap_write_unlock(mm);
+		mmap_read_unlock(mm);
 		return;
 	}
 
@@ -3138,6 +3121,10 @@ void exit_mmap(struct mm_struct *mm)
 	/* update_hiwater_rss(mm) here? but nobody should be looking */
 	/* Use -1 here to ensure all VMAs in the mm are unmapped */
 	unmap_vmas(&tlb, vma, 0, -1);
+	mmap_read_unlock(mm);
+	/* Set MMF_OOM_SKIP to disregard this mm from further consideration.*/
+	set_bit(MMF_OOM_SKIP, &mm->flags);
+	mmap_write_lock(mm);
 	free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING);
 	tlb_finish_mmu(&tlb);
 
mm/oom_kill.c +1 -1
index 49d7df39b02d..36355b162727 100644
@@ -509,7 +509,7 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait);
 static struct task_struct *oom_reaper_list;
 static DEFINE_SPINLOCK(oom_reaper_lock);
 
-bool __oom_reap_task_mm(struct mm_struct *mm)
+static bool __oom_reap_task_mm(struct mm_struct *mm)
 {
 	struct vm_area_struct *vma;
 	bool ret = true;
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
maintainer Acked-by on v1 2/3
#2 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

Quoted 6 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

The changelog is rather light on nasty details which might be good but for the sake of our future us let's be more verbose so that we do not have to reinvent the prior history each time we are looking into this code. I would go with something like this instead:
"
The primary reason to invoke the oom reaper from the exit_mmap path used to be a prevention of an excessive oom killing if the oom victim exit races with the oom reaper (see 212925802454 ("mm: oom: let oom_reap_task and exit_mmap run concurrently") for more details. The invocation has moved around since then because of the interaction with the munlock logic but the underlying reason has remained the same (see 27ae357fa82b ("mm, oom: fix concurrent munlock and oom reaper unmap, v3").

Munlock code is no longer a problem since a213e5cf71cb ("mm/munlock:
delete munlock_vma_pages_all(), allow oomreap") and there shouldn't be any blocking operation before the memory is unmapped by exit_mmap so the oom reaper invocation can be dropped. The unmapping part can be done with the non-exclusive mmap_sem and the exclusive one is only required when page tables are freed.

Remove the oom_reaper from exit_mmap which will make the code easier to read. This is really unlikely to make any observable difference although some microbenchmarks could benefit from one less branch that needs to be evaluated even though it almost never is true.
"

One minor comment below. Other than that \o/ this is finally going away. I strongly suspect that the history of this code is a nice example about how over optimizing code can cause more harm than good.

Thanks!

Quoted 8 lines
---
 include/linux/oom.h |  2 --
 mm/mmap.c           | 25 ++++++-------------------
 mm/oom_kill.c       |  2 +-
 3 files changed, 7 insertions(+), 22 deletions(-)

[...]

mm/mmap.c void exit_mmap(struct mm_struct *mm) · line 3126

Quoted 7 lines
mm/mmap.c +3 -0
@@ -3138,6 +3121,10 @@ void exit_mmap(struct mm_struct *mm)
 	/* update_hiwater_rss(mm) here? but nobody should be looking */
 	/* Use -1 here to ensure all VMAs in the mm are unmapped */
 	unmap_vmas(&tlb, vma, 0, -1);
+	mmap_read_unlock(mm);
+	/* Set MMF_OOM_SKIP to disregard this mm from further consideration.*/
+	set_bit(MMF_OOM_SKIP, &mm->flags);

I think that it would be slightly more readable to add an empty line above and below of this. Also the comment would be more helpful if it explaind what the further consideration actually means. I would go with

/*
* Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
* because the memory has been already freed. Do not bother
* checking mm_is_oom_victim because setting a bit
* unconditionally is just cheaper.
*/

Quoted 3 lines
mm/mmap.c +1 -0
+	mmap_write_lock(mm);
 	free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING);
 	tlb_finish_mmu(&tlb);
Signature · 2 lines
Michal Hocko
SUSE Labs
#3 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

Quoted 29 lines · Michal Hocko
Quoted 6 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

The changelog is rather light on nasty details which might be good but for the sake of our future us let's be more verbose so that we do not have to reinvent the prior history each time we are looking into this code. I would go with something like this instead:
"
The primary reason to invoke the oom reaper from the exit_mmap path used to be a prevention of an excessive oom killing if the oom victim exit races with the oom reaper (see 212925802454 ("mm: oom: let oom_reap_task and exit_mmap run concurrently") for more details. The invocation has moved around since then because of the interaction with the munlock logic but the underlying reason has remained the same (see 27ae357fa82b ("mm, oom: fix concurrent munlock and oom reaper unmap, v3").

Munlock code is no longer a problem since a213e5cf71cb ("mm/munlock:
delete munlock_vma_pages_all(), allow oomreap") and there shouldn't be any blocking operation before the memory is unmapped by exit_mmap so the oom reaper invocation can be dropped. The unmapping part can be done with the non-exclusive mmap_sem and the exclusive one is only required when page tables are freed.

Quoted 5 lines

Remove the oom_reaper from exit_mmap which will make the code easier to read. This is really unlikely to make any observable difference although some microbenchmarks could benefit from one less branch that needs to be evaluated even though it almost never is true.
"

Looks great! Thanks for collecting all the history. Will update the description.

Quoted 6 lines

One minor comment below. Other than that \o/ this is finally going away. I strongly suspect that the history of this code is a nice example about how over optimizing code can cause more harm than good.

Thanks.

Quoted 23 lines

Thanks!

Quoted 8 lines
---
 include/linux/oom.h |  2 --
 mm/mmap.c           | 25 ++++++-------------------
 mm/oom_kill.c       |  2 +-
 3 files changed, 7 insertions(+), 22 deletions(-)

[...]

mm/mmap.c void exit_mmap(struct mm_struct *mm) · line 3126

Quoted 7 lines
mm/mmap.c +3 -0
@@ -3138,6 +3121,10 @@ void exit_mmap(struct mm_struct *mm)
      /* update_hiwater_rss(mm) here? but nobody should be looking */
      /* Use -1 here to ensure all VMAs in the mm are unmapped */
      unmap_vmas(&tlb, vma, 0, -1);
+     mmap_read_unlock(mm);
+     /* Set MMF_OOM_SKIP to disregard this mm from further consideration.*/
+     set_bit(MMF_OOM_SKIP, &mm->flags);

I think that it would be slightly more readable to add an empty line above and below of this. Also the comment would be more helpful if it explaind what the further consideration actually means. I would go with

Quoted 7 lines

/*
* Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
* because the memory has been already freed. Do not bother
* checking mm_is_oom_victim because setting a bit
* unconditionally is just cheaper.
*/

Ack.

Quoted 7 lines

mm/mmap.c void exit_mmap(struct mm_struct *mm) · line 3129

Quoted 3 lines
mm/mmap.c +1 -0
+     mmap_write_lock(mm);
      free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING);
      tlb_finish_mmu(&tlb);

--
Michal Hocko
SUSE Labs

maintainer
#4 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

Quoted 30 lines · Suren Baghdasaryan
Quoted 29 lines · Michal Hocko
Quoted 6 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

The changelog is rather light on nasty details which might be good but for the sake of our future us let's be more verbose so that we do not have to reinvent the prior history each time we are looking into this code. I would go with something like this instead:
"
The primary reason to invoke the oom reaper from the exit_mmap path used to be a prevention of an excessive oom killing if the oom victim exit races with the oom reaper (see 212925802454 ("mm: oom: let oom_reap_task and exit_mmap run concurrently") for more details. The invocation has moved around since then because of the interaction with the munlock logic but the underlying reason has remained the same (see 27ae357fa82b ("mm, oom: fix concurrent munlock and oom reaper unmap, v3").

Munlock code is no longer a problem since a213e5cf71cb ("mm/munlock:
delete munlock_vma_pages_all(), allow oomreap") and there shouldn't be any blocking operation before the memory is unmapped by exit_mmap so the oom reaper invocation can be dropped. The unmapping part can be done with the non-exclusive mmap_sem and the exclusive one is only required when page tables are freed.

Quoted 7 lines
Quoted 5 lines

Remove the oom_reaper from exit_mmap which will make the code easier to read. This is really unlikely to make any observable difference although some microbenchmarks could benefit from one less branch that needs to be evaluated even though it almost never is true.
"

Looks great! Thanks for collecting all the history. Will update the description.

Please make sure you double check the story. This is mostly my recollection and brief reading through the said commits. I might misremember here and there.

Signature · 2 lines
Michal Hocko
SUSE Labs
#5 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

Quoted 38 lines · Michal Hocko
Quoted 36 lines · Suren Baghdasaryan
Quoted 34 lines · Michal Hocko
Quoted 6 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

The changelog is rather light on nasty details which might be good but for the sake of our future us let's be more verbose so that we do not have to reinvent the prior history each time we are looking into this code. I would go with something like this instead:
"
The primary reason to invoke the oom reaper from the exit_mmap path used to be a prevention of an excessive oom killing if the oom victim exit races with the oom reaper (see 212925802454 ("mm: oom: let oom_reap_task and exit_mmap run concurrently") for more details. The invocation has moved around since then because of the interaction with the munlock logic but the underlying reason has remained the same (see 27ae357fa82b ("mm, oom: fix concurrent munlock and oom reaper unmap, v3").

Munlock code is no longer a problem since a213e5cf71cb ("mm/munlock:
delete munlock_vma_pages_all(), allow oomreap") and there shouldn't be any blocking operation before the memory is unmapped by exit_mmap so the oom reaper invocation can be dropped. The unmapping part can be done with the non-exclusive mmap_sem and the exclusive one is only required when page tables are freed.

Remove the oom_reaper from exit_mmap which will make the code easier to read. This is really unlikely to make any observable difference although some microbenchmarks could benefit from one less branch that needs to be evaluated even though it almost never is true.
"

Quoted 5 lines
Quoted 1 line

Looks great! Thanks for collecting all the history. Will update the description.

Please make sure you double check the story. This is mostly my recollection and brief reading through the said commits. I might misremember here and there.

Will do. Thanks!

Quoted 3 lines

--
Michal Hocko
SUSE Labs

maintainer
#6 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

In reply to Suren Baghdasaryan
Quoted 7 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

Quoted 7 lines
---
  include/linux/oom.h |  2 --
  mm/mmap.c           | 25 ++++++-------------------
  mm/oom_kill.c       |  2 +-
  3 files changed, 7 insertions(+), 22 deletions(-)

How does this improve the test? Include the information on why this change is needed as opposed describing what this does?

thanks,
-- Shuah

#7 lore raw

Re: [PATCH 2/3] mm: drop oom code from exit_mmap

Quoted 17 lines · Shuah Khan
Quoted 7 lines · Suren Baghdasaryan

With the oom-killer being able to operate on locked pages, exit_mmap does not need to ensure that oom_reap_task_mm is done before it can proceed. Instead it can rely on mmap_lock write lock to prevent oom-killer from operating on the vma tree while it's freeing page tables. exit_mmap can hold mmap_lock read lock when unmapping vmas and then take mmap_lock write lock before freeing page tables.

Quoted 7 lines
---
  include/linux/oom.h |  2 --
  mm/mmap.c           | 25 ++++++-------------------
  mm/oom_kill.c       |  2 +-
  3 files changed, 7 insertions(+), 22 deletions(-)
Quoted 2 lines

How does this improve the test? Include the information on why this change is needed as opposed describing what this does?

It doesn't improve the test. I used the test to verify this change and wanted to keep them together so that others have an easy way to exercise the same code path. That's the only relation between the test and this cleanup. I'll split them into separate patchsets to avoid further confusion.

Quoted 3 lines

thanks,
-- Shuah

#8 lore raw

PATCH 3/3 mm: delete unused MMF_OOM_VICTIM flag

In reply to Suren Baghdasaryan

With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed.

---
 include/linux/oom.h            | 9 ---------
 include/linux/sched/coredump.h | 1 -
 mm/oom_kill.c                  | 4 +---
 3 files changed, 1 insertion(+), 13 deletions(-)
include/linux/oom.h +0 -9
index 6cdf0772dbae..25990e9d9e15 100644
@@ -77,15 +77,6 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk)
 	return tsk->signal->oom_mm;
 }
 
-/*
- * Use this helper if tsk->mm != mm and the victim mm needs a special
- * handling. This is guaranteed to stay true after once set.
- */
-static inline bool mm_is_oom_victim(struct mm_struct *mm)
-{
-	return test_bit(MMF_OOM_VICTIM, &mm->flags);
-}
-
 /*
  * Checks whether a page fault on the given mm is still reliable.
  * This is no longer true if the oom reaper started to reap the
include/linux/sched/coredump.h +0 -1
index 4d9e3a656875..746f6cb07a20 100644
@@ -70,7 +70,6 @@ static inline int get_dumpable(struct mm_struct *mm)
 #define MMF_UNSTABLE		22	/* mm is unstable for copy_from_user */
 #define MMF_HUGE_ZERO_PAGE	23      /* mm has ever used the global huge zero page */
 #define MMF_DISABLE_THP		24	/* disable THP for all VMAs */
-#define MMF_OOM_VICTIM		25	/* mm is the oom victim */
 #define MMF_OOM_REAP_QUEUED	26	/* mm was queued for oom_reaper */
 #define MMF_MULTIPROCESS	27	/* mm is shared between processes */
 /*
mm/oom_kill.c +1 -3
index 36355b162727..11291b99599f 100644
@@ -732,10 +732,8 @@ static void mark_oom_victim(struct task_struct *tsk)
 		return;
 
 	/* oom_mm is bound to the signal struct life time. */
-	if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) {
+	if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm))
 		mmgrab(tsk->signal->oom_mm);
-		set_bit(MMF_OOM_VICTIM, &mm->flags);
-	}
 
 	/*
 	 * Make sure that the task is woken up from uninterruptible sleep
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
maintainer Acked-by on v1 3/3
#9 lore raw

Re: [PATCH 3/3] mm: delete unused MMF_OOM_VICTIM flag

Quoted 4 lines · Suren Baghdasaryan

With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed.

LGTM

One question below
[...]

include/linux/sched/coredump.h static inline int get_dumpable(struct mm_struct *mm) · line 74

Quoted 11 lines
include/linux/sched/coredump.h +0 -1
index 4d9e3a656875..746f6cb07a20 100644
@@ -70,7 +70,6 @@ static inline int get_dumpable(struct mm_struct *mm)
 #define MMF_UNSTABLE		22	/* mm is unstable for copy_from_user */
 #define MMF_HUGE_ZERO_PAGE	23      /* mm has ever used the global huge zero page */
 #define MMF_DISABLE_THP		24	/* disable THP for all VMAs */
-#define MMF_OOM_VICTIM		25	/* mm is the oom victim */
 #define MMF_OOM_REAP_QUEUED	26	/* mm was queued for oom_reaper */
 #define MMF_MULTIPROCESS	27	/* mm is shared between processes */

Have you consider renumbering the follow up flags so that we do not have holes in there. Nothing really important but it can confuse somebody in the future.

Signature · 2 lines
Michal Hocko
SUSE Labs
#10 lore raw

Re: [PATCH 3/3] mm: delete unused MMF_OOM_VICTIM flag

Quoted 12 lines · Michal Hocko
Quoted 4 lines · Suren Baghdasaryan

With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed.

LGTM

One question below
[...]

Quoted 15 lines

include/linux/sched/coredump.h static inline int get_dumpable(struct mm_struct *mm) · line 74

Quoted 11 lines
include/linux/sched/coredump.h +0 -1
index 4d9e3a656875..746f6cb07a20 100644
@@ -70,7 +70,6 @@ static inline int get_dumpable(struct mm_struct *mm)
 #define MMF_UNSTABLE         22      /* mm is unstable for copy_from_user */
 #define MMF_HUGE_ZERO_PAGE   23      /* mm has ever used the global huge zero page */
 #define MMF_DISABLE_THP              24      /* disable THP for all VMAs */
-#define MMF_OOM_VICTIM               25      /* mm is the oom victim */
 #define MMF_OOM_REAP_QUEUED  26      /* mm was queued for oom_reaper */
 #define MMF_MULTIPROCESS     27      /* mm is shared between processes */

Have you consider renumbering the follow up flags so that we do not have holes in there. Nothing really important but it can confuse somebody in the future.

Missed this note until now. I will renumber the constants to avoid confusion. Thanks,
Suren.

Quoted 4 lines

--
Michal Hocko
SUSE Labs

maintainer
#11 lore raw

Re: [PATCH 3/3] mm: delete unused MMF_OOM_VICTIM flag

In reply to Suren Baghdasaryan
Quoted 49 lines · Suren Baghdasaryan

With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed.

---
  include/linux/oom.h            | 9 ---------
  include/linux/sched/coredump.h | 1 -
  mm/oom_kill.c                  | 4 +---
  3 files changed, 1 insertion(+), 13 deletions(-)
include/linux/oom.h +0 -9
index 6cdf0772dbae..25990e9d9e15 100644
@@ -77,15 +77,6 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk)
  	return tsk->signal->oom_mm;
  }
 
-/*
- * Use this helper if tsk->mm != mm and the victim mm needs a special
- * handling. This is guaranteed to stay true after once set.
- */
-static inline bool mm_is_oom_victim(struct mm_struct *mm)
-{
-	return test_bit(MMF_OOM_VICTIM, &mm->flags);
-}
-
  /*
   * Checks whether a page fault on the given mm is still reliable.
   * This is no longer true if the oom reaper started to reap the
include/linux/sched/coredump.h +0 -1
index 4d9e3a656875..746f6cb07a20 100644
@@ -70,7 +70,6 @@ static inline int get_dumpable(struct mm_struct *mm)
  #define MMF_UNSTABLE		22	/* mm is unstable for copy_from_user */
  #define MMF_HUGE_ZERO_PAGE	23      /* mm has ever used the global huge zero page */
  #define MMF_DISABLE_THP		24	/* disable THP for all VMAs */
-#define MMF_OOM_VICTIM		25	/* mm is the oom victim */
  #define MMF_OOM_REAP_QUEUED	26	/* mm was queued for oom_reaper */
  #define MMF_MULTIPROCESS	27	/* mm is shared between processes */
  /*
mm/oom_kill.c +0 -0
index 36355b162727..11291b99599f 100644
@@ -732,10 +732,8 @@ static void mark_oom_victim(struct task_struct *tsk)
  		return;

mm/oom_kill.c static void mark_oom_victim(struct task_struct *tsk) · line 739

Quoted 10 lines
mm/oom_kill.c +1 -3
 
  	/* oom_mm is bound to the signal struct life time. */
-	if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) {
+	if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm))
  		mmgrab(tsk->signal->oom_mm);
-		set_bit(MMF_OOM_VICTIM, &mm->flags);
-	}
 
  	/*
  	 * Make sure that the task is woken up from uninterruptible sleep

Thank you for working on the new tests and cleanups.

This series needs a cover-letter that explains why this series is needed that includes the information from this last patch.

Please send v2 with a proper cover letter starting with why this series is necessary. If you did that, it would have reviewers job is lot easier.

Also it appears you are combining new tests with cleanup patches. I think patches 2/3 and 3/3 can be a separate series and the new test can be a separate patch.

thanks,
-- Shuah

#12 lore raw

Re: [PATCH 3/3] mm: delete unused MMF_OOM_VICTIM flag

Quoted 58 lines · Shuah Khan
Quoted 56 lines · Suren Baghdasaryan

With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed.

---
  include/linux/oom.h            | 9 ---------
  include/linux/sched/coredump.h | 1 -
  mm/oom_kill.c                  | 4 +---
  3 files changed, 1 insertion(+), 13 deletions(-)
include/linux/oom.h +0 -9
index 6cdf0772dbae..25990e9d9e15 100644
@@ -77,15 +77,6 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk)
      return tsk->signal->oom_mm;
  }
 
-/*
- * Use this helper if tsk->mm != mm and the victim mm needs a special
- * handling. This is guaranteed to stay true after once set.
- */
-static inline bool mm_is_oom_victim(struct mm_struct *mm)
-{
-     return test_bit(MMF_OOM_VICTIM, &mm->flags);
-}
-
  /*
   * Checks whether a page fault on the given mm is still reliable.
   * This is no longer true if the oom reaper started to reap the
include/linux/sched/coredump.h +0 -1
index 4d9e3a656875..746f6cb07a20 100644
@@ -70,7 +70,6 @@ static inline int get_dumpable(struct mm_struct *mm)
  #define MMF_UNSTABLE                22      /* mm is unstable for copy_from_user */
  #define MMF_HUGE_ZERO_PAGE  23      /* mm has ever used the global huge zero page */
  #define MMF_DISABLE_THP             24      /* disable THP for all VMAs */
-#define MMF_OOM_VICTIM               25      /* mm is the oom victim */
  #define MMF_OOM_REAP_QUEUED 26      /* mm was queued for oom_reaper */
  #define MMF_MULTIPROCESS    27      /* mm is shared between processes */
  /*
mm/oom_kill.c +1 -3
index 36355b162727..11291b99599f 100644
@@ -732,10 +732,8 @@ static void mark_oom_victim(struct task_struct *tsk)
              return;
 
      /* oom_mm is bound to the signal struct life time. */
-     if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) {
+     if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm))
              mmgrab(tsk->signal->oom_mm);
-             set_bit(MMF_OOM_VICTIM, &mm->flags);
-     }
 
Quoted 5 lines

mm/oom_kill.c static void mark_oom_victim(struct task_struct *tsk) · line 739

Quoted 3 lines
mm/oom_kill.c +0 -0
      /*
       * Make sure that the task is woken up from uninterruptible sleep

Thank you for working on the new tests and cleanups.

Thanks for the review!

Quoted 10 lines

This series needs a cover-letter that explains why this series is needed that includes the information from this last patch.

Please send v2 with a proper cover letter starting with why this series is necessary. If you did that, it would have reviewers job is lot easier.

Also it appears you are combining new tests with cleanup patches. I think patches 2/3 and 3/3 can be a separate series and the new test can be a separate patch.

I used the new selftest to test the patches but otherwise it's true, they are unrelated. I was debating whether to send them separately and with your blessing I'll split them up.
Thanks,
Suren.

Quoted 3 lines

thanks,
-- Shuah

maintainer
#13 lore raw
In reply to Suren Baghdasaryan
Quoted 10 lines · Suren Baghdasaryan

Introduce process_mrelease syscall sanity tests. They include tests of invalid pidfd and flags inputs, attempting to call process_mrelease with a live process and a valid usage of process_mrelease. Because process_mrelease has to be used against a process with a pending SIGKILL, it's possible that the process exits before process_mrelease gets called. In such cases we retry the test with a victim that allocates twice more memory up to 1GB. This would require the victim process to spend more time during exit and process_mrelease has a better chance of catching the process before it exits.

+1 on Mike's comments on improving the change log. List what is getting tested as opposed to describing the test code.

Quoted 7 lines
---
  tools/testing/selftests/vm/Makefile        |   1 +
  tools/testing/selftests/vm/mrelease_test.c | 176 +++++++++++++++++++++
  tools/testing/selftests/vm/run_vmtests.sh  |  16 ++
  3 files changed, 193 insertions(+)

create mode 100644 tools/testing/selftests/vm/mrelease_test.c

Please update .gitignore with the new executable.

Quoted 28 lines
tools/testing/selftests/vm/Makefile +1 -0
index 04a49e876a46..733fccbff0ef 100644
@@ -43,6 +43,7 @@ TEST_GEN_FILES += map_populate
  TEST_GEN_FILES += memfd_secret
  TEST_GEN_FILES += mlock-random-test
  TEST_GEN_FILES += mlock2-tests
+TEST_GEN_FILES += mrelease_test
  TEST_GEN_FILES += mremap_dontunmap
  TEST_GEN_FILES += mremap_test
  TEST_GEN_FILES += on-fault-limit
tools/testing/selftests/vm/mrelease_test.c new file +9 -0
new file mode 100644
index 000000000000..a61061bf8433
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 Google LLC
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/wait.h>

tools/testing/selftests/vm/mrelease_test.c line 19

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+#include <unistd.h>
+
+#include "util.h"
+
+static inline int pidfd_open(pid_t pid, unsigned int flags)
+{
+#ifdef __NR_pidfd_open
+	return syscall(__NR_pidfd_open, pid, flags);
+#else
+	errno = ENOSYS;

This isn't an error - this would be skip because this syscall isn't supported.

tools/testing/selftests/vm/mrelease_test.c line 21

Quoted 2 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+	return -1;
+#endif

Key off of syscall return instead of these ifdefs - same comment on all of the ifdefs

tools/testing/selftests/vm/mrelease_test.c line 23

Quoted 2 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+}
+

I am not seeing any reason for breaking this code up have a separate routine for pidfd_open().

tools/testing/selftests/vm/mrelease_test.c line 33

Quoted 9 lines
tools/testing/selftests/vm/mrelease_test.c new file +9 -0
+static inline int process_mrelease(int pidfd, unsigned int flags)
+{
+#ifdef __NR_process_mrelease
+	return syscall(__NR_process_mrelease, pidfd, flags);
+#else
+	errno = ENOSYS;
+	return -1;
+#endif> +}
+

Same comments on ifdefs and skips here as well.

Quoted 7 lines
tools/testing/selftests/vm/mrelease_test.c new file +7 -0
+static void write_fault_pages(char *addr, unsigned long nr_pages)
+{
+	unsigned long i;
+
+	for (i = 0; i < nr_pages; i++)
+		*((unsigned long *)(addr + (i * PAGE_SIZE))) = i;
+}

tools/testing/selftests/vm/mrelease_test.c line 50

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+
+static int alloc_noexit(unsigned long nr_pages, int pipefd)
+{
+	int ppid = getppid();
+	void *buf;
+
+	buf = mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
+		   MAP_PRIVATE | MAP_ANON, 0, 0);
+	if (buf == MAP_FAILED) {
+		perror("mmap");

A bit more descriptive message what the test would do will be helpful. Also consider if this should be a skip or fail for the test.

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+		return 1;
+	}
+
+	write_fault_pages((char *)buf, nr_pages);
+

tools/testing/selftests/vm/mrelease_test.c line 65

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+	/* Signal the parent that the child is ready */
+	if (write(pipefd, "", 1) < 0) {
+		perror("write");
+		return 1;
+	}
+
+	/* Wait to be killed (when reparenting happens) */
+	while (getppid() == ppid)
+		sleep(1);
+

What happens if reparenting doesn't happen? Will this loop for ever? This test could hang?

Quoted 11 lines
tools/testing/selftests/vm/mrelease_test.c new file +11 -0
+	munmap(buf, nr_pages * PAGE_SIZE);
+
+	return 0;
+}
+
+
+#define MB(x) (x << 20)
+#define MAX_SIZE_MB 1024
+
+int main(void)
+{

tools/testing/selftests/vm/mrelease_test.c line 86

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+	int res;
+	int pipefd[2], pidfd;
+	pid_t pid;
+	char byte;
+	size_t size;
+	int negative_tests_done = 0;
+
+	/* Test a wrong pidfd */
+	if (!process_mrelease(-1, 0) || errno != EBADF) {
+		perror("process_mrelease with wring pidfd");

Incorrect spelling "wring/wrong"

Quoted 9 lines
tools/testing/selftests/vm/mrelease_test.c new file +9 -0
+		exit(1);
+	}
+
+	/*
+	 * Start the test with 1MB allocation and double every time
+	 * process_mrelease fails
+	 */
+	for (size = 1; size <= MAX_SIZE_MB; size *= 2) {
+		/*

tools/testing/selftests/vm/mrelease_test.c line 105

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+		 * Pipe for the child to signal when it's done allocating
+		 * memory
+		 */
+		if (pipe(pipefd)) {
+			perror("pipe");
+			exit(1);
+		}
+		pid = fork();
+		if (pid < 0) {
+			perror("fork");

Close the pipe?

Quoted 15 lines
tools/testing/selftests/vm/mrelease_test.c new file +15 -0
+			exit(1);
+		}
+
+		if (pid == 0) {
+			close(pipefd[0]);
+			res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
+			close(pipefd[1]);
+			exit(res);
+		}
+
+		close(pipefd[1]);
+		/* Block until the child is ready */
+		res = read(pipefd[0], &byte, 1);
+		close(pipefd[0]);
+		if (res < 0) {

tools/testing/selftests/vm/mrelease_test.c line 130

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+			perror("read");
+			exit(1);
+		}
+
+		pidfd = pidfd_open(pid, 0);
+		if (pidfd < 0) {
+			perror("pidfd_open");
+			exit(1);
+		}
+

The code is very hard to read. Add comments to indicate parent and child paths clearly so reviewers can follow the logic and be able to do effective review.

tools/testing/selftests/vm/mrelease_test.c line 137

Quoted 7 lines
tools/testing/selftests/vm/mrelease_test.c new file +7 -0
+		/* Run negative tests which require a valid child only once */
+		if (!negative_tests_done) {
+			/* Test invalid flags */
+			if (!process_mrelease(pidfd, (unsigned int)-1) ||
+			    errno != EINVAL) {
+				perror("process_mrelease with wrong flags");
+				exit(1);

So is this an expected fail or a test fail?

tools/testing/selftests/vm/mrelease_test.c line 142

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+			}
+			/* Test reapling while process is still alive */
+			if (!process_mrelease(pidfd, 0) ||
+			    errno != EINVAL) {
+				perror("process_mrelease on a live process");

So is this an expected fail or a test fail?

tools/testing/selftests/vm/mrelease_test.c line 146

Quoted 4 lines
tools/testing/selftests/vm/mrelease_test.c new file +4 -0
+				exit(1);
+			}
+			negative_tests_done = 1;
+		}

Now the above negative_tests_done block could be in a separate function --- All the others aren't really needed. It will be good for abstraction and readability.

tools/testing/selftests/vm/mrelease_test.c line 149

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +3 -0
+
+		if (kill(pid, SIGKILL)) {
+			perror("kill");

Include test results in the change log - so we can see the test report.

tools/testing/selftests/vm/mrelease_test.c line 161

Quoted 12 lines
tools/testing/selftests/vm/mrelease_test.c new file +12 -0
+			exit(1);
+		}
+
+		if (!process_mrelease(pidfd, 0)) {
+			/* Terminate the test once process_mrelease succeeds */
+			return 0;
+		}
+
+		/*
+		 * Ignore the failure if the child exited before mrelease got
+		 * called, increase allocation size and retry the test
+		 */

Add more info. on why allocating more memory helps.

Quoted 33 lines
tools/testing/selftests/vm/mrelease_test.c new file +15 -0
+		if (errno != ESRCH) {
+			perror("process_mrelease");
+			exit(1);
+		}
+
+		if (waitpid(pid, NULL, 0) < 0) {
+			perror("waitpid");
+			exit(1);
+		}
+		close(pidfd);
+	}
+
+	printf("All process_mrelease attempts failed!\n");
+	exit(1);
+}
tools/testing/selftests/vm/run_vmtests.sh +9 -0
index 352ba00cf26b..1986162fea39 100755
@@ -287,6 +287,22 @@ else
  	echo "[PASS]"
  fi
 
+echo "---------------------"
+echo "running mrelease_test"
+echo "---------------------"
+./mrelease_test
+ret_val=$?
+
+if [ $ret_val -eq 0 ]; then
+	echo "[PASS]"
+elif [ $ret_val -eq $ksft_skip ]; then

tools/testing/selftests/vm/run_vmtests.sh else · line 308

Quoted 10 lines
tools/testing/selftests/vm/run_vmtests.sh +7 -0
+	 echo "[SKIP]"
+	 exitcode=$ksft_skip
+else
+	echo "[FAIL]"
+	exitcode=1
+fi
+
  echo "-------------------"
  echo "running mremap_test"
  echo "-------------------"

In general, the code flow is hard to read to make sure resources are released e.g: pipefd in all the error paths. The code is broken up into smaller chunks where it isn't needed in some cases and left as a large block when it could benefit from abstraction e.g: negative test block.

Please make changes and send v2.

thanks,
-- Shuah

#14 lore raw
Quoted 13 lines · Shuah Khan
Quoted 10 lines · Suren Baghdasaryan

Introduce process_mrelease syscall sanity tests. They include tests of invalid pidfd and flags inputs, attempting to call process_mrelease with a live process and a valid usage of process_mrelease. Because process_mrelease has to be used against a process with a pending SIGKILL, it's possible that the process exits before process_mrelease gets called. In such cases we retry the test with a victim that allocates twice more memory up to 1GB. This would require the victim process to spend more time during exit and process_mrelease has a better chance of catching the process before it exits.

Quoted 2 lines

+1 on Mike's comments on improving the change log. List what is getting tested as opposed to describing the test code.

I'll try to improve the description but IMHO it does describe what it's testing - the process_mrelease syscall with valid and invalid inputs. I could omit the implementation details if that helps.

Quoted 10 lines
Quoted 8 lines
---
  tools/testing/selftests/vm/Makefile        |   1 +
  tools/testing/selftests/vm/mrelease_test.c | 176 +++++++++++++++++++++
  tools/testing/selftests/vm/run_vmtests.sh  |  16 ++
  3 files changed, 193 insertions(+)

create mode 100644 tools/testing/selftests/vm/mrelease_test.c

Please update .gitignore with the new executable.

Ack.

Quoted 34 lines
Quoted 34 lines
tools/testing/selftests/vm/Makefile +1 -0
index 04a49e876a46..733fccbff0ef 100644
@@ -43,6 +43,7 @@ TEST_GEN_FILES += map_populate
  TEST_GEN_FILES += memfd_secret
  TEST_GEN_FILES += mlock-random-test
  TEST_GEN_FILES += mlock2-tests
+TEST_GEN_FILES += mrelease_test
  TEST_GEN_FILES += mremap_dontunmap
  TEST_GEN_FILES += mremap_test
  TEST_GEN_FILES += on-fault-limit
tools/testing/selftests/vm/mrelease_test.c new file +14 -0
new file mode 100644
index 000000000000..a61061bf8433
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 Google LLC
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#include "util.h"
+
+static inline int pidfd_open(pid_t pid, unsigned int flags)
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 19

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+{
+#ifdef __NR_pidfd_open
+     return syscall(__NR_pidfd_open, pid, flags);
+#else
+     errno = ENOSYS;

This isn't an error - this would be skip because this syscall isn't supported.

Ack.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 21

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+     return -1;
+#endif

Key off of syscall return instead of these ifdefs - same comment on all of the ifdefs

Ack. I was using some other test as an example but I guess that was not a good model.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 23

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+}
+

I am not seeing any reason for breaking this code up have a separate routine for pidfd_open().

I'm a bit unclear what you mean. Do you mean that userspace headers should already define pidfd_open() and I don't need to define it?

Quoted 12 lines

tools/testing/selftests/vm/mrelease_test.c line 33

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +9 -0
+static inline int process_mrelease(int pidfd, unsigned int flags)
+{
+#ifdef __NR_process_mrelease
+     return syscall(__NR_process_mrelease, pidfd, flags);
+#else
+     errno = ENOSYS;
+     return -1;
+#endif> +}
+

Same comments on ifdefs and skips here as well.

Ack.

Quoted 13 lines
Quoted 13 lines
tools/testing/selftests/vm/mrelease_test.c new file +12 -0
+static void write_fault_pages(char *addr, unsigned long nr_pages)
+{
+     unsigned long i;
+
+     for (i = 0; i < nr_pages; i++)
+             *((unsigned long *)(addr + (i * PAGE_SIZE))) = i;
+}
+
+static int alloc_noexit(unsigned long nr_pages, int pipefd)
+{
+     int ppid = getppid();
+     void *buf;
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 50

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+
+     buf = mmap(NULL, nr_pages * PAGE_SIZE, PROT_READ | PROT_WRITE,
+                MAP_PRIVATE | MAP_ANON, 0, 0);
+     if (buf == MAP_FAILED) {
+             perror("mmap");

A bit more descriptive message what the test would do will be helpful. Also consider if this should be a skip or fail for the test.

I guess the question is whether an OOM condition should be considered a test failure? I guess we could skip since the test is specifically for process_mrelease.

Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+             return 1;
+     }
+
+     write_fault_pages((char *)buf, nr_pages);
+
+     /* Signal the parent that the child is ready */
+     if (write(pipefd, "", 1) < 0) {
+             perror("write");
+             return 1;
+     }
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 65

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+
+     /* Wait to be killed (when reparenting happens) */
+     while (getppid() == ppid)
+             sleep(1);
+

What happens if reparenting doesn't happen? Will this loop for ever? This test could hang?

Technically that should not happen because we kill the child and after that it has to reparent but to be safe I'll add a timeout to prevent an infinite loop.

Quoted 16 lines
Quoted 16 lines
tools/testing/selftests/vm/mrelease_test.c new file +15 -0
+     munmap(buf, nr_pages * PAGE_SIZE);
+
+     return 0;
+}
+
+
+#define MB(x) (x << 20)
+#define MAX_SIZE_MB 1024
+
+int main(void)
+{
+     int res;
+     int pipefd[2], pidfd;
+     pid_t pid;
+     char byte;
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 86

Quoted 6 lines
tools/testing/selftests/vm/mrelease_test.c new file +6 -0
+     size_t size;
+     int negative_tests_done = 0;
+
+     /* Test a wrong pidfd */
+     if (!process_mrelease(-1, 0) || errno != EBADF) {
+             perror("process_mrelease with wring pidfd");

Incorrect spelling "wring/wrong"

Ack.

Quoted 21 lines
Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +9 -0
+             exit(1);
+     }
+
+     /*
+      * Start the test with 1MB allocation and double every time
+      * process_mrelease fails
+      */
+     for (size = 1; size <= MAX_SIZE_MB; size *= 2) {
+             /*

tools/testing/selftests/vm/mrelease_test.c line 105

Quoted 10 lines
tools/testing/selftests/vm/mrelease_test.c new file +10 -0
+              * Pipe for the child to signal when it's done allocating
+              * memory
+              */
+             if (pipe(pipefd)) {
+                     perror("pipe");
+                     exit(1);
+             }
+             pid = fork();
+             if (pid < 0) {
+                     perror("fork");
Quoted 1 line

Close the pipe?

Ack.
Although I though all FDs are closed implicitly when the process exits (which is the case in here).

Quoted 22 lines
Quoted 22 lines
tools/testing/selftests/vm/mrelease_test.c new file +21 -0
+                     exit(1);
+             }
+
+             if (pid == 0) {
+                     close(pipefd[0]);
+                     res = alloc_noexit(MB(size) / PAGE_SIZE, pipefd[1]);
+                     close(pipefd[1]);
+                     exit(res);
+             }
+
+             close(pipefd[1]);
+             /* Block until the child is ready */
+             res = read(pipefd[0], &byte, 1);
+             close(pipefd[0]);
+             if (res < 0) {
+                     perror("read");
+                     exit(1);
+             }
+
+             pidfd = pidfd_open(pid, 0);
+             if (pidfd < 0) {
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 130

Quoted 4 lines
tools/testing/selftests/vm/mrelease_test.c new file +4 -0
+                     perror("pidfd_open");
+                     exit(1);
+             }
+

The code is very hard to read. Add comments to indicate parent and child paths clearly so reviewers can follow the logic and be able to do effective review.

Ack.

Quoted 10 lines

tools/testing/selftests/vm/mrelease_test.c line 137

Quoted 8 lines
tools/testing/selftests/vm/mrelease_test.c new file +7 -0
+             /* Run negative tests which require a valid child only once */
+             if (!negative_tests_done) {
+                     /* Test invalid flags */
+                     if (!process_mrelease(pidfd, (unsigned int)-1) ||
+                         errno != EINVAL) {
+                             perror("process_mrelease with wrong flags");
+                             exit(1);

So is this an expected fail or a test fail?

We expect it to fail with EINVAL. We fail the test if the call succeeds or fails for any other reason.

Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 142

Quoted 6 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+                     }
+                     /* Test reapling while process is still alive */
+                     if (!process_mrelease(pidfd, 0) ||
+                         errno != EINVAL) {
+                             perror("process_mrelease on a live process");

So is this an expected fail or a test fail?

Expected to fail with EINVAL.

Quoted 9 lines

tools/testing/selftests/vm/mrelease_test.c line 146

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +4 -0
+                             exit(1);
+                     }
+                     negative_tests_done = 1;
+             }

Now the above negative_tests_done block could be in a separate function --- All the others aren't really needed. It will be good for abstraction and readability.

I'll separate them into a stand-alone function.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 149

Quoted 4 lines
tools/testing/selftests/vm/mrelease_test.c new file +3 -0
+
+             if (kill(pid, SIGKILL)) {
+                     perror("kill");

Include test results in the change log - so we can see the test report.

Ack. Although the test does not print any results on success, it just succeeds or fails with an error description. I'll add outputs for both cases.

Quoted 15 lines

tools/testing/selftests/vm/mrelease_test.c line 161

Quoted 13 lines
tools/testing/selftests/vm/mrelease_test.c new file +12 -0
+                     exit(1);
+             }
+
+             if (!process_mrelease(pidfd, 0)) {
+                     /* Terminate the test once process_mrelease succeeds */
+                     return 0;
+             }
+
+             /*
+              * Ignore the failure if the child exited before mrelease got
+              * called, increase allocation size and retry the test
+              */

Add more info. on why allocating more memory helps.

Ack.

Quoted 51 lines
Quoted 34 lines
tools/testing/selftests/vm/mrelease_test.c new file +15 -0
+             if (errno != ESRCH) {
+                     perror("process_mrelease");
+                     exit(1);
+             }
+
+             if (waitpid(pid, NULL, 0) < 0) {
+                     perror("waitpid");
+                     exit(1);
+             }
+             close(pidfd);
+     }
+
+     printf("All process_mrelease attempts failed!\n");
+     exit(1);
+}
tools/testing/selftests/vm/run_vmtests.sh +9 -0
index 352ba00cf26b..1986162fea39 100755
@@ -287,6 +287,22 @@ else
      echo "[PASS]"
  fi
 
+echo "---------------------"
+echo "running mrelease_test"
+echo "---------------------"
+./mrelease_test
+ret_val=$?
+
+if [ $ret_val -eq 0 ]; then
+     echo "[PASS]"
+elif [ $ret_val -eq $ksft_skip ]; then

tools/testing/selftests/vm/run_vmtests.sh else · line 308

Quoted 10 lines
tools/testing/selftests/vm/run_vmtests.sh +7 -0
+      echo "[SKIP]"
+      exitcode=$ksft_skip
+else
+     echo "[FAIL]"
+     exitcode=1
+fi
+
  echo "-------------------"
  echo "running mremap_test"
  echo "-------------------"

In general, the code flow is hard to read to make sure resources are released e.g: pipefd in all the error paths. The code is broken up into smaller chunks where it isn't needed in some cases and left as a large block when it could benefit from abstraction e.g: negative test block.

Quoted 1 line

Please make changes and send v2.

Will try to refactor it for a better readability. Thanks for the review!

Quoted 3 lines

thanks,
-- Shuah

maintainer
#15 lore raw
Quoted 90 lines · Suren Baghdasaryan
Quoted 13 lines · Shuah Khan
Quoted 10 lines · Suren Baghdasaryan

Introduce process_mrelease syscall sanity tests. They include tests of invalid pidfd and flags inputs, attempting to call process_mrelease with a live process and a valid usage of process_mrelease. Because process_mrelease has to be used against a process with a pending SIGKILL, it's possible that the process exits before process_mrelease gets called. In such cases we retry the test with a victim that allocates twice more memory up to 1GB. This would require the victim process to spend more time during exit and process_mrelease has a better chance of catching the process before it exits.

Quoted 2 lines

+1 on Mike's comments on improving the change log. List what is getting tested as opposed to describing the test code.

I'll try to improve the description but IMHO it does describe what it's testing - the process_mrelease syscall with valid and invalid inputs. I could omit the implementation details if that helps.

Quoted 10 lines
Quoted 8 lines
---
   tools/testing/selftests/vm/Makefile        |   1 +
   tools/testing/selftests/vm/mrelease_test.c | 176 +++++++++++++++++++++
   tools/testing/selftests/vm/run_vmtests.sh  |  16 ++
   3 files changed, 193 insertions(+)

create mode 100644 tools/testing/selftests/vm/mrelease_test.c

Please update .gitignore with the new executable.

Ack.

Quoted 34 lines
Quoted 34 lines
tools/testing/selftests/vm/Makefile +1 -0
index 04a49e876a46..733fccbff0ef 100644
@@ -43,6 +43,7 @@ TEST_GEN_FILES += map_populate
   TEST_GEN_FILES += memfd_secret
   TEST_GEN_FILES += mlock-random-test
   TEST_GEN_FILES += mlock2-tests
+TEST_GEN_FILES += mrelease_test
   TEST_GEN_FILES += mremap_dontunmap
   TEST_GEN_FILES += mremap_test
   TEST_GEN_FILES += on-fault-limit
tools/testing/selftests/vm/mrelease_test.c new file +14 -0
new file mode 100644
index 000000000000..a61061bf8433
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 Google LLC
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#include "util.h"
+
+static inline int pidfd_open(pid_t pid, unsigned int flags)
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 19

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+{
+#ifdef __NR_pidfd_open
+     return syscall(__NR_pidfd_open, pid, flags);
+#else
+     errno = ENOSYS;

This isn't an error - this would be skip because this syscall isn't supported.

Ack.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 21

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+     return -1;
+#endif

Key off of syscall return instead of these ifdefs - same comment on all of the ifdefs

Ack. I was using some other test as an example but I guess that was not a good model.

Quoted 1 line
Quoted 9 lines
Quoted 5 lines

tools/testing/selftests/vm/mrelease_test.c line 23

Quoted 2 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+}
+

I am not seeing any reason for breaking this code up have a separate routine for pidfd_open().

I'm a bit unclear what you mean. Do you mean that userspace headers should already define pidfd_open() and I don't need to define it?

Do you need pidfd_open() or can this be part of main? Without the ifdefs, it is really a one line code.

thanks,
-- Shuah

#16 lore raw
Quoted 98 lines · Shuah Khan
Quoted 96 lines · Suren Baghdasaryan
Quoted 13 lines · Shuah Khan
Quoted 10 lines · Suren Baghdasaryan

Introduce process_mrelease syscall sanity tests. They include tests of invalid pidfd and flags inputs, attempting to call process_mrelease with a live process and a valid usage of process_mrelease. Because process_mrelease has to be used against a process with a pending SIGKILL, it's possible that the process exits before process_mrelease gets called. In such cases we retry the test with a victim that allocates twice more memory up to 1GB. This would require the victim process to spend more time during exit and process_mrelease has a better chance of catching the process before it exits.

Quoted 2 lines

+1 on Mike's comments on improving the change log. List what is getting tested as opposed to describing the test code.

I'll try to improve the description but IMHO it does describe what it's testing - the process_mrelease syscall with valid and invalid inputs. I could omit the implementation details if that helps.

Quoted 10 lines
Quoted 8 lines
---
   tools/testing/selftests/vm/Makefile        |   1 +
   tools/testing/selftests/vm/mrelease_test.c | 176 +++++++++++++++++++++
   tools/testing/selftests/vm/run_vmtests.sh  |  16 ++
   3 files changed, 193 insertions(+)

create mode 100644 tools/testing/selftests/vm/mrelease_test.c

Please update .gitignore with the new executable.

Ack.

Quoted 34 lines
Quoted 34 lines
tools/testing/selftests/vm/Makefile +1 -0
index 04a49e876a46..733fccbff0ef 100644
@@ -43,6 +43,7 @@ TEST_GEN_FILES += map_populate
   TEST_GEN_FILES += memfd_secret
   TEST_GEN_FILES += mlock-random-test
   TEST_GEN_FILES += mlock2-tests
+TEST_GEN_FILES += mrelease_test
   TEST_GEN_FILES += mremap_dontunmap
   TEST_GEN_FILES += mremap_test
   TEST_GEN_FILES += on-fault-limit
tools/testing/selftests/vm/mrelease_test.c new file +14 -0
new file mode 100644
index 000000000000..a61061bf8433
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2022 Google LLC
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#include "util.h"
+
+static inline int pidfd_open(pid_t pid, unsigned int flags)
Quoted 8 lines

tools/testing/selftests/vm/mrelease_test.c line 19

Quoted 5 lines
tools/testing/selftests/vm/mrelease_test.c new file +5 -0
+{
+#ifdef __NR_pidfd_open
+     return syscall(__NR_pidfd_open, pid, flags);
+#else
+     errno = ENOSYS;

This isn't an error - this would be skip because this syscall isn't supported.

Ack.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 21

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+     return -1;
+#endif

Key off of syscall return instead of these ifdefs - same comment on all of the ifdefs

Ack. I was using some other test as an example but I guess that was not a good model.

Quoted 6 lines

tools/testing/selftests/vm/mrelease_test.c line 23

Quoted 3 lines
tools/testing/selftests/vm/mrelease_test.c new file +2 -0
+}
+

I am not seeing any reason for breaking this code up have a separate routine for pidfd_open().

Quoted 6 lines
Quoted 3 lines

I'm a bit unclear what you mean. Do you mean that userspace headers should already define pidfd_open() and I don't need to define it?

Do you need pidfd_open() or can this be part of main? Without the ifdefs, it is really a one line code.

Ah, I see. I think it's cleaner that way but I'll make them one-line inline functions.

Quoted 3 lines

thanks,
-- Shuah