Skip to contentSkip to conversation

fstests: don't oom the box opening tmpfiles (take 2)

PATCH In review Amir Goldstein 2 participants 2 messages Jun 9, 2019 – Jun 11, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fstests: don't oom the box opening tmpfiles (take 2)

For the t_open_tmpfiles tests that run multiple jobs in parallel, limit ourselves to half of file-max for all jobs combined, so that we don't OOM the test machine.

---

Eryu,

Reducing max allowed file by factor of 2 wasn't good enough for the multi jobs variants of the tests. They still OOM mytest machine (with 2GB RAM).

Thanks,
Amir.

 tests/generic/531 | 4 ++--
 tests/xfs/502     | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
tests/generic/531 +2 -2
index 8ce4cc93..ea77eec2 100755
@@ -41,10 +41,10 @@ _scratch_mount
 # Try to load up all the CPUs, two threads per CPU.
 nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 ))
 
-# Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR)
+# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 50000 files per LOAD_FACTOR)
 # so that this test doesn't take forever or OOM the box
 max_files=$((50000 * LOAD_FACTOR))
-max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / 2 ))
+max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / $nr_cpus / 2 ))
 test $max_allowable_files -gt 0 && test $max_files -gt $max_allowable_files && \
 	max_files=$max_allowable_files
 ulimit -n $max_files
tests/xfs/502 +2 -2
index 0a7921a3..1b747a1a 100755
@@ -43,10 +43,10 @@ _scratch_mount
 # Load up all the CPUs, two threads per CPU.
 nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 ))
 
-# Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per cpu per LOAD_FACTOR)
+# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 30000 files per cpu per LOAD_FACTOR)
 # so that this test doesn't take forever or OOM the box
 max_files=$((30000 * LOAD_FACTOR))
-max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / 2 ))
+max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / $nr_cpus / 2 ))
 test $max_allowable_files -gt 0 && test $max_files -gt $max_allowable_files && \
 	max_files=$max_allowable_files
 ulimit -n $max_files
Signature · 1 line
2.17.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 5 lines · Amir Goldstein

For the t_open_tmpfiles tests that run multiple jobs in parallel, limit ourselves to half of file-max for all jobs combined, so that we don't OOM the test machine.

LGTM ...

--D

Quoted 52 lines

---

Eryu,

Reducing max allowed file by factor of 2 wasn't good enough for the multi jobs variants of the tests. They still OOM mytest machine (with 2GB RAM).

Thanks,
Amir.

 tests/generic/531 | 4 ++--
 tests/xfs/502     | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
tests/generic/531 +2 -2
index 8ce4cc93..ea77eec2 100755
@@ -41,10 +41,10 @@ _scratch_mount
 # Try to load up all the CPUs, two threads per CPU.
 nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 ))
 
-# Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR)
+# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 50000 files per LOAD_FACTOR)
 # so that this test doesn't take forever or OOM the box
 max_files=$((50000 * LOAD_FACTOR))
-max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / 2 ))
+max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / $nr_cpus / 2 ))
 test $max_allowable_files -gt 0 && test $max_files -gt $max_allowable_files && \
 	max_files=$max_allowable_files
 ulimit -n $max_files
tests/xfs/502 +2 -2
index 0a7921a3..1b747a1a 100755
@@ -43,10 +43,10 @@ _scratch_mount
 # Load up all the CPUs, two threads per CPU.
 nr_cpus=$(( $(getconf _NPROCESSORS_ONLN) * 2 ))
 
-# Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per cpu per LOAD_FACTOR)
+# Set ULIMIT_NOFILE to min(file-max / $nr_cpus / 2, 30000 files per cpu per LOAD_FACTOR)
 # so that this test doesn't take forever or OOM the box
 max_files=$((30000 * LOAD_FACTOR))
-max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / 2 ))
+max_allowable_files=$(( $(cat /proc/sys/fs/file-max) / $nr_cpus / 2 ))
 test $max_allowable_files -gt 0 && test $max_files -gt $max_allowable_files && \
 	max_files=$max_allowable_files
 ulimit -n $max_files
Signature · 1 line
2.17.1