Skip to contentSkip to conversation

shell: enable OOM protection by default

PATCH Superseded Andrea Cervesato 4 participants 5 messages Aug 4

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 shell: enable OOM protection by default

From: Andrea Cervesato <[email protected]>

Add TST_OOM_PROTECTION to activate/deactivate OOM protection in shell tests. When enabled, the shell harness shields itself from the OOM killer and runs the test in a child process, so it survives memory pressure and can still report results (e.g. during memcg stress tests).

---
Under Li's idea, implement a OOM protection mechanism for the shell
tests so we can avoid OOM for memcg stress tests.
---
Changes in v3:
- simplify oom code
- enable OOM protection by default
- Link to v2: https://lore.kernel.org/[email protected]

Changes in v2:
- update shell OOM protection functional test
- Link to v1: https://lore.kernel.org/[email protected]

---
 doc/developers/writing_tests.rst             |  3 +++
 lib/newlib_tests/runtest.sh                  |  1 +
 lib/newlib_tests/shell/tst_oom_protection.sh | 31 ++++++++++++++++++++++++
 testcases/lib/tst_test.sh                    | 35 +++++++++++++++++++++++++++-
 4 files changed, 69 insertions(+), 1 deletion(-)
doc/developers/writing_tests.rst +3 -0
index 4db57898fcf08b83e68be996f666e91c418838fc..2d5bc294083fa2b89212714f0a6c5e5c3f22777a 100644
@@ -549,6 +549,9 @@ LTP C And Shell Test API Comparison
     * - not applicable
       - TST_FS_TYPE
 
+    * - not applicable
+      - TST_OOM_PROTECTION
+
 .. list-table::
     :header-rows: 1
 
lib/newlib_tests/runtest.sh +1 -0
index 71808ef8b8d5545f52d6014bc070145f952915e9..7e2d0a2ac329fc1d825c2fe0b6f7c09d4f2b0064 100755
@@ -44,6 +44,7 @@ shell/tst_check_driver.sh
 shell/tst_check_kconfig0[1-5].sh
 shell/tst_mount_device.sh
 shell/tst_mount_device_tmpfs.sh
+shell/tst_oom_protection.sh
 shell/tst_skip_filesystems.sh
 shell/net/*.sh}"
 
lib/newlib_tests/shell/tst_oom_protection.sh new file +31 -0
new file mode 100755
index 0000000000000000000000000000000000000000..22993511c64845712e4fa9fc942e9869121d7e0b
@@ -0,0 +1,31 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linux Test Project
+
+TST_TESTFUNC=do_test
+
+read_oom_score_adj() {
+	cat "/proc/$1/oom_score_adj" 2>/dev/null
+}
+
+do_test() {
+	local harness_score child_score
+
+	harness_score=$(read_oom_score_adj "$$")
+
+	if [ "$harness_score" = -1000 ]; then
+		tst_res TPASS "shell harness is protected from OOM by default"
+
+		child_score=$(tst_oom_unprotect read_oom_score_adj self)
+		if [ "$child_score" = 0 ]; then
+			tst_res TPASS "unprotected child process has oom_score_adj reset to 0"
+		else
+			tst_res TFAIL "unprotected child process oom_score_adj is $child_score, expected 0"
+		fi
+	else
+		tst_res TCONF "shell harness OOM protection unavailable"
+	fi
+}
+
+. tst_test.sh
+tst_run
testcases/lib/tst_test.sh +34 -1
index b3e7e29bbf7b52de4cb65665751005cb4df156a8..70fa19e4dadfdceb92821a02fbce7cf6686d9c4e 100644
@@ -15,6 +15,7 @@ export TST_CONF=0
 export TST_COUNT=1
 export TST_ITERATIONS=1
 export TST_TMPDIR_RHOST=0
+export TST_OOM_PROTECTION="${TST_OOM_PROTECTION:-1}"
 export TST_LIB_LOADED=1
 
 # see testcases/lib/tst_runas.c
@@ -28,6 +29,34 @@ export TST_USR_GID="${LTP_USR_GID:-65534}"
 trap "tst_brk TBROK 'test interrupted'" INT
 trap "unset _tst_setup_timer_pid; tst_brk TBROK 'test terminated'" TERM
 
+_tst_set_oom_score_adj()
+{
+	local value="$1"
+	local path="/proc/self/oom_score_adj"
+
+	[ -e "$path" ] || return 0
+
+	echo "$value" > "$path" 2>/dev/null || return 0
+}
+
+_tst_enable_oom_protection()
+{
+	_tst_set_oom_score_adj -1000
+}
+
+_tst_disable_oom_protection()
+{
+	_tst_set_oom_score_adj 0
+}
+
+tst_oom_unprotect()
+{
+	_tst_disable_oom_protection
+	if [ $# -gt 0 ]; then
+		"$@"
+	fi
+}
+
 _tst_do_cleanup()
 {
 	if [ -n "$TST_DO_CLEANUP" -a -n "$TST_CLEANUP" -a -z "$LTP_NO_CLEANUP" ]; then
@@ -681,12 +710,16 @@ tst_run()
 	local _tst_pattern='[='\''"} \t\/:`$\;|].*'
 	local ret
 
+	if [ "$TST_OOM_PROTECTION" = 1 ]; then
+		_tst_enable_oom_protection
+	fi
+
 	if [ -n "$TST_TEST_PATH" ]; then
 		for _tst_i in $(grep '^[^#]*\<TST_' "$TST_TEST_PATH" | sed "s/.*TST_//; s/$_tst_pattern//"); do
 			case "$_tst_i" in
 			ALL_FILESYSTEMS|DISABLE_APPARMOR|DISABLE_SELINUX);;
 			SETUP|CLEANUP|TESTFUNC|ID|CNT|MIN_KVER);;
-			OPTS|USAGE|PARSE_ARGS|POS_ARGS);;
+			OPTS|USAGE|PARSE_ARGS|POS_ARGS|OOM_PROTECTION);;
 			NEEDS_ROOT|NEEDS_TMPDIR|TMPDIR|NEEDS_DEVICE|DEVICE);;
 			NEEDS_CMDS|NEEDS_MODULE|MODPATH|DATAROOT);;
 			NEEDS_DRIVERS|FS_TYPE|MNTPOINT|MNT_PARAMS);;

---
base-commit: 51f722d233efa06304b1d178e24a01048cf80f10
change-id: 20260713-shell_oom_protection-6221ab825220

Best regards,

Signature · 5 lines
Andrea Cervesato <[email protected]>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi Li, Petr,

I updated the patch and simplified it a bit. Maybe I missed something, but it looks good enough now. Let me know if you still have some feedbacks. It's somehow a new v1 :-)

--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
In reply to Andrea Cervesato

Hi Andrea,

testcases/lib/tst_test.sh tst_run() · line 715

Quoted 5 lines · Andrea Cervesato

shell: enable OOM protection by default

testcases/lib/tst_test.sh +3 -0
+	if [ "$TST_OOM_PROTECTION" = 1 ]; then
+		_tst_enable_oom_protection
+	fi

Major: Could the test body run in an unprotected child here, as it does in the C harness?

This sets -1000 on the same shell that later executes the test functions, so all workloads forked by existing tests inherit OOM protection. For example, memcg_limit_in_bytes.sh reaches test_proc_kill() and starts memcg_process in the background. Linux 7.2 copies oom_score_adj on fork, and its cgroup documentation says that tasks with -1000 are never killed. That path will therefore report that memcg_process was not killed instead of observing the expected cgroup OOM kill.

The commit message says that the test runs in a child, but this patch does not create that boundary. tst_oom_unprotect() is only called by the new self-test, so it does not reset the existing workload launch sites.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Hi,

Quoted 11 lines

Major: Could the test body run in an unprotected child here, as it does in the C harness?

This sets -1000 on the same shell that later executes the test functions, so all workloads forked by existing tests inherit OOM protection. For example, memcg_limit_in_bytes.sh reaches test_proc_kill() and starts memcg_process in the background. Linux 7.2 copies oom_score_adj on fork, and its cgroup documentation says that tasks with -1000 are never killed. That path will therefore report that memcg_process was not killed instead of observing the expected cgroup OOM kill.

Quoted 3 lines

The commit message says that the test runs in a child, but this patch does not create that boundary. tst_oom_unprotect() is only called by the new self-test, so it does not reset the existing workload launch sites.

the review is correct, I will sent a new version.

--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw
In reply to Andrea Cervesato

Hi Andrea,

Quoted 4 lines

Add TST_OOM_PROTECTION to activate/deactivate OOM protection in shell tests. When enabled, the shell harness shields itself from the OOM killer and runs the test in a child process, so it survives memory pressure and can still report results (e.g. during memcg stress tests).

C API just enables OOM protection unconditionally. Why we allow to disable it? Also, I would not allow to disable it until any test actually needs that.

Quoted 3 lines

---
Under Li's idea, implement a OOM protection mechanism for the shell

Li would deserve his credit via Suggested-by: :). And it's not only for the credit itself, but (maybe more important) when there is something later on others know whom to ask for the details.

Quoted 1 line

tests so we can avoid OOM for memcg stress tests.

Ah, it's for memcg stress tests. Maybe people in the future appreciate to know this, IMHO it should be part of the commit message.

Quoted 21 lines

---
Changes in v3:
- simplify oom code
- enable OOM protection by default
- Link to v2: https://lore.kernel.org/[email protected]

Changes in v2:
- update shell OOM protection functional test
- Link to v1: https://lore.kernel.org/[email protected]

---
 doc/developers/writing_tests.rst             |  3 +++
 lib/newlib_tests/runtest.sh                  |  1 +
 lib/newlib_tests/shell/tst_oom_protection.sh | 31 ++++++++++++++++++++++++
 testcases/lib/tst_test.sh                    | 35 +++++++++++++++++++++++++++-
 4 files changed, 69 insertions(+), 1 deletion(-)
doc/developers/writing_tests.rst +0 -0
index 4db57898fcf08b83e68be996f666e91c418838fc..2d5bc294083fa2b89212714f0a6c5e5c3f22777a 100644

doc/developers/writing_tests.rst LTP C And Shell Test API Comparison · line 553

Quoted 5 lines
doc/developers/writing_tests.rst +2 -0
@@ -549,6 +549,9 @@ LTP C And Shell Test API Comparison
     * - not applicable
       - TST_FS_TYPE
 
+    * - not applicable
+      - TST_OOM_PROTECTION

If we really want to keep the variable, I'd for C part instead of "not applicable" wrote:
_equivalent of OOM protection enabled in C API (tst_enable_oom_protection())_

And, more important, if we add new variable to tst_test.sh, IMHO it should be documented in the only docs we have for it: doc/old/Shell-Test-API.asciidoc.

lib/newlib_tests/runtest.sh shell/tst_check_driver.sh · line 47

Quoted 9 lines
lib/newlib_tests/runtest.sh +1 -0
index 71808ef8b8d5545f52d6014bc070145f952915e9..7e2d0a2ac329fc1d825c2fe0b6f7c09d4f2b0064 100755
@@ -44,6 +44,7 @@ shell/tst_check_driver.sh
 shell/tst_check_kconfig0[1-5].sh
 shell/tst_mount_device.sh
 shell/tst_mount_device_tmpfs.sh
+shell/tst_oom_protection.sh

+1

Quoted 9 lines
lib/newlib_tests/runtest.sh +0 -0
 shell/tst_skip_filesystems.sh
 shell/net/*.sh}"
 
lib/newlib_tests/shell/tst_oom_protection.sh new file +0 -0
new file mode 100755
index 0000000000000000000000000000000000000000..22993511c64845712e4fa9fc942e9869121d7e0b

lib/newlib_tests/shell/tst_oom_protection.sh line 8

Quoted 8 lines
lib/newlib_tests/shell/tst_oom_protection.sh new file +8 -0
@@ -0,0 +1,31 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linux Test Project
+
+TST_TESTFUNC=do_test
+
+read_oom_score_adj() {
+	cat "/proc/$1/oom_score_adj" 2>/dev/null

Why this masking stderr? It should be always OK to read.

Quoted 36 lines
lib/newlib_tests/shell/tst_oom_protection.sh new file +23 -0
+}
+
+do_test() {
+	local harness_score child_score
+
+	harness_score=$(read_oom_score_adj "$$")
+
+	if [ "$harness_score" = -1000 ]; then
+		tst_res TPASS "shell harness is protected from OOM by default"
+
+		child_score=$(tst_oom_unprotect read_oom_score_adj self)
+		if [ "$child_score" = 0 ]; then
+			tst_res TPASS "unprotected child process has oom_score_adj reset to 0"
+		else
+			tst_res TFAIL "unprotected child process oom_score_adj is $child_score, expected 0"
+		fi
+	else
+		tst_res TCONF "shell harness OOM protection unavailable"
+	fi
+}
+
+. tst_test.sh
+tst_run
testcases/lib/tst_test.sh +1 -0
index b3e7e29bbf7b52de4cb65665751005cb4df156a8..70fa19e4dadfdceb92821a02fbce7cf6686d9c4e 100644
@@ -15,6 +15,7 @@ export TST_CONF=0
 export TST_COUNT=1
 export TST_ITERATIONS=1
 export TST_TMPDIR_RHOST=0
+export TST_OOM_PROTECTION="${TST_OOM_PROTECTION:-1}"
 export TST_LIB_LOADED=1
 
 # see testcases/lib/tst_runas.c

testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 37

Quoted 9 lines
testcases/lib/tst_test.sh +6 -0
@@ -28,6 +29,34 @@ export TST_USR_GID="${LTP_USR_GID:-65534}"
 trap "tst_brk TBROK 'test interrupted'" INT
 trap "unset _tst_setup_timer_pid; tst_brk TBROK 'test terminated'" TERM
 
+_tst_set_oom_score_adj()
+{
+	local value="$1"
+	local path="/proc/self/oom_score_adj"
+
+	[ -e "$path" ] || return 0

If you don't reuse the return value you can use just "return"

Also, C API in set_oom_score_adj() warns:
tst_res(TINFO, "oom_score_adj does not exist, skipping the adjustment");

testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 39

Quoted 2 lines
testcases/lib/tst_test.sh +2 -0
+
+	echo "$value" > "$path" 2>/dev/null || return 0

Do we want to hide "permission denied"? C API in set_oom_score_adj() uses capability. If you want to really match the behavior with C API, you could create C binary helper testcases/lib/tst_*.c which would just call set_oom_score_adj(). But I'd be pragmatic, if the tests which needs it have TST_NEEDS_ROOT=1 (and memcg_lib.sh sets it), you might just write shell code which will behave similar like C code (except using capability, of course).

I would not hide "permission denied". At least similarly match C API in set_oom_score_adj():

if ! echo "$value" > "$path"; then
tst_res TWARN "Can't adjust score"
fi

I.e. no masking stderr), return is not used. And even if you want to use function return value, $? will be taken from echo, e.g.:

$ echo foo > /asdf; echo $?
bash: /asdf: Permission denied
1
Quoted 7 lines
testcases/lib/tst_test.sh +7 -0
+}
+
+_tst_enable_oom_protection()
+{
+	_tst_set_oom_score_adj -1000
+}
+

testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 56

Quoted 10 lines
testcases/lib/tst_test.sh +10 -0
+_tst_disable_oom_protection()
+{
+	_tst_set_oom_score_adj 0
+}
+
+tst_oom_unprotect()
+{
+	_tst_disable_oom_protection
+	if [ $# -gt 0 ]; then
+		"$@"

Because there is only single use of tst_oom_unprotect(), I would call "read_oom_score_adj self" here. More readable, more secure ("$@" is kind of eval).

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp