shell: enable OOM protection by default
PATCH v4 1/1 shell: enable OOM protection by default
From: Andrea Cervesato <[email protected]>
The shell harness shields itself from the OOM killer and runs the test in a child process, so it survives memory pressure and can still report results (e.g. during memcg stress tests).
---
Under Li's idea, implement a OOM protection mechanism for the shell
tests so we can avoid OOM for memcg stress tests.
---
Changes in v4:
- fix with Petr suggestions
- fix with AI agent suggestions
- Link to v3: https://lore.kernel.org/[email protected]
Changes in v3:
- simplify oom code
- enable OOM protection by default
- Link to v2: https://lore.kernel.org/[email protected]
Changes in v2:
- update shell OOM protection functional test
- Link to v1: https://lore.kernel.org/[email protected]
To: Linux Test Project <[email protected]>
--- lib/newlib_tests/runtest.sh | 1 + lib/newlib_tests/shell/tst_oom_protection.sh | 34 +++++++++++ testcases/lib/tst_test.sh | 84 ++++++++++++++++++++++++++-- 3 files changed, 115 insertions(+), 4 deletions(-)
lib/newlib_tests/runtest.sh +1 -0
@@ -44,6 +44,7 @@ shell/tst_check_driver.sh shell/tst_check_kconfig0[1-5].sh shell/tst_mount_device.sh shell/tst_mount_device_tmpfs.sh +shell/tst_oom_protection.sh shell/tst_skip_filesystems.sh shell/net/*.sh}"
lib/newlib_tests/shell/tst_oom_protection.sh new file +34 -0
@@ -0,0 +1,34 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2026 Linux Test Project + +TST_TESTFUNC=do_test + +read_oom_score_adj() { + cat "/proc/$1/oom_score_adj" +} + +do_test() { + local harness_score body_score + + # $$ points to the protected harness, while /proc/self is the + # unprotected child that actually runs the test body. + harness_score=$(read_oom_score_adj "$$") + body_score=$(read_oom_score_adj self) + + if [ "$harness_score" != -1000 ]; then + tst_res TCONF "shell harness OOM protection unavailable" + return + fi + + tst_res TPASS "shell harness is protected from OOM by default" + + if [ "$body_score" = 0 ]; then + tst_res TPASS "test body runs in an unprotected child (oom_score_adj=0)" + else + tst_res TFAIL "test body oom_score_adj is $body_score, expected 0" + fi +} + +. tst_test.sh +tst_run
testcases/lib/tst_test.sh +80 -4
108 lines not on this page · raw message
---
base-commit: ee945bb71ba7015dace7d983cf98a530b3eb53a4
change-id: 20260713-shell_oom_protection-6221ab825220
Best regards,
Signature · 5 lines
Andrea Cervesato <[email protected]> -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea,
testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 43
Quoted 5 lines · Andrea Cervesato
shell: enable OOM protection by default
testcases/lib/tst_test.sh +3
-0+ if ! echo "$value" > "$path"; then + tst_res TWARN "Can't adjust score" + fi
Could this avoid reporting TWARN when CAP_SYS_RESOURCE is absent, as tst_memutils.c does? Lowering oom_score_adj requires that capability, so every unprivileged shell test now gets a warning result. The new self-test itself reports TCONF but exits 36 because this TWARN remains counted.
Quoted 4 lines
testcases/lib/tst_test.sh +4
-0+ ( + TST_CHILD=1 + _tst_disable_oom_protection + trap '_tst_cleanup_timer; _tst_write_results' EXIT
testcases/lib/tst_test.sh tst_run() · line 863
Quoted 10 lines
testcases/lib/tst_test.sh +10
-0+ + if [ "$TST_ALL_FILESYSTEMS" = 1 ]; then + _tst_run_tcases_per_fs + else + _tst_run_iterations + fi + ) & + + wait $! + ret=$?
How can the protected parent perform cleanup if this child is OOM-killed or times out? TST_DO_CLEANUP and TST_MOUNT_FLAG are set only in the subshell. On an abnormal child exit, the parent therefore reports TBROK without calling TST_CLEANUP or unmounting a device mounted by _prepare_device(), leaving external test state behind.
Verdict - Needs revision
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea,
Quoted 5 lines
Hi Andrea,
Quoted 1 line · Andrea Cervesato
shell: enable OOM protection by default
Quoted 8 lines
testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 43
Quoted 3 lines
testcases/lib/tst_test.sh +3
-0+ if ! echo "$value" > "$path"; then + tst_res TWARN "Can't adjust score" + fiCould this avoid reporting TWARN when
CAP_SYS_RESOURCEis absent, astst_memutils.cdoes? Loweringoom_score_adjrequires that capability, so every unprivileged shell test now gets a warning result. The new self-test itself reports TCONF but exits 36 because this TWARN remains counted.
Ah, this really needs to be guarded behind if [ "$(id -ru)" != 0 ]; then
If you're in a mood for it, you could factor out [ "$(id -ru)" != 0 ] check into it's own function tst_check_root or tst_is_root which would be then used in tst_require_root and in this function.
I repeat my suggestion from last revision: add a helper into testcases/lib which will just call C function. That way the behavior will be the same (use capabilities). That would be more clearer solution.
Kind regards,
Petr
Quoted 12 lines
Quoted 12 lines
testcases/lib/tst_test.sh +12
-0+ ( + TST_CHILD=1 + _tst_disable_oom_protection + trap '_tst_cleanup_timer; _tst_write_results' EXIT + + if [ "$TST_ALL_FILESYSTEMS" = 1 ]; then + _tst_run_tcases_per_fs + else + _tst_run_iterations + fi + ) & +
Quoted 8 lines
testcases/lib/tst_test.sh tst_run() · line 863
Quoted 2 lines
testcases/lib/tst_test.sh +2
-0+ wait $! + ret=$?How can the protected parent perform cleanup if this child is OOM-killed or times out?
TST_DO_CLEANUPandTST_MOUNT_FLAGare set only in the subshell. On an abnormal child exit, the parent therefore reports TBROK without callingTST_CLEANUPor unmounting a device mounted by_prepare_device(), leaving external test state behind.
Sounds serious :). I'll comment on the patch itself.
Kind regards,
Petr
Quoted 11 lines
Verdict - Needs revision
---
Note:The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea,
...
Quoted 66 lines
lib/newlib_tests/shell/tst_oom_protection.sh new file +34
-0@@ -0,0 +1,34 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2026 Linux Test Project + +TST_TESTFUNC=do_test + +read_oom_score_adj() { + cat "/proc/$1/oom_score_adj" +} + +do_test() { + local harness_score body_score + + # $$ points to the protected harness, while /proc/self is the + # unprotected child that actually runs the test body. + harness_score=$(read_oom_score_adj "$$") + body_score=$(read_oom_score_adj self) + + if [ "$harness_score" != -1000 ]; then + tst_res TCONF "shell harness OOM protection unavailable" + return + fi + + tst_res TPASS "shell harness is protected from OOM by default" + + if [ "$body_score" = 0 ]; then + tst_res TPASS "test body runs in an unprotected child (oom_score_adj=0)" + else + tst_res TFAIL "test body oom_score_adj is $body_score, expected 0" + fi +} + +. tst_test.sh +tst_runtestcases/lib/tst_test.sh +18
-0@@ -28,6 +28,48 @@ export TST_USR_GID="${LTP_USR_GID:-65534}" trap "tst_brk TBROK 'test interrupted'" INT trap "unset _tst_setup_timer_pid; tst_brk TBROK 'test terminated'" TERM +_tst_set_oom_score_adj() +{ + local value="$1" + local path="/proc/self/oom_score_adj" + + if [ ! -e "$path" ]; then + tst_res TINFO "oom_score_adj does not exist, skipping the adjustment" + return + fi + + if ! echo "$value" > "$path"; then + tst_res TWARN "Can't adjust score" + fi +} + +_tst_enable_oom_protection() +{ + _tst_set_oom_score_adj -1000
testcases/lib/tst_test.sh export TST_USR_GID="${LTP_USR_GID:-65534}" · line 58
Quoted 10 lines
testcases/lib/tst_test.sh +10
-0+} + +_tst_disable_oom_protection() +{ + _tst_set_oom_score_adj 0 +} + +_tst_write_results() +{ + [ "$TST_CHILD" = 1 ] || return 0
Yes, $TST_CHILD is here visible, because it's run from the child...
Quoted 18 lines
testcases/lib/tst_test.sh +14
-0+ [ -n "$TST_RESULTS_FILE" ] || return 0 + + echo "$TST_PASS $TST_FAIL $TST_BROK $TST_WARN $TST_CONF $TST_COUNT" \ + > "$TST_RESULTS_FILE" +} + +_tst_read_results() +{ + [ -s "$TST_RESULTS_FILE" ] || return 0 + + read TST_PASS TST_FAIL TST_BROK TST_WARN TST_CONF TST_COUNT \ + < "$TST_RESULTS_FILE" +} + _tst_do_cleanup() { if [ -n "$TST_DO_CLEANUP" -a -n "$TST_CLEANUP" -a -z "$LTP_NO_CLEANUP" ]; then
testcases/lib/tst_test.sh _tst_do_exit() · line 96
Quoted 7 lines
testcases/lib/tst_test.sh +4
-0@@ -48,6 +90,15 @@ _tst_do_exit() _tst_do_cleanup + # When running as the unprotected test child, only propagate the + # results back to the protected harness which does the teardown and + # prints the summary. + if [ "$TST_CHILD" = 1 ]; then
... but as agent noted $TST_CHILD is not visible here in _tst_do_exit() because that's run from parent shell => nack.
Quoted 13 lines
testcases/lib/tst_test.sh +5
-1+ _tst_cleanup_timer + _tst_write_results + exit 0 + fi + cd "$LTPROOT" [ "$TST_MOUNT_FLAG" = 1 ] && tst_umount@@ -788,10 +839,34 @@ tst_run() TST_MNTPOINT="${TST_MNTPOINT:-$PWD/mntpoint}" - if [ "$TST_ALL_FILESYSTEMS" = 1 ]; then
testcases/lib/tst_test.sh tst_run() · line 848
Quoted 10 lines
testcases/lib/tst_test.sh +7
-3- _tst_run_tcases_per_fs - else - _tst_run_iterations + # Protect the harness so it survives memory pressure and can + # still report results, then run the test body in a child that + # resets OOM protection. This way workloads forked by the test + # remain killable, mirroring the C test harness. + _tst_enable_oom_protection + + TST_RESULTS_FILE=$(mktemp "${TMPDIR:-/tmp}/ltp_${TST_ID}_res.XXXXXX")
Hm, writing temporary file directly to TMPDIR is not optimal, but if that was the only problem I'd be ok with it.
testcases/lib/tst_test.sh tst_run() · line 851
Quoted 2 lines
testcases/lib/tst_test.sh +2
-0+ ( + TST_CHILD=1
Agent is correct, this is not visible in the parent shell => I doubt it will work.
Also this is supposed to fix memcg stress tests, but it touches all shell test => very effective way to broke many tests in single commit with great change to get it quickly reverted :).
IMHO we should really rewrite the tests which does not work in tst_test.sh API into C API (better) or to the shell loader. Further complicate tst_test.sh is a way to hell.
Kind regards,
Petr
Quoted 19 lines
testcases/lib/tst_test.sh +18
-0+ _tst_disable_oom_protection + trap '_tst_cleanup_timer; _tst_write_results' EXIT + + if [ "$TST_ALL_FILESYSTEMS" = 1 ]; then + _tst_run_tcases_per_fs + else + _tst_run_iterations + fi + ) & + + wait $! + ret=$? + + _tst_read_results + rm -f "$TST_RESULTS_FILE" + + if [ $ret -gt 128 ]; then + tst_res TBROK "Test killed by signal $((ret - 128))!" fi
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea, Petr,
After looking into it more, now I slightly think doing OOM protection in the shell harness isn't worth it: the benefit doesn't justify the complexity. Forking the whole test body means dealing with cleanup and timer ownership across the fork, plus passing results back from the child, and all of that would need revalidating for every shell test that goes through tst_test.sh.
Given that only a few tests actually produce memory pressure, I'd rather keep it simple and handle it locally in those tests instead of touching the common harness. Something like a small helper the test can wrap the memory hungry workload with:
tst_oom_run()
{
( echo 0 > /proc/self/oom_score_adj; exec "$@" )
}The harness (or the setup) protects itself with oom_score_adj -1000, and only the workload started via tst_oom_run stays killable. This keeps the harness alive to report results without forking the whole test body, and matches the C harness model where the worker is the unprotected part.
Does this direction make sense to you, or do you see a case that really needs the protection to be harness-wide?
Quoted 1 line
Suggested-by Li Wang <[email protected]>
This email address is no longer in use :).
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Li, Andrea,
Quoted 8 lines
Hi Andrea, Petr,
After looking into it more, now I slightly think doing OOM protection in the shell harness isn't worth it: the benefit doesn't justify the complexity. Forking the whole test body means dealing with cleanup and timer ownership across the fork, plus passing results back from the child, and all of that would need revalidating for every shell test that goes through
tst_test.sh.
+1
Quoted 9 lines
Given that only a few tests actually produce memory pressure, I'd rather keep it simple and handle it locally in those tests instead of touching the common harness. Something like a small helper the test can wrap the memory hungry workload with:
tst_oom_run() { ( echo 0 > /proc/self/oom_score_adj; exec "$@" ) }
+1
Maybe test should declare through TST_* variable that it needs OOM killer (approach on Andrea's v2 [1]). I'm sorry Andrea, it takes time to find a right solution.
[1] https://lore.kernel.org/ltp/[email protected]/
And again, IMHO the real solution would be to rewrite tests into C API or shell loader API.
Quoted 5 lines
The harness (or the setup) protects itself with
oom_score_adj-1000, and only the workload started viatst_oom_runstays killable. This keeps the harness alive to report results without forking the whole test body, and matches the C harness model where the worker is the unprotected part.
Yes, forking (the subshell) part looked to me fragile.
Quoted 6 lines
Does this direction make sense to you, or do you see a case that really needs the protection to be harness-wide?
Quoted 1 line
Suggested-by Li Wang <[email protected]>This email address is no longer in use :).
+1
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Li,
Quoted 19 lines
Hi Andrea, Petr,
After looking into it more, now I slightly think doing OOM protection in the shell harness isn't worth it: the benefit doesn't justify the complexity. Forking the whole test body means dealing with cleanup and timer ownership across the fork, plus passing results back from the child, and all of that would need revalidating for every shell test that goes through
tst_test.sh.Given that only a few tests actually produce memory pressure, I'd rather keep it simple and handle it locally in those tests instead of touching the common harness. Something like a small helper the test can wrap the memory hungry workload with:
tst_oom_run() { ( echo 0 > /proc/self/oom_score_adj; exec "$@" ) }
Quoted 8 lines
The harness (or the setup) protects itself with
oom_score_adj-1000, and only the workload started viatst_oom_runstays killable. This keeps the harness alive to report results without forking the whole test body, and matches the C harness model where the worker is the unprotected part.Does this direction make sense to you, or do you see a case that really needs the protection to be harness-wide?
I think you are right, perhaps these shell tests need to be rewritten in C anyway. Most of them are unstable and fificult to maintain (like all shell scripts anyway).
I can go back to my original idea of setting oom protection only for specific tests.
Quoted 4 lines
Quoted 2 lines
Suggested-by Li Wang <[email protected]>This email address is no longer in use :).
Ah damn.. :-D
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp