memcg/functional: Convert to newlib
PATCH 1/1 memcg/functional: Convert to newlib
From: Joerg Vehlow <[email protected]>
This is an intermediate step, to allow further improvements to tests in memcg/functional. The next steps are
- fixing synchronization between shell and memcg_process: At the moment signals may get lost (especially in warmup), because the memcg_process is signaled twice without any time in between. If the process is not fast enough, the second signal can get lost. I am already working on something for a better synchronization.
- Some tests fail always (at least for me). This can be due to the fact, that memory.use_hierarchy cannot be disabled easily on systems using systemd, because it enables it and ads lots of subgroups. (Disableing is only possible, if there are no subgroups).
I added TCONF for some tests, that obviously failed because of that, but I am not sure for the remaining errors
The second reason for failed tests is the fact, that the value in memory.limit_in_bytes is not exact. The kernel documentation does not say how this actually works, but I guess it is baed on per_cpu_counters. The checks will propably require an epsilon, but that needs some investigation in the kernel code.
In addtion to converting the tests to the new library:
- Moved non-shared code from memcg_lib to the respective test files
- Removed unused test_hugepage from memcg_lib
But even in this state, I think it should be merged, because it will simplify review for upcomming changes.
BTW: This was the last shell-user of TST_CHECKPOINT. In theory it could be removed from test.sh, if we want to gradually strip down unused code, until the whole file is gone.
J?rg
--- .../memcg/functional/memcg_failcnt.sh | 79 +-- .../memcg/functional/memcg_force_empty.sh | 64 +- .../controllers/memcg/functional/memcg_lib.sh | 559 ++++++------------ .../memcg/functional/memcg_limit_in_bytes.sh | 106 ++-- .../memcg_max_usage_in_bytes_test.sh | 111 ++-- .../memcg_memsw_limit_in_bytes_test.sh | 85 +-- .../memcg_move_charge_at_immigrate_test.sh | 101 ++-- .../memcg/functional/memcg_stat_rss.sh | 66 +-- .../memcg/functional/memcg_stat_test.sh | 93 ++- .../memcg/functional/memcg_subgroup_charge.sh | 79 +-- .../functional/memcg_usage_in_bytes_test.sh | 66 +-- .../functional/memcg_use_hierarchy_test.sh | 59 +- 12 files changed, 561 insertions(+), 907 deletions(-)
testcases/kernel/controllers/memcg/functional/memcg_failcnt.sh +30 -49
@@ -1,58 +1,39 @@ #!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2009 FUJITSU LIMITED +# Copyright (c) 2018-2019 ARM Ltd. All Rights Reserved. +# Copyright (c) 2021 Joerg Vehlow <[email protected]> +# +# Author: Li Zefan <[email protected]> +# Restructure for LTP: Shi Weihua <[email protected]> +# Added memcg enable/disable functionality: Rishikesh K Rajak <[email protected]> -################################################################################ -## ## -## Copyright (c) 2009 FUJITSU LIMITED ## -## ## -## This program is free software; you can redistribute it and#or modify ## -## it under the terms of the GNU General Public License as published by ## -## the Free Software Foundation; either version 2 of the License, or ## -## (at your option) any later version. ## -## ## -## This program is distributed in the hope that it will be useful, but ## -## WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ## -## or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License ## -## for more details. ## -## ## -## You should have received a copy of the GNU General Public License ## -## along with this program; if not, write to the Free Software Foundation, ## -## Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ## -## ## -## Author: Li Zefan <[email protected]> ## -## Restructure for LTP: Shi Weihua <[email protected]> ## -## Added memcg enable/disable functinality: Rishikesh K Rajak ## -## <[email protected] ## -## ## -################################################################################ - -TCID="memcg_failcnt" -TST_TOTAL=3 - +MEMCG_TESTFUNC=test +MEMCG_SHMMAX=1 +TST_TEST_DATA="--mmap-anon --mmap-file --shm" . memcg_lib.sh -# Test memory.failcnt -testcase_1() -{ - echo $PAGESIZE > memory.limit_in_bytes - malloc_free_memory "--mmap-anon" $(($PAGESIZE*2)) - test_failcnt "memory.failcnt" -} +MEMORY_LIMIT=$PAGESIZE +MEMORY_TO_ALLOCATE=$((MEMORY_LIMIT * 2)) -testcase_2() +test() { - echo $PAGESIZE > memory.limit_in_bytes - malloc_free_memory "--mmap-file" $(($PAGESIZE*2)) - test_failcnt "memory.failcnt" -} + echo $MEMORY_LIMIT > memory.limit_in_bytes -testcase_3() -{ - echo $PAGESIZE > memory.limit_in_bytes - malloc_free_memory "--shm" $(($PAGESIZE*2)) - test_failcnt "memory.failcnt" + start_memcg_process $2 -s ${MEMORY_TO_ALLOCATE} + echo $MEMCG_PROCESS_PID > tasks + + signal_memcg_process ${MEMORY_TO_ALLOCATE} + signal_memcg_process ${MEMORY_TO_ALLOCATE} + + stop_memcg_process + + failcnt=$(cat memory.failcnt) + if [ $failcnt -gt 0 ]; then + tst_res TPASS "memory.failcnt is $failcnt, > 0 as expected" + else + tst_res TFAIL "memory.failcnt is $failcnt, <= 0 expected" + fi } -shmmax_setup -LOCAL_CLEANUP=shmmax_cleanup -run_tests -tst_exit +tst_run
testcases/kernel/controllers/memcg/functional/memcg_force_empty.sh +22 -42
98 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_lib.sh +180 -379
708 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_limit_in_bytes.sh +47 -59
172 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_max_usage_in_bytes_test.sh +54 -57
133 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_memsw_limit_in_bytes_test.sh +25 -60
146 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_move_charge_at_immigrate_test.sh +60 -41
123 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_stat_rss.sh +23 -43
112 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_stat_test.sh +33 -60
156 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_subgroup_charge.sh +45 -34
95 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_usage_in_bytes_test.sh +21 -45
79 lines not on this page · raw message
testcases/kernel/controllers/memcg/functional/memcg_use_hierarchy_test.sh +21 -38
90 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH 1/1 memcg/functional: Convert to newlib
--- a/testcases/kernel/controllers/memcg/functional/memcg_failcnt.sh
testcases/kernel/controllers/memcg/functional/memcg_failcnt.sh line 4
Quoted 6 lines
+++
b/testcases/kernel/controllers/memcg/functional/memcg_failcnt.shtestcases/kernel/controllers/memcg/functional/memcg_failcnt.sh +3
-0@@ -1,58 +1,39 @@ #!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2009 FUJITSU LIMITED +# Copyright (c) 2018-2019 ARM Ltd. All Rights Reserved.
This is bad copy-and-paste for all files, where I mentioned arm, sorry...
J?rg
PATCH 1/1 memcg/functional: Convert to newlib
Hi J?rg,
Quoted 8 lines
---
a/testcases/kernel/controllers/memcg/functional/memcg_failcnt.shtestcases/kernel/controllers/memcg/functional/memcg_failcnt.sh line 4
Quoted 6 lines
+++
b/testcases/kernel/controllers/memcg/functional/memcg_failcnt.shtestcases/kernel/controllers/memcg/functional/memcg_failcnt.sh +3
-0@@ -1,58 +1,39 @@ #!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2009 FUJITSU LIMITED +# Copyright (c) 2018-2019 ARM Ltd. All Rights Reserved.This is bad copy-and-paste for all files, where I mentioned arm, sorry...
That's not a big deal.
On a first look good cleanup, thank you!
I hope these tests aren't planned to be rewritten to C (like NUMA tests). I suppose not, but better to ask earlier.
Kind regards,
Petr
PATCH 1/1 memcg/functional: Convert to newlib
Hello Petr,
Quoted 10 lines · Petr Vorel
Hi J?rg,
Quoted 8 lines
---
a/testcases/kernel/controllers/memcg/functional/memcg_failcnt.shtestcases/kernel/controllers/memcg/functional/memcg_failcnt.sh line 4
Quoted 6 lines
+++
b/testcases/kernel/controllers/memcg/functional/memcg_failcnt.shtestcases/kernel/controllers/memcg/functional/memcg_failcnt.sh +3
-0@@ -1,58 +1,39 @@ #!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2009 FUJITSU LIMITED +# Copyright (c) 2018-2019 ARM Ltd. All Rights Reserved.This is bad copy-and-paste for all files, where I mentioned arm, sorry...
Quoted 6 lines
That's not a big deal.
On a first look good cleanup, thank you!
I hope these tests aren't planned to be rewritten to C (like NUMA tests). I suppose not, but better to ask earlier.
I can look into that as they should use the new (WIP) CGroup C API. For now it is probably best to keep them in Shell.
Quoted 3 lines
Kind regards,
Petr
Signature · 2 lines
Thank you, Richard.
PATCH 1/1 memcg/functional: Convert to newlib
In reply to Petr VorelHi Petr,
Quoted 4 lines
On a first look good cleanup, thank you!
I hope these tests aren't planned to be rewritten to C (like NUMA tests). I suppose not, but better to ask earlier.
can this be merged then?
I'd like to submit some more patches on top of this.
J?rg
PATCH 1/1 memcg/functional: Convert to newlib
Hi Joerg,
Quoted 8 lines
Hi Petr,
Quoted 4 lines
On a first look good cleanup, thank you!
I hope these tests aren't planned to be rewritten to C (like NUMA tests). I suppose not, but better to ask earlier.
can this be merged then?
Good cleanup, thanks! I fixed copyright and merged.
Quoted 1 line
I'd like to submit some more patches on top of this.
Thanks, these tests certainly deserve more work.
Quoted 1 line
J?rg
Kind regards,
Petr
PATCH 1/1 memcg/functional: Convert to newlib
In reply to Joerg VehlowHi Joerg,
Quoted 1 line
I'd like to submit some more patches on top of this.
You probably noticed some failures (on 5.10.14 openSUSE kernel):
memcg_stat_rss, memcg_subgroup_charge, memcg_move_charge_at_immigrate, memcg_stat:
TBROK: timed out on memory.usage_in_bytes
=> maybe just timeout?
memcg_max_usage_in_bytesmemcg_max_usage_in_bytes_test 1 TFAIL: memory.max_usage_in_bytes is 4325376, 4194304 expected memcg_max_usage_in_bytes_test 2 TFAIL: memory.memsw.max_usage_in_bytes is 4325376, 4194304 expected memcg_max_usage_in_bytes_test 3 TFAIL: memory.max_usage_in_bytes is 4325376, 4194304 expected memcg_max_usage_in_bytes_test 3 TFAIL: memory.max_usage_in_bytes is 126976, 0 expected
memcg_usage_in_bytesmemcg_usage_in_bytes_test 1 TFAIL: memory.usage_in_bytes is 4325376, 4194304 expected memcg_usage_in_bytes_test 2 TFAIL: memory.memsw.usage_in_bytes is 4325376, 4194304 expected
Tests which needs further cleanup/rewrite (less important than fixing previous) testcases/kernel/controllers/memcg/regression/memcg_regression_test.sh testcases/kernel/controllers/memcg/regression/memcg_test_4.sh testcases/kernel/controllers/memcg/control/memcg_control_test.sh
Kind regards,
Petr
PATCH 1/1 memcg/functional: Convert to newlib
In reply to Joerg VehlowHi,
FYI there is bug related to these tests:
https://github.com/linux-test-project/ltp/issues/780
Kind regards,
Petr
PATCH 1/1 memcg/functional: Convert to newlib
Hi,
Quoted 4 lines · Petr Vorel
Hi,
FYI there is bug related to these tests:
https://github.com/linux-test-project/ltp/issues/780
Additionally there is
https://github.com/linux-test-project/ltp/issues/698
https://github.com/linux-test-project/ltp/issues/326
https://github.com/linux-test-project/ltp/issues/93
I guess the last one is invalid.
326 and 698 are basically duplicates due to inaccurate memory accounting in cgroups. The same as 780.
I did not fix these failures for now. I haven't checked the kernel sources. There is a hint in the documentation for memory.usage_in_bytes "Of course, when necessary, it's synchronized.".
Maybe it is possible to force synchronization somehow.
I will look into this, after this is merged.
Quoted 3 lines
Kind regards,
Petr