Skip to contentSkip to conversation

cgroup_regression_test: Use unique names for hierarchies in each test

PATCH In review Avinesh Kumar 4 participants 10 messages Jul 10, 2025 – Jul 30, 2025 2 messages not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 cgroup_regression_test: Use unique names for hierarchies in each test

Recently we started having failures where mounting cgroup in quick succession after umount of same named hierarchy fails with EBUSY

cgroup_regression_test 1 TPASS: no kernel bug was found
mount: /tmp/LTP_cgroup_regression_test.ZFwS3JkX4c/cgroup: cgroup already mounted or mount point busy.
dmesg(1) may have more information after failed mount system call.
cgroup_regression_test 2 TFAIL: Failed to mount cgroup filesystem

Updating test to use unique names for cgroup hierarchies in each subtest.

---
 .../controllers/cgroup/cgroup_regression_test.sh       | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +5 -5
index 276231fe8..c1bf4810f 100755
@@ -93,7 +93,7 @@ test1()
 	cgroup_regression_fork_processes &
 	sleep 1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo1 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		kill -TERM $!
@@ -118,7 +118,7 @@ test2()
 	local val1
 	local val2
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo2 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "Failed to mount cgroup filesystem"
 		return
@@ -204,7 +204,7 @@ test4()
 		return
 	fi
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo4 cgroup cgroup/
 	mkdir cgroup/0
 	rmdir cgroup/0
 	tst_umount $PWD/cgroup
@@ -309,7 +309,7 @@ test_7_2()
 {
 	local subsys=$1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo72 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup"
 		return
@@ -370,7 +370,7 @@ test7()
 #---------------------------------------------------------------------------
 test8()
 {
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo8 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		return
Signature · 5 lines
2.43.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

I forgot to mention that I also tried to use
tst_umount() helper routine, but that did not help as umount does not really report any failures here.

Quoted 67 lines · CEST Avinesh Kumar

Recently we started having failures where mounting cgroup in quick succession after umount of same named hierarchy fails with EBUSY

cgroup_regression_test 1 TPASS: no kernel bug was found
mount: /tmp/LTP_cgroup_regression_test.ZFwS3JkX4c/cgroup: cgroup already mounted or mount point busy.
dmesg(1) may have more information after failed mount system call.
cgroup_regression_test 2 TFAIL: Failed to mount cgroup filesystem

Updating test to use unique names for cgroup hierarchies in each subtest.

---
 .../controllers/cgroup/cgroup_regression_test.sh       | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +5 -5
index 276231fe8..c1bf4810f 100755
@@ -93,7 +93,7 @@ test1()
 	cgroup_regression_fork_processes &
 	sleep 1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo1 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		kill -TERM $!
@@ -118,7 +118,7 @@ test2()
 	local val1
 	local val2
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo2 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "Failed to mount cgroup filesystem"
 		return
@@ -204,7 +204,7 @@ test4()
 		return
 	fi
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo4 cgroup cgroup/
 	mkdir cgroup/0
 	rmdir cgroup/0
 	tst_umount $PWD/cgroup
@@ -309,7 +309,7 @@ test_7_2()
 {
 	local subsys=$1
 
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo72 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup"
 		return
@@ -370,7 +370,7 @@ test7()
 #---------------------------------------------------------------------------
 test8()
 {
-	mount -t cgroup -o none,name=foo cgroup cgroup/
+	mount -t cgroup -o none,name=foo8 cgroup cgroup/
 	if [ $? -ne 0 ]; then
 		tst_res TFAIL "failed to mount cgroup filesystem"
 		return
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
Quoted 3 lines · Avinesh Kumar

I forgot to mention that I also tried to use
tst_umount() helper routine, but that did not help as umount does not really report any failures here.

The solution doesn't really convince me. If it's true that a util-linux upgrade to v2.41 is causing this issue, we should probably research why this is happening in there and (eventually) fix it. Or, if it's not a library bug, we should fix tst_umount() in order to recognize when a cgroups folder hasn't been fully unmounted.

@Cyril WDYT ?

- Andrea

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Hi!

Quoted 11 lines
Quoted 3 lines

I forgot to mention that I also tried to use
tst_umount() helper routine, but that did not help as umount does not really report any failures here.

The solution doesn't really convince me. If it's true that a util-linux upgrade to v2.41 is causing this issue, we should probably research why this is happening in there and (eventually) fix it. Or, if it's not a library bug, we should fix tst_umount() in order to recognize when a cgroups folder hasn't been fully unmounted.

@Cyril WDYT ?

This indeed looks like a workaround for a bug either in the test or in the system.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw
Quoted 10 lines · Cyril Hrubis

Hi!

Quoted 9 lines
Quoted 3 lines

I forgot to mention that I also tried to use
tst_umount() helper routine, but that did not help as umount does not really report any failures here.

The solution doesn't really convince me. If it's true that a util-linux upgrade to v2.41 is causing this issue, we should probably research why this is happening in there and (eventually) fix it. Or, if it's not a library bug, we should fix tst_umount() in order to recognize when a cgroups folder hasn't been fully unmounted.

Quoted 4 lines
Quoted 1 line

@Cyril WDYT ?

This indeed looks like a workaround for a bug either in the test or in the system.

Perhaps, by placing sync before every new test which is going to mount a new cgroup folder we fix the issue as well. I'm wondering if we should dig into it a big deeper...

- Andrea

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp

A message that is not in the archive (sent off-list or not yet received): mqkxe24qlrcpb5jxgigbiarxxgemqjw3rmgnsprn55ubghrrda@etaozcz2nemy

#6 lore raw
Quoted 14 lines · Michal Koutný
Quoted 1 line · Andrea Cervesato

On 7/10/25 11:00 AM, Cyril Hrubis wrote:

...

Quoted 5 lines
Quoted 3 lines

This indeed looks like a workaround for a bug either in the test or in the system.

Perhaps, by placing sync before every new test which is going to mount a new cgroup folder we fix the issue as well.

That may only a workaround too. My bet is you'd rather need some RCU barriers or workqueue flushing, thus the simple separation by name. The cleanup path is just so asynchronous (the code had already attempted some nastiness [1], however, I'm suspecting its efficiency may be reduced under layers of VFS reworks since then).

Quoted 5 lines
Quoted 1 line

I'm wondering if we should dig into it a big deeper...

It may be worth checking that there's a change in util-linux that has (presumably) affected timings.
Note that unmounting a cgroup hierarchy is operation that typically happens once during "lifetime" (if at all) -- so it's your call.

I see, thanks for the suggestion. I will take a look.

Quoted 5 lines

HTH,
Michal

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=970317aa48c6ef66cd023c039c2650c897bad927

This git commit is part of cgroup v1 code and now in kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same mechanism used by cgroup v2, which is our main version. Is that something which might affect our system?

- Andrea

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp

A message that is not in the archive (sent off-list or not yet received): svs6pl6jmibp7uylvofkia5smvu2cuezfvnawlgkmjiulbrpfm@w7s7dlxxltap

#8 lore raw
Quoted 4 lines · Michal Koutný
Quoted 3 lines · Andrea Cervesato

This git commit is part of cgroup v1 code and now in kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same mechanism used by cgroup v2, which is our main version.

Quoted 8 lines

The patched tests use cgroup v1.

Quoted 1 line

Is that something which might affect our system?

Since there's only one cgroup v2 (unified) hierarchy, it sort of lives forever (statically alloc'd), so I wouldn't expect such residuals on v2 (with the current implementation).

Michal

I bisected util-linux, but it doesn't seem to be the problem. On my system (Tumbleweed), we are facing the same failure, no matter the commit that is tested in util-linux between 2.39 and 2.41 versions.

I guess this issue might be caused by a specific setup of the Linux kernel and its async nature, mixed with some services which are slowing down the umount process.

I'm wondering how to proceed, since this is actually a soft-failure and any fix to the test might hide future bugs.

- Andrea

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#9 lore raw

Hi all¸

I would like to ask if there are any other suggestions on how to address this issue.
I tried a rather simpler approach (as suggested by Martin) of just adding a delay using sleep after the umount call, which also seems to solve this issue of incomplete unmounting.

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -0
@@ -102,10 +102,11 @@ test1()
        cat cgroup/tasks > /dev/null
 
        kill -TERM $!
        wait $! 2>/dev/null
        umount cgroup/
+       sleep 2
        check_kernel_bug
 }
 

I will send this in the patch if we agree to use this approach instead of unique names.

Thanks,
Avinesh

Quoted 28 lines · CEST Andrea Cervesato
Quoted 4 lines · Michal Koutný
Quoted 3 lines · Andrea Cervesato

This git commit is part of cgroup v1 code and now in kernel/cgroup/cgroup-v1.c:cgroup1_root_to_use, but I don't see the same mechanism used by cgroup v2, which is our main version.

Quoted 8 lines

The patched tests use cgroup v1.

Quoted 1 line

Is that something which might affect our system?

Since there's only one cgroup v2 (unified) hierarchy, it sort of lives forever (statically alloc'd), so I wouldn't expect such residuals on v2 (with the current implementation).

Michal

I bisected util-linux, but it doesn't seem to be the problem. On my system (Tumbleweed), we are facing the same failure, no matter the commit that is tested in util-linux between 2.39 and 2.41 versions.

I guess this issue might be caused by a specific setup of the Linux kernel and its async nature, mixed with some services which are slowing down the umount process.

I'm wondering how to proceed, since this is actually a soft-failure and any fix to the test might hide future bugs.

- Andrea

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#10 lore raw
Quoted 20 lines · Avinesh Kumar

Hi all¸

I would like to ask if there are any other suggestions on how to address this issue.
I tried a rather simpler approach (as suggested by Martin) of just adding a delay using sleep after the umount call, which also seems to solve this issue of incomplete unmounting.

testcases/kernel/controllers/cgroup/cgroup_regression_test.sh +1 -0
@@ -102,10 +102,11 @@ test1()
         cat cgroup/tasks > /dev/null
 
         kill -TERM $!
         wait $! 2>/dev/null
         umount cgroup/
+       sleep 2
         check_kernel_bug
  }
 
Quoted 2 lines

I will send this in the patch if we agree to use this approach instead of unique names.

Well, this looks like the unmount happens asynchronously and the command exits before it's actually finished. In that case, sync would be the best solution. But I think we need some feedback from kernel developers whether the asynchronous behavior is intentional or not. Should the umount command wait for the task to finish?

Signature · 10 lines
Martin Doucha   [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#11 lore raw
In reply to Andrea Cervesato via ltp
Quoted 2 lines · Andrea Cervesato

I'm wondering how to proceed, since this is actually a soft-failure and any fix to the test might hide future bugs.

You can take the (unique names) fix from Avinesh and add a dedicated test for the unmounting (and solve (soft)failure only in that).

0.02€,
Michal

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp