NFS test fixes
cover NFS test fixes
The nfs03.sh test has a race condition which may randomly cause NFS unmount to fail and trigger strange behavior. Add NFS mount/unmount logging and unmount success check for easier debugging. Also fix the race condition in nfs03.sh
Patch 3 should be merged after new LTP release.
Martin Doucha (3):
nfs_lib.sh: Log NFS mount and unmount
nfs03.sh: Wait for all files to be removed
nfs_lib.sh: Fail the test if NFS unmount fails
testcases/network/nfs/nfs_stress/nfs03.sh | 2 ++ testcases/network/nfs/nfs_stress/nfs_lib.sh | 9 +++++++-- 2 files changed, 9 insertions(+), 2 deletions(-)
Signature · 5 lines
2.42.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 1/3 nfs_lib.sh: Log NFS mount and unmount
--- testcases/network/nfs/nfs_stress/nfs_lib.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
testcases/network/nfs/nfs_stress/nfs_lib.sh +7 -2
@@ -191,7 +191,9 @@ nfs_setup() remote_dir="$(get_remote_dir $i $type)" nfs_setup_server "$remote_dir" "$(($$ + n))" - nfs_mount "$(get_local_dir $i $n)" "$remote_dir" "-o proto=$type,vers=$i" + local_dir="$(get_local_dir $i $n)" + tst_res TINFO "Mounting $local_dir" + nfs_mount "$local_dir" "$remote_dir" "-o proto=$type,vers=$i" n=$(( n + 1 )) done
@@ -210,7 +212,10 @@ nfs_cleanup() local n=0 for i in $VERSION; do local_dir="$(get_local_dir $i $n)" - grep -q "$local_dir" /proc/mounts && umount $local_dir + if grep -q "$local_dir" /proc/mounts; then + tst_res TINFO "Unmounting $local_dir" + umount $local_dir + fi n=$(( n + 1 )) done sleep 2
Signature · 5 lines
2.42.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/3] nfs_lib.sh: Log NFS mount and unmount
Hi Martin,
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/3] nfs_lib.sh: Log NFS mount and unmount
In reply to Martin DouchaQuoted 34 lines · Martin Doucha
Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs_lib.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)testcases/network/nfs/nfs_stress/nfs_lib.sh +7
-2@@ -191,7 +191,9 @@ nfs_setup() remote_dir="$(get_remote_dir $i $type)" nfs_setup_server "$remote_dir" "$(($$ + n))" - nfs_mount "$(get_local_dir $i $n)" "$remote_dir" "-o proto=$type,vers=$i" + local_dir="$(get_local_dir $i $n)" + tst_res TINFO "Mounting $local_dir" + nfs_mount "$local_dir" "$remote_dir" "-o proto=$type,vers=$i" n=$(( n + 1 )) done@@ -210,7 +212,10 @@ nfs_cleanup() local n=0 for i in $VERSION; do local_dir="$(get_local_dir $i $n)" - grep -q "$local_dir" /proc/mounts && umount $local_dir + if grep -q "$local_dir" /proc/mounts; then + tst_res TINFO "Unmounting $local_dir" + umount $local_dir + fi n=$(( n + 1 )) done sleep 2Signature · 1 line
2.42.0
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 1/3] nfs_lib.sh: Log NFS mount and unmount
In reply to Martin DouchaHi!
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/3 nfs03.sh: Wait for all files to be removed
In reply to Martin DouchaA race condition between the two parallel calls of rm_files may cause NFS unmount to fail and then the NFS client will get confused by remote filesystem change which leads to strange failures. Make sure both rm_files calls finish before returning from the main test function.
--- testcases/network/nfs/nfs_stress/nfs03.sh | 2 ++ 1 file changed, 2 insertions(+)
testcases/network/nfs/nfs_stress/nfs03.sh +2 -0
@@ -66,11 +66,13 @@ do_test() cd ../dir1 wait $pid1 rm_files & + pid3=$! tst_res TINFO "cd dir2 & removing files" cd ../dir2 wait $pid2 rm_files + wait $pid3 tst_res TPASS "test done" }
Signature · 5 lines
2.42.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
Hi Martin,
Good idea, if the test timeouts we know it's due removing files.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
In reply to Martin DouchaHello,
Quoted 4 lines · Martin Doucha
A race condition between the two parallel calls of
rm_filesmay cause NFS unmount to fail and then the NFS client will get confused by remote filesystem change which leads to strange failures. Make sure bothrm_filescalls finish before returning from the main test function.
You mean unmounting fails if there are still operations in progress?
Like does umount fail with EBUSY?
Quoted 26 lines
Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs03.sh | 2 ++ 1 file changed, 2 insertions(+)testcases/network/nfs/nfs_stress/nfs03.sh +2
-0@@ -66,11 +66,13 @@ do_test() cd ../dir1 wait $pid1 rm_files & + pid3=$! tst_res TINFO "cd dir2 & removing files" cd ../dir2 wait $pid2 rm_files + wait $pid3 tst_res TPASS "test done" }Signature · 1 line
2.42.0
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
Quoted 4 lines · Richard Palethorpe
Hello,
Martin Doucha <[email protected]> writes:
Quoted 8 lines
Quoted 4 lines
A race condition between the two parallel calls of
rm_filesmay cause NFS unmount to fail and then the NFS client will get confused by remote filesystem change which leads to strange failures. Make sure bothrm_filescalls finish before returning from the main test function.You mean unmounting fails if there are still operations in progress?
Like does umount fail with EBUSY?
Yes, the unmount will fail with EBUSY. Then the test filesystem gets deleted while still mounted over NFS and a new NFS mount gets added on top. Then the NFS client randomly gets confused and complains about fsid change.
Quoted 29 lines
Quoted 27 lines
Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs03.sh | 2 ++ 1 file changed, 2 insertions(+)testcases/network/nfs/nfs_stress/nfs03.sh +2
-0@@ -66,11 +66,13 @@ do_test() cd ../dir1 wait $pid1 rm_files & + pid3=$! tst_res TINFO "cd dir2 & removing files" cd ../dir2 wait $pid2 rm_files + wait $pid3 tst_res TPASS "test done" }Signature · 1 line
2.42.0
Signature · 11 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
In reply to Martin DouchaQuoted 30 lines · Martin Doucha
A race condition between the two parallel calls of
rm_filesmay cause NFS unmount to fail and then the NFS client will get confused by remote filesystem change which leads to strange failures. Make sure bothrm_filescalls finish before returning from the main test function.Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs03.sh | 2 ++ 1 file changed, 2 insertions(+)testcases/network/nfs/nfs_stress/nfs03.sh +2
-0@@ -66,11 +66,13 @@ do_test() cd ../dir1 wait $pid1 rm_files & + pid3=$! tst_res TINFO "cd dir2 & removing files" cd ../dir2 wait $pid2 rm_files + wait $pid3 tst_res TPASS "test done" }Signature · 1 line
2.42.0
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
In reply to Martin DouchaHi!
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 2/3] nfs03.sh: Wait for all files to be removed
Hi all,
Quoted 2 lines
Hi!
Reviewed-by Cyril Hrubis <[email protected]>
I merged these two patches.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 3/3 nfs_lib.sh: Fail the test if NFS unmount fails
In reply to Martin Doucha--- testcases/network/nfs/nfs_stress/nfs_lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
testcases/network/nfs/nfs_stress/nfs_lib.sh +1 -1
@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed" fi n=$(( n + 1 )) done
Signature · 5 lines
2.42.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Hi Martin,
Quoted 10 lines
Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs_lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)testcases/network/nfs/nfs_stress/nfs_lib.sh +0
-0
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 5 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"
I'd personally add TWARN (i.e. less critical error). @Cyril WDYT?
Kind regards,
Petr
Quoted 3 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +0
-0fi n=$(( n + 1 )) done
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
In reply to Martin DouchaQuoted 20 lines · Martin Doucha
Signed-off-by Martin Doucha <[email protected]>--- testcases/network/nfs/nfs_stress/nfs_lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed" fi n=$(( n + 1 )) doneSignature · 1 line
2.42.0
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
In reply to Martin DouchaHi!
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 10 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"
I suppose that this should be TBROK instead. And that this, apart from the previous patches, should be applied after the release.
Otherwise:
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Quoted 13 lines
Hi!
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 10 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"I suppose that this should be TBROK instead. And that this, apart from
Right, TBROK looks to be the best.
Martin, if you're ok with the change, I'll update it before merge (after the release).
Kind regards,
Petr
Quoted 5 lines
the previous patches, should be applied after the release.
Otherwise:
Reviewed-by Cyril Hrubis <[email protected]>
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Quoted 14 lines · Petr Vorel
Quoted 13 lines
Hi!
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 10 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1@@ -214,7 +214,7 @@ nfs_cleanup() local_dir="$(get_local_dir $i $n)" if grep -q "$local_dir" /proc/mounts; then tst_res TINFO "Unmounting $local_dir" - umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"I suppose that this should be TBROK instead. And that this, apart from
Quoted 3 lines
Right, TBROK looks to be the best.
Martin, if you're ok with the change, I'll update it before merge (after the release).
I don't see the point. This is a cleanup function. The TBROK will be changed to TWARN and the test will continue anyway.
Signature · 11 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Hi,
...
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 2 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1- umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"I suppose that this should be TBROK instead. And that this, apart from
Right, TBROK looks to be the best.
Martin, if you're ok with the change, I'll update it before merge (after the release).I don't see the point. This is a cleanup function. The TBROK will be changed to TWARN and the test will continue anyway.
But TFAIL in cleanup function looks to me strange as well. Should we endup with TWARN then?
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Quoted 8 lines · Petr Vorel
Hi,
...
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 2 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1- umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"I suppose that this should be TBROK instead. And that this, apart from
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Right, TBROK looks to be the best.
Martin, if you're ok with the change, I'll update it before merge (after the release).I don't see the point. This is a cleanup function. The TBROK will be changed to TWARN and the test will continue anyway.
But TFAIL in cleanup function looks to me strange as well. Should we endup with TWARN then?
TWARN makes sense. Feel free to change it during merge.
Signature · 11 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH 3/3] nfs_lib.sh: Fail the test if NFS unmount fails
Quoted 12 lines · Petr Vorel
Quoted 11 lines
Hi,
...
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_cleanup() · line 217
Quoted 2 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +1
-1- umount $local_dir + umount $local_dir || tst_res TFAIL "Unmount failed"I suppose that this should be TBROK instead. And that this, apart from
Right, TBROK looks to be the best.
Martin, if you're ok with the change, I'll update it before merge (after the release).
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I don't see the point. This is a cleanup function. The TBROK will be changed to TWARN and the test will continue anyway.
But TFAIL in cleanup function looks to me strange as well. Should we endup with TWARN then?
TWARN makes sense. Feel free to change it during merge.
Good, changed and merged. Thanks!
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp