nfs_lib.sh: Add support for extra NFS mount options
PATCH v3 1/2 nfs_lib.sh: Add support for extra NFS mount options
---
Changes since v2: New patch
testcases/network/nfs/nfs_stress/nfs_lib.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
testcases/network/nfs/nfs_stress/nfs_lib.sh +8 -1
@@ -168,6 +168,7 @@ nfs_setup() local local_dir local remote_dir local mount_dir + local nfs_opts if [ "$(stat -f . | grep "Type: nfs")" ]; then tst_brk TCONF "Cannot run nfs-stress test on mounted NFS"
@@ -192,8 +193,14 @@ nfs_setup() remote_dir="$(get_remote_dir $i $type)" nfs_setup_server "$remote_dir" "$(($$ + n))" local_dir="$(get_local_dir $i $n)" + nfs_opts="-o proto=$type,vers=$i" + + if [ ! -z "$NFS_MOUNT_OPTS" ]; then + nfs_opts="$nfs_opts,$NFS_MOUNT_OPTS" + fi + tst_res TINFO "Mounting $local_dir" - nfs_mount "$local_dir" "$remote_dir" "-o proto=$type,vers=$i" + nfs_mount "$local_dir" "$remote_dir" "$nfs_opts" n=$(( n + 1 )) done
Signature · 5 lines
2.47.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v3 2/2 Add test for data integrity over NFS
Add NFS test which checks data integrity of random writes into a file, with both buffered and direct I/O.
---
Changes since v1: Added TST_TIMEOUT
Changes since v2: Force NFS blocksize to the underlying FS blocksize
By default, NFS uses very large blocksize (up to 1MB) to optimize network communication. For the FS integrity test this means a significant slowdown because the size of the test file and I/O operations increase with blocksize. Autodetect the blocksize of the underlying filesystem and force the same value during NFS mount to speed things up.
The TST_TIMEOUT value is calculated as worst-case scenario for 64KB blocksize. On reasonably modern x86_64 machines, the new tests take about 6 minutes to complete in total (with Btrfs, Ext4 and XFS being tested).
runtest/net.nfs | 11 ++++++ testcases/network/nfs/nfs_stress/nfs10.sh | 47 +++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100755 testcases/network/nfs/nfs_stress/nfs10.sh
runtest/net.nfs +11 -0
@@ -94,6 +94,17 @@ nfs09_v40_ip6t nfs09.sh -6 -v 4 -t tcp nfs09_v41_ip6t nfs09.sh -6 -v 4.1 -t tcp nfs09_v42_ip6t nfs09.sh -6 -v 4.2 -t tcp +nfs10_v30_ip4u nfs10.sh -v 3 -t udp +nfs10_v30_ip4t nfs10.sh -v 3 -t tcp +nfs10_v40_ip4t nfs10.sh -v 4 -t tcp +nfs10_v41_ip4t nfs10.sh -v 4.1 -t tcp +nfs10_v42_ip4t nfs10.sh -v 4.2 -t tcp +nfs10_v30_ip6u nfs10.sh -6 -v 3 -t udp +nfs10_v30_ip6t nfs10.sh -6 -v 3 -t tcp +nfs10_v40_ip6t nfs10.sh -6 -v 4 -t tcp +nfs10_v41_ip6t nfs10.sh -6 -v 4.1 -t tcp +nfs10_v42_ip6t nfs10.sh -6 -v 4.2 -t tcp + nfslock01_v30_ip4u nfslock01.sh -v 3 -t udp nfslock01_v30_ip4t nfslock01.sh -v 3 -t tcp nfslock01_v40_ip4t nfslock01.sh -v 4 -t tcp
testcases/network/nfs/nfs_stress/nfs10.sh new file +47 -0
47 lines not on this page · raw message
Signature · 5 lines
2.47.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] Add test for data integrity over NFS
Quoted 16 lines
Add NFS test which checks data integrity of random writes into a file, with both buffered and direct I/O.
Signed-off-by Martin Doucha <[email protected]>---
Changes since v1: Added
TST_TIMEOUTChanges since v2: Force NFS blocksize to the underlying FS blocksize
By default, NFS uses very large blocksize (up to 1MB) to optimize network communication. For the FS integrity test this means a significant slowdown because the size of the test file and I/O operations increase with blocksize. Autodetect the blocksize of the underlying filesystem and force the same value during NFS mount to speed things up.
Quoted 3 lines
The
TST_TIMEOUTvalue is calculated as worst-case scenario for 64KB blocksize. On reasonably modernx86_64machines, the new tests take about 6 minutes to complete in total (with Btrfs, Ext4 and XFS being tested).
+1. I also did some testing. BTW TST_TIMEOUT is for single filesystem, thus 11 min is really safe enough even for slow machines like emulated risc-v.
Quoted 9 lines
+++
b/testcases/network/nfs/nfs_stress/nfs10.shtestcases/network/nfs/nfs_stress/nfs10.sh new file +7
-0@@ -0,0 +1,47 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2024 SUSE LLC <[email protected]> +# +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT + +TST_CNT=4
testcases/network/nfs/nfs_stress/nfs10.sh line 17
Quoted 10 lines
testcases/network/nfs/nfs_stress/nfs10.sh new file +10
-0+TST_SETUP="nfs10_setup" +TST_TESTFUNC="do_test" +TST_DEVICE_SIZE=1024 +TST_TIMEOUT=660 + +nfs10_setup() +{ + local bsize=$(stat -f -c %s .) + + bsize=$(( $bsize < 1024 ? 1024 : $bsize ))
nit: this would work if stat is not installed (minimal systems) or fails for some reason:
if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then
bsize=1024
fi
If you agree, I can change it during merge.
Otherwise LGTM.
Thank you!
Kind regards,
Petr
Quoted 2 lines
testcases/network/nfs/nfs_stress/nfs10.sh new file +2
-0+ NFS_MOUNT_OPTS="rsize=$bsize,wsize=$bsize" + nfs_setup
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] Add test for data integrity over NFS
Hi,
Quoted 10 lines · Petr Vorel
testcases/network/nfs/nfs_stress/nfs10.sh line 17
Quoted 6 lines
testcases/network/nfs/nfs_stress/nfs10.sh new file +6
-0+ +nfs10_setup() +{ + local bsize=$(stat -f -c %s .) + + bsize=$(( $bsize < 1024 ? 1024 : $bsize ))nit: this would work if stat is not installed (minimal systems) or fails for some reason:
Quoted 6 lines
if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then
bsize=1024fi
If you agree, I can change it during merge.
Otherwise LGTM.
Please do, thank you.
Signature · 10 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] Add test for data integrity over NFS
Hi Martin,
Quoted 5 lines
Quoted 4 lines
testcases/network/nfs/nfs_stress/nfs10.sh line 17
Quoted 1 line
testcases/network/nfs/nfs_stress/nfs10.sh new file +1
-0+ bsize=$(( $bsize < 1024 ? 1024 : $bsize ))nit: this would work if stat is not installed (minimal systems) or fails for some reason:
Quoted 8 lines
Quoted 6 lines
if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then
bsize=1024fi
If you agree, I can change it during merge.
Otherwise LGTM.Please do, thank you.
The remaining two patches merged with agreed change. Thanks! I suppose you send a patch to modify vectorize_buffer() fsplough.c.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v3 2/2] Add test for data integrity over NFS
In reply to Martin DouchaHi,
testcases/network/nfs/nfs_stress/nfs10.sh line 10
Quoted 12 lines
+++
b/testcases/network/nfs/nfs_stress/nfs10.shtestcases/network/nfs/nfs_stress/nfs10.sh new file +10
-0@@ -0,0 +1,47 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2024 SUSE LLC <[email protected]> +# +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT + +TST_CNT=4 +TST_SETUP="nfs10_setup" +TST_TESTFUNC="do_test" +TST_DEVICE_SIZE=1024
FYI requiring 1GB free space can break the test (even for UDP which should be mostly skipped with TCONF due UDP being disabled on mainline kernel). On UDP it runs quickly (< 10s), but on TCP it runs for whole timeout (~ 33 min).
This is not a new limitation in shell API, it always existed. But recent commit 8532eb2c61 ("Add support for setting loop device size in shell tests") increased change of hitting the problem. I don't think it should block this patchset, I'll send a patch fixing the API soon.
Kind regards,
Petr
# ./nfs10.sh -v 3 -t udp nfs10 1 TINFO: Running: nfs10.sh -v 3 -t udp
nfs10 1 TINFO: 10.0.0.2/24 -- 10.0.0.1/24
nfs10 1 TINFO: fd00:1:1:1::2/64 -- fd00:1:1:1::1/64
nfs10 1 TINFO: Using /tmp/LTP_nfs10.XI7vLEjIkA as tmpdir (tmpfs filesystem) tst_device.c:299: TWARN: Failed to create test_dev.img: ENOSPC (28)
Usage:tst_device acquire [size [filename]]tst_device release /path/to/devicetst_device clear /path/to/device
nfs10 1 TWARN: Failed to acquire device tst_supported_fs_types.c:169: TINFO: Skipping ext2 as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping ext3 as requested by the test tst_supported_fs_types.c:97: TINFO: Kernel supports ext4 tst_supported_fs_types.c:62: TINFO: mkfs.ext4 does exist tst_supported_fs_types.c:97: TINFO: Kernel supports xfs tst_supported_fs_types.c:62: TINFO: mkfs.xfs does exist tst_supported_fs_types.c:97: TINFO: Kernel supports btrfs tst_supported_fs_types.c:62: TINFO: mkfs.btrfs does exist tst_supported_fs_types.c:97: TINFO: Kernel supports bcachefs tst_supported_fs_types.c:62: TINFO: mkfs.bcachefs does exist tst_supported_fs_types.c:169: TINFO: Skipping vfat as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping exfat as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping ntfs as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping tmpfs as requested by the test nfs10 1 TINFO: === Testing on ext4 ===
tst_device.c:389: TWARN: Failed to clear 512k block on
Usage:tst_device acquire [size [filename]]tst_device release /path/to/devicetst_device clear /path/to/device
...
Quoted 1 line
testcases/network/nfs/nfs_stress/nfs10.sh new file +1
-0+TST_TIMEOUT=660
...
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Martin,
...
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_setup() · line 198
Quoted 3 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +3
-0+ nfs_opts="-o proto=$type,vers=$i" + + if [ ! -z "$NFS_MOUNT_OPTS" ]; then
We have -n for non-null string. Even just [ "$NFS_MOUNT_OPTS" ] would work, but -n is more obvious. I can change it before merge:
if [ -n "$NFS_MOUNT_OPTS" ]; then
Kind regards,
Petr
Quoted 6 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +4
-1+ nfs_opts="$nfs_opts,$NFS_MOUNT_OPTS" + fi + tst_res TINFO "Mounting $local_dir" - nfs_mount "$local_dir" "$remote_dir" "-o proto=$type,vers=$i" + nfs_mount "$local_dir" "$remote_dir" "$nfs_opts"
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi,
Quoted 11 lines · Petr Vorel
Hi Martin,
...
testcases/network/nfs/nfs_stress/nfs_lib.sh nfs_setup() · line 198
Quoted 3 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +3
-0+ nfs_opts="-o proto=$type,vers=$i" + + if [ ! -z "$NFS_MOUNT_OPTS" ]; thenWe have -n for non-null string. Even just [ "$NFS_MOUNT_OPTS" ] would work, but -n is more obvious. I can change it before merge:
if [ -n "$NFS_MOUNT_OPTS" ]; then
Please do, thanks.
Quoted 11 lines
Reviewed-by Petr Vorel <[email protected]>Kind regards,
PetrQuoted 6 lines
testcases/network/nfs/nfs_stress/nfs_lib.sh +4
-1+ nfs_opts="$nfs_opts,$NFS_MOUNT_OPTS" + fi + tst_res TINFO "Mounting $local_dir" - nfs_mount "$local_dir" "$remote_dir" "-o proto=$type,vers=$i" + nfs_mount "$local_dir" "$remote_dir" "$nfs_opts"
Signature · 10 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp