nfs: test files written size as expected
Subsystems (2)
PATCH 1/1 nfs: test files written size as expected
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10M
As this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorro
tests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.out
tests/nfs/002 new file +43 -0
@@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes") +# +. ./common/preamble +_begin_fstest auto rw + +# real QA test starts here +_supported_fs nfs +_require_test + +localfile=$TEST_DIR/testfile.$seq +rm -rf $localfile + +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1 +block_size=`stat -c '%B' $localfile` +iblocks_expected=$((10 * 1024 * 1024 / $block_size)) +# Try several times for the speculative allocated file size can be shrunken +res=1 +for ((i=0; i<10; i++));do + iblocks_real=`stat -c '%b' $localfile` + if [ "$iblocks_expected" = "$iblocks_real" ];then + res=0 + break + fi + sleep 10 +done +if [ $res -ne 0 ];then + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks" +fi + +echo "Silence is golden" +# success, all done +status=0 +exit
tests/nfs/002.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 002 +Silence is golden
Signature · 1 line
2.31.1
Looks good to me.
Ccing linux-nfs@ for reviewing.
Quoted 87 lines · Zorro Lang
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
Signed-off-by Zorro Lang <[email protected]>---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10MAs this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorrotests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.outtests/nfs/002 new file +43
-0@@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes") +# +. ./common/preamble +_begin_fstest auto rw + +# real QA test starts here +_supported_fs nfs +_require_test + +localfile=$TEST_DIR/testfile.$seq +rm -rf $localfile + +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1 +block_size=`stat -c '%B' $localfile` +iblocks_expected=$((10 * 1024 * 1024 / $block_size)) +# Try several times for the speculative allocated file size can be shrunken +res=1 +for ((i=0; i<10; i++));do + iblocks_real=`stat -c '%b' $localfile` + if [ "$iblocks_expected" = "$iblocks_real" ];then + res=0 + break + fi + sleep 10 +done +if [ $res -ne 0 ];then + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks" +fi + +echo "Silence is golden" +# success, all done +status=0 +exittests/nfs/002.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 002 +Silence is golden --2.31.1
Quoted 33 lines · Zorro Lang
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
Signed-off-by Zorro Lang <[email protected]>---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10MAs this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorrotests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.outtests/nfs/002 new file +1
-0@@ -0,0 +1,43 @@ +#! /bin/bash
tests/nfs/002 line 11
Quoted 10 lines
tests/nfs/002 new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes")
_fixed_by_kernel_commit?
Signature · 1 line
- Andrey
Quoted 57 lines · Zorro Lang
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
Signed-off-by Zorro Lang <[email protected]>---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10MAs this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorrotests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.outtests/nfs/002 new file +25
-0@@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes") +# +. ./common/preamble +_begin_fstest auto rw + +# real QA test starts here +_supported_fs nfs +_require_test + +localfile=$TEST_DIR/testfile.$seq +rm -rf $localfile + +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1 +block_size=`stat -c '%B' $localfile` +iblocks_expected=$((10 * 1024 * 1024 / $block_size))
tests/nfs/002 line 35
Quoted 10 lines
tests/nfs/002 new file +10
-0+# Try several times for the speculative allocated file size can be shrunken +res=1 +for ((i=0; i<10; i++));do + iblocks_real=`stat -c '%b' $localfile` + if [ "$iblocks_expected" = "$iblocks_real" ];then + res=0 + break + fi + sleep 10 +done
Hmm... this case sometimes fails on kernel 6.1.0-rc6 [1] (nfs4.2 base on xfs), even I changed the sleep time to 20s * 10, it still fails. But I can't reproduce this failure if the underlying fs is ext4... cc linux-xfs, to check if I miss something for xfs? Or this's a xfs issue?
Thanks,
Zorro
[1]
# ./check nfs/002 FSTYP -- nfs PLATFORM -- Linux/x86_64 dell-per640-04 6.1.0-rc6 #1 SMP PREEMPT_DYNAMIC Mon Nov 21 00:51:20 EST 2022 MKFS_OPTIONS -- dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server MOUNT_OPTIONS -- -o vers=4.2 -o context=system_u:object_r:root_t:s0 dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server /mnt/xfstests/scratch/nfs-client
nfs/002 3s ... - output mismatch (see /var/lib/xfstests/results//nfs/002.out.bad)
--- tests/nfs/002.out 2022-11-21 01:29:33.861770474 -0500
+++ /var/lib/xfstests/results//nfs/002.out.bad 2022-11-21 13:27:37.424199056 -0500
@@ -1,2 +1,3 @@
QA output created by 002
+Write 20480 blocks, but get 32640 blocks
Silence is golden
...
(Run 'diff -u /var/lib/xfstests/tests/nfs/002.out /var/lib/xfstests/results//nfs/002.out.bad' to see the entire diff)Ran: nfs/002
Failures: nfs/002
Failed 1 of 1 tests
Quoted 19 lines
tests/nfs/002 new file +8
-0+if [ $res -ne 0 ];then + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks" +fi + +echo "Silence is golden" +# success, all done +status=0 +exittests/nfs/002.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 002 +Silence is goldenSignature · 1 line
2.31.1
Quoted 65 lines · Zorro Lang
Quoted 64 lines · Zorro Lang
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
Signed-off-by Zorro Lang <[email protected]>---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10MAs this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorrotests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.outtests/nfs/002 new file +32
-0@@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes") +# +. ./common/preamble +_begin_fstest auto rw + +# real QA test starts here +_supported_fs nfs +_require_test + +localfile=$TEST_DIR/testfile.$seq +rm -rf $localfile + +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1 +block_size=`stat -c '%B' $localfile` +iblocks_expected=$((10 * 1024 * 1024 / $block_size)) +# Try several times for the speculative allocated file size can be shrunken +res=1 +for ((i=0; i<10; i++));do + iblocks_real=`stat -c '%b' $localfile` + if [ "$iblocks_expected" = "$iblocks_real" ];then + res=0 + break
Quoted 8 lines
tests/nfs/002 line 35
Quoted 3 lines
tests/nfs/002 new file +3
-0+ fi + sleep 10 +doneHmm... this case sometimes fails on kernel 6.1.0-rc6 [1] (nfs4.2 base on xfs), even I changed the sleep time to 20s * 10, it still fails. But I can't reproduce this failure if the underlying fs is ext4... cc linux-xfs, to check if I miss something for xfs? Or this's a xfs issue?
Could be anything, really -- speculative preallocation on the server, or xattrs blowing up the attr fork. You'd have to go query the file mappings and whatnot of the xfs file on the server to find out.
--D
Quoted 44 lines
Thanks,
Zorro[1]
# ./check nfs/002 FSTYP -- nfs PLATFORM -- Linux/x86_64 dell-per640-04 6.1.0-rc6 #1 SMP PREEMPT_DYNAMIC Mon Nov 21 00:51:20 EST 2022 MKFS_OPTIONS -- dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server MOUNT_OPTIONS -- -o vers=4.2 -o context=system_u:object_r:root_t:s0 dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server /mnt/xfstests/scratch/nfs-clientnfs/002 3s ... - output mismatch (see
/var/lib/xfstests/results//nfs/002.out.bad)--- tests/nfs/002.out 2022-11-21 01:29:33.861770474 -0500 +++ /var/lib/xfstests/results//nfs/002.out.bad 2022-11-21 13:27:37.424199056 -0500 @@ -1,2 +1,3 @@ QA output created by 002 +Write 20480 blocks, but get 32640 blocks Silence is golden ... (Run 'diff -u /var/lib/xfstests/tests/nfs/002.out /var/lib/xfstests/results//nfs/002.out.bad' to see the entire diff)Ran: nfs/002
Failures: nfs/002
Failed 1 of 1 testsQuoted 19 lines
tests/nfs/002 new file +8
-0+if [ $res -ne 0 ];then + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks" +fi + +echo "Silence is golden" +# success, all done +status=0 +exittests/nfs/002.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 002 +Silence is goldenSignature · 1 line
2.31.1
Quoted 75 lines · Darrick J. Wong
Quoted 65 lines · Zorro Lang
Quoted 64 lines · Zorro Lang
Test nfs and its underlying fs, make sure file size as expected after writting a file, and the speculative allocation space can be shrunken.
Signed-off-by Zorro Lang <[email protected]>---
Hi,
The original bug reproducer is:
1. mount nfs3 backed by xfs
2. dd if=/dev/zero of=/nfs/10M bs=1M count=10
3. du -sh /nfs/10M
16M /nfs/10MAs this was a xfs issue, so cc linux-xfs@ to get review.
Thanks,
Zorrotests/nfs/002 | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/nfs/002.out | 2 ++ 2 files changed, 45 insertions(+) create mode 100755 tests/nfs/002 create mode 100644 tests/nfs/002.outtests/nfs/002 new file +32
-0@@ -0,0 +1,43 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2022 Red Hat, Inc. All Rights Reserved. +# +# FS QA Test 002 +# +# Make sure nfs gets expected file size after writting a big sized file. It's +# not only testing nfs, test its underlying fs too. For example a known old bug +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after +# writting 10M data to a file. It's fixed by a series of patches around +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes") +# +. ./common/preamble +_begin_fstest auto rw + +# real QA test starts here +_supported_fs nfs +_require_test + +localfile=$TEST_DIR/testfile.$seq +rm -rf $localfile + +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1 +block_size=`stat -c '%B' $localfile` +iblocks_expected=$((10 * 1024 * 1024 / $block_size)) +# Try several times for the speculative allocated file size can be shrunken +res=1 +for ((i=0; i<10; i++));do + iblocks_real=`stat -c '%b' $localfile` + if [ "$iblocks_expected" = "$iblocks_real" ];then + res=0 + breakQuoted 8 lines
tests/nfs/002 line 35
Quoted 3 lines
tests/nfs/002 new file +3
-0+ fi + sleep 10 +doneHmm... this case sometimes fails on kernel 6.1.0-rc6 [1] (nfs4.2 base on xfs), even I changed the sleep time to 20s * 10, it still fails. But I can't reproduce this failure if the underlying fs is ext4... cc linux-xfs, to check if I miss something for xfs? Or this's a xfs issue?
Quoted 3 lines
Could be anything, really -- speculative preallocation on the server, or xattrs blowing up the attr fork. You'd have to go query the file mappings and whatnot of the xfs file on the server to find out.
Hi Darrick,
Thanks for your reply.
When the number of blocks isn't as expected, I printed the bmap of the test file on xfs, it really shows 20480 blocks [1], not 32640 blocks.
And I checked the xattr of the file on xfs, it doesn't has attr blocks [2], it's local format. Even if it has some attr blocks, 32640 too much bigger than 20480, I don't think selinux or other xattr takes that many blocks.
Hmm... this test case passed on nfs3 and nfs4.0 with xfs, but fails on nfs4.1 or nfs4.2 with xfs. cc linux-nfs@ list to get more review.
BTW, this case is an original regression test for a rhel bug:
https://bugzilla.redhat.com/show_bug.cgi?id=955254 (you might can't open it) which blamed that "xfs files written over nfs are bigger due to speculative allocation and may never shrink while fs is mounted".
Dave said:
For NFS servers, keeping the speculative prealloc around for as long as possible is the desired behaviour. It will get truncated away when the inode is cycled out of the cache by memory pressure on the NFS server. So the code is currently behaving as designed.
But there are changes upstream that give different behaviour, so we merged below xfs patches to "fix" that bug:
xfs: add EOFBLOCKS inode tagging/untagging
xfs: support a tag-based inode_ag_iterator
xfs: create helper to check whether to free eofblocks on inode
xfs: make xfs_free_eofblocks() non-static, return EAGAIN on trylock failure
xfs: create function to scan and clear EOFBLOCKS inodes
xfs: add XFS_IOC_FREE_EOFBLOCKS ioctl
xfs: add inode id filtering to eofblocks scan
xfs: support multiple inode id filtering in eofblocks scan
xfs: add minimum file size filtering to eofblocks scan
xfs: add background scanning to clear eofblocks inodes
xfs: limit speculative prealloc near ENOSPC thresholds
xfs: limit speculative prealloc size on sparse files
xfs: fix potential infinite loop in xfs_iomap_prealloc_size()
xfs: increase prealloc size to double that of the previous extent
xfs: fix xfs_iomap_eof_prealloc_initial_size type
xfs: Define a new function xfs_this_quota_on()
xfs: Define a new function xfs_inode_dquot()
xfs: reorganize xfs_iomap_prealloc_size to remove indentation
xfs: push rounddown_pow_of_two() to after prealloc throttle
xfs: pass xfs_dquot to xfs_qm_adjust_dqlimits() instead of xfs_disk_dquot_t
xfs: xfs_dquot prealloc throttling watermarks and low free space
xfs: add quota-driven speculative preallocation throttling
xfs: xfs_iomap_prealloc_size() tracepoint
xfs: don't use speculative prealloc for small files
How I'm not sure if it's a bug again, or an expected result for nfs4.2 with xfs.
Thanks,
Zorro
[1]/mnt/xfstests/test/nfs-server/testfile.002:
EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..20479]: 192..20671 0 (192..20671) 20480 000000
FLAG Values:
0100000 Shared extent
0010000 Unwritten preallocated extent
0001000 Doesn't begin on stripe unit
0000100 Doesn't end on stripe unit
0000010 Doesn't begin on stripe width
0000001 Doesn't end on stripe width
[2]
core.format = 2 (extents)
...
core.size = 10485760
core.nblocks = 2560
core.extsize = 0
core.nextents = 1
core.naextents = 0
core.forkoff = 24
core.aformat = 1 (local)
...
u3.bmx[0] = [startoff,startblock,blockcount,extentflag] 0:[0,24,2560,0]
a.sfattr.hdr.totsize = 47
a.sfattr.hdr.count = 1
a.sfattr.list[0].namelen = 7 a.sfattr.list[0].valuelen = 33
a.sfattr.list[0].root = 0
a.sfattr.list[0].secure = 1
a.sfattr.list[0].name = "selinux" a.sfattr.list[0].value = "system_u:object_r:unlabeled_t:s0\000"
Quoted 48 lines
--D
Quoted 44 lines
Thanks,
Zorro[1]
# ./check nfs/002 FSTYP -- nfs PLATFORM -- Linux/x86_64 dell-per640-04 6.1.0-rc6 #1 SMP PREEMPT_DYNAMIC Mon Nov 21 00:51:20 EST 2022 MKFS_OPTIONS -- dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server MOUNT_OPTIONS -- -o vers=4.2 -o context=system_u:object_r:root_t:s0 dell-per640-04.dell2.lab.eng.bos.redhat.com:/mnt/xfstests/scratch/nfs-server /mnt/xfstests/scratch/nfs-clientnfs/002 3s ... - output mismatch (see
/var/lib/xfstests/results//nfs/002.out.bad)--- tests/nfs/002.out 2022-11-21 01:29:33.861770474 -0500 +++ /var/lib/xfstests/results//nfs/002.out.bad 2022-11-21 13:27:37.424199056 -0500 @@ -1,2 +1,3 @@ QA output created by 002 +Write 20480 blocks, but get 32640 blocks Silence is golden ... (Run 'diff -u /var/lib/xfstests/tests/nfs/002.out /var/lib/xfstests/results//nfs/002.out.bad' to see the entire diff)Ran: nfs/002
Failures: nfs/002
Failed 1 of 1 testsQuoted 19 lines
tests/nfs/002 new file +8
-0+if [ $res -ne 0 ];then + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks" +fi + +echo "Silence is golden" +# success, all done +status=0 +exittests/nfs/002.out new file +2
-0@@ -0,0 +1,2 @@ +QA output created by 002 +Silence is goldenSignature · 1 line
2.31.1