Skip to contentSkip to conversation

xfs: new EOF fragmentation tests

PATCH In review Dave Chinner 3 participants 7 messages Feb 11, 2019 – Feb 17, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: new EOF fragmentation tests

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

---
 tests/xfs/500     | 79 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/500.out |  9 +++++
 tests/xfs/501     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/501.out |  9 +++++
 tests/xfs/502     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/502.out |  9 +++++
 tests/xfs/503     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/503.out | 33 ++++++++++++++++++
 tests/xfs/group   |  4 +++
 9 files changed, 394 insertions(+)
 create mode 100755 tests/xfs/500
 create mode 100644 tests/xfs/500.out
 create mode 100755 tests/xfs/501
 create mode 100644 tests/xfs/501.out
 create mode 100755 tests/xfs/502
 create mode 100644 tests/xfs/502.out
 create mode 100755 tests/xfs/503
 create mode 100644 tests/xfs/503.out
tests/xfs/500 new file +79 -0
new file mode 100755
index 00000000..d0802e86
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/500
+#
+# Post-EOF preallocation defeat test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to
+# interleave allocations to fragment the files. Synchronous writes defeat the
+# open/write/close heuristics in xfs_release() that prevent EOF block removal,
+# so this should fragment badly. Typical problematic behaviour shows per-file
+# extent counts of >900 (almost worse case) whilst fixed behaviour typically
+# shows extent counts in the low 20s.
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/500.out new file +9 -0
new file mode 100644
index 00000000..1ffc1790
@@ -0,0 +1,9 @@
+QA output created by 500
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/501 new file +81 -0

81 lines not on this page · raw message

tests/xfs/501.out new file +9 -0
new file mode 100644
index 00000000..ead0b496
@@ -0,0 +1,9 @@
+QA output created by 501
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/502 new file +81 -0

81 lines not on this page · raw message

tests/xfs/502.out new file +9 -0
new file mode 100644
index 00000000..766a6efe
@@ -0,0 +1,9 @@
+QA output created by 502
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/503 new file +89 -0

89 lines not on this page · raw message

tests/xfs/503.out new file +33 -0
new file mode 100644
index 00000000..0089a698
@@ -0,0 +1,33 @@
+QA output created by 503
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
+file.8 extent count is in range
+file.9 extent count is in range
+file.10 extent count is in range
+file.11 extent count is in range
+file.12 extent count is in range
+file.13 extent count is in range
+file.14 extent count is in range
+file.15 extent count is in range
+file.16 extent count is in range
+file.17 extent count is in range
+file.18 extent count is in range
+file.19 extent count is in range
+file.20 extent count is in range
+file.21 extent count is in range
+file.22 extent count is in range
+file.23 extent count is in range
+file.24 extent count is in range
+file.25 extent count is in range
+file.26 extent count is in range
+file.27 extent count is in range
+file.28 extent count is in range
+file.29 extent count is in range
+file.30 extent count is in range
+file.31 extent count is in range
tests/xfs/group +4 -0
index 7b7d69f1..d0d33d73 100644
@@ -497,3 +497,7 @@
 497 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 498 dangerous_fuzzers dangerous_norepair
 499 auto quick
+500 auto prealloc rw
+501 auto prealloc rw
+502 auto prealloc rw
+503 auto prealloc rw
Signature · 1 line
2.20.1
#1 lore raw
Quoted 12 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

Quoted 2 lines

---

Forgot to add that these tests go along with this patch series:

https://marc.info/?l=linux-xfs&m=154951612101291&w=2

And can be used to test whatever way we decide to minimise fragmentation through maintianing post-EOF preallocation.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
In reply to Dave Chinner
Quoted 47 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

---
 tests/xfs/500     | 79 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/500.out |  9 +++++
 tests/xfs/501     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/501.out |  9 +++++
 tests/xfs/502     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/502.out |  9 +++++
 tests/xfs/503     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/503.out | 33 ++++++++++++++++++
 tests/xfs/group   |  4 +++
 9 files changed, 394 insertions(+)
 create mode 100755 tests/xfs/500
 create mode 100644 tests/xfs/500.out
 create mode 100755 tests/xfs/501
 create mode 100644 tests/xfs/501.out
 create mode 100755 tests/xfs/502
 create mode 100644 tests/xfs/502.out
 create mode 100755 tests/xfs/503
 create mode 100644 tests/xfs/503.out
tests/xfs/500 new file +8 -0
new file mode 100755
index 00000000..d0802e86
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/500
+#
+# Post-EOF preallocation defeat test
+#

...

Quoted 10 lines
tests/xfs/500 new file +10 -0
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do

tests/xfs/500 line 69

Quoted 10 lines
tests/xfs/500 new file +10 -0
+		$XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync

I'm not a huge fan of seeing these global syncs sprinkled around tests where they might not be needed. Are these here for a reason? If so, could they be replaced with an xfs_io -c syncfs or some such?

Quoted 10 lines
tests/xfs/500 new file +10 -0
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+# success, all done
+status=0
+exit

...

Quoted 6 lines
tests/xfs/502 new file +0 -0
new file mode 100755
index 00000000..9f314a3d
@@ -0,0 +1,81 @@

...

tests/xfs/502 line 47

Quoted 8 lines
tests/xfs/502 new file +8 -0
+
+# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
+# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
+# the open/write/close heuristics in xfs_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
+# shows extent counts in the low single digits (almost best case)
+#

This is essentially equivalent to the previous test, right? E.g., buffered writes with extsz hints essentially uses the same allocation path as direct I/O with an extent size hint in current XFS. Of course that may not always be so, but I wonder if these two could be combined.

Otherwise these all seem fine to me.

Brian

Quoted 198 lines
tests/xfs/502 new file +34 -0
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+extent_size=16m
+
+write_direct_file()
+{
+	idx=$1
+
+	$XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -d -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_direct_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-10
+	_within_tolerance "file.$n extent count" $count 2 1 8 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/502.out new file +9 -0
new file mode 100644
index 00000000..766a6efe
@@ -0,0 +1,9 @@
+QA output created by 502
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/503 new file +89 -0
new file mode 100755
index 00000000..ad303551
@@ -0,0 +1,89 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/503
+#
+# Post-EOF preallocation defeat test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to
+# interleave allocations to fragment the files. Assuming we've fixed the
+# sycnhronous write defeat, we can still trigger the same issue with a
+# open/read/close on O_RDONLY files. We should not be triggering EOF
+# preallocation removal on files we don't have permission to write, so until
+# this is fixed it should fragment badly.  Typical problematic behaviour shows
+# per-file extent counts of 50-350 whilst fixed behaviour typically demonstrates
+# post-eof speculative delalloc growth in extent size (~6 extents for 50MB
+# file).
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=32
+wsize=4096
+wcnt=1000
+
+write_file()
+{
+	idx=$1
+
+	$XFS_IO_PROG -f -s -c "pwrite -b 64k 0 50m" $workfile.$idx
+}
+
+read_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -r -c "pread 0 28" $workfile.$idx
+	done
+}
+
+rm -f $workdir/file*
+for ((n=0; n<$((nfiles)); n++)); do
+	write_file $n > /dev/null 2>&1 &
+	read_file $n > /dev/null 2>&1 &
+done
+wait
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 6 5 10 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/503.out new file +33 -0
new file mode 100644
index 00000000..0089a698
@@ -0,0 +1,33 @@
+QA output created by 503
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
+file.8 extent count is in range
+file.9 extent count is in range
+file.10 extent count is in range
+file.11 extent count is in range
+file.12 extent count is in range
+file.13 extent count is in range
+file.14 extent count is in range
+file.15 extent count is in range
+file.16 extent count is in range
+file.17 extent count is in range
+file.18 extent count is in range
+file.19 extent count is in range
+file.20 extent count is in range
+file.21 extent count is in range
+file.22 extent count is in range
+file.23 extent count is in range
+file.24 extent count is in range
+file.25 extent count is in range
+file.26 extent count is in range
+file.27 extent count is in range
+file.28 extent count is in range
+file.29 extent count is in range
+file.30 extent count is in range
+file.31 extent count is in range
tests/xfs/group +4 -0
index 7b7d69f1..d0d33d73 100644
@@ -497,3 +497,7 @@
 497 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 498 dangerous_fuzzers dangerous_norepair
 499 auto quick
+500 auto prealloc rw
+501 auto prealloc rw
+502 auto prealloc rw
+503 auto prealloc rw
Signature · 1 line
2.20.1
#3 lore raw
Quoted 12 lines · Brian Foster
Quoted 11 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

.....

Quoted 16 lines
Quoted 16 lines
tests/xfs/500 new file +16 -0
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
Quoted 8 lines

tests/xfs/500 line 69

Quoted 4 lines
tests/xfs/500 new file +4 -0
+	write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync

I'm not a huge fan of seeing these global syncs sprinkled around tests where they might not be needed. Are these here for a reason? If so, could they be replaced with an xfs_io -c syncfs or some such?

sync is there to ensure that everything is stable and I don't need to care about using sync flags for counting extents.

I could use syncfs (either binary or xfs_io) but that's not supported on all the OS's that fstests runs on. And I don't think using sync really matters - if "sync" causes problems for fstests that using syncfs fixes, then we've got bigger issues to worry about.

Quoted 18 lines
Quoted 10 lines
tests/xfs/500 new file +10 -0
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+# success, all done
+status=0
+exit

...

Quoted 6 lines
tests/xfs/502 new file +0 -0
new file mode 100755
index 00000000..9f314a3d
@@ -0,0 +1,81 @@

...

Quoted 10 lines

tests/xfs/502 line 47

Quoted 8 lines
tests/xfs/502 new file +8 -0
+
+# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
+# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
+# the open/write/close heuristics in xfs_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
+# shows extent counts in the low single digits (almost best case)
+#

This is essentially equivalent to the previous test, right? E.g.,

IMO, no. This is a different IO pattern that demonstrates a fragmentation problem, regardless of the underlying cause. We may fix the buffered write problem (e.g. via enabling delalloc on extent size hints) but that leaves the direct IO variant unchanged. Hence, if we drop this test, we'll never realise we still have a direct IO variant we have to ensure we don't break.

IOWs, I'm attempting to characterise and exercise the userspace IO patterns that cause fragmentation problems rather than exercise a specific bug in the filesystem implementation.

Quoted 2 lines

path as direct I/O with an extent size hint in current XFS. Of course that may not always be so, but I wonder if these two could be combined.

That's kinda my point - these IO patterns haven't always (and won't always) exercised the same code paths and allocation behaviours. Hence we nee dto exercise them both, even if they currently both expose the same underlying problem.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw
Quoted 39 lines · Dave Chinner
Quoted 12 lines · Brian Foster
Quoted 11 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

.....

Quoted 16 lines
Quoted 16 lines
tests/xfs/500 new file +16 -0
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
Quoted 8 lines

tests/xfs/500 line 69

Quoted 4 lines
tests/xfs/500 new file +4 -0
+	write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync

I'm not a huge fan of seeing these global syncs sprinkled around tests where they might not be needed. Are these here for a reason? If so, could they be replaced with an xfs_io -c syncfs or some such?

Quoted 9 lines

sync is there to ensure that everything is stable and I don't need to care about using sync flags for counting extents.

I could use syncfs (either binary or xfs_io) but that's not supported on all the OS's that fstests runs on. And I don't think using sync really matters - if "sync" causes problems for fstests that using syncfs fixes, then we've got bigger issues to worry about.

It's not a matter of sync causing problems, I just think it's a kludge to sync the entire system. syncfs by definition is localized to a target fs.

Having taken a closer look, xfs_io unconditionally uses the fiemap sync flag so this is an instance where we don't need either.

Quoted 40 lines
Quoted 18 lines
Quoted 10 lines
tests/xfs/500 new file +10 -0
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+# success, all done
+status=0
+exit

...

Quoted 6 lines
tests/xfs/502 new file +0 -0
new file mode 100755
index 00000000..9f314a3d
@@ -0,0 +1,81 @@

...

Quoted 10 lines

tests/xfs/502 line 47

Quoted 8 lines
tests/xfs/502 new file +8 -0
+
+# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
+# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
+# the open/write/close heuristics in xfs_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
+# shows extent counts in the low single digits (almost best case)
+#

This is essentially equivalent to the previous test, right? E.g.,

IMO, no. This is a different IO pattern that demonstrates a fragmentation problem, regardless of the underlying cause. We may fix the buffered write problem (e.g. via enabling delalloc on extent size hints) but that leaves the direct IO variant unchanged. Hence, if we drop this test, we'll never realise we still have a direct IO variant we have to ensure we don't break.

IOWs, I'm attempting to characterise and exercise the userspace IO patterns that cause fragmentation problems rather than exercise a specific bug in the filesystem implementation.

Quoted 8 lines
Quoted 2 lines

path as direct I/O with an extent size hint in current XFS. Of course that may not always be so, but I wonder if these two could be combined.

That's kinda my point - these IO patterns haven't always (and won't always) exercised the same code paths and allocation behaviours. Hence we nee dto exercise them both, even if they currently both expose the same underlying problem.

Ok, but I wasn't suggesting to not test any of these codepaths. I wanted to make sure the duplication of (current) behavior across the separate tests was intentional, for one. I'm also noting that the difference between this test and the previous is the xfs_io '-d' flag, and thus wondering why we wouldn't combine that into a single test (that tests extsz hint fragmentation on buffered and direct I/O).

Brian

Quoted 7 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw
In reply to Dave Chinner
Quoted 39 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

---
 tests/xfs/500     | 79 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/500.out |  9 +++++
 tests/xfs/501     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/501.out |  9 +++++
 tests/xfs/502     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/502.out |  9 +++++
 tests/xfs/503     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/503.out | 33 ++++++++++++++++++
 tests/xfs/group   |  4 +++
 9 files changed, 394 insertions(+)
 create mode 100755 tests/xfs/500
 create mode 100644 tests/xfs/500.out
 create mode 100755 tests/xfs/501
 create mode 100644 tests/xfs/501.out
 create mode 100755 tests/xfs/502
 create mode 100644 tests/xfs/502.out
 create mode 100755 tests/xfs/503
 create mode 100644 tests/xfs/503.out
tests/xfs/500 new file +0 -0
new file mode 100755
index 00000000..d0802e86

tests/xfs/500 line 7

Quoted 7 lines
tests/xfs/500 new file +7 -0
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/500
+#
+# Post-EOF preallocation defeat test

The test description of the four tests are all the same, I know there're detailed descriptions later in each test, but it's better to write high level descriptions here.

Quoted 17 lines
tests/xfs/500 new file +17 -0
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks

tests/xfs/500 line 34

Quoted 10 lines
tests/xfs/500 new file +10 -0
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic

_supported_fs xfs

The four tests all have the same issue.

Quoted 301 lines
tests/xfs/500 new file +45 -0
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to
+# interleave allocations to fragment the files. Synchronous writes defeat the
+# open/write/close heuristics in xfs_release() that prevent EOF block removal,
+# so this should fragment badly. Typical problematic behaviour shows per-file
+# extent counts of >900 (almost worse case) whilst fixed behaviour typically
+# shows extent counts in the low 20s.
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+
+write_sync_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -s -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_sync_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 21 19 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/500.out new file +9 -0
new file mode 100644
index 00000000..1ffc1790
@@ -0,0 +1,9 @@
+QA output created by 500
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/501 new file +81 -0
new file mode 100755
index 00000000..493e4d77
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/501
+#
+# Post-EOF preallocation defeat test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using buffered writes with extent size hints.
+# Aim is to interleave allocations to fragment the files. Writes w/ extent size
+# hints set defeat the open/write/close heuristics in xfs_release() that prevent
+# EOF block removal, so this should fragment badly. Typical problematic
+# behaviour shows per-file extent counts of 1000 (worst case!) whilst
+# fixed behaviour should show very few extents (almost best case).
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+extent_size=16m
+
+write_extsz_file()
+{
+	idx=$1
+
+	$XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_extsz_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-10
+	_within_tolerance "file.$n extent count" $count 2 1 8 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/501.out new file +9 -0
new file mode 100644
index 00000000..ead0b496
@@ -0,0 +1,9 @@
+QA output created by 501
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/502 new file +81 -0
new file mode 100755
index 00000000..9f314a3d
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/502
+#
+# Post-EOF preallocation defeat test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using O_DIRECT writes w/ extent size hints.
+# Aim is to interleave allocations to fragment the files. O_DIRECT writes defeat
+# the open/write/close heuristics in xfs_release() that prevent EOF block
+# removal, so this should fragment badly. Typical problematic behaviour shows
+# per-file extent counts of ~1000 (worst case) whilst fixed behaviour typically
+# shows extent counts in the low single digits (almost best case)
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=8
+wsize=4096
+wcnt=1000
+extent_size=16m
+
+write_direct_file()
+{
+	idx=$1
+
+	$XFS_IO_PROG -f -c "extsize $extent_size" $workfile.$idx
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -d -c "pwrite $((cnt * wsize)) $wsize" $workfile.$idx
+	done
+}
+
+rm -f $workfile*
+for ((n=0; n<$nfiles; n++)); do
+	write_direct_file $n > /dev/null 2>&1 &
+done
+wait
+sync
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-10
+	_within_tolerance "file.$n extent count" $count 2 1 8 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/502.out new file +9 -0
new file mode 100644
index 00000000..766a6efe
@@ -0,0 +1,9 @@
+QA output created by 502
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
tests/xfs/503 new file +31 -0
new file mode 100755
index 00000000..ad303551
@@ -0,0 +1,89 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
+#
+# FS QA Test xfs/503
+#
+# Post-EOF preallocation defeat test
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here

tests/xfs/503 line 41

Quoted 10 lines
tests/xfs/503 new file +10 -0
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to

This should be "using open/read/close loops" not "synchronous buffered writes"?

Thanks,
Eryu

Quoted 102 lines
tests/xfs/503 new file +48 -0
+# interleave allocations to fragment the files. Assuming we've fixed the
+# sycnhronous write defeat, we can still trigger the same issue with a
+# open/read/close on O_RDONLY files. We should not be triggering EOF
+# preallocation removal on files we don't have permission to write, so until
+# this is fixed it should fragment badly.  Typical problematic behaviour shows
+# per-file extent counts of 50-350 whilst fixed behaviour typically demonstrates
+# post-eof speculative delalloc growth in extent size (~6 extents for 50MB
+# file).
+#
+# Failure is determined by golden output mismatch from _within_tolerance().
+
+workfile=$SCRATCH_MNT/file
+nfiles=32
+wsize=4096
+wcnt=1000
+
+write_file()
+{
+	idx=$1
+
+	$XFS_IO_PROG -f -s -c "pwrite -b 64k 0 50m" $workfile.$idx
+}
+
+read_file()
+{
+	idx=$1
+
+	for ((cnt=0; cnt<$wcnt; cnt++)); do
+		$XFS_IO_PROG -f -r -c "pread 0 28" $workfile.$idx
+	done
+}
+
+rm -f $workdir/file*
+for ((n=0; n<$((nfiles)); n++)); do
+	write_file $n > /dev/null 2>&1 &
+	read_file $n > /dev/null 2>&1 &
+done
+wait
+
+for ((n=0; n<$nfiles; n++)); do
+	count=$(_count_extents $workfile.$n)
+	# Acceptible extent count range is 1-40
+	_within_tolerance "file.$n extent count" $count 6 5 10 -v
+done
+
+# success, all done
+status=0
+exit
tests/xfs/503.out new file +33 -0
new file mode 100644
index 00000000..0089a698
@@ -0,0 +1,33 @@
+QA output created by 503
+file.0 extent count is in range
+file.1 extent count is in range
+file.2 extent count is in range
+file.3 extent count is in range
+file.4 extent count is in range
+file.5 extent count is in range
+file.6 extent count is in range
+file.7 extent count is in range
+file.8 extent count is in range
+file.9 extent count is in range
+file.10 extent count is in range
+file.11 extent count is in range
+file.12 extent count is in range
+file.13 extent count is in range
+file.14 extent count is in range
+file.15 extent count is in range
+file.16 extent count is in range
+file.17 extent count is in range
+file.18 extent count is in range
+file.19 extent count is in range
+file.20 extent count is in range
+file.21 extent count is in range
+file.22 extent count is in range
+file.23 extent count is in range
+file.24 extent count is in range
+file.25 extent count is in range
+file.26 extent count is in range
+file.27 extent count is in range
+file.28 extent count is in range
+file.29 extent count is in range
+file.30 extent count is in range
+file.31 extent count is in range
tests/xfs/group +4 -0
index 7b7d69f1..d0d33d73 100644
@@ -497,3 +497,7 @@
 497 dangerous_fuzzers dangerous_scrub dangerous_online_repair
 498 dangerous_fuzzers dangerous_norepair
 499 auto quick
+500 auto prealloc rw
+501 auto prealloc rw
+502 auto prealloc rw
+503 auto prealloc rw
Signature · 1 line
2.20.1
#6 lore raw
Quoted 43 lines · Eryu Guan
Quoted 42 lines · Dave Chinner

From: Dave Chinner <[email protected]>

These tests create substantial file fragmentation as a result of application actions that defeat post-EOF preallocation optimisations. They are intended to replicate known vectors for these problems, and provide a check that the fragmentation levels have been controlled. The mitigations we make may not completely remove fragmentation (e.g. they may demonstrate speculative delalloc related extent size growth) so the checks don't assume we'll end up with perfect layouts and hence check for an exceptable level of fragmentation rather than none.

---
 tests/xfs/500     | 79 +++++++++++++++++++++++++++++++++++++++++
 tests/xfs/500.out |  9 +++++
 tests/xfs/501     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/501.out |  9 +++++
 tests/xfs/502     | 81 ++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/502.out |  9 +++++
 tests/xfs/503     | 89 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/503.out | 33 ++++++++++++++++++
 tests/xfs/group   |  4 +++
 9 files changed, 394 insertions(+)
 create mode 100755 tests/xfs/500
 create mode 100644 tests/xfs/500.out
 create mode 100755 tests/xfs/501
 create mode 100644 tests/xfs/501.out
 create mode 100755 tests/xfs/502
 create mode 100644 tests/xfs/502.out
 create mode 100755 tests/xfs/503
 create mode 100644 tests/xfs/503.out
tests/xfs/500 new file +3 -0
new file mode 100755
index 00000000..d0802e86
@@ -0,0 +1,79 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2019 Red Hat, Inc.  All Rights Reserved.
Quoted 8 lines

tests/xfs/500 line 7

Quoted 4 lines
tests/xfs/500 new file +4 -0
+#
+# FS QA Test xfs/500
+#
+# Post-EOF preallocation defeat test

The test description of the four tests are all the same, I know there're detailed descriptions later in each test, but it's better to write high level descriptions here.

Not really. lsqa.pl gets drowned by detailed test decsriptions in the header rather than just the one line description it was originally supposed to be. i.e. if I'm looking for preallocation defeat tests, then './lsqa.pl |grep -B 2 -i preallocation' should get me the description of all the tests that exercise preallocation in some way.

e.g.

$ ./lsqa.pl |grep -B 2 -i prealloc |grep "FS QA"
FS QA Test No. btrfs/013
FS QA Test 153
FS QA Test No. 094
FS QA Test No. 226
FS QA Test No. 274
FS QA Test No. 009
FS QA Test No. xfs/014
FS QA Test 118
FS QA Test No. 165
FS QA Test xfs/500
FS QA Test xfs/501
FS QA Test xfs/502
FS QA Test xfs/503

(you can also see why I put "FS QA Test <testdir>/<number>" in the identifier line, too)

IOWs, the rest of the test description is irrelevant to lsqa - like git commits, it's mostly the one-line description that matters for finding relevant tests....

AFAIC, if you're describing how/why the test code does what it does, then it should be with the code that does those operations, not 50 lines away where it will bitrot as the code gets changed over time. That's pretty much the rule we use for writing kernel/application code - the test code should be no different...

Quoted 30 lines
Quoted 17 lines
tests/xfs/500 new file +17 -0
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks

tests/xfs/500 line 34

Quoted 10 lines
tests/xfs/500 new file +10 -0
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic

_supported_fs xfs

Quoted 1 line

The four tests all have the same issue.

The contents of the test is generic and has no XFS specific code in them. I just placed it in tests/xfs because I don't care about what the other filesystems do in these situations.

Whatever, I'll change it.

Quoted 12 lines
Quoted 12 lines
tests/xfs/500 new file +12 -0
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_os Linux
+_require_scratch
Quoted 8 lines

tests/xfs/500 line 41

Quoted 5 lines
tests/xfs/500 new file +5 -0
+
+_scratch_mkfs 2>&1 >> $seqres.full
+_scratch_mount
+
+# Write multiple files in parallel using synchronous buffered writes. Aim is to

This should be "using open/read/close loops" not "synchronous buffered writes"?

No. write_file() uses synchronous buffered writes and they are written in parallel. And three lines later:

Quoted 3 lines

tests/xfs/500 line 54

Quoted 3 lines
tests/xfs/500 new file +3 -0
+# interleave allocations to fragment the files. Assuming we've fixed the
+# sycnhronous write defeat, we can still trigger the same issue with a
+# open/read/close on O_RDONLY files.

is the description of using open/read/close loops to trigger fragmentation.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]