Skip to contentSkip to conversation

fstests: add tests for btrfs' raid-stripe-tree feature

PATCH Superseded Anand Jain 2 participants 18 messages Dec 29, 2023 – Jan 4, 2024 Btrfs file system

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
author
#5 lore raw

PATCH v7 5/10 common: add _filter_trailing_whitespace

In reply to Anand Jain

The command 'btrfs inspect-internal dump-tree -t raid_stripe' introduces trailing whitespace in its output.
Apply a filter to remove it. Used in btrfs/30[4-8][.out].

---
 common/filter | 5 +++++
 1 file changed, 5 insertions(+)
common/filter +5 -0
index 509ee95039ac..016d213b8bee 100644
@@ -651,5 +651,10 @@ _filter_bash()
 	sed -e "s/^bash: line 1: /bash: /"
 }
 
+_filter_trailing_whitespace()
+{
+	sed -e "s/ $//"
+}
+
 # make sure this script returns success
 /bin/true
Signature · 1 line
2.39.3

Anand Jain

author
#0 lore raw

v7 cover fstests: add tests for btrfs' raid-stripe-tree feature

Changes in v7:
- Fixed trailing whitespace in the .out files
- Fixed the following test statement in 30[4-8]:

     test _get_page_size -eq 4096

- Link to v6: https://lore.kernel.org/r/[email protected]

--- original cover page from Johannes ----
Add tests for btrfs' raid-stripe-tree feature. All of these test work by writing a specific pattern to a newly created filesystem and afterwards using btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL to verify the placement and the layout of the metadata.

The md5sum of each file will be compared as well after a re-mount of the filesystem.

---
Changes in v6:
- require 4k pagesize for all tests as output depends on page size
- Add Filipe's Reviewed-by
- Link to v5: https://lore.kernel.org/r/[email protected]

Changes in v5:
- add _require_btrfs_free_space_tree helper and use in tests
- Link to v4: https://lore.kernel.org/r/[email protected]

Changes in v4:
- add _require_btrfs_no_compress to all tests
- add _require_btrfs_no_nodatacow helper and add to btrfs/308
- add _require_btrfs_feature "free_space_tree" to all tests
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it automatically
- Rename test cases as btrfs/302 and btrfs/303 already exist upstream
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Re-ordered series so the newly introduced group is added before the tests
- Changes Filipe requested to the tests.
- Link to v1: https://lore.kernel.org/r/[email protected]

Anand Jain (1):
common: add _filter_trailing_whitespace

Johannes Thumshirn (9):
fstests: doc: add new raid-stripe-tree group
common: add filter for btrfs raid-stripe dump
common: add _require_btrfs_no_nodatacow helper
common: add _require_btrfs_free_space_tree
btrfs: add fstest for stripe-tree metadata with 4k write
btrfs: add fstest for 8k write spanning two stripes on
raid-stripe-tree
btrfs: add fstest for writing to a file at an offset with RST
btrfs: add fstests to write 128k to a RST filesystem
btrfs: add fstest for overwriting a file partially with RST

 common/btrfs        |  17 +++++++
 common/filter       |   5 +++
 common/filter.btrfs |  14 ++++++
 doc/group-names.txt |   1 +
 tests/btrfs/304     |  59 ++++++++++++++++++++++++
 tests/btrfs/304.out |  58 ++++++++++++++++++++++++
 tests/btrfs/305     |  64 ++++++++++++++++++++++++++
 tests/btrfs/305.out |  82 ++++++++++++++++++++++++++++++++++
 tests/btrfs/306     |  62 ++++++++++++++++++++++++++
 tests/btrfs/306.out |  75 +++++++++++++++++++++++++++++++
 tests/btrfs/307     |  59 ++++++++++++++++++++++++
 tests/btrfs/307.out |  65 +++++++++++++++++++++++++++
 tests/btrfs/308     |  63 ++++++++++++++++++++++++++
 tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++
 14 files changed, 730 insertions(+)
 create mode 100755 tests/btrfs/304
 create mode 100644 tests/btrfs/304.out
 create mode 100755 tests/btrfs/305
 create mode 100644 tests/btrfs/305.out
 create mode 100755 tests/btrfs/306
 create mode 100644 tests/btrfs/306.out
 create mode 100755 tests/btrfs/307
 create mode 100644 tests/btrfs/307.out
 create mode 100755 tests/btrfs/308
 create mode 100644 tests/btrfs/308.out
Signature · 1 line
2.39.3
author
#1 lore raw

PATCH v7 1/10 fstests: doc: add new raid-stripe-tree group

From: Johannes Thumshirn <[email protected]>

Add a new test group for testing the raid-stripe-tree feature of btrfs with fstests.

---
 doc/group-names.txt | 1 +
 1 file changed, 1 insertion(+)
doc/group-names.txt +1 -0
index fec6bf71abcb..2ac95ac83a79 100644
@@ -94,6 +94,7 @@ punch			fallocate FALLOC_FL_PUNCH_HOLE
 qgroup			btrfs qgroup feature
 quota			filesystem usage quotas
 raid			btrfs RAID
+raid-stripe-tree	btrfs raid-stripe-tree feature
 read_repair		btrfs error correction on read failure
 realtime		XFS realtime volumes
 recoveryloop		crash recovery loops
Signature · 1 line
2.39.3
author
#2 lore raw

PATCH v7 2/10 common: add filter for btrfs raid-stripe dump

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

---
 common/filter.btrfs | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
common/filter.btrfs +14 -0
index 8c6fe5793663..e570648bb5e6 100644
@@ -126,5 +126,19 @@ _filter_btrfs_cloner_error()
 	sed -e "s/\(clone failed:\) Operation not supported/\1 Invalid argument/g"
 }
 
+# filter output of "btrfs inspect-internal dump-tree -t raid-stripe"
+_filter_stripe_tree()
+{
+	sed -E -e "s/leaf [0-9]+ items [0-9]+ free space [0-9]+ generation [0-9]+ owner RAID_STRIPE_TREE/leaf XXXXXXXXX items X free space XXXXX generation X owner RAID_STRIPE_TREE/" \
+		-e "s/leaf [0-9]+ flags 0x1\(WRITTEN\) backref revision 1/leaf XXXXXXXXX flags 0x1\(WRITTEN\) backref revision 1/" \
+		-e "s/checksum stored [0-9a-f]+/checksum stored <CHECKSUM>/"  \
+		-e "s/checksum calced [0-9a-f]+/checksum calced <CHECKSUM>/"  \
+		-e "s/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/<UUID>/" \
+		-e "s/item ([0-9]+) key \([0-9]+ RAID_STRIPE ([0-9]+)\) itemoff [0-9]+ itemsize ([0-9]+)/item \1 key \(XXXXXX RAID_STRIPE \2\) itemoff XXXXX itemsize \3/" \
+		-e "s/stripe ([0-9]+) devid ([0-9]+) physical [0-9]+/stripe \1 devid \2 physical XXXXXXXXX/" \
+		-e "s/total bytes [0-9]+/total bytes XXXXXXXX/" \
+		-e "s/bytes used [0-9]+/bytes used XXXXXX/"
+}
+
 # make sure this script returns success
 /bin/true
Signature · 1 line
2.39.3
author
#3 lore raw

PATCH v7 3/10 common: add _require_btrfs_no_nodatacow helper

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

---
 common/btrfs | 7 +++++++
 1 file changed, 7 insertions(+)
common/btrfs +7 -0
index f91f8dd869a1..9dd2a7f49e16 100644
@@ -120,6 +120,13 @@ _require_btrfs_no_compress()
 	fi
 }
 
+_require_btrfs_no_nodatacow()
+{
+	if _normalize_mount_options "$MOUNT_OPTIONS" | grep -q "nodatacow"; then
+		_notrun "This test requires no nodatacow enabled"
+	fi
+}
+
 _check_btrfs_filesystem()
 {
 	device=$1
Signature · 1 line
2.39.3
author
#4 lore raw

PATCH v7 4/10 common: add _require_btrfs_free_space_tree

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

---
 common/btrfs | 10 ++++++++++
 1 file changed, 10 insertions(+)
common/btrfs +10 -0
index 9dd2a7f49e16..e1b29c613767 100644
@@ -127,6 +127,16 @@ _require_btrfs_no_nodatacow()
 	fi
 }
 
+_require_btrfs_free_space_tree()
+{
+	_scratch_mkfs > /dev/null 2>&1
+	if ! $BTRFS_UTIL_PROG inspect-internal dump-super $SCRATCH_DEV | \
+		grep -q "FREE_SPACE_TREE"
+	then
+		_notrun "This test requires a free-space-tree"
+	fi
+}
+
 _check_btrfs_filesystem()
 {
 	device=$1
Signature · 1 line
2.39.3
author
#12 lore raw

PATCH v7 6/10 btrfs: add fstest for stripe-tree metadata with 4k write

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

Test a simple 4k write on all RAID profiles currently supported with the raid-stripe-tree.

[Fixed the test statement and trailing white space in the .out file.]
---
 tests/btrfs/304     | 59 +++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 117 insertions(+)
 create mode 100755 tests/btrfs/304
 create mode 100644 tests/btrfs/304.out
tests/btrfs/304 new file +59 -0
new file mode 100755
index 000000000000..186fa1646e48
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation.  All Rights Reserved.
+#
+# FS QA Test 304
+#
+# Test on-disk layout of RAID Stripe Tree Metadata writing 4k to a new file on
+# a pristine file system.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test $(_get_page_size) -eq 4096 || _notrun "this tests requires 4k pagesize"
+
+test_4k_write()
+{
+	local profile=$1
+	local ndevs=$2
+
+	_scratch_dev_pool_get $ndevs
+
+	echo "==== Testing $profile ===="
+	_scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+	_scratch_mount
+
+	$XFS_IO_PROG -fc "pwrite 0 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+	_scratch_cycle_mount
+	md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+	_scratch_unmount
+
+	$BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+		_filter_trailing_whitespace |\
+		_filter_btrfs_version |  _filter_stripe_tree
+
+	_scratch_dev_pool_put
+}
+
+echo "= Test basic 4k write ="
+test_4k_write raid0 2
+test_4k_write raid1 2
+test_4k_write raid10 4
+
+# success, all done
+status=0
+exit
tests/btrfs/304.out new file +58 -0

58 lines not on this page · raw message

Signature · 1 line
2.39.3
author
#13 lore raw

PATCH v7 7/10 btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

Add a test-case writing 8k to a raid-stripe-tree formatted filesystem with one stripe pre-filled to 60k so the 8k are split into a 4k write finishing stripe 1 and a 4k write starting the next stripe.

[Fixed the test statement and trailing white space in the .out file.]
---
 tests/btrfs/305     | 64 +++++++++++++++++++++++++++++++++++
 tests/btrfs/305.out | 82 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 146 insertions(+)
 create mode 100755 tests/btrfs/305
 create mode 100644 tests/btrfs/305.out
tests/btrfs/305 new file +64 -0
new file mode 100755
index 000000000000..f3bf0faa414c
@@ -0,0 +1,64 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation.  All Rights Reserved.
+#
+# FS QA Test 305
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 8k to a new file
+# with a filesystem prepropulated, so that 4k of the write are written to the
+# 1st stripe and 4k start a new stripe.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test $(_get_page_size) -eq 4096 || _notrun "this tests requires 4k pagesize"
+
+test_8k_new_stripe()
+{
+	local profile=$1
+	local ndevs=$2
+
+	_scratch_dev_pool_get $ndevs
+
+	echo "==== Testing $profile ===="
+	_scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+	_scratch_mount
+
+	# Fill the first stripe up to 64k - 4k
+	$XFS_IO_PROG -fc "pwrite 0 60k" -c fsync "$SCRATCH_MNT/bar" | _filter_xfs_io
+
+	# The actual 8k write
+	$XFS_IO_PROG -fc "pwrite 0 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+	_scratch_cycle_mount
+	md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+	_scratch_unmount
+
+	$BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+		_filter_trailing_whitespace |\
+		_filter_btrfs_version | _filter_stripe_tree
+
+	_scratch_dev_pool_put
+}
+
+echo "= Test 8k write to a new file so that 4k start a new stripe ="
+test_8k_new_stripe raid0 2
+test_8k_new_stripe raid1 2
+test_8k_new_stripe raid10 4
+
+# success, all done
+status=0
+exit
tests/btrfs/305.out new file +82 -0

82 lines not on this page · raw message

Signature · 1 line
2.39.3
author
#14 lore raw

PATCH v7 8/10 btrfs: add fstest for writing to a file at an offset with RST

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

Add a fstest writing 4k at offset 64k to a file with one RAID tripe already pre-filled for a raid-stripe-tree formatted file system.

[Fixed the test statement and trailing white space in the .out file.]
---
 tests/btrfs/306     | 62 +++++++++++++++++++++++++++++++++++++
 tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+)
 create mode 100755 tests/btrfs/306
 create mode 100644 tests/btrfs/306.out
tests/btrfs/306 new file +62 -0
new file mode 100755
index 000000000000..e2a9f804ac8b
@@ -0,0 +1,62 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation.  All Rights Reserved.
+#
+# FS QA Test 306
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 4k to an emppty
+# file at offset 64k with one stripe pre-filled on an otherwise pristine
+# filesystem.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test $(_get_page_size) -eq 4096 || _notrun "this tests requires 4k pagesize"
+
+test_4k_write_64koff()
+{
+	local profile=$1
+	local ndevs=$2
+
+	_scratch_dev_pool_get $ndevs
+
+	echo "==== Testing $profile ===="
+	_scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+	_scratch_mount
+
+	# precondition one stripe
+	$XFS_IO_PROG -fc "pwrite 0 64k" "$SCRATCH_MNT/bar" | _filter_xfs_io
+
+	$XFS_IO_PROG -fc "pwrite 64k 4k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+	_scratch_cycle_mount
+	md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+	_scratch_unmount
+
+	$BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+		_filter_trailing_whitespace |\
+		_filter_btrfs_version | _filter_stripe_tree
+
+	_scratch_dev_pool_put
+}
+echo "= Test 4k write to an empty file at offset 64k with one stripe prefilled ="
+test_4k_write_64koff raid0 2
+test_4k_write_64koff raid1 2
+test_4k_write_64koff raid10 4
+
+# success, all done
+status=0
+exit
tests/btrfs/306.out new file +75 -0

75 lines not on this page · raw message

Signature · 1 line
2.39.3
author
#15 lore raw

PATCH v7 9/10 btrfs: add fstests to write 128k to a RST filesystem

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

Add a test writing 128k to a file on an empty filesystem formatted with a raid-stripe-tree.

[Fixed the test statement and trailing white space in the .out file.]
---
 tests/btrfs/307     | 59 ++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/307.out | 65 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+)
 create mode 100755 tests/btrfs/307
 create mode 100644 tests/btrfs/307.out
tests/btrfs/307 new file +59 -0
new file mode 100755
index 000000000000..30656bcf0d96
@@ -0,0 +1,59 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation.  All Rights Reserved.
+#
+# FS QA Test 307
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to a new
+# file on a pristine filesystem
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+
+test $(_get_page_size) -eq 4096 || _notrun "this tests requires 4k pagesize"
+
+test_128k_write()
+{
+	local profile=$1
+	local ndevs=$2
+
+	_scratch_dev_pool_get $ndevs
+
+	echo "==== Testing $profile ===="
+	_scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+	_scratch_mount
+
+	$XFS_IO_PROG -fc "pwrite 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+	_scratch_cycle_mount
+	md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+	_scratch_unmount
+
+	$BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+		_filter_trailing_whitespace |\
+		_filter_btrfs_version | _filter_stripe_tree
+
+	_scratch_dev_pool_put
+}
+
+echo "= Test 128k write to empty file  ="
+test_128k_write raid0 2
+test_128k_write raid1 2
+test_128k_write raid10 4
+
+# success, all done
+status=0
+exit
tests/btrfs/307.out new file +65 -0

65 lines not on this page · raw message

Signature · 1 line
2.39.3
author
#16 lore raw

PATCH v7 10/10 btrfs: add fstest for overwriting a file partially with RST

In reply to Anand Jain

From: Johannes Thumshirn <[email protected]>

Add a test writing 128k to an empty file with one stripe already pre-filled on-disk. Then overwrite a portion of the file in the middle.

[Fixed the test statement and trailing white space in the .out file.]
---
 tests/btrfs/308     |  63 ++++++++++++++++++++++++++
 tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 169 insertions(+)
 create mode 100755 tests/btrfs/308
 create mode 100644 tests/btrfs/308.out
tests/btrfs/308 new file +63 -0
new file mode 100755
index 000000000000..7d651b2b4c9c
@@ -0,0 +1,63 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Western Digital Cooperation.  All Rights Reserved.
+#
+# FS QA Test 308
+#
+# Test on-disk layout of RAID Stripe Tree Metadata by writing 128k to an empty
+# file on a filesystem that has one stripe already pre-filled. Afterwards
+# overwrite a portion of the file.
+#
+. ./common/preamble
+_begin_fstest auto quick raid remount volume raid-stripe-tree
+
+. ./common/filter
+. ./common/filter.btrfs
+
+_supported_fs btrfs
+_require_btrfs_command inspect-internal dump-tree
+_require_btrfs_mkfs_feature "raid-stripe-tree"
+_require_scratch_dev_pool 4
+_require_btrfs_fs_feature "raid_stripe_tree"
+_require_btrfs_fs_feature "free_space_tree"
+_require_btrfs_free_space_tree
+_require_btrfs_no_compress
+_require_btrfs_no_nodatacow
+
+test $(_get_page_size) -eq 4096 || _notrun "this tests requires 4k pagesize"
+
+test_128k_write_overwrite()
+{
+	local profile=$1
+	local ndevs=$2
+
+	_scratch_dev_pool_get $ndevs
+
+	echo "==== Testing $profile ===="
+	_scratch_pool_mkfs -d $profile -m $profile -O raid-stripe-tree
+	_scratch_mount
+
+	$XFS_IO_PROG -fc "pwrite -W 0 32k" "$SCRATCH_MNT/bar" | _filter_xfs_io
+	$XFS_IO_PROG -fc "pwrite -W 0 128k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+	$XFS_IO_PROG -fc "pwrite -W 64k 8k" "$SCRATCH_MNT/foo" | _filter_xfs_io
+
+	_scratch_cycle_mount
+	md5sum "$SCRATCH_MNT/foo" | _filter_scratch
+
+	_scratch_unmount
+
+	$BTRFS_UTIL_PROG inspect-internal dump-tree -t raid_stripe $SCRATCH_DEV_POOL |\
+		_filter_trailing_whitespace |\
+		_filter_btrfs_version | _filter_stripe_tree
+
+	_scratch_dev_pool_put
+}
+
+echo "= Test 128k write to empty file with 1st stripe partially prefilled then overwrite ="
+test_128k_write_overwrite raid0 2
+test_128k_write_overwrite raid1 2
+test_128k_write_overwrite raid10 4
+
+# success, all done
+status=0
+exit
tests/btrfs/308.out new file +106 -0

106 lines not on this page · raw message

Signature · 1 line
2.39.3
author
#17 lore raw
In reply to Anand Jain
Quoted 5 lines · Anand Jain

Changes in v7:
- Fixed trailing whitespace in the .out files
- Fixed the following test statement in 30[4-8]:

      test _get_page_size -eq 4096

- Link to v6: https://lore.kernel.org/r/[email protected]

- Rebased on Zorro's for-next
Staged at https://github.com/asj/fstests/tree/ext-rst

Quoted 79 lines

--- original cover page from Johannes ----
Add tests for btrfs' raid-stripe-tree feature. All of these test work by writing a specific pattern to a newly created filesystem and afterwards using btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL to verify the placement and the layout of the metadata.

The md5sum of each file will be compared as well after a re-mount of the filesystem.

---
Changes in v6:
- require 4k pagesize for all tests as output depends on page size
- Add Filipe's Reviewed-by
- Link to v5: https://lore.kernel.org/r/[email protected]

Changes in v5:
- add _require_btrfs_free_space_tree helper and use in tests
- Link to v4: https://lore.kernel.org/r/[email protected]

Changes in v4:
- add _require_btrfs_no_compress to all tests
- add _require_btrfs_no_nodatacow helper and add to btrfs/308
- add _require_btrfs_feature "free_space_tree" to all tests
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it automatically
- Rename test cases as btrfs/302 and btrfs/303 already exist upstream
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Re-ordered series so the newly introduced group is added before the tests
- Changes Filipe requested to the tests.
- Link to v1: https://lore.kernel.org/r/[email protected]

Anand Jain (1):
common: add _filter_trailing_whitespace

Johannes Thumshirn (9):
fstests: doc: add new raid-stripe-tree group
common: add filter for btrfs raid-stripe dump
common: add _require_btrfs_no_nodatacow helper
common: add _require_btrfs_free_space_tree
btrfs: add fstest for stripe-tree metadata with 4k write
btrfs: add fstest for 8k write spanning two stripes on
raid-stripe-tree
btrfs: add fstest for writing to a file at an offset with RST
btrfs: add fstests to write 128k to a RST filesystem
btrfs: add fstest for overwriting a file partially with RST

  common/btrfs        |  17 +++++++
  common/filter       |   5 +++
  common/filter.btrfs |  14 ++++++
  doc/group-names.txt |   1 +
  tests/btrfs/304     |  59 ++++++++++++++++++++++++
  tests/btrfs/304.out |  58 ++++++++++++++++++++++++
  tests/btrfs/305     |  64 ++++++++++++++++++++++++++
  tests/btrfs/305.out |  82 ++++++++++++++++++++++++++++++++++
  tests/btrfs/306     |  62 ++++++++++++++++++++++++++
  tests/btrfs/306.out |  75 +++++++++++++++++++++++++++++++
  tests/btrfs/307     |  59 ++++++++++++++++++++++++
  tests/btrfs/307.out |  65 +++++++++++++++++++++++++++
  tests/btrfs/308     |  63 ++++++++++++++++++++++++++
  tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++
  14 files changed, 730 insertions(+)

create mode 100755 tests/btrfs/304
create mode 100644 tests/btrfs/304.out
create mode 100755 tests/btrfs/305
create mode 100644 tests/btrfs/305.out
create mode 100755 tests/btrfs/306
create mode 100644 tests/btrfs/306.out
create mode 100755 tests/btrfs/307
create mode 100644 tests/btrfs/307.out
create mode 100755 tests/btrfs/308
create mode 100644 tests/btrfs/308.out

#7 lore raw

Re: [PATCH 05/10] common: add _filter_trailing_whitespace

In reply to Anand Jain
Quoted 3 lines · Anand Jain

pls ignore this sole patch.

How can it be ignored if other patches in this patchset that introduce tests use the filter introduced by this patch?

Thanks.

#8 lore raw

Re: [PATCH 05/10] common: add _filter_trailing_whitespace

In reply to Anand Jain
Quoted 15 lines · Anand Jain

The command 'btrfs inspect-internal dump-tree -t raid_stripe' introduces trailing whitespace in its output.
Apply a filter to remove it. Used in btrfs/30[4-8][.out].

---
 common/filter | 5 +++++
 1 file changed, 5 insertions(+)
common/filter +0 -0
index 509ee95039ac..016d213b8bee 100644

common/filter _filter_bash() · line 657

Quoted 7 lines
common/filter +4 -0
@@ -651,5 +651,10 @@ _filter_bash()
        sed -e "s/^bash: line 1: /bash: /"
 }
 
+_filter_trailing_whitespace()
+{
+       sed -e "s/ $//"
+}

If we're having such a generic filter in common file, than I'd rather have it delete any number of trailing white spaces, not just a single one, and also account for tabs and other white spaces, so:

sed -e "s/\s+$//"

Also, since this is so specific to the raid stripe tree, I'd rather have this filter included in the raid stripe tree filter introduced in patch 2, _filter_stripe_tree(). That would make the tests shorter and cleaner by avoiding piping yet over another filter that is used only for the raid stripe tree dump...

Thanks.

Quoted 7 lines
common/filter +1 -1
+
 # make sure this script returns success
 /bin/true
--

2.39.3

author
#9 lore raw

Re: [PATCH 05/10] common: add _filter_trailing_whitespace

Quoted 28 lines · Filipe Manana
Quoted 15 lines · Anand Jain

The command 'btrfs inspect-internal dump-tree -t raid_stripe' introduces trailing whitespace in its output.
Apply a filter to remove it. Used in btrfs/30[4-8][.out].

---
  common/filter | 5 +++++
  1 file changed, 5 insertions(+)
common/filter +0 -0
index 509ee95039ac..016d213b8bee 100644

common/filter _filter_bash() · line 657

Quoted 7 lines
common/filter +4 -0
@@ -651,5 +651,10 @@ _filter_bash()
         sed -e "s/^bash: line 1: /bash: /"
  }
 
+_filter_trailing_whitespace()
+{
+       sed -e "s/ $//"
+}

If we're having such a generic filter in common file, than I'd rather have it delete any number of trailing white spaces, not just a single one, and also account for tabs and other white spaces, so:

Quoted 2 lines

sed -e "s/\s+$//"

I'll amend.

Quoted 5 lines

Also, since this is so specific to the raid stripe tree, I'd rather have this filter included in the raid stripe tree filter introduced in patch 2, _filter_stripe_tree(). That would make the tests shorter and cleaner by avoiding piping yet over another filter that is used only for the raid stripe tree dump...

I kept this as a separate function so that it can be used elsewhere when needed. Doesn't that make sense?

Thanks, Anand

#10 lore raw

Re: [PATCH 05/10] common: add _filter_trailing_whitespace

Quoted 38 lines · Anand Jain
Quoted 28 lines · Filipe Manana
Quoted 15 lines · Anand Jain

The command 'btrfs inspect-internal dump-tree -t raid_stripe' introduces trailing whitespace in its output.
Apply a filter to remove it. Used in btrfs/30[4-8][.out].

---
  common/filter | 5 +++++
  1 file changed, 5 insertions(+)
common/filter +0 -0
index 509ee95039ac..016d213b8bee 100644

common/filter _filter_bash() · line 657

Quoted 7 lines
common/filter +4 -0
@@ -651,5 +651,10 @@ _filter_bash()
         sed -e "s/^bash: line 1: /bash: /"
  }
 
+_filter_trailing_whitespace()
+{
+       sed -e "s/ $//"
+}

If we're having such a generic filter in common file, than I'd rather have it delete any number of trailing white spaces, not just a single one, and also account for tabs and other white spaces, so:

Quoted 2 lines

sed -e "s/\s+$//"

I'll amend.

Quoted 8 lines
Quoted 5 lines

Also, since this is so specific to the raid stripe tree, I'd rather have this filter included in the raid stripe tree filter introduced in patch 2, _filter_stripe_tree(). That would make the tests shorter and cleaner by avoiding piping yet over another filter that is used only for the raid stripe tree dump...

I kept this as a separate function so that it can be used elsewhere when needed. Doesn't that make sense?

Not so much if there's only one use case for it... specially if it's such a trivial filter...

Even if we had multiple cases, doing this pattern in the tests:

$BTRFS_UTIL_PROG inspect-internal dump-tree (... ) | _filter_trailing_whitespace | _filter_btrfs_version | _filter_stripe_tree

Is ugly and verbose. The filtering could be done in _filter_stripe_tree() by calling "_filter_triling_whitespace" there... And mentioning that, we could also call _filter_btrfs_version there, since it's always wanted and to make tests shorter and easier to read.

So in the end it would only be

$BTRFS_UTIL_PROG inspect-internal dump-tree (... ) | _filter_stripe_tree

With _filter_stripe_tree() as:

_filter_stripe_tree()
{
_filter_trailing_whitespace | _filter_btrfs_version | sed -E -e (....)
}

Or:

_filter_stripe_tree()

{
    _filter_btrfs_version | sed -E -e "s/\s+$//" -e (...)
}

A lot more clean.

Thanks.

Quoted 2 lines

Thanks, Anand

author
#11 lore raw

Re: [PATCH 05/10] common: add _filter_trailing_whitespace

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Also, since this is so specific to the raid stripe tree, I'd rather have this filter included in the raid stripe tree filter introduced in patch 2, _filter_stripe_tree(). That would make the tests shorter and cleaner by avoiding piping yet over another filter that is used only for the raid stripe tree dump...

Quoted 6 lines
Quoted 2 lines

I kept this as a separate function so that it can be used elsewhere when needed. Doesn't that make sense?

Not so much if there's only one use case for it... specially if it's such a trivial filter...

Hmm. Yes. Also, a completely avoidable coding nitpick in btrfs-progs.

Quoted 11 lines

Even if we had multiple cases, doing this pattern in the tests:

$BTRFS_UTIL_PROG inspect-internal dump-tree (... ) | _filter_trailing_whitespace | _filter_btrfs_version | _filter_stripe_tree

Is ugly and verbose. The filtering could be done in _filter_stripe_tree() by calling "_filter_triling_whitespace" there... And mentioning that, we could also call _filter_btrfs_version there, since it's always wanted and to make tests shorter and easier to read.

Yeah.

Quoted 11 lines

So in the end it would only be

$BTRFS_UTIL_PROG inspect-internal dump-tree (... ) | _filter_stripe_tree

With _filter_stripe_tree() as:

_filter_stripe_tree()
{
_filter_trailing_whitespace | _filter_btrfs_version | sed -E -e (....)
}

Quoted 9 lines

Or:

_filter_stripe_tree()

{
     _filter_btrfs_version | sed -E -e "s/\s+$//" -e (...)
}

A lot more clean.

I'm fine with either way. Since there is a choice here, I will keep
the former.

I'll send a reroll.

Thanks, Anand