Skip to contentSkip to conversation

fstests: generic/352 should accomodate other pwrite behaviors

PATCH Superseded Bill O'Donnell 4 participants 12 messages Aug 23, 2023 – Aug 24, 2023

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fstests: generic/352 should accomodate other pwrite behaviors

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +11 -5
index 52ec4850..c4ee8a44 100755
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out
 done
 
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full
 
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2
 
 # success, all done
 status=0
tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last
Signature · 1 line
2.41.0
#1 lore raw
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
 done
 
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

tests/generic/352.out line 5

Quoted 13 lines
tests/generic/352 +0 -0
 # success, all done
 status=0
tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#2 lore raw
Quoted 29 lines · Darrick J. Wong
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

Quoted 2 lines
tests/generic/352 +0 -0
 done
 
Quoted 8 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 6 lines
tests/generic/352 +2 -2
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

Do $tmp1 and $tmp2 actually need to be declared?

Quoted 16 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 11 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

Quoted 6 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

But how to [diff $tmp.out1 $tmp.out2]? I don't see that working.

Quoted 18 lines

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

Quoted 1 line
Quoted 16 lines

tests/generic/352.out line 5

Quoted 12 lines

# success, all done
status=0

tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

That's my interpretation as well.

Thanks-
Bill

Quoted 6 lines

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#3 lore raw
Quoted 57 lines · Bill O'Donnell
Quoted 29 lines · Darrick J. Wong
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

Quoted 2 lines
tests/generic/352 +0 -0
 done
 
Quoted 8 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 6 lines
tests/generic/352 +2 -2
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

Do $tmp1 and $tmp2 actually need to be declared?

Quoted 16 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 11 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

Quoted 8 lines
Quoted 6 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

But how to [diff $tmp.out1 $tmp.out2]? I don't see that working.

Oh, I suppose this would work:
diff <(cat $tmp.out1) <(cat $tmp.out2)

Quoted 49 lines
Quoted 19 lines

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

Quoted 1 line
Quoted 16 lines

tests/generic/352.out line 5

Quoted 12 lines

# success, all done
status=0

tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

That's my interpretation as well.

Thanks-
Bill

Quoted 6 lines

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#4 lore raw
Quoted 65 lines · Bill O'Donnell
Quoted 64 lines · Bill O'Donnell
Quoted 29 lines · Darrick J. Wong
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

Quoted 2 lines
tests/generic/352 +0 -0
 done
 
Quoted 8 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 6 lines
tests/generic/352 +2 -2
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

Do $tmp1 and $tmp2 actually need to be declared?

Quoted 22 lines

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 11 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Quoted 4 lines
Quoted 1 line

But how to [diff $tmp.out1 $tmp.out2]? I don't see that working.

Oh, I suppose this would work:
diff <(cat $tmp.out1) <(cat $tmp.out2)

Wrong :/

Quoted 51 lines
Quoted 50 lines
Quoted 20 lines

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

Quoted 1 line
Quoted 16 lines

tests/generic/352.out line 5

Quoted 12 lines

# success, all done
status=0

tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

That's my interpretation as well.

Thanks-
Bill

Quoted 6 lines

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#5 lore raw
In reply to Darrick J. Wong
Quoted 60 lines · Darrick J. Wong
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
 done
 
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Quoted 4 lines

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

-Bill

Quoted 35 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

tests/generic/352.out line 5

Quoted 13 lines
tests/generic/352 +0 -0
 # success, all done
 status=0
tests/generic/352.out +0 -2
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#6 lore raw
Quoted 61 lines · Bill O'Donnell
Quoted 60 lines · Darrick J. Wong
Quoted 18 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous. This behavior is acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Modify test to accept any layout for the reflinked logical range.

---
 tests/generic/352     | 16 +++++++++++-----
 tests/generic/352.out |  2 --
 2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
 # use reflink to create the rest of the file, whose all extents are all
 # pointing to the first extent
 for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
 done
 
 # then call fiemap on that file to test both the shared flag and if
 # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
 # refact the $LOAD_FACTOR to 1 to match the golden output
 sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
 	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Quoted 7 lines
Quoted 4 lines

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

...or I guess I could have used the current email addr instead of the one on the commit. :(

Qu: Question for you:

Quoted 10 lines
Quoted 10 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

Any thoughts?

--D

Quoted 25 lines
Quoted 24 lines
Quoted 9 lines
tests/generic/352 +0 -0
 # success, all done
 status=0
tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644

tests/generic/352.out line 5

Quoted 5 lines
tests/generic/352.out +0 -2
@@ -1,5 +1,3 @@
 QA output created by 352
 wrote 131072/131072 bytes at offset 0
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

--D

Quoted 3 lines
Signature · 1 line
2.41.0
#7 lore raw
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Bill O'Donnell
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous.

However this doesn't make much difference, at least for btrfs.

Btrfs would do the merging emitting the fiemap entry, thus even if the write didn't result a singular extent, as long as they are contiguous (under most cases they are) the fiemap result would still be a single one.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This behavior is
acceptable, but the test is flawed in that it expects a single extent for a pwrite.

I'm more interested in if you're hitting any test failure?

Quoted 60 lines
Quoted 60 lines
Quoted 59 lines
Quoted 14 lines

Modify test to accept any layout for the reflinked logical range.

---
  tests/generic/352     | 16 +++++++++++-----
  tests/generic/352.out |  2 --
  2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
  # use reflink to create the rest of the file, whose all extents are all
  # pointing to the first extent
  for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
  done
 
  # then call fiemap on that file to test both the shared flag and if
  # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
  # refact the $LOAD_FACTOR to 1 to match the golden output
  sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
  	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Quoted 7 lines
Quoted 2 lines

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

...or I guess I could have used the current email addr instead of the one on the commit. :(

Qu: Question for you:

Thanks a lot for referring it to me.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

It's quite sometime ago, thus my memory may not be reliable, but IIRC the blocksize has no specific requirement other than allowing all possible blocksize (4K to 64K).

And at that time, at least I was preferring to use golden output to detect errors, thus I choose a larger blocksize to allow all blocksizes to work.

Quoted 23 lines

Any thoughts?

--D

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
tests/generic/352 +0 -0
  # success, all done
  status=0
tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644

tests/generic/352.out line 5

Quoted 5 lines
tests/generic/352.out +0 -2
@@ -1,5 +1,3 @@
  QA output created by 352
  wrote 131072/131072 bytes at offset 0
  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

Could explain more on why the shared flag detection is not correct here?

If a file extent is shared, no matter if it's shared by another inode or not, shouldn't it be marked with SHARED flag?

Thanks,
Qu

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

--D

Quoted 3 lines

--
2.41.0

#8 lore raw
Quoted 15 lines · Qu Wenruo
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Bill O'Donnell
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous.

However this doesn't make much difference, at least for btrfs.

Btrfs would do the merging emitting the fiemap entry, thus even if the write didn't result a singular extent, as long as they are contiguous (under most cases they are) the fiemap result would still be a single one.

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This behavior is
acceptable, but the test is flawed in that it expects a single extent for a pwrite.

I'm more interested in if you're hitting any test failure?

Yes we are.

thanks-
Bill

Quoted 131 lines
Quoted 61 lines
Quoted 61 lines
Quoted 60 lines
Quoted 15 lines

Modify test to accept any layout for the reflinked logical range.

---
  tests/generic/352     | 16 +++++++++++-----
  tests/generic/352.out |  2 --
  2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
  # use reflink to create the rest of the file, whose all extents are all
  # pointing to the first extent
  for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
  done
 
  # then call fiemap on that file to test both the shared flag and if
  # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
  # refact the $LOAD_FACTOR to 1 to match the golden output
  sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
  	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Quoted 7 lines
Quoted 2 lines

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

...or I guess I could have used the current email addr instead of the one on the commit. :(

Qu: Question for you:

Thanks a lot for referring it to me.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

It's quite sometime ago, thus my memory may not be reliable, but IIRC the blocksize has no specific requirement other than allowing all possible blocksize (4K to 64K).

And at that time, at least I was preferring to use golden output to detect errors, thus I choose a larger blocksize to allow all blocksizes to work.

Quoted 23 lines

Any thoughts?

--D

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
tests/generic/352 +0 -0
  # success, all done
  status=0
tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644

tests/generic/352.out line 5

Quoted 5 lines
tests/generic/352.out +0 -2
@@ -1,5 +1,3 @@
  QA output created by 352
  wrote 131072/131072 bytes at offset 0
  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

Could explain more on why the shared flag detection is not correct here?

If a file extent is shared, no matter if it's shared by another inode or not, shouldn't it be marked with SHARED flag?

Thanks,
Qu

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

--D

Quoted 3 lines

--
2.41.0

#9 lore raw
Quoted 16 lines · Bill O'Donnell
Quoted 15 lines · Qu Wenruo
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Bill O'Donnell
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous.

However this doesn't make much difference, at least for btrfs.

Btrfs would do the merging emitting the fiemap entry, thus even if the write didn't result a singular extent, as long as they are contiguous (under most cases they are) the fiemap result would still be a single one.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This behavior is
acceptable, but the test is flawed in that it expects a single extent for a pwrite.

I'm more interested in if you're hitting any test failure?

Yes we are.

I guess it's using XFS?

Then the idea of using proper block size looks good to me.

And since we're getting rid of the golden output, it's also a good timing to get rid of the refactoring to match the golden output completely.

Thanks,
Qu

Quoted 137 lines

thanks-
Bill

Quoted 131 lines
Quoted 61 lines
Quoted 61 lines
Quoted 60 lines
Quoted 15 lines

Modify test to accept any layout for the reflinked logical range.

---
   tests/generic/352     | 16 +++++++++++-----
   tests/generic/352.out |  2 --
   2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
   # use reflink to create the rest of the file, whose all extents are all
   # pointing to the first extent
   for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
   done
 
   # then call fiemap on that file to test both the shared flag and if
   # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 65

Quoted 10 lines
tests/generic/352 +4 -2
   # refact the $LOAD_FACTOR to 1 to match the golden output
   sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
   	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 69

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Quoted 7 lines
Quoted 2 lines

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

...or I guess I could have used the current email addr instead of the one on the commit. :(

Qu: Question for you:

Thanks a lot for referring it to me.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

It's quite sometime ago, thus my memory may not be reliable, but IIRC the blocksize has no specific requirement other than allowing all possible blocksize (4K to 64K).

And at that time, at least I was preferring to use golden output to detect errors, thus I choose a larger blocksize to allow all blocksizes to work.

Quoted 23 lines

Any thoughts?

--D

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
tests/generic/352 +0 -0
   # success, all done
   status=0
tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644

tests/generic/352.out line 5

Quoted 5 lines
tests/generic/352.out +0 -2
@@ -1,5 +1,3 @@
   QA output created by 352
   wrote 131072/131072 bytes at offset 0
   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

Could explain more on why the shared flag detection is not correct here?

If a file extent is shared, no matter if it's shared by another inode or not, shouldn't it be marked with SHARED flag?

Thanks,
Qu

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

--D

Quoted 3 lines

--
2.41.0

#10 lore raw
Quoted 23 lines · Qu Wenruo
Quoted 20 lines · Bill O'Donnell
Quoted 19 lines · Qu Wenruo
Quoted 5 lines · Darrick J. Wong
Quoted 4 lines · Bill O'Donnell
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee that the resulting extent(s) will be singular or contiguous.

However this doesn't make much difference, at least for btrfs.

Btrfs would do the merging emitting the fiemap entry, thus even if the write didn't result a singular extent, as long as they are contiguous (under most cases they are) the fiemap result would still be a single one.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This behavior is
acceptable, but the test is flawed in that it expects a single extent for a pwrite.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I'm more interested in if you're hitting any test failure?

Yes we are.

I guess it's using XFS?

Yes. The problem begins with the initial pwrite, with unpredictable mapping. It's happening under the following configuration:

export TEST_DEV=/dev/pmem0
export TEST_DIR=/mnt/test
export SCRATCH_DEV=/dev/pmem0.1
export SCRATCH_MNT=/mnt/scratch
export MKFS_OPTIONS="-m reflink=1 -m rmapbt=0 -b size=4096" export MOUNT_OPTIONS="-o dax=always"
export TEST_FS_MOUNT_OPTS="-o dax=always"

-Bill

Quoted 146 lines

Then the idea of using proper block size looks good to me.

And since we're getting rid of the golden output, it's also a good timing to get rid of the refactoring to match the golden output completely.

Thanks,
Qu

Quoted 137 lines

thanks-
Bill

Quoted 131 lines
Quoted 61 lines
Quoted 61 lines
Quoted 60 lines
Quoted 15 lines

Modify test to accept any layout for the reflinked logical range.

---
   tests/generic/352     | 16 +++++++++++-----
   tests/generic/352.out |  2 --
   2 files changed, 11 insertions(+), 7 deletions(-)
tests/generic/352 +0 -0
index 52ec4850..c4ee8a44 100755

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 51

Quoted 5 lines
tests/generic/352 +1 -1
@@ -48,19 +48,25 @@ _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io
   # use reflink to create the rest of the file, whose all extents are all
   # pointing to the first extent
   for i in $(seq 1 $nr); do
-	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > /dev/null
+	_reflink_range $file 0 $file $(($i * $blocksize)) $blocksize > $tmp1.out

$tmp1 isnt defined anywhere.

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 57

Quoted 8 lines
tests/generic/352 +2 -2
   done
 
   # then call fiemap on that file to test both the shared flag and if
   # reserved extent mapping search will cause soft lockup
-$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp.out
-cat $tmp.out >> $seqres.full
+$XFS_IO_PROG -c "fiemap -v" $file | _filter_fiemap_flags > $tmp2.out
+cat $tmp2.out >> $seqres.full

Nor is $tmp2

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 64

Quoted 10 lines
tests/generic/352 +4 -2
   # refact the $LOAD_FACTOR to 1 to match the golden output
   sed -i -e "s/$(($last_extent - 1))/$(($orig_last_extent - 1))/" \
   	-e "s/$last_extent/$orig_last_extent/" \
-	-e "s/$end/$orig_end/" $tmp.out
-cat $tmp.out
+	-e "s/$end/$orig_end/" $tmp2.out
+
+cat $tmp1.out > tmp.1
+cat $tmp2.out > tmp.2

Not sure why you didn't make the _reflink_range and the fiemap above output to $tmp.out1 and $tmp.out2, respectively. If you had, then the default _cleanup would delete $tmp.* automatically...

tests/generic/352 _pwrite_byte 0xcdcdcdcd 0 $blocksize $file | _filter_xfs_io · line 68

Quoted 4 lines
tests/generic/352 +4 -0
+
+diff tmp.[12]
+rm tmp.1
+rm tmp.2

...and the rm here wouldn't be necessary.

Ok. Nitpicking over. Moving on to the weirder design questions of the original test:

[add original test author to cc]

Quoted 7 lines
Quoted 2 lines

Emails to [email protected] seem to be undeliverable. Maybe Joseph would know what btrfs intent was?

...or I guess I could have used the current email addr instead of the one on the commit. :(

Qu: Question for you:

Thanks a lot for referring it to me.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

I don't know why $blocksize is set to 128k above. If this test needs to guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

	blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

It's quite sometime ago, thus my memory may not be reliable, but IIRC the blocksize has no specific requirement other than allowing all possible blocksize (4K to 64K).

And at that time, at least I was preferring to use golden output to detect errors, thus I choose a larger blocksize to allow all blocksizes to work.

Quoted 23 lines

Any thoughts?

--D

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
tests/generic/352 +0 -0
   # success, all done
   status=0
tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644

tests/generic/352.out line 5

Quoted 5 lines
tests/generic/352.out +0 -2
@@ -1,5 +1,3 @@
   QA output created by 352
   wrote 131072/131072 bytes at offset 0
   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

Could explain more on why the shared flag detection is not correct here?

If a file extent is shared, no matter if it's shared by another inode or not, shouldn't it be marked with SHARED flag?

Thanks,
Qu

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

--D

Quoted 3 lines

--
2.41.0

#11 lore raw
In reply to Qu Wenruo
Quoted 10 lines · Qu Wenruo
Quoted 6 lines · Darrick J. Wong
Quoted 5 lines · Bill O'Donnell
Quoted 4 lines · Darrick J. Wong
Quoted 3 lines · Bill O'Donnell

xfs_io pwrite issues a series of block size writes, but there is no guarantee
that the resulting extent(s) will be singular or contiguous.

Quoted 5 lines

However this doesn't make much difference, at least for btrfs.

Btrfs would do the merging emitting the fiemap entry, thus even if the write didn't result a singular extent, as long as they are contiguous (under most cases they are) the fiemap result would still be a single one.

btrfs may merge physically adjacent extents in fiemap output, but if the write happened to create physically discontiguous extents for the logical range for the single pwrite, I think you would run into the same problem as Bill has described (golden output expects a single extent from fiemap), yes?

"under most cases" is the problem here, and results in a flakey test. :)

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This behavior is
acceptable, but the test is flawed in that it expects a single extent for a
pwrite.

I'm more interested in if you're hitting any test failure?

As Bill noted, yes - under some circumstances the pwrite on XFS creates physically discontiguous extents for the range. (Why it does so is also an interesting question, for later...)

<snip>

Quoted 17 lines
Quoted 1 line

Qu: Question for you:

Thanks a lot for referring it to me.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

I don't know why $blocksize is set to 128k above.  If this test needs to
guarantee that there would only be *one* extent (and the golden output implies this as you note), then it should have been written to say:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

    blocksize=$(_get_file_block_size $SCRATCH_MNT)

But I don't know if the "btrfs soft lock up and return wrong shared flag" behavior required sharing a (probably multi-block) 128k range, or if that was simply what the author selected because it reproduced the problem.

Quoted 7 lines

It's quite sometime ago, thus my memory may not be reliable, but IIRC the blocksize has no specific requirement other than allowing all possible blocksize (4K to 64K).

And at that time, at least I was preferring to use golden output to detect errors, thus I choose a larger blocksize to allow all blocksizes to work.

Ok, if it was simply "accomodate maximum blocksize" then I like Darrick's suggestion to simply get the actual filesystem block size, and use that.

Quoted 24 lines
Quoted 23 lines

Any thoughts?

--D

Quoted 18 lines
Quoted 18 lines
Quoted 9 lines

  # success, all done
  status=0

tests/generic/352.out +0 -0
index 4ff66c21..ad90ae0d 100644
@@ -1,5 +1,3 @@
Quoted 5 lines

  QA output created by 352
  wrote 131072/131072 bytes at offset 0
  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -0: [0..2097151]: shared
-1: [2097152..2097407]: shared|last

Also I suspect from the test description that the goal here was to detect the golden output failing because the shared flag does not get reported correctly.

Quoted 4 lines

Could explain more on why the shared flag detection is not correct here?

If a file extent is shared, no matter if it's shared by another inode or not, shouldn't it be marked with SHARED flag?

I think that what Darrick meant was that Bill's patch removed "shared" from the golden output, and if the entire point of the test was to detect proper sharing:

# This test case will check if reserved extent map searching go

# without problem and return correct SHARED flag.
# Which btrfs will soft lock up and return wrong shared flag.

then it was no longer testing as originally designed.

-Eric

Quoted 2 lines

Thanks,
Qu