generic: add a test to ensure that page is properly filled before write
PATCH v3 1/1 generic: add a test to ensure that page is properly filled before write
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.out
v3: adapt to new test template
tests/generic/639 new file +39 -0
@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions. +. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile + +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exit
tests/generic/639.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040
Signature · 1 line
2.31.1
Quoted 38 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.outv3: adapt to new test template
tests/generic/639 new file +16
-0@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions.
tests/generic/639 line 26
Quoted 10 lines
tests/generic/639 new file +10
-0+. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile
Looks good to me, if you've used this case to reproduce that issue.
Just one question, the comment says "fsync it out", does that mean we need an explicit "fsync" call (e.g. -c "pwrite -q 0 32" -c "fsync") to reproduce this bug? Or there's an implicit "fsync" somewhere?
Thanks,
Zorro
Quoted 27 lines
tests/generic/639 new file +13
-0+ +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/639.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040Signature · 1 line
2.31.1
Quoted 52 lines · Zorro Lang
Quoted 38 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.outv3: adapt to new test template
tests/generic/639 new file +16
-0@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions.tests/generic/639 line 26
Quoted 10 lines
tests/generic/639 new file +10
-0+. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfileLooks good to me, if you've used this case to reproduce that issue.
Quoted 3 lines
Just one question, the comment says "fsync it out", does that mean we need an explicit "fsync" call (e.g. -c "pwrite -q 0 32" -c "fsync") to reproduce this bug? Or there's an implicit "fsync" somewhere?
Oh, this patch has just been merged, sorry for my late review. You can ignore this message.
Thanks,
Zorro
Quoted 31 lines
Thanks,
ZorroQuoted 27 lines
tests/generic/639 new file +13
-0+ +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/639.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040Signature · 1 line
2.31.1
Quoted 52 lines · Zorro Lang
Quoted 38 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.outv3: adapt to new test template
tests/generic/639 new file +16
-0@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions.tests/generic/639 line 26
Quoted 10 lines
tests/generic/639 new file +10
-0+. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfileLooks good to me, if you've used this case to reproduce that issue.
Quoted 5 lines
Just one question, the comment says "fsync it out", does that mean we need an explicit "fsync" call (e.g. -c "pwrite -q 0 32" -c "fsync") to reproduce this bug? Or there's an implicit "fsync" somewhere?
No, there is no implicit fsync. This is holdover from copy/paste that I did from another test. The patch is already merged now, but if we expand this test to cover other cases, I'll plan to clean up the comment then.
Thanks,
Jeff
Quoted 29 lines
Quoted 28 lines
tests/generic/639 new file +13
-0+ +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/639.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040Signature · 1 line
2.31.1
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 63 lines · Jeff Layton
Quoted 52 lines · Zorro Lang
Quoted 38 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.outv3: adapt to new test template
tests/generic/639 new file +16
-0@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions.tests/generic/639 line 26
Quoted 10 lines
tests/generic/639 new file +10
-0+. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfileLooks good to me, if you've used this case to reproduce that issue.
Quoted 5 lines
Just one question, the comment says "fsync it out", does that mean we need an explicit "fsync" call (e.g. -c "pwrite -q 0 32" -c "fsync") to reproduce this bug? Or there's an implicit "fsync" somewhere?
No, there is no implicit fsync. This is holdover from copy/paste that I did from another test. The patch is already merged now, but if we expand this test to cover other cases, I'll plan to clean up the comment then.
Quoted 7 lines
Thanks,
JeffQuoted 4 lines
tests/generic/639 line 29
Quoted 4 lines
tests/generic/639 new file +3
-0+ +# cycle the mount to clean out the pagecache +_test_cycle_mount
I think this cycle mount accounts as an implicit fsync?
Thanks,
Eryu
Quoted 28 lines
Quoted 25 lines
Quoted 24 lines
tests/generic/639 new file +10
-0+ +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/639.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040Signature · 1 line
2.31.1Signature · 1 line
Jeff Layton <[email protected]>
Quoted 68 lines · Eryu Guan
Quoted 67 lines · Jeff Layton
Quoted 52 lines · Zorro Lang
Quoted 38 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/639 | 39 +++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 +++++ 2 files changed, 44 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.outv3: adapt to new test template
tests/generic/639 new file +16
-0@@ -0,0 +1,39 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +. ./common/preamble +_begin_fstest auto quick rw + +# Import common functions.tests/generic/639 line 26
Quoted 10 lines
tests/generic/639 new file +10
-0+. ./common/filter + +# real QA test starts here +_supported_fs generic +_require_test + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfileLooks good to me, if you've used this case to reproduce that issue.
Quoted 5 lines
Just one question, the comment says "fsync it out", does that mean we need an explicit "fsync" call (e.g. -c "pwrite -q 0 32" -c "fsync") to reproduce this bug? Or there's an implicit "fsync" somewhere?
No, there is no implicit fsync. This is holdover from copy/paste that I did from another test. The patch is already merged now, but if we expand this test to cover other cases, I'll plan to clean up the comment then.
Thanks,
JeffQuoted 1 line
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
tests/generic/639 line 29
Quoted 3 lines
tests/generic/639 new file +3
-0+ +# cycle the mount to clean out the pagecache +_test_cycle_mountI think this cycle mount accounts as an implicit fsync?
Yes, the data should get synced out naturally during a umount. The comment is still a bit misleading though since it may not happen until this point.
Quoted 28 lines
Quoted 28 lines
Quoted 25 lines
Quoted 24 lines
tests/generic/639 new file +10
-0+ +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/639.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040Signature · 1 line
2.31.1Signature · 1 line
Jeff Layton <[email protected]>
Signature · 1 line
Jeff Layton <[email protected]>