generic: add a test to ensure that page is properly filled before write
PATCH 1/1 generic: add a test to ensure that page is properly filled before write
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
--- tests/generic/XXX | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/XXX.out | 5 ++++ tests/generic/group | 1 + 3 files changed, 59 insertions(+) create mode 100755 tests/generic/XXX create mode 100644 tests/generic/XXX.out
tests/generic/XXX new file +53 -0
@@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. XXX +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. Prompted by a bug in ceph_write_begin. See: +# +# https://lore.kernel.org/ceph-devel/[email protected]/T/#m9f34b328623b493c529505e12f64f4dfdbddfb27 +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + + +_cleanup() +{ + cd / + rm -f $testfile +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +_supported_fs generic +_require_test + +rm -f $seqres.full + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile + +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exit
tests/generic/XXX.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by XXX +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040
tests/generic/group +1 -0
@@ -641,3 +641,4 @@ 636 auto quick swap 637 auto quick dir 638 auto quick rw +XXX auto quick rw
Signature · 1 line
2.31.1
Quoted 10 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/XXX | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/XXX.out | 5 ++++
Ah, all the 'XXX' in this test should be replaced with proper seq number.
Quoted 12 lines
tests/generic/group | 1 + 3 files changed, 59 insertions(+) create mode 100755 tests/generic/XXX create mode 100644 tests/generic/XXX.outtests/generic/XXX new file +1
-0@@ -0,0 +1,53 @@ +#! /bin/bash
tests/generic/XXX line 11
Quoted 10 lines
tests/generic/XXX new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. XXX +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. Prompted by a bug in ceph_write_begin. See: +# +# https://lore.kernel.org/ceph-devel/[email protected]/T/#m9f34b328623b493c529505e12f64f4dfdbddfb27
This looks like a thread 9 months ago. Was the fix merged? If so, it'd be better to reference the kernel commit here.
Otherwise test looks fine to me.
Thanks,
Eryu
Quoted 65 lines
tests/generic/XXX new file +42
-0+# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + + +_cleanup() +{ + cd / + rm -f $testfile +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +_supported_fs generic +_require_test + +rm -f $seqres.full + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile + +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/XXX.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by XXX +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040tests/generic/group +1
-0@@ -641,3 +641,4 @@ 636 auto quick swap 637 auto quick dir 638 auto quick rw +XXX auto quick rwSignature · 1 line
2.31.1
Quoted 7 lines · Eryu Guan
Quoted 6 lines · Jeff Layton
We had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
Quoted 8 lines
Quoted 4 lines
Signed-off-by Jeff Layton <[email protected]>--- tests/generic/XXX | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/XXX.out | 5 ++++Ah, all the 'XXX' in this test should be replaced with proper seq number.
I can do that and resend.
Quoted 26 lines
Quoted 12 lines
tests/generic/group | 1 + 3 files changed, 59 insertions(+) create mode 100755 tests/generic/XXX create mode 100644 tests/generic/XXX.outtests/generic/XXX new file +1
-0@@ -0,0 +1,53 @@ +#! /bin/bashtests/generic/XXX line 11
Quoted 10 lines
tests/generic/XXX new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. XXX +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. Prompted by a bug in ceph_write_begin. See: +# +# https://lore.kernel.org/ceph-devel/[email protected]/T/#m9f34b328623b493c529505e12f64f4dfdbddfb27This looks like a thread 9 months ago. Was the fix merged? If so, it'd be better to reference the kernel commit here.
Quoted 2 lines
Otherwise test looks fine to me.
Not yet. v5.13 will require a fix in the netfs lib, whereas ceph will need a one-off fix in v5.10.z - v5.12.z. We're sorting out both fixes now.
Thanks!
Quoted 66 lines
Quoted 66 lines
tests/generic/XXX new file +42
-0+# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + + +_cleanup() +{ + cd / + rm -f $testfile +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +_supported_fs generic +_require_test + +rm -f $seqres.full + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile + +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exittests/generic/XXX.out new file +5
-0@@ -0,0 +1,5 @@ +QA output created by XXX +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040tests/generic/group +1
-0@@ -641,3 +641,4 @@ 636 auto quick swap 637 auto quick dir 638 auto quick rw +XXX auto quick rwSignature · 1 line
2.31.1
Signature · 1 line
Jeff Layton <[email protected]>
PATCH v2 1/1 generic: add a test to ensure that page is properly filled before write
In reply to Jeff LaytonWe had a broken optimization in cephfs and netfs lib that could cause part of a page to be improperly zeroed-out when writing to an offset that was beyond the EOF but in an existing page.
Add a simple test that would have caught this.
--- tests/generic/639 | 53 +++++++++++++++++++++++++++++++++++++++++++ tests/generic/639.out | 5 ++++ tests/generic/group | 1 + 3 files changed, 59 insertions(+) create mode 100755 tests/generic/639 create mode 100644 tests/generic/639.out
The kernel patch for this finally got merged, so we can refer to it in the comments now. Also go ahead and set this test to be #639 instead of XXX.
tests/generic/639 new file +53 -0
@@ -0,0 +1,53 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021, Jeff Layton <[email protected]> +# +# FS QA Test No. 639 +# +# Open a file and write a little data to it. Unmount (to clean out the cache) +# and then mount again. Then write some data to it beyond the EOF and ensure +# the result is correct. +# +# Prompted by a bug in ceph_write_begin that was fixed by commit 827a746f405d. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + + +_cleanup() +{ + cd / + rm -f $testfile +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +_supported_fs generic +_require_test + +rm -f $seqres.full + +testfile="$TEST_DIR/test_write_begin.$$" + +# write some data to file and fsync it out +$XFS_IO_PROG -f -c "pwrite -q 0 32" $testfile + +# cycle the mount to clean out the pagecache +_test_cycle_mount + +# now, write to the file (near the end) +$XFS_IO_PROG -c "pwrite -q 32 32" $testfile + +# dump what we think is in there +echo "The result should be 64 bytes filled with 0xcd:" +hexdump -C $testfile + +status=0 +exit
tests/generic/639.out new file +5 -0
@@ -0,0 +1,5 @@ +QA output created by 639 +The result should be 64 bytes filled with 0xcd: +00000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd |................| +* +00000040
tests/generic/group +1 -0
@@ -641,3 +641,4 @@ 636 auto quick swap 637 auto quick dir 638 auto quick rw +639 auto quick rw
Signature · 1 line
2.31.1