tailtest: Add a test for the tail zeroing bug

This test tests tail zeroing by writing a byte at the start of a new file, then writing another byte at various offsets.

Currently, the test uses 4K/1M filesystems. The offsets are 10M, 1M, 512K, and 1K. This tests a new write location well past the existing extent, right after the existing extent, in the middle of the existing extent but past the last valid block of the file, and inside the last valid block of the file.

Each offset is tested twice. First with truncation out to the new location, then without truncation. This tests zeroing via truncate and zeroing via write.

The full set of tests is run on four different filesystem
configurations: sparse,inline-data sparse,noinline-data nosparse,inline-data and nosparse,noinline-data.

Signed-off-by Joel Becker <[email protected]>

---
 programs/tailtest/tailtest |  179 ++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 179 insertions(+), 0 deletions(-)
 create mode 100755 programs/tailtest/tailtest

programs/tailtest/tailtest new file +5 -0

new file mode 100755
index 0000000..5bf70f7
@@ -0,0 +1,179 @@
+#!/bin/bash
+#
+# Test for the ocfs2 tail zero bug
+#
+

programs/tailtest/tailtest new file +49 -0

+}
+trap "exitcode=\$?; cleanup 2>/dev/null && exit \$exitcode" 0
+trap "cleanup; die \"Signal caught, exiting\"" 1 2 13 15
+
+die()
+{
+    [ -n "$*" ] && echo "$@" >&2
+    exit 1
+}
+
+make_image()
+{
+    local image="$1"
+
+    dd if=/dev/urandom of="$image" bs=1M count=1 seek=10000 conv=notrunc 2>/dev/null ||
+        die "Unable to create image \"$image\""
+    dd if=/dev/urandom of="$image" bs=1M count=200 conv=notrunc 2>/dev/null ||
+        die "Unable to poison the start of image \"$image\""
+    losetup -f --show "$image" || die "Unable to attach loopback device"
+}
+
+make_fs()
+{
+    local device="$1"
+    local mountpoint="$2"
+    local extra_features="$3"
+    case "$extra_features" in
+        ,*) ;;
+        *)
+            extra_features=",$extra_features"
+            ;;
+    esac
+
+    mkfs -t ocfs2 -C 1M \
+        --fs-features=local"$extra_features" "$device" ||
+        die "Unable to create ocfs2 filesystem"
+    mount -t ocfs2 "$device" "$mountpoint" ||
+        die "Unable to mount ocfs2 filesystem"
+}
+
+clean_one_test()
+{
+    for f in "$@"
+    do
+        megs="$(ls -l "$f" | awk '{printf "%lu", ($5 + 1048576 - 1) / 1048576}')"
+        [ -z "$megs" ] && die "Unable to clean test file \"$f\""
+        dd if=/dev/urandom of="$f" bs=1M count="${megs}" 2>/dev/null ||
+            die "Unable to scribble over test file \"$f\""
+        rm -f "$f" || die "Unable to remove test file \"$f\""

programs/tailtest/tailtest new file +105 -0

+
+    local testfile="${mountpoint}/testfile"
+    local comparefile="${mountpoint}/diff"
+    local expectedfile="${mountpoint}/expected-$offset"
+
+    echo "Testing seek=$offset $trunc ..."
+    echo -n 'a' | dd of="$testfile" bs=1 count=1 2>/dev/null
+    umount "$mountpoint" || die "Unable to unmount \"$mountpoint\""
+    mount -t ocfs2 "$device" "$mountpoint" ||
+        die "Unable to remount \"$mountpoint\""
+    echo -n 'a' | dd of="$testfile" bs=1 count=1 seek=$offset $trunc 2>/dev/null
+    dd if="$testfile" bs=1M count=1 2>/dev/null | hexdump -C >"$comparefile"
+    if diff -q "$comparefile" "$expectedfile" >/dev/null 2>&1;
+    then
+        echo "    Test passed."
+    else
+        echo "    Test failed.  Extent contents:"
+        cat "$comparefile"
+    fi
+
+    echo "Cleaning up after test ..."
+    clean_one_test "$testfile" "$comparefile"
+}
+
+setup_expected()
+{
+    local mountpoint="$1"
+
+    cat >"${mountpoint}/expected-10M" <<EOCAT
+00000000  61 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |a...............|
+00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
+*
+00100000
+EOCAT
+    [ $? = 0 ] || die "Unable to create \"expected-10M\""
+
+    cp "${mountpoint}/expected-10M" "${mountpoint}/expected-1M" ||
+        die "Unable to create \"expected-1M\""
+
+    cat >"${mountpoint}/expected-512K" <<EOCAT
+00000000  61 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |a...............|
+00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
+*
+00080000  61                                                |a|
+00080001
+EOCAT
+    [ $? = 0 ] || die "Unable to create \"expected-512K\""
+
+    cat >"${mountpoint}/expected-1K" <<EOCAT
+00000000  61 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |a...............|
+00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
+*
+00000400  61                                                |a|
+00000401
+EOCAT
+    [ $? = 0 ] || die "Unable to create \"expected-1K\""
+
+}
+
+run_tests()
+{
+    local device="$1"
+    local mountpoint="$2"
+    local extra_features="$3"
+
+    echo "Generating filesystem ..."
+    make_fs "$device" "$mountpoint" $extra_features ||
+        die "Unable to create ocfs2 filesystem"
+    setup_expected "$mountpoint"
+
+    local offset
+    for offset in 10M 1M 512K 1K
+    do
+        run_one_test "$device" "$mountpoint" "$offset" "false"
+        run_one_test "$device" "$mountpoint" "$offset" "true"
+    done
+
+    echo "Destroying filesystem ..."
+    umount "$mountpoint" || die "Unable to unmount filesystem"
+    dd if=/dev/urandom of="$device" bs=1M count=1 2>/dev/null ||
+        die "Unable to reset device"
+}
+
+main()
+{
+    echo "Generating image ..."
+    _IMAGE="$(mktemp ${TMPDIR:-/tmp}/tailtest-image.XXXXXX 2>/dev/null)"
+    [ -z "$_IMAGE" ] && die "Unable to create image file"
+    _LOOP="$(make_image "$_IMAGE")"
+    [ -z "$_LOOP" ] && die "Unable to attach loopback device"
+
+    _MOUNTPOINT="$(mktemp -d ${TMPDIR:-/tmp}/tailtest-mountpoint.XXXXXX 2>/dev/null)"
+    [ -z "$_MOUNTPOINT" ] && die "Unable to create mountpoint"
+
+    run_tests "$_LOOP" "$_MOUNTPOINT" "sparse,inline-data"
+    run_tests "$_LOOP" "$_MOUNTPOINT" "sparse,noinline-data"
+    run_tests "$_LOOP" "$_MOUNTPOINT" "nosparse,inline-data"
+    run_tests "$_LOOP" "$_MOUNTPOINT" "nosparse,noinline-data"
+
+    return 0
+}
+
+main "$@"
+exit $?
+

Hi Joel,

I'm totally trusting the logic of your testcase to test tail zeroing.

Just two concerns as follows:

1. It should be better to make the script runnable for none-privilege users for the sake of security concerns.

2. Why not make variations with different bs and cs combinations

programs/tailtest/tailtest line 15

Quoted 6 lines

programs/tailtest/tailtest new file +6 -0

+cleanup()
+{
+    [ -d "$_MOUNTPOINT" ] && umount "$_MOUNTPOINT"
+    [ -d "$_MOUNTPOINT" ] && rmdir "$_MOUNTPOINT"
+    [ -n "$_LOOP" ] && losetup -d "$_LOOP"
+    [ -n "$_IMAGE" ] && rm -f "$_IMAGE"

You may also need to cleanup expected template files for comparison? if the test aborts abnormally.

or rmdir never succeeds. I guess using "rm -rf $MOUNTPOINT" here will be more aggressive to handle this.

Quoted 8 lines · tristan

Hi Joel,

I'm totally trusting the logic of your testcase to test tail zeroing.

Just two concerns as follows:

1. It should be better to make the script runnable for none-privilege users for the sake of security concerns.

Can't mount without root privileges, and the test requires
umount/mount cycles to expose the problem. I wish I had a better answer too.

Quoted 1 line

2. Why not make variations with different bs and cs combinations

This is a good idea for completeness, but I don't want to spend
the time on that right now. I picked a bs/cs combination that was sure to expose the bug as we know it.

Quoted 8 lines

programs/tailtest/tailtest line 15

Quoted 6 lines

programs/tailtest/tailtest new file +6 -0

+cleanup()
+{
+    [ -d "$_MOUNTPOINT" ] && umount "$_MOUNTPOINT"
+    [ -d "$_MOUNTPOINT" ] && rmdir "$_MOUNTPOINT"
+    [ -n "$_LOOP" ] && losetup -d "$_LOOP"
+    [ -n "$_IMAGE" ] && rm -f "$_IMAGE"

You may also need to cleanup expected template files for comparison? if the test aborts abnormally.

The template files are on the test filesystem. They go away
when we remove the image.

Quoted 2 lines

or rmdir never succeeds. I guess using "rm -rf $MOUNTPOINT" here will be more aggressive to handle this.

There's nothing under the mountpoint after we've unmounted.
that's why I use rmdir and not rm -rf.

Joel

Signed-off-by Tristan Ye <[email protected]>