Skip to contentSkip to conversation

xfs: log record crc mismatch test

PATCH In review Alexander Tsvetkov 3 participants 4 messages Apr 27, 2015 – May 13, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: log record crc mismatch test

Added new test case for simulation of V5 filesystem log crc record corruption using of external log.

---
 tests/xfs/005     | 28 ++++++++++++++++++++++++++--
 tests/xfs/005.out |  3 +++
 2 files changed, 29 insertions(+), 2 deletions(-)
tests/xfs/005 +26 -2
index fade4bb..ba0dcbb 100755
@@ -1,7 +1,7 @@
 #! /bin/bash
 # FS QA Test No. 005
 #
-# Test that a bad crc on a primary V5 superblock will fail the mount
+# Test that a bad crc on a V5 filesystem will fail the mount
 #
 # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
 # inadvertently caused primary SB CRC failures to not error out, this
@@ -55,10 +55,12 @@ _supported_os Linux
 
 _require_scratch_nocheck
 _require_xfs_mkfs_crc
+_require_logdev
+logsize=2560
 
 rm -f $seqres.full
 
-_scratch_mkfs_xfs -m crc=1 >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b >> $seqres.full 2>&1 || _fail "mkfs failed"
 
 # Zap the crc.  xfs_db updates the CRC post-write, so poke it directly
 $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
@@ -66,6 +68,28 @@ $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
 # should FAIL, the crc is bad; golden output contains mount failure
 _scratch_mount 2>&1 | filter_mount
 
+_scratch_unmount &> /dev/null
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b  >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
+
+# Run filesytem exercising for some time
+_param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
+$FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT &> /dev/null
+sleep 3
+
+# Shutdown filesystem and unmount
+src/godown -v $SCRATCH_MNT >> $seqres.full
+killall -q fsstress
+wait
+sleep 3
+_scratch_unmount
+
+# Zap log record crc.
+$XFS_IO_PROG -c "pwrite 5000 5" $SCRATCH_LOGDEV | _filter_xfs_io
+
+# should FAIL, log record crc is bad; golden output contains mount failure
+_scratch_mount 2>&1 | filter_mount
+
 # success, all done
 status=0
 exit
tests/xfs/005.out +3 -0
index 13d0125..8907301 100644
@@ -2,3 +2,6 @@ QA output created by 005
 wrote 4/4 bytes at offset 224
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 mount: Structure needs cleaning
+wrote 5/5 bytes at offset 5000
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning
Signature · 6 lines
1.9.3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw
Quoted 14 lines · Alexander Tsvetkov

Added new test case for simulation of V5 filesystem log crc record corruption using of external log.

---
 tests/xfs/005     | 28 ++++++++++++++++++++++++++--
 tests/xfs/005.out |  3 +++
 2 files changed, 29 insertions(+), 2 deletions(-)
tests/xfs/005 +0 -0
index fade4bb..ba0dcbb 100755

tests/xfs/005 line 4

Quoted 5 lines
tests/xfs/005 +1 -1
@@ -1,7 +1,7 @@
 #! /bin/bash
 # FS QA Test No. 005
 #
-# Test that a bad crc on a primary V5 superblock will fail the mount
+# Test that a bad crc on a V5 filesystem will fail the mount

bad crc on what?

tests/xfs/005 _supported_os Linux · line 59

Quoted 9 lines
tests/xfs/005 +2 -0
 #
 # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
 # inadvertently caused primary SB CRC failures to not error out, this
@@ -55,10 +55,12 @@ _supported_os Linux
 
 _require_scratch_nocheck
 _require_xfs_mkfs_crc
+_require_logdev
+logsize=2560

No, this means that xfs/05 will no longer work without an external logdev, right? So you've stopped testing the internal case, and we've lost that coverage.

And, we generally don't want to add new testcases to old tests because then it gets harder to investigate regressions. It'd probably be best to create a new test which specifically tests log crc corruption, and make it handle both internal and external logs automatically.

Thanks,
-Eric

Quoted 41 lines
tests/xfs/005 +23 -1
 rm -f $seqres.full
 
-_scratch_mkfs_xfs -m crc=1 >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b >> $seqres.full 2>&1 || _fail "mkfs failed"
 
 # Zap the crc.  xfs_db updates the CRC post-write, so poke it directly
 $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
@@ -66,6 +68,28 @@ $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
 # should FAIL, the crc is bad; golden output contains mount failure
 _scratch_mount 2>&1 | filter_mount
 
+_scratch_unmount &> /dev/null
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b  >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
+
+# Run filesytem exercising for some time
+_param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
+$FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT &> /dev/null
+sleep 3
+
+# Shutdown filesystem and unmount
+src/godown -v $SCRATCH_MNT >> $seqres.full
+killall -q fsstress
+wait
+sleep 3
+_scratch_unmount
+
+# Zap log record crc.
+$XFS_IO_PROG -c "pwrite 5000 5" $SCRATCH_LOGDEV | _filter_xfs_io
+
+# should FAIL, log record crc is bad; golden output contains mount failure
+_scratch_mount 2>&1 | filter_mount
+
 # success, all done
 status=0
 exit
tests/xfs/005.out +0 -0
index 13d0125..8907301 100644

tests/xfs/005.out QA output created by 005 · line 7

Quoted 6 lines
tests/xfs/005.out +3 -0
@@ -2,3 +2,6 @@ QA output created by 005
 wrote 4/4 bytes at offset 224
 XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 mount: Structure needs cleaning
+wrote 5/5 bytes at offset 5000
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw

Hi Eric,

Quoted 35 lines · Eric Sandeen
Quoted 14 lines · Alexander Tsvetkov

Added new test case for simulation of V5 filesystem log crc record corruption using of external log.

---
  tests/xfs/005     | 28 ++++++++++++++++++++++++++--
  tests/xfs/005.out |  3 +++
  2 files changed, 29 insertions(+), 2 deletions(-)
tests/xfs/005 +0 -0
index fade4bb..ba0dcbb 100755

tests/xfs/005 line 4

Quoted 5 lines
tests/xfs/005 +1 -1
@@ -1,7 +1,7 @@
  #! /bin/bash
  # FS QA Test No. 005
  #
-# Test that a bad crc on a primary V5 superblock will fail the mount
+# Test that a bad crc on a V5 filesystem will fail the mount

bad crc on what?

tests/xfs/005 _supported_os Linux · line 59

Quoted 9 lines
tests/xfs/005 +2 -0
  #
  # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
  # inadvertently caused primary SB CRC failures to not error out, this
@@ -55,10 +55,12 @@ _supported_os Linux
 
  _require_scratch_nocheck
  _require_xfs_mkfs_crc
+_require_logdev
+logsize=2560

No, this means that xfs/05 will no longer work without an external logdev, right? So you've stopped testing the internal case, and we've lost that coverage.

Quoted 4 lines

And, we generally don't want to add new testcases to old tests because then it gets harder to investigate regressions. It'd probably be best to create a new test which specifically tests log crc corruption, and make it handle both internal and external logs automatically.

I've prepared the test with both cases, see at the bottom of this mail

Quoted 42 lines

Thanks,
-Eric

Quoted 39 lines
tests/xfs/005 +23 -1
  rm -f $seqres.full
 
-_scratch_mkfs_xfs -m crc=1 >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b >> $seqres.full 2>&1 || _fail "mkfs failed"
 
  # Zap the crc.  xfs_db updates the CRC post-write, so poke it directly
  $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
@@ -66,6 +68,28 @@ $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
  # should FAIL, the crc is bad; golden output contains mount failure
  _scratch_mount 2>&1 | filter_mount
 
+_scratch_unmount &> /dev/null
+_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b  >> $seqres.full 2>&1 || _fail "mkfs failed"
+_scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
+
+# Run filesytem exercising for some time
+_param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
+$FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT &> /dev/null
+sleep 3
+
+# Shutdown filesystem and unmount
+src/godown -v $SCRATCH_MNT >> $seqres.full
+killall -q fsstress
+wait
+sleep 3
+_scratch_unmount
+
+# Zap log record crc.
+$XFS_IO_PROG -c "pwrite 5000 5" $SCRATCH_LOGDEV | _filter_xfs_io
+
+# should FAIL, log record crc is bad; golden output contains mount failure
+_scratch_mount 2>&1 | filter_mount
+
  # success, all done
  status=0
  exit
tests/xfs/005.out +0 -1
index 13d0125..8907301 100644
--- a/tests/xfs/005.out
Quoted 8 lines

tests/xfs/005.out QA output created by 005 · line 7

Quoted 8 lines

+++ b/tests/xfs/005.out

tests/xfs/005.out +3 -0
@@ -2,3 +2,6 @@ QA output created by 005
  wrote 4/4 bytes at offset 224
  XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
  mount: Structure needs cleaning
+wrote 5/5 bytes at offset 5000
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning

Added new test for simulation of V5 filesystem log crc record corruption in internal and external logs

---
 tests/xfs/306     | 110 
++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/306.out |   7 ++++
 tests/xfs/group   |   1 +
 3 files changed, 118 insertions(+)
 create mode 100755 tests/xfs/306
 create mode 100644 tests/xfs/306.out
tests/xfs/306 new file +110 -0
new file mode 100755
index 0000000..6e025fe
@@ -0,0 +1,110 @@
+#! /bin/bash
+# FS QA Test No. 306
+#
+# Test that bad crc on a V5 log record will fail the mount
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2015 Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+filter_mount()
+{
+    sed -e "s/mount .* failed: //"
+}
+
+filter_io()
+{
+    sed -e "s/at offset .*/at offset XXX/"
+}
+
+# $1 - log start offset
+# $2 - log device
+do_test()
+{
+    off=$1
+    dev=$2
+
+    _scratch_unmount >/dev/null 2>&1
+    _scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
+
+    # Run filesytem exercising for some time
+    param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
+    $FSSTRESS_PROG $param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT &> /dev/null
+    sleep 3
+
+    # Shutdown filesystem and unmount
+    src/godown -v $SCRATCH_MNT >> $seqres.full
+    killall -q fsstress
+    wait
+    sleep 3
+    _scratch_unmount
+
+    # Zap log record crc
+    off=$((off + 5000))
+    $XFS_IO_PROG -c "pwrite $off 5" $dev | _filter_xfs_io | filter_io
+
+    # should FAIL, log record crc is bad; golden output contains mount failure
+    _scratch_mount 2>&1 | filter_mount
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_supported_os Linux
+
+_require_scratch_nocheck
+_require_xfs_mkfs_crc
+_require_logdev
+
+rm -f $seqres.full
+fssize=$((32 * 1024 * 1024))
+MKFS_OPTIONS="-m crc=1 $MKFS_OPTIONS"
+_scratch_mkfs_sized $fssize 2> /dev/null | _filter_mkfs 2> $tmp.mkfs > /dev/null
+. $tmp.mkfs
+
+# internal log corruption
+unset USE_EXTERNAL
+_scratch_mkfs_sized $fssize >> $seqres.full 2>&1 || _fail "mkfs failed"
+logstart=$($XFS_DB_PROG -c "sb 0" -c "print" $SCRATCH_DEV | grep logstart | cut -f3 -d' ')
+offset=$((logstart * lbsize))
+do_test $offset $SCRATCH_DEV
+
+# external log corruption
+USE_EXTERNAL="yes"
+_scratch_mkfs_sized $fssize >> $seqres.full 2>&1 || _fail "mkfs failed"
+do_test 0 $SCRATCH_LOGDEV
+
+# success, all done
+status=0
+exit
tests/xfs/306.out new file +7 -0
new file mode 100644
index 0000000..aef98f2
@@ -0,0 +1,7 @@
+QA output created by 306
+wrote 5/5 bytes at offset XXX
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning
+wrote 5/5 bytes at offset XXX
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning
tests/xfs/group +1 -0
index 58144d2..f108377 100644
@@ -191,3 +191,4 @@
 303 auto quick quota
 304 auto quick quota
 305 auto quota
+306 auto quick
Signature · 6 lines
1.9.3

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw
In reply to Alexander Tsvetkov
Quoted 4 lines · Alexander Tsvetkov

Added new test case for simulation of V5 filesystem log crc record corruption using of external log.

Hi Alexander,

In future, can you please post xfstests patches to [email protected]? I've cc'd that list on my reply.

Quoted 19 lines
---
 tests/xfs/005     | 28 ++++++++++++++++++++++++++--
 tests/xfs/005.out |  3 +++
 2 files changed, 29 insertions(+), 2 deletions(-)
tests/xfs/005 +1 -1
index fade4bb..ba0dcbb 100755
@@ -1,7 +1,7 @@
 #! /bin/bash
 # FS QA Test No. 005
 #
-# Test that a bad crc on a primary V5 superblock will fail the mount
+# Test that a bad crc on a V5 filesystem will fail the mount
 #
 # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
 # inadvertently caused primary SB CRC failures to not error out, this

tests/xfs/005 _supported_os Linux · line 59

Quoted 5 lines
tests/xfs/005 +2 -0
@@ -55,10 +55,12 @@ _supported_os Linux
 
 _require_scratch_nocheck
 _require_xfs_mkfs_crc
+_require_logdev
+logsize=2560

As eric pointed out, this fundamentally changes the context of the test, and so should be written as a new test rather than a change to an existing test.

Also, you don't need an external log device to do this - you can use xfs_db to zero parts of the internal log just fine.

$ sudo mkfs.xfs -f -m crc=1,finobt=1 /dev/ram0
meta-data=/dev/ram0              isize=512    agcount=4, agsize=250000 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1
data     =                       bsize=4096   blocks=1000000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$ sudo xfs_db -x /dev/ram0
xfs_db> sb 0
xfs_db> p logstart
logstart = 524296
xfs_db> a logstart
xfs_db> type data
xfs_db> write fill 0xa5a5a5a5 0 4096
xfs_db> p
000: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
020: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
.....
fe0: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
xfs_db> q
$ sudo mount /dev/ram0 /mnt/test
[346491.179086] XFS (ram0): Mounting V5 Filesystem
[346491.180844] XFS (ram0): Log inconsistent or not a log (last==0, first!=1)
[346491.182946] XFS (ram0): empty log check failed
[346491.184403] XFS (ram0): log mount/recovery failed: error -22
[346491.186276] XFS (ram0): log mount failed
.....

Note that if you have an external log device, logstart will be zero, and you'll need to use xfs_io to overwrite it so you'll need to explicitly test for that case before calling xfs_db to modify the log.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]