Skip to contentSkip to conversation

New: xattr ext4_xattr_block_find, bad block on cleanly formatted ext4 partition

Bug 193661 [email protected] 1 participant 11 messages Jan 30, 2017 – Feb 28, 2017

lore
Share & tools
lore

[email protected]

author
#0 lore raw

https://bugzilla.kernel.org/show_bug.cgi?id=193661

Bug ID: 193661
Summary: xattr ext4_xattr_block_find, bad block on cleanly
formatted ext4 partition
Product: File System
Version: 2.5
Kernel Version: all versions tested, from 3.19 through to 4.10-rc6
Hardware: i386
OS: Linux
Tree: Mainline
Status: NEW
Severity: high
Priority: P1
Component: ext4
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Reproducer:

Ran inside a 8 proc VM instance, i386 kernel, 4MB of memory with the stress-ng xattr stress test on a cleanly formated ext4 partition (e.g. mkfs.ext4 /dev/vdb1):

using stress-ng - to build from source on ubuntu systems:
git clone git://kernel.ubuntu.com/cking/stress-ng
sudo apt-get build-dep stress-ng
cd stress-ng
make

test script to invoke the stressor, needs to be run as root:

#!/bin/bash -x
for i in $(seq 30)
do

        ./stress-ng/stress-ng --xattr 8 -t 10
        rc=$?
        if [ $rc -ne 0 ]; then
                exit 1
        fi

done

This will trigger errors such as:

EXT4-fs error (device vdb1): ext4_xattr_block_find:802: inode #131074: comm
stress-ng-xattr: bad block 532519

and more often than not one needs to fsck the partition.

I've tested this on mainline kernels from 3.19 through to 4.10-rc6 and I can trigger this on these kernels only on i386 systems. I've tested other 32 bit platforms (32 bit arm, raspberry pi 2) but can't trigger it. I cannot trigger this issue on amd64 builds of the same kernels in a VM.

I put some debug into ext4_xattr_block_set() at the /* update the inode. * comment and the bug does not trip, so it seems that this reduces the risk of teh race condition occurring.

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#4 lore raw

[Bug 193661] xattr ext4_xattr_block_find, bad block on cleanly formatted ext4 partition

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=193661

Theodore Tso <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #4 from Theodore Tso <[email protected]> --- Thanks for the bug report!

Does this require using the latest version of stress-ng (from the git tree), or is it reproducible using the version of stress-ng in Ubuntu or Debian Jessie?

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#6 lore raw

[Bug 193661] xattr ext4_xattr_block_find, bad block on cleanly formatted ext4 partition

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=193661

--- Comment #6 from Theodore Tso <[email protected]> ---
And since you appear to be the maintainer of the stress-ng package, while I'm here, any objections if upload a backport of stress-ng v0.07.16 to jessie-backports? :-)

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#9 lore raw

[Bug 193661] xattr ext4_xattr_block_find, bad block on cleanly formatted ext4 partition

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=193661

Christian Kujau ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #9 from Christian Kujau ([email protected]) --- Wow, this triggers in a Debian/unstable 64-bit VM here after the first run:

$ mkfs.ext4 /dev/sdd
$ mount -t ext4 /dev/sdd /mnt/disk && cd /mnt/disk
$ time ~/193661.sh
++ seq 30
+ for i in $(seq 30)
+ /usr/local/sbin/stress-ng --xattr 8 -t 10
stress-ng: info:  [2639] dispatching hogs: 8 xattr
stress-ng: info:  [2639] cache allocate: default cache size: 6144K
stress-ng: fail:  [2647] stress-ng-xattr: fsetxattr failed, errno=117
(Structure needs cleaning)
stress-ng: error: [2639] process 2647 (stress-ng-xattr) terminated with an
error, exit status=1
stress-ng: info:  [2639] unsuccessful run completed in 10.01s
+ rc=2
+ '[' 2 -ne 0 ']'
+ exit 1
real    0m10.019s
user    0m1.460s
sys     0m26.752s
$ dmesg | tail -2
[  494.311212] EXT4-fs (sdd): mounted filesystem with ordered data mode. Opts:
(null)
[  550.828337] EXT4-fs error (device sdd): ext4_xattr_block_find:786: inode
#524292: comm stress-ng-xattr: bad block 2105384
$ uname -rv
4.9.0-1-amd64 #1 SMP Debian 4.9.6-3 (2017-01-28)
Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.
author
#10 lore raw

[Bug 193661] xattr ext4_xattr_block_find, bad block on cleanly formatted ext4 partition

In reply to [email protected]

https://bugzilla.kernel.org/show_bug.cgi?id=193661

Theodore Tso ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

           Assignee|[email protected] |[email protected]
                   |g                           |

--- Comment #10 from Theodore Tso ([email protected]) --- Created attachment 254979
--> https://bugzilla.kernel.org/attachment.cgi?id=254979&action=edit
ext4: lock the the xattr block before calculating its checksum

Signature · 2 lines
You are receiving this mail because:
You are watching the assignee of the bug.