inline_data,metadata_csum a lot of bad csum for directories

DISCUSSION Dmitry Monakhov 4 participants 5 messages Dec 1, 2014 – Dec 5, 2014

Share & tools

Dmitry Monakhov

author

2014-12-01 14:09 UTC #0 lore raw

Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment
I use xfstest-bld for this purpose, but most test generates a lot of incorrect inode's csum for directories(almost any test which makes/rm directories). What I'm doing wrong? Are where any important fix I'm missing? Please post your test matrix

My config:
e2fsprogs git commit: g210baa5
ext4.git commit: recent tytso/dev ca83cc75a16f67
./kvm-xfstests -c inline generic/011 generic/013
generic/013 [13:51:44]

[  368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid

Andreas Dilger

2014-12-01 23:32 UTC #1 lore raw

Quoted 3 lines · Dmitry Monakhov

Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment

I suspect that since inline_data and metadata_csum were developed in isolation, they don't work well together yet. I believe that Darrick is working on fixing some issues in this area.

Quoted 5 lines

I use xfstest-bld for this purpose, but most test generates a lot of incorrect inode's csum for directories(almost any test which makes/rm directories). What I'm doing wrong? Are where any important fix I'm missing? Please post your test matrix

Quoted 8 lines

My config:
e2fsprogs git commit: g210baa5
ext4.git commit: recent tytso/dev ca83cc75a16f67
./kvm-xfstests -c inline generic/011 generic/013
generic/013 [13:51:44]
[  368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid

Cheers, Andreas

Theodore Ts'o

2014-12-02 03:58 UTC #2 lore raw

Quoted 8 lines · Andreas Dilger

Quoted 3 lines · Dmitry Monakhov
Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment
I suspect that since inline_data and metadata_csum were developed in isolation, they don't work well together yet. I believe that Darrick is working on fixing some issues in this area.

The standard xfstests-bld test configuration is with inline_data and metadata_csum in combination:

export FS=ext4
export TEST_DEV=$SM_TST_DEV
export TEST_DIR=$SM_TST_MNT
export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" export EXT_MOUNT_OPTIONS="-o block_validity" TESTNAME="Ext4 4k block w/inline"

Are you using the very latest version of e2fsprogs.git#next branch for your tests? If not, that could be why you are getting some test failures. It is absolutely required for any tests involving metdata_csum.

I have a xfstests-build 32-bit rootfs.img I can upload for your convenience if that's what you are using. Getting a formally upgraded xfstests-bld rootfs.img with the latest xfstests and some other bug fixes and the latest e2fsprogs was on my todo list; sorry, the metadata csum v3 patches that went in in 3.18-rc3 required an updated e2fsprogs that's not in the released rootfs.img.

Quoted 4 lines

Quoted 4 lines
Please post your test matrix
My config:
e2fsprogs git commit: g210baa5

OK, that should be sufficiently up to date. As far as my test matrix, please see:

http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf

with these test versions:

e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500)
fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600)
quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300)
xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000)
xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400)
xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100)

Quoted 6 lines

Quoted 6 lines
ext4.git commit: recent tytso/dev ca83cc75a16f67
./kvm-xfstests -c inline generic/011 generic/013
generic/013 [13:51:44]
[  368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid

Hmm, works for me:

./kvm-xfstests -c inline generic/011 generic/013

BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014
Device: /dev/vdd
mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit mount options: -o block_validity

FSTYP         -- ext4
PLATFORM      -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869
MKFS_OPTIONS  -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc

MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc

generic/011 2s ... [22:56:24] [22:56:26] 2s
generic/013 47s ... [22:56:26] [22:57:21] 55s
Ran: generic/011 generic/013
Passed all 2 tests

- Ted

Darrick J. Wong

2014-12-02 05:08 UTC #3 lore raw

Quoted 9 lines · Theodore Ts'o

Quoted 8 lines · Andreas Dilger
Quoted 3 lines · Dmitry Monakhov
Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment
I suspect that since inline_data and metadata_csum were developed in isolation, they don't work well together yet. I believe that Darrick is working on fixing some issues in this area.

Fixing metadata_csum bugs, yes. There could be plenty more in inline_data -- there are a couple of spots (setting extra_isize in particular) where I'm not entirely sure it's setting the inode checksum before writing the raw inode out to disk.

On the plus side, from running generic/011 in a loop I think I've found some sort of strange bug where unlinked inodes fail csum validation, causing e2fsck to complain. A cheap solution is not to bother revalidating an inode if its link count is zero, but I want to know why the kernel isn't doing the right thing with the unlinked inodes.

--D

Quoted 72 lines

The standard xfstests-bld test configuration is with inline_data and metadata_csum in combination:
export FS=ext4
export TEST_DEV=$SM_TST_DEV
export TEST_DIR=$SM_TST_MNT
export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" export EXT_MOUNT_OPTIONS="-o block_validity" TESTNAME="Ext4 4k block w/inline"
Are you using the very latest version of e2fsprogs.git#next branch for your tests? If not, that could be why you are getting some test failures. It is absolutely required for any tests involving metdata_csum.
I have a xfstests-build 32-bit rootfs.img I can upload for your convenience if that's what you are using. Getting a formally upgraded xfstests-bld rootfs.img with the latest xfstests and some other bug fixes and the latest e2fsprogs was on my todo list; sorry, the metadata csum v3 patches that went in in 3.18-rc3 required an updated e2fsprogs that's not in the released rootfs.img.
Quoted 4 lines
Quoted 4 lines
Please post your test matrix
My config:
e2fsprogs git commit: g210baa5
OK, that should be sufficiently up to date. As far as my test matrix, please see:
http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf
with these test versions:
e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500)
fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600)
quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300)
xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000)
xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400)
xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100)
Quoted 6 lines
Quoted 6 lines
ext4.git commit: recent tytso/dev ca83cc75a16f67
./kvm-xfstests -c inline generic/011 generic/013
generic/013 [13:51:44]
[  368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid
Hmm, works for me:
./kvm-xfstests -c inline generic/011 generic/013
BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014
Device: /dev/vdd
mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit mount options: -o block_validity
FSTYP         -- ext4
PLATFORM      -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869
MKFS_OPTIONS  -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc
MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc
generic/011 2s ... [22:56:24] [22:56:26] 2s
generic/013 47s ... [22:56:26] [22:57:21] 55s
Ran: generic/011 generic/013
Passed all 2 tests
- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Darrick J. Wong

2014-12-05 00:11 UTC #4 lore raw

Quoted 16 lines · Darrick J. Wong

Quoted 9 lines · Theodore Ts'o
Quoted 8 lines · Andreas Dilger
Quoted 3 lines · Dmitry Monakhov
Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment
I suspect that since inline_data and metadata_csum were developed in isolation, they don't work well together yet. I believe that Darrick is working on fixing some issues in this area.
Fixing metadata_csum bugs, yes. There could be plenty more in inline_data -- there are a couple of spots (setting extra_isize in particular) where I'm not entirely sure it's setting the inode checksum before writing the raw inode out to disk.

Quoted 5 lines

On the plus side, from running generic/011 in a loop I think I've found some sort of strange bug where unlinked inodes fail csum validation, causing e2fsck to complain. A cheap solution is not to bother revalidating an inode if its link count is zero, but I want to know why the kernel isn't doing the right thing with the unlinked inodes.

Ahah, I introduced a race condition with the crypto API rework in ext4_chksum. I failed to notice that _setkey() modifies s_chksum_driver without any sort of concurrency protection. Soo... I'll send a patch tonight that fixes it.

Sorry about that,

--D

Quoted 79 lines

--D
Quoted 67 lines
The standard xfstests-bld test configuration is with inline_data and metadata_csum in combination:
export FS=ext4
export TEST_DEV=$SM_TST_DEV
export TEST_DIR=$SM_TST_MNT
export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" export EXT_MOUNT_OPTIONS="-o block_validity" TESTNAME="Ext4 4k block w/inline"
Are you using the very latest version of e2fsprogs.git#next branch for your tests? If not, that could be why you are getting some test failures. It is absolutely required for any tests involving metdata_csum.
I have a xfstests-build 32-bit rootfs.img I can upload for your convenience if that's what you are using. Getting a formally upgraded xfstests-bld rootfs.img with the latest xfstests and some other bug fixes and the latest e2fsprogs was on my todo list; sorry, the metadata csum v3 patches that went in in 3.18-rc3 required an updated e2fsprogs that's not in the released rootfs.img.
Quoted 4 lines
Quoted 4 lines
Please post your test matrix
My config:
e2fsprogs git commit: g210baa5
OK, that should be sufficiently up to date. As far as my test matrix, please see:
http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf
with these test versions:
e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500)
fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600)
quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300)
xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000)
xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400)
xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100)
Quoted 6 lines
Quoted 6 lines
ext4.git commit: recent tytso/dev ca83cc75a16f67
./kvm-xfstests -c inline generic/011 generic/013
generic/013 [13:51:44]
[  368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid
[  369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid
Hmm, works for me:
./kvm-xfstests -c inline generic/011 generic/013
BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014
Device: /dev/vdd
mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit mount options: -o block_validity
FSTYP         -- ext4
PLATFORM      -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869
MKFS_OPTIONS  -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc
MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc
generic/011 2s ... [22:56:24] [22:56:26] 2s
generic/013 47s ... [22:56:26] [22:57:21] 55s
Ran: generic/011 generic/013
Passed all 2 tests
Quoted 5 lines
- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html