xfs: test dir/attr hash when loading module
PATCH 1/1 xfs: test dir/attr hash when loading module
From: Darrick J. Wong <[email protected]>
Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.
This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.
After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.
In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.
The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.
--- fs/xfs/Makefile | 1 fs/xfs/xfs_dahash_test.c | 662 ++++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_dahash_test.h | 12 + fs/xfs/xfs_super.c | 5 4 files changed, 680 insertions(+) create mode 100644 fs/xfs/xfs_dahash_test.c create mode 100644 fs/xfs/xfs_dahash_test.h
fs/xfs/Makefile +1 -0
@@ -127,6 +127,7 @@ xfs-y += xfs_aops.o \ xfs_bmap_util.o \ xfs_bio_io.o \ xfs_buf.o \ + xfs_dahash_test.o \ xfs_dir2_readdir.o \ xfs_discard.o \ xfs_error.o \
fs/xfs/xfs_dahash_test.c new file +662 -0
662 lines not on this page · raw message
fs/xfs/xfs_dahash_test.h new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2023 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __XFS_DAHASH_TEST_H__ +#define __XFS_DAHASH_TEST_H__ + +int xfs_dahash_test (void ); + +#endif /* __XFS_DAHASH_TEST_H__ */ +
fs/xfs/xfs_super.c +5 -0
@@ -41,6 +41,7 @@#include "xfs_attr_item.h" #include "xfs_xattr.h" #include "xfs_iunlink_item.h" +#include "xfs_dahash_test.h" #include <linux/magic.h> #include <linux/fs_context.h>
@@ -2286,6 +2287,10 @@ init_xfs_fs(void)xfs_check_ondisk_structs (); + error =xfs_dahash_test (); +if (error) +return error; + printk(KERN_INFO XFS_VERSION_STRING " with " XFS_BUILD_OPTIONS " enabled\n");
Quoted 27 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.
This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.
After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.
In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.
Quoted 8 lines
The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.
I'm going to trust that your binary tables exercise the hash in the manner needed because I don't have time right now to manually decode it. With that caveat, everything else looks fine.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 34 lines · Dave Chinner
Quoted 33 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.
This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.
After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.
In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.
The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.
Quoted 6 lines
Quoted 2 lines
I'm going to trust that your binary tables exercise the hash in the manner needed because I don't have time right now to manually decode it. With that caveat, everything else looks fine.
Yep. The kernel and userspace use the same 4k buffer of arbitrary bytes, and the test tables are identical. I don't know if the dahash function is *correct* mathematically speaking, but at least this will demonstrate consistency in behavior between the kernel and userspace.
--D
Quoted 4 lines
Reviewed-by Dave Chinner <[email protected]>Signature · 2 lines
Dave Chinner [email protected]