Skip to contentSkip to conversation

xfs: test dir/attr hash when loading module

PATCH In review Darrick J. Wong 2 participants 3 messages Mar 16, 2023 – Mar 18, 2023 XFS filesystem

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: test dir/attr hash when loading module

From: Darrick J. Wong <[email protected]>

Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.

This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.

After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.

In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.

The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.

---
 fs/xfs/Makefile          |    1 
 fs/xfs/xfs_dahash_test.c |  662 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_dahash_test.h |   12 +
 fs/xfs/xfs_super.c       |    5 
 4 files changed, 680 insertions(+)
 create mode 100644 fs/xfs/xfs_dahash_test.c
 create mode 100644 fs/xfs/xfs_dahash_test.h
fs/xfs/Makefile +1 -0
index 35c1b41a3ef1..bbabf898cabc 100644
@@ -127,6 +127,7 @@ xfs-y				+= xfs_aops.o \
 				   xfs_bmap_util.o \
 				   xfs_bio_io.o \
 				   xfs_buf.o \
+				   xfs_dahash_test.o \
 				   xfs_dir2_readdir.o \
 				   xfs_discard.o \
 				   xfs_error.o \
fs/xfs/xfs_dahash_test.c new file +662 -0

662 lines not on this page · raw message

fs/xfs/xfs_dahash_test.h new file +12 -0
new file mode 100644
index 000000000000..1a05bf4bd9e1
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Oracle.  All Rights Reserved.
+ * Author: Darrick J. Wong <[email protected]>
+ */
+#ifndef __XFS_DAHASH_TEST_H__
+#define __XFS_DAHASH_TEST_H__
+
+int xfs_dahash_test(void);
+
+#endif /* __XFS_DAHASH_TEST_H__ */
+
fs/xfs/xfs_super.c +5 -0
index 2479b5cbd75e..4f814f9e12ab 100644
@@ -41,6 +41,7 @@
 #include "xfs_attr_item.h"
 #include "xfs_xattr.h"
 #include "xfs_iunlink_item.h"
+#include "xfs_dahash_test.h"
 
 #include <linux/magic.h>
 #include <linux/fs_context.h>
@@ -2286,6 +2287,10 @@ init_xfs_fs(void)
 
 	xfs_check_ondisk_structs();
 
+	error = xfs_dahash_test();
+	if (error)
+		return error;
+
 	printk(KERN_INFO XFS_VERSION_STRING " with "
 			 XFS_BUILD_OPTIONS " enabled\n");
 
Reviewed-by on v1 1/1
#1 lore raw
Quoted 27 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.

This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.

After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.

In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.

Quoted 8 lines

The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.

I'm going to trust that your binary tables exercise the hash in the manner needed because I don't have time right now to manually decode it. With that caveat, everything else looks fine.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
Quoted 34 lines · Dave Chinner
Quoted 33 lines · Darrick J. Wong

From: Darrick J. Wong <[email protected]>

Back in the 6.2-rc1 days, Eric Whitney reported a fstests regression in ext4 against generic/454. The cause of this test failure was the unfortunate combination of setting an xattr name containing UTF8 encoded emoji, an xattr hash function that accepted a char pointer with no explicit signedness, signed type extension of those chars to an int, and the 6.2 build tools maintainers deciding to mandate -funsigned-char across the board. As a result, the ondisk extended attribute structure written out by 6.1 and 6.2 were not the same.

This discrepancy, in fact, had been noticeable if a filesystem with such an xattr were moved between any two architectures that don't employ the same signedness of a raw "char" declaration. The only reason anyone noticed is that x86 gcc defaults to signed, and no such -funsigned-char update was made to e2fsprogs, so e2fsck immediately started reporting data corruption.

After a day and a half of discussing how to handle this use case (xattrs with bit 7 set anywhere in the name) without breaking existing users, Linus merged his own patch and didn't tell the mailing list. None of the developers noticed until AUTOSEL made an announcement.

In the end, this problem could have been detected much earlier if there had been any useful tests of hash function(s) in use inside ext4 to make sure that they always produce the same outputs given the same inputs.

The XFS dirent/xattr name hash takes a uint8_t*, so I don't think it's vulnerable to this problem. However, let's avoid all this drama by adding our own self test to check that the da hash produces the same outputs for a static pile of inputs on various platforms. This will be followed up in xfsprogs with a similar patch.

Quoted 6 lines
Quoted 2 lines

I'm going to trust that your binary tables exercise the hash in the manner needed because I don't have time right now to manually decode it. With that caveat, everything else looks fine.

Yep. The kernel and userspace use the same 4k buffer of arbitrary bytes, and the test tables are identical. I don't know if the dahash function is *correct* mathematically speaking, but at least this will demonstrate consistency in behavior between the kernel and userspace.

--D

Quoted 4 lines
Signature · 2 lines
Dave Chinner
[email protected]