EXT4: unsupported inode size: 4096

DISCUSSION Herbert Poetzl 3 participants 4 messages Feb 6, 2020 – Feb 7, 2020 ext4 file system

Share & tools

Herbert Poetzl

author

2020-02-06 15:35 UTC #0 lore raw

I recently updated one of my servers from an older 4.19 Linux kernel to the latest 5.5 kernel mainly because of the many filesystem improvements, just to find that some of my filesystems simply cannot be mounted anymore.

The kernel reports: EXT4-fs: unsupported inode size: 4096

Here is a simple test to reproduce the issue:

  truncate --size 16G data
  losetup /dev/loop0 data
  mkfs.ext4 -I 4096 /dev/loop0
  mount /dev/loop0 /media

[33700.299204] EXT4-fs (loop0): unsupported inode size: 4096

Note: this works perfectly fine und 4.19.84 and 4.14.145.

My guess so far is that somehow the ext4 filesystem now checks that the inode size is not larger than the logical block size of the underlying block device.

  # cat /sys/block/loop0/queue/logical_block_size  
  512

Note that the logical block size is also 512 on many SATA drives which have a physical block size of 4096.

Any ideas how to address this problem and get the file- systems to mount under Linux 5.5?

Many thanks in advance,
Herbert

Theodore Y. Ts'o

maintainer

2020-02-06 22:52 UTC #1 lore raw

Quoted 5 lines · Herbert Poetzl

I recently updated one of my servers from an older 4.19 Linux kernel to the latest 5.5 kernel mainly because of the many filesystem improvements, just to find that some of my filesystems simply cannot be mounted anymore.

Thanks for the bug report! This was actually a regression caused by recent security fix (which landed after 5.5, but which backported to the 5.5 stable kernel).

The following should fix things. Please let me know if you still have problems after applying this fix.

- Ted

From e0c963ab41e21bea32565a36654523687be3e7f0 Mon Sep 17 00:00:00 2001
From: Theodore Ts'o <[email protected]>
Date: Thu, 6 Feb 2020 17:35:01 -0500
Subject: [PATCH] ext4: fix support for inode sizes > 1024 bytes

A recent commit, 9803387c55f7 ("ext4: validate the debug_want_extra_isize mount option at parse time"), moved mount-time checks around. One of those changes moved the inode size check before the blocksize variable was set to the blocksize of the file system. After 9803387c55f7 was set to the minimum allowable blocksize, which in practice on most systems would be 1024 bytes. This cuased file systems with inode sizes larger than 1024 bytes to be rejected with a message:

EXT4-fs (sdXX): unsupported inode size: 4096

Fixes 9803387c55f7 ("ext4: validate the debug_want_extra_isize mount option at parse time")

Cc [email protected]

Reported-by Herbert Poetzl <[email protected]>

Signed-off-by Theodore Ts'o <[email protected]>

---
 fs/ext4/super.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

fs/ext4/super.c +10 -8

index 88b213bd32bc..eebcf9b9272d 100644
@@ -3814,6 +3814,15 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 	 */
 	sbi->s_li_wait_mult = EXT4_DEF_LI_WAIT_MULT;
 
+	blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size);
+	if (blocksize < EXT4_MIN_BLOCK_SIZE ||
+	    blocksize > EXT4_MAX_BLOCK_SIZE) {
+		ext4_msg(sb, KERN_ERR,
+		       "Unsupported filesystem blocksize %d (%d log_block_size)",
+			 blocksize, le32_to_cpu(es->s_log_block_size));
+		goto failed_mount;
+	}
+
 	if (le32_to_cpu(es->s_rev_level) == EXT4_GOOD_OLD_REV) {
 		sbi->s_inode_size = EXT4_GOOD_OLD_INODE_SIZE;
 		sbi->s_first_ino = EXT4_GOOD_OLD_FIRST_INO;

@@ -3831,6 +3840,7 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 			ext4_msg(sb, KERN_ERR,
 			       "unsupported inode size: %d",
 			       sbi->s_inode_size);
+			ext4_msg(sb, KERN_ERR, "blocksize: %d", blocksize);
 			goto failed_mount;
 		}
 		/*

@@ -4033,14 +4043,6 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)
 	if (!ext4_feature_set_ok(sb, (sb_rdonly(sb))))
 		goto failed_mount;
 
-	blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size);
-	if (blocksize < EXT4_MIN_BLOCK_SIZE ||
-	    blocksize > EXT4_MAX_BLOCK_SIZE) {
-		ext4_msg(sb, KERN_ERR,
-		       "Unsupported filesystem blocksize %d (%d log_block_size)",
-			 blocksize, le32_to_cpu(es->s_log_block_size));
-		goto failed_mount;
-	}
 	if (le32_to_cpu(es->s_log_block_size) >
 	    (EXT4_MAX_BLOCK_LOG_SIZE - EXT4_MIN_BLOCK_LOG_SIZE)) {
 		ext4_msg(sb, KERN_ERR,

Signature · 1 line

2.24.1

Andreas Dilger

2020-02-06 22:55 UTC #2 lore raw

In reply to Herbert Poetzl

Quoted 7 lines · Herbert Poetzl

I recently updated one of my servers from an older 4.19 Linux kernel to the latest 5.5 kernel mainly because of the many filesystem improvements, just to find that some of my filesystems simply cannot be mounted anymore.

Quoted 8 lines

The kernel reports: EXT4-fs: unsupported inode size: 4096
Here is a simple test to reproduce the issue:
truncate --size 16G data
losetup /dev/loop0 data
mkfs.ext4 -I 4096 /dev/loop0
mount /dev/loop0 /media

Does this still fail if you also specify "-b 4096"?

Quoted 1 line

[33700.299204] EXT4-fs (loop0): unsupported inode size: 4096

It looks like this is a bug in the code? This check is using

3641: blocksize = sb_min_blocksize(sb, EXT4_MIN_BLOCK_SIZE);

3782: if ((sbi->s_inode_size < EXT4_GOOD_OLD_INODE_SIZE) ||
3783: (!is_power_of_2(sbi->s_inode_size)) ||
3784: (sbi->s_inode_size > blocksize)) {
3785: ext4_msg(sb, KERN_ERR,
3786: "unsupported inode size: %d",
3787: sbi->s_inode_size);
3788: goto failed_mount;
3789: }

which is set from the hardware sector size of the device, while the ext4 filesystem blocksize is not set until later during mount:

3991: blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size);

It looks like this was just introduced in commit v5.4-rc3-96-g9803387 "ext4: validate the debug_want_extra_isize mount option at parse time" so it is a relatively recent change, and looks to be unintentional. This check was previously on line 4033, after "blocksize" was updated from the superblock, but it wasn't noticed because it works for all "normal" filesystems.

I suspect nobody has noticed because having an inode *size* of 4KB is very unusual, while having an inode *ratio* of 4KB is more normal (one 256-byte inode for each 4096-byte block in the filesystem). Was the use of "-I 4096" intentional, or did you mean to use "-i 4096"?

The only reason to have a 4096-byte inode *size* is if you have a ton of xattrs for every file and/or you have tiny files (< 3.5KB) and you are using inline data.

Quoted 8 lines

Note: this works perfectly fine und 4.19.84 and 4.14.145.
My guess so far is that somehow the ext4 filesystem now checks that the inode size is not larger than the logical block size of the underlying block device.
 # cat /sys/block/loop0/queue/logical_block_size
 512

Yes, this appears to be the case. We have LOT of filesystems that are using 1024-byte inodes, but I suspect that most of them are on devices that report 4096-byte sector size and/or are running older kernels that have not included this bug.

Quoted 2 lines

Any ideas how to address this problem and get the file- systems to mount under Linux 5.5?

Probably the easiest, and likely correct, fix is to move the update of "blocksize" from line 3991: up to before this check. There are a bunch of sanity checks that should also be moved for a proper patch, but the one-line fix is enough to get your filesystems mounting again.

Cheers, Andreas

Herbert Poetzl

author

2020-02-07 03:13 UTC #3 lore raw

Quoted 14 lines · Andreas Dilger

--Apple-Mail=_7CDF64A9-C7FD-4E08-9AB4-1843C57439EC
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii
Quoted 6 lines · Herbert Poetzl
I recently updated one of my servers from an older 4.19 Linux kernel to the latest 5.5 kernel mainly because of the many filesystem improvements, just to find that some of my filesystems simply cannot be mounted anymore.
The kernel reports: EXT4-fs: unsupported inode size: 4096

Quoted 8 lines

Quoted 6 lines
Here is a simple test to reproduce the issue:
truncate --size 16G data
losetup /dev/loop0 data
mkfs.ext4 -I 4096 /dev/loop0
mount /dev/loop0 /media
Does this still fail if you also specify "-b 4096"?

mkfs.ext4 -b 4096 -I 4096 /dev/loop0
mount /dev/loop0 /media

[66723.456449] EXT4-fs (loop0): unsupported inode size: 4096

Quoted 36 lines

Quoted 1 line
[33700.299204] EXT4-fs (loop0): unsupported inode size: 4096
It looks like this is a bug in the code? This check is using
3641: blocksize = sb_min_blocksize(sb, EXT4_MIN_BLOCK_SIZE);
3782: if ((sbi->s_inode_size < EXT4_GOOD_OLD_INODE_SIZE) ||
3783: (!is_power_of_2(sbi->s_inode_size)) ||
3784: (sbi->s_inode_size > blocksize)) {
3785: ext4_msg(sb, KERN_ERR,
3786: "unsupported inode size: %d",
3787: sbi->s_inode_size);
3788: goto failed_mount;
3789: }
which is set from the hardware sector size of the device, while the ext4 filesystem blocksize is not set until later during mount:
3991: blocksize = BLOCK_SIZE << le32_to_cpu(es->s_log_block_size);
It looks like this was just introduced in commitv5.4-rc3-96-g9803387 "ext4: validate the debug_want_extra_isize mount option at parse time" so it is a relatively recent change, and looks to be unintentional.
This check was previously on line 4033, after "blocksize" was updated from the superblock, but it wasn't noticed because it works for all "normal" filesystems.
I suspect nobody has noticed because having an inode *size* of 4KB is very unusual, while having an inode *ratio* of 4KB is more normal (one 256-byte inode for each 4096-byte block in the filesystem). Was the use of "-I 4096" intentional, or did you mean to use "-i 4096"?

Quoted 3 lines

The only reason to have a 4096-byte inode *size* is if you have a ton of xattrs for every file and/or you have tiny files (< 3.5KB) and you are using inline data.

Indeed the filesystems in question have a huge number of small files with lots and lots of xattrs for each file.

IIRC, back when I created them, I ran some tests iterating over various block and group sizes and simply chose the one with the best performance over a given testset.

Quoted 17 lines

Quoted 8 lines
Note: this works perfectly fine under 4.19.84 and 4.14.145.
My guess so far is that somehow the ext4 filesystem now checks that the inode size is not larger than the logical block size of the underlying block device.
 # cat /sys/block/loop0/queue/logical_block_size
 512
Yes, this appears to be the case. We have LOT of filesystems that are using 1024-byte inodes, but I suspect that most of them are on devices that report 4096-byte sector size and/or are running older kernels that have not included this bug.
Quoted 2 lines
Any ideas how to address this problem and get the file- systems to mount under Linux 5.5?

Quoted 6 lines

Probably the easiest, and likely correct, fix is to move the update of "blocksize" from line 3991: up to before this check.
There are a bunch of sanity checks that should also be moved for a proper patch, but the one-line fix is enough to get your filesystems mounting again.

Yes, I can confirm that moving line 3991 before the check fixes the issue and the test as well as the filesystem mount passes without problems.

Thanks a bunch for the quick and accurate information! Appreciated!

Quoted 1 line

Cheers, Andreas

All the best,
Herbert

Quoted 27 lines

--Apple-Mail=_7CDF64A9-C7FD-4E08-9AB4-1843C57439EC
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
iQIzBAEBCAAdFiEEDb73u6ZejP5ZMprvcqXauRfMH+AFAl48mUgACgkQcqXauRfM H+ASGg/8DycAju0NbXzVaKiOvovbvjZRyJq7nF6M+KBJevm0928uLjg8qkWvIdXp Jj1AM93mikp4A/BULggBBpa8wOCIG9Z7bx1tATaQrvQh/3cI5KuWd7ssfTR9INWJ yzgZ1Y/1vjwiU/YD1i922CK4M3sEwmB5fzrNC/H9HruwHpuMe0ek44lNmsuNPjGh c+hBkTFlmOPF9n9bW4mr2Da/v1BA+ffSI2NJW3TejR7k6UvvNKWpLrbzheMSMVCf y5xuD9mWuh/1FL77tdDfDVbPo6VRS6I1JBoz14EUl9mz6IrCWulVgIIi/7NzRviF onDLo/t3pA/2Yx5G+AAVsIM9tClXXGbNT4WquU2vrO9CdnuRT6rr1pc8vKCz7lch 2US+UhmorTVVd/NeXQMxT2i6NPNbRsoaBqxP5TcLAtp8b5aDAUCUSAHyIEWtoydm GRPRfXZJauqBYDffGdBWsvsMmepceMC4CMiezfoIWBbfnMfH8wVI+D3qEO6gLDkr sNm1/dl/7BfIFjF3ndItsgKTVCGIiFgQ86juEDwDwO/+UB9O9K7nngoEe0ZLt/sy Kn7RLdkOGR689vc/1WArbM31HntWbp88xTe3s2tPlWv4r9hVZebZXFIAYrwvqviS NrZwqOyjeAmlHWJcqaXQS7kV6tYDpT6Je7weNgZmQA1Xc7Ig12o= =rKs+
-----END PGP SIGNATURE-----
--Apple-Mail=_7CDF64A9-C7FD-4E08-9AB4-1843C57439EC-