Emergency remount readonly and EFBIG errors when unlinking files on 3.18 android kernel
Hi
As you know Android uses emergency remount instead of doing something like "umount -a" in its shutdown/reboot path.
https://android.googlesource.com/platform/system/core/+/master/libcutils/android_reboot.c#132
I have seen a strange issue that sometimes occurs when there are a large number of writes to an ext4 file system and an adb reboot is issued ( triggering an emergency remount readonly and a reboot)
Teh issue doesnt happen all the writer processes are killed before the emergency remount
And on disk we see that one of the files being written to has incorrect ext4_inode->i_blocks_lo ( which is less than the the size of the file by something like 2k)
When unlinking this file the vfs inode->iblocks underflows and we end up with EFBIG if EXT4_FEATURE_RO_COMPAT_HUGE_FILE is not enabled in the superblock.
Is this a known issue?
I am still trying to figure out why we have a incorrect i_blocks_lo on the disk.
Running fsck on the partition does fix the issue but i am trying to figure out why this would happen and how to fix it.
I would appreciate if you could point me in the right direction and any help you can give me.
Signature · 6 lines
Thanks Nikhilesh Reddy Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Quoted 2 lines · Nikhilesh Reddy
As you know Android uses emergency remount instead of doing something like "umount -a" in its shutdown/reboot path.
No, I didn't know that. (And I wish I didn't. Yelch, what an ugly hack.)
Quoted 2 lines
Teh issue doesnt happen all the writer processes are killed before the emergency remount
Is there a missing "if", as in "if all the writer processes".... ?
Note that an emergency remount is very much an emergency. So we don't do a graceful shutdown of any pending writes. (Normally we would return EBUSY if there anything that would prevent a clean remount.) In the emergency remount path, we bypass all of these checks.
Quoted 9 lines
And on disk we see that one of the files being written to has incorrect
ext4_inode->i_blocks_lo( which is less than the the size of the file by something like 2k)When unlinking this file the vfs
inode->iblocksunderflows and we end up with EFBIG ifEXT4_FEATURE_RO_COMPAT_HUGE_FILEis not enabled in the superblock.Is this a known issue?
No, this isn't a known issue. I've never seen anything like this, but all of the tests we do assume a forced poweroff, which we simulate using dm-flakey. We do *not* test the blunt-force-trauma which is inflected on the file system structures which results from doing an emergency remount.
Off by 2k really doesn't make sense. I could see if it was off by 4k, but 2k is really wierd.
Quoted 2 lines
I would appreciate if you could point me in the right direction and any help you can give me.
Well, what I'd do is create a new ioctl interface which simulates an emergency ro on just the one device, and try to create a reliable repro. Eventually we'll want to add some tests for this in xfstests.
- Ted
Quoted 7 lines · Theodore Ts'o
Quoted 2 lines · Nikhilesh Reddy
I would appreciate if you could point me in the right direction and any help you can give me.
Well, what I'd do is create a new ioctl interface which simulates an emergency ro on just the one device, and try to create a reliable repro. Eventually we'll want to add some tests for this in xfstests.
That's pretty much XFS_IOC_GOINGDOWN, controlled by xfs_io -c shutdown. I'd suggest that adding a new flag:
#define XFS_FSOP_GOING_FLAGS_EMERG_REMOUNT 0x4
Would be in line with it's existing usage in xfstests for forcing different shutdown conditions on filesystems....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Hi
Thanks for your reply
Yes the sentence below should have the *if* .. sorry about the typo The issue doesnt happen if all the writer processes/threads are killed before the emergency remount
Quoted 16 lines
Note that an emergency remount is very much an emergency. So we don't do a graceful shutdown of any pending writes. (Normally we would return EBUSY if there anything that would prevent a clean remount.) In the emergency remount path, we bypass all of these checks.
Quoted 9 lines
And on disk we see that one of the files being written to has incorrect
ext4_inode->i_blocks_lo( which is less than the the size of the file by something like 2k)When unlinking this file the vfs
inode->iblocksunderflows and we end up with EFBIG ifEXT4_FEATURE_RO_COMPAT_HUGE_FILEis not enabled in the superblock.Is this a known issue?
Quoted 8 lines
No, this isn't a known issue. I've never seen anything like this, but all of the tests we do assume a forced poweroff, which we simulate using dm-flakey. We do *not* test the blunt-force-trauma which is inflected on the file system structures which results from doing an emergency remount.
Off by 2k really doesn't make sense. I could see if it was off by 4k, but 2k is really wierd.
Just to clarify when i say off by 2k .. i meant the i_blocks count not the actual size file ( which would be 2k * 512 if i am not wrong)
For example we see fsck report
Pass 1: Checking inodes, blocks, and sizes
Inode XXX, i_blocks is 854024, should be 856072.
Quoted 8 lines
Quoted 3 lines
I would appreciate if you could point me in the right direction and any help you can give me.
Well, what I'd do is create a new ioctl interface which simulates an emergency ro on just the one device, and try to create a reliable repro. Eventually we'll want to add some tests for this in xfstests.
Thanks so much for your suggestion.
I will try to see if i can reliably reproduce the issue after implementing the ioctl as you suggested.
I have some issues getting the xfs tests to run on the device which i have been meaning to work on .. maybe this is the time to do so.
Signature · 6 lines
Thanks Nikhilesh Reddy Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.