New: fsck.ext4: unable to set superblock flags, even after mounting ok
https://bugzilla.kernel.org/show_bug.cgi?id=196677
Bug ID: 196677
Summary: fsck.ext4: unable to set superblock flags, even after
mounting ok
Product: File System
Version: 2.5
Kernel Version: 4.12.4
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: ext4
Assignee: [email protected]
Reporter: [email protected]
Regression: NoJust had a Raspberry Pi crash, and upon reboot fsck.ext4 was unable to repair the filesystem. Moving the SD card to a PC and running fsck there showed the same behaviour. I am not sure what exactly you need to know so please advise if you need anything else.
The reason for logging this bug is that I can't work out why fsck.ext4 is telling me "unable to set superblock" when I can successfully mount the filesystem and write to it without problem, however I can't boot the Raspberry Pi from it because it tries to run fsck on every boot which fails, falling back to single user mode instead of booting properly. I also don't understand why the old journal apparently remains even after cleanly unmounting the filesystem, making it look like there are still errors and causing fsck to keep failing.
$ fsck.ext4 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Run journal anyway<y>? yes fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ fsck.ext4 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Run journal anyway<y>? no Clear journal<y>? yes fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ fsck.ext4 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Run journal anyway<y>? no Clear journal<y>? no fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ mkfs.ext4 -n /dev/sdb2
mke2fs 1.43.4 (31-Jan-2017)
/dev/sdb2 contains a ext4 file system
last mounted on / on Sun Aug 13 19:58:40 2017
Proceed anyway? (y,N) y
Creating filesystem with 3885568 4k blocks and 972944 inodes
Filesystem UUID: b0aabda8-ecd4-49a7-8d49-847fdf66b795
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208$ fsck.ext4 -b 98304 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ fsck.ext4 -b 163840 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. Superblock needs_recovery flag is clear, but journal has data. Recovery flag not set in backup superblock, so running journal anyway. fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ mkdir temp $ mount /dev/sdb2 temp $ ls temp total 80K drwxr-xr-x 17 root root 4.0K Jul 26 11:29 . drwxr-x--- 29 root root 4.0K Aug 16 18:05 .. lrwxrwxrwx 1 root root 7 Apr 1 11:34 bin -> usr/bin drwxr-xr-x 2 root root 4.0K Mar 11 2016 boot drwxr-xr-x 2 root root 4.0K Mar 1 2016 dev drwxr-xr-x 45 root root 4.0K Aug 16 15:44 etc drwxr-xr-x 3 root root 4.0K Mar 11 2016 home lrwxrwxrwx 1 root root 7 Apr 1 11:34 lib -> usr/lib drwx------ 2 root root 16K Mar 11 2016 lost+found drwxr-xr-x 2 root root 4.0K Oct 6 2015 mnt drwxr-xr-x 3 root root 4.0K Mar 1 2016 opt dr-xr-xr-x 2 root root 4.0K Mar 1 2016 proc drwxr-x--- 8 root root 4.0K Aug 16 15:44 root drwxr-xr-x 2 root root 4.0K Mar 1 2016 run lrwxrwxrwx 1 root root 7 Apr 1 11:34 sbin -> usr/bin drwxr-xr-x 4 root root 4.0K Oct 6 2015 srv dr-xr-xr-x 2 root root 4.0K Mar 1 2016 sys drwxrwxrwt 2 root root 4.0K Mar 1 2016 tmp drwxr-xr-x 8 root root 4.0K Aug 16 15:44 usr drwxr-xr-x 12 root root 4.0K Jul 26 11:29 var
$ umount temp $ fsck.ext4 /dev/sdb2 e2fsck 1.43.4 (31-Jan-2017) /dev/sdb2: recovering journal Superblock needs_recovery flag is clear, but journal has data. Run journal anyway<y>? no Clear journal<y>? yes fsck.ext4: unable to set superblock flags on /dev/sdb2
/dev/sdb2: ********** WARNING: Filesystem still has errors **********
$ dmesg | tail [881102.628229] scsi host7: usb-storage 3-1:1.0 [881103.655218] scsi 7:0:0:0: Direct-Access Generic STORAGE DEVICE 0821 PQ: 0 ANSI: 6 [881103.655493] sd 7:0:0:0: Attached scsi generic sg2 type 0 [881104.004041] sd 7:0:0:0: [sdb] 31291392 512-byte logical blocks: (16.0 GB/14.9 GiB) [881104.004576] sd 7:0:0:0: [sdb] Write Protect is off [881104.004579] sd 7:0:0:0: [sdb] Mode Sense: 23 00 00 00 [881104.005161] sd 7:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [881104.008549] sdb: sdb1 sdb2 [881104.010750] sd 7:0:0:0: [sdb] Attached SCSI removable disk [881151.920150] EXT4-fs (sdb2): recovery complete [881151.920669] EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null) [881178.944509] EXT4-fs (sdb2): recovery complete [881178.944973] EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null) [881658.714126] EXT4-fs (sdb2): recovery complete [881658.714609] EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null) [882732.745664] EXT4-fs (sdb2): recovery complete [882732.746169] EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null)
$ strace fsck.ext4 -y /dev/sdb2 ... write(1, "Run ", 4Run ) = 4 write(1, "journal", 7journal) = 7 write(1, " anyway", 7 anyway) = 7 write(1, "? yes\n\n", 7? yes
) = 7pwrite64(4, "\266\3\0\0\306\3\0\0\326\3\0\0\21#\227\24$\1\4\0\0\0\0\0\0\0\0\0R\v\343\273"..., 4096, 4096) = 4096pwrite64(4, "\300;9\230\0\0\0\4\0\0\0\0\0\0\20\0\0\0\200\0\0\0\0\1\0\0\221o\0\0@,"..., 4096, 6580559872) = 4096
fsync(4) = 0
lseek(4, 1072, SEEK_SET) = 1072
write(4, "\247\375\223Y", 4) = 4
lseek(4, 1400, SEEK_SET) = 1400
write(4, "E\321", 2) = 2
fsync(4) = 0
close(4) = 0
open("/dev/sdb2", O_RDWR|O_EXCL) = 4
getuid() = 0
geteuid() = 0
getgid() = 0
getegid() = 0
prctl(PR_GET_DUMPABLE) = 1 (SUID_DUMP_USER)fstat(4, {st_mode=S_IFBLK|0660, st_rdev=makedev(8, 18), ...}) = 0ioctl(4, BLKDISCARDZEROES, [0]) = 0ioctl(4, BLKROGET, [0]) = 0uname({sysname="Linux", nodename="korath.teln.shikadi.net", ...}) = 0 pread64(4, "\220\330\16\0\0J;\0\346\366\2\0\233\217,\0\312\311\r\0\0\0\0\0\2\0\0\0\2\0\0\0"..., 1024, 1024) = 1024pread64(4, "\266\3\0\0\306\3\0\0\326\3\0\0\21#\227\24$\1\4\0\0\0\0\0\0\0\0\0R\v\343\273"..., 4096, 4096) = 4096
pread64(4, "\0\0\0\0\0\0\0\0\330\241\342V\330\241\342V\330\241\342V\0\0\0\0\0\0\0\0\0\0\0\0"..., 4096, 4022272) = 4096pread64(4, "\300;9\230\0\0\0\4\0\0\0\0\0\0\20\0\0\0\200\0\0\0\0\1\0\0\221o\0\0@,"..., 4096, 6580559872) = 4096write(2, "fsck.ext4", 9fsck.ext4) = 9write(2, ": ", 2: ) = 2
write(2, "unable to set superblock flags o"..., 44unable to set superblock flags on /dev/sdb2
) = 44
...
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 196677] fsck.ext4: unable to set superblock flags, even after mounting ok
https://bugzilla.kernel.org/show_bug.cgi?id=196677
--- Comment #1 from Theodore Tso ([email protected]) ---
Quoted 5 lines · On Wed, Aug 16, 2017 at 08:14:26AM +0000, [email protected]…
Just had a Raspberry Pi crash, and upon reboot fsck.ext4 was unable to repair the filesystem. Moving the SD card to a PC and running fsck there showed the same behaviour. I am not sure what exactly you need to know so please advise if you need anything else.
This is a e2fsprogs bugs, not a kernel bug, but it's fine to discuss it here.
Can you send me the output of "dumpe2fs /dev/sdb2", "debugfs -R 'stat <8>' /dev/sdb2", and "debugfs -R logdump /dev/sdb2"?
Also could you run:
debugfs -R 'dump_inode <8> /tmp/journal.dat' /dev/sdb2
and then send me a copy of /tmp/journal.dat?
I'm guessing the journal must be corrupted in some interesting way, since what should have happened if it was just a case of the journal data being present but the superblock flags not being set sould have been:
e2fsck 1.43.5 (04-Aug-2017)/tmp/foo.img: recovering journal
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong (26854, counted=24490).
Fix? yes
Free inodes count wrong (8181, counted=7281).
Fix? yes
/tmp/foo.img: ***** FILE SYSTEM WAS MODIFIED ***** /tmp/foo.img: 911/8192 files (0.0% non-contiguous), 8278/32768 blocks
Cheers,
- Ted
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.
[Bug 196677] fsck.ext4: unable to set superblock flags, even after mounting ok
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=196677
Adam Nielsen ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |INVALID--- Comment #2 from Adam Nielsen ([email protected]) ---
Just to follow up on this, after performing a couple of troubleshooting steps suggested by Ted, it became apparent that the SD card containing the ext4 filesystem has become read only.
Writes succeed without error, but the original data is always returned unmodified.
For the benefit of anyone else finding their way here, I was able to verify the problem by making a copy of the filesystem on another device and running fsck on the copy, which succeeded.
To avoid needing a lot of disk space for the copy, I used e2image to copy the metadata into a sparse file:
$ e2image -r /dev/sdb2 sdb2.e2i e2image 1.43.4 (31-Jan-2017)
$ fsck.ext4 sdb2.e2i e2fsck 1.43.4 (31-Jan-2017) sdb2.e2i: recovering journal Setting free inodes count to 902960 (was 903626) Setting free blocks count to 2918014 (was 2920347) sdb2.e2i: clean, 69984/972944 files, 967554/3885568 blocks
This only took 180MB of on-disk space instead of the full 15GB.
So the underlying cause of this bug is a hardware failure on the SD card. It appears to have gone into some sort of failsafe read-only mode, but unfortunately it doesn't reject write operations, it just discards them. This means the host (and also the user, thanks to caching) does not realise there is a problem unless, like fsck, a program does a verify-after-write and reveals the issue.
Signature · 2 lines
You are receiving this mail because: You are watching the assignee of the bug.