ext4: fix bug for rename with RENAME_WHITEOUT
PATCH v3 1/1 ext4: fix bug for rename with RENAME_WHITEOUT
We got a "deleted inode referenced" warning cross our fsstress test. The bug can be reproduced easily with following steps:
cd /dev/shm
mkdir test/
fallocate -l 128M img
mkfs.ext4 -b 1024 img
mount img test/
dd if=/dev/zero of=test/foo bs=1M count=128
mkdir test/dir/ && cd test/dir/
for ((i=0;i<1000;i++)); do touch file$i; done # consume all block cd ~ && renameat2(AT_FDCWD, /dev/shm/test/dir/file1, AT_FDCWD,
/dev/shm/test/dir/dst_file, RENAME_WHITEOUT) # ext4_add_entry in ext4_rename will return ENOSPC!!
cd /dev/shm/ && umount test/ && mount img test/ && ls -li test/dir/file1 We will get the output:
"ls: cannot access 'test/dir/file1': Structure needs cleaning" and the dmesg show:
"EXT4-fs error (device loop0): ext4_lookup:1626: inode #2049: comm ls:
deleted inode referenced: 139"
ext4_rename will create a special inode for whiteout and use this 'ino' to replace the source file's dir entry 'ino'. Once error happens latter(the error above was the ENOSPC return from ext4_add_entry in ext4_rename since all space has been consumed), the cleanup do drop the nlink for whiteout, but forget to restore 'ino' with source file. This will trigger the bug describle as above.
--- fs/ext4/namei.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
fs/ext4/namei.c +9 -8
@@ -3593,9 +3593,6 @@ static int ext4_setent(handle_t *handle, struct ext4_renament *ent,return retval2; } } -brelse (ent->bh); - ent->bh =NULL ; -return retval; }
@@ -3794,6 +3791,7 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, } } + old_file_type = old.de->file_type;if (IS_DIRSYNC (old.dir) ||IS_DIRSYNC (new.dir))ext4_handle_sync (handle);
@@ -3821,7 +3819,6 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, force_reread = (new.dir->i_ino == old.dir->i_ino &&ext4_test_inode_flag (new.dir,EXT4_INODE_INLINE_DATA )); - old_file_type = old.de->file_type;if (whiteout) {/* * Do this before adding a new entry, so the old entry is sure
@@ -3919,15 +3916,19 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, retval =0 ; end_rename: -brelse (old.dir_bh); -brelse (old.bh); -brelse (new.bh);if (whiteout) { -if (retval) +if (retval) { +ext4_setent (handle, &old, + old.inode->i_ino, old_file_type);drop_nlink (whiteout); + }unlock_new_inode (whiteout);iput (whiteout); + } +brelse (old.dir_bh); +brelse (old.bh); +brelse (new.bh);if (handle)ext4_journal_stop (handle);return retval;
Signature · 1 line
2.25.4
Quoted 21 lines · yangerkun
We got a "deleted inode referenced" warning cross our fsstress test. The bug can be reproduced easily with following steps:
cd
/dev/shm
mkdir test/
fallocate -l 128M img
mkfs.ext4 -b 1024 img
mount img test/
dd if=/dev/zero of=test/foo bs=1M count=128
mkdir test/dir/ && cd test/dir/
for ((i=0;i<1000;i++)); do touch file$i; done # consume all block cd ~ && renameat2(AT_FDCWD,/dev/shm/test/dir/file1,AT_FDCWD,
/dev/shm/test/dir/dst_file,RENAME_WHITEOUT) #ext4_add_entryinext4_renamewill return ENOSPC!!
cd/dev/shm/&& umount test/ && mount img test/ && ls -li test/dir/file1 We will get the output:
"ls: cannot access 'test/dir/file1': Structure needs cleaning" and the dmesg show:
"EXT4-fs error (device loop0): ext4_lookup:1626: inode #2049: comm ls:
deleted inode referenced: 139"
Quoted 8 lines
ext4_renamewill create a special inode for whiteout and use this 'ino' to replace the source file's dir entry 'ino'. Once error happens latter(the error above was the ENOSPC return fromext4_add_entryinext4_renamesince all space has been consumed), the cleanup do drop the nlink for whiteout, but forget to restore 'ino' with source file. This will trigger the bug describle as above.Signed-off-by yangerkun <[email protected]>
Thanks! The patch looks good to me now. You can add:
Honza
Quoted 61 lines
--- fs/ext4/namei.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/ext4/namei.c +9
-8@@ -3593,9 +3593,6 @@ static int ext4_setent(handle_t *handle, struct ext4_renament *ent,return retval2; } } -brelse (ent->bh); - ent->bh =NULL ; -return retval; }@@ -3794,6 +3791,7 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, } } + old_file_type = old.de->file_type;if (IS_DIRSYNC (old.dir) ||IS_DIRSYNC (new.dir))ext4_handle_sync (handle);@@ -3821,7 +3819,6 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, force_reread = (new.dir->i_ino == old.dir->i_ino &&ext4_test_inode_flag (new.dir,EXT4_INODE_INLINE_DATA )); - old_file_type = old.de->file_type;if (whiteout) {/* * Do this before adding a new entry, so the old entry is sure @@ -3919,15 +3916,19 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, retval =0 ; end_rename: -brelse (old.dir_bh); -brelse (old.bh); -brelse (new.bh);if (whiteout) { -if (retval) +if (retval) { +ext4_setent (handle, &old, + old.inode->i_ino, old_file_type);drop_nlink (whiteout); + }unlock_new_inode (whiteout);iput (whiteout); + } +brelse (old.dir_bh); +brelse (old.bh); +brelse (new.bh);if (handle)ext4_journal_stop (handle);return retval;Signature · 1 line
2.25.4
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Theodore Ts'o
maintainerQuoted 21 lines · yangerkun
We got a "deleted inode referenced" warning cross our fsstress test. The bug can be reproduced easily with following steps:
cd
/dev/shm
mkdir test/
fallocate -l 128M img
mkfs.ext4 -b 1024 img
mount img test/
dd if=/dev/zero of=test/foo bs=1M count=128
mkdir test/dir/ && cd test/dir/
for ((i=0;i<1000;i++)); do touch file$i; done # consume all block cd ~ && renameat2(AT_FDCWD,/dev/shm/test/dir/file1,AT_FDCWD,
/dev/shm/test/dir/dst_file,RENAME_WHITEOUT) #ext4_add_entryinext4_renamewill return ENOSPC!!
cd/dev/shm/&& umount test/ && mount img test/ && ls -li test/dir/file1 We will get the output:
"ls: cannot access 'test/dir/file1': Structure needs cleaning" and the dmesg show:
"EXT4-fs error (device loop0): ext4_lookup:1626: inode #2049: comm ls:
deleted inode referenced: 139"
Quoted 8 lines
ext4_renamewill create a special inode for whiteout and use this 'ino' to replace the source file's dir entry 'ino'. Once error happens latter(the error above was the ENOSPC return fromext4_add_entryinext4_renamesince all space has been consumed), the cleanup do drop the nlink for whiteout, but forget to restore 'ino' with source file. This will trigger the bug describle as above.Signed-off-by yangerkun <[email protected]>
Thanks, replied.
- Ted
Quoted 23 lines · Theodore Ts'o
Quoted 21 lines · yangerkun
We got a "deleted inode referenced" warning cross our fsstress test. The bug can be reproduced easily with following steps:
cd
/dev/shm
mkdir test/
fallocate -l 128M img
mkfs.ext4 -b 1024 img
mount img test/
dd if=/dev/zero of=test/foo bs=1M count=128
mkdir test/dir/ && cd test/dir/
for ((i=0;i<1000;i++)); do touch file$i; done # consume all block cd ~ && renameat2(AT_FDCWD,/dev/shm/test/dir/file1,AT_FDCWD,
/dev/shm/test/dir/dst_file,RENAME_WHITEOUT) #ext4_add_entryinext4_renamewill return ENOSPC!!
cd/dev/shm/&& umount test/ && mount img test/ && ls -li test/dir/file1 We will get the output:
"ls: cannot access 'test/dir/file1': Structure needs cleaning" and the dmesg show:
"EXT4-fs error (device loop0): ext4_lookup:1626: inode #2049: comm ls:
deleted inode referenced: 139"
Quoted 9 lines
Quoted 8 lines
ext4_renamewill create a special inode for whiteout and use this 'ino' to replace the source file's dir entry 'ino'. Once error happens latter(the error above was the ENOSPC return fromext4_add_entryinext4_renamesince all space has been consumed), the cleanup do drop the nlink for whiteout, but forget to restore 'ino' with source file. This will trigger the bug describle as above.Signed-off-by yangerkun <[email protected]>
Apropos RENAME_WHITEOUT, it seems to be missing __ext4_fc_track_link(). I guess test coverage of RENAME_WHITEOUT in fstests is not much. I have been seeing trickles of bug fixes for RENAME_WHITEOUT for almost every filesystem that supports it.
But I must say it would have been very hard to catch missing ext4_fc_track_* without specialized fs fuzzer such as the CrashMonkey generated tests.
And as long as I am ranting, I'd like to point out that it is a shame that whiteout
was not implemented as a special (constant) inode whose nlink is irrelevant (or a special dirent with d_ino 0 and d_type DT_WHT for that matter). It would have been a rather small RO_COMPAT on-disk change for ext4. It could also be implemented in slightly more backward compat manner by maintaining a valid nlink and postpone setting the RO_COMPAT flag until EXT4_LINK_MAX is reached.
As things stand now, overlayfs makes an effort to maintain a singleton hardlinked whiteout inode, without being able to use it with RENAME_WHITEOUT and filesystems have to take special care to journal the metadata of all individual whiteout inodes, without any added value to the only user (overlayfs).
But I guess that train has left the station long ago...
Thanks,
Amir.
Quoted 9 lines · Amir Goldstein
And as long as I am ranting, I'd like to point out that it is a shame that whiteout
was not implemented as a special (constant) inode whose nlink is irrelevant (or a special dirent withd_ino0 andd_typeDT_WHTfor that matter). It would have been a rather smallRO_COMPATon-disk change for ext4. It could also be implemented in slightly more backward compat manner by maintaining a valid nlink and postpone setting theRO_COMPATflag untilEXT4_LINK_MAXis reached.
Quoted 7 lines
As things stand now, overlayfs makes an effort to maintain a singleton hardlinked whiteout inode, without being able to use it with
RENAME_WHITEOUTand filesystems have to take special care to journal the metadata of all individual whiteout inodes, without any added value to the only user (overlayfs).But I guess that train has left the station long ago...
Not so, I believe. Kernel internal interfaces are easy to change, and adding support for DT_WHT to overlayfs would mostly be a trivial undertaking.
The big issue (as always) is userspace API's and not introducing DT_WHT there was a very deliberate choice. Adding a translation layer from an internal whiteout representation to the userspace API also does not seem to be a very complex problem, but I haven't looked into that deeply.
So AFAICS there's really nothing preventing the addition of whiteout objects to filesystems, other than developer dedication.
Thanks,
Miklos
Thanks Amir for pointing that out. Yes we are missing fast commit tracking in whiteout. I'll send out a fix for that.
Quoted 2 lines
But I must say it would have been very hard to catch missing ext4_fc_track_* without specialized fs fuzzer such as the CrashMonkey generated tests.
I agree, it's been on my to-do list to run CrashMonkey tests with fast commits. I'm curious what kind of CrashMonkey tests you ran that helped you catch this? Were you running Overlayfs on top of Ext4 with fast commits?
Thanks,
Harshad
Quoted 35 lines · Miklos Szeredi
Quoted 9 lines · Amir Goldstein
And as long as I am ranting, I'd like to point out that it is a shame that whiteout
was not implemented as a special (constant) inode whose nlink is irrelevant (or a special dirent withd_ino0 andd_typeDT_WHTfor that matter). It would have been a rather smallRO_COMPATon-disk change for ext4. It could also be implemented in slightly more backward compat manner by maintaining a valid nlink and postpone setting theRO_COMPATflag untilEXT4_LINK_MAXis reached.Quoted 7 lines
As things stand now, overlayfs makes an effort to maintain a singleton hardlinked whiteout inode, without being able to use it with
RENAME_WHITEOUTand filesystems have to take special care to journal the metadata of all individual whiteout inodes, without any added value to the only user (overlayfs).But I guess that train has left the station long ago...
Not so, I believe. Kernel internal interfaces are easy to change, and adding support for
DT_WHTto overlayfs would mostly be a trivial undertaking.The big issue (as always) is userspace API's and not introducing
DT_WHTthere was a very deliberate choice. Adding a translation layer from an internal whiteout representation to the userspace API also does not seem to be a very complex problem, but I haven't looked into that deeply.So AFAICS there's really nothing preventing the addition of whiteout objects to filesystems, other than developer dedication.
Thanks,
Miklos
Quoted 4 lines · harshad shirwadkar
Thanks Amir for pointing that out. Yes we are missing fast commit tracking in whiteout. I'll send out a fix for that.
Quoted 8 lines
Quoted 2 lines
But I must say it would have been very hard to catch missing ext4_fc_track_* without specialized fs fuzzer such as the CrashMonkey generated tests.
I agree, it's been on my to-do list to run CrashMonkey tests with fast commits. I'm curious what kind of CrashMonkey tests you ran that helped you catch this? Were you running Overlayfs on top of Ext4 with fast commits?
Neither. I just guessed RENAME_WHITEOUT might be missed as developers are rarely aware of it.
I never ran CrashMonkey tests myself.
I found a few crash consistency bugs using xfstest generic/455. I suggest that you run it with fast commits
and try using NUM_OPS and NUM_FILES larger than the test defaults to let the test run for a longer time.
Thanks,
Amir.
Quoted 4 lines · harshad shirwadkar
Thanks Amir for pointing that out. Yes we are missing fast commit tracking in whiteout. I'll send out a fix for that.
Ping.
Harshad,
Did you forget or did I miss the patch?
Thanks,
Amir.