Skip to content

fs: Fix missed inode write during fsync

Jan Kara linux-ext4 linux-fsdevel stable 5 versions – Applied

Status from this message: Applied to the vfs-7.3.sync branch of the vfs/vfs.git tree.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest20 patches 4 repliesApplied
  2. v421 patches 3 repliesApplied
  3. v319 patches 5 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v210 patches4 repliesSuperseded
v19 patches9 repliesSuperseded

Changes in v5 from the cover letter

* Fixed where ext4 calls set_inode_metadata_writeback() so that metadata staged directly with ext4_mark_inode_dirty() gets properly tracked for writeout with fsync(2)
* Added __GFP_ACCOUNT for ext4 mapping_metadata_bh struct allocation to match how memory was tracked when embedded in the inode
* Removed cleanup patch removing sync_dirty_buffers() calls from FAT. The error handling would need more work and things get too complicated for this series.
* Improved error reporting for ext2
* Fixed typo in bfs leading to compilation failure

Patches in v5 Applied

#Subject and filesSizeRepliesApplied
1/20affs: Drop support for metadata bh trackingfs/affs/affs.h fs/affs/amigaffs.c fs/affs/file.c +3 more+26 -360 replies
2/20fs: Fix possible UAF in mark_buffer_write_io_error()fs/buffer.c+8 -20 replies
3/20fs: Fix missed inode writeback when racing with __writeback_single_inodefs/buffer.c fs/libfs.c+6 -40 replies
4/20ext4: Allocate mapping_metadata_bhs struct on demandfs/ext4/ext4.h fs/ext4/ext4_jbd2.c fs/ext4/fsync.c +2 more+55 -160 replies
5/20fs: Provide way for filesystem to wait for metadata writebackfs/fs-writeback.c fs/libfs.c include/linux/fs.h +1 more+41 -100 replies
6/20ext2: Fix lost inode updates for IS_SYNC inodesfs/ext2/inode.c fs/ext2/xattr.c+4 -70 replies
7/20ext2: Drop __ext2_write_inode()fs/ext2/inode.c+8 -100 replies
8/20ext2: Avoid unnecessary inode buffer writeback for sync(2)fs/ext2/inode.c+5 -10 replies
9/20ext2: Fix data integrity writeout issuesfs/ext2/dir.c fs/ext2/ext2.h fs/ext2/file.c +2 more+34 -370 replies
10/20udf: Fix data integrity writeout issuesfs/udf/dir.c fs/udf/file.c fs/udf/inode.c +2 more+33 -210 replies
11/20udf: Use sync_inode_metadata() to writeout IS_SYNC inodefs/udf/file.c fs/udf/inode.c+3 -40 replies
12/20udf: Drop udf_sync_inode()fs/udf/inode.c+1 -100 replies
13/20udf: Use sync_inode_metadata() in udf_evict_inode()fs/udf/inode.c+1 -30 replies
14/20udf: Fold udf_update_inode() into udf_write_inode()fs/udf/inode.c+1 -70 replies
15/20bfs: Fix data integrity writeout issuesfs/bfs/dir.c fs/bfs/inode.c+25 -140 replies
16/20minix: Fix data integrity writeout issuesfs/minix/dir.c fs/minix/file.c fs/minix/inode.c +1 more+37 -270 replies
17/20ext4: Fix data integrity writeout issues in nojournal modefs/ext4/ext4.h fs/ext4/fsync.c fs/ext4/inode.c +1 more+72 -540 replies
18/20fat: Fix missed inode writeback during fsync(2)fs/fat/file.c fs/fat/inode.c+45 -120 replies
19/20fat: Replace fat_sync_inode() with sync_inode_metadata()fs/fat/dir.c fs/fat/fat.h fs/fat/file.c +4 more+34 -500 replies
20/20vfs: Remove mmb_fsync()fs/buffer.c include/linux/buffer_head.h+0 -780 replies

Reviews trailers per patch

2 of 20 patches have a Reviewed-by or Acked-by.

PatchDavid SterbaTheodore Ts'o
1/20A-b
2/20
3/20
4/20A-b
5/20
6/20
7/20
8/20
9/20
10/20
11/20
12/20
13/20
14/20
15/20
16/20
17/20
18/20
19/20
20/20

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v5 thread

Cover letter [PATCH v5 0/20] fs: Fix missed inode write during fsync

Hello,

here is v5 of the patch series which fixes the possibly missing inode write during fsync(2) for filesystems using generic metadata bh tracking. This revision contains mostly small fixes and improvements since v3, neither Sashiko nor Christian found fundamental issues with this approach.

Patches have survived some testing with fstests so they should be reasonably sound. I'd just like Ogawa to have a look at FAT changes as they are not completely trivial (although I've opted for a less cleanups than in v4).

Changes since v4:
* Fixed where ext4 calls set_inode_metadata_writeback() so that metadata staged directly with ext4_mark_inode_dirty() gets properly tracked for writeout with fsync(2)
* Added __GFP_ACCOUNT for ext4 mapping_metadata_bh struct allocation to match how memory was tracked when embedded in the inode
* Removed cleanup patch removing sync_dirty_buffers() calls from FAT. The error handling would need more work and things get too complicated for this series.
* Improved error reporting for ext2
* Fixed typo in bfs leading to compilation failure

Changes since v3:
* Add a fix for ext2 lost inode updates for IS_SYNC inodes (independent issue found during testing)
* Check I_METADATA_WRITEBACK in simple_fsync_noflush() to avoid skipping inodes with pending bh writeout
* Reorder buffer writeback in to first write out metadata blocks and only then inodes. It should lead to a more consistent filesystem in case of a crash.
* Added handling of IO errors from metadata writeback for IS_SYNC inodes for FAT
* Fix inode writeout during eviction for ext2
* Fix conversion of ext4_nfs_commit_metadata() to work for filesystems with a journal
* Fix races when handling of IS_SYNC inodes for UDF by using sync_inode_metadata()
* Add ext4 helper for accessing i_metadata_bhs
* Fix some bisectability hazards
* Removed some dead branches after FAT conversion

Changes since v2:
* redesigned the patch set to use new superblock operation
* fixed race of inode freeing with IO error processing in mark_buffer_write_io_error()

Changes since v1:
* Fixed freeing for ext4 dynamically allocated mmb struct
* Optimized tracking of block carrying the inode so that we don't flush it unnecessarily on fsync
* Add forgotten check for reclaimed bh to mmb_sync() to avoid NULL ptr deref
* Couple other smaller fixups pointed out by Sashiko

Honza

Original design explanation:

The series strives to make sure sync_inode_metadata() -> writeback_single_inode() will end up to properly persisting not only the inode but also all metadata associated with the inode. This is beneficial for several reasons:

1) This removes the need for separate mmb_fsync() implementations. Filesystems can just use simple_fsync().

2) This makes sure all metadata is written for IS_SYNC / IS_DIRSYNC inodes. Currently most filesystems just write inode itself.

3) This fixes races when multiple fsyncs race for a while where mmb_sync() could return before all buffers were really persisted (now I_SYNC state flag properly serializes everything).

To be able to achieve this we add new .sync_inode_metadata superblock operation which gets called from __writeback_single_inode() and new I_METADATA_WRITEBACK state flag (putting everything into .write_inode is possible but it looked too ugly so I've decided for the new operation).

This scheme with I_METADATA_WRITEBACK state flag also fixes the problem that when WB_SYNC_NONE writeback happened between write(2) and fsync(2), fsync(2) would fail to properly persist the inode. Note that this problem isn't specific to filesystems using the generic metadata bh tracking. This patch set doesn't fix such filesystems as it is already big enough but the universality of this problem among simple filesystems is another reason why I've chosen the new superblock operation instead of trying to somehow hack up metadata bh tracking infrastructure.

Previous versions: