ext4: delayed inode update for the consistency of file size after a crash

For one write() system call, the file size in the ext4_inode can be updated twice; at the write() system call and when dirty pages of the file are written to storage (i.g. fsync() or kworker thread). The write() system call that appends data to a file but does not need to allocate a block updates the page cache entry and marks the page as dirty. Then, the write() system call updates the file size in ext4_inode to the size of appended data and inserts the ext4_inode into a running transaction. After that, if the application calls an fsync(), the application thread dispatches the dirty page of the file and wakes up the JBD2. Once the JBD2 is wakened up, the JBD2 commits a running transaction. When the JBD2 commits the running transaction, it is possible that there are some ext4_inodes which file size is updated even though the dirty pages are not written to storage in the running transaction. If an unexpected power failure occurs after the fsync(), the EXT4 recovery module recovers the ext4_inodes in the journal area, but without the appropriate data blocks on disk.

Consider the following scenario. There are the two files, namely, fileA and fileB. The sizes of these two files are 14 KB. Mount options of EXT4 include the delayed allocation and the ordered mode journaling.

1. Current file offset of fileA is 14 KB. An application appends 2 KB data to fileA by executing a write() system call. At this time, the file size in the ext4_inode of fileA is updated to 16 KB by ext4_da_write_end().
2. Current file offset of fileB is 14 KB. An application appends 2 KB data to fileB by executing a write() system call. At this time, the file size in the ext4_inode of fileB is updated to 16 KB by ext4_da_write_end().
3. A fsync(fileB) is called before the kworker thread runs. At this time, the application thread transfers the data block of fileB to storage and wakes up the JBD2. Then, JBD2 writes the ext4_inodes of fileA and fileB in the running transaction to the journal area. The ext4_inode of fileA in the journal area has the file size, 16 KB, even though the data block of fileA has not been written to storage.
4. Assume that a system crash occurs. The EXT4 recovery module recovers the inodes of fileA and fileB. The recovered inode of fileA has the updated file size, 16 KB, even though the data of fileA has not been made durable. The data block of fileA between 14 KB and 16 KB is seen as zeros.

EXT4 updates the file size of the ext4_inode at ext4_da_write_end() so that both the time stamps and the file size of the ext4_inode get updated at the write() system call. This saves EXT4 from having to journal the ext4_inode again when dirty pages of the file are written to storage. If the file size in the ext4_inode is updated again when dirty pages of the file are written to storage, the ext4_inode can be inserted into two different transactions by the update of time stamps and the update of the file size respectively. This causes the amount of blocks written to the journal area to get increased. In order to address the problem that the ext4_inode has a wrong file size after an unexpected power failure, this commit does not update the file size in the ext4_inode at the write() system call, but delays the update of the file size in the ext4_inode until dirty pages of the file are written to storage (i.g. fsync() or kworker thread). So, ext4_inode can be inserted into two different transactions in this technique that this commit suggests. In order to keep the above EXT4 optimization, our technique delays journaling the ext4_inode in which the time stamps are updated until dirty pages of the file are written to storage. Therefore, the time stamps and the file size of the ext4_inode get updated at the same time in our technique as well.

Additionally, we should consider the kswapd behavior for this commit. When there is a memory pressure, the kswapd cleans dirty pages in the page cache. Consider that all dirty pages of a file are written to storage by the kswapd. The ext4_inode of the file is not journaled due to this commit. After the fsync() system call or the kswapd writes the dirty page to storage, the state of the page is changed to clean or writeback and the ext4_inode associated with the page is still not in the journal transaction and will not be inserted to the transaction ever after. In order to update and insert the ext4_inode into a running transaction in the above situation, I make the kswapd be re-dirty the last selected victim page among the dirty pages of a file.

This commit is applied to kernel 4.15-rc2.

Details can be found as follows.

Son et al. "Guaranteeing the Metadata Update Atomicity in EXT4 Filesystem”, In Proc. of APSYS 2017, Mumbai, India

---
 fs/ext4/ext4.h     |   4 ++
 fs/ext4/file.c     |  10 +++-
 fs/ext4/inode.c    | 170 ++++++++++++++++++++++++++++++++++++++++++-----------
 include/linux/fs.h |   1 +
 mm/filemap.c       |  88 +++++++++++++++++++++++++++
 5 files changed, 238 insertions(+), 35 deletions(-)

index 4e091ea..31aba55 100644
@@ -1045,6 +1045,9 @@ struct ext4_inode_info {
 	tid_t i_sync_tid;
 	tid_t i_datasync_tid;
 
+	/* Flag checking whether timestamps of ext4_inode is updated or not */
+	__u16 is_ts_updated;
+
 #ifdef CONFIG_QUOTA
 	struct dquot *i_dquot[MAXQUOTAS];
 #endif

@@ -2453,6 +2456,7 @@ extern void ext4_clear_inode(struct inode *);
 extern int  ext4_file_getattr(const struct path *, struct kstat *, u32, unsigned int);
 extern int  ext4_sync_inode(handle_t *, struct inode *);
 extern void ext4_dirty_inode(struct inode *, int);
+extern int ext4_update_time(struct inode *);
 extern int ext4_change_inode_journal_flag(struct inode *, int);
 extern int ext4_get_inode_loc(struct inode *, struct ext4_iloc *);
 extern int ext4_inode_attach_jinode(struct inode *inode);

index a0ae27b..d2ac76c 100644
@@ -263,7 +263,15 @@ ext4_file_write_iter(struct kiocb *iocb, struct iov_iter *from)
 		}
 	}
 
-	ret = __generic_file_write_iter(iocb, from);
+	/*
+	 * Update timestamps of ext4_inode without inserting the updated inode
+	 * into a transaction.
+	 */
+	ret = ext4_update_time(inode);
+	if (ret)
+		goto out;
+
+	ret = _generic_file_write_iter(iocb, from);
 	inode_unlock(inode);
 
 	if (ret > 0)

index 511fbaa..b001f65 100644
@@ -2931,6 +2931,7 @@ extern int generic_file_readonly_mmap(struct file *, struct vm_area_struct *);
 extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *);
 extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *);
 extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *);
+extern ssize_t _generic_file_write_iter(struct kiocb *, struct iov_iter *);
 extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *);
 extern ssize_t generic_file_direct_write(struct kiocb *, struct iov_iter *);
 extern ssize_t generic_perform_write(struct file *, struct iov_iter *, loff_t);

2.7.4

There's nothing wrong with this. The user space application called fsync on fileB, and *not* on fileA. Therefore, there is absolutely no guarantee that fileA's data contents are valid.

Consider the exact same thing will happen if the application had written data to fileA at offsets 6k to 8k. If those offsets were previously zero, then after the crash, those offsets *might* still be zero after the crash, *unless* the application had first called fsync() or fdatasync() first.

This is behind a paywall, so I can't access it. I am sorry I wasn't on the program committee, or I would have pointed this out while the paper was being reviewed.

The problem with providing more guarantees than what is strictly provided for by POSIX is that it degrades the performance of the file system. It can also promote application writes to depend on semantics which are non-portable, which can cause problems when they try to run that progam on other operating systems or other file systems.

Cheers,

- Ted