Skip to contentSkip to conversation

ext4: do not mark inode dirty which is already dirtied under append-write scenario

PATCH In review Liu Song 2 participants 4 messages Jul 11, 2023 – Aug 10, 2023 ext4 file system

lore
Share & tools
lore

Liu Song

author
#0 lore raw

PATCH 1/1 ext4: do not mark inode dirty which is already dirtied under append-write scenario

In the append-write scenario, after ensuring that the dirty inode can be seen by the writeback process, there is no need to execute "mark_inode_dirty" for every write. Instead, we can rely on "ext4_mark_inode_dirty" executed when updating i_disksize in "mpage_map_and_submit_extent" to ensure data consistency, which can significantly improve performance in high-frequency append-write scenarios.

In test scenarios of Kafka version 2.6.2, using packet size of 2K resulted in a 10% performance improvement.

---
 fs/ext4/inode.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 56 insertions(+), 2 deletions(-)
fs/ext4/inode.c +56 -2
index 9d9f414f99fe..d1aa775c9936 100644
@@ -3128,6 +3128,57 @@ static int ext4_da_should_update_i_disksize(struct page *page,
 	return 1;
 }
 
+/*
+ * Copy from generic_write_end, add conditions to execute mark_inode_dirty
+ * to avoid additional overhead caused by frequent dirty inode operations
+ */
+static int ext4_da_generic_write_end(struct file *file, struct address_space *mapping,
+			loff_t pos, unsigned len, unsigned copied, bool need_dirty,
+			struct page *page, void *fsdata)
+{
+	struct inode *inode = mapping->host;
+	loff_t old_size = inode->i_size;
+	bool i_size_changed = false;
+	int was_dirty;
+
+	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
+
+	/*
+	 * No need to use i_size_read() here, the i_size cannot change under us
+	 * because we hold i_rwsem.
+	 *
+	 * But it's important to update i_size while still holding page lock:
+	 * page writeout could otherwise come in and zero beyond i_size.
+	 */
+	if (pos + copied > inode->i_size) {
+		i_size_write(inode, pos + copied);
+		i_size_changed = true;
+	}
+
+	unlock_page(page);
+	put_page(page);
+
+	if (old_size < pos)
+		pagecache_isize_extended(inode, old_size, pos);
+
+	/*
+	 * In the append-write scenario, if the inode is marked as dirty,
+	 * it is ensured that the inode will be seen by the writeback process.
+	 * In the ext4_writepages process, when updating i_disksize,
+	 * corresponding metadata updates are also performed.
+	 * Therefore, it is unnecessary to repeatedly execute mark_inode_dirty
+	 * to improve performance.
+	 */
+	if (i_size_changed) {
+		spin_lock(&inode->i_lock);
+		was_dirty = inode->i_state & I_DIRTY;
+		spin_unlock(&inode->i_lock);
+		if (!was_dirty || need_dirty)
+			mark_inode_dirty(inode);
+	}
+	return copied;
+}
+
 static int ext4_da_write_end(struct file *file,
 			     struct address_space *mapping,
 			     loff_t pos, unsigned len, unsigned copied,
@@ -3137,6 +3188,7 @@ static int ext4_da_write_end(struct file *file,
 	loff_t new_i_size;
 	unsigned long start, end;
 	int write_mode = (int)(unsigned long)fsdata;
+	bool need_dirty = false;
 
 	if (write_mode == FALL_BACK_TO_NONDELALLOC)
 		return ext4_write_end(file, mapping, pos,
@@ -3169,10 +3221,12 @@ static int ext4_da_write_end(struct file *file,
 	 */
 	new_i_size = pos + copied;
 	if (copied && new_i_size > inode->i_size &&
-	    ext4_da_should_update_i_disksize(page, end))
+	    ext4_da_should_update_i_disksize(page, end)) {
 		ext4_update_i_disksize(inode, new_i_size);
+		need_dirty = true;
+	}
 
-	return generic_write_end(file, mapping, pos, len, copied, page, fsdata);
+	return ext4_da_generic_write_end(file, mapping, pos, len, copied, need_dirty, page, fsdata);
 }
 
 /*
Signature · 1 line
2.19.1.6.gb485710b
author
#1 lore raw

PATCH 1/1 ext4: do not mark inode dirty which is already dirtied under append-write scenario

Hi,

Any comments on this? Considering the most extreme scenario where a file is appended with only one byte each time, "jbd2_journal_dirty_metadata" will be executed because i_size has changed. Therefore, it is better to update raw_inode's size after the writeback is completed, which can also ensure the consistency of metadata and data, and obtain better performance benefits in high-frequency append write scenarios.

Thanks

reviewer
#2 lore raw
In reply to Liu Song
Quoted 8 lines · Liu Song

In the append-write scenario, after ensuring that the dirty inode can be seen by the writeback process, there is no need to execute "mark_inode_dirty" for every write. Instead, we can rely on "ext4_mark_inode_dirty" executed when updating i_disksize in "mpage_map_and_submit_extent" to ensure data consistency, which can significantly improve performance in high-frequency append-write scenarios.

Quoted 4 lines

In test scenarios of Kafka version 2.6.2, using packet size of 2K resulted in a 10% performance improvement.

Overall the benefit looks interesting. Nice work.

Quoted 31 lines
fs/ext4/inode.c +23 -0
index 9d9f414f99fe..d1aa775c9936 100644
@@ -3128,6 +3128,57 @@ static int ext4_da_should_update_i_disksize(struct page *page,
 	return 1;
 }
 
+/*
+ * Copy from generic_write_end, add conditions to execute mark_inode_dirty
+ * to avoid additional overhead caused by frequent dirty inode operations
+ */
+static int ext4_da_generic_write_end(struct file *file, struct address_space *mapping,
+			loff_t pos, unsigned len, unsigned copied, bool need_dirty,
+			struct page *page, void *fsdata)
+{
+	struct inode *inode = mapping->host;
+	loff_t old_size = inode->i_size;
+	bool i_size_changed = false;
+	int was_dirty;
+
+	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
+
+	/*
+	 * No need to use i_size_read() here, the i_size cannot change under us
+	 * because we hold i_rwsem.
+	 *
+	 * But it's important to update i_size while still holding page lock:
+	 * page writeout could otherwise come in and zero beyond i_size.
+	 */
+	if (pos + copied > inode->i_size) {

fs/ext4/inode.c static int ext4_da_should_update_i_disksize(struct page *page, · line 3163

Quoted 10 lines
fs/ext4/inode.c +10 -0
+		i_size_write(inode, pos + copied);
+		i_size_changed = true;
+	}
+
+	unlock_page(page);
+	put_page(page);
+
+	if (old_size < pos)
+		pagecache_isize_extended(inode, old_size, pos);
+

I dislike the duplication of generic_write_end() but exporting a variant of generic_write_end() not doing the dirtying (and additionally returning the i_size_changed value) doesn't look appealing either. So I guess I'll just live with this...

fs/ext4/inode.c static int ext4_da_should_update_i_disksize(struct page *page, · line 3171

Quoted 8 lines
fs/ext4/inode.c +8 -0
+	/*
+	 * In the append-write scenario, if the inode is marked as dirty,
+	 * it is ensured that the inode will be seen by the writeback process.
+	 * In the ext4_writepages process, when updating i_disksize,
+	 * corresponding metadata updates are also performed.
+	 * Therefore, it is unnecessary to repeatedly execute mark_inode_dirty
+	 * to improve performance.
+	 */

Note that block_write_end() will mark the inode as dirty with I_DIRTY_PAGES flag (which all that's needed to trigger page writeback). Since i_size is never really written to disk by ext4, there's simply never a need to mark inode dirty because of that. Also if inode timestamps were updated inode was already properly marked dirty. Hence we can just mostly drop the code below.

Quoted 22 lines
fs/ext4/inode.c +11 -0
+	if (i_size_changed) {
+		spin_lock(&inode->i_lock);
+		was_dirty = inode->i_state & I_DIRTY;
+		spin_unlock(&inode->i_lock);
+		if (!was_dirty || need_dirty)
+			mark_inode_dirty(inode);
+	}
+	return copied;
+}
+
 static int ext4_da_write_end(struct file *file,
 			     struct address_space *mapping,
 			     loff_t pos, unsigned len, unsigned copied,
@@ -3137,6 +3188,7 @@ static int ext4_da_write_end(struct file *file,
 	loff_t new_i_size;
 	unsigned long start, end;
 	int write_mode = (int)(unsigned long)fsdata;
+	bool need_dirty = false;
 
 	if (write_mode == FALL_BACK_TO_NONDELALLOC)
 		return ext4_write_end(file, mapping, pos,

fs/ext4/inode.c static int ext4_da_write_end(struct file *file, · line 3227

Quoted 8 lines
fs/ext4/inode.c +3 -1
@@ -3169,10 +3221,12 @@ static int ext4_da_write_end(struct file *file,
 	 */
 	new_i_size = pos + copied;
 	if (copied && new_i_size > inode->i_size &&
-	    ext4_da_should_update_i_disksize(page, end))
+	    ext4_da_should_update_i_disksize(page, end)) {
 		ext4_update_i_disksize(inode, new_i_size);
+		need_dirty = true;
+	}

So when we create our own new helper function anyway I'd just move all this logic for delalloc write end there. Something like:

static int ext4_da_do_write_end(struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied, struct page *page)

{
	struct inode *inode = mapping->host;
	loff_t old_size = inode->i_size;
	bool disksize_changed = false;
	loff_t new_i_size;

	copied = block_write_end(NULL, mapping, pos, len, copied, page, NULL);

	new_i_size = pos + copied;
	/*
	 * It's important to update i_size while still holding page lock:
	 * page writeout could otherwise come in and zero beyond i_size.
	 *
	 * Since we are holding inode lock, we are sure i_disksize <=
	 * i_size. We also know that if i_disksize < i_size, there are
	 * delalloc writes pending in the range upto i_size. If the end of
	 * the current write is <= i_size, there's no need to touch
	 * i_disksize since writeback will push i_disksize upto i_size
	 * eventually. If the end of the current write is > i_size and
	 * inside an allocated block (ext4_da_should_update_i_disksize()
	 * check), we need to update i_disksize here as certain
	 * ext4_writepages() paths not allocating blocks update i_disksize.
	 */
	if (new_i_size > inode->i_size) {
		unsigned long end;

		i_size_write(inode, new_i_size);
		end = (new_i_size - 1) & (PAGE_SIZE - 1);
		if (copied && ext4_da_should_update_i_disksize(folio, end)) {
			ext4_update_i_disksize(inode, new_i_size);
			disksize_changed = true;
		}
	}

	unlock_page(page);
	put_page(page);

	if (old_size < pos)
		pagecache_isize_extended(inode, old_size, pos);

	if (disksize_changed) {
		handle_t *handle;

		handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
		if (IS_ERR(handle))
			return PTR_ERR(handle);
		ext4_mark_inode_dirty(handle, inode);
		ext4_journal_stop(handle);
	}

	return copied;
}

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#3 lore raw
Quoted 95 lines · Jan Kara
Quoted 8 lines · Liu Song

In the append-write scenario, after ensuring that the dirty inode can be seen by the writeback process, there is no need to execute "mark_inode_dirty" for every write. Instead, we can rely on "ext4_mark_inode_dirty" executed when updating i_disksize in "mpage_map_and_submit_extent" to ensure data consistency, which can significantly improve performance in high-frequency append-write scenarios.

Quoted 4 lines

In test scenarios of Kafka version 2.6.2, using packet size of 2K resulted in a 10% performance improvement.

Overall the benefit looks interesting. Nice work.

Quoted 31 lines
fs/ext4/inode.c +23 -0
index 9d9f414f99fe..d1aa775c9936 100644
@@ -3128,6 +3128,57 @@ static int ext4_da_should_update_i_disksize(struct page *page,
  	return 1;
  }
 
+/*
+ * Copy from generic_write_end, add conditions to execute mark_inode_dirty
+ * to avoid additional overhead caused by frequent dirty inode operations
+ */
+static int ext4_da_generic_write_end(struct file *file, struct address_space *mapping,
+			loff_t pos, unsigned len, unsigned copied, bool need_dirty,
+			struct page *page, void *fsdata)
+{
+	struct inode *inode = mapping->host;
+	loff_t old_size = inode->i_size;
+	bool i_size_changed = false;
+	int was_dirty;
+
+	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
+
+	/*
+	 * No need to use i_size_read() here, the i_size cannot change under us
+	 * because we hold i_rwsem.
+	 *
+	 * But it's important to update i_size while still holding page lock:
+	 * page writeout could otherwise come in and zero beyond i_size.
+	 */
+	if (pos + copied > inode->i_size) {

fs/ext4/inode.c static int ext4_da_should_update_i_disksize(struct page *page, · line 3163

Quoted 10 lines
fs/ext4/inode.c +10 -0
+		i_size_write(inode, pos + copied);
+		i_size_changed = true;
+	}
+
+	unlock_page(page);
+	put_page(page);
+
+	if (old_size < pos)
+		pagecache_isize_extended(inode, old_size, pos);
+

I dislike the duplication of generic_write_end() but exporting a variant of generic_write_end() not doing the dirtying (and additionally returning the i_size_changed value) doesn't look appealing either. So I guess I'll just live with this...

fs/ext4/inode.c static int ext4_da_should_update_i_disksize(struct page *page, · line 3171

Quoted 8 lines
fs/ext4/inode.c +8 -0
+	/*
+	 * In the append-write scenario, if the inode is marked as dirty,
+	 * it is ensured that the inode will be seen by the writeback process.
+	 * In the ext4_writepages process, when updating i_disksize,
+	 * corresponding metadata updates are also performed.
+	 * Therefore, it is unnecessary to repeatedly execute mark_inode_dirty
+	 * to improve performance.
+	 */

Note that block_write_end() will mark the inode as dirty with I_DIRTY_PAGES flag (which all that's needed to trigger page writeback). Since i_size is never really written to disk by ext4, there's simply never a need to mark inode dirty because of that. Also if inode timestamps were updated inode was already properly marked dirty. Hence we can just mostly drop the code below.

Quoted 19 lines
fs/ext4/inode.c +11 -0
+	if (i_size_changed) {
+		spin_lock(&inode->i_lock);
+		was_dirty = inode->i_state & I_DIRTY;
+		spin_unlock(&inode->i_lock);
+		if (!was_dirty || need_dirty)
+			mark_inode_dirty(inode);
+	}
+	return copied;
+}
+
  static int ext4_da_write_end(struct file *file,
  			     struct address_space *mapping,
  			     loff_t pos, unsigned len, unsigned copied,
@@ -3137,6 +3188,7 @@ static int ext4_da_write_end(struct file *file,
  	loff_t new_i_size;
  	unsigned long start, end;
  	int write_mode = (int)(unsigned long)fsdata;
+	bool need_dirty = false;
 
Quoted 11 lines

fs/ext4/inode.c static int ext4_da_write_end(struct file *file, · line 3227

Quoted 11 lines
fs/ext4/inode.c +3 -1
  	if (write_mode == FALL_BACK_TO_NONDELALLOC)
  		return ext4_write_end(file, mapping, pos,
@@ -3169,10 +3221,12 @@ static int ext4_da_write_end(struct file *file,
  	 */
  	new_i_size = pos + copied;
  	if (copied && new_i_size > inode->i_size &&
-	    ext4_da_should_update_i_disksize(page, end))
+	    ext4_da_should_update_i_disksize(page, end)) {
  		ext4_update_i_disksize(inode, new_i_size);
+		need_dirty = true;
+	}

Thank you very much for your response, and the implementation of "ext4_da_do_write_end" is great. I will refer to your suggestions and send a V2 version of the patch.

Thanks

Quoted 61 lines

So when we create our own new helper function anyway I'd just move all this logic for delalloc write end there. Something like:

static int ext4_da_do_write_end(struct address_space *mapping,
loff_t pos, unsigned len, unsigned copied, struct page *page)

{
	struct inode *inode = mapping->host;
	loff_t old_size = inode->i_size;
	bool disksize_changed = false;
	loff_t new_i_size;

	copied = block_write_end(NULL, mapping, pos, len, copied, page, NULL);

	new_i_size = pos + copied;
	/*
	 * It's important to update i_size while still holding page lock:
	 * page writeout could otherwise come in and zero beyond i_size.
	 *
	 * Since we are holding inode lock, we are sure i_disksize <=
	 * i_size. We also know that if i_disksize < i_size, there are
	 * delalloc writes pending in the range upto i_size. If the end of
	 * the current write is <= i_size, there's no need to touch
	 * i_disksize since writeback will push i_disksize upto i_size
	 * eventually. If the end of the current write is > i_size and
	 * inside an allocated block (ext4_da_should_update_i_disksize()
	 * check), we need to update i_disksize here as certain
	 * ext4_writepages() paths not allocating blocks update i_disksize.
	 */
	if (new_i_size > inode->i_size) {
		unsigned long end;

		i_size_write(inode, new_i_size);
		end = (new_i_size - 1) & (PAGE_SIZE - 1);
		if (copied && ext4_da_should_update_i_disksize(folio, end)) {
			ext4_update_i_disksize(inode, new_i_size);
			disksize_changed = true;
		}
	}

	unlock_page(page);
	put_page(page);

	if (old_size < pos)
		pagecache_isize_extended(inode, old_size, pos);

	if (disksize_changed) {
		handle_t *handle;

		handle = ext4_journal_start(inode, EXT4_HT_INODE, 2);
		if (IS_ERR(handle))
			return PTR_ERR(handle);
		ext4_mark_inode_dirty(handle, inode);
		ext4_journal_stop(handle);
	}

	return copied;
}

Honza