Skip to contentSkip to conversation

iomap: Add support for per-block dirty state to improve write performance

RFC Superseded Ritesh Harjani (IBM) 7 participants 48 messages May 7, 2023 – May 23, 2023

Subsystems (5)

v5 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v5 cover iomap: Add support for per-block dirty state to improve write performance

Hello,

Please find version-5 of this series. v5 mainly addresses the review comments from Matthew to have a better iop->state bitmap handling functions. Please do let me know if any comments/feedback.

Testing
=========
Continuing to run more tests, but so far I haven't observed any surprises on my 1k and 4k blocksize with default options & -g "auto" runs.

<Pasting RFCv3 => RFCv4 relevant changelog for v5 and dopping everything else>
========================================================================
This addresses a problem reported by Brian for a short write case with delalloc
release. This is addressed in patch-5 in function iomap_write_delalloc_scan(). I suppose this is a major change from the previous rfcv3.
I did test a unit test which Brian provided with xfs_io -f option. Before those changes, the kernel caused a bug_on during unmount with the unit test. This gets fixed with the changes added in v4.

	i.e. After v4
	=================
	root@ubuntu# ./xfs_io -fc "truncate 4k" -c "mmap 0 4k" -c "mread 0 4k" -c "pwrite 0 1" -c "pwrite -f 2k 1" -c fsync /mnt1/tmnt/testfile
	wrote 1/1 bytes at offset 0
	1.000000 bytes, 1 ops; 0.0001 sec (7.077 KiB/sec and 7246.3768 ops/sec)
	pwrite: Bad address
	root@ubuntu# ./xfs_io -c "fiemap -v" /mnt1/tmnt/testfile
	/mnt1/tmnt/testfile:
	 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
	   0: [0..1]:          22..23               2   0x1
	   1: [2..7]:          hole                 6
	root@ubuntu# filefrag -v /mnt1/tmnt/testfile
	Filesystem type is: 58465342
	File size of /mnt1/tmnt/testfile is 4096 (4 blocks of 1024 bytes)
	 ext:     logical_offset:        physical_offset: length:   expected: flags:
	   0:        0..       0:         11..        11:      1:             last
	/mnt1/tmnt/testfile: 1 extent found
	root@ubuntu# umount /mnt1/tmnt
	root@ubuntu#

	Before v4
	===========
	root@ubuntu# mount /dev/loop8 /mnt1/test
	root@ubuntu# ./xfs_io -fc "truncate 4k" -c "mmap 0 4k" -c "mread 0 4k" -c "pwrite 0 1" -c "pwrite -f 2k 1" -c fsync /mnt1/test/testfile
	wrote 1/1 bytes at offset 0
	1.000000 bytes, 1 ops; 0.0000 sec (10.280 KiB/sec and 10526.3158 ops/sec)
	pwrite: Bad address
	root@ubuntu# ./xfs_io -c "fiemap -v" /mnt1/test/testfile
	/mnt1/test/testfile:
	 EXT: FILE-OFFSET      BLOCK-RANGE      TOTAL FLAGS
	   0: [0..1]:          22..23               2   0x0
	   1: [2..3]:          hole                 2
	   2: [4..5]:          0..1                 2   0x7
	   3: [6..7]:          hole                 2
	root@ubuntu# filefrag -v /mnt1/test/testfile
	Filesystem type is: 58465342
	File size of /mnt1/test/testfile is 4096 (4 blocks of 1024 bytes)
	 ext:     logical_offset:        physical_offset: length:   expected: flags:
	   0:        0..       0:         11..        11:      1:
	   1:        2..       2:          0..         0:      0:             last,unknown_loc,delalloc
	/mnt1/test/testfile: 2 extents found
	root@ubuntu# umount /mnt1/test
	<dmesg snip>
	[  156.581188] XFS (loop8): Unmounting Filesystem 7889507d-fc7f-4a1c-94d5-06797f2cc790
	[  156.584455] XFS (loop8): ino 43 data fork has delalloc extent at [0x2:0x1]
	[  156.587847] XFS: Assertion failed: 0, file: fs/xfs/xfs_icache.c, line: 1816
	[  156.591675] ------------[ cut here ]------------
	[  156.594133] kernel BUG at fs/xfs/xfs_message.c:102!
	[  156.596669] invalid opcode: 0000 [#1] PREEMPT SMP PTI
	[  156.599277] CPU: 7 PID: 435 Comm: kworker/7:5 Not tainted 6.3.0-rc6-xfstests-00003-g99a844f4e411-dirty #105
	[  156.603721] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
	[  156.608701] Workqueue: xfs-inodegc/loop8 xfs_inodegc_worker
	[  156.611426] RIP: 0010:assfail+0x38/0x40
	[  156.646981] Call Trace:
	[  156.647951]  <TASK>
	[  156.648904]  xfs_inodegc_set_reclaimable+0x15b/0x160
	[  156.651270]  xfs_inodegc_worker+0x95/0x1d0
	[  156.653202]  process_one_work+0x274/0x550
	[  156.655305]  worker_thread+0x4f/0x300
	[  156.657081]  ? __pfx_worker_thread+0x10/0x10
	[  156.658977]  kthread+0xf6/0x120
	[  156.660657]  ? __pfx_kthread+0x10/0x10
	[  156.662565]  ret_from_fork+0x2c/0x50
	[  156.664421]  </TASK>

Previous changelogs & TODOs at [1]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/

Ritesh Harjani (IBM) (5):
iomap: Rename iomap_page_create/release() to iop_alloc/free()
iomap: Refactor iop_set_range_uptodate() function
iomap: Add iop's uptodate state handling functions
iomap: Allocate iop in ->write_begin() early
iomap: Add per-block dirty state tracking to improve performance

 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 264 ++++++++++++++++++++++++++++++++---------
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 210 insertions(+), 61 deletions(-)

--
2.39.2

#1 lore raw

RFC v5 1/5 iomap: Rename iomap_page_create/release() to iop_alloc/free()

This patch renames the iomap_page_create/release() functions to iop_alloc/free() calls. Later patches adds more functions for handling iop structure with iop_** naming conventions. Hence iop_alloc/free() makes more sense.

Note, this patch also move folio_detach_private() to happen later after checking for bitmap_full(). This is just another small refactor because in later patches we will move bitmap_** helpers to iop_** related helpers which will only take a folio and hence we should move folio_detach_private() to the end before calling kfree(iop).

---
 fs/iomap/buffered-io.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)
fs/iomap/buffered-io.c +13 -11
index 6f4c97a6d7e9..cbd945d96584 100644
@@ -43,8 +43,8 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)
 
 static struct bio_set iomap_ioend_bioset;
 
-static struct iomap_page *
-iomap_page_create(struct inode *inode, struct folio *folio, unsigned int flags)
+static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
+				    unsigned int flags)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
 	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
@@ -69,9 +69,9 @@ iomap_page_create(struct inode *inode, struct folio *folio, unsigned int flags)
 	return iop;
 }
 
-static void iomap_page_release(struct folio *folio)
+static void iop_free(struct folio *folio)
 {
-	struct iomap_page *iop = folio_detach_private(folio);
+	struct iomap_page *iop = to_iomap_page(folio);
 	struct inode *inode = folio->mapping->host;
 	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
 
@@ -81,6 +81,7 @@ static void iomap_page_release(struct folio *folio)
 	WARN_ON_ONCE(atomic_read(&iop->write_bytes_pending));
 	WARN_ON_ONCE(bitmap_full(iop->uptodate, nr_blocks) !=
 			folio_test_uptodate(folio));
+	folio_detach_private(folio);
 	kfree(iop);
 }
 
@@ -231,7 +232,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	if (WARN_ON_ONCE(size > iomap->length))
 		return -EIO;
 	if (offset > 0)
-		iop = iomap_page_create(iter->inode, folio, iter->flags);
+		iop = iop_alloc(iter->inode, folio, iter->flags);
 	else
 		iop = to_iomap_page(folio);
 
@@ -269,7 +270,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 		return iomap_read_inline_data(iter, folio);
 
 	/* zero post-eof blocks as the page may be mapped */
-	iop = iomap_page_create(iter->inode, folio, iter->flags);
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 	iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen);
 	if (plen == 0)
 		goto done;
@@ -497,7 +498,7 @@ bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags)
 	 */
 	if (folio_test_dirty(folio) || folio_test_writeback(folio))
 		return false;
-	iomap_page_release(folio);
+	iop_free(folio);
 	return true;
 }
 EXPORT_SYMBOL_GPL(iomap_release_folio);
@@ -514,12 +515,12 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)
 	if (offset == 0 && len == folio_size(folio)) {
 		WARN_ON_ONCE(folio_test_writeback(folio));
 		folio_cancel_dirty(folio);
-		iomap_page_release(folio);
+		iop_free(folio);
 	} else if (folio_test_large(folio)) {
 		/* Must release the iop so the page can be split */
 		WARN_ON_ONCE(!folio_test_uptodate(folio) &&
 			     folio_test_dirty(folio));
-		iomap_page_release(folio);
+		iop_free(folio);
 	}
 }
 EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
@@ -566,7 +567,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 		return 0;
 	folio_clear_error(folio);
 
-	iop = iomap_page_create(iter->inode, folio, iter->flags);
+	iop = iop_alloc(iter->inode, folio, iter->flags);
+
 	if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1)
 		return -EAGAIN;
 
@@ -1619,7 +1621,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iomap_page_create(inode, folio, 0);
+	struct iomap_page *iop = iop_alloc(inode, folio, 0);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
Signature · 1 line
2.39.2
#2 lore raw

Re: [RFCv5 1/5] iomap: Rename iomap_page_create/release() to iop_alloc/free()

Quoted 4 lines · Ritesh Harjani (IBM)

This patch renames the iomap_page_create/release() functions to iop_alloc/free() calls. Later patches adds more functions for handling iop structure with iop_** naming conventions. Hence iop_alloc/free() makes more sense.

I can't say I like the iop_* naming all that much, especially as we it is very generic and we use an iomap_ prefix every else.

Quoted 5 lines

Note, this patch also move folio_detach_private() to happen later after checking for bitmap_full(). This is just another small refactor because in later patches we will move bitmap_** helpers to iop_** related helpers which will only take a folio and hence we should move folio_detach_private() to the end before calling kfree(iop).

Please don't mix renames and code movements.

#3 lore raw

Re: [RFCv5 1/5] iomap: Rename iomap_page_create/release() to iop_alloc/free()

Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Ritesh Harjani (IBM)

This patch renames the iomap_page_create/release() functions to iop_alloc/free() calls. Later patches adds more functions for handling iop structure with iop_** naming conventions. Hence iop_alloc/free() makes more sense.

I can't say I like the iop_* naming all that much, especially as we it is very generic and we use an iomap_ prefix every else.

I can prefix iomap_ so it will then become iomap_iop_alloc()/iomap_iop_free(). All other helpers will be like...
- iomap_iop_set_range_uptodate()
- iomap_iop_set_range_dirty()
- ...

'iomap_iop' prefix just helps distinguish all the APIs which are working over iomap_page (iop) structure.

Quoted 7 lines
Quoted 5 lines

Note, this patch also move folio_detach_private() to happen later after checking for bitmap_full(). This is just another small refactor because in later patches we will move bitmap_** helpers to iop_** related helpers which will only take a folio and hence we should move folio_detach_private() to the end before calling kfree(iop).

Please don't mix renames and code movements.

Sure. Will take care of it in the next rev.

#4 lore raw

RFC v5 2/5 iomap: Refactor iop_set_range_uptodate() function

In reply to Ritesh Harjani (IBM)

This patch moves up and combine the definitions of two functions (iomap_iop_set_range_uptodate() & iomap_set_range_uptodate()) into iop_set_range_uptodate() & refactors it's arguments a bit.

No functionality change in this patch.

---
 fs/iomap/buffered-io.c | 57 ++++++++++++++++++++----------------------
 1 file changed, 27 insertions(+), 30 deletions(-)
fs/iomap/buffered-io.c +27 -30
index cbd945d96584..e732581dc2d4 100644
@@ -43,6 +43,27 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)
 
 static struct bio_set iomap_ioend_bioset;
 
+static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
+				   size_t off, size_t len)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	unsigned int first_blk = off >> inode->i_blkbits;
+	unsigned int last_blk = (off + len - 1) >> inode->i_blkbits;
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (iop) {
+		spin_lock_irqsave(&iop->uptodate_lock, flags);
+		bitmap_set(iop->uptodate, first_blk, nr_blks);
+		if (bitmap_full(iop->uptodate,
+				i_blocks_per_folio(inode, folio)))
+			folio_mark_uptodate(folio);
+		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+	} else {
+		folio_mark_uptodate(folio);
+	}
+}
+
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 				    unsigned int flags)
 {
@@ -145,30 +166,6 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 	*lenp = plen;
 }
 
-static void iomap_iop_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	struct inode *inode = folio->mapping->host;
-	unsigned first = off >> inode->i_blkbits;
-	unsigned last = (off + len - 1) >> inode->i_blkbits;
-	unsigned long flags;
-
-	spin_lock_irqsave(&iop->uptodate_lock, flags);
-	bitmap_set(iop->uptodate, first, last - first + 1);
-	if (bitmap_full(iop->uptodate, i_blocks_per_folio(inode, folio)))
-		folio_mark_uptodate(folio);
-	spin_unlock_irqrestore(&iop->uptodate_lock, flags);
-}
-
-static void iomap_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	if (iop)
-		iomap_iop_set_range_uptodate(folio, iop, off, len);
-	else
-		folio_mark_uptodate(folio);
-}
-
 static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		size_t len, int error)
 {
@@ -178,7 +175,8 @@ static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		folio_clear_uptodate(folio);
 		folio_set_error(folio);
 	} else {
-		iomap_set_range_uptodate(folio, iop, offset, len);
+		iop_set_range_uptodate(folio->mapping->host, folio, offset,
+				       len);
 	}
 
 	if (!iop || atomic_sub_and_test(len, &iop->read_bytes_pending))
@@ -240,7 +238,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	memcpy(addr, iomap->inline_data, size);
 	memset(addr + size, 0, PAGE_SIZE - poff - size);
 	kunmap_local(addr);
-	iomap_set_range_uptodate(folio, iop, offset, PAGE_SIZE - poff);
+	iop_set_range_uptodate(iter->inode, folio, offset, PAGE_SIZE - poff);
 	return 0;
 }
 
@@ -277,7 +275,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 
 	if (iomap_block_needs_zeroing(iter, pos)) {
 		folio_zero_range(folio, poff, plen);
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 		goto done;
 	}
 
@@ -598,7 +596,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 			if (status)
 				return status;
 		}
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 	} while ((block_start += plen) < block_end);
 
 	return 0;
@@ -705,7 +703,6 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos,
 static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 		size_t copied, struct folio *folio)
 {
-	struct iomap_page *iop = to_iomap_page(folio);
 	flush_dcache_folio(folio);
 
 	/*
@@ -721,7 +718,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 	 */
 	if (unlikely(copied < len && !folio_test_uptodate(folio)))
 		return 0;
-	iomap_set_range_uptodate(folio, iop, offset_in_folio(folio, pos), len);
+	iop_set_range_uptodate(inode, folio, offset_in_folio(folio, pos), len);
 	filemap_dirty_folio(inode->i_mapping, folio);
 	return copied;
 }

--
2.39.2

#5 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

kernel test robot noticed the following build warnings:

Hi Ritesh,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on xfs-linux/for-next]
[also build test WARNING on gfs2/for-next linus/master v6.3 next-20230505] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903
base:   https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next
patch link:    https://lore.kernel.org/r/203a9e25873f6c94c9de89823439aa1f6a7dc714.1683485700.git.ritesh.list%40gmail.com
patch subject: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function
config: m68k-allyesconfig (https://download.01.org/0day-ci/archive/20230508/[email protected]/config)
compiler: m68k-linux-gcc (GCC) 12.1.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/9f4d2751ba6c4b9022e0361ce14274eb156cf39b
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903 git checkout 9f4d2751ba6c4b9022e0361ce14274eb156cf39b
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k SHELL=/bin/bash fs/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

fs/iomap/buffered-io.c: In function 'iomap_read_inline_data':

>> fs/iomap/buffered-io.c:215:28: warning: variable 'iop' set but not used [-Wunused-but-set-variable]
     215 |         struct iomap_page *iop;
         |                            ^~~

vim +/iop +215 fs/iomap/buffered-io.c

afc51aaa22f26c Darrick J. Wong         2019-07-15  202  
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  203  /**
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  204   * iomap_read_inline_data - copy inline data into the page cache
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  205   * @iter: iteration structure
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  206)  * @folio: folio to copy to
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  207   *
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  208)  * Copy the inline data in @iter into @folio and zero out the rest of the folio.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  209   * Only a single IOMAP_INLINE extent is allowed at the end of each file.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  210   * Returns zero for success to complete the read, or the usual negative errno.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  211   */
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  212  static int iomap_read_inline_data(const struct iomap_iter *iter,
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  213) 		struct folio *folio)
afc51aaa22f26c Darrick J. Wong         2019-07-15  214  {
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28 @215) 	struct iomap_page *iop;
fad0a1ab34f777 Christoph Hellwig       2021-08-10  216  	const struct iomap *iomap = iomap_iter_srcmap(iter);
1b5c1e36dc0e0f Christoph Hellwig       2021-08-10  217  	size_t size = i_size_read(iter->inode) - iomap->offset;
b405435b419cb6 Matthew Wilcox (Oracle  2021-08-02  218) 	size_t poff = offset_in_page(iomap->offset);
431c0566bb6078 Matthew Wilcox (Oracle  2021-04-28  219) 	size_t offset = offset_in_folio(folio, iomap->offset);
afc51aaa22f26c Darrick J. Wong         2019-07-15  220  	void *addr;
afc51aaa22f26c Darrick J. Wong         2019-07-15  221  
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  222) 	if (folio_test_uptodate(folio))
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  223  		return 0;
afc51aaa22f26c Darrick J. Wong         2019-07-15  224  
ae44f9c286da3f Matthew Wilcox (Oracle  2021-08-04  225) 	if (WARN_ON_ONCE(size > PAGE_SIZE - poff))
ae44f9c286da3f Matthew Wilcox (Oracle  2021-08-04  226) 		return -EIO;
69f4a26c1e0c7c Gao Xiang               2021-08-03  227  	if (WARN_ON_ONCE(size > PAGE_SIZE -
69f4a26c1e0c7c Gao Xiang               2021-08-03  228  			 offset_in_page(iomap->inline_data)))
69f4a26c1e0c7c Gao Xiang               2021-08-03  229  		return -EIO;
69f4a26c1e0c7c Gao Xiang               2021-08-03  230  	if (WARN_ON_ONCE(size > iomap->length))
69f4a26c1e0c7c Gao Xiang               2021-08-03  231  		return -EIO;
431c0566bb6078 Matthew Wilcox (Oracle  2021-04-28  232) 	if (offset > 0)
bd964cc84262c6 Ritesh Harjani (IBM     2023-05-08  233) 		iop = iop_alloc(iter->inode, folio, iter->flags);
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28  234) 	else
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28  235) 		iop = to_iomap_page(folio);
afc51aaa22f26c Darrick J. Wong         2019-07-15  236  
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  237) 	addr = kmap_local_folio(folio, offset);
afc51aaa22f26c Darrick J. Wong         2019-07-15  238  	memcpy(addr, iomap->inline_data, size);
b405435b419cb6 Matthew Wilcox (Oracle  2021-08-02  239) 	memset(addr + size, 0, PAGE_SIZE - poff - size);
ab069d5fdcd145 Matthew Wilcox (Oracle  2021-08-04  240) 	kunmap_local(addr);
9f4d2751ba6c4b Ritesh Harjani (IBM     2023-05-08  241) 	iop_set_range_uptodate(iter->inode, folio, offset, PAGE_SIZE - poff);
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  242  	return 0;
afc51aaa22f26c Darrick J. Wong         2019-07-15  243  }
afc51aaa22f26c Darrick J. Wong         2019-07-15  244  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#6 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

In reply to Ritesh Harjani (IBM)
kernel test robot noticed the following build warnings:

Hi Ritesh,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on xfs-linux/for-next]
[also build test WARNING on gfs2/for-next linus/master v6.4-rc1 next-20230505] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903
base:   https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next
patch link:    https://lore.kernel.org/r/203a9e25873f6c94c9de89823439aa1f6a7dc714.1683485700.git.ritesh.list%40gmail.com
patch subject: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function
config: i386-randconfig-a001-20230508 (https://download.01.org/0day-ci/archive/20230508/[email protected]/config)
compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/9f4d2751ba6c4b9022e0361ce14274eb156cf39b
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903 git checkout 9f4d2751ba6c4b9022e0361ce14274eb156cf39b
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash fs/iomap/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> fs/iomap/buffered-io.c:215:21: warning: variable 'iop' set but not used [-Wunused-but-set-variable]
           struct iomap_page *iop;
                              ^

1 warning generated.

vim +/iop +215 fs/iomap/buffered-io.c

afc51aaa22f26c Darrick J. Wong         2019-07-15  202  
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  203  /**
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  204   * iomap_read_inline_data - copy inline data into the page cache
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  205   * @iter: iteration structure
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  206)  * @folio: folio to copy to
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  207   *
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  208)  * Copy the inline data in @iter into @folio and zero out the rest of the folio.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  209   * Only a single IOMAP_INLINE extent is allowed at the end of each file.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  210   * Returns zero for success to complete the read, or the usual negative errno.
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  211   */
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  212  static int iomap_read_inline_data(const struct iomap_iter *iter,
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  213) 		struct folio *folio)
afc51aaa22f26c Darrick J. Wong         2019-07-15  214  {
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28 @215) 	struct iomap_page *iop;
fad0a1ab34f777 Christoph Hellwig       2021-08-10  216  	const struct iomap *iomap = iomap_iter_srcmap(iter);
1b5c1e36dc0e0f Christoph Hellwig       2021-08-10  217  	size_t size = i_size_read(iter->inode) - iomap->offset;
b405435b419cb6 Matthew Wilcox (Oracle  2021-08-02  218) 	size_t poff = offset_in_page(iomap->offset);
431c0566bb6078 Matthew Wilcox (Oracle  2021-04-28  219) 	size_t offset = offset_in_folio(folio, iomap->offset);
afc51aaa22f26c Darrick J. Wong         2019-07-15  220  	void *addr;
afc51aaa22f26c Darrick J. Wong         2019-07-15  221  
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  222) 	if (folio_test_uptodate(folio))
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  223  		return 0;
afc51aaa22f26c Darrick J. Wong         2019-07-15  224  
ae44f9c286da3f Matthew Wilcox (Oracle  2021-08-04  225) 	if (WARN_ON_ONCE(size > PAGE_SIZE - poff))
ae44f9c286da3f Matthew Wilcox (Oracle  2021-08-04  226) 		return -EIO;
69f4a26c1e0c7c Gao Xiang               2021-08-03  227  	if (WARN_ON_ONCE(size > PAGE_SIZE -
69f4a26c1e0c7c Gao Xiang               2021-08-03  228  			 offset_in_page(iomap->inline_data)))
69f4a26c1e0c7c Gao Xiang               2021-08-03  229  		return -EIO;
69f4a26c1e0c7c Gao Xiang               2021-08-03  230  	if (WARN_ON_ONCE(size > iomap->length))
69f4a26c1e0c7c Gao Xiang               2021-08-03  231  		return -EIO;
431c0566bb6078 Matthew Wilcox (Oracle  2021-04-28  232) 	if (offset > 0)
bd964cc84262c6 Ritesh Harjani (IBM     2023-05-08  233) 		iop = iop_alloc(iter->inode, folio, iter->flags);
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28  234) 	else
cd1e5afe5503ed Matthew Wilcox (Oracle  2021-04-28  235) 		iop = to_iomap_page(folio);
afc51aaa22f26c Darrick J. Wong         2019-07-15  236  
874628a2c59003 Matthew Wilcox (Oracle  2021-07-23  237) 	addr = kmap_local_folio(folio, offset);
afc51aaa22f26c Darrick J. Wong         2019-07-15  238  	memcpy(addr, iomap->inline_data, size);
b405435b419cb6 Matthew Wilcox (Oracle  2021-08-02  239) 	memset(addr + size, 0, PAGE_SIZE - poff - size);
ab069d5fdcd145 Matthew Wilcox (Oracle  2021-08-04  240) 	kunmap_local(addr);
9f4d2751ba6c4b Ritesh Harjani (IBM     2023-05-08  241) 	iop_set_range_uptodate(iter->inode, folio, offset, PAGE_SIZE - poff);
5ad448ce2976f8 Andreas Gruenbacher     2021-11-24  242  	return 0;
afc51aaa22f26c Darrick J. Wong         2019-07-15  243  }
afc51aaa22f26c Darrick J. Wong         2019-07-15  244  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#7 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

In reply to Ritesh Harjani (IBM)
Quoted 8 lines · Ritesh Harjani (IBM)

This patch moves up and combine the definitions of two functions (iomap_iop_set_range_uptodate() & iomap_set_range_uptodate()) into iop_set_range_uptodate() & refactors it's arguments a bit.

No functionality change in this patch.

---

Hi Ritesh,

I just have a few random and nitty comments/questions on the series..

Quoted 8 lines
 fs/iomap/buffered-io.c | 57 ++++++++++++++++++++----------------------
 1 file changed, 27 insertions(+), 30 deletions(-)
fs/iomap/buffered-io.c +0 -0
index cbd945d96584..e732581dc2d4 100644

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 48

Quoted 6 lines
fs/iomap/buffered-io.c +3 -0
@@ -43,6 +43,27 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)
 
 static struct bio_set iomap_ioend_bioset;
 
+static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
+				   size_t off, size_t len)
+{

Any particular reason this now takes the inode as a param now instead of continuing to use the folio?

Brian

Quoted 109 lines
fs/iomap/buffered-io.c +24 -31
+	struct iomap_page *iop = to_iomap_page(folio);
+	unsigned int first_blk = off >> inode->i_blkbits;
+	unsigned int last_blk = (off + len - 1) >> inode->i_blkbits;
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (iop) {
+		spin_lock_irqsave(&iop->uptodate_lock, flags);
+		bitmap_set(iop->uptodate, first_blk, nr_blks);
+		if (bitmap_full(iop->uptodate,
+				i_blocks_per_folio(inode, folio)))
+			folio_mark_uptodate(folio);
+		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+	} else {
+		folio_mark_uptodate(folio);
+	}
+}
+
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 				    unsigned int flags)
 {
@@ -145,30 +166,6 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 	*lenp = plen;
 }
 
-static void iomap_iop_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	struct inode *inode = folio->mapping->host;
-	unsigned first = off >> inode->i_blkbits;
-	unsigned last = (off + len - 1) >> inode->i_blkbits;
-	unsigned long flags;
-
-	spin_lock_irqsave(&iop->uptodate_lock, flags);
-	bitmap_set(iop->uptodate, first, last - first + 1);
-	if (bitmap_full(iop->uptodate, i_blocks_per_folio(inode, folio)))
-		folio_mark_uptodate(folio);
-	spin_unlock_irqrestore(&iop->uptodate_lock, flags);
-}
-
-static void iomap_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	if (iop)
-		iomap_iop_set_range_uptodate(folio, iop, off, len);
-	else
-		folio_mark_uptodate(folio);
-}
-
 static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		size_t len, int error)
 {
@@ -178,7 +175,8 @@ static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		folio_clear_uptodate(folio);
 		folio_set_error(folio);
 	} else {
-		iomap_set_range_uptodate(folio, iop, offset, len);
+		iop_set_range_uptodate(folio->mapping->host, folio, offset,
+				       len);
 	}
 
 	if (!iop || atomic_sub_and_test(len, &iop->read_bytes_pending))
@@ -240,7 +238,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	memcpy(addr, iomap->inline_data, size);
 	memset(addr + size, 0, PAGE_SIZE - poff - size);
 	kunmap_local(addr);
-	iomap_set_range_uptodate(folio, iop, offset, PAGE_SIZE - poff);
+	iop_set_range_uptodate(iter->inode, folio, offset, PAGE_SIZE - poff);
 	return 0;
 }
 
@@ -277,7 +275,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 
 	if (iomap_block_needs_zeroing(iter, pos)) {
 		folio_zero_range(folio, poff, plen);
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 		goto done;
 	}
 
@@ -598,7 +596,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 			if (status)
 				return status;
 		}
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 	} while ((block_start += plen) < block_end);
 
 	return 0;
@@ -705,7 +703,6 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos,
 static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 		size_t copied, struct folio *folio)
 {
-	struct iomap_page *iop = to_iomap_page(folio);
 	flush_dcache_folio(folio);
 
 	/*
@@ -721,7 +718,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 	 */
 	if (unlikely(copied < len && !folio_test_uptodate(folio)))
 		return 0;
-	iomap_set_range_uptodate(folio, iop, offset_in_folio(folio, pos), len);
+	iop_set_range_uptodate(inode, folio, offset_in_folio(folio, pos), len);
 	filemap_dirty_folio(inode->i_mapping, folio);
 	return copied;
 }
--

2.39.2

#8 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

Quoted 5 lines · Brian Foster
Quoted 4 lines · Ritesh Harjani (IBM)

This patch moves up and combine the definitions of two functions (iomap_iop_set_range_uptodate() & iomap_set_range_uptodate()) into iop_set_range_uptodate() & refactors it's arguments a bit.

Quoted 9 lines
Quoted 4 lines

No functionality change in this patch.

---

Hi Ritesh,

I just have a few random and nitty comments/questions on the series..

Thanks a lot for reviewing.

Quoted 9 lines
Quoted 9 lines
 fs/iomap/buffered-io.c | 57 ++++++++++++++++++++----------------------
 1 file changed, 27 insertions(+), 30 deletions(-)
fs/iomap/buffered-io.c +0 -0
index cbd945d96584..e732581dc2d4 100644
@@ -43,6 +43,27 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)
 
Quoted 9 lines

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 48

Quoted 5 lines
fs/iomap/buffered-io.c +3 -0
 static struct bio_set iomap_ioend_bioset;
 
+static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
+				   size_t off, size_t len)
+{

Any particular reason this now takes the inode as a param now instead of continuing to use the folio?

Mainly for blkbits (or blocksize) and blocks_per_folio. Earlier we were explicitly passing this info, but with recent comments, I felt the api make sense to have inode and folio (given that we want to work on blocksize, blocks_per_folio and folio).

-ritesh

Quoted 111 lines

Brian

Quoted 109 lines
fs/iomap/buffered-io.c +24 -31
+	struct iomap_page *iop = to_iomap_page(folio);
+	unsigned int first_blk = off >> inode->i_blkbits;
+	unsigned int last_blk = (off + len - 1) >> inode->i_blkbits;
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (iop) {
+		spin_lock_irqsave(&iop->uptodate_lock, flags);
+		bitmap_set(iop->uptodate, first_blk, nr_blks);
+		if (bitmap_full(iop->uptodate,
+				i_blocks_per_folio(inode, folio)))
+			folio_mark_uptodate(folio);
+		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+	} else {
+		folio_mark_uptodate(folio);
+	}
+}
+
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 				    unsigned int flags)
 {
@@ -145,30 +166,6 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 	*lenp = plen;
 }
 
-static void iomap_iop_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	struct inode *inode = folio->mapping->host;
-	unsigned first = off >> inode->i_blkbits;
-	unsigned last = (off + len - 1) >> inode->i_blkbits;
-	unsigned long flags;
-
-	spin_lock_irqsave(&iop->uptodate_lock, flags);
-	bitmap_set(iop->uptodate, first, last - first + 1);
-	if (bitmap_full(iop->uptodate, i_blocks_per_folio(inode, folio)))
-		folio_mark_uptodate(folio);
-	spin_unlock_irqrestore(&iop->uptodate_lock, flags);
-}
-
-static void iomap_set_range_uptodate(struct folio *folio,
-		struct iomap_page *iop, size_t off, size_t len)
-{
-	if (iop)
-		iomap_iop_set_range_uptodate(folio, iop, off, len);
-	else
-		folio_mark_uptodate(folio);
-}
-
 static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		size_t len, int error)
 {
@@ -178,7 +175,8 @@ static void iomap_finish_folio_read(struct folio *folio, size_t offset,
 		folio_clear_uptodate(folio);
 		folio_set_error(folio);
 	} else {
-		iomap_set_range_uptodate(folio, iop, offset, len);
+		iop_set_range_uptodate(folio->mapping->host, folio, offset,
+				       len);
 	}
 
 	if (!iop || atomic_sub_and_test(len, &iop->read_bytes_pending))
@@ -240,7 +238,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	memcpy(addr, iomap->inline_data, size);
 	memset(addr + size, 0, PAGE_SIZE - poff - size);
 	kunmap_local(addr);
-	iomap_set_range_uptodate(folio, iop, offset, PAGE_SIZE - poff);
+	iop_set_range_uptodate(iter->inode, folio, offset, PAGE_SIZE - poff);
 	return 0;
 }
 
@@ -277,7 +275,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 
 	if (iomap_block_needs_zeroing(iter, pos)) {
 		folio_zero_range(folio, poff, plen);
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 		goto done;
 	}
 
@@ -598,7 +596,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 			if (status)
 				return status;
 		}
-		iomap_set_range_uptodate(folio, iop, poff, plen);
+		iop_set_range_uptodate(iter->inode, folio, poff, plen);
 	} while ((block_start += plen) < block_end);
 
 	return 0;
@@ -705,7 +703,6 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos,
 static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 		size_t copied, struct folio *folio)
 {
-	struct iomap_page *iop = to_iomap_page(folio);
 	flush_dcache_folio(folio);
 
 	/*
@@ -721,7 +718,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len,
 	 */
 	if (unlikely(copied < len && !folio_test_uptodate(folio)))
 		return 0;
-	iomap_set_range_uptodate(folio, iop, offset_in_folio(folio, pos), len);
+	iop_set_range_uptodate(inode, folio, offset_in_folio(folio, pos), len);
 	filemap_dirty_folio(inode->i_mapping, folio);
 	return copied;
 }
--

2.39.2

#9 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

In reply to Ritesh Harjani (IBM)

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 64

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+	if (iop) {
+		spin_lock_irqsave(&iop->uptodate_lock, flags);
+		bitmap_set(iop->uptodate, first_blk, nr_blks);
+		if (bitmap_full(iop->uptodate,
+				i_blocks_per_folio(inode, folio)))
+			folio_mark_uptodate(folio);
+		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+	} else {
+		folio_mark_uptodate(folio);
+	}

If we did a:

	if (!iop) {
		folio_mark_uptodate(folio);
		return;
	}

we can remove a leel of identation and keep thing a bit simpler. But I can live with either style.

#10 lore raw

Re: [RFCv5 2/5] iomap: Refactor iop_set_range_uptodate() function

Quoted 13 lines · Christoph Hellwig

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 64

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+	if (iop) {
+		spin_lock_irqsave(&iop->uptodate_lock, flags);
+		bitmap_set(iop->uptodate, first_blk, nr_blks);
+		if (bitmap_full(iop->uptodate,
+				i_blocks_per_folio(inode, folio)))
+			folio_mark_uptodate(folio);
+		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+	} else {
+		folio_mark_uptodate(folio);
+	}

If we did a:

Quoted 7 lines
	if (!iop) {
		folio_mark_uptodate(folio);
		return;
	}

we can remove a leel of identation and keep thing a bit simpler. But I can live with either style.

Yup, it pleases my eyes too!! I will change it.

-ritesh

#11 lore raw

RFC v5 3/5 iomap: Add iop's uptodate state handling functions

In reply to Ritesh Harjani (IBM)

Firstly this patch renames iop->uptodate to iop->state bitmap. This is because we will add dirty state to iop->state bitmap in later patches. So it makes sense to rename the iop->uptodate bitmap to iop->state.

Secondly this patch also adds other helpers for uptodate state bitmap handling of iop->state.

No functionality change in this patch.

---
 fs/iomap/buffered-io.c | 78 +++++++++++++++++++++++++++++++-----------
 1 file changed, 58 insertions(+), 20 deletions(-)
fs/iomap/buffered-io.c +58 -20
index e732581dc2d4..5103b644e115 100644
@@ -24,14 +24,14 @@
 #define IOEND_BATCH_SIZE	4096
 
 /*
- * Structure allocated for each folio when block size < folio size
- * to track sub-folio uptodate status and I/O completions.
+ * Structure allocated for each folio to track per-block uptodate state
+ * and I/O completions.
  */
 struct iomap_page {
 	atomic_t		read_bytes_pending;
 	atomic_t		write_bytes_pending;
-	spinlock_t		uptodate_lock;
-	unsigned long		uptodate[];
+	spinlock_t		state_lock;
+	unsigned long		state[];
 };
 
 static inline struct iomap_page *to_iomap_page(struct folio *folio)
@@ -43,6 +43,47 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)
 
 static struct bio_set iomap_ioend_bioset;
 
+/*
+ * inline helpers for bitmap operations on iop->state
+ */
+static inline void iop_set_range(struct iomap_page *iop, unsigned int start_blk,
+				 unsigned int nr_blks)
+{
+	bitmap_set(iop->state, start_blk, nr_blks);
+}
+
+static inline bool iop_test_block(struct iomap_page *iop, unsigned int block)
+{
+	return test_bit(block, iop->state);
+}
+
+static inline bool iop_bitmap_full(struct iomap_page *iop,
+				   unsigned int blks_per_folio)
+{
+	return bitmap_full(iop->state, blks_per_folio);
+}
+
+/*
+ * iop related helpers for checking uptodate/dirty state of per-block
+ * or range of blocks within a folio
+ */
+static bool iop_test_full_uptodate(struct folio *folio)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	struct inode *inode = folio->mapping->host;
+
+	WARN_ON(!iop);
+	return iop_bitmap_full(iop, i_blocks_per_folio(inode, folio));
+}
+
+static bool iop_test_block_uptodate(struct folio *folio, unsigned int block)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+
+	WARN_ON(!iop);
+	return iop_test_block(iop, block);
+}
+
 static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 				   size_t off, size_t len)
 {
@@ -53,12 +94,11 @@ static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 	unsigned long flags;
 
 	if (iop) {
-		spin_lock_irqsave(&iop->uptodate_lock, flags);
-		bitmap_set(iop->uptodate, first_blk, nr_blks);
-		if (bitmap_full(iop->uptodate,
-				i_blocks_per_folio(inode, folio)))
+		spin_lock_irqsave(&iop->state_lock, flags);
+		iop_set_range(iop, first_blk, nr_blks);
+		if (iop_test_full_uptodate(folio))
 			folio_mark_uptodate(folio);
-		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+		spin_unlock_irqrestore(&iop->state_lock, flags);
 	} else {
 		folio_mark_uptodate(folio);
 	}
@@ -79,12 +119,12 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)),
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
 		      gfp);
 	if (iop) {
-		spin_lock_init(&iop->uptodate_lock);
+		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
-			bitmap_fill(iop->uptodate, nr_blocks);
+			iop_set_range(iop, 0, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
 	return iop;
@@ -93,15 +133,13 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 static void iop_free(struct folio *folio)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
-	struct inode *inode = folio->mapping->host;
-	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
 
 	if (!iop)
 		return;
 	WARN_ON_ONCE(atomic_read(&iop->read_bytes_pending));
 	WARN_ON_ONCE(atomic_read(&iop->write_bytes_pending));
-	WARN_ON_ONCE(bitmap_full(iop->uptodate, nr_blocks) !=
-			folio_test_uptodate(folio));
+	WARN_ON_ONCE(iop_test_full_uptodate(folio) !=
+		     folio_test_uptodate(folio));
 	folio_detach_private(folio);
 	kfree(iop);
 }
@@ -132,7 +170,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* move forward for each leading block marked uptodate */
 		for (i = first; i <= last; i++) {
-			if (!test_bit(i, iop->uptodate))
+			if (!iop_test_block_uptodate(folio, i))
 				break;
 			*pos += block_size;
 			poff += block_size;
@@ -142,7 +180,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* truncate len if we find any trailing uptodate block(s) */
 		for ( ; i <= last; i++) {
-			if (test_bit(i, iop->uptodate)) {
+			if (iop_test_block_uptodate(folio, i)) {
 				plen -= (last - i + 1) * block_size;
 				last = i - 1;
 				break;
@@ -450,7 +488,7 @@ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count)
 	last = (from + count - 1) >> inode->i_blkbits;
 
 	for (i = first; i <= last; i++)
-		if (!test_bit(i, iop->uptodate))
+		if (!iop_test_block_uptodate(folio, i))
 			return false;
 	return true;
 }
@@ -1634,7 +1672,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !test_bit(i, iop->uptodate))
+		if (iop && !iop_test_block_uptodate(folio, i))
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
Signature · 1 line
2.39.2
#12 lore raw

Re: [RFCv5 3/5] iomap: Add iop's uptodate state handling functions

Quoted 19 lines · Ritesh Harjani (IBM)

Firstly this patch renames iop->uptodate to iop->state bitmap. This is because we will add dirty state to iop->state bitmap in later patches. So it makes sense to rename the iop->uptodate bitmap to iop->state.

Secondly this patch also adds other helpers for uptodate state bitmap handling of iop->state.

No functionality change in this patch.

---
 fs/iomap/buffered-io.c | 78 +++++++++++++++++++++++++++++++-----------
 1 file changed, 58 insertions(+), 20 deletions(-)
fs/iomap/buffered-io.c +0 -0
index e732581dc2d4..5103b644e115 100644

...

Quoted 1 line
fs/iomap/buffered-io.c +0 -0
@@ -43,6 +43,47 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)

...

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 75

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+/*
+ * iop related helpers for checking uptodate/dirty state of per-block
+ * or range of blocks within a folio
+ */
+static bool iop_test_full_uptodate(struct folio *folio)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	struct inode *inode = folio->mapping->host;
+
+	WARN_ON(!iop);

It looks like an oops or something is imminent here if iop is NULL. Why the warn (here and in a couple other places)?

Brian

Quoted 104 lines
fs/iomap/buffered-io.c +24 -16
+	return iop_bitmap_full(iop, i_blocks_per_folio(inode, folio));
+}
+
+static bool iop_test_block_uptodate(struct folio *folio, unsigned int block)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+
+	WARN_ON(!iop);
+	return iop_test_block(iop, block);
+}
+
 static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 				   size_t off, size_t len)
 {
@@ -53,12 +94,11 @@ static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 	unsigned long flags;
 
 	if (iop) {
-		spin_lock_irqsave(&iop->uptodate_lock, flags);
-		bitmap_set(iop->uptodate, first_blk, nr_blks);
-		if (bitmap_full(iop->uptodate,
-				i_blocks_per_folio(inode, folio)))
+		spin_lock_irqsave(&iop->state_lock, flags);
+		iop_set_range(iop, first_blk, nr_blks);
+		if (iop_test_full_uptodate(folio))
 			folio_mark_uptodate(folio);
-		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+		spin_unlock_irqrestore(&iop->state_lock, flags);
 	} else {
 		folio_mark_uptodate(folio);
 	}
@@ -79,12 +119,12 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)),
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
 		      gfp);
 	if (iop) {
-		spin_lock_init(&iop->uptodate_lock);
+		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
-			bitmap_fill(iop->uptodate, nr_blocks);
+			iop_set_range(iop, 0, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
 	return iop;
@@ -93,15 +133,13 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 static void iop_free(struct folio *folio)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
-	struct inode *inode = folio->mapping->host;
-	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
 
 	if (!iop)
 		return;
 	WARN_ON_ONCE(atomic_read(&iop->read_bytes_pending));
 	WARN_ON_ONCE(atomic_read(&iop->write_bytes_pending));
-	WARN_ON_ONCE(bitmap_full(iop->uptodate, nr_blocks) !=
-			folio_test_uptodate(folio));
+	WARN_ON_ONCE(iop_test_full_uptodate(folio) !=
+		     folio_test_uptodate(folio));
 	folio_detach_private(folio);
 	kfree(iop);
 }
@@ -132,7 +170,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* move forward for each leading block marked uptodate */
 		for (i = first; i <= last; i++) {
-			if (!test_bit(i, iop->uptodate))
+			if (!iop_test_block_uptodate(folio, i))
 				break;
 			*pos += block_size;
 			poff += block_size;
@@ -142,7 +180,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* truncate len if we find any trailing uptodate block(s) */
 		for ( ; i <= last; i++) {
-			if (test_bit(i, iop->uptodate)) {
+			if (iop_test_block_uptodate(folio, i)) {
 				plen -= (last - i + 1) * block_size;
 				last = i - 1;
 				break;
@@ -450,7 +488,7 @@ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count)
 	last = (from + count - 1) >> inode->i_blkbits;
 
 	for (i = first; i <= last; i++)
-		if (!test_bit(i, iop->uptodate))
+		if (!iop_test_block_uptodate(folio, i))
 			return false;
 	return true;
 }
@@ -1634,7 +1672,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !test_bit(i, iop->uptodate))
+		if (iop && !iop_test_block_uptodate(folio, i))
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
Signature · 1 line
2.39.2
#13 lore raw

Re: [RFCv5 3/5] iomap: Add iop's uptodate state handling functions

Quoted 23 lines · Brian Foster
Quoted 19 lines · Ritesh Harjani (IBM)

Firstly this patch renames iop->uptodate to iop->state bitmap. This is because we will add dirty state to iop->state bitmap in later patches. So it makes sense to rename the iop->uptodate bitmap to iop->state.

Secondly this patch also adds other helpers for uptodate state bitmap handling of iop->state.

No functionality change in this patch.

---
 fs/iomap/buffered-io.c | 78 +++++++++++++++++++++++++++++++-----------
 1 file changed, 58 insertions(+), 20 deletions(-)
fs/iomap/buffered-io.c +0 -0
index e732581dc2d4..5103b644e115 100644

...

Quoted 1 line
fs/iomap/buffered-io.c +0 -0
@@ -43,6 +43,47 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio)

...

Quoted 14 lines

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 75

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+/*
+ * iop related helpers for checking uptodate/dirty state of per-block
+ * or range of blocks within a folio
+ */
+static bool iop_test_full_uptodate(struct folio *folio)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	struct inode *inode = folio->mapping->host;
+
+	WARN_ON(!iop);

It looks like an oops or something is imminent here if iop is NULL. Why the warn (here and in a couple other places)?

I agree. I will remove it in the next revision.

-ritesh

Quoted 106 lines

Brian

Quoted 104 lines
fs/iomap/buffered-io.c +24 -17
+	return iop_bitmap_full(iop, i_blocks_per_folio(inode, folio));
+}
+
+static bool iop_test_block_uptodate(struct folio *folio, unsigned int block)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+
+	WARN_ON(!iop);
+	return iop_test_block(iop, block);
+}
+
 static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 				   size_t off, size_t len)
 {
@@ -53,12 +94,11 @@ static void iop_set_range_uptodate(struct inode *inode, struct folio *folio,
 	unsigned long flags;
 
 	if (iop) {
-		spin_lock_irqsave(&iop->uptodate_lock, flags);
-		bitmap_set(iop->uptodate, first_blk, nr_blks);
-		if (bitmap_full(iop->uptodate,
-				i_blocks_per_folio(inode, folio)))
+		spin_lock_irqsave(&iop->state_lock, flags);
+		iop_set_range(iop, first_blk, nr_blks);
+		if (iop_test_full_uptodate(folio))
 			folio_mark_uptodate(folio);
-		spin_unlock_irqrestore(&iop->uptodate_lock, flags);
+		spin_unlock_irqrestore(&iop->state_lock, flags);
 	} else {
 		folio_mark_uptodate(folio);
 	}
@@ -79,12 +119,12 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)),
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
 		      gfp);
 	if (iop) {
-		spin_lock_init(&iop->uptodate_lock);
+		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
-			bitmap_fill(iop->uptodate, nr_blocks);
+			iop_set_range(iop, 0, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
 	return iop;
@@ -93,15 +133,13 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 static void iop_free(struct folio *folio)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
-	struct inode *inode = folio->mapping->host;
-	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
 
 	if (!iop)
 		return;
 	WARN_ON_ONCE(atomic_read(&iop->read_bytes_pending));
 	WARN_ON_ONCE(atomic_read(&iop->write_bytes_pending));
-	WARN_ON_ONCE(bitmap_full(iop->uptodate, nr_blocks) !=
-			folio_test_uptodate(folio));
+	WARN_ON_ONCE(iop_test_full_uptodate(folio) !=
+		     folio_test_uptodate(folio));
 	folio_detach_private(folio);
 	kfree(iop);
 }
@@ -132,7 +170,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* move forward for each leading block marked uptodate */
 		for (i = first; i <= last; i++) {
-			if (!test_bit(i, iop->uptodate))
+			if (!iop_test_block_uptodate(folio, i))
 				break;
 			*pos += block_size;
 			poff += block_size;
@@ -142,7 +180,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio,
 
 		/* truncate len if we find any trailing uptodate block(s) */
 		for ( ; i <= last; i++) {
-			if (test_bit(i, iop->uptodate)) {
+			if (iop_test_block_uptodate(folio, i)) {
 				plen -= (last - i + 1) * block_size;
 				last = i - 1;
 				break;
@@ -450,7 +488,7 @@ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count)
 	last = (from + count - 1) >> inode->i_blkbits;
 
 	for (i = first; i <= last; i++)
-		if (!test_bit(i, iop->uptodate))
+		if (!iop_test_block_uptodate(folio, i))
 			return false;
 	return true;
 }
@@ -1634,7 +1672,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !test_bit(i, iop->uptodate))
+		if (iop && !iop_test_block_uptodate(folio, i))
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
--

2.39.2

#14 lore raw

Re: [RFCv5 3/5] iomap: Add iop's uptodate state handling functions

In reply to Ritesh Harjani (IBM)
Quoted 8 lines

+ * inline helpers for bitmap operations on iop->state
+ */

fs/iomap/buffered-io.c +6 -0
+static inline void iop_set_range(struct iomap_page *iop, unsigned int start_blk,
+				 unsigned int nr_blks)
+{
+	bitmap_set(iop->state, start_blk, nr_blks);
+}
+

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 64

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+static inline bool iop_test_block(struct iomap_page *iop, unsigned int block)
+{
+	return test_bit(block, iop->state);
+}
+
+static inline bool iop_bitmap_full(struct iomap_page *iop,
+				   unsigned int blks_per_folio)
+{
+	return bitmap_full(iop->state, blks_per_folio);
+}

I don't really see much poin in these helpers, any particular reason for adding them?

Quoted 4 lines

+/*
+ * iop related helpers for checking uptodate/dirty state of per-block
+ * or range of blocks within a folio
+ */

I'm also not sure this comment adds a whole lot of value.

The rest looks good modulo the WARN_ONs already mentined by Brian.

#15 lore raw

Re: [RFCv5 3/5] iomap: Add iop's uptodate state handling functions

Quoted 13 lines · Christoph Hellwig
Quoted 13 lines

+ * inline helpers for bitmap operations on iop->state
+ */

fs/iomap/buffered-io.c +11 -0
+static inline void iop_set_range(struct iomap_page *iop, unsigned int start_blk,
+				 unsigned int nr_blks)
+{
+	bitmap_set(iop->state, start_blk, nr_blks);
+}
+
+static inline bool iop_test_block(struct iomap_page *iop, unsigned int block)
+{
+	return test_bit(block, iop->state);
+}
+
Quoted 9 lines

fs/iomap/buffered-io.c static inline struct iomap_page *to_iomap_page(struct folio *folio) · line 64

Quoted 5 lines
fs/iomap/buffered-io.c +5 -0
+static inline bool iop_bitmap_full(struct iomap_page *iop,
+				   unsigned int blks_per_folio)
+{
+	return bitmap_full(iop->state, blks_per_folio);
+}

I don't really see much poin in these helpers, any particular reason for adding them?

We ended up modifying the APIs in v5. The idea on v4 was we will keep iop_set_range() function which will be same for both uptodate and dirty. The caller can pass start_blk depending upon whether we dirty/uptodate needs to be marked.
But I guess with the API changes, we don't need this low level helpers anymore. So If no one has any objection, I can kill this one liners.

Quoted 8 lines
Quoted 4 lines

+/*
+ * iop related helpers for checking uptodate/dirty state of per-block
+ * or range of blocks within a folio
+ */

I'm also not sure this comment adds a whole lot of value.

The rest looks good modulo the WARN_ONs already mentined by Brian.

Sure. Thanks for the review!

-ritesh

#16 lore raw

Re: [RFCv5 3/5] iomap: Add iop's uptodate state handling functions

Quoted 6 lines · Ritesh Harjani

We ended up modifying the APIs in v5. The idea on v4 was we will keep iop_set_range() function which will be same for both uptodate and dirty. The caller can pass start_blk depending upon whether we dirty/uptodate needs to be marked.
But I guess with the API changes, we don't need this low level helpers anymore. So If no one has any objection, I can kill this one liners.

Yes, the actually uptodate/dirty helper isolate the caller from the API. We don't really need another tiny wrapper here.

#17 lore raw

RFC v5 4/5 iomap: Allocate iop in ->write_begin() early

In reply to Ritesh Harjani (IBM)

Earlier when the folio is uptodate, we only allocate iop at writeback time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iop, this could cause some performance degradation. The reason is that if we don't allocate iop during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now.

However, for all the writes with (pos, len) which completely overlaps the given folio, there is no need to allocate an iop during ->write_begin(). So skip those cases.

---
 fs/iomap/buffered-io.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
fs/iomap/buffered-io.c +12 -2
index 5103b644e115..25f20f269214 100644
@@ -599,15 +599,25 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 	size_t from = offset_in_folio(folio, pos), to = from + len;
 	size_t poff, plen;
 
-	if (folio_test_uptodate(folio))
+	/*
+	 * If the write completely overlaps the current folio, then
+	 * entire folio will be dirtied so there is no need for
+	 * per-block state tracking structures to be attached to this folio.
+	 */
+	if (pos <= folio_pos(folio) &&
+	    pos + len >= folio_pos(folio) + folio_size(folio))
 		return 0;
-	folio_clear_error(folio);
 
 	iop = iop_alloc(iter->inode, folio, iter->flags);
 
 	if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1)
 		return -EAGAIN;
 
+	if (folio_test_uptodate(folio))
+		return 0;
+	folio_clear_error(folio);
+
+
 	do {
 		iomap_adjust_read_range(iter->inode, folio, &block_start,
 				block_end - block_start, &poff, &plen);
Signature · 1 line
2.39.2
#18 lore raw

Re: [RFCv5 4/5] iomap: Allocate iop in ->write_begin() early

Quoted 1 line · Ritesh Harjani (IBM)

Earlier when the folio is uptodate, we only allocate iop at writeback

s/Earlier/Currently/ ?

Quoted 11 lines

time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iop, this could cause some performance degradation. The reason is that if we don't allocate iop during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now.

However, for all the writes with (pos, len) which completely overlaps the given folio, there is no need to allocate an iop during ->write_begin(). So skip those cases.

This reads a bit backwards, I'd suggest to mention early allocation only happens for sub-page writes before going into the details.

The changes themselves looks good to me.

#19 lore raw

Re: [RFCv5 4/5] iomap: Allocate iop in ->write_begin() early

Quoted 13 lines · Christoph Hellwig
Quoted 1 line · Ritesh Harjani (IBM)

Earlier when the folio is uptodate, we only allocate iop at writeback

s/Earlier/Currently/ ?

Quoted 8 lines

time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iop, this could cause some performance degradation. The reason is that if we don't allocate iop during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now.

Quoted 8 lines
Quoted 3 lines

However, for all the writes with (pos, len) which completely overlaps the given folio, there is no need to allocate an iop during ->write_begin(). So skip those cases.

This reads a bit backwards, I'd suggest to mention early allocation only happens for sub-page writes before going into the details.

sub-page is a bit confusing here. Because we can have a large folio too with blocks within that folio. So we decided to go with per-block terminology [1].

[1]: https://lore.kernel.org/linux-xfs/ZFR%[email protected]/

I am guessing you would like to me to re-write the above para. Is this better?

"We dont need to allocate an iop in ->write_begin() for writes where the position and length completely overlap with the given folio. Therefore, such cases are skipped."

Quoted 1 line

The changes themselves looks good to me.

Sure. Thanks!

-ritesh

#20 lore raw

Re: [RFCv5 4/5] iomap: Allocate iop in ->write_begin() early

Quoted 28 lines · Ritesh Harjani
Quoted 13 lines · Christoph Hellwig
Quoted 1 line · Ritesh Harjani (IBM)

Earlier when the folio is uptodate, we only allocate iop at writeback

s/Earlier/Currently/ ?

Quoted 8 lines

time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iop, this could cause some performance degradation. The reason is that if we don't allocate iop during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now.

Quoted 8 lines
Quoted 3 lines

However, for all the writes with (pos, len) which completely overlaps the given folio, there is no need to allocate an iop during ->write_begin(). So skip those cases.

This reads a bit backwards, I'd suggest to mention early allocation only happens for sub-page writes before going into the details.

sub-page is a bit confusing here. Because we can have a large folio too with blocks within that folio. So we decided to go with per-block terminology [1].

Quoted 7 lines

[1]: https://lore.kernel.org/linux-xfs/ZFR%[email protected]/

I am guessing you would like to me to re-write the above para. Is this better?

"We dont need to allocate an iop in ->write_begin() for writes where the position and length completely overlap with the given folio. Therefore, such cases are skipped."

... and reorder that paragraph to be first.

#21 lore raw

Re: [RFCv5 4/5] iomap: Allocate iop in ->write_begin() early

Quoted 31 lines · Matthew Wilcox
Quoted 30 lines · Ritesh Harjani
Quoted 13 lines · Christoph Hellwig
Quoted 1 line · Ritesh Harjani (IBM)

Earlier when the folio is uptodate, we only allocate iop at writeback

s/Earlier/Currently/ ?

Quoted 8 lines

time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iop, this could cause some performance degradation. The reason is that if we don't allocate iop during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now.

Quoted 8 lines
Quoted 3 lines

However, for all the writes with (pos, len) which completely overlaps the given folio, there is no need to allocate an iop during ->write_begin(). So skip those cases.

This reads a bit backwards, I'd suggest to mention early allocation only happens for sub-page writes before going into the details.

sub-page is a bit confusing here. Because we can have a large folio too with blocks within that folio. So we decided to go with per-block terminology [1].

[1]: https://lore.kernel.org/linux-xfs/ZFR%[email protected]/

Quoted 7 lines
Quoted 5 lines

I am guessing you would like to me to re-write the above para. Is this better?

"We dont need to allocate an iop in ->write_begin() for writes where the position and length completely overlap with the given folio. Therefore, such cases are skipped."

... and reorder that paragraph to be first.

Sure.

-ritesh

#22 lore raw

RFC v5 5/5 iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)
fs/gfs2/aops.c +1 -1
index a5f4be6b9213..75efec3c3b71 100644
@@ -746,7 +746,7 @@ static const struct address_space_operations gfs2_aops = {
 	.writepages = gfs2_writepages,
 	.read_folio = gfs2_read_folio,
 	.readahead = gfs2_readahead,
-	.dirty_folio = filemap_dirty_folio,
+	.dirty_folio = iomap_dirty_folio,
 	.release_folio = iomap_release_folio,
 	.invalidate_folio = iomap_invalidate_folio,
 	.bmap = gfs2_bmap,
fs/iomap/buffered-io.c +108 -7

205 lines not on this page · raw message

fs/xfs/xfs_aops.c +1 -1
index 2ef78aa1d3f6..77c7332ae197 100644
@@ -578,7 +578,7 @@ const struct address_space_operations xfs_address_space_operations = {
 	.read_folio		= xfs_vm_read_folio,
 	.readahead		= xfs_vm_readahead,
 	.writepages		= xfs_vm_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.bmap			= xfs_vm_bmap,
fs/zonefs/file.c +1 -1
index 132f01d3461f..e508c8e97372 100644
@@ -175,7 +175,7 @@ const struct address_space_operations zonefs_file_aops = {
 	.read_folio		= zonefs_read_folio,
 	.readahead		= zonefs_readahead,
 	.writepages		= zonefs_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.migrate_folio		= filemap_migrate_folio,
include/linux/iomap.h +1 -0
index 0f8123504e5e..0c2bee80565c 100644
@@ -264,6 +264,7 @@ bool iomap_is_partially_uptodate(struct folio *, size_t from, size_t count);
 struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos);
 bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags);
 void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len);
+bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);
 int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,
 		const struct iomap_ops *ops);
 int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len,

--
2.39.2

#23 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

kernel test robot noticed the following build warnings:

Hi Ritesh,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on xfs-linux/for-next]
[also build test WARNING on gfs2/for-next linus/master v6.4-rc1 next-20230505] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903
base:   https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next
patch link:    https://lore.kernel.org/r/86987466d8d7863bd0dca81e9d6c3eff7abd4964.1683485700.git.ritesh.list%40gmail.com
patch subject: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance
config: m68k-allyesconfig (https://download.01.org/0day-ci/archive/20230508/[email protected]/config)
compiler: m68k-linux-gcc (GCC) 12.1.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/a05d7e56a951c074e0d13c4e9908a90b4e14ad92
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903 git checkout a05d7e56a951c074e0d13c4e9908a90b4e14ad92
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k SHELL=/bin/bash fs/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

fs/iomap/buffered-io.c: In function 'iomap_read_inline_data':
fs/iomap/buffered-io.c:311:28: warning: variable 'iop' set but not used [-Wunused-but-set-variable]

     311 |         struct iomap_page *iop;
         |                            ^~~
   fs/iomap/buffered-io.c: In function 'iomap_dirty_folio':
   fs/iomap/buffered-io.c:626:28: warning: variable 'iop' set but not used [-Wunused-but-set-variable]
     626 |         struct iomap_page *iop;
         |                            ^~~
   fs/iomap/buffered-io.c: In function 'iomap_write_delalloc_scan':
>> fs/iomap/buffered-io.c:1032:51: warning: variable 'blks_per_folio' set but not used [-Wunused-but-set-variable]
    1032 |                 unsigned int first_blk, last_blk, blks_per_folio, i;
         |                                                   ^~~~~~~~~~~~~~

vim +/blks_per_folio +1032 fs/iomap/buffered-io.c

1007
1008 /*
1009 * Scan the data range passed to us for dirty page cache folios. If we find a
1010 * dirty folio, punch out the preceeding range and update the offset from which
1011 * the next punch will start from.
1012 *
1013 * We can punch out storage reservations under clean pages because they either
1014 * contain data that has been written back - in which case the delalloc punch
1015 * over that range is a no-op - or they have been read faults in which case they
1016 * contain zeroes and we can remove the delalloc backing range and any new
1017 * writes to those pages will do the normal hole filling operation...
1018 *
1019 * This makes the logic simple: we only need to keep the delalloc extents only
1020 * over the dirty ranges of the page cache.
1021 *
1022 * This function uses [start_byte, end_byte) intervals (i.e. open ended) to
1023 * simplify range iterations.
1024 */
1025 static int iomap_write_delalloc_scan(struct inode *inode,
1026 loff_t *punch_start_byte, loff_t start_byte, loff_t end_byte,
1027 int (*punch)(struct inode *inode, loff_t offset, loff_t length))
1028 {
1029 while (start_byte < end_byte) {
1030 struct folio *folio;
1031 struct iomap_page *iop;

> 1032			unsigned int first_blk, last_blk, blks_per_folio, i;

1033 loff_t last_byte;
1034 u8 blkbits = inode->i_blkbits;
1035
1036 /* grab locked page */
1037 folio = filemap_lock_folio(inode->i_mapping,
1038 start_byte >> PAGE_SHIFT);
1039 if (!folio) {
1040 start_byte = ALIGN_DOWN(start_byte, PAGE_SIZE) +
1041 PAGE_SIZE;
1042 continue;
1043 }
1044
1045 /* if dirty, punch up to offset */
1046 if (folio_test_dirty(folio)) {
1047 if (start_byte > *punch_start_byte) {
1048 int error;
1049
1050 error = punch(inode, *punch_start_byte,
1051 start_byte - *punch_start_byte);
1052 if (error) {
1053 folio_unlock(folio);
1054 folio_put(folio);
1055 return error;
1056 }
1057 }
1058
1059 /*
1060 * When we have per-block dirty tracking, there can be
1061 * blocks within a folio which are marked uptodate
1062 * but not dirty. In that case it is necessary to punch
1063 * out such blocks to avoid leaking any delalloc blocks.
1064 */
1065 iop = to_iomap_page(folio);
1066 if (!iop)
1067 goto skip_iop_punch;
1068 last_byte = min_t(loff_t, end_byte - 1,
1069 (folio_next_index(folio) << PAGE_SHIFT) - 1);
1070 first_blk = offset_in_folio(folio, start_byte) >>
1071 blkbits;
1072 last_blk = offset_in_folio(folio, last_byte) >>
1073 blkbits;
1074 blks_per_folio = i_blocks_per_folio(inode, folio);
1075 for (i = first_blk; i <= last_blk; i++) {
1076 if (!iop_test_block_dirty(folio, i))
1077 punch(inode, i << blkbits,
1078 1 << blkbits);
1079 }
1080 skip_iop_punch:
1081 /*
1082 * Make sure the next punch start is correctly bound to
1083 * the end of this data range, not the end of the folio.
1084 */
1085 *punch_start_byte = min_t(loff_t, end_byte,
1086 folio_next_index(folio) << PAGE_SHIFT);
1087 }
1088
1089 /* move offset to start of next folio in range */
1090 start_byte = folio_next_index(folio) << PAGE_SHIFT;
1091 folio_unlock(folio);
1092 folio_put(folio);
1093 }
1094 return 0;
1095 }
1096

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#24 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)
kernel test robot noticed the following build warnings:

Hi Ritesh,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on xfs-linux/for-next]
[also build test WARNING on gfs2/for-next linus/master v6.4-rc1 next-20230505] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903
base:   https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git for-next
patch link:    https://lore.kernel.org/r/86987466d8d7863bd0dca81e9d6c3eff7abd4964.1683485700.git.ritesh.list%40gmail.com
patch subject: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance
config: i386-randconfig-a001-20230508 (https://download.01.org/0day-ci/archive/20230508/[email protected]/config)
compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/a05d7e56a951c074e0d13c4e9908a90b4e14ad92
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ritesh-Harjani-IBM/iomap-Rename-iomap_page_create-release-to-iop_alloc-free/20230508-032903 git checkout a05d7e56a951c074e0d13c4e9908a90b4e14ad92
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash fs/iomap/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

fs/iomap/buffered-io.c:311:21: warning: variable 'iop' set but not used [-Wunused-but-set-variable]

           struct iomap_page *iop;
                              ^
   fs/iomap/buffered-io.c:626:21: warning: variable 'iop' set but not used [-Wunused-but-set-variable]
           struct iomap_page *iop;
                              ^
>> fs/iomap/buffered-io.c:1032:37: warning: variable 'blks_per_folio' set but not used [-Wunused-but-set-variable]
                   unsigned int first_blk, last_blk, blks_per_folio, i;
                                                     ^

3 warnings generated.

vim +/blks_per_folio +1032 fs/iomap/buffered-io.c

1007
1008 /*
1009 * Scan the data range passed to us for dirty page cache folios. If we find a
1010 * dirty folio, punch out the preceeding range and update the offset from which
1011 * the next punch will start from.
1012 *
1013 * We can punch out storage reservations under clean pages because they either
1014 * contain data that has been written back - in which case the delalloc punch
1015 * over that range is a no-op - or they have been read faults in which case they
1016 * contain zeroes and we can remove the delalloc backing range and any new
1017 * writes to those pages will do the normal hole filling operation...
1018 *
1019 * This makes the logic simple: we only need to keep the delalloc extents only
1020 * over the dirty ranges of the page cache.
1021 *
1022 * This function uses [start_byte, end_byte) intervals (i.e. open ended) to
1023 * simplify range iterations.
1024 */
1025 static int iomap_write_delalloc_scan(struct inode *inode,
1026 loff_t *punch_start_byte, loff_t start_byte, loff_t end_byte,
1027 int (*punch)(struct inode *inode, loff_t offset, loff_t length))
1028 {
1029 while (start_byte < end_byte) {
1030 struct folio *folio;
1031 struct iomap_page *iop;

> 1032			unsigned int first_blk, last_blk, blks_per_folio, i;

1033 loff_t last_byte;
1034 u8 blkbits = inode->i_blkbits;
1035
1036 /* grab locked page */
1037 folio = filemap_lock_folio(inode->i_mapping,
1038 start_byte >> PAGE_SHIFT);
1039 if (!folio) {
1040 start_byte = ALIGN_DOWN(start_byte, PAGE_SIZE) +
1041 PAGE_SIZE;
1042 continue;
1043 }
1044
1045 /* if dirty, punch up to offset */
1046 if (folio_test_dirty(folio)) {
1047 if (start_byte > *punch_start_byte) {
1048 int error;
1049
1050 error = punch(inode, *punch_start_byte,
1051 start_byte - *punch_start_byte);
1052 if (error) {
1053 folio_unlock(folio);
1054 folio_put(folio);
1055 return error;
1056 }
1057 }
1058
1059 /*
1060 * When we have per-block dirty tracking, there can be
1061 * blocks within a folio which are marked uptodate
1062 * but not dirty. In that case it is necessary to punch
1063 * out such blocks to avoid leaking any delalloc blocks.
1064 */
1065 iop = to_iomap_page(folio);
1066 if (!iop)
1067 goto skip_iop_punch;
1068 last_byte = min_t(loff_t, end_byte - 1,
1069 (folio_next_index(folio) << PAGE_SHIFT) - 1);
1070 first_blk = offset_in_folio(folio, start_byte) >>
1071 blkbits;
1072 last_blk = offset_in_folio(folio, last_byte) >>
1073 blkbits;
1074 blks_per_folio = i_blocks_per_folio(inode, folio);
1075 for (i = first_blk; i <= last_blk; i++) {
1076 if (!iop_test_block_dirty(folio, i))
1077 punch(inode, i << blkbits,
1078 1 << blkbits);
1079 }
1080 skip_iop_punch:
1081 /*
1082 * Make sure the next punch start is correctly bound to
1083 * the end of this data range, not the end of the folio.
1084 */
1085 *punch_start_byte = min_t(loff_t, end_byte,
1086 folio_next_index(folio) << PAGE_SHIFT);
1087 }
1088
1089 /* move offset to start of next folio in range */
1090 start_byte = folio_next_index(folio) << PAGE_SHIFT;
1091 folio_unlock(folio);
1092 folio_put(folio);
1093 }
1094 return 0;
1095 }
1096

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#25 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)
Quoted 10 lines
fs/iomap/buffered-io.c +1 -1
@@ -1666,7 +1766,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0);
+	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);

fs/iomap/buffered-io.c iomap_writepage_map(struct iomap_writepage_ctx *wpc, · line 1785

Quoted 5 lines
fs/iomap/buffered-io.c +1 -1
@@ -1682,7 +1782,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !iop_test_block_uptodate(folio, i))
+		if (iop && !iop_test_block_dirty(folio, i))

Shouldn't this be if(iop && iop_test_block_dirty(folio, i))?

Before we were skipping if the blocks were not uptodate but now we are skipping if the blocks are not dirty (which means they are uptodate)?

I am new to iomap but let me know if I am missing something here.

Quoted 3 lines
fs/iomap/buffered-io.c +0 -0
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
#26 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 18 lines · Pankaj Raghav
Quoted 10 lines
fs/iomap/buffered-io.c +1 -1
@@ -1666,7 +1766,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0);
+	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);

fs/iomap/buffered-io.c iomap_writepage_map(struct iomap_writepage_ctx *wpc, · line 1785

Quoted 5 lines
fs/iomap/buffered-io.c +1 -1
@@ -1682,7 +1782,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !iop_test_block_uptodate(folio, i))
+		if (iop && !iop_test_block_dirty(folio, i))

Shouldn't this be if(iop && iop_test_block_dirty(folio, i))?

Quoted 5 lines

Before we were skipping if the blocks were not uptodate but now we are skipping if the blocks are not dirty (which means they are uptodate)?

I am new to iomap but let me know if I am missing something here.

We set the per-block dirty status in ->write_begin. The check above happens in the writeback path when we are about to write the dirty data to the disk. What the check is doing is that, it checks if the block state is not dirty then skip it which means the block was not written to in the ->write_begin(). Also the block with dirty status always means that the block is uptodate anyways.

Hope it helps!

-ritesh

#27 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 10 lines

We set the per-block dirty status in ->write_begin. The check above happens in the writeback path when we are about to write the dirty data to the disk. What the check is doing is that, it checks if the block state is not dirty then skip it which means the block was not written to in the ->write_begin(). Also the block with dirty status always means that the block is uptodate anyways.

Hope it helps!

Definitely. I also checked your 1st RFC version to get more context. Thanks for the explanation.

#28 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)
Quoted 47 lines · Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
index 25f20f269214..c7f41b26280a 100644

...

Quoted 9 lines
fs/iomap/buffered-io.c +4 -1
@@ -119,12 +169,20 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
+	/*
+	 * iop->state tracks two sets of state flags when the
+	 * filesystem block size is smaller than the folio size.
+	 * The first state tracks per-block uptodate and the

fs/iomap/buffered-io.c static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio, · line 185

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
+	 * second tracks per-block dirty state.
+	 */
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(2 * nr_blocks)),
 		      gfp);
 	if (iop) {
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
+		if (is_dirty)
+			iop_set_range(iop, nr_blocks, nr_blocks);

I find the is_dirty logic here a bit confusing. AFAICT, this is primarily to handle the case where a folio is completely overwritten, so no iop is allocated at write time, and so then writeback needs to allocate the iop as fully dirty to reflect that. I.e., all iop_alloc() callers other than iomap_writepage_map() either pass the result of folio_test_dirty() or explicitly dirty the entire range of the folio anyways. iomap_dirty_folio() essentially does the latter because it needs to dirty the entire folio regardless of whether the iop already exists or not, right?

If so and if I'm following all of that correctly, could this complexity be isolated to iomap_writepage_map() by simply checking for the !iop case first, then call iop_alloc() immediately followed by set_range_dirty() of the entire folio? Then presumably iop_alloc() could always just dirty based on folio state with the writeback path exception case handled explicitly. Hm?

Quoted 3 lines
fs/iomap/buffered-io.c +0 -0
 		folio_attach_private(folio, iop);
 	}
 	return iop;

...

fs/iomap/buffered-io.c void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len) · line 628

Quoted 9 lines
fs/iomap/buffered-io.c +5 -0
@@ -561,6 +621,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)
 }
 EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
+bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
+{
+	struct iomap_page *iop;
+	struct inode *inode = mapping->host;
+	size_t len = i_blocks_per_folio(inode, folio) << inode->i_blkbits;

folio_size()?

Quoted 10 lines
fs/iomap/buffered-io.c +7 -0
+
+	iop = iop_alloc(inode, folio, 0, false);
+	iop_set_range_dirty(inode, folio, 0, len);
+	return filemap_dirty_folio(mapping, folio);
+}
+EXPORT_SYMBOL_GPL(iomap_dirty_folio);
+
 static void
 iomap_write_failed(struct inode *inode, loff_t pos, unsigned len)
 {

...

Quoted 10 lines
fs/iomap/buffered-io.c +6 -0
@@ -978,6 +1056,28 @@ static int iomap_write_delalloc_scan(struct inode *inode,
 				}
 			}
 
+			/*
+			 * When we have per-block dirty tracking, there can be
+			 * blocks within a folio which are marked uptodate
+			 * but not dirty. In that case it is necessary to punch
+			 * out such blocks to avoid leaking any delalloc blocks.
+			 */

fs/iomap/buffered-io.c static int iomap_write_delalloc_scan(struct inode *inode, · line 1074

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+			iop = to_iomap_page(folio);
+			if (!iop)
+				goto skip_iop_punch;
+			last_byte = min_t(loff_t, end_byte - 1,
+				(folio_next_index(folio) << PAGE_SHIFT) - 1);
+			first_blk = offset_in_folio(folio, start_byte) >>
+						    blkbits;
+			last_blk = offset_in_folio(folio, last_byte) >>
+						   blkbits;
+			blks_per_folio = i_blocks_per_folio(inode, folio);

Unused?

fs/iomap/buffered-io.c static int iomap_write_delalloc_scan(struct inode *inode, · line 1080

Quoted 6 lines
fs/iomap/buffered-io.c +6 -0
+			for (i = first_blk; i <= last_blk; i++) {
+				if (!iop_test_block_dirty(folio, i))
+					punch(inode, i << blkbits,
+						     1 << blkbits);
+			}
+skip_iop_punch:

Looks reasonable at first glance, though the deep indentation here kind of makes my eyes cross. Could we stuff this loop into a iomap_write_delalloc_scan_folio() helper or some such, and then maybe update the comment at the top of the whole branch to explain both punches?

WRT to the !iop case.. I _think_ this is handled correctly here because that means we'd handle the folio as completely dirty at writeback time. Is that the case? If so, it might be nice to document that assumption somewhere (here or perhaps in the writeback path).

Brian

Quoted 70 lines
fs/iomap/buffered-io.c +3 -2
 			/*
 			 * Make sure the next punch start is correctly bound to
 			 * the end of this data range, not the end of the folio.
@@ -1666,7 +1766,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0);
+	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
@@ -1682,7 +1782,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !iop_test_block_uptodate(folio, i))
+		if (iop && !iop_test_block_dirty(folio, i))
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
@@ -1726,6 +1826,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		}
 	}
 
+	iop_clear_range_dirty(folio, 0, end_pos - folio_pos(folio));
 	folio_start_writeback(folio);
 	folio_unlock(folio);
 
fs/xfs/xfs_aops.c +1 -1
index 2ef78aa1d3f6..77c7332ae197 100644
@@ -578,7 +578,7 @@ const struct address_space_operations xfs_address_space_operations = {
 	.read_folio		= xfs_vm_read_folio,
 	.readahead		= xfs_vm_readahead,
 	.writepages		= xfs_vm_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.bmap			= xfs_vm_bmap,
fs/zonefs/file.c +1 -1
index 132f01d3461f..e508c8e97372 100644
@@ -175,7 +175,7 @@ const struct address_space_operations zonefs_file_aops = {
 	.read_folio		= zonefs_read_folio,
 	.readahead		= zonefs_readahead,
 	.writepages		= zonefs_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.migrate_folio		= filemap_migrate_folio,
include/linux/iomap.h +1 -1
index 0f8123504e5e..0c2bee80565c 100644
@@ -264,6 +264,7 @@ bool iomap_is_partially_uptodate(struct folio *, size_t from, size_t count);
 struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos);
 bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags);
 void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len);
+bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);
 int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,
 		const struct iomap_ops *ops);
 int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len,
--

2.39.2

#29 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 75 lines · Brian Foster
Quoted 47 lines · Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
index 25f20f269214..c7f41b26280a 100644

...

Quoted 9 lines
fs/iomap/buffered-io.c +4 -1
@@ -119,12 +169,20 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
+	/*
+	 * iop->state tracks two sets of state flags when the
+	 * filesystem block size is smaller than the folio size.
+	 * The first state tracks per-block uptodate and the

fs/iomap/buffered-io.c static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio, · line 185

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
+	 * second tracks per-block dirty state.
+	 */
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(2 * nr_blocks)),
 		      gfp);
 	if (iop) {
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
+		if (is_dirty)
+			iop_set_range(iop, nr_blocks, nr_blocks);

I find the is_dirty logic here a bit confusing. AFAICT, this is

Quoted 8 lines

primarily to handle the case where a folio is completely overwritten, so no iop is allocated at write time, and so then writeback needs to allocate the iop as fully dirty to reflect that. I.e., all iop_alloc() callers other than iomap_writepage_map() either pass the result of folio_test_dirty() or explicitly dirty the entire range of the folio anyways. iomap_dirty_folio() essentially does the latter because it needs to dirty the entire folio regardless of whether the iop already exists or not, right?

Yes.

Quoted 8 lines

If so and if I'm following all of that correctly, could this complexity be isolated to iomap_writepage_map() by simply checking for the !iop case first, then call iop_alloc() immediately followed by set_range_dirty() of the entire folio? Then presumably iop_alloc() could always just dirty based on folio state with the writeback path exception case handled explicitly. Hm?

Hi Brian,

It was discussed here [1] to pass is_dirty flag at the time of iop allocation. We can do what you are essentially suggesting, but it's just extra work i.e. we will again do some calculations of blocks_per_folio, start, end and more importantly take and release iop->state_lock spinlock. Whereas with above approach we could get away with this at the time of iop allocation itself.

Besides, isn't it easier this way? which as you also stated we will dirty all the blocks based on is_dirty flag, which is folio_test_dirty() except at the writeback time.

Quoted 8 lines
Quoted 3 lines
fs/iomap/buffered-io.c +0 -0
 		folio_attach_private(folio, iop);
 	}
 	return iop;

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
@@ -561,6 +621,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)
 }
 EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
Quoted 8 lines

fs/iomap/buffered-io.c void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len) · line 628

Quoted 5 lines
fs/iomap/buffered-io.c +5 -0
+bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
+{
+	struct iomap_page *iop;
+	struct inode *inode = mapping->host;
+	size_t len = i_blocks_per_folio(inode, folio) << inode->i_blkbits;

folio_size()?

sure.

Quoted 25 lines
Quoted 10 lines
fs/iomap/buffered-io.c +7 -0
+
+	iop = iop_alloc(inode, folio, 0, false);
+	iop_set_range_dirty(inode, folio, 0, len);
+	return filemap_dirty_folio(mapping, folio);
+}
+EXPORT_SYMBOL_GPL(iomap_dirty_folio);
+
 static void
 iomap_write_failed(struct inode *inode, loff_t pos, unsigned len)
 {

...

Quoted 14 lines
fs/iomap/buffered-io.c +10 -0
@@ -978,6 +1056,28 @@ static int iomap_write_delalloc_scan(struct inode *inode,
 				}
 			}
 
+			/*
+			 * When we have per-block dirty tracking, there can be
+			 * blocks within a folio which are marked uptodate
+			 * but not dirty. In that case it is necessary to punch
+			 * out such blocks to avoid leaking any delalloc blocks.
+			 */
+			iop = to_iomap_page(folio);
+			if (!iop)
+				goto skip_iop_punch;
+			last_byte = min_t(loff_t, end_byte - 1,
Quoted 9 lines

fs/iomap/buffered-io.c static int iomap_write_delalloc_scan(struct inode *inode, · line 1074

Quoted 6 lines
fs/iomap/buffered-io.c +6 -0
+				(folio_next_index(folio) << PAGE_SHIFT) - 1);
+			first_blk = offset_in_folio(folio, start_byte) >>
+						    blkbits;
+			last_blk = offset_in_folio(folio, last_byte) >>
+						   blkbits;
+			blks_per_folio = i_blocks_per_folio(inode, folio);

Unused?

Yes. I will fix it in next rev somehow compilation didn't give me any warnings.

Quoted 4 lines
Quoted 4 lines
fs/iomap/buffered-io.c +4 -0
+			for (i = first_blk; i <= last_blk; i++) {
+				if (!iop_test_block_dirty(folio, i))
+					punch(inode, i << blkbits,
+						     1 << blkbits);
Quoted 8 lines

fs/iomap/buffered-io.c static int iomap_write_delalloc_scan(struct inode *inode, · line 1080

Quoted 2 lines
fs/iomap/buffered-io.c +2 -0
+			}
+skip_iop_punch:

Looks reasonable at first glance, though the deep indentation here kind of makes my eyes cross. Could we stuff this loop into a iomap_write_delalloc_scan_folio() helper or some such, and then maybe update the comment at the top of the whole branch to explain both punches?

I felt that too. Sure will give it a try in the next spin.

Quoted 6 lines

WRT to the !iop case.. I _think_ this is handled correctly here because that means we'd handle the folio as completely dirty at writeback time. Is that the case? If so, it might be nice to document that assumption somewhere (here or perhaps in the writeback path).

!iop case is simply when we don't have a large folio and blocksize ==
pagesize. In that case we don't allocate any iop and simply returns from iop_alloc().
So then we just skip the loop which is only meant when we have blocks within a folio.

-ritesh

Quoted 72 lines

Brian

Quoted 70 lines
fs/iomap/buffered-io.c +3 -2
 			/*
 			 * Make sure the next punch start is correctly bound to
 			 * the end of this data range, not the end of the folio.
@@ -1666,7 +1766,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0);
+	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
@@ -1682,7 +1782,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	 * invalid, grab a new one.
 	 */
 	for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) {
-		if (iop && !iop_test_block_uptodate(folio, i))
+		if (iop && !iop_test_block_dirty(folio, i))
 			continue;
 
 		error = wpc->ops->map_blocks(wpc, inode, pos);
@@ -1726,6 +1826,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		}
 	}
 
+	iop_clear_range_dirty(folio, 0, end_pos - folio_pos(folio));
 	folio_start_writeback(folio);
 	folio_unlock(folio);
 
fs/xfs/xfs_aops.c +1 -1
index 2ef78aa1d3f6..77c7332ae197 100644
@@ -578,7 +578,7 @@ const struct address_space_operations xfs_address_space_operations = {
 	.read_folio		= xfs_vm_read_folio,
 	.readahead		= xfs_vm_readahead,
 	.writepages		= xfs_vm_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.bmap			= xfs_vm_bmap,
fs/zonefs/file.c +1 -1
index 132f01d3461f..e508c8e97372 100644
@@ -175,7 +175,7 @@ const struct address_space_operations zonefs_file_aops = {
 	.read_folio		= zonefs_read_folio,
 	.readahead		= zonefs_readahead,
 	.writepages		= zonefs_writepages,
-	.dirty_folio		= filemap_dirty_folio,
+	.dirty_folio		= iomap_dirty_folio,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.migrate_folio		= filemap_migrate_folio,
include/linux/iomap.h +1 -1
index 0f8123504e5e..0c2bee80565c 100644
@@ -264,6 +264,7 @@ bool iomap_is_partially_uptodate(struct folio *, size_t from, size_t count);
 struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos);
 bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags);
 void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len);
+bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);
 int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,
 		const struct iomap_ops *ops);
 int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len,
--

2.39.2

#30 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 97 lines · Ritesh Harjani
Quoted 75 lines · Brian Foster
Quoted 47 lines · Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
index 25f20f269214..c7f41b26280a 100644

...

Quoted 9 lines
fs/iomap/buffered-io.c +4 -1
@@ -119,12 +169,20 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
+	/*
+	 * iop->state tracks two sets of state flags when the
+	 * filesystem block size is smaller than the folio size.
+	 * The first state tracks per-block uptodate and the

fs/iomap/buffered-io.c static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio, · line 185

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
+	 * second tracks per-block dirty state.
+	 */
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(2 * nr_blocks)),
 		      gfp);
 	if (iop) {
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
+		if (is_dirty)
+			iop_set_range(iop, nr_blocks, nr_blocks);

I find the is_dirty logic here a bit confusing. AFAICT, this is

Quoted 8 lines

primarily to handle the case where a folio is completely overwritten, so no iop is allocated at write time, and so then writeback needs to allocate the iop as fully dirty to reflect that. I.e., all iop_alloc() callers other than iomap_writepage_map() either pass the result of folio_test_dirty() or explicitly dirty the entire range of the folio anyways. iomap_dirty_folio() essentially does the latter because it needs to dirty the entire folio regardless of whether the iop already exists or not, right?

Yes.

Quoted 8 lines

If so and if I'm following all of that correctly, could this complexity be isolated to iomap_writepage_map() by simply checking for the !iop case first, then call iop_alloc() immediately followed by set_range_dirty() of the entire folio? Then presumably iop_alloc() could always just dirty based on folio state with the writeback path exception case handled explicitly. Hm?

Quoted 9 lines

Hi Brian,

It was discussed here [1] to pass is_dirty flag at the time of iop allocation. We can do what you are essentially suggesting, but it's just extra work i.e. we will again do some calculations of blocks_per_folio, start, end and more importantly take and release iop->state_lock spinlock. Whereas with above approach we could get away with this at the time of iop allocation itself.

Hi Ritesh,

Isn't that extra work already occurring in iomap_dirty_folio()? I was just thinking that maybe moving it to where it's apparently needed (i.e. writeback) might eliminate the need for the param.

I suppose iomap_dirty_folio() would need to call filemap_dirty_folio() first to make sure iop_alloc() sees the dirty state, but maybe that would also allow skipping the iop alloc if the folio was already dirty (i.e. if the folio was previously dirtied by a full buffered overwite for example)?

I've appended a quick diff below (compile tested only) just to explain what I mean. When doing that it also occurred to me that if we really care about the separate call, we could keep the is_dirty param but do the __iop_alloc() wrapper thing where iop_alloc() always passes folio_test_dirty().

BTW, I think you left off your [1] discussion reference..

Quoted 4 lines

Besides, isn't it easier this way? which as you also stated we will dirty all the blocks based on is_dirty flag, which is folio_test_dirty() except at the writeback time.

My thinking was just that I kind of had to read through all of the iop_alloc() callsites to grok the purpose of the parameter, which made it seem unnecessarily confusing. But ultimately it made sense, so I don't insist on changing it or anything if this approach is intentional and/or preferred by others. That's just my .02 and I'll defer to your preference. :)

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
 		folio_attach_private(folio, iop);
 	}
 	return iop;

...

Quoted 1 line
fs/iomap/buffered-io.c +0 -0
@@ -561,6 +621,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)

...

Quoted 7 lines
Quoted 6 lines

WRT to the !iop case.. I _think_ this is handled correctly here because that means we'd handle the folio as completely dirty at writeback time. Is that the case? If so, it might be nice to document that assumption somewhere (here or perhaps in the writeback path).

Quoted 6 lines

!iop case is simply when we don't have a large folio and blocksize ==
pagesize. In that case we don't allocate any iop and simply returns from iop_alloc().
So then we just skip the loop which is only meant when we have blocks within a folio.

Isn't it also the case that iop might be NULL at this point if the fs has sub-folio blocks, but the folio was dirtied by a full overwrite of the folio? Or did I misunderstand patch 4?

Brian

--- 8< ---

fs/iomap/buffered-io.c +15 -16
index 92e1e1061225..89b3053e3f2d 100644
@@ -155,7 +155,7 @@ static void iop_clear_range_dirty(struct folio *folio, size_t off, size_t len)
 }
 
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
-				    unsigned int flags, bool is_dirty)
+				    unsigned int flags)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
 	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
@@ -181,7 +181,7 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
-		if (is_dirty)
+		if (folio_test_dirty(folio))
 			iop_set_range(iop, nr_blocks, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
@@ -326,8 +326,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	if (WARN_ON_ONCE(size > iomap->length))
 		return -EIO;
 	if (offset > 0)
-		iop = iop_alloc(iter->inode, folio, iter->flags,
-				folio_test_dirty(folio));
+		iop = iop_alloc(iter->inode, folio, iter->flags);
 	else
 		iop = to_iomap_page(folio);
 
@@ -365,8 +364,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 		return iomap_read_inline_data(iter, folio);
 
 	/* zero post-eof blocks as the page may be mapped */
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 	iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen);
 	if (plen == 0)
 		goto done;
@@ -616,13 +614,10 @@ EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
 {
-	struct iomap_page *iop;
-	struct inode *inode = mapping->host;
-	size_t len = i_blocks_per_folio(inode, folio) << inode->i_blkbits;
-
-	iop = iop_alloc(inode, folio, 0, false);
-	iop_set_range_dirty(inode, folio, 0, len);
-	return filemap_dirty_folio(mapping, folio);
+	bool dirtied = filemap_dirty_folio(mapping, folio);
+	if (dirtied)
+		iop_alloc(mapping->host, folio, 0);
+	return dirtied;
 }
 EXPORT_SYMBOL_GPL(iomap_dirty_folio);
 
@@ -673,8 +668,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 	    pos + len >= folio_pos(folio) + folio_size(folio))
 		return 0;
 
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 
 	if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1)
 		return -EAGAIN;
@@ -1759,7 +1753,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
+	struct iomap_page *iop = to_iomap_page(folio);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
@@ -1767,6 +1761,11 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	int error = 0, count = 0, i;
 	LIST_HEAD(submit_list);
 
+	if (!iop) {
+		iop = iop_alloc(inode, folio, 0);
+		iop_set_range_dirty(inode, folio, 0, folio_size(folio));
+	}
+
 	WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0);
 
 	/*
#31 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 120 lines · Brian Foster
Quoted 97 lines · Ritesh Harjani
Quoted 75 lines · Brian Foster
Quoted 47 lines · Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
index 25f20f269214..c7f41b26280a 100644

...

Quoted 9 lines
fs/iomap/buffered-io.c +4 -1
@@ -119,12 +169,20 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
+	/*
+	 * iop->state tracks two sets of state flags when the
+	 * filesystem block size is smaller than the folio size.
+	 * The first state tracks per-block uptodate and the

fs/iomap/buffered-io.c static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio, · line 185

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
+	 * second tracks per-block dirty state.
+	 */
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(2 * nr_blocks)),
 		      gfp);
 	if (iop) {
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
+		if (is_dirty)
+			iop_set_range(iop, nr_blocks, nr_blocks);

I find the is_dirty logic here a bit confusing. AFAICT, this is

Quoted 8 lines

primarily to handle the case where a folio is completely overwritten, so no iop is allocated at write time, and so then writeback needs to allocate the iop as fully dirty to reflect that. I.e., all iop_alloc() callers other than iomap_writepage_map() either pass the result of folio_test_dirty() or explicitly dirty the entire range of the folio anyways. iomap_dirty_folio() essentially does the latter because it needs to dirty the entire folio regardless of whether the iop already exists or not, right?

Yes.

Quoted 8 lines

If so and if I'm following all of that correctly, could this complexity be isolated to iomap_writepage_map() by simply checking for the !iop case first, then call iop_alloc() immediately followed by set_range_dirty() of the entire folio? Then presumably iop_alloc() could always just dirty based on folio state with the writeback path exception case handled explicitly. Hm?

Quoted 9 lines

Hi Brian,

It was discussed here [1] to pass is_dirty flag at the time of iop allocation. We can do what you are essentially suggesting, but it's just extra work i.e. we will again do some calculations of blocks_per_folio, start, end and more importantly take and release iop->state_lock spinlock. Whereas with above approach we could get away with this at the time of iop allocation itself.

Hi Ritesh,

Isn't that extra work already occurring in iomap_dirty_folio()? I was just thinking that maybe moving it to where it's apparently needed (i.e. writeback) might eliminate the need for the param.

I suppose iomap_dirty_folio() would need to call filemap_dirty_folio() first to make sure iop_alloc() sees the dirty state, but maybe that would also allow skipping the iop alloc if the folio was already dirty (i.e. if the folio was previously dirtied by a full buffered overwite for example)?

Quoted 5 lines

I've appended a quick diff below (compile tested only) just to explain what I mean. When doing that it also occurred to me that if we really care about the separate call, we could keep the is_dirty param but do the __iop_alloc() wrapper thing where iop_alloc() always passes folio_test_dirty().

Sure. Brian. I guess when we got the comment, it was still in the intial work & I was anyway passing a from_writeback flag. Thanks for the diff, it does make sense to me. I will try to add that change in the next revision.

Quoted 3 lines

BTW, I think you left off your [1] discussion reference..

Sorry, my bad.

[1]: https://lore.kernel.org/linux-xfs/[email protected]/

Quoted 5 lines
Quoted 4 lines

Besides, isn't it easier this way? which as you also stated we will dirty all the blocks based on is_dirty flag, which is folio_test_dirty() except at the writeback time.

Quoted 7 lines

My thinking was just that I kind of had to read through all of the iop_alloc() callsites to grok the purpose of the parameter, which made it seem unnecessarily confusing. But ultimately it made sense, so I don't insist on changing it or anything if this approach is intentional and/or preferred by others. That's just my .02 and I'll defer to your preference. :)

Sure Thanks!

Quoted 21 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
 		folio_attach_private(folio, iop);
 	}
 	return iop;

...

Quoted 1 line
fs/iomap/buffered-io.c +0 -0
@@ -561,6 +621,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)

...

Quoted 7 lines
Quoted 6 lines

WRT to the !iop case.. I _think_ this is handled correctly here because that means we'd handle the folio as completely dirty at writeback time. Is that the case? If so, it might be nice to document that assumption somewhere (here or perhaps in the writeback path).

Quoted 6 lines

!iop case is simply when we don't have a large folio and blocksize ==
pagesize. In that case we don't allocate any iop and simply returns from iop_alloc().
So then we just skip the loop which is only meant when we have blocks within a folio.

Quoted 4 lines

Isn't it also the case that iop might be NULL at this point if the fs has sub-folio blocks, but the folio was dirtied by a full overwrite of the folio? Or did I misunderstand patch 4?

Yes. Both of the cases. We can either have bs == ps or we can have a complete overwritten folio in which case we don't allocate any iop in iomap_writepage_map() function.

Sure. I will document that when we factor out that change in a seperate function.

Quoted 85 lines

Brian

--- 8< ---

fs/iomap/buffered-io.c +10 -16
index 92e1e1061225..89b3053e3f2d 100644
@@ -155,7 +155,7 @@ static void iop_clear_range_dirty(struct folio *folio, size_t off, size_t len)
 }
 
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
-				    unsigned int flags, bool is_dirty)
+				    unsigned int flags)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
 	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
@@ -181,7 +181,7 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
-		if (is_dirty)
+		if (folio_test_dirty(folio))
 			iop_set_range(iop, nr_blocks, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
@@ -326,8 +326,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	if (WARN_ON_ONCE(size > iomap->length))
 		return -EIO;
 	if (offset > 0)
-		iop = iop_alloc(iter->inode, folio, iter->flags,
-				folio_test_dirty(folio));
+		iop = iop_alloc(iter->inode, folio, iter->flags);
 	else
 		iop = to_iomap_page(folio);
 
@@ -365,8 +364,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 		return iomap_read_inline_data(iter, folio);
 
 	/* zero post-eof blocks as the page may be mapped */
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 	iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen);
 	if (plen == 0)
 		goto done;
@@ -616,13 +614,10 @@ EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
 {
-	struct iomap_page *iop;
-	struct inode *inode = mapping->host;
-	size_t len = i_blocks_per_folio(inode, folio) << inode->i_blkbits;
-
-	iop = iop_alloc(inode, folio, 0, false);
-	iop_set_range_dirty(inode, folio, 0, len);
-	return filemap_dirty_folio(mapping, folio);
+	bool dirtied = filemap_dirty_folio(mapping, folio);
+	if (dirtied)
+		iop_alloc(mapping->host, folio, 0);
+	return dirtied;
 }
 EXPORT_SYMBOL_GPL(iomap_dirty_folio);
 
@@ -673,8 +668,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 	    pos + len >= folio_pos(folio) + folio_size(folio))
 		return 0;
 
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 
 	if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1)
 		return -EAGAIN;
@@ -1759,7 +1753,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
+	struct iomap_page *iop = to_iomap_page(folio);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
@@ -1767,6 +1761,11 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	int error = 0, count = 0, i;

fs/iomap/buffered-io.c iomap_writepage_map(struct iomap_writepage_ctx *wpc, · line 1771

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
 	LIST_HEAD(submit_list);
 
+	if (!iop) {
+		iop = iop_alloc(inode, folio, 0);
+		iop_set_range_dirty(inode, folio, 0, folio_size(folio));
+	}
+
 	WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0);
 
 	/*

Thanks for the review!

-ritesh

#32 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 128 lines · Ritesh Harjani
Quoted 120 lines · Brian Foster
Quoted 97 lines · Ritesh Harjani
Quoted 75 lines · Brian Foster
Quoted 47 lines · Ritesh Harjani (IBM)

When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_page. We currently only track uptodate state.

This patch implements support for tracking per-block dirty state in iomap_page->state bitmap. This should help improve the filesystem write performance and help reduce write amplification.

Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps.

1. <test_randwrite.fio>
[global]

	ioengine=psync
	rw=randwrite
	overwrite=1
	pre_read=1
	direct=0
	bs=4k
	size=1G
	dir=./
	numjobs=8
	fdatasync=1
	runtime=60
	iodepth=64
	group_reporting=1

[fio-run]

2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power)

---
 fs/gfs2/aops.c         |   2 +-
 fs/iomap/buffered-io.c | 115 ++++++++++++++++++++++++++++++++++++++---
 fs/xfs/xfs_aops.c      |   2 +-
 fs/zonefs/file.c       |   2 +-
 include/linux/iomap.h  |   1 +
 5 files changed, 112 insertions(+), 10 deletions(-)

...

Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
index 25f20f269214..c7f41b26280a 100644

...

Quoted 9 lines
fs/iomap/buffered-io.c +4 -1
@@ -119,12 +169,20 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 	else
 		gfp = GFP_NOFS | __GFP_NOFAIL;
 
-	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(nr_blocks)),
+	/*
+	 * iop->state tracks two sets of state flags when the
+	 * filesystem block size is smaller than the folio size.
+	 * The first state tracks per-block uptodate and the

fs/iomap/buffered-io.c static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio, · line 185

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
+	 * second tracks per-block dirty state.
+	 */
+	iop = kzalloc(struct_size(iop, state, BITS_TO_LONGS(2 * nr_blocks)),
 		      gfp);
 	if (iop) {
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
+		if (is_dirty)
+			iop_set_range(iop, nr_blocks, nr_blocks);

I find the is_dirty logic here a bit confusing. AFAICT, this is

Quoted 8 lines

primarily to handle the case where a folio is completely overwritten, so no iop is allocated at write time, and so then writeback needs to allocate the iop as fully dirty to reflect that. I.e., all iop_alloc() callers other than iomap_writepage_map() either pass the result of folio_test_dirty() or explicitly dirty the entire range of the folio anyways. iomap_dirty_folio() essentially does the latter because it needs to dirty the entire folio regardless of whether the iop already exists or not, right?

Yes.

Quoted 8 lines

If so and if I'm following all of that correctly, could this complexity be isolated to iomap_writepage_map() by simply checking for the !iop case first, then call iop_alloc() immediately followed by set_range_dirty() of the entire folio? Then presumably iop_alloc() could always just dirty based on folio state with the writeback path exception case handled explicitly. Hm?

Quoted 9 lines

Hi Brian,

It was discussed here [1] to pass is_dirty flag at the time of iop allocation. We can do what you are essentially suggesting, but it's just extra work i.e. we will again do some calculations of blocks_per_folio, start, end and more importantly take and release iop->state_lock spinlock. Whereas with above approach we could get away with this at the time of iop allocation itself.

Hi Ritesh,

Isn't that extra work already occurring in iomap_dirty_folio()? I was just thinking that maybe moving it to where it's apparently needed (i.e. writeback) might eliminate the need for the param.

I suppose iomap_dirty_folio() would need to call filemap_dirty_folio() first to make sure iop_alloc() sees the dirty state, but maybe that would also allow skipping the iop alloc if the folio was already dirty (i.e. if the folio was previously dirtied by a full buffered overwite for example)?

Quoted 5 lines

I've appended a quick diff below (compile tested only) just to explain what I mean. When doing that it also occurred to me that if we really care about the separate call, we could keep the is_dirty param but do the __iop_alloc() wrapper thing where iop_alloc() always passes folio_test_dirty().

Quoted 4 lines

Sure. Brian. I guess when we got the comment, it was still in the intial work & I was anyway passing a from_writeback flag. Thanks for the diff, it does make sense to me. I will try to add that change in the next revision.

Ok, though I think what I did to iomap_folio_dirty() might be wrong... If we have a folio/iop that is already partially dirty with sub-folio blocks, and then that folio is mapped and write faulted, we still need to explicitly dirty the rest of the iop during the fault, right? If so, then I guess we'd still need to keep the iop_set_range_dirty() call there at least for that case.

Hmm.. so I suppose I could see doing that a couple different ways. One is just to just keep it as you already have it and just drop the dirty param. I.e.:

bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)

{
        iop_alloc(mapping->host, folio, 0);
        iop_set_range_dirty(mapping->host, folio, 0, folio_size(folio));
        return filemap_dirty_folio(mapping, folio);
}

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop? I.e., I think what I was trying to do in the previous diff was actually something like this:

bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)

{
        iop_set_range_dirty(mapping->host, folio, 0, folio_size(folio));
        return filemap_dirty_folio(mapping, folio);
}

... which would only dirty the full iop if it already exists. Thoughts?

Brian

Quoted 155 lines
Quoted 3 lines

BTW, I think you left off your [1] discussion reference..

Sorry, my bad.

[1]: https://lore.kernel.org/linux-xfs/[email protected]/

Quoted 5 lines
Quoted 4 lines

Besides, isn't it easier this way? which as you also stated we will dirty all the blocks based on is_dirty flag, which is folio_test_dirty() except at the writeback time.

Quoted 7 lines

My thinking was just that I kind of had to read through all of the iop_alloc() callsites to grok the purpose of the parameter, which made it seem unnecessarily confusing. But ultimately it made sense, so I don't insist on changing it or anything if this approach is intentional and/or preferred by others. That's just my .02 and I'll defer to your preference. :)

Sure Thanks!

Quoted 21 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
fs/iomap/buffered-io.c +0 -0
 		folio_attach_private(folio, iop);
 	}
 	return iop;

...

Quoted 1 line
fs/iomap/buffered-io.c +0 -0
@@ -561,6 +621,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)

...

Quoted 7 lines
Quoted 6 lines

WRT to the !iop case.. I _think_ this is handled correctly here because that means we'd handle the folio as completely dirty at writeback time. Is that the case? If so, it might be nice to document that assumption somewhere (here or perhaps in the writeback path).

Quoted 6 lines

!iop case is simply when we don't have a large folio and blocksize ==
pagesize. In that case we don't allocate any iop and simply returns from iop_alloc().
So then we just skip the loop which is only meant when we have blocks within a folio.

Quoted 4 lines

Isn't it also the case that iop might be NULL at this point if the fs has sub-folio blocks, but the folio was dirtied by a full overwrite of the folio? Or did I misunderstand patch 4?

Yes. Both of the cases. We can either have bs == ps or we can have a complete overwritten folio in which case we don't allocate any iop in iomap_writepage_map() function.

Sure. I will document that when we factor out that change in a seperate function.

Quoted 85 lines

Brian

--- 8< ---

fs/iomap/buffered-io.c +10 -16
index 92e1e1061225..89b3053e3f2d 100644
@@ -155,7 +155,7 @@ static void iop_clear_range_dirty(struct folio *folio, size_t off, size_t len)
 }
 
 static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
-				    unsigned int flags, bool is_dirty)
+				    unsigned int flags)
 {
 	struct iomap_page *iop = to_iomap_page(folio);
 	unsigned int nr_blocks = i_blocks_per_folio(inode, folio);
@@ -181,7 +181,7 @@ static struct iomap_page *iop_alloc(struct inode *inode, struct folio *folio,
 		spin_lock_init(&iop->state_lock);
 		if (folio_test_uptodate(folio))
 			iop_set_range(iop, 0, nr_blocks);
-		if (is_dirty)
+		if (folio_test_dirty(folio))
 			iop_set_range(iop, nr_blocks, nr_blocks);
 		folio_attach_private(folio, iop);
 	}
@@ -326,8 +326,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter,
 	if (WARN_ON_ONCE(size > iomap->length))
 		return -EIO;
 	if (offset > 0)
-		iop = iop_alloc(iter->inode, folio, iter->flags,
-				folio_test_dirty(folio));
+		iop = iop_alloc(iter->inode, folio, iter->flags);
 	else
 		iop = to_iomap_page(folio);
 
@@ -365,8 +364,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter,
 		return iomap_read_inline_data(iter, folio);
 
 	/* zero post-eof blocks as the page may be mapped */
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 	iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen);
 	if (plen == 0)
 		goto done;
@@ -616,13 +614,10 @@ EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
 {
-	struct iomap_page *iop;
-	struct inode *inode = mapping->host;
-	size_t len = i_blocks_per_folio(inode, folio) << inode->i_blkbits;
-
-	iop = iop_alloc(inode, folio, 0, false);
-	iop_set_range_dirty(inode, folio, 0, len);
-	return filemap_dirty_folio(mapping, folio);
+	bool dirtied = filemap_dirty_folio(mapping, folio);
+	if (dirtied)
+		iop_alloc(mapping->host, folio, 0);
+	return dirtied;
 }
 EXPORT_SYMBOL_GPL(iomap_dirty_folio);
 
@@ -673,8 +668,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos,
 	    pos + len >= folio_pos(folio) + folio_size(folio))
 		return 0;
 
-	iop = iop_alloc(iter->inode, folio, iter->flags,
-			folio_test_dirty(folio));
+	iop = iop_alloc(iter->inode, folio, iter->flags);
 
 	if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1)
 		return -EAGAIN;
@@ -1759,7 +1753,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 		struct writeback_control *wbc, struct inode *inode,
 		struct folio *folio, u64 end_pos)
 {
-	struct iomap_page *iop = iop_alloc(inode, folio, 0, true);
+	struct iomap_page *iop = to_iomap_page(folio);
 	struct iomap_ioend *ioend, *next;
 	unsigned len = i_blocksize(inode);
 	unsigned nblocks = i_blocks_per_folio(inode, folio);
@@ -1767,6 +1761,11 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc,
 	int error = 0, count = 0, i;

fs/iomap/buffered-io.c iomap_writepage_map(struct iomap_writepage_ctx *wpc, · line 1771

Quoted 10 lines
fs/iomap/buffered-io.c +5 -0
 	LIST_HEAD(submit_list);
 
+	if (!iop) {
+		iop = iop_alloc(inode, folio, 0);
+		iop_set_range_dirty(inode, folio, 0, folio_size(folio));
+	}
+
 	WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0);
 
 	/*

Thanks for the review!

-ritesh

#33 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Quoted 10 lines

I.e., I think what I was trying to do in the
previous diff was actually something like this:

bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)

{
        iop_set_range_dirty(mapping->host, folio, 0, folio_size(folio));
        return filemap_dirty_folio(mapping, folio);
}

... which would only dirty the full iop if it already exists. Thoughts?

That does sound pretty good to me, and I can't think of any obvious pitfall.

#34 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

#35 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#36 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 43 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 3 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Think about it at a higher level than the implementation ("do we allocate an iop or not"). If userspace dirties one page in a folio, should it dirty all pages in the folio, or just the page that was actually dirtied? I appreciate you're thinking about this from the point of view of 64kB pages on PPC and using single-page folios, but pretend we've allocated a 1MB folio, mmaped it (or a part of it?) and now userspace stores to it. How much of it do we want to write back?

My argument is that this is RARE. Userspace generally does not mmap(MAP_SHARED), store to it and call msync() to do writes. Writes are almost always done using the write() syscall. Userspace gets a lot more control about when the writeback happens, and they actually get errors back from the write() syscall.

If we attempt to track which pages have actually been dirtied, I worry the fs and the mm will lose track of what the other needs to know. eg the mm will make every page in the folio writable and then not notify the fs when subsequent pages in the folio are stored to.

#37 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts? To optimize the case where pagecache is dirtied but truncated or something and thus never written back? Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred? Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#38 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#39 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#40 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 92 lines · Brian Foster
Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

Quoted 5 lines

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Here is the 1st discussion/reasoning where the deferred iop allocation in the readpage path got discussed [1].
And here is the discussion when I first pointed out the deferred allocation in writepage path. IMO, it got slipped in with the discussions maybe only on mailing list but nothing in the commit messages or comments.[2]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/
[2]: https://lore.kernel.org/linux-xfs/20230130202150.pfohy5yg6dtu64ce@rh-tp/

Quoted 26 lines
Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Quoted 9 lines

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Based on the discussion so far, I would like to think of this as follow:
We already have some sort of lazy iop allocation in the buffered I/O path (discussed above). This patch series does not changes that behavior. For now I would like to keep the page mkwrite page as is without any lazy iop allocation optimization.
I am ok to pick this optimization work as a seperate series because, IIUC, Christoph has some ideas on deferring iop allocations even further [2] (from link shared above).

Does that sound good?

Quoted 5 lines

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Agree that when we work on such optimizations as a seperate series, it will be worth measuring the performance benefits of that.

-ritesh

Quoted 16 lines

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#41 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 155 lines · Ritesh Harjani
Quoted 92 lines · Brian Foster
Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

Quoted 5 lines

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Here is the 1st discussion/reasoning where the deferred iop allocation in the readpage path got discussed [1].
And here is the discussion when I first pointed out the deferred allocation in writepage path. IMO, it got slipped in with the discussions maybe only on mailing list but nothing in the commit messages or comments.[2]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/
[2]: https://lore.kernel.org/linux-xfs/20230130202150.pfohy5yg6dtu64ce@rh-tp/

Quoted 26 lines
Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Quoted 9 lines

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Based on the discussion so far, I would like to think of this as follow:
We already have some sort of lazy iop allocation in the buffered I/O path (discussed above). This patch series does not changes that behavior. For now I would like to keep the page mkwrite page as is without any lazy iop allocation optimization.
I am ok to pick this optimization work as a seperate series because, IIUC, Christoph has some ideas on deferring iop allocations even further [2] (from link shared above).

Quoted 2 lines

Does that sound good?

Could you do that in two steps where the buffered I/O path variant is replaced by explicit dirty tracking in the initial patch, and then is restored by a subsequent patch in the same series? That would allow keeping it around and documenting it explicitly in the commit log for the separate patch, but IMO makes this a bit easier to review (and potentially debug/bisect if needed down the road).

But I don't insist if that's too troublesome for some reason...

Brian

Quoted 29 lines
Quoted 5 lines

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Agree that when we work on such optimizations as a seperate series, it will be worth measuring the performance benefits of that.

-ritesh

Quoted 16 lines

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#42 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 156 lines · Brian Foster
Quoted 155 lines · Ritesh Harjani
Quoted 92 lines · Brian Foster
Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

Quoted 5 lines

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Here is the 1st discussion/reasoning where the deferred iop allocation in the readpage path got discussed [1].
And here is the discussion when I first pointed out the deferred allocation in writepage path. IMO, it got slipped in with the discussions maybe only on mailing list but nothing in the commit messages or comments.[2]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/
[2]: https://lore.kernel.org/linux-xfs/20230130202150.pfohy5yg6dtu64ce@rh-tp/

Quoted 26 lines
Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Quoted 9 lines

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Based on the discussion so far, I would like to think of this as follow:
We already have some sort of lazy iop allocation in the buffered I/O path (discussed above). This patch series does not changes that behavior. For now I would like to keep the page mkwrite page as is without any lazy iop allocation optimization.
I am ok to pick this optimization work as a seperate series because, IIUC, Christoph has some ideas on deferring iop allocations even further [2] (from link shared above).

Quoted 6 lines
Quoted 2 lines

Does that sound good?

Could you do that in two steps where the buffered I/O path variant is replaced by explicit dirty tracking in the initial patch, and then is restored by a subsequent patch in the same series? That would allow

Sorry, I couldn't follow it. Can you please elaborate.

So, what I was suggesting was - for buffered I/O path we should continue to have the lazy iop allocation scheme whereever possible. Rest of the optimizations of further deferring the iop allocation including in the ->mkwrite path should be dealt seperately in a later patch series.

Also we already have a seperate patch in this series which defers the iop allocation if the write completely overwrites the folio [1]. Earlier the behavior was that it skips it entirely if the folio was uptodate, but since we require it for per-block dirty tracking, we defer iop allocation only if it was a complete overwrite of the folio.

[1]: https://lore.kernel.org/linux-xfs/ZGzRX9YVkAYJGLqV@bfoster/T/#m048d0a097f7abb09da1c12c9a02afbcc3b9d39ee

-ritesh

Quoted 37 lines

keeping it around and documenting it explicitly in the commit log for the separate patch, but IMO makes this a bit easier to review (and potentially debug/bisect if needed down the road).

But I don't insist if that's too troublesome for some reason...

Brian

Quoted 29 lines
Quoted 5 lines

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Agree that when we work on such optimizations as a seperate series, it will be worth measuring the performance benefits of that.

-ritesh

Quoted 16 lines

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#43 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 158 lines · Ritesh Harjani
Quoted 156 lines · Brian Foster
Quoted 155 lines · Ritesh Harjani
Quoted 92 lines · Brian Foster
Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

Quoted 5 lines

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Here is the 1st discussion/reasoning where the deferred iop allocation in the readpage path got discussed [1].
And here is the discussion when I first pointed out the deferred allocation in writepage path. IMO, it got slipped in with the discussions maybe only on mailing list but nothing in the commit messages or comments.[2]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/
[2]: https://lore.kernel.org/linux-xfs/20230130202150.pfohy5yg6dtu64ce@rh-tp/

Quoted 26 lines
Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Quoted 9 lines

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Based on the discussion so far, I would like to think of this as follow:
We already have some sort of lazy iop allocation in the buffered I/O path (discussed above). This patch series does not changes that behavior. For now I would like to keep the page mkwrite page as is without any lazy iop allocation optimization.
I am ok to pick this optimization work as a seperate series because, IIUC, Christoph has some ideas on deferring iop allocations even further [2] (from link shared above).

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Does that sound good?

Could you do that in two steps where the buffered I/O path variant is replaced by explicit dirty tracking in the initial patch, and then is restored by a subsequent patch in the same series? That would allow

Sorry, I couldn't follow it. Can you please elaborate.

Sorry for the confusion...

Quoted 6 lines

So, what I was suggesting was - for buffered I/O path we should continue to have the lazy iop allocation scheme whereever possible. Rest of the optimizations of further deferring the iop allocation including in the ->mkwrite path should be dealt seperately in a later patch series.

Yup, agree.

Quoted 6 lines

Also we already have a seperate patch in this series which defers the iop allocation if the write completely overwrites the folio [1]. Earlier the behavior was that it skips it entirely if the folio was uptodate, but since we require it for per-block dirty tracking, we defer iop allocation only if it was a complete overwrite of the folio.

That is a prepatory patch before iop dirty tracking is enabled in patch 5, right? I was mainly just suggesting to make the overwrite checking part of this patch come after dirty tracking is enabled (as a small optimization patch) rather than before.

I don't want to harp on it if that's difficult or still doesn't make sense for some reason. I'll take a closer look the next go around when I have a bit more time and just send a diff if it seems it can be done cleanly..

Brian

Quoted 43 lines

[1]: https://lore.kernel.org/linux-xfs/ZGzRX9YVkAYJGLqV@bfoster/T/#m048d0a097f7abb09da1c12c9a02afbcc3b9d39ee

-ritesh

Quoted 37 lines

keeping it around and documenting it explicitly in the commit log for the separate patch, but IMO makes this a bit easier to review (and potentially debug/bisect if needed down the road).

But I don't insist if that's too troublesome for some reason...

Brian

Quoted 29 lines
Quoted 5 lines

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Agree that when we work on such optimizations as a seperate series, it will be worth measuring the performance benefits of that.

-ritesh

Quoted 16 lines

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#44 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 187 lines · Brian Foster
Quoted 158 lines · Ritesh Harjani
Quoted 156 lines · Brian Foster
Quoted 155 lines · Ritesh Harjani
Quoted 92 lines · Brian Foster
Quoted 77 lines · Darrick J. Wong
Quoted 37 lines · Brian Foster
Quoted 29 lines · Ritesh Harjani
Quoted 15 lines · Matthew Wilcox
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Brian Foster

But I also wonder.. if we can skip the iop alloc on full folio buffered overwrites, isn't that also true of mapped writes to folios that don't already have an iop?

Yes.

Hm, well, maybe? If somebody stores to a page, we obviously set the dirty flag on the folio, but depending on the architecture, we may or may not have independent dirty bits on the PTEs (eg if it's a PMD, we have one dirty bit for the entire folio; similarly if ARM uses the contiguous PTE bit). If we do have independent dirty bits, we could dirty only the blocks corresponding to a single page at a time.

Quoted 7 lines

This has potential for causing some nasty bugs, so I'm inclined to rule that if a folio is mmaped, then it's all dirty from any writable page fault. The fact is that applications generally do not perform writes through mmap because the error handling story is so poor.

There may be a different answer for anonymous memory, but that doesn't feel like my problem and shouldn't feel like any FS developer's problem.

Although I am skeptical too to do the changes which Brian is suggesting here. i.e. not making all the blocks of the folio dirty when we are going to call ->dirty_folio -> filemap_dirty_folio() (mmaped writes).

Quoted 6 lines

However, I am sorry but I coudn't completely follow your reasoning above. I think what Brian is suggesting here is that filemap_dirty_folio() should be similar to complete buffered overwrite case where we do not allocate the iop at the ->write_begin() time. Then at the writeback time we allocate an iop and mark all blocks dirty.

Quoted 7 lines

Yeah... I think what Willy is saying (i.e. to not track sub-page dirty granularity of intra-folio faults) makes sense, but I'm also not sure what it has to do with the idea of being consistent with how full folio overwrites are implemented (between buffered or mapped writes). We're not changing historical dirtying granularity either way. I think this is just a bigger picture thought for future consideration as opposed to direct feedback on this patch..

<nod>

Quoted 9 lines
Quoted 8 lines

In a way it is also the similar case as for mmapped writes too but my only worry is the way mmaped writes work and it makes more sense to keep the dirty state of folio and per-block within iop in sync. For that matter, we can even just make sure we always allocate an iop in the complete overwrites case as well. I didn't change that code because it was kept that way for uptodate state as well and based on one of your inputs for complete overwrite case.

Quoted 6 lines

Can you elaborate on your concerns, out of curiosity?

Either way, IMO it also seems reasonable to drop this behavior for the basic implementation of dirty tracking (so always allocate the iop for sub-folio tracking as you suggest above) and then potentially restore it as a separate optimization patch at the end of the series.

Agree.

Quoted 3 lines

That said, I'm not totally clear why it exists in the first place, so that might warrant some investigation. Is it primarily to defer allocations out of task write/fault contexts?

(Assuming by 'it' you mean the behavior where we don't unconditionally allocate iops for blocksize < foliosize...)

IIRC the reason is to reduce memory usage by eliding iop allocations unless it's absolutely necessary for correctness was /my/ understanding of why we don't always allocate the iop...

Quoted 7 lines
Quoted 2 lines

To optimize the case where pagecache is dirtied but truncated or something and thus never written back?

...because this might very well happen. Write a temporary .o file to the filesystem, then delete the whole thing before writeback ever gets its hands on the file.

I don't think a simple temp write will trigger this scenario currently because the folios would have to be uptodate at the time of the write to bypass the iop alloc. I guess you'd have to read folios (even if backed by holes) first to start seeing the !iop case at writeback time (for bs != ps).

Quoted 5 lines

That could change with these patches, but I was trying to reason about the intent of the existing code and whether there was some known reason to continue to try and defer the iop allocation as the need/complexity for deferring it grows with the addition of more (i.e. dirty) tracking.

Here is the 1st discussion/reasoning where the deferred iop allocation in the readpage path got discussed [1].
And here is the discussion when I first pointed out the deferred allocation in writepage path. IMO, it got slipped in with the discussions maybe only on mailing list but nothing in the commit messages or comments.[2]

[1]: https://lore.kernel.org/linux-xfs/[email protected]/
[2]: https://lore.kernel.org/linux-xfs/20230130202150.pfohy5yg6dtu64ce@rh-tp/

Quoted 26 lines
Quoted 8 lines
Quoted 2 lines

Is there any room for further improvement where the alloc could be avoided completely for folio overwrites instead of just deferred?

Once writeback starts, though, we need the iop so that we can know when all the writeback for that folio is actually complete, no matter how many IOs might be in flight for that folio. I don't know how you'd get around this problem.

Ok. I noticed some kind of counter or something being updated last time I looked through that code, so it sounds like that's the reason the iop eventually needs to exist. Thanks.

Quoted 6 lines
Quoted 3 lines

Was that actually the case at some point and then something later decided the iop was needed at writeback time, leading to current behavior?

It's been in iomap since the beginning when we lifted it from xfs.

Not sure exactly what you're referring to here. iomap_writepage_map() would warn on the (bs != ps && !iop) case up until commit 8e1bcef8e18d ("iomap: Permit pages without an iop to enter writeback"), so I don't see how iop allocs were deferred (other than when bs == ps, obviously) prior to that.

Quoted 9 lines

Heh, but I'm starting to get my wires crossed just trying to piece things together here. Ritesh, ISTM the (writeback && !iop && bs != ps) case is primarily a subtle side effect of the current writeback behavior being driven by uptodate status. I think it's probably wise to drop it at least initially, always alloc and dirty the appropriate iop ranges for sub-folio blocks, and then if you or others think there is value in the overwrite optimization to defer iop allocs, tack that on as a separate patch and try to be consistent between buffered and mapped writes.

Based on the discussion so far, I would like to think of this as follow:
We already have some sort of lazy iop allocation in the buffered I/O path (discussed above). This patch series does not changes that behavior. For now I would like to keep the page mkwrite page as is without any lazy iop allocation optimization.
I am ok to pick this optimization work as a seperate series because, IIUC, Christoph has some ideas on deferring iop allocations even further [2] (from link shared above).

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Does that sound good?

Could you do that in two steps where the buffered I/O path variant is replaced by explicit dirty tracking in the initial patch, and then is restored by a subsequent patch in the same series? That would allow

Sorry, I couldn't follow it. Can you please elaborate.

Sorry for the confusion...

Quoted 6 lines

So, what I was suggesting was - for buffered I/O path we should continue to have the lazy iop allocation scheme whereever possible. Rest of the optimizations of further deferring the iop allocation including in the ->mkwrite path should be dealt seperately in a later patch series.

Yup, agree.

Quoted 6 lines

Also we already have a seperate patch in this series which defers the iop allocation if the write completely overwrites the folio [1]. Earlier the behavior was that it skips it entirely if the folio was uptodate, but since we require it for per-block dirty tracking, we defer iop allocation only if it was a complete overwrite of the folio.

Quoted 2 lines

That is a prepatory patch before iop dirty tracking is enabled in patch 5, right?

Yes, right.

Quoted 8 lines

I was mainly just suggesting to make the overwrite checking part of this patch come after dirty tracking is enabled (as a small optimization patch) rather than before.

I don't want to harp on it if that's difficult or still doesn't make sense for some reason. I'll take a closer look the next go around when I have a bit more time and just send a diff if it seems it can be done cleanly..

It should not be difficult. Will make the change in next rev.

Thanks for the help. Seems a lot of things have been sorted out now :)

I will be working on the review comments, finish off few of the remaining todos, get more testing done and will spin off the next revision.

-ritesh

Quoted 46 lines

Brian

Quoted 43 lines

[1]: https://lore.kernel.org/linux-xfs/ZGzRX9YVkAYJGLqV@bfoster/T/#m048d0a097f7abb09da1c12c9a02afbcc3b9d39ee

-ritesh

Quoted 37 lines

keeping it around and documenting it explicitly in the commit log for the separate patch, but IMO makes this a bit easier to review (and potentially debug/bisect if needed down the road).

But I don't insist if that's too troublesome for some reason...

Brian

Quoted 29 lines
Quoted 5 lines

Darrick noted above that he also agrees with that separate patch approach. For me, I think it would also be useful to show that there is some measurable performance benefit on at least one reasonable workload to help justify it.

Agree that when we work on such optimizations as a seperate series, it will be worth measuring the performance benefits of that.

-ritesh

Quoted 16 lines

Brian

Quoted 13 lines

--D (who is now weeks behind on reviewing things and stressed out)

Quoted 10 lines

Brian

Quoted 7 lines

Though I agree that we should ideally be allocatting & marking all blocks in iop as dirty in the call to ->dirty_folio(), I just wanted to understand your reasoning better.

Thanks!
-ritesh

#45 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Brian Foster
Quoted 9 lines · Brian Foster

That is a prepatory patch before iop dirty tracking is enabled in patch 5, right? I was mainly just suggesting to make the overwrite checking part of this patch come after dirty tracking is enabled (as a small optimization patch) rather than before.

I don't want to harp on it if that's difficult or still doesn't make sense for some reason. I'll take a closer look the next go around when I have a bit more time and just send a diff if it seems it can be done cleanly..

Honestly, I think the way the patchset is structured now is fine. There are always multiple ways to structure a set of patches, and I don't think we need be too concerned about what order the things are done in, as long as they get done.

#46 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

In reply to Ritesh Harjani (IBM)

fs/iomap/buffered-io.c static inline void iop_set_range(struct iomap_page *iop, unsigned int start_blk, · line 59

Quoted 5 lines · Ritesh Harjani (IBM)
fs/iomap/buffered-io.c +5 -0
+static inline void iop_clear_range(struct iomap_page *iop,
+				   unsigned int start_blk, unsigned int nr_blks)
+{
+	bitmap_clear(iop->state, start_blk, nr_blks);
+}

Similar to the other trivial bitmap wrappers added earlier in the series I don't think this one is very useful.

fs/iomap/buffered-io.c static void iop_set_range_uptodate(struct inode *inode, struct folio *folio, · line 137

Quoted 12 lines
fs/iomap/buffered-io.c +12 -0
+	struct iomap_page *iop = to_iomap_page(folio);
+	unsigned int blks_per_folio = i_blocks_per_folio(inode, folio);
+	unsigned int first_blk = (off >> inode->i_blkbits);
+	unsigned int last_blk = ((off + len - 1) >> inode->i_blkbits);
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (!iop)
+		return;
+	spin_lock_irqsave(&iop->state_lock, flags);
+	iop_set_range(iop, first_blk + blks_per_folio, nr_blks);
+	spin_unlock_irqrestore(&iop->state_lock, flags);

All the variable initializations except for ios should really move into a branch here. Or we just use separate helpers for the case where we actually have an iops and isolate all that, which would be my preference (but goes counter to the direction of changes earlier in the series to the existing functions).

Quoted 6 lines
fs/iomap/buffered-io.c +6 -0
+static void iop_clear_range_dirty(struct folio *folio, size_t off, size_t len)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	struct inode *inode = folio->mapping->host;
+	unsigned int blks_per_folio = i_blocks_per_folio(inode, folio);
+	unsigned int first_blk = (off >> inode->i_blkbits);

fs/iomap/buffered-io.c static void iop_set_range_uptodate(struct inode *inode, struct folio *folio, · line 155

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+	unsigned int last_blk = ((off + len - 1) >> inode->i_blkbits);
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (!iop)
+		return;
+	spin_lock_irqsave(&iop->state_lock, flags);
+	iop_clear_range(iop, first_blk + blks_per_folio, nr_blks);
+	spin_unlock_irqrestore(&iop->state_lock, flags);
+}

Same here.

#47 lore raw

Re: [RFCv5 5/5] iomap: Add per-block dirty state tracking to improve performance

Quoted 10 lines · Christoph Hellwig

fs/iomap/buffered-io.c static inline void iop_set_range(struct iomap_page *iop, unsigned int start_blk, · line 59

Quoted 5 lines · Ritesh Harjani (IBM)
fs/iomap/buffered-io.c +5 -0
+static inline void iop_clear_range(struct iomap_page *iop,
+				   unsigned int start_blk, unsigned int nr_blks)
+{
+	bitmap_clear(iop->state, start_blk, nr_blks);
+}

Similar to the other trivial bitmap wrappers added earlier in the series I don't think this one is very useful.

Quoted 15 lines

fs/iomap/buffered-io.c static void iop_set_range_uptodate(struct inode *inode, struct folio *folio, · line 137

Quoted 12 lines
fs/iomap/buffered-io.c +12 -0
+	struct iomap_page *iop = to_iomap_page(folio);
+	unsigned int blks_per_folio = i_blocks_per_folio(inode, folio);
+	unsigned int first_blk = (off >> inode->i_blkbits);
+	unsigned int last_blk = ((off + len - 1) >> inode->i_blkbits);
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
+	if (!iop)
+		return;
+	spin_lock_irqsave(&iop->state_lock, flags);
+	iop_set_range(iop, first_blk + blks_per_folio, nr_blks);
+	spin_unlock_irqrestore(&iop->state_lock, flags);

All the variable initializations except for ios should really move into a branch here.

Branch won't be needed I guess, will just move the initialization after the return.

Quoted 15 lines

Or we just use separate helpers for the case
where we actually have an iops and isolate all that, which would be my preference (but goes counter to the direction of changes earlier in the series to the existing functions).

Quoted 10 lines
fs/iomap/buffered-io.c +10 -0
+static void iop_clear_range_dirty(struct folio *folio, size_t off, size_t len)
+{
+	struct iomap_page *iop = to_iomap_page(folio);
+	struct inode *inode = folio->mapping->host;
+	unsigned int blks_per_folio = i_blocks_per_folio(inode, folio);
+	unsigned int first_blk = (off >> inode->i_blkbits);
+	unsigned int last_blk = ((off + len - 1) >> inode->i_blkbits);
+	unsigned int nr_blks = last_blk - first_blk + 1;
+	unsigned long flags;
+
Quoted 8 lines

fs/iomap/buffered-io.c static void iop_set_range_uptodate(struct inode *inode, struct folio *folio, · line 155

Quoted 6 lines
fs/iomap/buffered-io.c +6 -0
+	if (!iop)
+		return;
+	spin_lock_irqsave(&iop->state_lock, flags);
+	iop_clear_range(iop, first_blk + blks_per_folio, nr_blks);
+	spin_unlock_irqrestore(&iop->state_lock, flags);
+}

Same here.

Sure.

-ritesh