Skip to contentSkip to conversation

btrfs: reduce extent map lookup overhead for data write

PATCH In review Qu Wenruo 4 participants 9 messages Aug 8, 2024 – Aug 29, 2024 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs: reduce extent map lookup overhead for data write

Unlike data read path, which use cached extent map to reduce overhead, data write path always do the extent map lookup no matter what.

So this patchset will improve the situation by:

- Move em_cached into bio_ctrl
Since the lifespan of them is the same, it's a perfect match.

- Make data write path to use bio_ctrl::em_cached

Unfortunately since my last relocation, I no longer have any dedicated storage attached to my VMs (my laptop only has one NVME slot, and my main workhorse aarch64 board only has one NVME attached either).

So no benchmark yet, and any extra benchmark would be very appreciated.

Qu Wenruo (2):
btrfs: introduce extent_map::em_cached member
btrfs: utilize cached extent map for data writeback

 fs/btrfs/extent_io.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)
Signature · 1 line
2.45.2
author
#1 lore raw

PATCH 1/2 btrfs: introduce extent_map::em_cached member

For data read, we always pass an extent_map pointer as @em_cached for btrfs_readpage().
And that @em_cached pointer has the same lifespan as the @bio_ctrl we passed in, so it's a perfect match to move @em_cached into @bio_ctrl.

---
 fs/btrfs/extent_io.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
fs/btrfs/extent_io.c +9 -10
index 822e2bf8bc99..d4ad98488c03 100644
@@ -101,6 +101,8 @@ struct btrfs_bio_ctrl {
 	blk_opf_t opf;
 	btrfs_bio_end_io_t end_io_func;
 	struct writeback_control *wbc;
+	/* For read/write extent map cache. */
+	struct extent_map *em_cached;
 };
 
 static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl)
@@ -1003,8 +1005,8 @@ static struct extent_map *__get_extent_map(struct inode *inode,
  * XXX JDM: This needs looking at to ensure proper page locking
  * return 0 on success, otherwise return error
  */
-static int btrfs_do_readpage(struct folio *folio, struct extent_map **em_cached,
-		      struct btrfs_bio_ctrl *bio_ctrl, u64 *prev_em_start)
+static int btrfs_do_readpage(struct folio *folio, struct btrfs_bio_ctrl *bio_ctrl,
+			     u64 *prev_em_start)
 {
 	struct inode *inode = folio->mapping->host;
 	struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
@@ -1052,7 +1054,7 @@ static int btrfs_do_readpage(struct folio *folio, struct extent_map **em_cached,
 			break;
 		}
 		em = __get_extent_map(inode, folio, cur, end - cur + 1,
-				      em_cached);
+				      &bio_ctrl->em_cached);
 		if (IS_ERR(em)) {
 			unlock_extent(tree, cur, end, NULL);
 			end_folio_read(folio, false, cur, end + 1 - cur);
@@ -1160,13 +1162,12 @@ int btrfs_read_folio(struct file *file, struct folio *folio)
 	u64 start = folio_pos(folio);
 	u64 end = start + folio_size(folio) - 1;
 	struct btrfs_bio_ctrl bio_ctrl = { .opf = REQ_OP_READ };
-	struct extent_map *em_cached = NULL;
 	int ret;
 
 	btrfs_lock_and_flush_ordered_range(inode, start, end, NULL);
 
-	ret = btrfs_do_readpage(folio, &em_cached, &bio_ctrl, NULL);
-	free_extent_map(em_cached);
+	ret = btrfs_do_readpage(folio, &bio_ctrl, NULL);
+	free_extent_map(bio_ctrl.em_cached);
 
 	/*
 	 * If btrfs_do_readpage() failed we will want to submit the assembled
@@ -2349,16 +2350,14 @@ void btrfs_readahead(struct readahead_control *rac)
 	struct folio *folio;
 	u64 start = readahead_pos(rac);
 	u64 end = start + readahead_length(rac) - 1;
-	struct extent_map *em_cached = NULL;
 	u64 prev_em_start = (u64)-1;
 
 	btrfs_lock_and_flush_ordered_range(inode, start, end, NULL);
 
 	while ((folio = readahead_folio(rac)) != NULL)
-		btrfs_do_readpage(folio, &em_cached, &bio_ctrl, &prev_em_start);
+		btrfs_do_readpage(folio, &bio_ctrl, &prev_em_start);
 
-	if (em_cached)
-		free_extent_map(em_cached);
+	free_extent_map(bio_ctrl.em_cached);
 	submit_one_bio(&bio_ctrl);
 }
 
Signature · 1 line
2.45.2
author
#4 lore raw

PATCH 2/2 btrfs: utilize cached extent map for data writeback

In reply to Qu Wenruo

Unlike data read, we never really utilize the cached extent_map for data write.

This means we will do the costly extent map lookup for each sector we need to write back.

Change it to utilize the same function, __get_extent_map(), just like the data read path, to reduce the overhead of extent map lookup.

---
 fs/btrfs/extent_io.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/btrfs/extent_io.c +4 -1
index d4ad98488c03..9492cd9d4f04 100644
@@ -1357,7 +1357,8 @@ static int submit_one_sector(struct btrfs_inode *inode,
 	/* @filepos >= i_size case should be handled by the caller. */
 	ASSERT(filepos < i_size);
 
-	em = btrfs_get_extent(inode, NULL, filepos, sectorsize);
+	em = __get_extent_map(&inode->vfs_inode, NULL, filepos, sectorsize,
+			      &bio_ctrl->em_cached);
 	if (IS_ERR(em))
 		return PTR_ERR_OR_ZERO(em);
 
@@ -2320,6 +2321,7 @@ void extent_write_locked_range(struct inode *inode, const struct folio *locked_f
 		cur = cur_end + 1;
 	}
 
+	free_extent_map(bio_ctrl.em_cached);
 	submit_write_bio(&bio_ctrl, found_error ? ret : 0);
 }
 
@@ -2338,6 +2340,7 @@ int btrfs_writepages(struct address_space *mapping, struct writeback_control *wb
 	 */
 	btrfs_zoned_data_reloc_lock(BTRFS_I(inode));
 	ret = extent_write_cache_pages(mapping, &bio_ctrl);
+	free_extent_map(bio_ctrl.em_cached);
 	submit_write_bio(&bio_ctrl, ret);
 	btrfs_zoned_data_reloc_unlock(BTRFS_I(inode));
 	return ret;
Signature · 1 line
2.45.2
#6 lore raw
In reply to Qu Wenruo
Quoted 9 lines · Qu Wenruo

Unlike data read path, which use cached extent map to reduce overhead, data write path always do the extent map lookup no matter what.

So this patchset will improve the situation by:

- Move em_cached into bio_ctrl
Since the lifespan of them is the same, it's a perfect match.

Quoted 7 lines

- Make data write path to use bio_ctrl::em_cached

Unfortunately since my last relocation, I no longer have any dedicated storage attached to my VMs (my laptop only has one NVME slot, and my main workhorse aarch64 board only has one NVME attached either).

So no benchmark yet, and any extra benchmark would be very appreciated.

You don't need to test with dedicated NVME or any dedicated hardware.

Use a VM and use a null block device.
Make the test scenario doing a 1M write into a file that has a few thousand extent maps loaded in the inode's io tree, then measure the total time taken by submit_one_sector() calls with a bpftrace script.

Quoted 3 lines

Qu Wenruo (2):
btrfs: introduce extent_map::em_cached member

The subject doesn't match the change, should be btrfs_bio_ctrl::em_cached. The name is also odd, "cached_em" would be more correct, but given it's inside a context structure, just "em" would be fine.

Quoted 9 lines

btrfs: utilize cached extent map for data writeback

 fs/btrfs/extent_io.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

--
2.45.2

#2 lore raw

Re: [PATCH 1/2] btrfs: introduce extent_map::em_cached member

In reply to Qu Wenruo
Quoted 16 lines · Qu Wenruo

For data read, we always pass an extent_map pointer as @em_cached for btrfs_readpage().
And that @em_cached pointer has the same lifespan as the @bio_ctrl we passed in, so it's a perfect match to move @em_cached into @bio_ctrl.

---
 fs/btrfs/extent_io.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
fs/btrfs/extent_io.c +0 -0
index 822e2bf8bc99..d4ad98488c03 100644

fs/btrfs/extent_io.c struct btrfs_bio_ctrl { · line 105

Quoted 5 lines
fs/btrfs/extent_io.c +2 -0
@@ -101,6 +101,8 @@ struct btrfs_bio_ctrl {
        blk_opf_t opf;
        btrfs_bio_end_io_t end_io_func;
        struct writeback_control *wbc;
+       /* For read/write extent map cache. */
+       struct extent_map *em_cached;

As mentioned before, this can be named just "em", it's clear enough given the comment and the fact the structure is contextual. The naming "em_cached" is odd, and would be more logical as "cached_em" if it were outside the structure.

Quoted 45 lines
fs/btrfs/extent_io.c +5 -7
 };
 
 static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl)
@@ -1003,8 +1005,8 @@ static struct extent_map *__get_extent_map(struct inode *inode,
  * XXX JDM: This needs looking at to ensure proper page locking
  * return 0 on success, otherwise return error
  */
-static int btrfs_do_readpage(struct folio *folio, struct extent_map **em_cached,
-                     struct btrfs_bio_ctrl *bio_ctrl, u64 *prev_em_start)
+static int btrfs_do_readpage(struct folio *folio, struct btrfs_bio_ctrl *bio_ctrl,
+                            u64 *prev_em_start)
 {
        struct inode *inode = folio->mapping->host;
        struct btrfs_fs_info *fs_info = inode_to_fs_info(inode);
@@ -1052,7 +1054,7 @@ static int btrfs_do_readpage(struct folio *folio, struct extent_map **em_cached,
                        break;
                }
                em = __get_extent_map(inode, folio, cur, end - cur + 1,
-                                     em_cached);
+                                     &bio_ctrl->em_cached);
                if (IS_ERR(em)) {
                        unlock_extent(tree, cur, end, NULL);
                        end_folio_read(folio, false, cur, end + 1 - cur);
@@ -1160,13 +1162,12 @@ int btrfs_read_folio(struct file *file, struct folio *folio)
        u64 start = folio_pos(folio);
        u64 end = start + folio_size(folio) - 1;
        struct btrfs_bio_ctrl bio_ctrl = { .opf = REQ_OP_READ };
-       struct extent_map *em_cached = NULL;
        int ret;
 
        btrfs_lock_and_flush_ordered_range(inode, start, end, NULL);
 
-       ret = btrfs_do_readpage(folio, &em_cached, &bio_ctrl, NULL);
-       free_extent_map(em_cached);
+       ret = btrfs_do_readpage(folio, &bio_ctrl, NULL);
+       free_extent_map(bio_ctrl.em_cached);
 
        /*
         * If btrfs_do_readpage() failed we will want to submit the assembled
@@ -2349,16 +2350,14 @@ void btrfs_readahead(struct readahead_control *rac)
        struct folio *folio;
        u64 start = readahead_pos(rac);
        u64 end = start + readahead_length(rac) - 1;
-       struct extent_map *em_cached = NULL;
        u64 prev_em_start = (u64)-1;

fs/btrfs/extent_io.c void btrfs_readahead(struct readahead_control *rac) · line 2360

Quoted 10 lines
fs/btrfs/extent_io.c +2 -3
 
        btrfs_lock_and_flush_ordered_range(inode, start, end, NULL);
 
        while ((folio = readahead_folio(rac)) != NULL)
-               btrfs_do_readpage(folio, &em_cached, &bio_ctrl, &prev_em_start);
+               btrfs_do_readpage(folio, &bio_ctrl, &prev_em_start);
 
-       if (em_cached)
-               free_extent_map(em_cached);
+       free_extent_map(bio_ctrl.em_cached);

So instead of calling free_extent_map() before each submit_one_bio() call, it would be better to do the call inside submit_one_bio() and then set bio_ctrl->em_cached to NULL there,
to avoid any use-after-free surprises and leaks in case someone later forgets to call free_extent_map(). Also removes duplicated code.

Also, as mentioned before, the subject of the patch is incorrect, it should mention btrfs_bio_ctrl:: instead of extent_map::

Thanks.

Quoted 8 lines
fs/btrfs/extent_io.c +0 -1
        submit_one_bio(&bio_ctrl);
 }
 
 
--

2.45.2

#5 lore raw

Re: [PATCH 2/2] btrfs: utilize cached extent map for data writeback

In reply to Qu Wenruo
Quoted 38 lines · Qu Wenruo

Unlike data read, we never really utilize the cached extent_map for data write.

This means we will do the costly extent map lookup for each sector we need to write back.

Change it to utilize the same function, __get_extent_map(), just like the data read path, to reduce the overhead of extent map lookup.

---
 fs/btrfs/extent_io.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/btrfs/extent_io.c +3 -1
index d4ad98488c03..9492cd9d4f04 100644
@@ -1357,7 +1357,8 @@ static int submit_one_sector(struct btrfs_inode *inode,
        /* @filepos >= i_size case should be handled by the caller. */
        ASSERT(filepos < i_size);
 
-       em = btrfs_get_extent(inode, NULL, filepos, sectorsize);
+       em = __get_extent_map(&inode->vfs_inode, NULL, filepos, sectorsize,
+                             &bio_ctrl->em_cached);
        if (IS_ERR(em))
                return PTR_ERR_OR_ZERO(em);
 
@@ -2320,6 +2321,7 @@ void extent_write_locked_range(struct inode *inode, const struct folio *locked_f
                cur = cur_end + 1;
        }
 
+       free_extent_map(bio_ctrl.em_cached);
        submit_write_bio(&bio_ctrl, found_error ? ret : 0);
 }
 

fs/btrfs/extent_io.c int btrfs_writepages(struct address_space *mapping, struct writeback_control *wb · line 2344

Quoted 5 lines
fs/btrfs/extent_io.c +1 -0
@@ -2338,6 +2340,7 @@ int btrfs_writepages(struct address_space *mapping, struct writeback_control *wb
         */
        btrfs_zoned_data_reloc_lock(BTRFS_I(inode));
        ret = extent_write_cache_pages(mapping, &bio_ctrl);
+       free_extent_map(bio_ctrl.em_cached);
        submit_write_bio(&bio_ctrl, ret);

Same comment as in the first patch, instead of duplicating free_extent_map() calls before submit_write_bio() calls, just move the free_extent_map() to inside of submit_write_bio() and
then also set ->em_cached to NULL after the free. This helps avoid use-after-free issues and someone forgetting a free call in the future.

Thanks.

Quoted 6 lines
fs/btrfs/extent_io.c +0 -1
        btrfs_zoned_data_reloc_unlock(BTRFS_I(inode));
        return ret;
--

2.45.2

#3 lore raw

Re: [PATCH 1/2] btrfs: introduce extent_map::em_cached member

In reply to Filipe Manana
Quoted 12 lines · Filipe Manana
Quoted 11 lines · Qu Wenruo

For data read, we always pass an extent_map pointer as @em_cached for btrfs_readpage().
And that @em_cached pointer has the same lifespan as the @bio_ctrl we passed in, so it's a perfect match to move @em_cached into @bio_ctrl.

---
  fs/btrfs/extent_io.c | 19 +++++++++----------
  1 file changed, 9 insertions(+), 10 deletions(-)
Quoted 16 lines

fs/btrfs/extent_io.c struct btrfs_bio_ctrl { · line 105

Quoted 10 lines
fs/btrfs/extent_io.c +2 -0
index 822e2bf8bc99..d4ad98488c03 100644
@@ -101,6 +101,8 @@ struct btrfs_bio_ctrl {
         blk_opf_t opf;
         btrfs_bio_end_io_t end_io_func;
         struct writeback_control *wbc;
+       /* For read/write extent map cache. */
+       struct extent_map *em_cached;

As mentioned before, this can be named just "em", it's clear enough given the comment and the fact the structure is contextual. The naming "em_cached" is odd, and would be more logical as "cached_em" if it were outside the structure.

Definitely will fix the naming and subjective line.

[...]

Quoted 6 lines

So instead of calling free_extent_map() before each submit_one_bio() call, it would be better to do the call inside submit_one_bio() and then set bio_ctrl->em_cached to NULL there,
to avoid any use-after-free surprises and leaks in case someone later forgets to call free_extent_map(). Also removes duplicated code.

This is not that straightforward. E.g.

	|<- em 1 ->|<- em 2 ->|
        | 1  |  2  |  3  |  4 | <- Pages

For page 1, we got em1, and now bio_ctrl->em = em1, add the page into the bio_ctrl->bio.
For page 2, the same, we just reuse bio_ctrl->em, add the page into bio_ctrl->bio.

For page 3, we find bio_ctrl->em no longer matches our range, so we get the new em2 and stored it into bio_ctrl->em, so far so good.

But at submit_extent_folio(), we found page 3 is not contig for bio_ctrl->bio, thus we should submit the bio.

Remember the bio_ctrl->em is now em2, if we clear it right now, the next page, page 4 will still need to do an em lookup.

So unfortunately we can not free the cached em at submit_one_bio(), as at that timing, our cached em can be switched to the next range.

This is not a huge deal, as we're only doing at most 2 em lookups per dirty range, but it's still not the ideal 1 em lookup of the existing code.

And considering there are only 4 call sites allocating an on-stack bio_ctrl, the explicit free_extent_map() call is not that a big deal.

Thanks,
Qu

Quoted 15 lines

Also, as mentioned before, the subject of the patch is incorrect, it should mention btrfs_bio_ctrl:: instead of extent_map::

Thanks.

Quoted 8 lines
fs/btrfs/extent_io.c +0 -1
         submit_one_bio(&bio_ctrl);
  }
 
 
--

2.45.2

#7 lore raw
In reply to Qu Wenruo
Quoted 14 lines · Qu Wenruo

Unlike data read path, which use cached extent map to reduce overhead, data write path always do the extent map lookup no matter what.

So this patchset will improve the situation by:

- Move em_cached into bio_ctrl
Since the lifespan of them is the same, it's a perfect match.

- Make data write path to use bio_ctrl::em_cached

Unfortunately since my last relocation, I no longer have any dedicated storage attached to my VMs (my laptop only has one NVME slot, and my main workhorse aarch64 board only has one NVME attached either).

Quoted 5 lines

So no benchmark yet, and any extra benchmark would be very appreciated.

Qu Wenruo (2):
btrfs: introduce extent_map::em_cached member
btrfs: utilize cached extent map for data writeback

This looks like a good optimization, we're approaching code freeze (next week) so it would be good to get it merged by then. As it's an optimization we can also postpone it if you're busy with other things.

#8 lore raw
Quoted 17 lines · David Sterba
Quoted 16 lines · Qu Wenruo

Unlike data read path, which use cached extent map to reduce overhead, data write path always do the extent map lookup no matter what.

So this patchset will improve the situation by:

- Move em_cached into bio_ctrl
Since the lifespan of them is the same, it's a perfect match.

- Make data write path to use bio_ctrl::em_cached

Unfortunately since my last relocation, I no longer have any dedicated storage attached to my VMs (my laptop only has one NVME slot, and my main workhorse aarch64 board only has one NVME attached either).

So no benchmark yet, and any extra benchmark would be very appreciated.

Quoted 8 lines
Quoted 3 lines

Qu Wenruo (2):
btrfs: introduce extent_map::em_cached member
btrfs: utilize cached extent map for data writeback

This looks like a good optimization, we're approaching code freeze (next week) so it would be good to get it merged by then. As it's an optimization we can also postpone it if you're busy with other things.

I'd like to postpone this one.

The biggest problem is, this is not working the last time I micro-benchmarked the whole thing.

It turns out that, with the new code the latency to lookup an extent map is in fact larger than the existing code.

Maybe related to the extent map shrinker which is doing a too good job.

I'll refresh this series when the em shrinker get stable and I can re-benchmark the result to see if it's still worthy.

Thanks,
Qu