btrfs: Refactor main loop in extent_readpages

PATCH In review Nikolay Borisov 2 participants 3 messages Nov 29, 2018 – Dec 12, 2018 Btrfs file system

Share & tools

Nikolay Borisov

author

2018-11-29 16:41 UTC #0 lore raw

PATCH 1/1 btrfs: Refactor main loop in extent_readpages

extent_readpages processes all pages in the readlist in batches of 16, this is implemented by a single for loop but thanks to an if condition the loop does 2 things based on whether we've filled the batch or not. Additionally due to the structure of the code there is an additional check which deals with partial batches.

Streamline all of this by explicitly using two loops. The outter one is used to process all pages while the inner one just fills in the batch of 16 (currently). Due to this new structure the code guarantees that all pages are processed in the loop hence the code to deal with any leftovers is eliminated.

This also enable the compiler to inline __extent_readpages:

./scripts/bloat-o-meter fs/btrfs/extent_io.o extent_io.for

add/remove: 0/1 grow/shrink: 1/0 up/down: 660/-820 (-160)
Function old new delta
extent_readpages 476 1136 +660
__extent_readpages 820 - -820
Total: Before=44315, After=44155, chg -0.36%

Signed-off-by Nikolay Borisov <[email protected]>

---
 fs/btrfs/extent_io.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)

fs/btrfs/extent_io.c +16 ~~-21~~

index 8332c5f4b1c3..233f835dd6f8 100644
@@ -4093,43 +4093,38 @@ int extent_writepages(struct address_space *mapping,
 int extent_readpages(struct address_space *mapping, struct list_head *pages,
 		     unsigned nr_pages)
 {
+#define BTRFS_PAGES_BATCH 16
+
 	struct bio *bio = NULL;
-	unsigned page_idx;
 	unsigned long bio_flags = 0;
-	struct page *pagepool[16];
-	struct page *page;
+	struct page *pagepool[BTRFS_PAGES_BATCH];
 	struct extent_map *em_cached = NULL;
 	struct extent_io_tree *tree = &BTRFS_I(mapping->host)->io_tree;
 	int nr = 0;
 	u64 prev_em_start = (u64)-1;
 
-	for (page_idx = 0; page_idx < nr_pages; page_idx++) {
-		page = lru_to_page(pages);
+	while (!list_empty(pages)) {
+		for (nr = 0; nr < BTRFS_PAGES_BATCH && !list_empty(pages);) {
+			struct page *page = lru_to_page(pages);
 
-		prefetchw(&page->flags);
-		list_del(&page->lru);
-		if (add_to_page_cache_lru(page, mapping,
-					page->index,
-					readahead_gfp_mask(mapping))) {
-			put_page(page);
-			continue;
+			prefetchw(&page->flags);
+			list_del(&page->lru);
+			if (add_to_page_cache_lru(page, mapping, page->index,
+						readahead_gfp_mask(mapping))) {
+				put_page(page);
+				continue;
+			}
+
+			pagepool[nr++] = page;
 		}
 
-		pagepool[nr++] = page;
-		if (nr < ARRAY_SIZE(pagepool))
-			continue;
 		__extent_readpages(tree, pagepool, nr, &em_cached, &bio,
-				&bio_flags, &prev_em_start);
-		nr = 0;
+				   &bio_flags, &prev_em_start);
 	}
-	if (nr)
-		__extent_readpages(tree, pagepool, nr, &em_cached, &bio,
-				&bio_flags, &prev_em_start);
 
 	if (em_cached)
 		free_extent_map(em_cached);
 
-	BUG_ON(!list_empty(pages));
 	if (bio)
 		return submit_one_bio(bio, 0, bio_flags);
 	return 0;

Signature · 1 line

2.17.1

David Sterba

Reviewed-by on v1 1/1

2018-12-12 14:41 UTC #1 lore raw

Quoted 32 lines · Nikolay Borisov

extent_readpages processes all pages in the readlist in batches of 16, this is implemented by a single for loop but thanks to an if condition the loop does 2 things based on whether we've filled the batch or not. Additionally due to the structure of the code there is an additional check which deals with partial batches.
Streamline all of this by explicitly using two loops. The outter one is used to process all pages while the inner one just fills in the batch of 16 (currently). Due to this new structure the code guarantees that all pages are processed in the loop hence the code to deal with any leftovers is eliminated.
This also enable the compiler to inline __extent_readpages:
./scripts/bloat-o-meter fs/btrfs/extent_io.o extent_io.for
add/remove: 0/1 grow/shrink: 1/0 up/down: 660/-820 (-160)
Function old new delta
extent_readpages 476 1136 +660
__extent_readpages 820 - -820
Total: Before=44315, After=44155, chg -0.36%
Signed-off-by Nikolay Borisov <[email protected]>
---
 fs/btrfs/extent_io.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)
fs/btrfs/extent_io.c +0 -0
index 8332c5f4b1c3..233f835dd6f8 100644

fs/btrfs/extent_io.c int extent_writepages(struct address_space *mapping, · line 4094

Quoted 2 lines

fs/btrfs/extent_io.c +0 -0

@@ -4093,43 +4093,38 @@ int extent_writepages(struct address_space *mapping,
 int extent_readpages(struct address_space *mapping, struct list_head *pages,
 		     unsigned nr_pages)

nr_pages becomes unused now. I've checked the MM code that calls here and it's guaranteed that the number of list 'pages' members is same as nr_pages.

fs/btrfs/extent_io.c int extent_writepages(struct address_space *mapping, · line 4100

Quoted 9 lines

fs/btrfs/extent_io.c +3 -3

 {
+#define BTRFS_PAGES_BATCH 16
+
 	struct bio *bio = NULL;
-	unsigned page_idx;
 	unsigned long bio_flags = 0;
-	struct page *pagepool[16];
-	struct page *page;
+	struct page *pagepool[BTRFS_PAGES_BATCH];

I don't think we need the extra define, if

	struct page *pagepool[16];

	...

	for (nr = 0; nr < ARRAY_SIZE(pagepool) ...

Otherwise ok, nice cleanup.

Reviewed-by David Sterba <[email protected]>

Nikolay Borisov

author

2018-12-12 15:17 UTC #2 lore raw

Quoted 28 lines · David Sterba

Quoted 27 lines · Nikolay Borisov
extent_readpages processes all pages in the readlist in batches of 16, this is implemented by a single for loop but thanks to an if condition the loop does 2 things based on whether we've filled the batch or not. Additionally due to the structure of the code there is an additional check which deals with partial batches.
Streamline all of this by explicitly using two loops. The outter one is used to process all pages while the inner one just fills in the batch of 16 (currently). Due to this new structure the code guarantees that all pages are processed in the loop hence the code to deal with any leftovers is eliminated.
This also enable the compiler to inline __extent_readpages:
./scripts/bloat-o-meter fs/btrfs/extent_io.o extent_io.for
add/remove: 0/1 grow/shrink: 1/0 up/down: 660/-820 (-160)
Function old new delta
extent_readpages 476 1136 +660
__extent_readpages 820 - -820
Total: Before=44315, After=44155, chg -0.36%
Signed-off-by Nikolay Borisov <[email protected]>
---
 fs/btrfs/extent_io.c | 37 ++++++++++++++++---------------------
 1 file changed, 16 insertions(+), 21 deletions(-)

Quoted 11 lines

fs/btrfs/extent_io.c int extent_writepages(struct address_space *mapping, · line 4094
Quoted 7 lines
fs/btrfs/extent_io.c +0 -0
index 8332c5f4b1c3..233f835dd6f8 100644
@@ -4093,43 +4093,38 @@ int extent_writepages(struct address_space *mapping,
 int extent_readpages(struct address_space *mapping, struct list_head *pages,
 		     unsigned nr_pages)
nr_pages becomes unused now. I've checked the MM code that calls here and it's guaranteed that the number of list 'pages' members is same as nr_pages.

Indeed, I missed that.

Quoted 13 lines

fs/btrfs/extent_io.c int extent_writepages(struct address_space *mapping, · line 4100
Quoted 10 lines
fs/btrfs/extent_io.c +3 -3
 {
+#define BTRFS_PAGES_BATCH 16
+
 	struct bio *bio = NULL;
-	unsigned page_idx;
 	unsigned long bio_flags = 0;
-	struct page *pagepool[16];
-	struct page *page;
+	struct page *pagepool[BTRFS_PAGES_BATCH];
I don't think we need the extra define, if

Quoted 7 lines

	struct page *pagepool[16];

	...

	for (nr = 0; nr < ARRAY_SIZE(pagepool) ...

Otherwise ok, nice cleanup.

Fair enough, I'm ok with that as well.

Quoted 3 lines

Reviewed-by David Sterba <[email protected]>