Submit ->readpages() IO as read-ahead
Subsystems (3)
cover Submit ->readpages() IO as read-ahead
The only caller of ->readpages() is from read-ahead, yet we don't submit IO flagged with REQ_RAHEAD. This means we don't see it in blktrace, for instance, which is a shame. We already make assumptions about ->readpages() just being for read-ahead in the mpage implementation, using readahead_gfp_mask(mapping) as out GFP mask of choice.
This small series fixes up mpage_readpages() to submit with REQ_RAHEAD, which takes care of file systems using mpage_readpages(). The last two fixup ext4 and btrfs.
Signature · 1 line
Jens Axboe
PATCH 1/3 mpage: mpage_readpages() should submit IO as read-ahead
a_ops->readpages() is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that uses mpage_readpages() as it's ->readpages() implementation will now get this right.
--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
fs/mpage.c +9 -8
@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead) { struct inode *inode = page->mapping->host;const unsigned blkbits = inode->i_blkbits;
@@ -161,6 +161,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,struct block_device *bdev =NULL ;int length;int fully_mapped =1 ; +int op_flags = is_readahead ?REQ_RAHEAD :0 ;unsigned nblocks;unsigned relative_block;
@@ -274,7 +275,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,* This page will go to BIO. Do we need to send this BIO off first? */ if (bio && (*last_block_in_bio != blocks[0 ] -1 )) - bio =mpage_bio_submit (REQ_OP_READ ,0 , bio); + bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); alloc_new:if (bio ==NULL ) {
@@ -291,7 +292,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, length = first_hole << blkbits;if (bio_add_page (bio, page, length,0 ) < length) { - bio =mpage_bio_submit (REQ_OP_READ ,0 , bio); + bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); goto alloc_new; }
@@ -299,7 +300,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, nblocks = map_bh->b_size >> blkbits;if ((buffer_boundary (map_bh) && relative_block == nblocks) || (first_hole != blocks_per_page)) - bio =mpage_bio_submit (REQ_OP_READ ,0 , bio); + bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio);else *last_block_in_bio = blocks[blocks_per_page -1 ]; out:
@@ -307,7 +308,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, confused:if (bio) - bio =mpage_bio_submit (REQ_OP_READ ,0 , bio); + bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio);if (!PageUptodate (page))block_read_full_page (page, get_block);else
@@ -384,13 +385,13 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages, nr_pages - page_idx, &last_block_in_bio, &map_bh, &first_logical_block, - get_block, gfp); + get_block, gfp, true); }put_page (page); }BUG_ON (!list_empty (pages));if (bio) -mpage_bio_submit (REQ_OP_READ ,0 , bio); +mpage_bio_submit (REQ_OP_READ ,REQ_RAHEAD , bio);return 0 ; }EXPORT_SYMBOL (mpage_readpages);
@@ -409,7 +410,7 @@ int mpage_readpage(struct page *page, get_block_t get_block) map_bh.b_state =0 ; map_bh.b_size =0 ; bio =do_mpage_readpage (bio, page,1 , &last_block_in_bio, - &map_bh, &first_logical_block, get_block, gfp); + &map_bh, &first_logical_block, get_block, gfp, false);if (bio)mpage_bio_submit (REQ_OP_READ ,0 , bio);return 0 ;
Signature · 1 line
2.7.4
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 15 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
a_ops->readpages()is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that usesmpage_readpages()as it's ->readpages() implementation will now get this right.Signed-off-by Jens Axboe <[email protected]>--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/mpage.c +0
-0
fs/mpage.c static struct bio * · line 149
Quoted 5 lines
fs/mpage.c +1
-1@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)
That's a lot of arguments.
I suspect we'll have a faster kernel if we mark this __always_inline. I think my ancient "This isn't called much at all" over mpage_readpage() remains true. Almost all callers come in via mpage_readpages(), which would benefit from the inlining. But mpage.o gets 1.5k fatter. hm.
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 8 lines · Andrew Morton
fs/mpage.c static struct bio * · line 149
Quoted 5 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
do_mpage_readpage(struct bio *bio, struct page *page, unsigned
nr_pages,fs/mpage.c +1
-1sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
struct mpage_args args = {
.bio = NULL,
.first_logical_block = 0,
.last_block_in_bio = 0,
.is_readahead = true,
.map_bh = {
.b_state = 0;
.b_size = 0;
},
.get_block = get_block,
.gfp = readahead_gfp_mask(mapping);
};...
do_mpage_readpages(&args, page, nr_pages - page_idx);
better than inlining?
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 23 lines · Matthew Wilcox
Quoted 8 lines · Andrew Morton
Review
Quoted 5 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
do_mpage_readpage(struct bio *bio, struct page *page, unsigned
nr_pages,(quoted diff) +1
-1sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
struct mpage_args args = { .bio = NULL, .first_logical_block = 0, .last_block_in_bio = 0, .is_readahead = true, .map_bh = { .b_state = 0; .b_size = 0; }, .get_block = get_block, .gfp = readahead_gfp_mask(mapping); };
Quoted 4 lines
...
do_mpage_readpages(&args, page, nr_pages - page_idx);better than inlining?
It makes it smaller, at least.
fs/mpage.c +74 -71
@@ -133,6 +133,18 @@ map_buffer_to_page(struct page *page, struct buffer_head *bh, int page_block) }while (page_bh != head); } +struct mpage_args { +struct bio *bio; +struct page *page; +unsigned nr_pages; +sector_t last_block_in_bio; +struct buffer_head map_bh; +unsigned long first_logical_block; +get_block_t *get_block; +gfp_t gfp; +bool is_readahead; +}; +/* * This is the worker routine which does all the work of mapping the disk * blocks and constructs largest possible bios, submits them for IO if the
@@ -142,12 +154,9 @@ map_buffer_to_page(struct page *page, struct buffer_head *bh, int page_block)* represent the validity of its disk mapping and to decide when to do the next * get_block() call. */ -static struct bio * -do_mpage_readpage (struct bio *bio,struct page *page,unsigned nr_pages, -sector_t *last_block_in_bio,struct buffer_head *map_bh, -unsigned long *first_logical_block,get_block_t get_block, -gfp_t gfp,bool is_readahead) +static struct bio *do_mpage_readpage (struct mpage_args *args) { +struct page *page = args->page;struct inode *inode = page->mapping->host;const unsigned blkbits = inode->i_blkbits;const unsigned blocks_per_page =PAGE_SIZE >> blkbits;
@@ -161,7 +170,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,struct block_device *bdev =NULL ;int length;int fully_mapped =1 ; -int op_flags = is_readahead ?REQ_RAHEAD :0 ; +int op_flags = args->is_readahead ?REQ_RAHEAD :0 ;unsigned nblocks;unsigned relative_block;
@@ -169,7 +178,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, goto confused; block_in_file = (sector_t )page->index << (PAGE_SHIFT - blkbits); - last_block = block_in_file + nr_pages * blocks_per_page; + last_block = block_in_file + args->nr_pages * blocks_per_page; last_block_in_file = (i_size_read (inode) + blocksize -1 ) >> blkbits;if (last_block > last_block_in_file) last_block = last_block_in_file;
@@ -178,43 +187,43 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,/* * Map blocks using the result from the previous get_blocks call first. */ - nblocks = map_bh->b_size >> blkbits; -if (buffer_mapped (map_bh) && block_in_file > *first_logical_block && - block_in_file < (*first_logical_block + nblocks)) { -unsigned map_offset = block_in_file - *first_logical_block; + nblocks = args->map_bh.b_size >> blkbits; +if (buffer_mapped (&args->map_bh) && block_in_file > args->first_logical_block && + block_in_file < (args->first_logical_block + nblocks)) { +unsigned map_offset = block_in_file - args->first_logical_block;unsigned last = nblocks - map_offset;for (relative_block =0 ; ; relative_block++) {if (relative_block == last) { -clear_buffer_mapped (map_bh); +clear_buffer_mapped (&args->map_bh);break ; }if (page_block == blocks_per_page)break ; - blocks[page_block] = map_bh->b_blocknr + map_offset + + blocks[page_block] = args->map_bh.b_blocknr + map_offset + relative_block; page_block++; block_in_file++; } - bdev = map_bh->b_bdev; + bdev = args->map_bh.b_bdev; }/* * Then do more get_blocks calls until we are done with this page. */ - map_bh->b_page = page; + args->map_bh.b_page = page;while (page_block < blocks_per_page) { - map_bh->b_state =0 ; - map_bh->b_size =0 ; + args->map_bh.b_state =0 ; + args->map_bh.b_size =0 ;if (block_in_file < last_block) { - map_bh->b_size = (last_block-block_in_file) << blkbits; -if (get_block (inode, block_in_file, map_bh,0 )) + args->map_bh.b_size = (last_block-block_in_file) << blkbits; +if (args->get_block (inode, block_in_file, &args->map_bh,0 )) goto confused; - *first_logical_block = block_in_file; + args->first_logical_block = block_in_file; } -if (!buffer_mapped (map_bh)) { +if (!buffer_mapped (&args->map_bh)) { fully_mapped =0 ;if (first_hole == blocks_per_page) first_hole = page_block;
@@ -229,8 +238,8 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,* we just collected from get_block into the page's buffers * so readpage doesn't have to repeat the get_block call */ -if (buffer_uptodate (map_bh)) { -map_buffer_to_page (page, map_bh, page_block); +if (buffer_uptodate (&args->map_bh)) { +map_buffer_to_page (page, &args->map_bh, page_block); goto confused; }
@@ -238,20 +247,20 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, goto confused;/* hole -> non-hole */ /* Contiguous blocks? */ -if (page_block && blocks[page_block-1 ] != map_bh->b_blocknr-1 ) +if (page_block && blocks[page_block-1 ] != args->map_bh.b_blocknr-1 ) goto confused; - nblocks = map_bh->b_size >> blkbits; + nblocks = args->map_bh.b_size >> blkbits;for (relative_block =0 ; ; relative_block++) {if (relative_block == nblocks) { -clear_buffer_mapped (map_bh); +clear_buffer_mapped (&args->map_bh);break ; }else if (page_block == blocks_per_page)break ; - blocks[page_block] = map_bh->b_blocknr+relative_block; + blocks[page_block] = args->map_bh.b_blocknr+relative_block; page_block++; block_in_file++; } - bdev = map_bh->b_bdev; + bdev = args->map_bh.b_bdev; }if (first_hole != blocks_per_page) {
@@ -274,43 +283,43 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages,/* * This page will go to BIO. Do we need to send this BIO off first? */ -if (bio && (*last_block_in_bio != blocks[0 ] -1 )) - bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); +if (args->bio && (args->last_block_in_bio != blocks[0 ] -1 )) + args->bio =mpage_bio_submit (REQ_OP_READ , op_flags, args->bio); alloc_new: -if (bio ==NULL ) { +if (args->bio ==NULL ) {if (first_hole == blocks_per_page) {if (!bdev_read_page (bdev, blocks[0 ] << (blkbits -9 ), page)) goto out; } - bio =mpage_alloc (bdev, blocks[0 ] << (blkbits -9 ), -min_t (int, nr_pages,BIO_MAX_PAGES ), gfp); -if (bio ==NULL ) + args->bio =mpage_alloc (bdev, blocks[0 ] << (blkbits -9 ), +min_t (int, args->nr_pages,BIO_MAX_PAGES ), args->gfp); +if (args->bio ==NULL ) goto confused; } length = first_hole << blkbits; -if (bio_add_page (bio, page, length,0 ) < length) { - bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); +if (bio_add_page (args->bio, page, length,0 ) < length) { + args->bio =mpage_bio_submit (REQ_OP_READ , op_flags, args->bio); goto alloc_new; } - relative_block = block_in_file - *first_logical_block; - nblocks = map_bh->b_size >> blkbits; -if ((buffer_boundary (map_bh) && relative_block == nblocks) || + relative_block = block_in_file - args->first_logical_block; + nblocks = args->map_bh.b_size >> blkbits; +if ((buffer_boundary (&args->map_bh) && relative_block == nblocks) || (first_hole != blocks_per_page)) - bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); + args->bio =mpage_bio_submit (REQ_OP_READ , op_flags, args->bio);else - *last_block_in_bio = blocks[blocks_per_page -1 ]; + args->last_block_in_bio = blocks[blocks_per_page -1 ]; out: -return bio; +return args->bio; confused: -if (bio) - bio =mpage_bio_submit (REQ_OP_READ , op_flags, bio); +if (args->bio) + args->bio =mpage_bio_submit (REQ_OP_READ , op_flags, args->bio);if (!PageUptodate (page)) -block_read_full_page (page, get_block); +block_read_full_page (page, args->get_block);else unlock_page (page); goto out;
@@ -364,15 +373,13 @@ int mpage_readpages(struct address_space *mapping, struct list_head *pages, unsigned nr_pages, get_block_t get_block) { - struct bio *bio = NULL; + struct mpage_args args = { + .get_block = get_block, + .gfp =readahead_gfp_mask (mapping), + .is_readahead = true, + };unsigned page_idx; -sector_t last_block_in_bio =0 ; -struct buffer_head map_bh; -unsigned long first_logical_block =0 ; -gfp_t gfp =readahead_gfp_mask (mapping); - map_bh.b_state =0 ; - map_bh.b_size =0 ;for (page_idx =0 ; page_idx < nr_pages; page_idx++) {struct page *page =lru_to_page (pages);
@@ -380,18 +387,16 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages,list_del (&page->lru);if (!add_to_page_cache_lru (page, mapping, page->index, - gfp)) { - bio =do_mpage_readpage (bio, page, - nr_pages - page_idx, - &last_block_in_bio, &map_bh, - &first_logical_block, - get_block, gfp, true); + args.gfp)) { + args.page = page; + args.nr_pages = nr_pages - page_idx; + args.bio =do_mpage_readpage (&args); }put_page (page); }BUG_ON (!list_empty (pages)); -if (bio) -mpage_bio_submit (REQ_OP_READ ,REQ_RAHEAD , bio); +if (args.bio) +mpage_bio_submit (REQ_OP_READ ,REQ_RAHEAD , args.bio);return 0 ; }EXPORT_SYMBOL (mpage_readpages);
@@ -401,18 +406,16 @@ EXPORT_SYMBOL(mpage_readpages);*/ int mpage_readpage (struct page *page,get_block_t get_block) { -struct bio *bio =NULL ; -sector_t last_block_in_bio =0 ; -struct buffer_head map_bh; -unsigned long first_logical_block =0 ; -gfp_t gfp =mapping_gfp_constraint (page->mapping,GFP_KERNEL ); +struct mpage_args args = { + .page = page, + .nr_pages =1 , + .get_block = get_block, + .gfp =mapping_gfp_constraint (page->mapping,GFP_KERNEL ), + }; - map_bh.b_state =0 ; - map_bh.b_size =0 ; - bio =do_mpage_readpage (bio, page,1 , &last_block_in_bio, - &map_bh, &first_logical_block, get_block, gfp, false); -if (bio) -mpage_bio_submit (REQ_OP_READ ,0 , bio); + args.bio =do_mpage_readpage (&args); +if (args.bio) +mpage_bio_submit (REQ_OP_READ ,0 , args.bio);return 0 ; }EXPORT_SYMBOL (mpage_readpage);
Signature · 1 line
Jens Axboe
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
In reply to Andrew MortonQuoted 25 lines · Andrew Morton
Quoted 15 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
a_ops->readpages()is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that usesmpage_readpages()as it's ->readpages() implementation will now get this right.Signed-off-by Jens Axboe <[email protected]>--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/mpage.c +0
-0fs/mpage.c static struct bio * · line 149
Quoted 5 lines
fs/mpage.c +1
-1@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
Quoted 5 lines
I suspect we'll have a faster kernel if we mark this
__always_inline. I think my ancient "This isn't called much at all" overmpage_readpage()remains true. Almost all callers come in viampage_readpages(), which would benefit from the inlining. But mpage.o gets 1.5k fatter. hm.
Was going to send out a v2, but would be nice to get some consensus on what you prefer here. I can either do the struct version, or I can keep it as-is (going from 8 to 9 arguments). For the struct version, I'd prefer to do that as a prep patch, so the functional change is clear.
Signature · 1 line
Jens Axboe
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 32 lines · On Tue, 29 May 2018 08:36:41 -0600 Jens Axboe <[email protected]> wrote:
Quoted 25 lines · Andrew Morton
Quoted 15 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
a_ops->readpages()is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that usesmpage_readpages()as it's ->readpages() implementation will now get this right.Signed-off-by Jens Axboe <[email protected]>--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/mpage.c +0
-0fs/mpage.c static struct bio * · line 149
Quoted 5 lines
fs/mpage.c +1
-1@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
Quoted 5 lines
I suspect we'll have a faster kernel if we mark this
__always_inline. I think my ancient "This isn't called much at all" overmpage_readpage()remains true. Almost all callers come in viampage_readpages(), which would benefit from the inlining. But mpage.o gets 1.5k fatter. hm.
Quoted 5 lines
Was going to send out a v2, but would be nice to get some consensus on what you prefer here. I can either do the struct version, or I can keep it as-is (going from 8 to 9 arguments). For the struct version, I'd prefer to do that as a prep patch, so the functional change is clear.
The struct thing makes the code smaller, and presumably faster, doesn't it? I suppose it saves a bit of stack as well, by letting the callee access the caller's locals rather than a copy of them. All sounds good to me?
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 40 lines · Andrew Morton
Quoted 32 lines · On Tue, 29 May 2018 08:36:41 -0600 Jens Axboe <[email protected]> wrote:
Quoted 25 lines · Andrew Morton
Quoted 15 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
a_ops->readpages()is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that usesmpage_readpages()as it's ->readpages() implementation will now get this right.Signed-off-by Jens Axboe <[email protected]>--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/mpage.c +0
-0fs/mpage.c static struct bio * · line 149
Quoted 5 lines
fs/mpage.c +1
-1@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
Quoted 5 lines
I suspect we'll have a faster kernel if we mark this
__always_inline. I think my ancient "This isn't called much at all" overmpage_readpage()remains true. Almost all callers come in viampage_readpages(), which would benefit from the inlining. But mpage.o gets 1.5k fatter. hm.Quoted 5 lines
Was going to send out a v2, but would be nice to get some consensus on what you prefer here. I can either do the struct version, or I can keep it as-is (going from 8 to 9 arguments). For the struct version, I'd prefer to do that as a prep patch, so the functional change is clear.
Quoted 4 lines
The struct thing makes the code smaller, and presumably faster, doesn't it? I suppose it saves a bit of stack as well, by letting the callee access the caller's locals rather than a copy of them. All sounds good to me?
That's what I thought to, so already prepped the series. Sending it out.
Signature · 1 line
Jens Axboe
Re: [PATCH 1/3] mpage: mpage_readpages() should submit IO as read-ahead
Quoted 41 lines · Jens Axboe
Quoted 40 lines · Andrew Morton
Quoted 37 lines · On Tue, 29 May 2018 08:36:41 -0600 Jens Axboe <[email protected]> wrote:
Quoted 25 lines · Andrew Morton
Quoted 15 lines · On Thu, 24 May 2018 10:02:52 -0600 Jens Axboe <[email protected]> wrote:
a_ops->readpages()is only ever used for read-ahead, yet we don't flag the IO being submitted as such. Fix that up. Any file system that usesmpage_readpages()as it's ->readpages() implementation will now get this right.Signed-off-by Jens Axboe <[email protected]>--- fs/mpage.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)fs/mpage.c +0
-0fs/mpage.c static struct bio * · line 149
Quoted 5 lines
fs/mpage.c +1
-1@@ -146,7 +146,7 @@ static struct bio * do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, sector_t *last_block_in_bio, struct buffer_head *map_bh, unsigned long *first_logical_block, get_block_t get_block, - gfp_t gfp) + gfp_t gfp, bool is_readahead)That's a lot of arguments.
Quoted 5 lines
I suspect we'll have a faster kernel if we mark this
__always_inline. I think my ancient "This isn't called much at all" overmpage_readpage()remains true. Almost all callers come in viampage_readpages(), which would benefit from the inlining. But mpage.o gets 1.5k fatter. hm.Was going to send out a v2, but would be nice to get some consensus on what you prefer here. I can either do the struct version, or I can keep it as-is (going from 8 to 9 arguments). For the struct version, I'd prefer to do that as a prep patch, so the functional change is clear.
Quoted 6 lines
Quoted 4 lines
The struct thing makes the code smaller, and presumably faster, doesn't it? I suppose it saves a bit of stack as well, by letting the callee access the caller's locals rather than a copy of them. All sounds good to me?
That's what I thought to, so already prepped the series. Sending it out.
We could actually kill args->gfp as well, since that's dependent on args->is_readahead anyway. Separate patch, or fold into patch #2? Incremental below.
fs/mpage.c +13 -7
@@ -137,12 +137,11 @@ struct mpage_readpage_args {struct bio *bio;struct page *page;unsigned nr_pages; +bool is_readahead;sector_t last_block_in_bio;struct buffer_head map_bh;unsigned long first_logical_block;get_block_t *get_block; -gfp_t gfp; -bool is_readahead; };/*
@@ -171,9 +170,18 @@ static struct bio *do_mpage_readpage(struct mpage_readpage_args *args)struct block_device *bdev =NULL ;int length;int fully_mapped =1 ; -int op_flags = args->is_readahead ?REQ_RAHEAD :0 ; +int op_flags;unsigned nblocks;unsigned relative_block; +gfp_t gfp; + +if (args->is_readahead) { + op_flags =REQ_RAHEAD ; + gfp =readahead_gfp_mask (page->mapping); + }else { + op_flags =0 ; + gfp =mapping_gfp_constraint (page->mapping,GFP_KERNEL ); + }if (page_has_buffers (page)) goto confused;
@@ -295,7 +303,7 @@ static struct bio *do_mpage_readpage(struct mpage_readpage_args *args) goto out; } args->bio =mpage_alloc (bdev, blocks[0 ] << (blkbits -9 ), -min_t (int, args->nr_pages,BIO_MAX_PAGES ), args->gfp); +min_t (int, args->nr_pages,BIO_MAX_PAGES ), gfp);if (args->bio ==NULL ) goto confused; }
@@ -376,7 +384,6 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages, {struct mpage_readpage_args args = { .get_block = get_block, - .gfp =readahead_gfp_mask (mapping), .is_readahead = true, };unsigned page_idx;
@@ -388,7 +395,7 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages,list_del (&page->lru);if (!add_to_page_cache_lru (page, mapping, page->index, - args.gfp)) { +readahead_gfp_mask (mapping))) { args.page = page; args.nr_pages = nr_pages - page_idx; args.bio =do_mpage_readpage (&args);
@@ -411,7 +418,6 @@ int mpage_readpage(struct page *page, get_block_t get_block) .page = page, .nr_pages = 1, .get_block = get_block, - .gfp = mapping_gfp_constraint(page->mapping, GFP_KERNEL), }; args.bio =do_mpage_readpage (&args);
Signature · 1 line
Jens Axboe
PATCH 2/3 btrfs: readpages() should submit IO as read-ahead
In reply to Jens Axboea_ops->readpages() is only ever used for read-ahead. Ensure that we pass this information down to the block layer.
--- fs/btrfs/extent_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/extent_io.c +1 -1
@@ -3119,7 +3119,7 @@ static inline void __do_contiguous_readpages(struct extent_io_tree *tree,for (index =0 ; index < nr_pages; index++) {__do_readpage (tree, pages[index], btrfs_get_extent, em_cached, - bio,0 , bio_flags,0 , prev_em_start); + bio,0 , bio_flags,REQ_RAHEAD , prev_em_start);put_page (pages[index]); } }
Signature · 1 line
2.7.4
PATCH 3/3 ext4: readpages() should submit IO as read-ahead
In reply to Jens Axboea_ops->readpages() is only ever used for read-ahead. Ensure that we pass this information down to the block layer.
--- fs/ext4/readpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/readpage.c +1 -1
@@ -259,7 +259,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0 ] << (blkbits -9 ); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; -bio_set_op_attrs (bio,REQ_OP_READ ,0 ); +bio_set_op_attrs (bio,REQ_OP_READ ,REQ_RAHEAD ); } length = first_hole << blkbits;
Signature · 1 line
2.7.4
Theodore Y. Ts'o
maintainerRe: [PATCH 3/3] ext4: readpages() should submit IO as read-ahead
Quoted 13 lines · Jens Axboe
a_ops->readpages()is only ever used for read-ahead. Ensure that we pass this information down to the block layer.Signed-off-by Jens Axboe <[email protected]>--- fs/ext4/readpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/ext4/readpage.c +0
-0
fs/ext4/readpage.c int ext4_mpage_readpages(struct address_space *mapping, · line 263
Quoted 6 lines
fs/ext4/readpage.c +1
-1@@ -259,7 +259,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0 ] << (blkbits -9 ); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; -bio_set_op_attrs (bio,REQ_OP_READ ,0 ); +bio_set_op_attrs (bio,REQ_OP_READ ,REQ_RAHEAD ); }
The problem is that ext4_readpage() also calls ext4_mpage_readpages(). So you can't set REQ_RAHEAD unconditionally here.
- Ted
Re: [PATCH 3/3] ext4: readpages() should submit IO as read-ahead
Quoted 9 lines · Theodore Y. Ts'o
Quoted 8 lines · Jens Axboe
a_ops->readpages()is only ever used for read-ahead. Ensure that we pass this information down to the block layer.Signed-off-by Jens Axboe <[email protected]>--- fs/ext4/readpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
fs/ext4/readpage.c int ext4_mpage_readpages(struct address_space *mapping, · line 263
Quoted 11 lines
fs/ext4/readpage.c +1
-1@@ -259,7 +259,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0 ] << (blkbits -9 ); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; -bio_set_op_attrs (bio,REQ_OP_READ ,0 ); +bio_set_op_attrs (bio,REQ_OP_READ ,REQ_RAHEAD ); }The problem is that
ext4_readpage()also callsext4_mpage_readpages(). So you can't setREQ_RAHEADunconditionally here.
Oh shoot, yeah I did miss that. I'll update it, thanks Ted.
Signature · 1 line
Jens Axboe
Re: [PATCH 3/3] ext4: readpages() should submit IO as read-ahead
Quoted 25 lines · Jens Axboe
Quoted 9 lines · Theodore Y. Ts'o
Quoted 8 lines · Jens Axboe
a_ops->readpages()is only ever used for read-ahead. Ensure that we pass this information down to the block layer.Signed-off-by Jens Axboe <[email protected]>--- fs/ext4/readpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 14 lines
fs/ext4/readpage.c int ext4_mpage_readpages(struct address_space *mapping, · line 263
Quoted 11 lines
fs/ext4/readpage.c +1
-1@@ -259,7 +259,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0 ] << (blkbits -9 ); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; -bio_set_op_attrs (bio,REQ_OP_READ ,0 ); +bio_set_op_attrs (bio,REQ_OP_READ ,REQ_RAHEAD ); }The problem is that
ext4_readpage()also callsext4_mpage_readpages(). So you can't setREQ_RAHEADunconditionally here.
Quoted 1 line
Oh shoot, yeah I did miss that. I'll update it, thanks Ted.
Updated version:
Signature · 1 line
Jens Axboe
Quoted 10 lines · Jens Axboe
The only caller of ->readpages() is from read-ahead, yet we don't submit IO flagged with
REQ_RAHEAD. This means we don't see it in blktrace, for instance, which is a shame. We already make assumptions about ->readpages() just being for read-ahead in the mpage implementation, usingreadahead_gfp_mask(mapping)as out GFP mask of choice.This small series fixes up
mpage_readpages()to submit withREQ_RAHEAD, which takes care of file systems usingmpage_readpages(). The last two fixup ext4 and btrfs.
What are the benefits? Any setup where this buys us anything? Any setup where this actually regressed because drivers/hardware are doing stupid things with REQ_RAHEAD?
Quoted 8 lines · Christoph Hellwig
Quoted 7 lines · Jens Axboe
The only caller of ->readpages() is from read-ahead, yet we don't submit IO flagged with
REQ_RAHEAD. This means we don't see it in blktrace, for instance, which is a shame. We already make assumptions about ->readpages() just being for read-ahead in the mpage implementation, usingreadahead_gfp_mask(mapping)as out GFP mask of choice.
Quoted 5 lines
Quoted 3 lines
This small series fixes up
mpage_readpages()to submit withREQ_RAHEAD, which takes care of file systems usingmpage_readpages(). The last two fixup ext4 and btrfs.What are the benefits? Any setup where this buys us anything? Any
The first benefit is getting the tracing right. Before you could not see which parts where readahead in blktrace, and which parts were not. This now works fine.
The second motivation is fixing an issue around big read ahead windows where a severely bogged down box takes forever to exit a task that was killed, because it's going through tons of IOs on an oversubscribed disk. We see this at FB. 4MB window, and IOs being largely non-sequential. 1000 requests on a rotating drive that takes seconds to do each one. This requires fixing on top, which can be pretty easily done as long as we acknowledge that ->readpages() is strictly for read-ahead.
Quoted 2 lines
setup where this actually regressed because
drivers/hardwareare doing stupid things withREQ_RAHEAD?
I'd sure hope not, we are using REQ_RAHEAD in various meta data related bits. But the main read-ahead was not.
Signature · 1 line
Jens Axboe