add an ordered_extent pointer to struct btrfs_bio v2
Subsystems (2)
Hi all,
this series adds a pointer to the ordered_extent to struct btrfs_bio to reduce the repeated lookups in the rbtree. For non-buffered I/Os the I/O will now never do a lookup of the ordered extent tree (other places like waiting for I/O still do). For buffered I/O there is still a lookup as the writepages code is structured in a way that makes it impossible to just pass the ordered_extent down. With some of the work from Goldwyn this should eventually become possible as well, though.
Changes since v1:
- rebased to the latest misc-next tree with the changes to not split ordered extents for zoned writes
- rename is_data_bio to is_data_bbio
- add a new bbio_has_ordered_extent helper
Diffstat:
fs/btrfs/bio.c | 61 ++++++++++++--- fs/btrfs/bio.h | 12 --- fs/btrfs/compression.c | 40 ++++------ fs/btrfs/compression.h | 5 - fs/btrfs/extent_io.c | 28 +++---- fs/btrfs/file-item.c | 59 --------------- fs/btrfs/inode.c | 108 ++++++++++++++------------- fs/btrfs/ordered-data.c | 169 +++++++++++++++++++++++-------------------- fs/btrfs/ordered-data.h | 7 - fs/btrfs/relocation.c | 36 +++------ fs/btrfs/relocation.h | 2 include/trace/events/btrfs.h | 29 +++++++ 12 files changed, 278 insertions(+), 278 deletions(-)
PATCH 1/17 btrfs: fix file_offset for REQ_BTRFS_ONE_ORDERED bios that get split
If a bio gets split, it needs to have a proper file_offset for checksum validation and repair to work properly.
Based on feedback from Josef, commit 852eee62d31a ("btrfs: allow btrfs_submit_bio to split bios") skipped this adjustment for ONE_ORDERED bios. But if we actually ever need to split a ONE_ORDERED read bio, this will lead to a wrong file offset in the repair code. Right now the only user of the file_offset is logging of an error message so this is mostly harmless, but the wrong offset might be more problematic for additional users in the future.
--- fs/btrfs/bio.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/bio.c +1 -2
@@ -81,8 +81,7 @@ static struct btrfs_bio *btrfs_split_bio(struct btrfs_fs_info *fs_info,btrfs_bio_init (bbio, fs_info,NULL , orig_bbio); bbio->inode = orig_bbio->inode; bbio->file_offset = orig_bbio->file_offset; -if (!(orig_bbio->bio.bi_opf &REQ_BTRFS_ONE_ORDERED )) - orig_bbio->file_offset += map_length; + orig_bbio->file_offset += map_length;atomic_inc (&orig_bbio->pending_ios);return bbio;
Signature · 1 line
2.39.2
PATCH 2/17 btrfs: limit write bios to a single ordered extent
In reply to Christoph HellwigCurrently buffered writeback bios are allowed to span multiple ordered_extents, although that basically never actually happens since commit 4a445b7b6178 ("btrfs: don't merge pages into bio if their page offset is not contiguous").
Supporting bios than span ordered_extents complicates the file checksumming code, and prevents us from adding an ordered_extent pointer to the btrfs_bio structure. Use the existing code to limit a bio to single ordered_extent for zoned device writes for all writes.
This allows to remove the REQ_BTRFS_ONE_ORDERED flags, and the handling of multiple ordered_extents in btrfs_csum_one_bio.
--- fs/btrfs/bio.c | 3 --- fs/btrfs/bio.h | 3 --- fs/btrfs/compression.c | 2 -- fs/btrfs/extent_io.c | 9 ++------- fs/btrfs/file-item.c | 38 -------------------------------------- 5 files changed, 2 insertions(+), 53 deletions(-)
fs/btrfs/bio.c +0 -3
@@ -458,9 +458,6 @@ static void btrfs_submit_mirrored_bio(struct btrfs_io_context *bioc, int dev_nr)static void __btrfs_submit_bio (struct bio *bio,struct btrfs_io_context *bioc,struct btrfs_io_stripe *smap,int mirror_num) { -/* Do not leak our private flag into the block layer. */ - bio->bi_opf &= ~REQ_BTRFS_ONE_ORDERED ; -if (!bioc) {/* Single mirror read/write fast path. */ btrfs_bio (bio)->mirror_num = mirror_num;
fs/btrfs/bio.h +0 -3
@@ -102,9 +102,6 @@ static inline void btrfs_bio_end_io(struct btrfs_bio *bbio, blk_status_t status) bbio->end_io (bbio); } -/* Bio only refers to one ordered extent. */ -#define REQ_BTRFS_ONE_ORDERED REQ_DRV -/* Submit using blkcg_punt_bio_submit. */ #define REQ_BTRFS_CGROUP_PUNT REQ_FS_PRIVATE
fs/btrfs/compression.c +0 -2
@@ -295,8 +295,6 @@ void btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start,ASSERT (IS_ALIGNED (start, fs_info->sectorsize) &&IS_ALIGNED (len, fs_info->sectorsize)); - write_flags |=REQ_BTRFS_ONE_ORDERED ; - cb =alloc_compressed_bio (inode, start,REQ_OP_WRITE | write_flags, end_compressed_bio_write); cb->start = start;
fs/btrfs/extent_io.c +2 -7
@@ -796,12 +796,9 @@ static void alloc_new_bio(struct btrfs_inode *inode, bio_ctrl->len_to_oe_boundary =U32_MAX ;/* -* Limit the extent to the ordered boundary for Zone Append. -* Compressed bios aren't submitted directly, so it doesn't apply to -* them. +* Limit data write bios to the ordered boundary. */ -if (bio_ctrl->compress_type ==BTRFS_COMPRESS_NONE && -btrfs_use_zone_append (bbio)) { +if (bio_ctrl->wbc) {struct btrfs_ordered_extent *ordered; ordered =btrfs_lookup_ordered_extent (inode, file_offset);
@@ -811,9 +808,7 @@ static void alloc_new_bio(struct btrfs_inode *inode, ordered->disk_num_bytes - file_offset);btrfs_put_ordered_extent (ordered); } - } -if (bio_ctrl->wbc) {/* * Pick the last added device to support cgroup writeback. For * multi-device file systems this means blk-cgroup policies have
fs/btrfs/file-item.c +0 -38
@@ -733,8 +733,6 @@ blk_status_t btrfs_csum_one_bio(struct btrfs_bio *bbio)struct bio_vec bvec;int index;unsigned int blockcount; -unsigned long total_bytes =0 ; -unsigned long this_sum_bytes =0 ;int i;unsigned nofs_flag;
@@ -776,34 +774,6 @@ blk_status_t btrfs_csum_one_bio(struct btrfs_bio *bbio) - 1);for (i =0 ; i < blockcount; i++) { -if (!(bio->bi_opf &REQ_BTRFS_ONE_ORDERED ) && - !in_range (offset, ordered->file_offset, - ordered->num_bytes)) { -unsigned long bytes_left; - - sums->len = this_sum_bytes; - this_sum_bytes =0 ; -btrfs_add_ordered_sum (ordered, sums); -btrfs_put_ordered_extent (ordered); - - bytes_left = bio->bi_iter.bi_size - total_bytes; - - nofs_flag =memalloc_nofs_save (); - sums =kvzalloc (btrfs_ordered_sum_size (fs_info, - bytes_left),GFP_KERNEL ); -memalloc_nofs_restore (nofs_flag); -if (!sums) -return BLK_STS_RESOURCE ; - - sums->len = bytes_left; - ordered =btrfs_lookup_ordered_extent (inode, - offset); -ASSERT (ordered);/* Logic error */ - sums->logical = (bio->bi_iter.bi_sector <<SECTOR_SHIFT ) - + total_bytes; - index =0 ; - } - data =bvec_kmap_local (&bvec); crypto_shash_digest(shash, data + (i * fs_info->sectorsize),
@@ -812,18 +782,10 @@ blk_status_t btrfs_csum_one_bio(struct btrfs_bio *bbio)kunmap_local (data); index += fs_info->csum_size; offset += fs_info->sectorsize; - this_sum_bytes += fs_info->sectorsize; - total_bytes += fs_info->sectorsize; } } - this_sum_bytes =0 ; -/* -* The ->sums assignment is for zoned writes, where a bio never spans -* ordered extents and is only done unconditionally because that's cheaper -* than a branch. -*/ bbio->sums = sums;btrfs_add_ordered_sum (ordered, sums);btrfs_put_ordered_extent (ordered);
Signature · 1 line
2.39.2
PATCH 3/17 btrfs: merge the two calls to btrfs_add_ordered_extent in run_delalloc_nocow
In reply to Christoph HellwigRefactor run_delalloc_nocow a little bit so that there is only a single call to btrfs_add_ordered_extent instead of two.
--- fs/btrfs/inode.c | 37 +++++++++++++++---------------------- 1 file changed, 15 insertions(+), 22 deletions(-)
fs/btrfs/inode.c +15 -22
@@ -2128,6 +2128,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,u64 ram_bytes;u64 nocow_end;int extent_type; +bool is_prealloc; nocow = false;
@@ -2266,8 +2267,8 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode, } nocow_end = cur_offset + nocow_args.num_bytes -1 ; - -if (extent_type ==BTRFS_FILE_EXTENT_PREALLOC ) { + is_prealloc = extent_type ==BTRFS_FILE_EXTENT_PREALLOC ; +if (is_prealloc) {u64 orig_start = found_key.offset - nocow_args.extent_offset;struct extent_map *em;
@@ -2283,29 +2284,21 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode, goto error; }free_extent_map (em); - ret =btrfs_add_ordered_extent (inode, - cur_offset, nocow_args.num_bytes, - nocow_args.num_bytes, - nocow_args.disk_bytenr, - nocow_args.num_bytes,0 , -1 <<BTRFS_ORDERED_PREALLOC , -BTRFS_COMPRESS_NONE ); -if (ret) { + } + + ret =btrfs_add_ordered_extent (inode, cur_offset, + nocow_args.num_bytes, nocow_args.num_bytes, + nocow_args.disk_bytenr, nocow_args.num_bytes,0 , + is_prealloc ? + (1 <<BTRFS_ORDERED_PREALLOC ) : + (1 <<BTRFS_ORDERED_NOCOW ), +BTRFS_COMPRESS_NONE ); +if (ret) { +if (is_prealloc) {btrfs_drop_extent_map_range (inode, cur_offset, nocow_end, false); - goto error; } - }else { - ret =btrfs_add_ordered_extent (inode, cur_offset, - nocow_args.num_bytes, - nocow_args.num_bytes, - nocow_args.disk_bytenr, - nocow_args.num_bytes, -0 , -1 <<BTRFS_ORDERED_NOCOW , -BTRFS_COMPRESS_NONE ); -if (ret) - goto error; + goto error; }if (nocow) {
Signature · 1 line
2.39.2
Re: [PATCH 03/17] btrfs: merge the two calls to btrfs_add_ordered_extent in run_delalloc_nocow
Looks good,
PATCH 4/17 btrfs: pass an ordered_extent to btrfs_reloc_clone_csums
In reply to Christoph HellwigBoth callers of btrfs_reloc_clone_csums allocate the ordered_extent that btrfs_reloc_clone_csums operates on. Switch them to use btrfs_alloc_ordered_extent instead of btrfs_add_ordered_extent and pass the ordered_extent to btrfs_reloc_clone_csums instead of doing an extra lookup.
--- fs/btrfs/inode.c | 29 ++++++++++++++++++----------- fs/btrfs/relocation.c | 36 +++++++++++++++--------------------- fs/btrfs/relocation.h | 2 +- 3 files changed, 34 insertions(+), 33 deletions(-)
fs/btrfs/inode.c +18 -11
@@ -1487,6 +1487,8 @@ static noinline int cow_file_range(struct btrfs_inode *inode, min_alloc_size = fs_info->sectorsize;while (num_bytes >0 ) { +struct btrfs_ordered_extent *ordered; + cur_alloc_size = num_bytes; ret = btrfs_reserve_extent(root, cur_alloc_size, cur_alloc_size, min_alloc_size, 0, alloc_hint,
@@ -1528,16 +1530,18 @@ static noinline int cow_file_range(struct btrfs_inode *inode, }free_extent_map (em); - ret =btrfs_add_ordered_extent (inode, start, ram_size, ram_size, - ins.objectid, cur_alloc_size,0 , -1 <<BTRFS_ORDERED_REGULAR , -BTRFS_COMPRESS_NONE ); -if (ret) + ordered =btrfs_alloc_ordered_extent (inode, start, ram_size, + ram_size, ins.objectid, cur_alloc_size, +0 ,1 <<BTRFS_ORDERED_REGULAR , +BTRFS_COMPRESS_NONE ); +if (IS_ERR (ordered)) { + ret =PTR_ERR (ordered); goto out_drop_extent_cache; + }if (btrfs_is_data_reloc_root (root)) { - ret =btrfs_reloc_clone_csums (inode, start, - cur_alloc_size); + ret =btrfs_reloc_clone_csums (ordered); +/* * Only drop cache here, and process as normal. *
@@ -1554,6 +1558,7 @@ static noinline int cow_file_range(struct btrfs_inode *inode, start + ram_size -1 , false); } +btrfs_put_ordered_extent (ordered);btrfs_dec_block_group_reservations (fs_info, ins.objectid);
@@ -2121,6 +2126,7 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode, nocow_args.writeback_path = true;while (1 ) { +struct btrfs_ordered_extent *ordered;struct btrfs_key found_key;struct btrfs_file_extent_item *fi;struct extent_buffer *leaf;
@@ -2286,18 +2292,19 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,free_extent_map (em); } - ret =btrfs_add_ordered_extent (inode, cur_offset, + ordered =btrfs_alloc_ordered_extent (inode, cur_offset, nocow_args.num_bytes, nocow_args.num_bytes, nocow_args.disk_bytenr, nocow_args.num_bytes,0 , is_prealloc ? (1 <<BTRFS_ORDERED_PREALLOC ) : (1 <<BTRFS_ORDERED_NOCOW ),BTRFS_COMPRESS_NONE ); -if (ret) { +if (IS_ERR (ordered)) {if (is_prealloc) {btrfs_drop_extent_map_range (inode, cur_offset, nocow_end, false); } + ret =PTR_ERR (ordered); goto error; }
@@ -2312,8 +2319,8 @@ static noinline int run_delalloc_nocow(struct btrfs_inode *inode,* extent_clear_unlock_delalloc() in error handler * from freeing metadata of created ordered extent. */ - ret =btrfs_reloc_clone_csums (inode, cur_offset, - nocow_args.num_bytes); + ret =btrfs_reloc_clone_csums (ordered); +btrfs_put_ordered_extent (ordered); extent_clear_unlock_delalloc(inode, cur_offset, nocow_end, locked_page, EXTENT_LOCKED |
fs/btrfs/relocation.c +15 -21
@@ -4342,29 +4342,25 @@ int btrfs_recover_relocation(struct btrfs_fs_info *fs_info)* cloning checksum properly handles the nodatasum extents. * it also saves CPU time to re-calculate the checksum. */ -int btrfs_reloc_clone_csums (struct btrfs_inode *inode,u64 file_pos,u64 len) +int btrfs_reloc_clone_csums (struct btrfs_ordered_extent *ordered) { +struct btrfs_inode *inode =BTRFS_I (ordered->inode);struct btrfs_fs_info *fs_info = inode->root->fs_info; -struct btrfs_root *csum_root; -struct btrfs_ordered_sum *sums; -struct btrfs_ordered_extent *ordered; -int ret; -u64 disk_bytenr; -u64 new_bytenr; +u64 disk_bytenr = ordered->file_offset + inode->index_cnt; +struct btrfs_root *csum_root =btrfs_csum_root (fs_info, disk_bytenr);LIST_HEAD (list); +int ret; - ordered =btrfs_lookup_ordered_extent (inode, file_pos); -BUG_ON (ordered->file_offset != file_pos || ordered->num_bytes != len); - - disk_bytenr = file_pos + inode->index_cnt; - csum_root =btrfs_csum_root (fs_info, disk_bytenr); ret =btrfs_lookup_csums_list (csum_root, disk_bytenr, - disk_bytenr + len -1 , &list,0 , false); + disk_bytenr + ordered->num_bytes -1 , + &list,0 , false);if (ret) - goto out; +return ret;while (!list_empty (&list)) { - sums = list_entry(list.next, struct btrfs_ordered_sum, list); +struct btrfs_ordered_sum *sums = + list_entry(list.next, struct btrfs_ordered_sum, list); +list_del_init (&sums->list);/*
@@ -4379,14 +4375,12 @@ int btrfs_reloc_clone_csums(struct btrfs_inode *inode, u64 file_pos, u64 len)* disk_len vs real len like with real inodes since it's all * disk length. */ - new_bytenr = ordered->disk_bytenr + sums->logical - disk_bytenr; - sums->logical = new_bytenr; - + sums->logical = + ordered->disk_bytenr + sums->logical - disk_bytenr;btrfs_add_ordered_sum (ordered, sums); } -out: -btrfs_put_ordered_extent (ordered); -return ret; + +return 0 ; } int btrfs_reloc_cow_block(struct btrfs_trans_handle *trans,
fs/btrfs/relocation.h +1 -1
@@ -8,7 +8,7 @@ int btrfs_init_reloc_root(struct btrfs_trans_handle *trans, struct btrfs_root *rint btrfs_update_reloc_root (struct btrfs_trans_handle *trans,struct btrfs_root *root);int btrfs_recover_relocation (struct btrfs_fs_info *fs_info); -int btrfs_reloc_clone_csums (struct btrfs_inode *inode,u64 file_pos,u64 len); +int btrfs_reloc_clone_csums (struct btrfs_ordered_extent *ordered);int btrfs_reloc_cow_block (struct btrfs_trans_handle *trans,struct btrfs_root *root,struct extent_buffer *buf,struct extent_buffer *cow);
Signature · 1 line
2.39.2
PATCH 5/17 btrfs: pass an ordered_extent to btrfs_submit_compressed_write
In reply to Christoph Hellwigbtrfs_submit_compressed_write alwas operates on a single ordered_extent. Make that explicit by using btrfs_alloc_ordered_extent in the callers and passing the ordered_exten to btrfs_submit_compressed_write. This will help with storing and ordered_extent pointer in the btrfs_bio in subsequent patches.
--- fs/btrfs/compression.c | 29 +++++++++++++++-------------- fs/btrfs/compression.h | 5 ++--- fs/btrfs/inode.c | 21 ++++++++++----------- 3 files changed, 27 insertions(+), 28 deletions(-)
fs/btrfs/compression.c +15 -14
@@ -281,33 +281,34 @@ static void btrfs_add_compressed_bio_pages(struct compressed_bio *cb)* This also checksums the file bytes and gets things ready for * the end io hooks. */ -void btrfs_submit_compressed_write (struct btrfs_inode *inode,u64 start, -unsigned int len,u64 disk_start, -unsigned int compressed_len, -struct page **compressed_pages, -unsigned int nr_pages, -blk_opf_t write_flags, -bool writeback) +void btrfs_submit_compressed_write (struct btrfs_ordered_extent *ordered, +struct page **compressed_pages, +unsigned int nr_pages, +blk_opf_t write_flags, +bool writeback) { +struct btrfs_inode *inode =BTRFS_I (ordered->inode);struct btrfs_fs_info *fs_info = inode->root->fs_info;struct compressed_bio *cb; -ASSERT (IS_ALIGNED (start, fs_info->sectorsize) && -IS_ALIGNED (len, fs_info->sectorsize)); +ASSERT (IS_ALIGNED (ordered->file_offset, fs_info->sectorsize)); +ASSERT (IS_ALIGNED (ordered->num_bytes, fs_info->sectorsize)); - cb =alloc_compressed_bio (inode, start,REQ_OP_WRITE | write_flags, + cb =alloc_compressed_bio (inode, ordered->file_offset, +REQ_OP_WRITE | write_flags, end_compressed_bio_write); - cb->start = start; - cb->len = len; + cb->start = ordered->file_offset; + cb->len = ordered->num_bytes; cb->compressed_pages = compressed_pages; - cb->compressed_len = compressed_len; + cb->compressed_len = ordered->disk_num_bytes; cb->writeback = writeback;INIT_WORK (&cb->write_end_work, btrfs_finish_compressed_write_work); cb->nr_pages = nr_pages; - cb->bbio.bio.bi_iter.bi_sector = disk_start >>SECTOR_SHIFT ; + cb->bbio.bio.bi_iter.bi_sector = ordered->disk_bytenr >>SECTOR_SHIFT ;btrfs_add_compressed_bio_pages (cb);btrfs_submit_bio (&cb->bbio,0 ); +btrfs_put_ordered_extent (ordered); }/*
fs/btrfs/compression.h +2 -3
@@ -10,6 +10,7 @@#include "bio.h" struct btrfs_inode ; +struct btrfs_ordered_extent ;/* * We want to make sure that amount of RAM required to uncompress an extent is
@@ -86,9 +87,7 @@ int btrfs_decompress(int type, const u8 *data_in, struct page *dest_page,int btrfs_decompress_buf2page (const char *buf,u32 buf_len,struct compressed_bio *cb,u32 decompressed); -void btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start, - unsigned int len, u64 disk_start, - unsigned int compressed_len, +void btrfs_submit_compressed_write(struct btrfs_ordered_extent *ordered, struct page **compressed_pages, unsigned int nr_pages, blk_opf_t write_flags,
fs/btrfs/inode.c +10 -11
@@ -1175,6 +1175,7 @@ static int submit_one_async_extent(struct btrfs_inode *inode,struct extent_io_tree *io_tree = &inode->io_tree;struct btrfs_root *root = inode->root;struct btrfs_fs_info *fs_info = root->fs_info; +struct btrfs_ordered_extent *ordered;struct btrfs_key ins;struct page *locked_page =NULL ;struct extent_map *em;
@@ -1236,7 +1237,7 @@ static int submit_one_async_extent(struct btrfs_inode *inode, }free_extent_map (em); - ret = btrfs_add_ordered_extent(inode, start, /* file_offset */ + ordered = btrfs_alloc_ordered_extent(inode, start, /* file_offset */ async_extent->ram_size, /* num_bytes */ async_extent->ram_size, /* ram_bytes */ ins.objectid, /* disk_bytenr */
@@ -1244,8 +1245,9 @@ static int submit_one_async_extent(struct btrfs_inode *inode,0 ,/* offset */ 1 <<BTRFS_ORDERED_COMPRESSED , async_extent->compress_type); -if (ret) { +if (IS_ERR (ordered)) {btrfs_drop_extent_map_range (inode, start, end, false); + ret =PTR_ERR (ordered); goto out_free_reserve; }btrfs_dec_block_group_reservations (fs_info, ins.objectid);
@@ -1254,11 +1256,7 @@ static int submit_one_async_extent(struct btrfs_inode *inode,extent_clear_unlock_delalloc (inode, start, end,NULL ,EXTENT_LOCKED |EXTENT_DELALLOC ,PAGE_UNLOCK |PAGE_START_WRITEBACK ); - -btrfs_submit_compressed_write (inode, start,/* file_offset */ - async_extent->ram_size,/* num_bytes */ - ins.objectid,/* disk_bytenr */ - ins.offset,/* compressed_len */ +btrfs_submit_compressed_write (ordered, async_extent->pages,/* compressed_pages */ async_extent->nr_pages, async_chunk->write_flags, true);
@@ -10256,6 +10254,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,struct extent_io_tree *io_tree = &inode->io_tree;struct extent_changeset *data_reserved =NULL ;struct extent_state *cached_state =NULL ; +struct btrfs_ordered_extent *ordered;int compression;size_t orig_count;u64 start, end;
@@ -10432,14 +10431,15 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, }free_extent_map (em); - ret =btrfs_add_ordered_extent (inode, start, num_bytes, ram_bytes, + ordered =btrfs_alloc_ordered_extent (inode, start, num_bytes, ram_bytes, ins.objectid, ins.offset, encoded->unencoded_offset, (1 <<BTRFS_ORDERED_ENCODED ) | (1 <<BTRFS_ORDERED_COMPRESSED ), compression); -if (ret) { +if (IS_ERR (ordered)) {btrfs_drop_extent_map_range (inode, start, end, false); + ret =PTR_ERR (ordered); goto out_free_reserved; }btrfs_dec_block_group_reservations (fs_info, ins.objectid);
@@ -10451,8 +10451,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from,btrfs_delalloc_release_extents (inode, num_bytes); -btrfs_submit_compressed_write (inode, start, num_bytes, ins.objectid, - ins.offset, pages, nr_pages,0 , false); +btrfs_submit_compressed_write (ordered, pages, nr_pages,0 , false); ret = orig_count; goto out;
Signature · 1 line
2.39.2
PATCH 6/17 btrfs: remove btrfs_add_ordered_extent
In reply to Christoph HellwigAll callers are gone now.
--- fs/btrfs/ordered-data.c | 25 +------------------------ fs/btrfs/ordered-data.h | 4 ---- 2 files changed, 1 insertion(+), 28 deletions(-)
fs/btrfs/ordered-data.c +1 -24
@@ -279,29 +279,6 @@ struct btrfs_ordered_extent *btrfs_alloc_ordered_extent(return entry; } -/* -* Add a new btrfs_ordered_extent for the range, but drop the reference instead -* of returning it to the caller. -*/ -int btrfs_add_ordered_extent (struct btrfs_inode *inode,u64 file_offset, -u64 num_bytes,u64 ram_bytes,u64 disk_bytenr, -u64 disk_num_bytes,u64 offset,unsigned long flags, -int compress_type) -{ -struct btrfs_ordered_extent *ordered; - - ordered =btrfs_alloc_ordered_extent (inode, file_offset, num_bytes, - ram_bytes, disk_bytenr, - disk_num_bytes, offset, flags, - compress_type); - -if (IS_ERR (ordered)) -return PTR_ERR (ordered); -btrfs_put_ordered_extent (ordered); - -return 0 ; -} -/* * Add a struct btrfs_ordered_sum into the list of checksums to be inserted * when an ordered extent is finished. If the list covers more than one
@@ -579,7 +556,7 @@ void btrfs_remove_ordered_extent(struct btrfs_inode *btrfs_inode, freespace_inode =btrfs_is_free_space_inode (btrfs_inode);btrfs_lockdep_acquire (fs_info, btrfs_trans_pending_ordered); -/* This is paired with btrfs_add_ordered_extent. */ +/* This is paired with btrfs_alloc_ordered_extent. */ spin_lock (&btrfs_inode->lock);btrfs_mod_outstanding_extents (btrfs_inode,-1 );spin_unlock (&btrfs_inode->lock);
fs/btrfs/ordered-data.h +0 -4
@@ -178,10 +178,6 @@ struct btrfs_ordered_extent *btrfs_alloc_ordered_extent( u64 num_bytes, u64 ram_bytes, u64 disk_bytenr, u64 disk_num_bytes, u64 offset, unsigned long flags, int compress_type); -int btrfs_add_ordered_extent (struct btrfs_inode *inode,u64 file_offset, -u64 num_bytes,u64 ram_bytes,u64 disk_bytenr, -u64 disk_num_bytes,u64 offset,unsigned long flags, -int compress_type);void btrfs_add_ordered_sum (struct btrfs_ordered_extent *entry,struct btrfs_ordered_sum *sum); struct btrfs_ordered_extent *btrfs_lookup_ordered_extent(struct btrfs_inode *inode,
Signature · 1 line
2.39.2
PATCH 7/17 btrfs: add a is_data_bbio helper
In reply to Christoph HellwigAdd a helper to check for that a btrfs_bio has a valid inde inode, and that it is a data inode to key off all the special handling for data path checksumming. Note that this uses is_data_inode instead of REQ_META as REQ_META is only set directly before submission in submit_one_bio and we'll also want to use this helper for error handling where REQ_META isn't set yet.
--- fs/btrfs/bio.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
fs/btrfs/bio.c +9 -4
@@ -27,6 +27,12 @@ struct btrfs_failed_bio {atomic_t repair_count; }; +/* Is this a data path I/O that needs storage layer checksum and repair? */ +static inline bool is_data_bbio (struct btrfs_bio *bbio) +{ +return bbio->inode &&is_data_inode (&bbio->inode->vfs_inode); +} +/* * Initialize a btrfs_bio structure. This skips the embedded bio itself as it * is already initialized by the block layer.
@@ -312,7 +318,7 @@ static void btrfs_end_bio_work(struct work_struct *work) struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, end_io_work);/* Metadata reads are checked and repaired by the submitter. */ -if (bbio->inode && !(bbio->bio.bi_opf &REQ_META )) +if (is_data_bbio (bbio))btrfs_check_read_bio (bbio, bbio->bio.bi_private);else btrfs_orig_bbio_end_io (bbio);
@@ -346,8 +352,7 @@ static void btrfs_raid56_end_io(struct bio *bio)btrfs_bio_counter_dec (bioc->fs_info); bbio->mirror_num = bioc->mirror_num; -if (bio_op (bio) ==REQ_OP_READ && bbio->inode && - !(bbio->bio.bi_opf &REQ_META )) +if (bio_op (bio) ==REQ_OP_READ &&is_data_bbio (bbio))btrfs_check_read_bio (bbio,NULL );else btrfs_orig_bbio_end_io (bbio);
@@ -638,7 +643,7 @@ static bool btrfs_submit_chunk(struct btrfs_bio *bbio, int mirror_num)* Save the iter for the end_io handler and preload the checksums for * data reads. */ -if (bio_op (bio) ==REQ_OP_READ && inode && !(bio->bi_opf &REQ_META )) { +if (bio_op (bio) ==REQ_OP_READ &&is_data_bbio (bbio)) { bbio->saved_iter = bio->bi_iter; ret =btrfs_lookup_bio_sums (bbio);if (ret)
Signature · 1 line
2.39.2
Re: [PATCH 07/17] btrfs: add a is_data_bbio helper
Looks good,
PATCH 8/17 btrfs: open code btrfs_bio_end_io in btrfs_dio_submit_io
In reply to Christoph Hellwigbtrfs_dio_submit_io is the only place that uses btrfs_bio_end_io to end a bio that hasn't been submitted using btrfs_submit_bio yet, and this invariant will become a problem with upcoming changes to the btrfs bio layer. Just open code the assignment of bi_status and the call to btrfs_dio_end_io.
--- fs/btrfs/inode.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/btrfs/inode.c +2 -1
@@ -7827,7 +7827,8 @@ static void btrfs_dio_submit_io(const struct iomap_iter *iter, struct bio *bio, ret =btrfs_extract_ordered_extent (bbio, dio_data->ordered);if (ret) { -btrfs_bio_end_io (bbio,errno_to_blk_status (ret)); + bbio->bio.bi_status =errno_to_blk_status (ret); +btrfs_dio_end_io (bbio);return ; } }
Signature · 1 line
2.39.2
PATCH 9/17 btrfs: add an ordered_extent pointer to struct btrfs_bio
In reply to Christoph HellwigAdd a pointer to the ordered_extent to the existing union in struct btrfs_bio, so all code dealing with data write bios can just use a pointer dereference to retrieve the ordered_extent instead of doing multiple rbtree lookups per I/O.
The reference to this ordered_extent is dropped at end I/O time, which implies that an extra one must be acquired when the bio is split. This also requires moving the btrfs_extract_ordered_extent call into btrfs_split_bio so that the invariant of always having a valid ordered_extent reference for the btrfs_bio is kept.
--- fs/btrfs/bio.c | 42 ++++++++++++++++++++++++++++++++++++++---- fs/btrfs/bio.h | 9 +++------ fs/btrfs/compression.c | 2 +- fs/btrfs/extent_io.c | 2 +- fs/btrfs/inode.c | 8 +++++--- 5 files changed, 48 insertions(+), 15 deletions(-)
fs/btrfs/bio.c +38 -4
@@ -33,6 +33,11 @@ static inline bool is_data_bbio(struct btrfs_bio *bbio)return bbio->inode &&is_data_inode (&bbio->inode->vfs_inode); } +static bool bbio_has_ordered_extent (struct btrfs_bio *bbio) +{ +return is_data_bbio (bbio) &&btrfs_op (&bbio->bio) ==BTRFS_MAP_WRITE ; +} +/* * Initialize a btrfs_bio structure. This skips the embedded bio itself as it * is already initialized by the block layer.
@@ -88,11 +93,40 @@ static struct btrfs_bio *btrfs_split_bio(struct btrfs_fs_info *fs_info, bbio->inode = orig_bbio->inode; bbio->file_offset = orig_bbio->file_offset; orig_bbio->file_offset += map_length; - +if (bbio_has_ordered_extent (bbio)) { +refcount_inc (&orig_bbio->ordered->refs); + bbio->ordered = orig_bbio->ordered; + }atomic_inc (&orig_bbio->pending_ios);return bbio; } +/* Free a bio that was never submitted to the underlying device */ +static void btrfs_cleanup_bio (struct btrfs_bio *bbio) +{ +if (bbio_has_ordered_extent (bbio)) +btrfs_put_ordered_extent (bbio->ordered); +bio_put (&bbio->bio); +} + +static void __btrfs_bio_end_io (struct btrfs_bio *bbio) +{ +if (bbio_has_ordered_extent (bbio)) { +struct btrfs_ordered_extent *ordered = bbio->ordered; + + bbio->end_io (bbio); +btrfs_put_ordered_extent (ordered); + }else { + bbio->end_io (bbio); + } +} + +void btrfs_bio_end_io (struct btrfs_bio *bbio,blk_status_t status) +{ + bbio->bio.bi_status = status; +__btrfs_bio_end_io (bbio); +} +static void btrfs_orig_write_end_io (struct bio *bio); static void btrfs_bbio_propagate_error(struct btrfs_bio *bbio,
@@ -121,12 +155,12 @@ static void btrfs_orig_bbio_end_io(struct btrfs_bio *bbio)if (bbio->bio.bi_status)btrfs_bbio_propagate_error (bbio, orig_bbio); -bio_put (&bbio->bio); +btrfs_cleanup_bio (bbio); bbio = orig_bbio; }if (atomic_dec_and_test (&bbio->pending_ios)) - bbio->end_io (bbio); +__btrfs_bio_end_io (bbio); }static int next_repair_mirror (struct btrfs_failed_bio *fbio,int cur_mirror)
@@ -679,7 +713,7 @@ static bool btrfs_submit_chunk(struct btrfs_bio *bbio, int mirror_num) fail_put_bio:if (map_length < length) -bio_put (bio); +btrfs_cleanup_bio (bbio); fail:btrfs_bio_counter_dec (fs_info);btrfs_bio_end_io (orig_bbio, ret);
fs/btrfs/bio.h +3 -6
@@ -50,11 +50,13 @@ struct btrfs_bio {/* * For data writes: +* - ordered extent covering the bio * - pointer to the checksums for this bio * - original physical address from the allocator * (for zone append only) */ struct { +struct btrfs_ordered_extent *ordered;struct btrfs_ordered_sum *sums;u64 orig_physical; };
@@ -95,12 +97,7 @@ void btrfs_bio_init(struct btrfs_bio *bbio, struct btrfs_fs_info *fs_info,struct btrfs_bio *btrfs_bio_alloc (unsigned int nr_vecs,blk_opf_t opf,struct btrfs_fs_info *fs_info,btrfs_bio_end_io_t end_io,void *private); - -static inline void btrfs_bio_end_io (struct btrfs_bio *bbio,blk_status_t status) -{ - bbio->bio.bi_status = status; - bbio->end_io (bbio); -} +void btrfs_bio_end_io (struct btrfs_bio *bbio,blk_status_t status);/* Submit using blkcg_punt_bio_submit. */ #define REQ_BTRFS_CGROUP_PUNT REQ_FS_PRIVATE
fs/btrfs/compression.c +1 -1
@@ -305,10 +305,10 @@ void btrfs_submit_compressed_write(struct btrfs_ordered_extent *ordered,INIT_WORK (&cb->write_end_work, btrfs_finish_compressed_write_work); cb->nr_pages = nr_pages; cb->bbio.bio.bi_iter.bi_sector = ordered->disk_bytenr >>SECTOR_SHIFT ; + cb->bbio.ordered = ordered;btrfs_add_compressed_bio_pages (cb);btrfs_submit_bio (&cb->bbio,0 ); -btrfs_put_ordered_extent (ordered); }/*
fs/btrfs/extent_io.c +1 -1
@@ -806,7 +806,7 @@ static void alloc_new_bio(struct btrfs_inode *inode, bio_ctrl->len_to_oe_boundary =min_t (u32,U32_MAX , ordered->file_offset + ordered->disk_num_bytes - file_offset); -btrfs_put_ordered_extent (ordered); + bbio->ordered = ordered; }/*
fs/btrfs/inode.c +5 -3
@@ -2704,8 +2704,11 @@ static int btrfs_extract_ordered_extent(struct btrfs_bio *bbio,return -EINVAL ;/* No need to split if the ordered extent covers the entire bio. */ -if (ordered->disk_num_bytes == len) +if (ordered->disk_num_bytes == len) { +refcount_inc (&ordered->refs); + bbio->ordered = ordered;return 0 ; + }/* * Don't split the extent_map for NOCOW extents, as we're writing into
@@ -2722,8 +2725,7 @@ static int btrfs_extract_ordered_extent(struct btrfs_bio *bbio, new =btrfs_split_ordered_extent (ordered, len);if (IS_ERR (new))return PTR_ERR (new); -btrfs_put_ordered_extent (new); - + bbio->ordered = new;return 0 ; }
Signature · 1 line
2.39.2
PATCH 10/17 btrfs: use bbio->ordered in btrfs_csum_one_bio
In reply to Christoph HellwigUse the ordered_extent pointer in the btrfs_bio instead of looking it up manually.
--- fs/btrfs/file-item.c | 21 +-------------------- 1 file changed, 1 insertion(+), 20 deletions(-)
fs/btrfs/file-item.c +1 -20
@@ -721,13 +721,12 @@ int btrfs_lookup_csums_bitmap(struct btrfs_root *root, u64 start, u64 end,*/ blk_status_t btrfs_csum_one_bio (struct btrfs_bio *bbio) { +struct btrfs_ordered_extent *ordered = bbio->ordered;struct btrfs_inode *inode = bbio->inode;struct btrfs_fs_info *fs_info = inode->root->fs_info;SHASH_DESC_ON_STACK (shash, fs_info->csum_shash);struct bio *bio = &bbio->bio; -u64 offset = bbio->file_offset;struct btrfs_ordered_sum *sums; -struct btrfs_ordered_extent *ordered =NULL ;char *data;struct bvec_iter iter;struct bio_vec bvec;
@@ -753,22 +752,6 @@ blk_status_t btrfs_csum_one_bio(struct btrfs_bio *bbio) shash->tfm = fs_info->csum_shash;bio_for_each_segment (bvec, bio, iter) { -if (!ordered) { - ordered =btrfs_lookup_ordered_extent (inode, offset); -/* -* The bio range is not covered by any ordered extent, -* must be a code logic error. -*/ -if (unlikely (!ordered)) { -WARN (1 ,KERN_WARNING -"no ordered extent for root %llu ino %llu offset %llu\n" , - inode->root->root_key.objectid, -btrfs_ino (inode), offset); -kvfree (sums); -return BLK_STS_IOERR ; - } - } - blockcount =BTRFS_BYTES_TO_BLKS (fs_info, bvec.bv_len + fs_info->sectorsize -1 );
@@ -781,14 +764,12 @@ blk_status_t btrfs_csum_one_bio(struct btrfs_bio *bbio) sums->sums + index);kunmap_local (data); index += fs_info->csum_size; - offset += fs_info->sectorsize; } } bbio->sums = sums;btrfs_add_ordered_sum (ordered, sums); -btrfs_put_ordered_extent (ordered);return 0 ; }
Signature · 1 line
2.39.2
PATCH 11/17 btrfs: factor out a can_finish_ordered_extent helper
In reply to Christoph HellwigFactor out a helper from btrfs_mark_ordered_io_finished that does the actual per-ordered_extent work to check if we want to schedule an I/O completion. This new helper will later be used complete an ordered_extent without first doing a lookup.
--- fs/btrfs/ordered-data.c | 103 ++++++++++++++++++++++------------------ 1 file changed, 58 insertions(+), 45 deletions(-)
fs/btrfs/ordered-data.c +58 -45
@@ -303,6 +303,63 @@ static void finish_ordered_fn(struct btrfs_work *work)btrfs_finish_ordered_io (ordered_extent); } +static bool can_finish_ordered_extent (struct btrfs_ordered_extent *ordered, +struct page *page,u64 file_offset, +u64 len,bool uptodate) +{ +struct btrfs_inode *inode =BTRFS_I (ordered->inode); +struct btrfs_fs_info *fs_info = inode->root->fs_info; + +lockdep_assert_held (&inode->ordered_tree.lock); + +if (page) { +ASSERT (page->mapping); +ASSERT (page_offset (page) <= file_offset); +ASSERT (file_offset + len <=page_offset (page) +PAGE_SIZE ); + +/* +* Ordered (Private2) bit indicates whether we still +* have pending io unfinished for the ordered extent. +* +* If there's no such bit, we need to skip to next range. +*/ +if (!btrfs_page_test_ordered (fs_info, page, file_offset, len)) +return false; +btrfs_page_clear_ordered (fs_info, page, file_offset, len); + } + +/* Now we're fine to update the accounting */ +if (WARN_ON_ONCE (len > ordered->bytes_left)) { +btrfs_crit (fs_info, +"bad ordered extent accounting, root=%llu ino=%llu OE offset=%llu OE len=%llu to_dec=%llu left=%llu" , + inode->root->root_key.objectid, +btrfs_ino (inode), + ordered->file_offset, + ordered->num_bytes, + len, + ordered->bytes_left); + ordered->bytes_left =0 ; + }else { + ordered->bytes_left -= len; + } + +if (!uptodate) +set_bit (BTRFS_ORDERED_IOERR , &ordered->flags); + +if (ordered->bytes_left) +return false; + +/* +* All the IO of the ordered extent is finished, we need to queue +* the finish_func to be executed. +*/ +set_bit (BTRFS_ORDERED_IO_DONE , &ordered->flags); +cond_wake_up (&ordered->wait); +refcount_inc (&ordered->refs); +trace_btrfs_ordered_extent_mark_finished (inode, ordered); +return true; +} +/* * Mark all ordered extents io inside the specified range finished. *
@@ -333,10 +390,6 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,else wq = fs_info->endio_write_workers; -if (page) -ASSERT (page->mapping &&page_offset (page) <= file_offset && - file_offset + num_bytes <=page_offset (page) +PAGE_SIZE ); -spin_lock_irqsave (&tree->lock, flags);while (cur < file_offset + num_bytes) {u64 entry_end;
@@ -389,47 +442,7 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,ASSERT (end +1 - cur <U32_MAX ); len = end +1 - cur; -if (page) { -/* -* Ordered (Private2) bit indicates whether we still -* have pending io unfinished for the ordered extent. -* -* If there's no such bit, we need to skip to next range. -*/ -if (!btrfs_page_test_ordered (fs_info, page, cur, len)) { - cur += len; -continue ; - } -btrfs_page_clear_ordered (fs_info, page, cur, len); - } - -/* Now we're fine to update the accounting */ -if (unlikely (len > entry->bytes_left)) { -WARN_ON (1 ); -btrfs_crit (fs_info, -"bad ordered extent accounting, root=%llu ino=%llu OE offset=%llu OE len=%llu to_dec=%u left=%llu" , - inode->root->root_key.objectid, -btrfs_ino (inode), - entry->file_offset, - entry->num_bytes, - len, entry->bytes_left); - entry->bytes_left =0 ; - }else { - entry->bytes_left -= len; - } - -if (!uptodate) -set_bit (BTRFS_ORDERED_IOERR , &entry->flags); - -/* -* All the IO of the ordered extent is finished, we need to queue -* the finish_func to be executed. -*/ -if (entry->bytes_left ==0 ) { -set_bit (BTRFS_ORDERED_IO_DONE , &entry->flags); -cond_wake_up (&entry->wait); -refcount_inc (&entry->refs); -trace_btrfs_ordered_extent_mark_finished (inode, entry); +if (can_finish_ordered_extent (entry, page, cur, len, uptodate)) {spin_unlock_irqrestore (&tree->lock, flags);btrfs_init_work (&entry->work, finish_ordered_fn,NULL ,NULL );btrfs_queue_work (wq, &entry->work);
Signature · 1 line
2.39.2
PATCH 12/17 btrfs: factor out a btrfs_queue_ordered_fn helper
In reply to Christoph HellwigFactor out a helper to queue up an ordered_extent completion in a work queue. This new helper will later be used complete an ordered_extent without first doing a lookup.
--- fs/btrfs/ordered-data.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
fs/btrfs/ordered-data.c +12 -9
@@ -360,6 +360,17 @@ static bool can_finish_ordered_extent(struct btrfs_ordered_extent *ordered,return true; } +static void btrfs_queue_ordered_fn (struct btrfs_ordered_extent *ordered) +{ +struct btrfs_inode *inode =BTRFS_I (ordered->inode); +struct btrfs_fs_info *fs_info = inode->root->fs_info; +struct btrfs_workqueue *wq =btrfs_is_free_space_inode (inode) ? + fs_info->endio_freespace_worker : fs_info->endio_write_workers; + +btrfs_init_work (&ordered->work, finish_ordered_fn,NULL ,NULL ); +btrfs_queue_work (wq, &ordered->work); +} +/* * Mark all ordered extents io inside the specified range finished. *
@@ -378,18 +389,11 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode, u64 num_bytes, bool uptodate) {struct btrfs_ordered_inode_tree *tree = &inode->ordered_tree; -struct btrfs_fs_info *fs_info = inode->root->fs_info; -struct btrfs_workqueue *wq;struct rb_node *node;struct btrfs_ordered_extent *entry =NULL ;unsigned long flags;u64 cur = file_offset; -if (btrfs_is_free_space_inode (inode)) - wq = fs_info->endio_freespace_worker; -else - wq = fs_info->endio_write_workers; -spin_lock_irqsave (&tree->lock, flags);while (cur < file_offset + num_bytes) {u64 entry_end;
@@ -444,8 +448,7 @@ void btrfs_mark_ordered_io_finished(struct btrfs_inode *inode,if (can_finish_ordered_extent (entry, page, cur, len, uptodate)) {spin_unlock_irqrestore (&tree->lock, flags); -btrfs_init_work (&entry->work, finish_ordered_fn,NULL ,NULL ); -btrfs_queue_work (wq, &entry->work); +btrfs_queue_ordered_fn (entry);spin_lock_irqsave (&tree->lock, flags); } cur += len;
Signature · 1 line
2.39.2
PATCH 13/17 btrfs: add a btrfs_finish_ordered_extent helper
In reply to Christoph HellwigAdd a helper to complete an ordered_extent without first doing a lookup.
--- fs/btrfs/ordered-data.c | 20 ++++++++++++++++++++ fs/btrfs/ordered-data.h | 3 +++ include/trace/events/btrfs.h | 29 +++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+)
fs/btrfs/ordered-data.c +20 -0
@@ -371,6 +371,26 @@ static void btrfs_queue_ordered_fn(struct btrfs_ordered_extent *ordered)btrfs_queue_work (wq, &ordered->work); } +bool btrfs_finish_ordered_extent (struct btrfs_ordered_extent *ordered, +struct page *page,u64 file_offset,u64 len, +bool uptodate) +{ +struct btrfs_inode *inode =BTRFS_I (ordered->inode); +unsigned long flags; +bool ret; + +trace_btrfs_finish_ordered_extent (inode, file_offset, len, uptodate); + +spin_lock_irqsave (&inode->ordered_tree.lock, flags); + ret =can_finish_ordered_extent (ordered, page, file_offset, len, + uptodate); +spin_unlock_irqrestore (&inode->ordered_tree.lock, flags); + +if (ret) +btrfs_queue_ordered_fn (ordered); +return ret; +} +/* * Mark all ordered extents io inside the specified range finished. *
fs/btrfs/ordered-data.h +3 -0
@@ -167,6 +167,9 @@ int btrfs_finish_ordered_io(struct btrfs_ordered_extent *ordered_extent);void btrfs_put_ordered_extent (struct btrfs_ordered_extent *entry);void btrfs_remove_ordered_extent (struct btrfs_inode *btrfs_inode,struct btrfs_ordered_extent *entry); +bool btrfs_finish_ordered_extent (struct btrfs_ordered_extent *ordered, +struct page *page,u64 file_offset,u64 len, +bool uptodate);void btrfs_mark_ordered_io_finished (struct btrfs_inode *inode,struct page *page,u64 file_offset,u64 num_bytes,bool uptodate);
include/trace/events/btrfs.h +29 -0
@@ -661,6 +661,35 @@ DEFINE_EVENT(btrfs__ordered_extent, btrfs_ordered_extent_mark_finished,TP_ARGS (inode, ordered) ); +TRACE_EVENT (btrfs_finish_ordered_extent, + + TP_PROTO(const struct btrfs_inode *inode, u64 start, u64 len, + int uptodate), + +TP_ARGS (inode, start, len, uptodate), + +TP_STRUCT__entry_btrfs ( + __field( u64, ino ) + __field( u64, start ) + __field( u64, len ) + __field( int, uptodate ) +__field ( u64, root_objectid ) + ), + +TP_fast_assign_btrfs (inode->root->fs_info, + __entry->ino = btrfs_ino(inode); + __entry->start = start; + __entry->len = len; + __entry->uptodate = uptodate; + __entry->root_objectid = inode->root->root_key.objectid; + ), + +TP_printk_btrfs ("root=%llu(%s) ino=%llu start=%llu len=%llu uptodate=%d" , +show_root_type (__entry->root_objectid), + __entry->ino, __entry->start, + __entry->len, __entry->uptodate) +); + DECLARE_EVENT_CLASS(btrfs__writepage, TP_PROTO(const struct page *page, const struct inode *inode,
Signature · 1 line
2.39.2
Re: [PATCH 13/17] btrfs: add a btrfs_finish_ordered_extent helper
Looks good,
PATCH 14/17 btrfs: open code end_extent_writepage in end_bio_extent_writepage
In reply to Christoph HellwigThis prepares for switching to more efficient ordered_extent processing and already removes the forth and back conversion from len to end back to len.
--- fs/btrfs/extent_io.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
fs/btrfs/extent_io.c +9 -8
@@ -505,8 +505,6 @@ static void end_bio_extent_writepage(struct btrfs_bio *bbio)struct bio *bio = &bbio->bio;int error =blk_status_to_errno (bio->bi_status);struct bio_vec *bvec; -u64 start; -u64 end;struct bvec_iter_all iter_all;ASSERT (!bio_flagged (bio,BIO_CLONED ));
@@ -515,6 +513,8 @@ static void end_bio_extent_writepage(struct btrfs_bio *bbio)struct inode *inode = page->mapping->host;struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);const u32 sectorsize = fs_info->sectorsize; +u64 start =page_offset (page) + bvec->bv_offset; +u32 len = bvec->bv_len;/* Our read/write should always be sector aligned. */ if (!IS_ALIGNED (bvec->bv_offset, sectorsize))
@@ -526,12 +526,13 @@ static void end_bio_extent_writepage(struct btrfs_bio *bbio)"incomplete page write with offset %u and length %u" , bvec->bv_offset, bvec->bv_len); - start =page_offset (page) + bvec->bv_offset; - end = start + bvec->bv_len -1 ; - -end_extent_writepage (page, error, start, end); - -btrfs_page_clear_writeback (fs_info, page, start, bvec->bv_len); +btrfs_writepage_endio_finish_ordered (BTRFS_I (inode), page, start, + start + len -1 , !error); +if (error) { +btrfs_page_clear_uptodate (fs_info, page, start, len); +mapping_set_error (page->mapping, error); + } +btrfs_page_clear_writeback (fs_info, page, start, len); }bio_put (bio);
Signature · 1 line
2.39.2
PATCH 15/17 btrfs: use btrfs_finish_ordered_extent to complete compressed writes
In reply to Christoph HellwigUse the btrfs_finish_ordered_extent helper to complete compressed writes using the bbio->ordered pointer instead of requiring an rbtree lookup in the otherwise equivalent btrfs_mark_ordered_io_finished called from btrfs_writepage_endio_finish_ordered.
--- fs/btrfs/compression.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-)
fs/btrfs/compression.c +2 -7
@@ -226,13 +226,8 @@ static void btrfs_finish_compressed_write_work(struct work_struct *work)struct compressed_bio *cb = container_of(work, struct compressed_bio, write_end_work); -/* -* Ok, we're the last bio for this extent, step one is to call back -* into the FS and do all the end_io operations. -*/ -btrfs_writepage_endio_finish_ordered (cb->bbio.inode,NULL , - cb->start, cb->start + cb->len -1 , - cb->bbio.bio.bi_status ==BLK_STS_OK ); +btrfs_finish_ordered_extent (cb->bbio.ordered,NULL , cb->start, cb->len, + cb->bbio.bio.bi_status ==BLK_STS_OK );if (cb->writeback)end_compressed_writeback (cb);
Signature · 1 line
2.39.2
PATCH 16/17 btrfs: use btrfs_finish_ordered_extent to complete direct writes
In reply to Christoph HellwigUse the btrfs_finish_ordered_extent helper to complete compressed writes using the bbio->ordered pointer instead of requiring an rbtree lookup in the otherwise equivalent btrfs_mark_ordered_io_finished called from btrfs_writepage_endio_finish_ordered.
--- fs/btrfs/inode.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
fs/btrfs/inode.c +8 -6
@@ -7757,8 +7757,8 @@ static int btrfs_dio_iomap_end(struct inode *inode, loff_t pos, loff_t length, pos += submitted; length -= submitted;if (write) -btrfs_mark_ordered_io_finished (BTRFS_I (inode),NULL , - pos, length, false); +btrfs_finish_ordered_extent (dio_data->ordered,NULL , + pos, length, false);else unlock_extent (&BTRFS_I (inode)->io_tree, pos, pos + length -1 ,NULL );
@@ -7788,12 +7788,14 @@ static void btrfs_dio_end_io(struct btrfs_bio *bbio) dip->file_offset, dip->bytes, bio->bi_status); } -if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) -btrfs_mark_ordered_io_finished (inode,NULL , dip->file_offset, - dip->bytes, !bio->bi_status); -else +if (btrfs_op (bio) ==BTRFS_MAP_WRITE ) { +btrfs_finish_ordered_extent (bbio->ordered,NULL , + dip->file_offset, dip->bytes, + !bio->bi_status); + }else {unlock_extent (&inode->io_tree, dip->file_offset, dip->file_offset + dip->bytes -1 ,NULL ); + } bbio->bio.bi_private = bbio->private;iomap_dio_bio_end_io (bio);
Signature · 1 line
2.39.2
PATCH 17/17 btrfs: use btrfs_finish_ordered_extent to complete buffered writes
In reply to Christoph HellwigUse the btrfs_finish_ordered_extent helper to complete compressed writes using the bbio->ordered pointer instead of requiring an rbtree lookup in the otherwise equivalent btrfs_mark_ordered_io_finished called from btrfs_writepage_endio_finish_ordered.
--- fs/btrfs/extent_io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/extent_io.c +2 -2
@@ -526,8 +526,8 @@ static void end_bio_extent_writepage(struct btrfs_bio *bbio)"incomplete page write with offset %u and length %u" , bvec->bv_offset, bvec->bv_len); -btrfs_writepage_endio_finish_ordered (BTRFS_I (inode), page, start, - start + len -1 , !error); +btrfs_finish_ordered_extent (bbio->ordered, page, start, len, + !error);if (error) {btrfs_page_clear_uptodate (fs_info, page, start, len);mapping_set_error (page->mapping, error);
Signature · 1 line
2.39.2
Josef Bacik
Reviewed-byQuoted 10 lines · Christoph Hellwig
Hi all,
this series adds a pointer to the
ordered_extentto structbtrfs_bioto reduce the repeated lookups in the rbtree. For non-buffered I/Os the I/O will now never do a lookup of the ordered extent tree (other places like waiting for I/O still do). For buffered I/O there is still a lookup as the writepages code is structured in a way that makes it impossible to just pass theordered_extentdown. With some of the work from Goldwyn this should eventually become possible as well, though.
Quoted 6 lines
Changes since v1:
- rebased to the latest misc-next tree with the changes to not split ordered extents for zoned writes
- renameis_data_biotois_data_bbio
- add a newbbio_has_ordered_extenthelper
Thanks,
Josef
Quoted 10 lines · Christoph Hellwig
Hi all,
this series adds a pointer to the
ordered_extentto structbtrfs_bioto reduce the repeated lookups in the rbtree. For non-buffered I/Os the I/O will now never do a lookup of the ordered extent tree (other places like waiting for I/O still do). For buffered I/O there is still a lookup as the writepages code is structured in a way that makes it impossible to just pass theordered_extentdown. With some of the work from Goldwyn this should eventually become possible as well, though.
Quoted 5 lines
Changes since v1:
- rebased to the latest misc-next tree with the changes to not split ordered extents for zoned writes
- renameis_data_biotois_data_bbio
- add a newbbio_has_ordered_extenthelper
For the record, this was added to misc-next last week, thanks.