Reduce scope of extent locking while reading
cover Reduce scope of extent locking while reading
From: Goldwyn Rodrigues <[email protected]>
Extent locking is not required for the entire read process. Once the extent map is read all information to retrieve is available in the extent map, and is cached in em_cached for later retrieval. The extent map cached is also refcounted so it would not disappear. Limit the extent locking while reading the extnet maps only. The rest is just creating bio and fetching/decompressing the data according to the extent map provided.
The only reason this was not working is because we would get EIO as the CRCs would not match (or were not committed to disk as yet) for folios that were written and released. In order to get over it, mark the extent as finished only after all folios are cleared of writeback.
I have run the xfstests on it without any deadlocks or corruptions. However, a fresh outlook on what could go wrong with a limiting the scope of locks would be good.
Goldwyn Rodrigues (2):
btrfs: clear folio writeback after completing ordered range
btrfs: take extent lock only while reading extent map
fs/btrfs/compression.c | 5 --- fs/btrfs/extent_io.c | 99 +++--------------------------------------- 2 files changed, 7 insertions(+), 97 deletions(-)
Signature · 1 line
2.46.0
PATCH 1/2 btrfs: clear folio writeback after completing ordered range
From: Goldwyn Rodrigues <[email protected]>
Mark the folio as completed writeback only after ordered range has been marked as finished. This is to make sure that the CRCs are written to disk before a disk read is performed.
--- fs/btrfs/extent_io.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/extent_io.c +2 -2
@@ -470,11 +470,11 @@ static void end_bbio_data_write(struct btrfs_bio *bbio)"incomplete page write with offset %zu and length %zu" , fi.offset, fi.length); -btrfs_finish_ordered_extent (bbio->ordered, folio, start, len, - !error);if (error)mapping_set_error (folio->mapping, error);btrfs_folio_clear_writeback (fs_info, folio, start, len); +btrfs_finish_ordered_extent (bbio->ordered, folio, start, len, + !error); }bio_put (bio);
Signature · 1 line
2.46.0
PATCH 2/2 btrfs: take extent lock only while reading extent map
In reply to Goldwyn RodriguesFrom: Goldwyn Rodrigues <[email protected]>
Limit the scope of the extent lock only while reading extent map. The extent_map is refcounted so we don't need to worry about holding the lock for it's lifecycle (in em_cached).
This removes all the processed extent structures and functions for unlocking the extent as a whole on finishing reads.
This also removes all unlock extents for page boundaries in case of a failure while performing reads.
--- fs/btrfs/compression.c | 5 --- fs/btrfs/extent_io.c | 95 +++--------------------------------------- 2 files changed, 5 insertions(+), 95 deletions(-)
fs/btrfs/compression.c +0 -5
@@ -424,11 +424,9 @@ static noinline int add_ra_bio_pages(struct inode *inode,struct extent_map *em;struct address_space *mapping = inode->i_mapping;struct extent_map_tree *em_tree; -struct extent_io_tree *tree;int sectors_missed =0 ; em_tree = &BTRFS_I (inode)->extent_tree; - tree = &BTRFS_I (inode)->io_tree;if (isize ==0 )return 0 ;
@@ -499,7 +497,6 @@ static noinline int add_ra_bio_pages(struct inode *inode, } page_end = (pg_index <<PAGE_SHIFT ) +folio_size (folio) -1 ; -lock_extent (tree, cur, page_end,NULL );read_lock (&em_tree->lock); em =lookup_extent_mapping (em_tree, cur, page_end +1 - cur);read_unlock (&em_tree->lock);
@@ -514,7 +511,6 @@ static noinline int add_ra_bio_pages(struct inode *inode, (extent_map_block_start(em) >> SECTOR_SHIFT) != orig_bio->bi_iter.bi_sector) {free_extent_map (em); -unlock_extent (tree, cur, page_end,NULL );folio_unlock (folio);folio_put (folio);break ;
@@ -534,7 +530,6 @@ static noinline int add_ra_bio_pages(struct inode *inode,if (!bio_add_folio (orig_bio, folio, add_size,offset_in_folio (folio, cur))) { -unlock_extent (tree, cur, page_end,NULL );folio_unlock (folio);folio_put (folio);break ;
fs/btrfs/extent_io.c +5 -90
185 lines not on this page · raw message
Signature · 1 line
2.46.0
Quoted 15 lines · Goldwyn Rodrigues
From: Goldwyn Rodrigues <[email protected]>
Extent locking is not required for the entire read process. Once the extent map is read all information to retrieve is available in the extent map, and is cached in
em_cachedfor later retrieval. The extent map cached is also refcounted so it would not disappear. Limit the extent locking while reading the extnet maps only. The rest is just creating bio and fetching/decompressing the data according to the extent map provided.The only reason this was not working is because we would get EIO as the CRCs would not match (or were not committed to disk as yet) for folios that were written and released. In order to get over it, mark the extent as finished only after all folios are cleared of writeback.
Quoted 7 lines
I have run the xfstests on it without any deadlocks or corruptions. However, a fresh outlook on what could go wrong with a limiting the scope of locks would be good.
Goldwyn Rodrigues (2):
btrfs: clear folio writeback after completing ordered range
btrfs: take extent lock only while reading extent map
The first patch seems relevant but it had been sent before Josef added the read locking updates so I don't know if it still needs to be merged. The second patch is covered by "btrfs: do not hold the extent lock for entire read".
Quoted 20 lines · David Sterba
Quoted 19 lines · Goldwyn Rodrigues
From: Goldwyn Rodrigues <[email protected]>
Extent locking is not required for the entire read process. Once the extent map is read all information to retrieve is available in the extent map, and is cached in
em_cachedfor later retrieval. The extent map cached is also refcounted so it would not disappear. Limit the extent locking while reading the extnet maps only. The rest is just creating bio and fetching/decompressing the data according to the extent map provided.The only reason this was not working is because we would get EIO as the CRCs would not match (or were not committed to disk as yet) for folios that were written and released. In order to get over it, mark the extent as finished only after all folios are cleared of writeback.
I have run the xfstests on it without any deadlocks or corruptions. However, a fresh outlook on what could go wrong with a limiting the scope of locks would be good.
Quoted 8 lines
Quoted 3 lines
Goldwyn Rodrigues (2):
btrfs: clear folio writeback after completing ordered range
btrfs: take extent lock only while reading extent mapThe first patch seems relevant but it had been sent before Josef added the read locking updates so I don't know if it still needs to be merged. The second patch is covered by "btrfs: do not hold the extent lock for entire read".
The first patch is not relevant anymore because of the way extent locking is used for reads. ie, a read must make sure all ordered extents are submitted beore proceeding. This was not the case before and hence I used to receive CRC errors for reads. This was a small race, where a process would writeback and invalidate pages, and a fresh read is performed simultaneously. The CRC tree was not updated and reads would fail with EIO.
The second patch is incorporated in Josef's v2 version of the patchset.
In short, ignore this series.
Signature · 1 line
Goldwyn