Limit scope of extent locks in btrfs_buffered_write()
cover Limit scope of extent locks in btrfs_buffered_write()
From: Goldwyn Rodrigues <[email protected]>
This is in preparation for using iomap for btrfs. It will help in keeping the extent lock limited to iomap_end() function.
The extent locks are taken for a majority of the write sequence. With Josef patches limiting the extent locks to recording extent changes, we can reduce the scope of locking of extent locks during write to only when recording the extent as delalloc. Restrict the locking to btrfs_dirty_pages(). However, the write needs to make sure that there are no pending ordered extents. So, wait for any pending ordered extents before performing the copying of data from userspace.
As for testing, it did go through a round of xfstests without any crash or failure. However, since this touches a crucial part of write, please make sure this is correct especially in terms of data corruption by overwrites/staleness.
Goldwyn Rodrigues (2):
btrfs: btrfs_has_ordered_extent() to check for ordered extent in range
btrfs: reduce scope of extent locks during buffered write
fs/btrfs/file.c | 109 ++++++---------------------------------- fs/btrfs/ordered-data.c | 11 ++++ fs/btrfs/ordered-data.h | 1 + 3 files changed, 28 insertions(+), 93 deletions(-)
Signature · 1 line
2.46.0
PATCH 1/2 btrfs: btrfs_has_ordered_extent() to check for ordered extent in range
From: Goldwyn Rodrigues <[email protected]>
A simple function which checks if there are ordered extent in range supplied.
--- fs/btrfs/ordered-data.c | 11 +++++++++++ fs/btrfs/ordered-data.h | 1 + 2 files changed, 12 insertions(+)
fs/btrfs/ordered-data.c +11 -0
@@ -1006,6 +1006,17 @@ struct btrfs_ordered_extent *btrfs_lookup_ordered_range(return entry; } +bool btrfs_has_ordered_extent (struct btrfs_inode *inode,u64 file_offset,u64 len) +{ +struct btrfs_ordered_extent *oe =NULL ; + + oe =btrfs_lookup_ordered_range (inode, file_offset, len); +if (!oe) +return false; +btrfs_put_ordered_extent (oe); +return true; +} +/* * Adds all ordered extents to the given list. The list ends up sorted by the * file_offset of the ordered extents.
fs/btrfs/ordered-data.h +1 -0
@@ -201,6 +201,7 @@ struct btrfs_ordered_extent *btrfs_lookup_ordered_range(struct btrfs_inode *inode, u64 file_offset, u64 len); +bool btrfs_has_ordered_extent (struct btrfs_inode *inode,u64 file_offset,u64 len);void btrfs_get_ordered_extents_for_logging (struct btrfs_inode *inode,struct list_head *list); u64 btrfs_wait_ordered_extents(struct btrfs_root *root, u64 nr,
Signature · 1 line
2.46.0
PATCH 2/2 btrfs: reduce scope of extent locks during buffered write
In reply to Goldwyn RodriguesFrom: Goldwyn Rodrigues <[email protected]>
Reduce the scope of locking while performing writes. The scope is limited to changing extent bits, which is in btrfs_dirty_pages(). btrfs_dirty_pages() is also called from __btrfs_write_out_cache(), and it expects extent locks held. So, perform extent locking around btrfs_dirty_pages() only.
The inode lock will insure that no other process is writing to this file, so we don't need to worry about multiple processes dirtying folios.
However, the write has to make sure that there are no ordered extents in the range specified. So, call btrfs_wait_ordered_range() before initiating the write. In case of nowait, bail if there exists an ordered range within the write range.
--- fs/btrfs/file.c | 109 +++++++----------------------------------------- 1 file changed, 16 insertions(+), 93 deletions(-)
fs/btrfs/file.c +16 -93
@@ -976,83 +976,6 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, } -/* -* This function locks the extent and properly waits for data=ordered extents -* to finish before allowing the pages to be modified if need. -* -* The return value: -* 1 - the extent is locked -* 0 - the extent is not locked, and everything is OK -* -EAGAIN - need re-prepare the pages -* the other < 0 number - Something wrong happens -*/ -static noinline int -lock_and_cleanup_extent_if_need (struct btrfs_inode *inode,struct page **pages, -size_t num_pages,loff_t pos, -size_t write_bytes, -u64 *lockstart,u64 *lockend,bool nowait, -struct extent_state **cached_state) -{ -struct btrfs_fs_info *fs_info = inode->root->fs_info; -u64 start_pos; -u64 last_pos; -int i; -int ret =0 ; - - start_pos =round_down (pos, fs_info->sectorsize); - last_pos =round_up (pos + write_bytes, fs_info->sectorsize) -1 ; - -if (start_pos < inode->vfs_inode.i_size) { -struct btrfs_ordered_extent *ordered; - -if (nowait) { -if (!try_lock_extent (&inode->io_tree, start_pos, last_pos, - cached_state)) { -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - pages[i] =NULL ; - } - -return -EAGAIN ; - } - }else { -lock_extent (&inode->io_tree, start_pos, last_pos, cached_state); - } - - ordered =btrfs_lookup_ordered_range (inode, start_pos, - last_pos - start_pos +1 ); -if (ordered && - ordered->file_offset + ordered->num_bytes > start_pos && - ordered->file_offset <= last_pos) { -unlock_extent (&inode->io_tree, start_pos, last_pos, - cached_state); -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - } -btrfs_start_ordered_extent (ordered); -btrfs_put_ordered_extent (ordered); -return -EAGAIN ; - } -if (ordered) -btrfs_put_ordered_extent (ordered); - - *lockstart = start_pos; - *lockend = last_pos; - ret =1 ; - } - -/* -* We should be called after prepare_pages() which should have locked -* all pages in the range. -*/ -for (i =0 ; i < num_pages; i++) -WARN_ON (!PageLocked (pages[i])); - -return ret; -} -/* * Check if we can do nocow write into the range [@pos, @pos + @write_bytes) *
@@ -1246,7 +1169,7 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)size_t copied;size_t dirty_sectors;size_t num_sectors; -int extents_locked; +int extents_locked = false;/* * Fault pages before locking them in prepare_pages
@@ -1310,13 +1233,19 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) } release_bytes = reserve_bytes; -again: ret =balance_dirty_pages_ratelimited_flags (inode->i_mapping, bdp_flags);if (ret) {btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes);break ; } +if (nowait && !btrfs_has_ordered_extent (BTRFS_I (inode), pos, write_bytes)) { +btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes); +break ; + }else { +btrfs_wait_ordered_range (BTRFS_I (inode), pos, write_bytes); + } +/* * This is going to setup the pages array with the number of * pages we want, so we don't really need to worry about the
@@ -1330,20 +1259,6 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)break ; } - extents_locked =lock_and_cleanup_extent_if_need ( -BTRFS_I (inode), pages, - num_pages, pos, write_bytes, &lockstart, - &lockend, nowait, &cached_state); -if (extents_locked <0 ) { -if (!nowait && extents_locked == -EAGAIN ) - goto again; - -btrfs_delalloc_release_extents (BTRFS_I (inode), - reserve_bytes); - ret = extents_locked; -break ; - } - copied =btrfs_copy_from_user (pos, write_bytes, pages, i); num_sectors =BTRFS_BYTES_TO_BLKS (fs_info, reserve_bytes);
@@ -1389,6 +1304,14 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) release_bytes =round_up (copied + sector_offset, fs_info->sectorsize); +if (pos < inode->i_size) { + lockstart =round_down (pos, fs_info->sectorsize); + lockend =round_up (pos + copied, fs_info->sectorsize); +lock_extent (&BTRFS_I (inode)->io_tree, lockstart, + lockend, &cached_state); + extents_locked = true; + } + ret =btrfs_dirty_pages (BTRFS_I (inode), pages, dirty_pages, pos, copied, &cached_state, only_release_metadata);
Signature · 1 line
2.46.0
Re: [PATCH 2/2] btrfs: reduce scope of extent locks during buffered write
Quoted 9 lines · Goldwyn Rodrigues
From: Goldwyn Rodrigues <[email protected]>
Reduce the scope of locking while performing writes. The scope is limited to changing extent bits, which is in
btrfs_dirty_pages().btrfs_dirty_pages()is also called from__btrfs_write_out_cache(), and it expects extent locks held. So, perform extent locking aroundbtrfs_dirty_pages()only.
Quoted 8 lines
The inode lock will insure that no other process is writing to this file, so we don't need to worry about multiple processes dirtying folios.
However, the write has to make sure that there are no ordered extents in the range specified. So, call
btrfs_wait_ordered_range()before initiating the write. In case of nowait, bail if there exists an ordered range within the write range.
if there exists an ordered range -> if there exists an ordered extent
Quoted 95 lines
Signed-off-by Goldwyn Rodrigues <[email protected]>--- fs/btrfs/file.c | 109 +++++++----------------------------------------- 1 file changed, 16 insertions(+), 93 deletions(-)fs/btrfs/file.c +0
-77@@ -976,83 +976,6 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, } -/* -* This function locks the extent and properly waits for data=ordered extents -* to finish before allowing the pages to be modified if need. -* -* The return value: -* 1 - the extent is locked -* 0 - the extent is not locked, and everything is OK -* -EAGAIN - need re-prepare the pages -* the other < 0 number - Something wrong happens -*/ -static noinline int -lock_and_cleanup_extent_if_need (struct btrfs_inode *inode,struct page **pages, -size_t num_pages,loff_t pos, -size_t write_bytes, -u64 *lockstart,u64 *lockend,bool nowait, -struct extent_state **cached_state) -{ -struct btrfs_fs_info *fs_info = inode->root->fs_info; -u64 start_pos; -u64 last_pos; -int i; -int ret =0 ; - - start_pos =round_down (pos, fs_info->sectorsize); - last_pos =round_up (pos + write_bytes, fs_info->sectorsize) -1 ; - -if (start_pos < inode->vfs_inode.i_size) { -struct btrfs_ordered_extent *ordered; - -if (nowait) { -if (!try_lock_extent (&inode->io_tree, start_pos, last_pos, - cached_state)) { -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - pages[i] =NULL ; - } - -return -EAGAIN ; - } - }else { -lock_extent (&inode->io_tree, start_pos, last_pos, cached_state); - } - - ordered =btrfs_lookup_ordered_range (inode, start_pos, - last_pos - start_pos +1 ); -if (ordered && - ordered->file_offset + ordered->num_bytes > start_pos && - ordered->file_offset <= last_pos) { -unlock_extent (&inode->io_tree, start_pos, last_pos, - cached_state); -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - } -btrfs_start_ordered_extent (ordered); -btrfs_put_ordered_extent (ordered); -return -EAGAIN ; - } -if (ordered) -btrfs_put_ordered_extent (ordered); - - *lockstart = start_pos; - *lockend = last_pos; - ret =1 ; - } - -/* -* We should be called after prepare_pages() which should have locked -* all pages in the range. -*/ -for (i =0 ; i < num_pages; i++) -WARN_ON (!PageLocked (pages[i])); - -return ret; -} -/* * Check if we can do nocow write into the range [@pos, @pos + @write_bytes) *
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1172
Quoted 5 lines
fs/btrfs/file.c +1
-1@@ -1246,7 +1169,7 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)size_t copied;size_t dirty_sectors;size_t num_sectors; -int extents_locked; +int extents_locked = false;
This is an int. So either assign to 0 or change the type to bool (preferable).
Quoted 5 lines
fs/btrfs/file.c +0
-0/* * Fault pages before locking them in prepare_pages @@ -1310,13 +1233,19 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) }
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1242
Quoted 10 lines
fs/btrfs/file.c +1
-1release_bytes = reserve_bytes; -again: ret =balance_dirty_pages_ratelimited_flags (inode->i_mapping, bdp_flags);if (ret) {btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes);break ; } +if (nowait && !btrfs_has_ordered_extent (BTRFS_I (inode), pos, write_bytes)) {
The logic is not correct, the ! should be removed.
I.e. if it's a nowait write and there are ordered extents, we want to stop because it would make us block waiting for ordered extents.
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1244
Quoted 2 lines
fs/btrfs/file.c +2
-0+btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes); +break ;
Before the break we also need a: ret = -EAGAIN
We should also prevent looking up ordered extents if the write starts at or beyond i_size, just like before this patch.
Quoted 29 lines
fs/btrfs/file.c +4
-14+ }else { +btrfs_wait_ordered_range (BTRFS_I (inode), pos, write_bytes); + } +/* * This is going to setup the pages array with the number of * pages we want, so we don't really need to worry about the @@ -1330,20 +1259,6 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)break ; } - extents_locked =lock_and_cleanup_extent_if_need ( -BTRFS_I (inode), pages, - num_pages, pos, write_bytes, &lockstart, - &lockend, nowait, &cached_state); -if (extents_locked <0 ) { -if (!nowait && extents_locked == -EAGAIN ) - goto again; - -btrfs_delalloc_release_extents (BTRFS_I (inode), - reserve_bytes); - ret = extents_locked; -break ; - } - copied =btrfs_copy_from_user (pos, write_bytes, pages, i); num_sectors =BTRFS_BYTES_TO_BLKS (fs_info, reserve_bytes);
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1311
Quoted 8 lines
fs/btrfs/file.c +5
-0@@ -1389,6 +1304,14 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) release_bytes =round_up (copied + sector_offset, fs_info->sectorsize); +if (pos < inode->i_size) { + lockstart =round_down (pos, fs_info->sectorsize); + lockend =round_up (pos + copied, fs_info->sectorsize); +lock_extent (&BTRFS_I (inode)->io_tree, lockstart, + lockend, &cached_state);
We should respect the nowait semantics and do a try_lock_extent() if we're a nowait write.
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1312
Quoted 1 line
fs/btrfs/file.c +1
-0+ extents_locked = true;
Same here, the type is int and not bool.
Thanks.
Quoted 9 lines
fs/btrfs/file.c +2
-1+ } + ret =btrfs_dirty_pages (BTRFS_I (inode), pages, dirty_pages, pos, copied, &cached_state, only_release_metadata); --2.46.0
Re: [PATCH 1/2] btrfs: btrfs_has_ordered_extent() to check for ordered extent in range
In reply to Goldwyn RodriguesQuoted 17 lines · Goldwyn Rodrigues
From: Goldwyn Rodrigues <[email protected]>
A simple function which checks if there are ordered extent in range supplied.
Signed-off-by Goldwyn Rodrigues <[email protected]>--- fs/btrfs/ordered-data.c | 11 +++++++++++ fs/btrfs/ordered-data.h | 1 + 2 files changed, 12 insertions(+)fs/btrfs/ordered-data.c +0
-0
fs/btrfs/ordered-data.c struct btrfs_ordered_extent *btrfs_lookup_ordered_range( · line 1011
Quoted 6 lines
fs/btrfs/ordered-data.c +3
-0@@ -1006,6 +1006,17 @@ struct btrfs_ordered_extent *btrfs_lookup_ordered_range(return entry; } +bool btrfs_has_ordered_extent (struct btrfs_inode *inode,u64 file_offset,u64 len) +{ +struct btrfs_ordered_extent *oe =NULL ;
There's no point in this assignment, as we don't ever use the variable before assigning it below.
Stuff like this has resulted in warnings from tools (and therefore reports and patches), see for example:
Thanks.
Quoted 27 lines
fs/btrfs/ordered-data.c +8
-0+ + oe =btrfs_lookup_ordered_range (inode, file_offset, len); +if (!oe) +return false; +btrfs_put_ordered_extent (oe); +return true; +} +/* * Adds all ordered extents to the given list. The list ends up sorted by the * file_offset of the ordered extents. fs/btrfs/ordered-data.h +1
-1@@ -201,6 +201,7 @@ struct btrfs_ordered_extent *btrfs_lookup_ordered_range(struct btrfs_inode *inode, u64 file_offset, u64 len); +bool btrfs_has_ordered_extent (struct btrfs_inode *inode,u64 file_offset,u64 len);void btrfs_get_ordered_extents_for_logging (struct btrfs_inode *inode,struct list_head *list); u64 btrfs_wait_ordered_extents(struct btrfs_root *root, u64 nr, --2.46.0
Re: [PATCH 2/2] btrfs: reduce scope of extent locks during buffered write
In reply to Goldwyn RodriguesQuoted 8 lines · Goldwyn Rodrigues
From: Goldwyn Rodrigues <[email protected]>
Reduce the scope of locking while performing writes. The scope is limited to changing extent bits, which is in
btrfs_dirty_pages().btrfs_dirty_pages()is also called from__btrfs_write_out_cache(), and it expects extent locks held. So, perform extent locking aroundbtrfs_dirty_pages()only.
There's one fundamental problem I missed before. Comments inlined below.
Quoted 4 lines
The inode lock will insure that no other process is writing to this file, so we don't need to worry about multiple processes dirtying folios.
Well, there's an exception: memory mapped writes, as they don't take the inode lock.
So this patch introduces a race, see below.
Quoted 114 lines
However, the write has to make sure that there are no ordered extents in the range specified. So, call
btrfs_wait_ordered_range()before initiating the write. In case of nowait, bail if there exists an ordered range within the write range.Signed-off-by Goldwyn Rodrigues <[email protected]>--- fs/btrfs/file.c | 109 +++++++----------------------------------------- 1 file changed, 16 insertions(+), 93 deletions(-)fs/btrfs/file.c +1
-79@@ -976,83 +976,6 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, } -/* -* This function locks the extent and properly waits for data=ordered extents -* to finish before allowing the pages to be modified if need. -* -* The return value: -* 1 - the extent is locked -* 0 - the extent is not locked, and everything is OK -* -EAGAIN - need re-prepare the pages -* the other < 0 number - Something wrong happens -*/ -static noinline int -lock_and_cleanup_extent_if_need (struct btrfs_inode *inode,struct page **pages, -size_t num_pages,loff_t pos, -size_t write_bytes, -u64 *lockstart,u64 *lockend,bool nowait, -struct extent_state **cached_state) -{ -struct btrfs_fs_info *fs_info = inode->root->fs_info; -u64 start_pos; -u64 last_pos; -int i; -int ret =0 ; - - start_pos =round_down (pos, fs_info->sectorsize); - last_pos =round_up (pos + write_bytes, fs_info->sectorsize) -1 ; - -if (start_pos < inode->vfs_inode.i_size) { -struct btrfs_ordered_extent *ordered; - -if (nowait) { -if (!try_lock_extent (&inode->io_tree, start_pos, last_pos, - cached_state)) { -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - pages[i] =NULL ; - } - -return -EAGAIN ; - } - }else { -lock_extent (&inode->io_tree, start_pos, last_pos, cached_state); - } - - ordered =btrfs_lookup_ordered_range (inode, start_pos, - last_pos - start_pos +1 ); -if (ordered && - ordered->file_offset + ordered->num_bytes > start_pos && - ordered->file_offset <= last_pos) { -unlock_extent (&inode->io_tree, start_pos, last_pos, - cached_state); -for (i =0 ; i < num_pages; i++) { -unlock_page (pages[i]); -put_page (pages[i]); - } -btrfs_start_ordered_extent (ordered); -btrfs_put_ordered_extent (ordered); -return -EAGAIN ; - } -if (ordered) -btrfs_put_ordered_extent (ordered); - - *lockstart = start_pos; - *lockend = last_pos; - ret =1 ; - } - -/* -* We should be called after prepare_pages() which should have locked -* all pages in the range. -*/ -for (i =0 ; i < num_pages; i++) -WARN_ON (!PageLocked (pages[i])); - -return ret; -} -/* * Check if we can do nocow write into the range [@pos, @pos + @write_bytes) * @@ -1246,7 +1169,7 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)size_t copied;size_t dirty_sectors;size_t num_sectors; -int extents_locked; +int extents_locked = false;/* * Fault pages before locking them in prepare_pages @@ -1310,13 +1233,19 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) } release_bytes = reserve_bytes; -again: ret =balance_dirty_pages_ratelimited_flags (inode->i_mapping, bdp_flags);
fs/btrfs/file.c ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) · line 1246
Quoted 10 lines
fs/btrfs/file.c +5
-0if (ret) {btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes);break ; } +if (nowait && !btrfs_has_ordered_extent (BTRFS_I (inode), pos, write_bytes)) { +btrfs_delalloc_release_extents (BTRFS_I (inode), reserve_bytes); +break ; + }else { +btrfs_wait_ordered_range (BTRFS_I (inode), pos, write_bytes);
So after waiting for ordered extents, we call prepare_pages() below, which is what gets and locks the pages.
But after this wait and before we lock the pages, a mmap write can happen, it dirties pages and if after it completes and before we lock the pages at prepare_pages(), delalloc is flushed, an ordered extent for the range is created - and we will miss it here and not wait for it to complete.
Before this patch that couldn't happen, as we always lock the extent range before locking pages.
Thanks.
Quoted 45 lines
fs/btrfs/file.c +10
-15+ } +/* * This is going to setup the pages array with the number of * pages we want, so we don't really need to worry about the @@ -1330,20 +1259,6 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i)break ; } - extents_locked =lock_and_cleanup_extent_if_need ( -BTRFS_I (inode), pages, - num_pages, pos, write_bytes, &lockstart, - &lockend, nowait, &cached_state); -if (extents_locked <0 ) { -if (!nowait && extents_locked == -EAGAIN ) - goto again; - -btrfs_delalloc_release_extents (BTRFS_I (inode), - reserve_bytes); - ret = extents_locked; -break ; - } - copied =btrfs_copy_from_user (pos, write_bytes, pages, i); num_sectors =BTRFS_BYTES_TO_BLKS (fs_info, reserve_bytes);@@ -1389,6 +1304,14 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) release_bytes =round_up (copied + sector_offset, fs_info->sectorsize); +if (pos < inode->i_size) { + lockstart =round_down (pos, fs_info->sectorsize); + lockend =round_up (pos + copied, fs_info->sectorsize); +lock_extent (&BTRFS_I (inode)->io_tree, lockstart, + lockend, &cached_state); + extents_locked = true; + } + ret =btrfs_dirty_pages (BTRFS_I (inode), pages, dirty_pages, pos, copied, &cached_state, only_release_metadata); --2.46.0