Btrfs: make full fsyncs always operate on the entire file again
PATCH 1/1 Btrfs: make full fsyncs always operate on the entire file again
From: Filipe Manana <[email protected]>
Commit 0a8068a3dd4294 ("btrfs: make ranged full fsyncs more efficient") made full fsyncs operate on the given range only as it assumed it was safe when using the NO_HOLES feature, since the hole detection was simplified some time ago and no longer was a source for races with ordered extent completion of adjacent file ranges.
However it's still not safe to have a full fsync only operate on the given range, because extent maps for new extents might not be present in memory due to inode eviction or extent cloning. Consider the following example:
1) We are currently at transaction N;
2) We write to the file range [0, 1Mb[;
3) Writeback finishes for the whole range and ordered extents complete, while we are still at transaction N;
4) The inode is evicted;
5) We open the file for writing, causing the inode to be loaded to memory again, which sets the 'full sync' bit on its flags. At this point the inode's list of modified extent maps is empty (figuring out which extents were created in the current transaction and were not yet logged by an fsync is expensive, that's why we set the 'full sync' bit when loading an inode);
6) We write to the file range [512Kb, 768Kb[;
7) We do a ranged fsync (such as msync()) for file range [512Kb, 768Kb[. This correctly flushes this range and logs its extent into the log tree. When the writeback started an extent map for range [512Kb, 768Kb[ was added to the inode's list of modified extents, and when the fsync() finishes logging it removes that extent map from the list of modified extent maps. This fsync also clears the 'full sync' bit;
8) We do a regular fsync() (full ranged). This fsync() ends up doing nothing because the inode's list of modified extents is empty and no other changes happened since the previous ranged fsync(), so it just returns success (0) and we end up never logging extents for the file ranges [0, 512Kb[ and [768Kb, 1Mb[.
Another scenario where this can happen is if we replace steps 2 to 4 with cloning from another file into our test file, as that sets the 'full sync' bit in our inode's flags and does not populate its list of modified extent maps.
This was causing test case generic/457 to fail sporadically when using the NO_HOLES feature, as it exercised this later case where the inode has the 'full sync' bit set and has no extent maps in memory to represent the new extents due to extent cloning.
Fix this by reverting commit 0a8068a3dd4294 ("btrfs: make ranged full fsyncs more efficient") since there is no easy way to work around it.
--- fs/btrfs/file.c | 14 +++++++++ fs/btrfs/tree-log.c | 91 ++++++++--------------------------------------------- 2 files changed, 27 insertions(+), 78 deletions(-)
fs/btrfs/file.c +14 -0
@@ -2097,6 +2097,20 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync)atomic_inc (&root->log_batch); +/* +* If the inode needs a full sync, make sure we use a full range to avoid +* log tree corruption, due to hole detection racing with ordered extent +* completion for adjacent ranges and races between logging and completion +* of ordered extents for adjance ranges - both races could lead to file +* extent items in the log with overlapping ranges. +* Do this while holding the inode lock, to avoid races with other tasks. +*/ +if (test_bit (BTRFS_INODE_NEEDS_FULL_SYNC , + &BTRFS_I (inode)->runtime_flags)) { + start =0 ; + end =LLONG_MAX ; + } +/* * Before we acquired the inode's lock, someone may have dirtied more * pages in the target range. We need to make sure that writeback for
fs/btrfs/tree-log.c +13 -78
184 lines not on this page · raw message
Signature · 1 line
2.11.0
Quoted 1 line · On Tue, Apr 07, 2020 at 11:37:44AM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
I've put the note about revert at the beginning, fixed the comment formatting in btrfs_sync_file, and updated the parameters of copy_inode_items_to_log in btrfs_log_inode so the diff against 0a8068a3dd4294 minimal.
Quoted 2 lines
Fix this by reverting commit
0a8068a3dd4294("btrfs: make ranged full fsyncs more efficient") since there is no easy way to work around it.
Oh well, you tried and that's appreciated, but the fsync matters are undoubtedly complicated. Patch added to misc-next and will go to upcoming rc, thanks.
Quoted 6 lines · David Sterba
Quoted 1 line · On Tue, Apr 07, 2020 at 11:37:44AM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
I've put the note about revert at the beginning, fixed the comment formatting in
btrfs_sync_file, and updated the parameters of
I totally missed that when looking at the diff in a console, don't know what happened.
Quoted 2 lines
copy_inode_items_to_loginbtrfs_log_inodeso the diff against0a8068a3dd4294minimal.
Sounds good.
Thanks.
Quoted 7 lines
Quoted 3 lines
Fix this by reverting commit
0a8068a3dd4294("btrfs: make ranged full fsyncs more efficient") since there is no easy way to work around it.Oh well, you tried and that's appreciated, but the fsync matters are undoubtedly complicated. Patch added to misc-next and will go to upcoming rc, thanks.