ext4: Use filemap_write_and_wait_range() correctly in collapse range
PATCH 1/5 ext4: Use filemap_write_and_wait_range() correctly in collapse range
Currently we're passing -1 as lend argumnet for filemap_write_and_wait_range() which is wrong since lend is signed type so it would cause some confusion and we might not write_and_wait for the entire range we're expecting to write.
Fix it by using LLONG_MAX instead.
--- fs/ext4/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len) punch_stop = (offset + len) >>EXT4_BLOCK_SIZE_BITS (sb);/* Write out all dirty pages */ - ret =filemap_write_and_wait_range (inode->i_mapping, offset,-1 ); + ret =filemap_write_and_wait_range (inode->i_mapping, offset,LLONG_MAX );if (ret)return ret;
Signature · 1 line
1.8.3.1
PATCH 2/5 ext4: fix removing status extents in ext4_collapse_range()
Currently in ext4_collapse_range() when calling ext4_es_remove_extent() to remove status extents we're passing (EXT_MAX_BLOCKS - punch_start - 1) in order to remove all extents from start of the collapse range to the end of the file. However this is wrong because we might miss the possible extent covering the last block of the file.
Fix it by removing the -1.
--- fs/ext4/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
@@ -5422,7 +5422,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)ext4_discard_preallocations (inode); ret =ext4_es_remove_extent (inode, punch_start, -EXT_MAX_BLOCKS - punch_start -1 ); +EXT_MAX_BLOCKS - punch_start);if (ret) {up_write (&EXT4_I (inode)->i_data_sem); goto out_stop;
Signature · 1 line
1.8.3.1
PATCH 3/5 ext4: No need to truncate pagecache twice in collapse range
In reply to Lukas CzernerWe're already calling truncate_pagecache_range() before we attempt to do any actual job so there is not need to truncate pagecache once more using truncate_setsize() after we're finished.
Remove truncate_setsize() and replace it just with i_size_write() note that we're holding appropriate locks.
--- fs/ext4/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
@@ -5442,7 +5442,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len) } new_size =i_size_read (inode) - len; -truncate_setsize (inode, new_size); +i_size_write (inode, new_size);EXT4_I (inode)->i_disksize = new_size;ext4_discard_preallocations (inode);
Signature · 1 line
1.8.3.1
Re: [PATCH 3/5] ext4: No need to truncate pagecache twice in collapse range
Quoted 8 lines · Lukas Czerner
We're already calling
truncate_pagecache_range()before we attempt to do any actual job so there is not need to truncate pagecache once more usingtruncate_setsize()after we're finished.Remove
truncate_setsize()and replace it just withi_size_write()note that we're holding appropriate locks.Signed-off-by Lukas Czerner <[email protected]>
Applied, with truncate_pagecache_range() changed to be truncate_pagecache() in the commit description.
Thanks!!
- Ted
PATCH 4/5 ext4: Discard preallocations after removing space
In reply to Lukas CzernerCurrently in ext4_collapse_range() and ext4_punch_hole() we're discarding preallocation twice. Once before we attempt to do any changes and second time after we're done with the changes.
While the second call to ext4_discard_preallocations() in ext4_punch_hole() case is not needed, we need to discard preallocation right after ext4_ext_remove_space() in collapse range case because in the case we had to restart a transaction in the middle of removing space we might have new preallocations created.
Remove unneeded ext4_discard_preallocations() ext4_punch_hole() and move it to the better place in ext4_collapse_range()
--- fs/ext4/extents.c | 2 +- fs/ext4/inode.c | 1 - 2 files changed, 1 insertion(+), 2 deletions(-)
fs/ext4/extents.c +1 -1
@@ -5433,6 +5433,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)up_write (&EXT4_I (inode)->i_data_sem); goto out_stop; } +ext4_discard_preallocations (inode); ret =ext4_ext_shift_extents (inode, handle, punch_stop, punch_stop - punch_start);
@@ -5445,7 +5446,6 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)i_size_write (inode, new_size);EXT4_I (inode)->i_disksize = new_size; -ext4_discard_preallocations (inode);up_write (&EXT4_I (inode)->i_data_sem);if (IS_SYNC (inode))ext4_handle_sync (handle);
fs/ext4/inode.c +0 -1
@@ -3613,7 +3613,6 @@ int ext4_punch_hole(struct inode *inode, loff_t offset, loff_t length) ret =ext4_free_hole_blocks (handle, inode, first_block, stop_block); -ext4_discard_preallocations (inode);up_write (&EXT4_I (inode)->i_data_sem);if (IS_SYNC (inode))ext4_handle_sync (handle);
Signature · 1 line
1.8.3.1
PATCH 5/5 ext4: Fix extent merging in ext4_ext_shift_path_extents()
In reply to Lukas CzernerThere is a bug in ext4_ext_shift_path_extents() where if we actually manage to merge a extent we would skip shifting the next extent. This will result in in one extent in the extent tree not being properly shifted.
This is causing failure in various xfstests tests using fsx or fsstress with collapse range support. It will also cause file system corruption which looks something like:
e2fsck 1.42.9 (4-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Inode 20 has out of order extents
(invalid logical block 3, physical block 492938, len 2)
Clear? yes
...
when running e2fsck.
It's also very easily reproducible just by running fsx without any parameters. I can usually hit the problem within a minute.
Fix it by increasing ex_start only if we're not merging the extent.
--- fs/ext4/extents.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
fs/ext4/extents.c +8 -7
@@ -5230,13 +5230,14 @@ ext4_ext_shift_path_extents(struct ext4_ext_path *path, ext4_lblk_t shift,while (ex_start <= ex_last) { ex_start->ee_block -= shift; -if (ex_start > -EXT_FIRST_EXTENT (path[depth].p_hdr)) { -if (ext4_ext_try_to_merge_right (inode, - path, ex_start -1 )) - ex_last--; - } - ex_start++; +/* Try to merge to the left. */ +if ((ex_start > +EXT_FIRST_EXTENT (path[depth].p_hdr)) && +ext4_ext_try_to_merge_right (inode, + path, ex_start -1 )) + ex_last--; +else + ex_start++; } err =ext4_ext_dirty (handle, inode, path + depth);if (err)
Signature · 1 line
1.8.3.1
Ted,
there are still endianness problems with collapse range, I kind of remember seeing some patches to fix that but I can not find those anywhere. So let me know if you have those, otherwise I'll send some.
Also since this patch set (or rather the patch #5) fixes the collapse range so that it does not fail on fsx and fsstress we can drop the fallocate mode block patches I think :)
Thanks!
-Lukas
Quoted 33 lines · Lukas Czerner
Date: Wed, 16 Apr 2014 20:32:58 +0200
From: Lukas Czerner <[email protected]>
To: [email protected]
Cc: [email protected], Lukas Czerner <[email protected]>
Subject: [PATCH 1/5] ext4: Usefilemap_write_and_wait_range()correctly in
collapse rangeCurrently we're passing -1 as lend argumnet for
filemap_write_and_wait_range()which is wrong since lend is signed type so it would cause some confusion and we might notwrite_and_waitfor the entire range we're expecting to write.Fix it by using
LLONG_MAXinstead.Signed-off-by Lukas Czerner <[email protected]>--- fs/ext4/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/ext4/extents.c +1
-1@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len) punch_stop = (offset + len) >>EXT4_BLOCK_SIZE_BITS (sb);/* Write out all dirty pages */ - ret =filemap_write_and_wait_range (inode->i_mapping, offset,-1 ); + ret =filemap_write_and_wait_range (inode->i_mapping, offset,LLONG_MAX );if (ret)return ret;
Theodore Ts'o
maintainerQuoted 4 lines · Lukáš Czerner
there are still endianness problems with collapse range, I kind of remember seeing some patches to fix that but I can not find those anywhere. So let me know if you have those, otherwise I'll send some.
Yes, those are in the tree already. I had to adjust the patch #5 slightly so it would apply as a result.
Quoted 3 lines
Also since this patch set (or rather the patch #5) fixes the collapse range so that it does not fail on fsx and fsstress we can drop the fallocate mode block patches I think :)
I'll move it to the unstable part of the tree as a debugging patch, pending the xfstests changes to make it easy to block things like the on-deck INSERT_RANGE patchset. (BTW, it would be great if the xfstests changes for INSERT_RANGE had an easy way to disable INSERT_RANGE with a single environment variable.)
The point is that otherwise, I can't let the INSERT_RANGE patches into the dev branch until it's completely regression free, or else it interferes the testing of other patches during the development cycle. Of course, if there are still problems that aren't solved by the end of the development cycle, I'll still drop the patches and hold them for the next merge window.
Thanks,
- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
So unfortunately this does not fix all the problems. Even though this fixes all the problems with collapse rage when page size == block size it still fails some tests when page size > block size so I guess we still have some work to do.
Thanks!
-Lukas
Quoted 33 lines · Lukas Czerner
Date: Wed, 16 Apr 2014 20:32:58 +0200
From: Lukas Czerner <[email protected]>
To: [email protected]
Cc: [email protected], Lukas Czerner <[email protected]>
Subject: [PATCH 1/5] ext4: Usefilemap_write_and_wait_range()correctly in
collapse rangeCurrently we're passing -1 as lend argumnet for
filemap_write_and_wait_range()which is wrong since lend is signed type so it would cause some confusion and we might notwrite_and_waitfor the entire range we're expecting to write.Fix it by using
LLONG_MAXinstead.Signed-off-by Lukas Czerner <[email protected]>--- fs/ext4/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/ext4/extents.c +1
-1@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len) punch_stop = (offset + len) >>EXT4_BLOCK_SIZE_BITS (sb);/* Write out all dirty pages */ - ret =filemap_write_and_wait_range (inode->i_mapping, offset,-1 ); + ret =filemap_write_and_wait_range (inode->i_mapping, offset,LLONG_MAX );if (ret)return ret;
Quoted 8 lines · Lukas Czerner
Currently we're passing -1 as lend argumnet for
filemap_write_and_wait_range()which is wrong since lend is signed type so it would cause some confusion and we might notwrite_and_waitfor the entire range we're expecting to write.Fix it by using
LLONG_MAXinstead.Signed-off-by Lukas Czerner <[email protected]>
Thanks, applied.
- Ted