Skip to contentSkip to conversation

ext4: Use filemap_write_and_wait_range() correctly in collapse range

PATCH Applied Lukas Czerner 2 participants 10 messages Apr 16, 2014 – Apr 18, 2014 ext4 file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/5 ext4: Use filemap_write_and_wait_range() correctly in collapse range

Currently we're passing -1 as lend argumnet for filemap_write_and_wait_range() which is wrong since lend is signed type so it would cause some confusion and we might not write_and_wait for the entire range we're expecting to write.

Fix it by using LLONG_MAX instead.

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
index ff823b7..821c1d4 100644
@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
 
 	/* Write out all dirty pages */
-	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
+	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
 	if (ret)
 		return ret;
 
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 2/5 ext4: fix removing status extents in ext4_collapse_range()

Currently in ext4_collapse_range() when calling ext4_es_remove_extent() to remove status extents we're passing (EXT_MAX_BLOCKS - punch_start - 1) in order to remove all extents from start of the collapse range to the end of the file. However this is wrong because we might miss the possible extent covering the last block of the file.

Fix it by removing the -1.

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
index 821c1d4..25ed60f 100644
@@ -5422,7 +5422,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	ext4_discard_preallocations(inode);
 
 	ret = ext4_es_remove_extent(inode, punch_start,
-				    EXT_MAX_BLOCKS - punch_start - 1);
+				    EXT_MAX_BLOCKS - punch_start);
 	if (ret) {
 		up_write(&EXT4_I(inode)->i_data_sem);
 		goto out_stop;
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 3/5 ext4: No need to truncate pagecache twice in collapse range

In reply to Lukas Czerner

We're already calling truncate_pagecache_range() before we attempt to do any actual job so there is not need to truncate pagecache once more using truncate_setsize() after we're finished.

Remove truncate_setsize() and replace it just with i_size_write() note that we're holding appropriate locks.

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
index 25ed60f..9cd762c 100644
@@ -5442,7 +5442,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	}
 
 	new_size = i_size_read(inode) - len;
-	truncate_setsize(inode, new_size);
+	i_size_write(inode, new_size);
 	EXT4_I(inode)->i_disksize = new_size;
 
 	ext4_discard_preallocations(inode);
Signature · 1 line
1.8.3.1
maintainer Applied
#3 lore raw

Re: [PATCH 3/5] ext4: No need to truncate pagecache twice in collapse range

Quoted 8 lines · Lukas Czerner

We're already calling truncate_pagecache_range() before we attempt to do any actual job so there is not need to truncate pagecache once more using truncate_setsize() after we're finished.

Remove truncate_setsize() and replace it just with i_size_write() note that we're holding appropriate locks.

Applied, with truncate_pagecache_range() changed to be truncate_pagecache() in the commit description.

Thanks!!

- Ted

#4 lore raw

PATCH 4/5 ext4: Discard preallocations after removing space

In reply to Lukas Czerner

Currently in ext4_collapse_range() and ext4_punch_hole() we're discarding preallocation twice. Once before we attempt to do any changes and second time after we're done with the changes.

While the second call to ext4_discard_preallocations() in ext4_punch_hole() case is not needed, we need to discard preallocation right after ext4_ext_remove_space() in collapse range case because in the case we had to restart a transaction in the middle of removing space we might have new preallocations created.

Remove unneeded ext4_discard_preallocations() ext4_punch_hole() and move it to the better place in ext4_collapse_range()

---
 fs/ext4/extents.c | 2 +-
 fs/ext4/inode.c   | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)
fs/ext4/extents.c +1 -1
index 9cd762c..84bb668 100644
@@ -5433,6 +5433,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 		up_write(&EXT4_I(inode)->i_data_sem);
 		goto out_stop;
 	}
+	ext4_discard_preallocations(inode);
 
 	ret = ext4_ext_shift_extents(inode, handle, punch_stop,
 				     punch_stop - punch_start);
@@ -5445,7 +5446,6 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	i_size_write(inode, new_size);
 	EXT4_I(inode)->i_disksize = new_size;
 
-	ext4_discard_preallocations(inode);
 	up_write(&EXT4_I(inode)->i_data_sem);
 	if (IS_SYNC(inode))
 		ext4_handle_sync(handle);
fs/ext4/inode.c +0 -1
index 56f1ff4..8765d49 100644
@@ -3613,7 +3613,6 @@ int ext4_punch_hole(struct inode *inode, loff_t offset, loff_t length)
 		ret = ext4_free_hole_blocks(handle, inode, first_block,
 					    stop_block);
 
-	ext4_discard_preallocations(inode);
 	up_write(&EXT4_I(inode)->i_data_sem);
 	if (IS_SYNC(inode))
 		ext4_handle_sync(handle);
Signature · 1 line
1.8.3.1
#5 lore raw

PATCH 5/5 ext4: Fix extent merging in ext4_ext_shift_path_extents()

In reply to Lukas Czerner

There is a bug in ext4_ext_shift_path_extents() where if we actually manage to merge a extent we would skip shifting the next extent. This will result in in one extent in the extent tree not being properly shifted.

This is causing failure in various xfstests tests using fsx or fsstress with collapse range support. It will also cause file system corruption which looks something like:

e2fsck 1.42.9 (4-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Inode 20 has out of order extents
(invalid logical block 3, physical block 492938, len 2)
Clear? yes
...

when running e2fsck.

It's also very easily reproducible just by running fsx without any parameters. I can usually hit the problem within a minute.

Fix it by increasing ex_start only if we're not merging the extent.

---
 fs/ext4/extents.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
fs/ext4/extents.c +8 -7
index 84bb668..5fa31cb 100644
@@ -5230,13 +5230,14 @@ ext4_ext_shift_path_extents(struct ext4_ext_path *path, ext4_lblk_t shift,
 
 			while (ex_start <= ex_last) {
 				ex_start->ee_block -= shift;
-				if (ex_start >
-					EXT_FIRST_EXTENT(path[depth].p_hdr)) {
-					if (ext4_ext_try_to_merge_right(inode,
-						path, ex_start - 1))
-						ex_last--;
-				}
-				ex_start++;
+				/* Try to merge to the left. */
+				if ((ex_start >
+				     EXT_FIRST_EXTENT(path[depth].p_hdr)) &&
+				    ext4_ext_try_to_merge_right(inode,
+							path, ex_start - 1))
+					ex_last--;
+				else
+					ex_start++;
 			}
 			err = ext4_ext_dirty(handle, inode, path + depth);
 			if (err)
Signature · 1 line
1.8.3.1
#6 lore raw
In reply to Lukas Czerner

Ted,

there are still endianness problems with collapse range, I kind of remember seeing some patches to fix that but I can not find those anywhere. So let me know if you have those, otherwise I'll send some.

Also since this patch set (or rather the patch #5) fixes the collapse range so that it does not fail on fsx and fsstress we can drop the fallocate mode block patches I think :)

Thanks!
-Lukas

Quoted 33 lines · Lukas Czerner

Date: Wed, 16 Apr 2014 20:32:58 +0200
From: Lukas Czerner <[email protected]>
To: [email protected]
Cc: [email protected], Lukas Czerner <[email protected]>
Subject: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in
collapse range

Currently we're passing -1 as lend argumnet for filemap_write_and_wait_range() which is wrong since lend is signed type so it would cause some confusion and we might not write_and_wait for the entire range we're expecting to write.

Fix it by using LLONG_MAX instead.

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
index ff823b7..821c1d4 100644
@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
 
 	/* Write out all dirty pages */
-	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
+	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
 	if (ret)
 		return ret;
 
maintainer
#7 lore raw
Quoted 4 lines · Lukáš Czerner

there are still endianness problems with collapse range, I kind of remember seeing some patches to fix that but I can not find those anywhere. So let me know if you have those, otherwise I'll send some.

Yes, those are in the tree already. I had to adjust the patch #5 slightly so it would apply as a result.

Quoted 3 lines

Also since this patch set (or rather the patch #5) fixes the collapse range so that it does not fail on fsx and fsstress we can drop the fallocate mode block patches I think :)

I'll move it to the unstable part of the tree as a debugging patch, pending the xfstests changes to make it easy to block things like the on-deck INSERT_RANGE patchset. (BTW, it would be great if the xfstests changes for INSERT_RANGE had an easy way to disable INSERT_RANGE with a single environment variable.)

The point is that otherwise, I can't let the INSERT_RANGE patches into the dev branch until it's completely regression free, or else it interferes the testing of other patches during the development cycle. Of course, if there are still problems that aren't solved by the end of the development cycle, I'll still drop the patches and hold them for the next merge window.

Thanks,

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to Lukas Czerner

So unfortunately this does not fix all the problems. Even though this fixes all the problems with collapse rage when page size == block size it still fails some tests when page size > block size so I guess we still have some work to do.

Thanks!
-Lukas

Quoted 33 lines · Lukas Czerner

Date: Wed, 16 Apr 2014 20:32:58 +0200
From: Lukas Czerner <[email protected]>
To: [email protected]
Cc: [email protected], Lukas Czerner <[email protected]>
Subject: [PATCH 1/5] ext4: Use filemap_write_and_wait_range() correctly in
collapse range

Currently we're passing -1 as lend argumnet for filemap_write_and_wait_range() which is wrong since lend is signed type so it would cause some confusion and we might not write_and_wait for the entire range we're expecting to write.

Fix it by using LLONG_MAX instead.

---
 fs/ext4/extents.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext4/extents.c +1 -1
index ff823b7..821c1d4 100644
@@ -5378,7 +5378,7 @@ int ext4_collapse_range(struct inode *inode, loff_t offset, loff_t len)
 	punch_stop = (offset + len) >> EXT4_BLOCK_SIZE_BITS(sb);
 
 	/* Write out all dirty pages */
-	ret = filemap_write_and_wait_range(inode->i_mapping, offset, -1);
+	ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
 	if (ret)
 		return ret;
 
maintainer Applied
#9 lore raw
In reply to Lukas Czerner
Quoted 8 lines · Lukas Czerner

Currently we're passing -1 as lend argumnet for filemap_write_and_wait_range() which is wrong since lend is signed type so it would cause some confusion and we might not write_and_wait for the entire range we're expecting to write.

Fix it by using LLONG_MAX instead.

Thanks, applied.

- Ted