Skip to contentSkip to conversation

new new aops patchset

ANNOUNCE Nick Piggin 5 participants 43 messages Apr 2, 2007 – Apr 6, 2007

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

New stuff: patches from Mark and Steve for the cluster filesystems.

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested, but I have tried to put some effort in, so if maintainers would kindly take a look... ext3 still needs review and porting to ext4, which I hope someone will do eventually (I presume ext3 is still maintained)... does NFS have a lock_page vs lock_kernel deadlock in its commit_write? (if yes, this is fixable with the new write aops).

I chose a various smattering of weird and wonderful filesystems to try converting, and so far I can't see major problems.

A first cut at FAT and cont_prepare_write support. The FAT conversion is actually tested and seems to work quite well for various truncates and expands. Breaks reiserfs though, which might be a bit tricky to fix within the generic cont infrastructure.

Nick

#1 lore raw
Quoted 9 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 8 lines

New stuff: patches from Mark and Steve for the cluster filesystems.

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested, but I have tried to put some effort in, so if maintainers would kindly take a look... ext3 still needs review and porting to ext4, which I hope someone will do eventually (I presume ext3 is still maintained)... does NFS have a lock_page vs lock_kernel deadlock in its commit_write? (if yes, this is fixable with the new write aops).

In case of error, write_begin() is *supposed* to unlock and release the page it locked. Right ?

If so, ext3 error handling is not quite right :(
Here is the fix.

Thanks,
Badari

---
 fs/ext3/inode.c |   10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
fs/ext3/inode.c +8 -2
Index: linux-2.6.21-rc5.aop/fs/ext3/inode.c
===================================================================
@@ -1236,8 +1236,11 @@ retry:
 	*pagep = page;
 
 	handle = ext3_journal_start(inode, needed_blocks);
-	if (IS_ERR(handle))
+	if (IS_ERR(handle)) {
+		unlock_page(page);
+		page_cache_release(page);
 		return PTR_ERR(handle);
+	}
 	ret = block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
 							ext3_get_block);
 	if (ret)
@@ -1246,9 +1249,12 @@ retry:
 	if (ext3_should_journal_data(inode)) {
 		ret = walk_page_buffers(handle, page_buffers(page),
 				start, end, NULL, do_journal_get_write_access);
-		if (ret)
+		if (ret) {
 			/* fatal error, just put the handle and return */
+			unlock_page(page);
+			page_cache_release(page);
 			journal_stop(handle);
+		}
 	}
 	return ret;
 
#2 lore raw
Quoted 19 lines · Badari Pulavarty
Quoted 9 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 8 lines

New stuff: patches from Mark and Steve for the cluster filesystems.

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested, but I have tried to put some effort in, so if maintainers would kindly take a look... ext3 still needs review and porting to ext4, which I hope someone will do eventually (I presume ext3 is still maintained)... does NFS have a lock_page vs lock_kernel deadlock in its commit_write? (if yes, this is fixable with the new write aops).

Quoted 2 lines

In case of error, write_begin() is *supposed* to unlock and release the page it locked. Right ?

Thanks.

BTW. I also had another instance of that problem in the NFS conversion.

#3 lore raw
In reply to Nick Piggin
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

ext3_write_failure() conversion is NOT quite correct. Old code was returing failure of do_journal_get_write_access(), where as your changes will return journal_stop().

Thanks,
Badari

---
 fs/ext3/inode.c |    7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
fs/ext3/inode.c +5 -2
Index: linux-2.6.21-rc5.aop/fs/ext3/inode.c
===================================================================
@@ -1196,8 +1196,11 @@ skip:
 			break;
 		if (ext3_should_journal_data(mapping->host)) {
 			ret = do_journal_get_write_access(handle, bh);
-			if (ret)
-				goto skip;
+			if (ret) {
+				unlock_page(page);
+				page_cache_release(page);
+				return ret;
+			}
 		}
 	/*
 	 * block_start here becomes the first block where the current iteration
#4 lore raw
Quoted 7 lines · Badari Pulavarty
Quoted 6 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Quoted 7 lines
Quoted 2 lines

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

ext3_write_failure() conversion is NOT quite correct. Old code was returing failure of do_journal_get_write_access(), where as your changes will return journal_stop().

Good catch. This still requires a journal_stop, however?

How's this look?

--

fs/ext3/inode.c +7 -3
Index: linux-2.6/fs/ext3/inode.c
===================================================================
@@ -1167,11 +1167,13 @@ static int ext3_write_failure(struct fil
 	handle_t *handle = ext3_journal_current_handle();
 
 	if (ext3_should_writeback_data(mapping->host)) {
+skip_and_stop:
 		/* optimization: no constraints about data */
+		ret = ext3_journal_stop(handle);
 skip:
 		unlock_page(page);
 		page_cache_release(page);
-		return ext3_journal_stop(handle);
+		return ret;
 	}
 
 	from = pos & (PAGE_CACHE_SIZE - 1);
@@ -1196,8 +1198,10 @@ skip:
 			break;
 		if (ext3_should_journal_data(mapping->host)) {
 			ret = do_journal_get_write_access(handle, bh);
-			if (ret)
+			if (ret) {
+				ext3_journal_stop(handle);
 				goto skip;
+			}
 		}
 	/*
 	 * block_start here becomes the first block where the current iteration
@@ -1205,7 +1209,7 @@ skip:
 	 */
 	}
 	if (block_start <= from)
-		goto skip;
+		goto skip_and_stop;
 
 	/* commit allocated and zeroed buffers */
 	return mapping->a_ops->write_end(file, mapping, pos, len,
#5 lore raw
Quoted 12 lines · Nick Piggin
Quoted 11 lines · Badari Pulavarty
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 5 lines
Quoted 3 lines

ext3_write_failure() conversion is NOT quite correct. Old code was returing failure of do_journal_get_write_access(), where as your changes will return journal_stop().

Good catch. This still requires a journal_stop, however?

Yes.

Quoted 2 lines

How's this look?

Looks good. I am little worried about the page-lock vs journal start/stop order. I remember running into deadlock while trying to do writepages() for ordered mode. I think we are okay here. I will take a closer look.

Thanks,
Badari

Quoted 42 lines

--
Index: linux-2.6/fs/ext3/inode.c
===================================================================

fs/ext3/inode.c +7 -3
@@ -1167,11 +1167,13 @@ static int ext3_write_failure(struct fil
 	handle_t *handle = ext3_journal_current_handle();
 
 	if (ext3_should_writeback_data(mapping->host)) {
+skip_and_stop:
 		/* optimization: no constraints about data */
+		ret = ext3_journal_stop(handle);
 skip:
 		unlock_page(page);
 		page_cache_release(page);
-		return ext3_journal_stop(handle);
+		return ret;
 	}
 
 	from = pos & (PAGE_CACHE_SIZE - 1);
@@ -1196,8 +1198,10 @@ skip:
 			break;
 		if (ext3_should_journal_data(mapping->host)) {
 			ret = do_journal_get_write_access(handle, bh);
-			if (ret)
+			if (ret) {
+				ext3_journal_stop(handle);
 				goto skip;
+			}
 		}
 	/*
 	 * block_start here becomes the first block where the current iteration
@@ -1205,7 +1209,7 @@ skip:
 	 */
 	}
 	if (block_start <= from)
-		goto skip;
+		goto skip_and_stop;
 
 	/* commit allocated and zeroed buffers */
 	return mapping->a_ops->write_end(file, mapping, pos, len,
#6 lore raw
Quoted 17 lines · Badari Pulavarty
Quoted 16 lines · Nick Piggin
Quoted 14 lines · Badari Pulavarty
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

ext3_write_failure() conversion is NOT quite correct. Old code was returing failure of do_journal_get_write_access(), where as your changes will return journal_stop().

Quoted 8 lines
Quoted 1 line

Good catch. This still requires a journal_stop, however?

Yes.

Quoted 2 lines

How's this look?

Looks good. I am little worried about the page-lock vs journal start/stop order.

Yeah that did cross my mind, but the old code does it. You can just unlock before the journal_stop, it just takes a bit more code.

BTW. this is one thing the new interface is good for (ie. the filesystem completely controls the page lock).

Of course, this whole thing needs updating for the ext3_write_failure backout...

#7 lore raw
In reply to Nick Piggin
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

[root@elm3a241 linux-2.6.21-rc5]# make -j8 modules
CHK include/linux/version.h
CHK include/linux/utsrelease.h
Building modules, stage 2.
MODPOST 1281 modules
WARNING: "cont_write_begin" [fs/fat/fat.ko] undefined! make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2

Here is the fix.

Thanks,
Badari

---
 fs/buffer.c |    1 +
 1 file changed, 1 insertion(+)
fs/buffer.c +1 -0
Index: linux-2.6.21-rc5/fs/buffer.c
===================================================================
@@ -3128,6 +3128,7 @@ EXPORT_SYMBOL(block_sync_page);
 EXPORT_SYMBOL(block_truncate_page);
 EXPORT_SYMBOL(block_write_full_page);
 EXPORT_SYMBOL(cont_prepare_write);
+EXPORT_SYMBOL(cont_write_begin);
 EXPORT_SYMBOL(end_buffer_read_sync);
 EXPORT_SYMBOL(end_buffer_write_sync);
 EXPORT_SYMBOL(file_fsync);
#8 lore raw
In reply to Nick Piggin
Quoted 6 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Simple cleanup for block_write_begin().

Thanks,
Badari

---
 fs/buffer.c |    1 -
 1 file changed, 1 deletion(-)
fs/buffer.c +0 -1
Index: linux-2.6.21-rc5/fs/buffer.c
===================================================================
@@ -1977,7 +1977,6 @@ int block_write_begin(struct file *file,
 			if (pos + len > inode->i_size)
 				vmtruncate(inode, inode->i_size);
 		}
-		goto out;
 	}
 
 out:
#9 lore raw
In reply to Nick Piggin
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Baaah !! You took away ext3 -nobh option :(

Do you have plans to support nobh versions of block_write_begin/end ?

BTW, I don't see how block_write_end() can ever return < 0. If so, here is the cleanup fix for ext3 (no unnecessay checks).

Thanks,
Badari

---
 fs/ext3/inode.c |   11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
fs/ext3/inode.c +3 -8
Index: linux-2.6.21-rc5/fs/ext3/inode.c
===================================================================
@@ -1328,8 +1328,6 @@ static int ext3_ordered_write_end(struct
 		if (new_i_size > EXT3_I(inode)->i_disksize)
 			EXT3_I(inode)->i_disksize = new_i_size;
 		copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
-		if (copied < 0)
-			ret = copied;
 	}
 	ret2 = ext3_journal_stop(handle);
 	if (!ret)
@@ -1344,7 +1342,7 @@ static int ext3_writeback_write_end(stru
 {
 	handle_t *handle = ext3_journal_current_handle();
 	struct inode *inode = file->f_mapping->host;
-	int ret = 0, ret2;
+	int ret;
 	loff_t new_i_size;
 
 	new_i_size = pos + copied;
@@ -1352,12 +1350,9 @@ static int ext3_writeback_write_end(stru
 		EXT3_I(inode)->i_disksize = new_i_size;
 
 	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
-	if (copied < 0)
-		ret = copied;
 
-	ret2 = ext3_journal_stop(handle);
-	if (!ret)
-		ret = ret2;
+	ret = ext3_journal_stop(handle);
+
 	return ret ? ret : copied;
 }
 
#10 lore raw
Quoted 8 lines · Badari Pulavarty
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Baaah !! You took away ext3 -nobh option :(

Ahh, just the person I wanted to ask! ;) How useful is it, out of curiosity? What sort of users use it, and what sort of improvements do they get?

Quoted 1 line

Do you have plans to support nobh versions of block_write_begin/end ?

At the moment I'm looking at doing it another way. Having the seperate nobh path is quite annoying -- there are still bugs in it and it is simply less tested (not that the bh path is bug-free either, but it is better to be able to concentrate on one). So I hope to merge them at some point to restore that functionality.

Quoted 2 lines

BTW, I don't see how block_write_end() can ever return < 0. If so, here is the cleanup fix for ext3 (no unnecessay checks).

Shouldn't we allow for the possibility?

Quoted 42 lines
---
 fs/ext3/inode.c |   11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

Index: linux-2.6.21-rc5/fs/ext3/inode.c
===================================================================

fs/ext3/inode.c +3 -8
@@ -1328,8 +1328,6 @@ static int ext3_ordered_write_end(struct
 		if (new_i_size > EXT3_I(inode)->i_disksize)
 			EXT3_I(inode)->i_disksize = new_i_size;
 		copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
-		if (copied < 0)
-			ret = copied;
 	}
 	ret2 = ext3_journal_stop(handle);
 	if (!ret)
@@ -1344,7 +1342,7 @@ static int ext3_writeback_write_end(stru
 {
 	handle_t *handle = ext3_journal_current_handle();
 	struct inode *inode = file->f_mapping->host;
-	int ret = 0, ret2;
+	int ret;
 	loff_t new_i_size;
 
 	new_i_size = pos + copied;
@@ -1352,12 +1350,9 @@ static int ext3_writeback_write_end(stru
 		EXT3_I(inode)->i_disksize = new_i_size;
 
 	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
-	if (copied < 0)
-		ret = copied;
 
-	ret2 = ext3_journal_stop(handle);
-	if (!ret)
-		ret = ret2;
+	ret = ext3_journal_stop(handle);
+
 	return ret ? ret : copied;
 }
 
#11 lore raw
Quoted 4 lines · Nick Piggin
Quoted 3 lines · Badari Pulavarty
Quoted 2 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Baaah !! You took away ext3 -nobh option :(

Ahh, just the person I wanted to ask! ;) How useful is it, out of curiosity? What sort of users use it, and what sort of improvements do they get?

Well, at the time it helped lowmem issue on x86. Bufferheads end up using lots of lowmem and caused bunch of issues. We used it heavily on our local machines. Not sure if any customers use it (my guess is not).

I wanted to completely kill bufferhead usage from ext3. writeback mode is simple and easy. Ordered mode, need attach the buffers to transaction - which needed complete rework. So never gotten around to doing it.

Quoted 10 lines
Quoted 2 lines

Do you have plans to support nobh versions of block_write_begin/end ?

At the moment I'm looking at doing it another way. Having the seperate nobh path is quite annoying -- there are still bugs in it and it is simply less tested (not that the bh path is bug-free either, but it is better to be able to concentrate on one). So I hope to merge them at some point to restore that functionality.

Quoted 4 lines
Quoted 2 lines

BTW, I don't see how block_write_end() can ever return < 0. If so, here is the cleanup fix for ext3 (no unnecessay checks).

Shouldn't we allow for the possibility?

Well, if there is a (remote) possibility of failure - yes we should handle it. Otherwise its dead code, unnecessary checks in hotpath and makes code ugly by confusing non-possible error cases from possible ones.

Even today - ext3 code has checks to handle generic_commit_write() failures. But it never returns failure. I wanted to clean up ext3 code, but I left it for allowing for possibility.

Thanks,
Badari

#12 lore raw
Quoted 9 lines · Badari Pulavarty
Quoted 3 lines · Nick Piggin

Ahh, just the person I wanted to ask! ;) How useful is it, out of curiosity? What sort of users use it, and what sort of improvements do they get?

Well, at the time it helped lowmem issue on x86. Bufferheads end up using lots of lowmem and caused bunch of issues. We used it heavily on our local machines. Not sure if any customers use it (my guess is not).

Hmm, I think we need better reclaim for buffers as well. That might help the problem. The thing that doesn't make sense with nobh is that you deliberately throw out buffers for dirty pages, which you know will be needed again for writeout. You also want to do as little work as possible in the writeout path (ie. do it all in the dirtying path) so as to avoid deadlocks and stuff.

Anyway, I will keep nobh in mind. A port to the new aops shouldn't be impossible.

Quoted 12 lines
Quoted 11 lines

At the moment I'm looking at doing it another way. Having the seperate nobh path is quite annoying -- there are still bugs in it and it is simply less tested (not that the bh path is bug-free either, but it is better to be able to concentrate on one). So I hope to merge them at some point to restore that functionality.

Quoted 2 lines

BTW, I don't see how block_write_end() can ever return < 0. If so, here is the cleanup fix for ext3 (no unnecessay checks).

Shouldn't we allow for the possibility?

Quoted 8 lines

Well, if there is a (remote) possibility of failure - yes we should handle it. Otherwise its dead code, unnecessary checks in hotpath and makes code ugly by confusing non-possible error cases from possible ones.

Even today - ext3 code has checks to handle generic_commit_write() failures. But it never returns failure. I wanted to clean up ext3 code, but I left it for allowing for possibility.

Well I'm just thinking we should all for it in the interface from the start, so that if we ever have a need to error out there, we can without having to update all filesystems. I can't imagine a reason today though...

#13 lore raw
In reply to Nick Piggin
Quoted 4 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Sorry to send you so many silly fixes, but I though it would be easy to review individual ones.

Thanks,
Badari

---
 mm/filemap.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/filemap.c +1 -1
Index: linux-2.6.21-rc5/mm/filemap.c
===================================================================
@@ -2107,7 +2107,7 @@ int pagecache_write_end(struct file *fil
 	const struct address_space_operations *aops = mapping->a_ops;
 	int ret;
 
-	if (aops->write_begin) {
+	if (aops->write_end) {
 		ret = aops->write_end(file, mapping, pos, len, copied,
 							page, fsdata);
 	} else {
#14 lore raw
Quoted 8 lines · Badari Pulavarty
Quoted 4 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Sorry to send you so many silly fixes, but I though it would be easy to review individual ones.

On the contrary, thanks for sending them all :) In regards to this hunk, my aim is simply to mirror generic_perform_write test which gives us a free oops if write_end is NULL. I'll add a comment there (eventually this will all go away anyway).

The rest of your patches look good I think, so consider them applied unless I reply individually.

Thanks,
Nick

Quoted 24 lines

Thanks,
Badari

---
 mm/filemap.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6.21-rc5/mm/filemap.c
===================================================================

mm/filemap.c +1 -1
@@ -2107,7 +2107,7 @@ int pagecache_write_end(struct file *fil
 	const struct address_space_operations *aops = mapping->a_ops;
 	int ret;
 
-	if (aops->write_begin) {
+	if (aops->write_end) {
 		ret = aops->write_end(file, mapping, pos, len, copied,
 							page, fsdata);
 	} else {
#15 lore raw
In reply to Nick Piggin
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

ext3_write_begin() is computing "start" incorrectly.

Thanks,
Badari

---
 fs/ext3/inode.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ext3/inode.c +1 -1
Index: linux-2.6.21-rc5/fs/ext3/inode.c
===================================================================
@@ -1229,7 +1229,7 @@ static int ext3_write_begin(struct file 
 	unsigned start, end;
 
 	index = pos >> PAGE_CACHE_SHIFT;
-	start = pos * (PAGE_CACHE_SIZE - 1);
+	start = pos & (PAGE_CACHE_SIZE - 1);
 	end = start + len;
 
 retry:
#16 lore raw
Quoted 8 lines · Badari Pulavarty
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

ext3_write_begin() is computing "start" incorrectly.

Good spotting again, thanks!

Quoted 22 lines

Thanks,
Badari

---
 fs/ext3/inode.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6.21-rc5/fs/ext3/inode.c
===================================================================

fs/ext3/inode.c +1 -1
@@ -1229,7 +1229,7 @@ static int ext3_write_begin(struct file 
 	unsigned start, end;
 
 	index = pos >> PAGE_CACHE_SHIFT;
-	start = pos * (PAGE_CACHE_SIZE - 1);
+	start = pos & (PAGE_CACHE_SIZE - 1);
 	end = start + len;
 
 retry:
#17 lore raw
In reply to Nick Piggin
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

One more cleanup. "index" is unused.

BTW, I will take a shot at ext4 tomorrow.

(If you want me to roll all my fixes and send them to you,u,

5.)

1 c link

5.orig�


h2
(

and send them to you,
let me know).

Thanks,
Badari

---
 mm/filemap.c |    2 --
 1 file changed, 2 deletions(-)
mm/filemap.c +0 -2
Index: linux-2.6.21-rc5/mm/filemap.c
===================================================================
@@ -2383,14 +2383,12 @@ static ssize_t generic_perform_write(str
 
 	do {
 		struct page *page;
-		pgoff_t index;		/* Pagecache index for current page */
 		unsigned long offset;	/* Offset into pagecache page */
 		unsigned long bytes;	/* Bytes to write to page */
 		size_t copied;		/* Bytes copied from user */
 		void *fsdata;
 
 		offset = (pos & (PAGE_CACHE_SIZE - 1));
-		index = pos >> PAGE_CACHE_SHIFT;
 		bytes = min_t(unsigned long, PAGE_CACHE_SIZE - offset,
 						iov_iter_count(i));
 

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#18 lore raw
Quoted 8 lines · Badari Pulavarty
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

One more cleanup. "index" is unused.

Indeed. I wonder why gcc (at least the 4.0 I'm using) doesn't warn if a local, non volatile is only ever assigned to?

Quoted 1 line

BTW, I will take a shot at ext4 tomorrow.

Thanks, so long as you think ext3 is looking OK?

BTW. is it a known issue that ext3 fails fsx-linux? (I tried 2.6.21-rc3 IIRC, and ordered and writeback both eventually failed I think). ext2 does not.

Quoted 6 lines

(If you want me to roll all my fixes and send them to you,u,

5.)

1

c link

Quoted 6 lines

5.orig?


h2
(

^^^ I hope that wasn't due to your running the new aops patches! ;)

Quoted 3 lines

and send them to you,
let me know).

I have merged them all up, so it should be fine. Thanks very much.

Nick

Quoted 30 lines

Thanks,
Badari

---
 mm/filemap.c |    2 --
 1 file changed, 2 deletions(-)

Index: linux-2.6.21-rc5/mm/filemap.c
===================================================================

mm/filemap.c +0 -2
@@ -2383,14 +2383,12 @@ static ssize_t generic_perform_write(str
 
 	do {
 		struct page *page;
-		pgoff_t index;		/* Pagecache index for current page */
 		unsigned long offset;	/* Offset into pagecache page */
 		unsigned long bytes;	/* Bytes to write to page */
 		size_t copied;		/* Bytes copied from user */
 		void *fsdata;
 
 		offset = (pos & (PAGE_CACHE_SIZE - 1));
-		index = pos >> PAGE_CACHE_SHIFT;
 		bytes = min_t(unsigned long, PAGE_CACHE_SIZE - offset,
 						iov_iter_count(i));
 
#19 lore raw
Quoted 7 lines · Nick Piggin
Quoted 1 line

BTW, I will take a shot at ext4 tomorrow.

Thanks, so long as you think ext3 is looking OK?

BTW. is it a known issue that ext3 fails fsx-linux? (I tried 2.6.21-rc3 IIRC, and ordered and writeback both eventually failed I think). ext2 does not.

Well I just tested, and it is not fixed by the recent patch to revert ext3_prepare_failure....

Is this a known issue? Is it an fsx-linux shortcoming? It is fairly surprising because it basically makes it impossible to test ext3 changes with that nice tool :( I can submit the traces if anyone is interested, however I can reproduce in UML on an ext3 writeback filesystem with no arguments (except the filename).

#20 lore raw
Quoted 9 lines · Nick Piggin
Quoted 7 lines · Nick Piggin
Quoted 1 line

BTW, I will take a shot at ext4 tomorrow.

Thanks, so long as you think ext3 is looking OK?

BTW. is it a known issue that ext3 fails fsx-linux? (I tried 2.6.21-rc3 IIRC, and ordered and writeback both eventually failed I think). ext2 does not.

Quoted 9 lines

Well I just tested, and it is not fixed by the recent patch to revert ext3_prepare_failure....

Is this a known issue? Is it an fsx-linux shortcoming? It is fairly surprising because it basically makes it impossible to test ext3 changes with that nice tool :( I can submit the traces if anyone is interested, however I can reproduce in UML on an ext3 writeback filesystem with no arguments (except the filename).

I haven't seen an fsx failure recently. I ran 4 copies of fsx on 2.6.21-rc5 and 2.6.21-rc5+aops, without any problems for 12+ hours. What am I missing ?

Mingming, do you know of any fsx failures recently ?

Thanks,
Badari

#21 lore raw
Quoted 20 lines · Badari Pulavarty
Quoted 9 lines · Nick Piggin
Quoted 7 lines · Nick Piggin
Quoted 1 line

BTW, I will take a shot at ext4 tomorrow.

Thanks, so long as you think ext3 is looking OK?

BTW. is it a known issue that ext3 fails fsx-linux? (I tried 2.6.21-rc3 IIRC, and ordered and writeback both eventually failed I think). ext2 does not.

Quoted 9 lines

Well I just tested, and it is not fixed by the recent patch to revert ext3_prepare_failure....

Is this a known issue? Is it an fsx-linux shortcoming? It is fairly surprising because it basically makes it impossible to test ext3 changes with that nice tool :( I can submit the traces if anyone is interested, however I can reproduce in UML on an ext3 writeback filesystem with no arguments (except the filename).

Quoted 3 lines

I haven't seen an fsx failure recently. I ran 4 copies of fsx on 2.6.21-rc5 and 2.6.21-rc5+aops, without any problems for 12+ hours. What am I missing ?

Well this is in UML and mounted on loopback, so it could be a failure in the driver stack somewhere. However ext2 is completely solid, so I did suspect ext3.

In my configuration, some IO operations that would take a reasonable amount of time on a real system will complete much more quickly. There might be an issue there? Using a ramdisk or loop over tmpfs might produce something.

I'll ping you again if I can reproduce outside of UML.

#22 lore raw
In reply to Nick Piggin
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Here is the ext4 support for it. This is a simple port from ext3 code. Ran fsx without any problems :)

BTW, I never clearly understood what exactly the problem these new interfaces are solving and how :( I can dig through the archives and try to figure out. Would you care to put a small description of the *actual* problem and how these new aops are needed (vs hacking the existing methods).

Thanks,
Badari

---
 fs/ext4/inode.c |  171 +++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 113 insertions(+), 58 deletions(-)
fs/ext4/inode.c +113 -58
Index: linux-2.6.21-rc5/fs/ext4/inode.c
===================================================================
@@ -1154,23 +1154,30 @@ static int do_journal_get_write_access(h
  * This content is expected to be set to zeroes by block_prepare_write().
  * 2006/10/14  SAW
  */
-static int ext4_prepare_failure(struct file *file, struct page *page,
-				unsigned from, unsigned to)
+static int ext4_write_failure(struct file *file, struct address_space *mapping,
+				loff_t pos, unsigned len,
+				struct page *page, void *fsdata)
 {
-	struct address_space *mapping;
 	struct buffer_head *bh, *head, *next;
 	unsigned block_start, block_end;
 	unsigned blocksize;
+	unsigned from, to;
 	int ret;
 	handle_t *handle = ext4_journal_current_handle();
 
-	mapping = page->mapping;
 	if (ext4_should_writeback_data(mapping->host)) {
 		/* optimization: no constraints about data */
+skip_and_stop:
+		ret = ext4_journal_stop(handle);
 skip:
-		return ext4_journal_stop(handle);
+		unlock_page(page);
+		page_cache_release(page);
+		return ret;
 	}
 
+	from = pos & (PAGE_CACHE_SIZE - 1);
+	to = from + len;
+
 	head = page_buffers(page);
 	blocksize = head->b_size;
 	for (	bh = head, block_start = 0;
@@ -1192,7 +1199,7 @@ skip:
 			ret = do_journal_get_write_access(handle, bh);
 			if (ret) {
 				ext4_journal_stop(handle);
-				return ret;
+				goto skip;
 			}
 		}
 	/*
@@ -1201,43 +1208,64 @@ skip:
 	 */
 	}
 	if (block_start <= from)
-		goto skip;
+		goto skip_and_stop;
 
 	/* commit allocated and zeroed buffers */
-	return mapping->a_ops->commit_write(file, page, from, block_start);
+	return mapping->a_ops->write_end(file, mapping, pos, len,
+					block_start - from, page, fsdata);
 }
 
-static int ext4_prepare_write(struct file *file, struct page *page,
-			      unsigned from, unsigned to)
+static int ext4_write_begin(struct file *file, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata)
 {
-	struct inode *inode = page->mapping->host;
-	int ret, ret2;
+	struct inode *inode = mapping->host;
 	int needed_blocks = ext4_writepage_trans_blocks(inode);
+	int ret, ret2;
 	handle_t *handle;
 	int retries = 0;
+	struct page *page;
+	pgoff_t index;
+	unsigned start, end;
+
+	index = pos >> PAGE_CACHE_SHIFT;
+	start = pos * (PAGE_CACHE_SIZE - 1);
+	end = start + len;
 
 retry:
+	page = __grab_cache_page(mapping, index);
+	if (!page)
+		return -ENOMEM;
+	*pagep = page;
+
 	handle = ext4_journal_start(inode, needed_blocks);
-	if (IS_ERR(handle))
+	if (IS_ERR(handle)) {
+		unlock_page(page);
+		page_cache_release(page);
 		return PTR_ERR(handle);
-	if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
-		ret = nobh_prepare_write(page, from, to, ext4_get_block);
-	else
-		ret = block_prepare_write(page, from, to, ext4_get_block);
+	}
+	ret = block_write_begin(file, mapping, pos, len, flags, pagep, fsdata,
+							ext4_get_block);
 	if (ret)
 		goto failure;
 
 	if (ext4_should_journal_data(inode)) {
 		ret = walk_page_buffers(handle, page_buffers(page),
-				from, to, NULL, do_journal_get_write_access);
-		if (ret)
+				start, end, NULL, do_journal_get_write_access);
+		if (ret) {
 			/* fatal error, just put the handle and return */
 			ext4_journal_stop(handle);
+			unlock_page(page);
+			page_cache_release(page);
+		}
 	}
 	return ret;
 
 failure:
-	ret2 = ext4_prepare_failure(file, page, from, to);
+	ret2 = ext4_write_failure(file, mapping, pos, len, page, *fsdata);
+	/* trim off blocks (XXX: need better helpers than vmtruncate) */
+	if (pos + len > inode->i_size)
+		vmtruncate(inode, inode->i_size);
 	if (ret2 < 0)
 		return ret2;
 	if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
@@ -1251,12 +1279,12 @@ int ext4_journal_dirty_data(handle_t *ha
 	int err = jbd2_journal_dirty_data(handle, bh);
 	if (err)
 		ext4_journal_abort_handle(__FUNCTION__, __FUNCTION__,
-						bh, handle,err);
+						bh, handle, err);
 	return err;
 }
 
-/* For commit_write() in data=journal mode */
-static int commit_write_fn(handle_t *handle, struct buffer_head *bh)
+/* For write_end() in data=journal mode */
+static int write_end_fn(handle_t *handle, struct buffer_head *bh)
 {
 	if (!buffer_mapped(bh) || buffer_freed(bh))
 		return 0;
@@ -1271,78 +1299,103 @@ static int commit_write_fn(handle_t *han
  * ext4 never places buffers on inode->i_mapping->private_list.  metadata
  * buffers are managed internally.
  */
-static int ext4_ordered_commit_write(struct file *file, struct page *page,
-			     unsigned from, unsigned to)
+static int ext4_ordered_write_end(struct file *file,
+				struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata)
 {
 	handle_t *handle = ext4_journal_current_handle();
-	struct inode *inode = page->mapping->host;
+	struct inode *inode = file->f_mapping->host;
+	unsigned from, to;
 	int ret = 0, ret2;
 
+	from = pos & (PAGE_CACHE_SIZE - 1);
+	to = from + len;
+
 	ret = walk_page_buffers(handle, page_buffers(page),
 		from, to, NULL, ext4_journal_dirty_data);
 
 	if (ret == 0) {
 		/*
-		 * generic_commit_write() will run mark_inode_dirty() if i_size
+		 * block_write_end() will run mark_inode_dirty() if i_size
 		 * changes.  So let's piggyback the i_disksize mark_inode_dirty
 		 * into that.
 		 */
 		loff_t new_i_size;
 
-		new_i_size = ((loff_t)page->index << PAGE_CACHE_SHIFT) + to;
+		new_i_size = pos + copied;
 		if (new_i_size > EXT4_I(inode)->i_disksize)
 			EXT4_I(inode)->i_disksize = new_i_size;
-		ret = generic_commit_write(file, page, from, to);
+		copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
+		if (copied < 0)
+			ret = copied;
 	}
 	ret2 = ext4_journal_stop(handle);
 	if (!ret)
 		ret = ret2;
-	return ret;
+	return ret ? ret : copied;
 }
 
-static int ext4_writeback_commit_write(struct file *file, struct page *page,
-			     unsigned from, unsigned to)
+static int ext4_writeback_write_end(struct file *file,
+				struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata)
 {
 	handle_t *handle = ext4_journal_current_handle();
-	struct inode *inode = page->mapping->host;
+	struct inode *inode = file->f_mapping->host;
 	int ret = 0, ret2;
 	loff_t new_i_size;
 
-	new_i_size = ((loff_t)page->index << PAGE_CACHE_SHIFT) + to;
+	new_i_size = pos + copied;
 	if (new_i_size > EXT4_I(inode)->i_disksize)
 		EXT4_I(inode)->i_disksize = new_i_size;
 
-	if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode))
-		ret = nobh_commit_write(file, page, from, to);
-	else
-		ret = generic_commit_write(file, page, from, to);
+	copied = block_write_end(file, mapping, pos, len, copied, page, fsdata);
+	if (copied < 0)
+		ret = copied;
 
 	ret2 = ext4_journal_stop(handle);
 	if (!ret)
 		ret = ret2;
-	return ret;
+	return ret ? ret : copied;
 }
 
-static int ext4_journalled_commit_write(struct file *file,
-			struct page *page, unsigned from, unsigned to)
+static int ext4_journalled_write_end(struct file *file,
+				struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata)
 {
 	handle_t *handle = ext4_journal_current_handle();
-	struct inode *inode = page->mapping->host;
+	struct inode *inode = mapping->host;
 	int ret = 0, ret2;
 	int partial = 0;
-	loff_t pos;
+	unsigned from, to;
 
-	/*
-	 * Here we duplicate the generic_commit_write() functionality
-	 */
-	pos = ((loff_t)page->index << PAGE_CACHE_SHIFT) + to;
+	from = pos & (PAGE_CACHE_SIZE - 1);
+	to = from + len;
+
+	if (copied < len) {
+		if (PageUptodate(page))
+			copied = len;
+		else {
+			/* XXX: don't need to zero new buffers because we abort? */
+			copied = 0;
+			if (!is_handle_aborted(handle))
+				jbd2_journal_abort_handle(handle);
+			unlock_page(page);
+			page_cache_release(page);
+			goto out;
+		}
+	}
 
 	ret = walk_page_buffers(handle, page_buffers(page), from,
-				to, &partial, commit_write_fn);
+				to, &partial, write_end_fn);
 	if (!partial)
 		SetPageUptodate(page);
-	if (pos > inode->i_size)
-		i_size_write(inode, pos);
+	unlock_page(page);
+	page_cache_release(page);
+	if (pos+copied > inode->i_size)
+		i_size_write(inode, pos+copied);
 	EXT4_I(inode)->i_state |= EXT4_STATE_JDATA;
 	if (inode->i_size > EXT4_I(inode)->i_disksize) {
 		EXT4_I(inode)->i_disksize = inode->i_size;
@@ -1350,10 +1403,12 @@ static int ext4_journalled_commit_write(
 		if (!ret)
 			ret = ret2;
 	}
+
+out:
 	ret2 = ext4_journal_stop(handle);
 	if (!ret)
 		ret = ret2;
-	return ret;
+	return ret ? ret : copied;
 }
 
 /*
@@ -1611,7 +1666,7 @@ static int ext4_journalled_writepage(str
 			PAGE_CACHE_SIZE, NULL, do_journal_get_write_access);
 
 		err = walk_page_buffers(handle, page_buffers(page), 0,
-				PAGE_CACHE_SIZE, NULL, commit_write_fn);
+				PAGE_CACHE_SIZE, NULL, write_end_fn);
 		if (ret == 0)
 			ret = err;
 		EXT4_I(inode)->i_state |= EXT4_STATE_JDATA;
@@ -1771,8 +1826,8 @@ static const struct address_space_operat
 	.readpages	= ext4_readpages,
 	.writepage	= ext4_ordered_writepage,
 	.sync_page	= block_sync_page,
-	.prepare_write	= ext4_prepare_write,
-	.commit_write	= ext4_ordered_commit_write,
+	.write_begin	= ext4_write_begin,
+	.write_end	= ext4_ordered_write_end,
 	.bmap		= ext4_bmap,
 	.invalidatepage	= ext4_invalidatepage,
 	.releasepage	= ext4_releasepage,
@@ -1785,8 +1840,8 @@ static const struct address_space_operat
 	.readpages	= ext4_readpages,
 	.writepage	= ext4_writeback_writepage,
 	.sync_page	= block_sync_page,
-	.prepare_write	= ext4_prepare_write,
-	.commit_write	= ext4_writeback_commit_write,
+	.write_begin	= ext4_write_begin,
+	.write_end	= ext4_writeback_write_end,
 	.bmap		= ext4_bmap,
 	.invalidatepage	= ext4_invalidatepage,
 	.releasepage	= ext4_releasepage,
@@ -1799,8 +1854,8 @@ static const struct address_space_operat
 	.readpages	= ext4_readpages,
 	.writepage	= ext4_journalled_writepage,
 	.sync_page	= block_sync_page,
-	.prepare_write	= ext4_prepare_write,
-	.commit_write	= ext4_journalled_commit_write,
+	.write_begin	= ext4_write_begin,
+	.write_end	= ext4_journalled_write_end,
 	.set_page_dirty	= ext4_journalled_set_page_dirty,
 	.bmap		= ext4_bmap,
 	.invalidatepage	= ext4_invalidatepage,
#23 lore raw
Quoted 9 lines · Badari Pulavarty
Quoted 5 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Here is the ext4 support for it. This is a simple port from ext3 code. Ran fsx without any problems :)

Thanks. Again, I guess this needs updating for write_failure backout :(

Quoted 5 lines

BTW, I never clearly understood what exactly the problem these new interfaces are solving and how :( I can dig through the archives and try to figure out. Would you care to put a small description of the *actual* problem and how these new aops are needed (vs hacking the existing methods).

OK, so the problem from my point of view is the pagecache write deadlock:

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/2.6.21-rc5-new-aops/broken-out/mm-pagecache-write-deadlocks.patch

The problem with that fix is that it requires an extra copy (I thought long and hard about different ways to fix this without changing the aops API and without taking a performance hit).

It was pretty clear that the problem can be fixed easily and efficiently in the mm/ code if the the API can be changed a bit (eg. allow commit_write to commit a range which is smaller than the prepare_write range).

The problem with changing the API is that you break all filesystems, and out of tree filesystems. Even if we ignore the out of tree ones, the existing ones can have difficult and subtle issues that I cannot fix alone.

So I decided that the best way to go is to introduce the "slow" fix for everyone, then add these new aops. Once everyone is moved to the new aops, we remove the old ones and a lot of old cruft with them.

So at this point we can make a clean break and come up with an API that is not only capable of fixing the deadlock, but is also better suited to modern filesystems (prepare_write was intorduced in 2.2 or 2.0 IIRC, and probably suffered from deadlocks even then!).

I don't know if write_begin/ write_end is final yet (which is why we need your input), but it already does help a lot with the clustered filesystems right now.

The reason we also have the (still unused) perform_write, is that was my first attempt at a new interface. The problem with that is that it is difficult to provide generic block helpers that are usable in more complex ways. It also cannot be used for source data other than memory in iovecs. However the reason why it is still there is because I already introduced the iov_iter and infrastructure for it, and I think it is a very nice, clean interface, and can be good for high performance writes (I got positive results even with a naive ext2 implementation).

Does that answer your question?

Thanks,
Nick

#24 lore raw
In reply to Nick Piggin
Quoted 3 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Thanks again for pushing this stuff out Nick.

Fwiw, I had merge conflicts working against the latest git code from Linus - it looks like Jens pushed some splice changes (some from you). A refreshed fs-new-write-aops.patch is attached.

The ext3 patch also caused me some problems, but I just temporarily dropped it for my immediate work.

	--Mark

--
Mark Fasheh
Senior Software Developer, Oracle
[email protected]

Introduce write_begin, write_end, and perform_write aops.

These are intended to replace prepare_write and commit_write with more flexible alternatives that are also able to avoid the buffered write deadlock problems efficiently (which prepare_write is unable to do).

 Documentation/filesystems/Locking |    9 -
 Documentation/filesystems/vfs.txt |   36 +++++
 drivers/block/loop.c              |   69 +++--------
 fs/buffer.c                       |  161 +++++++++++++++++++++++---
 fs/libfs.c                        |   40 ++++++
 fs/namei.c                        |   47 +------
 fs/splice.c                       |  107 +----------------
 include/linux/buffer_head.h       |    7 +
 include/linux/fs.h                |   31 +++++
 include/linux/pagemap.h           |    2 
 mm/filemap.c                      |  229 ++++++++++++++++++++++++++++++++++----
 11 files changed, 513 insertions(+), 225 deletions(-)
include/linux/fs.h +31 -0
Index: linux-2.6.git/include/linux/fs.h
===================================================================
@@ -389,6 +389,8 @@ enum positive_aop_returns {
 	AOP_TRUNCATED_PAGE	= 0x80001,
 };
 
+#define AOP_FLAG_UNINTERRUPTIBLE	0x0001 /* will not do a short write */
+
 /*
  * oh the beauties of C type declarations.
  */
@@ -449,6 +451,17 @@ struct address_space_operations {
 	 */
 	int (*prepare_write)(struct file *, struct page *, unsigned, unsigned);
 	int (*commit_write)(struct file *, struct page *, unsigned, unsigned);
+
+	int (*write_begin)(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata);
+	int (*write_end)(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata);
+
+	ssize_t (*perform_write)(struct file *, struct address_space *,
+				struct iov_iter *, loff_t);
+
 	/* Unfortunately this kludge is needed for FIBMAP. Don't use it */
 	sector_t (*bmap)(struct address_space *, sector_t);
 	void (*invalidatepage) (struct page *, unsigned long);
@@ -463,6 +476,18 @@ struct address_space_operations {
 	int (*launder_page) (struct page *);
 };
 
+/*
+ * pagecache_write_begin/pagecache_write_end must be used by general code
+ * to write into the pagecache.
+ */
+int pagecache_write_begin(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata);
+
+int pagecache_write_end(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata);
+
 struct backing_dev_info;
 struct address_space {
 	struct inode		*host;		/* owner: inode, block_device */
@@ -1907,6 +1932,12 @@ extern int simple_prepare_write(struct f
 			unsigned offset, unsigned to);
 extern int simple_commit_write(struct file *file, struct page *page,
 				unsigned offset, unsigned to);
+extern int simple_write_begin(struct file *file, struct address_space *mapping,
+			loff_t pos, unsigned len, unsigned flags,
+			struct page **pagep, void **fsdata);
+extern int simple_write_end(struct file *file, struct address_space *mapping,
+			loff_t pos, unsigned len, unsigned copied,
+			struct page *page, void *fsdata);
 
 extern struct dentry *simple_lookup(struct inode *, struct dentry *, struct nameidata *);
 extern ssize_t generic_read_dir(struct file *, char __user *, size_t, loff_t *);
mm/filemap.c +211 -24

302 lines not on this page · raw message

fs/buffer.c +150 -30

206 lines not on this page · raw message

include/linux/buffer_head.h +7 -0
Index: linux-2.6.git/include/linux/buffer_head.h
===================================================================
@@ -202,6 +202,13 @@ void block_invalidatepage(struct page *p
 int block_write_full_page(struct page *page, get_block_t *get_block,
 				struct writeback_control *wbc);
 int block_read_full_page(struct page*, get_block_t*);
+int block_write_begin(struct file *, struct address_space *,
+				loff_t, unsigned, unsigned,
+				struct page **, void **, get_block_t*);
+int block_write_end(struct file *, struct address_space *,
+				loff_t, unsigned, unsigned,
+				struct page *, void *);
+void page_zero_new_buffers(struct page *page, unsigned from, unsigned to);
 int block_prepare_write(struct page*, unsigned, unsigned, get_block_t*);
 int cont_prepare_write(struct page*, unsigned, unsigned, get_block_t*,
 				loff_t *);
include/linux/pagemap.h +2 -0
Index: linux-2.6.git/include/linux/pagemap.h
===================================================================
@@ -85,6 +85,8 @@ unsigned find_get_pages_contig(struct ad
 unsigned find_get_pages_tag(struct address_space *mapping, pgoff_t *index,
 			int tag, unsigned int nr_pages, struct page **pages);
 
+struct page *__grab_cache_page(struct address_space *mapping, pgoff_t index);
+
 /*
  * Returns locked page at given index in given cache, creating it if needed.
  */
fs/libfs.c +44 -0

62 lines not on this page · raw message

drivers/block/loop.c +24 -45

102 lines not on this page · raw message

fs/namei.c +12 -35

65 lines not on this page · raw message

fs/splice.c +8 -62

99 lines not on this page · raw message

Documentation/filesystems/Locking +6 -3
Index: linux-2.6.git/Documentation/filesystems/Locking
===================================================================
@@ -176,15 +176,18 @@ prototypes:
 locking rules:
 	All except set_page_dirty may block
 
-			BKL	PageLocked(page)
+			BKL	PageLocked(page)	i_sem
 writepage:		no	yes, unlocks (see below)
 readpage:		no	yes, unlocks
 sync_page:		no	maybe
 writepages:		no
 set_page_dirty		no	no
 readpages:		no
-prepare_write:		no	yes
-commit_write:		no	yes
+prepare_write:		no	yes			yes
+commit_write:		no	yes			yes
+write_begin:		no	locks the page		yes
+write_end:		no	yes, unlocks		yes
+perform_write:		no	n/a			yes
 bmap:			yes
 invalidatepage:		no	yes
 releasepage:		no	yes
Documentation/filesystems/vfs.txt +36 -0
Index: linux-2.6.git/Documentation/filesystems/vfs.txt
===================================================================
@@ -534,6 +534,14 @@ struct address_space_operations {
 			struct list_head *pages, unsigned nr_pages);
 	int (*prepare_write)(struct file *, struct page *, unsigned, unsigned);
 	int (*commit_write)(struct file *, struct page *, unsigned, unsigned);
+	int (*write_begin)(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata);
+	int (*write_end)(struct file *, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned copied,
+				struct page *page, void *fsdata);
+	int (*perform_write)(struct file *, struct address_space *,
+				struct iov_iter *, loff_t);
 	sector_t (*bmap)(struct address_space *, sector_t);
 	int (*invalidatepage) (struct page *, unsigned long);
 	int (*releasepage) (struct page *, int);
@@ -629,6 +637,34 @@ struct address_space_operations {
         operations.  It should avoid returning an error if possible -
         errors should have been handled by prepare_write.
 
+  write_begin: This is intended as a replacement for prepare_write. Called
+        by the generic buffered write code to ask the filesystem to prepare
+        to write len bytes at the given offset in the file. flags is a field
+        for AOP_FLAG_xxx flags, described in include/linux/mm.h.
+
+        The filesystem must return the locked pagecache page for the caller
+        to write into.
+
+        A void * may be returned in fsdata, which then gets passed into
+        write_end.
+
+        Returns < 0 on failure, in which case all cleanup must be done and
+        write_end not called. 0 on success, in which case write_end must
+        be called.
+
+  write_end: After a successful write_begin, and data copy, write_end must
+        be called. len is the original len passed to write_begin, and copied
+        is the amount that was able to be copied (they must be equal if
+        write_begin was called with intr == 0).
+
+        The filesystem must take care of unlocking the page and dropping its
+        refcount, and updating i_size.
+
+        Returns < 0 on failure, otherwise the number of bytes (<= 'copied')
+        that were able to be copied into the file.
+
+  perform_write:
+
   bmap: called by the VFS to map a logical block offset within object to
   	physical block number. This method is used by the FIBMAP
   	ioctl and for working with swap-files.  To be able to swap to
#25 lore raw
Quoted 5 lines · Mark Fasheh
Quoted 3 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Quoted 8 lines

Thanks again for pushing this stuff out Nick.

Fwiw, I had merge conflicts working against the latest git code from Linus - it looks like Jens pushed some splice changes (some from you). A refreshed fs-new-write-aops.patch is attached.

The ext3 patch also caused me some problems, but I just temporarily dropped it for my immediate work.

Could you elaborate on issues ext3 caused ? Its pretty stable for me (after bunch of simple fixes, I posted earlier). Anything I need to be aware ?

Thanks,
Badari

#26 lore raw
Quoted 3 lines · Badari Pulavarty

Could you elaborate on issues ext3 caused ? Its pretty stable for me (after bunch of simple fixes, I posted earlier). Anything I need to be aware ?

Oh, I meant merge issues - that's all. And that's just because I was trying to be "fancy" by merging the stuff on top of a git branch I'm working on ;)

	--Mark

--
Mark Fasheh
Senior Software Developer, Oracle
[email protected]

#27 lore raw
Quoted 8 lines · Mark Fasheh
Quoted 3 lines · Badari Pulavarty

Could you elaborate on issues ext3 caused ? Its pretty stable for me (after bunch of simple fixes, I posted earlier). Anything I need to be aware ?

Oh, I meant merge issues - that's all. And that's just because I was trying to be "fancy" by merging the stuff on top of a git branch I'm working on ;)

	--Mark

Okay, Thanks.

#28 lore raw
In reply to Mark Fasheh

On Wed, 2007-04-04 at 15:10 -0700, Mark Fasheh wrote:
...

Quoted 41 lines
mm/filemap.c +41 -0
+int pagecache_write_begin(struct file *file, struct address_space *mapping,
+				loff_t pos, unsigned len, unsigned flags,
+				struct page **pagep, void **fsdata)
+{
+	const struct address_space_operations *aops = mapping->a_ops;
+
+	if (aops->write_begin) {
+		return aops->write_begin(file, mapping, pos, len, flags,
+							pagep, fsdata);
+	} else {
+		int ret;
+		pgoff_t index = pos >> PAGE_CACHE_SHIFT;
+		unsigned offset = pos & (PAGE_CACHE_SIZE - 1);
+		struct inode *inode = mapping->host;
+		struct page *page;
+again:
+		page = __grab_cache_page(mapping, index);
+		*pagep = page;
+		if (!page)
+			return -ENOMEM;
+
+		if (flags & AOP_FLAG_UNINTERRUPTIBLE && !PageUptodate(page)) {
+			/*
+			 * There is no way to resolve a short write situation
+			 * for a !Uptodate page (except by double copying in
+			 * the caller done by generic_perform_write_2copy).
+			 *
+			 * Instead, we have to bring it uptodate here.
+			 */
+			ret = aops->readpage(file, page);
+			page_cache_release(page);
+			if (ret) {
+				if (ret == AOP_TRUNCATED_PAGE)
+					goto again;
+				return ret;
+			}
+			goto again;
+		}
+
+		ret = aops->prepare_write(file, page, offset, offset+len);
+		if (ret) {

mm/filemap.c inline int generic_write_checks(struct f · line 2102

Quoted 10 lines
mm/filemap.c +10 -0
+			if (ret != AOP_TRUNCATED_PAGE)
+				unlock_page(page);
+			page_cache_release(page);
+			if (pos + len > inode->i_size)
+				vmtruncate(inode, inode->i_size);
+			if (ret == AOP_TRUNCATED_PAGE)
+				goto again;
+		}
+		return ret;
+	}

Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of them are switched to have write_begin()/write_end(), why do we need this code to handle AOP_TRUNCATED_PAGE (in the else part) ? Can't we just cleanup/nuke all the AOP_TRUNCATED_PAGE handling ?

Dumb question ?

Thanks,
Badari

#29 lore raw
Quoted 5 lines · Badari Pulavarty

Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of them are switched to have write_begin()/write_end(), why do we need this code to handle AOP_TRUNCATED_PAGE (in the else part) ? Can't we just cleanup/nuke all the AOP_TRUNCATED_PAGE handling ?

We don't - I'm pretty sure that fs-no-AOP_TRUNCATED_PAGE.patch gets rid of them.

	--Mark

--
Mark Fasheh
Senior Software Developer, Oracle
[email protected]

#30 lore raw
Quoted 10 lines · Mark Fasheh
Quoted 5 lines · Badari Pulavarty

Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of them are switched to have write_begin()/write_end(), why do we need this code to handle AOP_TRUNCATED_PAGE (in the else part) ? Can't we just cleanup/nuke all the AOP_TRUNCATED_PAGE handling ?

We don't - I'm pretty sure that fs-no-AOP_TRUNCATED_PAGE.patch gets rid of them.

	--Mark

It didn't, completely get rid of them :(

Thanks,
Badari

#31 lore raw
Quoted 8 lines · Badari Pulavarty
Quoted 7 lines · Mark Fasheh
Quoted 5 lines · Badari Pulavarty

Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of them are switched to have write_begin()/write_end(), why do we need this code to handle AOP_TRUNCATED_PAGE (in the else part) ? Can't we just cleanup/nuke all the AOP_TRUNCATED_PAGE handling ?

Quoted 5 lines
Quoted 3 lines

We don't - I'm pretty sure that fs-no-AOP_TRUNCATED_PAGE.patch gets rid of them.

	--Mark

It didn't, completely get rid of them :(

->readpage can still return AOP_TRUNCATED_PAGE. Were there any from prepare_write or commit_write still around?

#32 lore raw
Quoted 9 lines · Nick Piggin
Quoted 8 lines · Badari Pulavarty
Quoted 7 lines · Mark Fasheh
Quoted 5 lines · Badari Pulavarty

Hmm.. Okay, only filesystems that could return AOP_TRUNCATED_PAGE are ocf2 and gfs2. Now that both of them are switched to have write_begin()/write_end(), why do we need this code to handle AOP_TRUNCATED_PAGE (in the else part) ? Can't we just cleanup/nuke all the AOP_TRUNCATED_PAGE handling ?

Quoted 10 lines
Quoted 5 lines
Quoted 3 lines

We don't - I'm pretty sure that fs-no-AOP_TRUNCATED_PAGE.patch gets rid of them.

	--Mark

It didn't, completely get rid of them :(

->readpage can still return AOP_TRUNCATED_PAGE. Were there any from prepare_write or commit_write still around?

Not a big deal. But trying to understand it better.

int pagecache_write_begin()
{

        if (aops->write_begin) {
                return aops->write_begin(file, mapping, pos, len, flags,
                                                        pagep, fsdata);
        } else {
                .....
                ret = aops->readpage(file, page);
                page_cache_release(page);
                if (ret) {
			if (ret == AOP_TRUNCATED_PAGE)
                                        goto again;
                        return ret;
                }
                goto again;
		
	      ....
	}
}

filesystems (ocfs2, gfs2) which can return AOP_TRUNCATED_PAGE for prepare_write or readpage would never come to this case. They have write_begin() method set. Isn't it ? Why this check ?

Thanks,
Badari

#33 lore raw
Quoted 31 lines · Badari Pulavarty
Quoted 5 lines · Nick Piggin

->readpage can still return AOP_TRUNCATED_PAGE. Were there any from prepare_write or commit_write still around?

Not a big deal. But trying to understand it better.

int pagecache_write_begin()
{

        if (aops->write_begin) {
                return aops->write_begin(file, mapping, pos, len, flags,
                                                        pagep, fsdata);
        } else {
                .....
                ret = aops->readpage(file, page);
                page_cache_release(page);
                if (ret) {
			if (ret == AOP_TRUNCATED_PAGE)
                                        goto again;
                        return ret;
                }
                goto again;
		
	      ....
	}
}
Quoted 3 lines

filesystems (ocfs2, gfs2) which can return AOP_TRUNCATED_PAGE for prepare_write or readpage would never come to this case. They have write_begin() method set. Isn't it ? Why this check ?

Ah you're right. In that case, I'll replace that with a comment ;)

Thanks!

#34 lore raw
In reply to Mark Fasheh
Quoted 10 lines · Mark Fasheh
Quoted 3 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Thanks again for pushing this stuff out Nick.

Fwiw, I had merge conflicts working against the latest git code from Linus - it looks like Jens pushed some splice changes (some from you). A refreshed fs-new-write-aops.patch is attached.

Thanks, added.

Quoted 2 lines

The ext3 patch also caused me some problems, but I just temporarily dropped it for my immediate work.

Yeah, good idea ;)

#35 lore raw
In reply to Nick Piggin
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

......

Quoted 1 line

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested,

Failed to compile in UDF and reiser for me, but no doubt you know that already. Don't have time to look at why - I just disabled them so I get some QA done on the XFS and core changes.

On a related note - what's the rules for a perform_write() implementation? I noticed that wasn't documented with write_begin and write_end and I don't see any other filesystem implementing it yet....

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#36 lore raw
Quoted 9 lines · David Chinner
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 7 lines

......

Quoted 1 line

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested,

Failed to compile in UDF and reiser for me, but no doubt you know that already. Don't have time to look at why - I just disabled them so I get some QA done on the XFS and core changes.

Thanks, yeah the "cont_prepare_write" rework is a little intrusive and affects reiserfs in ways I haven't had time to try fixing yet.

Quoted 3 lines

On a related note - what's the rules for a perform_write() implementation? I noticed that wasn't documented with write_begin and write_end and I don't see any other filesystem implementing it yet....

Ah, so it isn't, thanks I'll document it. (today I'm looking at doing a simple_perform_write and perhaps another easy one, which should also help filesystem maintainers to have a reference).

Thanks,
Nick

#37 lore raw
Quoted 7 lines · Nick Piggin
Quoted 3 lines

On a related note - what's the rules for a perform_write() implementation? I noticed that wasn't documented with write_begin and write_end and I don't see any other filesystem implementing it yet....

Ah, so it isn't, thanks I'll document it. (today I'm looking at doing a simple_perform_write and perhaps another easy one, which should also help filesystem maintainers to have a reference).

I think ->perform_write should go away. It's really just duplicating ->write for the !O_DIRECT case. Instead we should factor the remaining bits of generic_file_aio_write into nice helpers so that people that don't want to use the generic part can build their own ->write from these pieces.

#38 lore raw
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Nick Piggin
Quoted 3 lines

On a related note - what's the rules for a perform_write() implementation? I noticed that wasn't documented with write_begin and write_end and I don't see any other filesystem implementing it yet....

Quoted 8 lines
Quoted 3 lines

Ah, so it isn't, thanks I'll document it. (today I'm looking at doing a simple_perform_write and perhaps another easy one, which should also help filesystem maintainers to have a reference).

I think ->perform_write should go away. It's really just duplicating ->write for the !O_DIRECT case. Instead we should factor the remaining bits of generic_file_aio_write into nice helpers so that people that don't want to use the generic part can build their own ->write from these pieces.

I probably don't disagree with you. If we use the iov_iter further down the stack (which I was going to do as a cleanup following this patchset), then you're right that it isn't too painful...

Anyway, I'll leave it in the next release: I've converted a few users, so that will give people a better idea of whether they agree/disagree or don't care ;)

#39 lore raw
In reply to David Chinner
Quoted 9 lines · David Chinner
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 7 lines

......

Quoted 1 line

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested,

Failed to compile in UDF and reiser for me, but no doubt you know that already. Don't have time to look at why - I just disabled them so I get some QA done on the XFS and core changes.

Nick - the XFSQA randholes test locks up. As you can probably guess it creates holey files and does stuff to them ;)

Both pdflush and a write doing an a blocked waiting for a page to become unlocked. They are both waiting on teh same page (0xa0007fffffe8bd68).

[0]kdb> btt 0xe0000039ec2b8000
Stack traceback for pid 113
0xe0000039ec2b8000 113 19 0 3 D 0xe0000039ec2b82f0 pdflush
0xa0000001008bcdb0 schedule+0x9b0

        args (0xe0000039ec207020, 0xe0000039ec2070a8, 0xa0000001001f3fa0, 0x206, 0xa000000100eee5a0)
0xa0000001008bf4b0 io_schedule+0x50
        args (0xe00000b0030258d4, 0xa0000001001338c0, 0x205, 0xa000000100eee5a0)
0xa0000001001338c0 sync_page+0x100
        args (0xa0007fffffe8bd68, 0xa0000001008bfd20, 0x58e, 0xa000000100eee5a0)
0xa0000001008bfd20 __wait_on_bit_lock+0x160
        args (0xe00000b005905e20, 0xe0000039ec2bfb40, 0xa000000100accf30, 0x2, 0xe0000039ec2bfb90)
0xa000000100133760 __lock_page+0x180
        args (0xa0007fffffe8bd68, 0xa000000100143b30, 0x917, 0x917)
0xa000000100143b30 generic_writepages+0x450
        args (0xe00000b47b2a65f0, 0xe0000039ec2bfcd8, 0xa0007fffffe8bd68, 0xe0000039ec2bfbe8, 0x2)
0xa00000010044be70 xfs_vm_writepages+0x50
        args (0xe00000b47b2a65f0, 0xe0000039ec2bfcd8, 0xa000000100144030, 0x308, 0xa0000001000d1870)
0xa000000100144030 do_writepages+0xb0
        args (0xe00000b47b2a65f0, 0xe0000039ec2bfcd8, 0xe0000039ec2bfd10, 0xa0000001001eb8e0, 0x691)
0xa0000001001eb8e0 __writeback_single_inode+0x140
        args (0xe00000b47b2a64c8, 0xe0000039ec2bfcd8, 0x7, 0xe00000b47b2a6700, 0xe00000b47b2a65d8)
0xa0000001001ecbd0 sync_sb_inodes+0x490
        args (0xe0000039eb96fb48, 0xe0000039ec2bfcd8, 0xe0000039ec207140, 0xe00000b47b2a64c8, 0xe0000039eb96fc30)
0xa0000001001ed940 writeback_inodes+0x160
        args (0xe0000039ec2bfcd8, 0xe0000039eb96fb48, 0xe0000039eb96fbb8, 0xa000000100c51e48, 0xe0000039ec2bfcf0)
0xa0000001001452a0 wb_kupdate+0x280
        args (0x34eb, 0xe0000039ec2bfcf0, 0xe0000039ec2bfd10, 0x1000036af, 0xe0000039ec2bfcd8)
0xa000000100145f70 pdflush+0x2d0
        args (0xa000000100cbfa10, 0xe0000039ec2bfd88, 0xe0000039ec2bfd90, 0xe0000039ec2bfd78, 0xa000000100bb1a80)
0xa0000001000fb640 kthread+0x260
        args (0xe0000039ee1c7d10, 0xffffffffffffffff, 0x0, 0xa000000100acd590, 0xfffffffffffffffc)
0xa0000001000124b0 kernel_thread_helper+0xd0
        args (0xa000000100acead0, 0xe0000039ee1c7d10, 0xa0000001000094c0, 0x2, 0xa000000100eee5a0)
0xa0000001000094c0 start_kernel_thread+0x20
        args (0xa000000100acead0, 0xe0000039ee1c7d10)

[0]kdb> btt 0xe0000030101d8000
Stack traceback for pid 8416
0xe0000030101d8000 8416 8242 0 3 D 0xe0000030101d82f0 randholes
0xa0000001008bcdb0 schedule+0x9b0

        args (0xe0000039ec207020, 0xe0000039ec2070a8, 0xa0000001001f3fa0, 0x206, 0xa000000100eee5a0)
0xa0000001008bf4b0 io_schedule+0x50
        args (0xe00000b0030258d4, 0xa0000001001338c0, 0x205, 0xa000000100eee5a0)
0xa0000001001338c0 sync_page+0x100
        args (0xa0007fffffe8bd68, 0xa0000001008bfd20, 0x58e, 0xa000000100eee5a0)
0xa0000001008bfd20 __wait_on_bit_lock+0x160
        args (0xe00000b005905e20, 0xe0000030101dfbb0, 0xa000000100accf30, 0x2, 0xa0000001008c2520)
0xa000000100133760 __lock_page+0x180
        args (0xa0007fffffe8bd68, 0xa000000100133d10, 0x48b, 0x0)
0xa000000100133d10 find_lock_page+0xf0
        args (0xe00000b47b2a65f0, 0x275a, 0xa0007fffffe8bd68, 0xe00000b47b2a6608, 0xe00000b47b2a65f8)
0xa000000100134550 __grab_cache_page+0x30
        args (0xe00000b47b2a65f0, 0x275a, 0x3fff, 0xe00000b47b2a6660, 0xfffff)
0xa0000001001f9fa0 block_write_begin+0x1a0
        args (0x0, 0xe00000b47b2a65f0, 0x9d69000, 0x3000, 0x1)
0xa000000100447630 xfs_vm_write_begin+0x50
        args (0x0, 0xe00000b47b2a65f0, 0x9d69000, 0x3000, 0x1)
0xa000000100138d50 pagecache_write_begin+0x90
        args (0x0, 0xe00000b47b2a65f0, 0x9d69000, 0x3000, 0x1)
0xa00000010045df90 xfs_iozero+0x90
        args (0xe00000b47b2a65f0, 0x9d69000, 0x7000, 0xe00000300e23e0b8, 0x3000)
0xa00000010045eaa0 xfs_zero_eof+0x500
        args (0xe00000b47b2a64c8, 0xe00000300e23e150, 0xad36000, 0x9d70000, 0xad35)
0xa0000001004611f0 xfs_write+0xc10
        args (0xe00000300e23e018, 0xe0000030101dfd40, 0xe0000030101dfd30, 0x200, 0xe0000030101dfdc0)
0xa0000001004543e0 __xfs_file_write+0xe0
        args (0xe0000030101dfd40, 0xe0000030101dfd30, 0x1, 0x1, 0xad36000)
0xa0000001004544e0 xfs_file_aio_write+0x40
        args (0xe0000030101dfd40, 0xe0000030101dfd30, 0x1, 0xad36000, 0xa0000001001a11b0)
0xa0000001001a11b0 do_sync_write+0x170
        args (0xa000000100cd7c78, 0xe00000b006679650, 0xa000000100cd7c50, 0xe0000030101dfe38, 0xe0000030101dfd40)
0xa0000001001a2860 vfs_write+0x1a0
        args (0xe00000b006679630, 0x6000000000018020, 0x200, 0xe0000030101dfe38, 0xe00000b006679650)
0xa0000001001a3530 sys_write+0x70
        args (0x3, 0x6000000000018020, 0x200, 0xc000000000000793, 0x250)
0xa00000010000bc40 ia64_ret_from_syscall
        args (0x3, 0x6000000000018020, 0x200, 0xc000000000000793)
0xa000000000010620 __kernel_syscall_via_break
        args (0x3, 0x6000000000018020, 0x200, 0xc000000000000793)

The page:

[0]kdb> md8c8 0xa0007fffffe8bd68
0xa0007fffffe8bd68 0100000000000835 ffffffff00000005 5...............
0xa0007fffffe8bd78 e00000b47aa88228 e00000b47b2a65f0 (..z.....e*{....

                   ^^^^^^^^^^^^^^^^ buffers

0xa0007fffffe8bd88 000000000000275a a0007fffffe8a660 Z'......`.......
0xa0007fffffe8bd98 a0007ffff7aab928 0100000000080000 (...............

kdb> inode_pages 0xe00000b47b2a64c8
page_struct index cnt zone nid flags
e000003010324000 5274 3 0 0 0x824 Referenced LRU Private

        bh 0xe000003479f8d0a8 bno 18446744073709551615 []
        bh 0xe00000b47ad7d0a8 bno    53132 [Uptodate Mapped]
        bh 0xe000003479f8e228 bno 18446744073709551615 []
        bh 0xe00000b47ad7fca8 bno 18446744073709551615 []
 mapping= e00000b47b2a65f0
e00000b079f80000   7637     3   0   1 0x100000000000824 Referenced LRU Private
        bh 0xe00000b47ad7cc28 bno 18446744073709551615 []
        bh 0xe000003479f8cfa8 bno 18446744073709551615 []
        bh 0xe00000b47ad7f028 bno 18446744073709551615 []
        bh 0xe000003479f8f7a8 bno    53082 [Uptodate Mapped]
 mapping= e00000b47b2a65f0
e00000b9e56ac000  10074     5   0   1 0x100000000000835 Locked Referenced Dirty LRU Private
        bh 0xe00000b47aa88228 bno 18446744073709551615 [Uptodate Dirty Mapped Delay]
           ^^^^^^^^^^^^^^^^^^
        bh 0xe000003479f8e3a8 bno 18446744073709551615 []
        bh 0xe00000b47aa882a8 bno 18446744073709551615 []
        bh 0xe000003479f8fd28 bno 18446744073709551615 []

mapping= e00000b47b2a65f0

So the page is locked and it has a delalloc buffer on it that used to be a hole. The problem page is at EOF....

No idea what is going write yet.....

Cheers,

Dave.

Signature · 3 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group
#40 lore raw
Quoted 23 lines · David Chinner
Quoted 9 lines · David Chinner
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 7 lines

......

Quoted 1 line

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested,

Failed to compile in UDF and reiser for me, but no doubt you know that already. Don't have time to look at why - I just disabled them so I get some QA done on the XFS and core changes.

Nick - the XFSQA randholes test locks up. As you can probably guess it creates holey files and does stuff to them ;)

Both pdflush and a write doing an a blocked waiting for a page to become unlocked. They are both waiting on teh same page (0xa0007fffffe8bd68).

[...]

Quoted 4 lines

So the page is locked and it has a delalloc buffer on it that used to be a hole. The problem page is at EOF....

No idea what is going write yet.....

OK, thanks for testing. This is with the full patchset applied, I guess. No real ideas here (brute force we put a last-locked stack trace into struct page :P), but I wonder if it may be any of my earlier patches? (ie. not the new aops path or XFS conversion itself).

Anyway I think randholes is available online, right? I'll try reproduce here and help.

Thanks,
Nick

#41 lore raw
Quoted 25 lines · Nick Piggin
Quoted 18 lines · David Chinner
Quoted 9 lines · David Chinner
Quoted 8 lines · Nick Piggin

Updated aops patchset against 2.6.21-rc5.

http://www.kernel.org/pub/linux/kernel/people/npiggin/patches/new-aops/

Files/dirs are 2.6.21-rc5-new-aops*

Contains numerous fixes from Mark and myself -- I'd say the core code is getting reasonably stable at this point.

Quoted 7 lines

......

Quoted 1 line

(compile only) patches for NFS, XFS, FUSE, eCryptfs. OK, they're untested,

Failed to compile in UDF and reiser for me, but no doubt you know that already. Don't have time to look at why - I just disabled them so I get some QA done on the XFS and core changes.

Quoted 5 lines

Nick - the XFSQA randholes test locks up. As you can probably guess it creates holey files and does stuff to them ;)

Both pdflush and a write doing an a blocked waiting for a page to become unlocked. They are both waiting on teh same page (0xa0007fffffe8bd68).

Quoted 8 lines

[...]

Quoted 4 lines

So the page is locked and it has a delalloc buffer on it that used to be a hole. The problem page is at EOF....

No idea what is going write yet.....

OK, thanks for testing. This is with the full patchset applied, I guess.

Yes.

Quoted 3 lines

No real ideas here (brute force we put a last-locked stack trace into struct page :P), but I wonder if it may be any of my earlier patches? (ie. not the new aops path or XFS conversion itself).

It's not.

I've had a look at the files created by the test - it's the third file in the test that is causing the problem:

_do_test 1 50 "-l 5000000 -c 50 -b $pgsize"
_do_test 2 100 "-l 10000000 -c 100 -b $pgsize"
_do_test 3 100 "-l 10000000 -c 100 -b 512" # test partial pages

i.e. the partial page test. This is the effective command that locks it up:

# randholes -v -l 10000000 -c 100 -b 512 holes.out

Ah, now i see what the I/o pattern is that is triggering this:

budgie:/mnt/test # ~/dgc/xfstests/src/randholes -v -v -v -l 10000000 -c 100 -b 512 dgc.holes.out
randholes: Seed = 1175749165 (use "-s 1175749165" to re-execute this test)
randholes: blocksize=512, filesize=268435456, seed=1175749165
randholes: count=100, offset=0, extsize=0
randholes: verbose=3, wsync=0, direct=0, rt=0, alloconly=0, preserve=0, test=0 write
.writing data at offset=acbd800, value 0xacbd800 and 0xacbd800
writing data at offset=5c09600, value 0x5c09600 and 0x5c09600
writing data at offset=97bde00, value 0x97bde00 and 0x97bde00

And we writing to offset:

0xa0000001004611f0 xfs_write+0xc10

        args (0xe00000347895e750, 0xe00000347aa1fd40, 0xe00000347aa1fd30, 0x200, 0xe00000347aa1fdc0)

.....
[0]kdb> md8c1 0xe00000347aa1fdc0
0xe00000347aa1fdc0 000000000c885400 .T......

Which is well past teh last EOF.
So when we extend the file past the EOF, the old EOF had a partial block (and page) that needed to have the remaining chunk zeroed. This will have found the page already in the cache, but it is locked.

That tends to imply that it wasn't unlocked correctly on teh first call to xfs_iozero it was originally written...

Ah, found it. page_cache_write_begin() returns zero on success. So this:

               status = pagecache_write_begin(NULL, mapping, pos, bytes,
                                       AOP_FLAG_UNINTERRUPTIBLE,
                                       &page, &fsdata);
               if (!status)
                        break;

Will abort on success, never zero the range of the page it's supposed to and leave without calling write_end. Then when we extend the file again....

Hmmm - an interesting question just came up. The return from ->write_end() will either be equal to 'copied' or zero. What do you do if it returns zero? it indicates a failure of some kind, but what?

In xfs_iozero, because we loop based on the number of bytes we get returned from pagecache_write_end(); a return of zero bytes will result in trying the same operation again. If it fails repeatedly in the same manner (likely), then we've got an endless loop. I can break out of the loop, but I've got no idea what the error was or how to handle it. Any thoughts on this?

Also, when pagecache_write_begin() (and ->write_begin()) returns zero, it indicates success. When pagecache_write_end() (and ->write_end()) returns zero, it indicates some kind of failure occurred. Can you make the return values more consistent, Nick?

Patch below

Cheers,

Dave.

Signature · 36 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group

---
 fs/xfs/linux-2.6/xfs_lrw.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

Index: 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_lrw.c
===================================================================
--- 2.6.x-xfs-new.orig/fs/xfs/linux-2.6/xfs_lrw.c	2007-04-05 15:07:54.000000000 +1000
+++ 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_lrw.c	2007-04-05 15:41:44.912532457 +1000
@@ -151,18 +151,17 @@ xfs_iozero(
 		status = pagecache_write_begin(NULL, mapping, pos, bytes,
 					AOP_FLAG_UNINTERRUPTIBLE,
 					&page, &fsdata);
-		if (!status)
+		if (status)
 			break;
 
 		memclear_highpage_flush(page, offset, bytes);
 
 		status = pagecache_write_end(NULL, mapping, pos, bytes, bytes,
 					page, fsdata);
-		if (status < 0)
-			break;
-		bytes = status;
-		pos += bytes;
-		count -= bytes;
+		WARN_ON(status <= 0); /* can't return less than zero! */
+		pos += status;
+		count -= status;
+		status = 0;
 	} while (count);
 
 	return (-status);
#42 lore raw

On Thu, Apr 05, 2007 at 04:18:09PM +1000, David Chinner wrote:

[lots of interesting stuff]

Quoted 10 lines

Ah, found it. page_cache_write_begin() returns zero on success. So this:

               status = pagecache_write_begin(NULL, mapping, pos, bytes,
                                       AOP_FLAG_UNINTERRUPTIBLE,
                                       &page, &fsdata);
               if (!status)
                        break;

Will abort on success, never zero the range of the page it's supposed to and leave without calling write_end. Then when we extend the file again....

Ah, and I just downloaded xfs-cmds :P

Thanks for finding that. Sorry to waste your time with a simple thinko :(

Quoted 3 lines

Hmmm - an interesting question just came up. The return from ->write_end() will either be equal to 'copied' or zero. What do you do if it returns zero? it indicates a failure of some kind, but what?

zero means that it couldn't copy anything, but there was no permanent failure (ie. we could retry as we might with any other short write).

The reason why write_end returns "copied" is that some filesystems (jffs was one) that want to return a short copy. I would prefer if they could all to the required setup in write_begin so a short write couldn't occur at write_end-time, however I'm not an expert on filesystems, so I left this feature in. It will probably be easier to rip it out later than to introduce it later. I think?

Quoted 6 lines

In xfs_iozero, because we loop based on the number of bytes we get returned from pagecache_write_end(); a return of zero bytes will result in trying the same operation again. If it fails repeatedly in the same manner (likely), then we've got an endless loop. I can break out of the loop, but I've got no idea what the error was or how to handle it. Any thoughts on this?

Basically you can probably take some advantage of knowing that your filesystem never returns 0 here, but we should note that the filesystem must either return an error or copy some bytes eventually, rather than always return 0. I'll at that to the docs.

Quoted 4 lines

Also, when pagecache_write_begin() (and ->write_begin()) returns zero, it indicates success. When pagecache_write_end() (and ->write_end()) returns zero, it indicates some kind of failure occurred. Can you make the return values more consistent, Nick?

As I said, zero is not technically an error (and it isn't a great deal more code because most callers have to deal with short copies anyway).

If we get agreement that the feature of allowing the filesystem to shorten the copy at write_end-time is unnecessary, then I would like that because it does make interfaces easier.

Quoted 2 lines

Patch below

Thanks a lot, applied.

Quoted 41 lines

Cheers,

Dave.

Signature · 10 lines
Dave Chinner
Principal Engineer
SGI Australian Software Group

---
 fs/xfs/linux-2.6/xfs_lrw.c |   11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

Index: 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_lrw.c
===================================================================
fs/xfs/linux-2.6/xfs_lrw.c +5 -6
@@ -151,18 +151,17 @@ xfs_iozero(
 		status = pagecache_write_begin(NULL, mapping, pos, bytes,
 					AOP_FLAG_UNINTERRUPTIBLE,
 					&page, &fsdata);
-		if (!status)
+		if (status)
 			break;
 
 		memclear_highpage_flush(page, offset, bytes);
 
 		status = pagecache_write_end(NULL, mapping, pos, bytes, bytes,
 					page, fsdata);
-		if (status < 0)
-			break;
-		bytes = status;
-		pos += bytes;
-		count -= bytes;
+		WARN_ON(status <= 0); /* can't return less than zero! */
+		pos += status;
+		count -= status;
+		status = 0;
 	} while (count);
 
 	return (-status);