Skip to contentSkip to conversation

ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

PATCH New Wengang Wang 4 participants 8 messages Nov 18, 2010 – Dec 31, 2010 OCFS2

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

---
 fs/ocfs2/aops.c |   10 +++++++---
 fs/ocfs2/mmap.c |   10 +++++++---
 2 files changed, 14 insertions(+), 6 deletions(-)
fs/ocfs2/aops.c +7 -3
index f1e962c..6b7e1f2 100644
@@ -1130,14 +1130,18 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
 
 		if (index == target_index && mmap_page) {
 			/*
-			 * ocfs2_pagemkwrite() is a little different
+			 * ocfs2_page_mkwrite() is a little different
 			 * and wants us to directly use the page
 			 * passed in.
+			 * So far, only __ocfs2_page_mkwrite() passes in a
+			 * non-NULL page. And we locked the page there, so we
+			 * just assert the page is locked. If another path will
+			 * be passing in a non-NULL page in future, the locking
+			 * should be well considered.
 			 */
-			lock_page(mmap_page);
+			BUG_ON(!PageLocked(mmap_page));
 
 			if (mmap_page->mapping != mapping) {
-				unlock_page(mmap_page);
 				/*
 				 * Sanity check - the locking in
 				 * ocfs2_pagemkwrite() should ensure
fs/ocfs2/mmap.c +7 -3
index 7e32db9..bb29335 100644
@@ -79,9 +79,10 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
 	 * from do_generic_file_read.
 	 */
 	last_index = (size - 1) >> PAGE_CACHE_SHIFT;
+	lock_page(page);
 	if (unlikely(!size || page->index > last_index)) {
 		ret = -EINVAL;
-		goto out;
+		goto out_unlock;
 	}
 
 	/*
@@ -96,7 +97,7 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
 		 * So return 0 here and let VFS retry.
 		 */
 		ret = 0;
-		goto out;
+		goto out_unlock;
 	}
 
 	/*
@@ -117,7 +118,7 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
 	if (ret) {
 		if (ret != -ENOSPC)
 			mlog_errno(ret);
-		goto out;
+		goto out_unlock;
 	}
 
 	ret = ocfs2_write_end_nolock(mapping, pos, len, len, locked_page,
@@ -128,6 +129,9 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
 	}
 	BUG_ON(ret != len);
 	ret = 0;
+	goto out;
+out_unlock:
+	unlock_page(page);
 out:
 	return ret;
 }
Signature · 1 line
1.7.2.3
#1 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Quoted 5 lines · Wengang Wang

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

This looks like a good implementation of the concept. I'm still
wary of holding the page lock that long, though I *think* it's the right thing.
Mark, do you have any thoughts on this? I believe this is
originally your code.

Joel

Signature · 8 lines
"Not being known doesn't stop the truth from being true."
        - Richard Bach

Joel Becker
Senior Development Manager
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127
#2 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Hi Mark,

Do you have any comment on this patch? Seems Joel wants your ACK.

regards,
wengang.

Quoted 25 lines · Joel Becker
Quoted 5 lines · Wengang Wang

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

This looks like a good implementation of the concept. I'm still
wary of holding the page lock that long, though I *think* it's the right thing.
Mark, do you have any thoughts on this? I believe this is
originally your code.

Joel

Signature · 8 lines
"Not being known doesn't stop the truth from being true."
        - Richard Bach

Joel Becker
Senior Development Manager
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127
#3 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

In reply to Wengang Wang

Hi Wengang,

Quoted 5 lines · Wengang Wang

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

Thanks for doing this work. Review is below.

Quoted 31 lines
fs/ocfs2/aops.c +7 -3
index f1e962c..6b7e1f2 100644
@@ -1130,14 +1130,18 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
 
 		if (index == target_index && mmap_page) {
 			/*
-			 * ocfs2_pagemkwrite() is a little different
+			 * ocfs2_page_mkwrite() is a little different
 			 * and wants us to directly use the page
 			 * passed in.
+			 * So far, only __ocfs2_page_mkwrite() passes in a
+			 * non-NULL page. And we locked the page there, so we
+			 * just assert the page is locked. If another path will
+			 * be passing in a non-NULL page in future, the locking
+			 * should be well considered.
 			 */
-			lock_page(mmap_page);
+			BUG_ON(!PageLocked(mmap_page));
 
 			if (mmap_page->mapping != mapping) {
-				unlock_page(mmap_page);
 				/*
 				 * Sanity check - the locking in
 				 * ocfs2_pagemkwrite() should ensure
fs/ocfs2/mmap.c +0 -0
index 7e32db9..bb29335 100644

fs/ocfs2/mmap.c static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh, · line 82

Quoted 4 lines
fs/ocfs2/mmap.c +1 -0
@@ -79,9 +79,10 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
 	 * from do_generic_file_read.
 	 */
 	last_index = (size - 1) >> PAGE_CACHE_SHIFT;
+	lock_page(page);

This breaks the page lock ordering in ocfs2_grab_pages_for_write() -- note that the call to find_or_create_page() in that function returns pages in a locked state. So roughly, we want to be taking them in index order.

	--Mark

--
Mark Fasheh

#4 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Quoted 43 lines · Mark Fasheh

Hi Wengang,

Quoted 5 lines · Wengang Wang

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

Thanks for doing this work. Review is below.

Quoted 31 lines
fs/ocfs2/aops.c +7 -3
index f1e962c..6b7e1f2 100644
@@ -1130,14 +1130,18 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
 
 		if (index == target_index && mmap_page) {
 			/*
-			 * ocfs2_pagemkwrite() is a little different
+			 * ocfs2_page_mkwrite() is a little different
 			 * and wants us to directly use the page
 			 * passed in.
+			 * So far, only __ocfs2_page_mkwrite() passes in a
+			 * non-NULL page. And we locked the page there, so we
+			 * just assert the page is locked. If another path will
+			 * be passing in a non-NULL page in future, the locking
+			 * should be well considered.
 			 */
-			lock_page(mmap_page);
+			BUG_ON(!PageLocked(mmap_page));
 
 			if (mmap_page->mapping != mapping) {
-				unlock_page(mmap_page);
 				/*
 				 * Sanity check - the locking in
 				 * ocfs2_pagemkwrite() should ensure
fs/ocfs2/mmap.c +0 -0
index 7e32db9..bb29335 100644
@@ -79,9 +79,10 @@ static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh,
Quoted 8 lines

fs/ocfs2/mmap.c static int __ocfs2_page_mkwrite(struct file *file, struct buffer_head *di_bh, · line 82

Quoted 4 lines
fs/ocfs2/mmap.c +1 -0
 	 * from do_generic_file_read.
 	 */
 	last_index = (size - 1) >> PAGE_CACHE_SHIFT;
+	lock_page(page);

This breaks the page lock ordering in ocfs2_grab_pages_for_write() -- note that the call to find_or_create_page() in that function returns pages in a locked state. So roughly, we want to be taking them in index order.

Yes, this sounds reasonable. Let me see if there could be a better fix.

thanks,
wengang.

#5 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

In reply to Mark Fasheh

Hi Mark,

Quoted 9 lines · Mark Fasheh

Hi Wengang,

Quoted 5 lines · Wengang Wang

Lock the page in __ocfs2_page_mkwrite(). Or we may get -EINVAL error from ocfs2_grab_pages_for_write() if the page(page cache) gets truncated while someone is flushing the pagecache. In the original code, we are checking mapping earlier in __ocfs2_page_mkwrite() but we don't lock it. so the mapping can change during flushing pagecache.

<snip>

Quoted 4 lines

This breaks the page lock ordering in ocfs2_grab_pages_for_write() -- note that the call to find_or_create_page() in that function returns pages in a locked state. So roughly, we want to be taking them in index order.

In ocfs2_grab_pages_for_write() code, there is a comment that

1131 if (index == target_index && mmap_page) {
1132 /*

1133                          * ocfs2_pagemkwrite() is a little different
1134                          * and wants us to directly use the page
1135                          * passed in.
1136                          */

1137 lock_page(mmap_page);

I guess it's commented by you? So how it "wants" us to use the page passed in?
The main purpose of page_mkwrite() is to setup the mapping of page index to sector(block) numbers on the device. I don't see using the page passed in is very important. Correct me if I am wrong.
So also for (index == target_index && mmap_page) situation, we can use find_or_create_page() just like where it's used in other situation. find_lock_page() takes care of (page->mapping != mapping). How about it?

#patch attached

regards,
wengang.
-------------- next part --------------

fs/ocfs2/aops.c +5 -30
index f1e962c..e9145db 100644
@@ -1128,36 +1128,11 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
 	for(i = 0; i < wc->w_num_pages; i++) {
 		index = start + i;
 
-		if (index == target_index && mmap_page) {
-			/*
-			 * ocfs2_pagemkwrite() is a little different
-			 * and wants us to directly use the page
-			 * passed in.
-			 */
-			lock_page(mmap_page);
-
-			if (mmap_page->mapping != mapping) {
-				unlock_page(mmap_page);
-				/*
-				 * Sanity check - the locking in
-				 * ocfs2_pagemkwrite() should ensure
-				 * that this code doesn't trigger.
-				 */
-				ret = -EINVAL;
-				mlog_errno(ret);
-				goto out;
-			}
-
-			page_cache_get(mmap_page);
-			wc->w_pages[i] = mmap_page;
-		} else {
-			wc->w_pages[i] = find_or_create_page(mapping, index,
-							     GFP_NOFS);
-			if (!wc->w_pages[i]) {
-				ret = -ENOMEM;
-				mlog_errno(ret);
-				goto out;
-			}
+		wc->w_pages[i] = find_or_create_page(mapping, index, GFP_NOFS);
+		if (!wc->w_pages[i]) {
+			ret = -ENOMEM;
+			mlog_errno(ret);
+			goto out;
 		}
 
 		if (index == target_index)
#6 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Quoted 10 lines · Wengang Wang

In ocfs2_grab_pages_for_write() code, there is a comment that

1131 if (index == target_index && mmap_page) {
1132 /*

1133                          * ocfs2_pagemkwrite() is a little different
1134                          * and wants us to directly use the page
1135                          * passed in.
1136                          */

1137 lock_page(mmap_page);

Quoted 8 lines

I guess it's commented by you? So how it "wants" us to use the page passed in?
The main purpose of page_mkwrite() is to setup the mapping of page index to sector(block) numbers on the device. I don't see using the page passed in is very important. Correct me if I am wrong.
So also for (index == target_index && mmap_page) situation, we can use find_or_create_page() just like where it's used in other situation. find_lock_page() takes care of (page->mapping != mapping). How about it?

Boy, this is a rabbit hole. The return codes for page_mkwrite()
got changed a year ago, and no one updated ocfs2 (or ext4 or gfs2 for that matter). I've emailed Nick about it.

Joel

Signature · 8 lines
"Senator let's be sincere,
 As much as you can."

Joel Becker
Senior Development Manager
Oracle
E-mail: joel.becker at oracle.com
Phone: (650) 506-8127
#7 lore raw

PATCH 1/1 ocfs2: lock the page in __ocfs2_page_mkwrite() -v2

Quoted 20 lines · Joel Becker
Quoted 10 lines · Wengang Wang

In ocfs2_grab_pages_for_write() code, there is a comment that

1131 if (index == target_index&& mmap_page) {
1132 /*

1133                          * ocfs2_pagemkwrite() is a little different
1134                          * and wants us to directly use the page
1135                          * passed in.
1136                          */

1137 lock_page(mmap_page);

Quoted 8 lines

I guess it's commented by you? So how it "wants" us to use the page passed in?
The main purpose of page_mkwrite() is to setup the mapping of page index to sector(block) numbers on the device. I don't see using the page passed in is very important. Correct me if I am wrong.
So also for (index == target_index&& mmap_page) situation, we can use find_or_create_page() just like where it's used in other situation. find_lock_page() takes care of (page->mapping != mapping). How about it?

Quoted 3 lines

Boy, this is a rabbit hole. The return codes for page_mkwrite()
got changed a year ago, and no one updated ocfs2 (or ext4 or gfs2 for that matter). I've emailed Nick about it.

Which patch and why it is a hole? would you mind describing it to me? Just curious.

Regards,
Tao