Skip to contentSkip to conversation

btrfs: inode: btrfs_invalidatepage() related refactor and fix for subpage

PATCH In review Qu Wenruo 7 participants 16 messages Dec 17, 2020 – Dec 22, 2020 Btrfs file system

lore
Share & tools
lore
author
#1 lore raw

PATCH 1/4 btrfs: inode: use min() to replace open-code in btrfs_invalidatepage()

In reply to Qu Wenruo

In btrfs_invalidatepage() we introduce a temporary variable, new_len, to update ordered->truncated_len.

But we can use min() to replace it completely and no need for the variable.

---
 fs/btrfs/inode.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
fs/btrfs/inode.c +2 -4
index af3439b21bc4..dced71bccaac 100644
@@ -8219,15 +8219,13 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 		 */
 		if (TestClearPagePrivate2(page)) {
 			struct btrfs_ordered_inode_tree *tree;
-			u64 new_len;
 
 			tree = &inode->ordered_tree;
 
 			spin_lock_irq(&tree->lock);
 			set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
-			new_len = start - ordered->file_offset;
-			if (new_len < ordered->truncated_len)
-				ordered->truncated_len = new_len;
+			ordered->truncated_len = min(ordered->truncated_len,
+					start - ordered->file_offset);
 			spin_unlock_irq(&tree->lock);
 
 			ASSERT(end - start + 1 < U32_MAX);
Signature · 1 line
2.29.2
author
#2 lore raw

PATCH 2/4 btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

---
 fs/btrfs/inode.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/inode.c +3 -6
index dced71bccaac..b4d36d138008 100644
@@ -8169,6 +8169,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 				 unsigned int length)
 {
 	struct btrfs_inode *inode = BTRFS_I(page->mapping->host);
+	struct btrfs_ordered_inode_tree *ordered_tree = &inode->ordered_tree;
 	struct extent_io_tree *tree = &inode->io_tree;
 	struct btrfs_ordered_extent *ordered;
 	struct extent_state *cached_state = NULL;
@@ -8218,15 +8219,11 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 		 * for the finish_ordered_io
 		 */
 		if (TestClearPagePrivate2(page)) {
-			struct btrfs_ordered_inode_tree *tree;
-
-			tree = &inode->ordered_tree;
-
-			spin_lock_irq(&tree->lock);
+			spin_lock_irq(&ordered_tree->lock);
 			set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
 			ordered->truncated_len = min(ordered->truncated_len,
 					start - ordered->file_offset);
-			spin_unlock_irq(&tree->lock);
+			spin_unlock_irq(&ordered_tree->lock);
 
 			ASSERT(end - start + 1 < U32_MAX);
 			if (btrfs_dec_test_ordered_pending(inode, &ordered,
Signature · 1 line
2.29.2

Qu Wenruo

author
#0 lore raw

cover btrfs: inode: btrfs_invalidatepage() related refactor and fix for subpage

This small patchset contains 3 refactors which are subpage independent.

And the last patch is RFC where I'm not certain about the existing code, but it solves the problem for subpage during test.

Thus I'm here asking for help on the btrfs_invalidatepage() behavior.

Qu Wenruo (4):
btrfs: inode: use min() to replace open-code in btrfs_invalidatepage()
btrfs: inode: remove variable shadowing in btrfs_invalidatepage()
btrfs: inode: move the timing of TestClearPagePrivate() in
btrfs_invalidatepage()
btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

 fs/btrfs/inode.c | 45 +++++++++++++++++++++++++++------------------
 1 file changed, 27 insertions(+), 18 deletions(-)
Signature · 1 line
2.29.2
author
#10 lore raw

PATCH 3/4 btrfs: inode: move the timing of TestClearPagePrivate() in btrfs_invalidatepage()

For current sectorsize == PAGE_ZIE case, there will only be at most one ordered extent for one page.

But for subpage case, one page can contain several ordered extents, thus current TestClearPagePrivate2() call will only finish ordered IO for the first ordered extent, the remaining ones will be skipped, and cause never-end ordered io.

This patch will move the TestClearPagePrivate2 before the loop, and save the result, so that we can finish all ordered extents.

---
 fs/btrfs/inode.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
fs/btrfs/inode.c +6 -1
index b4d36d138008..eb493fbb65f9 100644
@@ -8178,6 +8178,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 	u64 start;
 	u64 end;
 	int inode_evicting = inode->vfs_inode.i_state & I_FREEING;
+	bool cleared_private2;
 	bool found_ordered = false;
 	bool completed_ordered = false;
 
@@ -8197,6 +8198,8 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 
 	if (!inode_evicting)
 		lock_extent_bits(tree, page_start, page_end, &cached_state);
+
+	cleared_private2 = TestClearPagePrivate2(page);
 again:
 	start = page_start;
 	ordered = btrfs_lookup_ordered_range(inode, start, page_end - start + 1);
@@ -8214,11 +8217,12 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 					 EXTENT_DELALLOC |
 					 EXTENT_LOCKED | EXTENT_DO_ACCOUNTING |
 					 EXTENT_DEFRAG, 1, 0, &cached_state);
+
 		/*
 		 * whoever cleared the private bit is responsible
 		 * for the finish_ordered_io
 		 */
-		if (TestClearPagePrivate2(page)) {
+		if (cleared_private2) {
 			spin_lock_irq(&ordered_tree->lock);
 			set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
 			ordered->truncated_len = min(ordered->truncated_len,
@@ -8233,6 +8237,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 				completed_ordered = true;
 			}
 		}
+
 		btrfs_put_ordered_extent(ordered);
 		if (!inode_evicting) {
 			cached_state = NULL;
Signature · 1 line
2.29.2
author
#11 lore raw

RFC PATCH 4/4 btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

In reply to Qu Wenruo

[BUG]
With current subpage RW patchset, the following script can lead to filesystem hang:

  # mkfs.btrfs -f -s 4k $dev
  # mount $dev -o nospace_cache $mnt
  # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

The file system will hang at wait_event() of
btrfs_start_ordered_extent().

[CAUSE]
The root cause is, btrfs_invalidatepage() is freeing page::private which still has subpage dirty bit set.

The offending situation happens like this:
btrfs_fllocate()
|- btrfs_zero_range()
|- btrfs_punch_hole_lock_range()
|- truncate_pagecache_range()
|- btrfs_invalidatepage()

The involved range looks like:

0 32K 64K 96K 128K

	|///////||//////|
	| Range to drop |

For the [32K, 64K) range, since the offset is 32K, the page won't be invalidated.

But for the [64K, 96K) range, the offset is 0, current btrfs_invalidatepage() will call clear_page_extent_mapped() which will detach page::private, making the subpage dirty bitmap being cleared.

This prevents later __extent_writepage_io() to locate any range to write, thus no way to wake up the ordered extents.

[FIX]
To fix the problem this patch will:
- Only clear page status and detach page private when the full page is invalidated

- Change how we handle unfinished ordered extent
If there is any ordered extent unfinished in the page range, we can't call clear_extent_bit() with delete == true.

[REASON FOR RFC]
There is still uncertainty around the btrfs_releasepage() call.

1. Why we need btrfs_releasepage() call for non-full-page condition? Other fs (aka. xfs) just exit without doing special handling if invalidatepage() is called with part of the page.

Thus I didn't completely understand why btrfs_releasepage() here is needed for non-full page call.

2. Why "if (offset)" is not causing problem for current code? This existing if (offset) call can be skipped for cases like offset == 0 length == 2K.
As MM layer can call invalidatepage() with unaligned offset/length, for cases like truncate_inode_pages_range().
This will make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page.

---
 fs/btrfs/inode.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
fs/btrfs/inode.c +16 -7
index eb493fbb65f9..872c5309b4ca 100644
@@ -8180,7 +8180,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 	int inode_evicting = inode->vfs_inode.i_state & I_FREEING;
 	bool cleared_private2;
 	bool found_ordered = false;
-	bool completed_ordered = false;
+	bool incompleted_ordered = false;
 
 	/*
 	 * we have the page locked, so new writeback can't start,
@@ -8191,7 +8191,13 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 	 */
 	wait_on_page_writeback(page);
 
-	if (offset) {
+	/*
+	 * The range doesn't cover the full page, just let btrfs_releasepage() to
+	 * check if we can release the extent mapping.
+	 * Any locked/pinned/logged extent map would prevent us freeing the
+	 * extent mapping.
+	 */
+	if (!(offset == 0 && length == PAGE_SIZE)) {
 		btrfs_releasepage(page, GFP_NOFS);
 		return;
 	}
@@ -8208,9 +8214,10 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 		end = min(page_end,
 			  ordered->file_offset + ordered->num_bytes - 1);
 		/*
-		 * IO on this page will never be started, so we need to account
-		 * for any ordered extents now. Don't clear EXTENT_DELALLOC_NEW
-		 * here, must leave that up for the ordered extent completion.
+		 * IO on this ordered extent will never be started, so we need
+		 * to account for any ordered extents now. Don't clear
+		 * EXTENT_DELALLOC_NEW here, must leave that up for the
+		 * ordered extent completion.
 		 */
 		if (!inode_evicting)
 			clear_extent_bit(tree, start, end,
@@ -8234,7 +8241,8 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 							   start,
 							   end - start + 1, 1)) {
 				btrfs_finish_ordered_io(ordered);
-				completed_ordered = true;
+			} else {
+				incompleted_ordered = true;
 			}
 		}
 
@@ -8276,7 +8284,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 		 * is cleared if we don't delete, otherwise it can lead to
 		 * corruptions if the i_size is extented later.
 		 */
-		if (found_ordered && !completed_ordered)
+		if (found_ordered && incompleted_ordered)
 			delete = false;
 		clear_extent_bit(tree, page_start, page_end, EXTENT_LOCKED |
 				 EXTENT_DELALLOC | EXTENT_UPTODATE |
@@ -8286,6 +8294,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
 		__btrfs_releasepage(page, GFP_NOFS);
 	}
 
+	ClearPagePrivate2(page);
 	ClearPageChecked(page);
 	clear_page_extent_mapped(page);
 }
Signature · 1 line
2.29.2
#3 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Qu Wenruo
Quoted 15 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

---
 fs/btrfs/inode.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
fs/btrfs/inode.c +0 -0
index dced71bccaac..b4d36d138008 100644

fs/btrfs/inode.c static void btrfs_invalidatepage(struct · line 8175

Quoted 7 lines
fs/btrfs/inode.c +1 -0
@@ -8169,6 +8169,7 @@ static void btrfs_invalidatepage(struct 
 page *page, unsigned int offset,
 				 unsigned int length)
 {
 	struct btrfs_inode *inode = BTRFS_I(page->mapping->host);
+	struct btrfs_ordered_inode_tree *ordered_tree = 
 &inode->ordered_tree;
 

Any reason for the declaration here? I didn't find that patch[3/4] use it.

fs/btrfs/inode.c static void btrfs_invalidatepage(struct · line 8223

Quoted 10 lines
fs/btrfs/inode.c +0 -2
 	struct extent_io_tree *tree = &inode->io_tree;
 	struct btrfs_ordered_extent *ordered;
 	struct extent_state *cached_state = NULL;
@@ -8218,15 +8219,11 @@ static void btrfs_invalidatepage(struct 
 page *page, unsigned int offset,
 		 * for the finish_ordered_io
 		 */
 		if (TestClearPagePrivate2(page)) {
-			struct btrfs_ordered_inode_tree *tree;
-

Better to just rename the @tree to @ordered_tree.

Quoted 12 lines
fs/btrfs/inode.c +2 -4
-			tree = &inode->ordered_tree;
-
-			spin_lock_irq(&tree->lock);
+			spin_lock_irq(&ordered_tree->lock);
 			set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
 			ordered->truncated_len = min(ordered->truncated_len,
 					start - ordered->file_offset);
-			spin_unlock_irq(&tree->lock);
+			spin_unlock_irq(&ordered_tree->lock);
 
 			ASSERT(end - start + 1 < U32_MAX);

if (btrfs_dec_test_ordered_pending(inode, &ordered,

#4 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

Quoted 13 lines · Su Yue
Quoted 10 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

---
 fs/btrfs/inode.c | 9 +++------

 1 file changed, 3 insertions(+), 6 deletions(-)
Quoted 13 lines

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page · line 8175

Quoted 12 lines
fs/btrfs/inode.c +1 -0
index dced71bccaac..b4d36d138008 100644
@@ -8169,6 +8169,7 @@ static void btrfs_invalidatepage(struct page 
 *page, unsigned int offset,
                   unsigned int length)
  {
      struct btrfs_inode *inode = BTRFS_I(page->mapping->host);
+    struct btrfs_ordered_inode_tree *ordered_tree = 
 &inode->ordered_tree;
 

Any reason for the declaration here? I didn't find that patch[3/4] use it.

Didn't that ordered_tree get used lines below?

Quoted 12 lines

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page · line 8223

Quoted 11 lines

     struct extent_io_tree *tree = &inode->io_tree; struct btrfs_ordered_extent *ordered; struct extent_state *cached_state = NULL;

fs/btrfs/inode.c +0 -2
@@ -8218,15 +8219,11 @@ static void btrfs_invalidatepage(struct page 
 *page, unsigned int offset,
           * for the finish_ordered_io
           */
          if (TestClearPagePrivate2(page)) {
-            struct btrfs_ordered_inode_tree *tree;
-

Better to just rename the @tree to @ordered_tree.

Isn't that exactly what I did?

Thanks,
Qu

Quoted 14 lines
Quoted 13 lines
fs/btrfs/inode.c +2 -4
-            tree = &inode->ordered_tree;
-
-            spin_lock_irq(&tree->lock);
+            spin_lock_irq(&ordered_tree->lock);
              set_bit(BTRFS_ORDERED_TRUNCATED, &ordered->flags);
              ordered->truncated_len = min(ordered->truncated_len,
                      start - ordered->file_offset);
-            spin_unlock_irq(&tree->lock);
+            spin_unlock_irq(&ordered_tree->lock);
 
              ASSERT(end - start + 1 < U32_MAX);

             if (btrfs_dec_test_ordered_pending(inode, &ordered,

#6 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Qu Wenruo
Quoted 4 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

You can't do that, because lock_extent_bits expects extent_io_tree !

#7 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

Quoted 10 lines · Nikolay Borisov
Quoted 4 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

You can't do that, because lock_extent_bits expects extent_io_tree !

Ok, nvm, you just factored the var at the beginning of the functions. OTOH since the ordered tree is used just for lock/unlock why not do spin_(un)lock(&inode->ordered_tree->lock);

#5 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Qu Wenruo
Quoted 45 lines · Qu Wenruo
Quoted 13 lines · Su Yue
Quoted 10 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

---
 fs/btrfs/inode.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
Quoted 14 lines
Quoted 8 lines
fs/btrfs/inode.c +0 -0
index dced71bccaac..b4d36d138008 100644
@@ -8169,6 +8169,7 @@ static void btrfs_invalidatepage(struct 

page
*page, unsigned int offset,
unsigned int length)

Quoted 5 lines
 {
     struct btrfs_inode *inode = BTRFS_I(page->mapping->host);

+ struct btrfs_ordered_inode_tree *ordered_tree =
&inode->ordered_tree;

Any reason for the declaration here? I didn't find that patch[3/4] use it.

Didn't that ordered_tree get used lines below?

Quoted 13 lines

fs/btrfs/inode.c

Quoted 12 lines
     struct extent_io_tree *tree = &inode->io_tree;
     struct btrfs_ordered_extent *ordered;
     struct extent_state *cached_state = NULL;
fs/btrfs/inode.c +0 -0
@@ -8218,15 +8219,11 @@ static void 

btrfs_invalidatepage(struct page
*page, unsigned int offset,

          * for the finish_ordered_io
          */
         if (TestClearPagePrivate2(page)) {
fs/btrfs/inode.c +0 -2
-            struct btrfs_ordered_inode_tree *tree;
-

Better to just rename the @tree to @ordered_tree.

Quoted 1 line

Isn't that exactly what I did?

What I mean is that keep the declaration in the block since no further use of it.

Quoted 20 lines

Thanks,
Qu

Quoted 17 lines
Quoted 16 lines
fs/btrfs/inode.c +2 -4
-            tree = &inode->ordered_tree;
-
-            spin_lock_irq(&tree->lock);
+            spin_lock_irq(&ordered_tree->lock);
             set_bit(BTRFS_ORDERED_TRUNCATED, 
             &ordered->flags);
             ordered->truncated_len = 
             min(ordered->truncated_len,
                     start - ordered->file_offset);
-            spin_unlock_irq(&tree->lock);
+            spin_unlock_irq(&ordered_tree->lock);
 
             ASSERT(end - start + 1 < U32_MAX);
             if (btrfs_dec_test_ordered_pending(inode, 
             &ordered,
author
#8 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Nikolay Borisov
Quoted 9 lines · Nikolay Borisov
Quoted 6 lines · Nikolay Borisov
Quoted 3 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

Remove such variable shadowing which can be very confusing.

You can't do that, because lock_extent_bits expects extent_io_tree !

Ok, nvm, you just factored the var at the beginning of the functions. OTOH since the ordered tree is used just for lock/unlock why not do spin_(un)lock(&inode->ordered_tree->lock);

Oh, that indeed looks better and since Su is also complaining about the declaration at the beginning of the function, I guess that's the better way to go.

Thanks,
Qu

#12 lore raw

Re: [PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

In reply to Qu Wenruo
Quoted 57 lines · Qu Wenruo

[BUG]
With current subpage RW patchset, the following script can lead to filesystem hang:

  # mkfs.btrfs -f -s 4k $dev
  # mount $dev -o nospace_cache $mnt
  # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

The file system will hang at wait_event() of
btrfs_start_ordered_extent().

[CAUSE]
The root cause is, btrfs_invalidatepage() is freeing page::private which still has subpage dirty bit set.

The offending situation happens like this:
btrfs_fllocate()
|- btrfs_zero_range()
|- btrfs_punch_hole_lock_range()
|- truncate_pagecache_range()
|- btrfs_invalidatepage()

The involved range looks like:

0       32K     64K     96K     128K
        |///////||//////|
        | Range to drop |

For the [32K, 64K) range, since the offset is 32K, the page won't be invalidated.

But for the [64K, 96K) range, the offset is 0, current btrfs_invalidatepage() will call clear_page_extent_mapped() which will detach page::private, making the subpage dirty bitmap being cleared.

This prevents later __extent_writepage_io() to locate any range to write, thus no way to wake up the ordered extents.

[FIX]
To fix the problem this patch will:
- Only clear page status and detach page private when the full page is invalidated

- Change how we handle unfinished ordered extent
If there is any ordered extent unfinished in the page range, we can't call clear_extent_bit() with delete == true.

[REASON FOR RFC]
There is still uncertainty around the btrfs_releasepage() call.

1. Why we need btrfs_releasepage() call for non-full-page condition? Other fs (aka. xfs) just exit without doing special handling if invalidatepage() is called with part of the page.

Thus I didn't completely understand why btrfs_releasepage() here is needed for non-full page call.

Quoted 7 lines

2. Why "if (offset)" is not causing problem for current code? This existing if (offset) call can be skipped for cases like offset == 0 length == 2K.
As MM layer can call invalidatepage() with unaligned offset/length, for cases like truncate_inode_pages_range().
This will make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page.

I don't think you can ever get offset == 0 and length < PAGE_SIZE unless this is the last page in the file, the one containing eof, in which it's perfectly valid to invalidate the whole page.

Quoted 35 lines
---
 fs/btrfs/inode.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
fs/btrfs/inode.c +8 -2
index eb493fbb65f9..872c5309b4ca 100644
@@ -8180,7 +8180,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
        int inode_evicting = inode->vfs_inode.i_state & I_FREEING;
        bool cleared_private2;
        bool found_ordered = false;
-       bool completed_ordered = false;
+       bool incompleted_ordered = false;
 
        /*
         * we have the page locked, so new writeback can't start,
@@ -8191,7 +8191,13 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
         */
        wait_on_page_writeback(page);
 
-       if (offset) {
+       /*
+        * The range doesn't cover the full page, just let btrfs_releasepage() to
+        * check if we can release the extent mapping.
+        * Any locked/pinned/logged extent map would prevent us freeing the
+        * extent mapping.
+        */
+       if (!(offset == 0 && length == PAGE_SIZE)) {
                btrfs_releasepage(page, GFP_NOFS);
                return;
        }

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page *page, unsigned int offset, · line 8218

Quoted 8 lines
fs/btrfs/inode.c +2 -3
@@ -8208,9 +8214,10 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                end = min(page_end,
                          ordered->file_offset + ordered->num_bytes - 1);
                /*
-                * IO on this page will never be started, so we need to account
-                * for any ordered extents now. Don't clear EXTENT_DELALLOC_NEW
-                * here, must leave that up for the ordered extent completion.
+                * IO on this ordered extent will never be started, so we need
+                * to account for any ordered extents now. Don't clear

So this comment update states that "IO on this ordered extent will never be started".
That is not true, unless some other patch not in misc-next changed something and I missed it (like splitting ordered extents).

If you have a 1M ordered extent, for file range [0, 1M[ for example, and then truncate the file to 512K or punch a hole for the range [512K, 1M[, then IO for the first 512K of the ordered extent is still done.

So I think what you wanted to say is more like "IO for this subpage will never be started ...".

Quoted 16 lines
fs/btrfs/inode.c +4 -1
+                * EXTENT_DELALLOC_NEW here, must leave that up for the
+                * ordered extent completion.
                 */
                if (!inode_evicting)
                        clear_extent_bit(tree, start, end,
@@ -8234,7 +8241,8 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                                                           start,
                                                           end - start + 1, 1)) {
                                btrfs_finish_ordered_io(ordered);
-                               completed_ordered = true;
+                       } else {
+                               incompleted_ordered = true;
                        }
                }
 

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page *page, unsigned int offset, · line 8287

Quoted 5 lines
fs/btrfs/inode.c +1 -1
@@ -8276,7 +8284,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                 * is cleared if we don't delete, otherwise it can lead to
                 * corruptions if the i_size is extented later.
                 */
-               if (found_ordered && !completed_ordered)
+               if (found_ordered && incompleted_ordered)

I find this naming, "incompleted_ordered" confusing, I think "incompleted" is not even a valid english word.

What you mean is that if there is some ordered extent for the page range that we could not complete ourselves.
I would suggest naming it to "completed_all_ordered", initialize it to true and then set it to false when we can't complete an ordered extent ourselves.

Then it would just be "if (found_ordered && !completed_all_ordered) delete = false;".

Also, I haven't checked the other patchsets for subpage support, but from looking only at this patchset, I'm assuming we can't set ranges in the io tree smaller than page size, is that correct? Otherwise this would be calling clear_extent_bit for each subpage range.

Thanks.

Quoted 14 lines
fs/btrfs/inode.c +1 -1
                        delete = false;
                clear_extent_bit(tree, page_start, page_end, EXTENT_LOCKED |
                                 EXTENT_DELALLOC | EXTENT_UPTODATE |
@@ -8286,6 +8294,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                __btrfs_releasepage(page, GFP_NOFS);
        }
 
+       ClearPagePrivate2(page);
        ClearPageChecked(page);
        clear_page_extent_mapped(page);
 }
--

2.29.2

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
#9 lore raw

Re: [PATCH 2/4] btrfs: inode: remove variable shadowing in btrfs_invalidatepage()

In reply to Qu Wenruo
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · Nikolay Borisov
Quoted 10 lines · Nikolay Borisov
Quoted 4 lines · Qu Wenruo

In btrfs_invalidatepage() we re-declare @tree variable as btrfs_ordered_inode_tree.

Remove such variable shadowing which can be very confusing.

You can't do that, because lock_extent_bits expects extent_io_tree !

Quoted 7 lines
Quoted 4 lines

Ok, nvm, you just factored the var at the beginning of the functions. OTOH since the ordered tree is used just for lock/unlock why not do spin_(un)lock(&inode->ordered_tree->lock);

Oh, that indeed looks better and since Su is also complaining about the declaration at the beginning of the function, I guess that's the better way to go.

The preferred style is to declare variables in the closest scope, so there's not a huge blob of declarations that are randomly used inside nested blocks. It's more like a recommendation, eg. when the function is short and there are a few variables used inside a for/while.

#14 lore raw

Re: [PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

In reply to Qu Wenruo
Quoted 56 lines · Qu Wenruo

[BUG]
With current subpage RW patchset, the following script can lead to filesystem hang:

   # mkfs.btrfs -f -s 4k $dev
   # mount $dev -o nospace_cache $mnt
   # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

The file system will hang at wait_event() of
btrfs_start_ordered_extent().

[CAUSE]
The root cause is, btrfs_invalidatepage() is freeing page::private which still has subpage dirty bit set.

The offending situation happens like this:
btrfs_fllocate()
|- btrfs_zero_range()
|- btrfs_punch_hole_lock_range()
|- truncate_pagecache_range()
|- btrfs_invalidatepage()

The involved range looks like:

0 32K 64K 96K 128K

	|///////||//////|
	| Range to drop |

For the [32K, 64K) range, since the offset is 32K, the page won't be invalidated.

But for the [64K, 96K) range, the offset is 0, current btrfs_invalidatepage() will call clear_page_extent_mapped() which will detach page::private, making the subpage dirty bitmap being cleared.

This prevents later __extent_writepage_io() to locate any range to write, thus no way to wake up the ordered extents.

[FIX]
To fix the problem this patch will:
- Only clear page status and detach page private when the full page is invalidated

- Change how we handle unfinished ordered extent
If there is any ordered extent unfinished in the page range, we can't call clear_extent_bit() with delete == true.

[REASON FOR RFC]
There is still uncertainty around the btrfs_releasepage() call.

1. Why we need btrfs_releasepage() call for non-full-page condition? Other fs (aka. xfs) just exit without doing special handling if invalidatepage() is called with part of the page.

Thus I didn't completely understand why btrfs_releasepage() here is needed for non-full page call.

Quoted 8 lines

2. Why "if (offset)" is not causing problem for current code? This existing if (offset) call can be skipped for cases like offset == 0 length == 2K.
As MM layer can call invalidatepage() with unaligned offset/length, for cases like truncate_inode_pages_range().
This will make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page.

Are we ever calling with a different length when pagesize == sectorsize? That's probably why it works fine now.

But I think we should follow what all the other file systems do, if len != PAGE_SIZE || offset != 0 then just skip it, that would probably be easier and work for you as well? Thanks,

Josef

#15 lore raw

Re: [PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

Quoted 66 lines · Josef Bacik
Quoted 56 lines · Qu Wenruo

[BUG]
With current subpage RW patchset, the following script can lead to filesystem hang:

   # mkfs.btrfs -f -s 4k $dev
   # mount $dev -o nospace_cache $mnt
   # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

The file system will hang at wait_event() of
btrfs_start_ordered_extent().

[CAUSE]
The root cause is, btrfs_invalidatepage() is freeing page::private which still has subpage dirty bit set.

The offending situation happens like this:
btrfs_fllocate()
|- btrfs_zero_range()
    |- btrfs_punch_hole_lock_range()
       |- truncate_pagecache_range()
          |- btrfs_invalidatepage()

The involved range looks like:

0    32K    64K    96K    128K

    |///////||//////|
    | Range to drop |

For the [32K, 64K) range, since the offset is 32K, the page won't be invalidated.

But for the [64K, 96K) range, the offset is 0, current btrfs_invalidatepage() will call clear_page_extent_mapped() which will detach page::private, making the subpage dirty bitmap being cleared.

This prevents later __extent_writepage_io() to locate any range to write, thus no way to wake up the ordered extents.

[FIX]
To fix the problem this patch will:
- Only clear page status and detach page private when the full page is invalidated

- Change how we handle unfinished ordered extent
   If there is any ordered extent unfinished in the page range, we can't call clear_extent_bit() with delete == true.

[REASON FOR RFC]
There is still uncertainty around the btrfs_releasepage() call.

1. Why we need btrfs_releasepage() call for non-full-page condition? Other fs (aka. xfs) just exit without doing special handling if invalidatepage() is called with part of the page.

    Thus I didn't completely understand why btrfs_releasepage() here is needed for non-full page call.

Quoted 8 lines

2. Why "if (offset)" is not causing problem for current code? This existing if (offset) call can be skipped for cases like offset == 0 length == 2K.
    As MM layer can call invalidatepage() with unaligned offset/length, for cases like truncate_inode_pages_range().
    This will make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page.

Quoted 2 lines

Are we ever calling with a different length when pagesize == sectorsize?  That's probably why it works fine now.

The range passed in can be unaligned at all.

MM layer functions like truncate_inode_pages_range() relies on that.

That's why I'm wondering why the current code is working.

As for start == 0 and length != PAGE_SIZE case it may clear the Private2/Checked bit unintentionally.

Or is that CoW fixup saving the problem?

Quoted 4 lines

But I think we should follow what all the other file systems do, if len != PAGE_SIZE || offset != 0 then just skip it, that would probably be easier and work for you as well?  Thanks,

Definitely it would work for me.

Thanks,
Qu

Quoted 2 lines

Josef

#13 lore raw

Re: [PATCH RFC 4/4] btrfs: inode: make btrfs_invalidatepage() to be subpage compatible

In reply to Filipe Manana
Quoted 66 lines · Filipe Manana
Quoted 57 lines · Qu Wenruo

[BUG]
With current subpage RW patchset, the following script can lead to filesystem hang:

   # mkfs.btrfs -f -s 4k $dev
   # mount $dev -o nospace_cache $mnt
   # fsstress -w -n 100 -p 1 -s 1608140256 -v -d $mnt

The file system will hang at wait_event() of
btrfs_start_ordered_extent().

[CAUSE]
The root cause is, btrfs_invalidatepage() is freeing page::private which still has subpage dirty bit set.

The offending situation happens like this:
btrfs_fllocate()
|- btrfs_zero_range()
|- btrfs_punch_hole_lock_range()
|- truncate_pagecache_range()
|- btrfs_invalidatepage()

The involved range looks like:

0 32K 64K 96K 128K

         |///////||//////|
         | Range to drop |

For the [32K, 64K) range, since the offset is 32K, the page won't be invalidated.

But for the [64K, 96K) range, the offset is 0, current btrfs_invalidatepage() will call clear_page_extent_mapped() which will detach page::private, making the subpage dirty bitmap being cleared.

This prevents later __extent_writepage_io() to locate any range to write, thus no way to wake up the ordered extents.

[FIX]
To fix the problem this patch will:
- Only clear page status and detach page private when the full page is invalidated

- Change how we handle unfinished ordered extent
If there is any ordered extent unfinished in the page range, we can't call clear_extent_bit() with delete == true.

[REASON FOR RFC]
There is still uncertainty around the btrfs_releasepage() call.

1. Why we need btrfs_releasepage() call for non-full-page condition? Other fs (aka. xfs) just exit without doing special handling if invalidatepage() is called with part of the page.

Thus I didn't completely understand why btrfs_releasepage() here is needed for non-full page call.

Quoted 7 lines

2. Why "if (offset)" is not causing problem for current code? This existing if (offset) call can be skipped for cases like offset == 0 length == 2K.
As MM layer can call invalidatepage() with unaligned offset/length, for cases like truncate_inode_pages_range().
This will make btrfs_invalidatepage() to truncate the whole page when we only need to zero part of the page.

Quoted 3 lines

I don't think you can ever get offset == 0 and length < PAGE_SIZE unless this is the last page in the file, the one containing eof, in which it's perfectly valid to invalidate the whole page.

You're right.

After more testing, it indeed shows except setsize call which could pass unaligned range, all other call sites are using sector aligned ranges.

Thus the existing code won't cause any problem for current code base.

Either we're invalidating the last page of the inode, or we're invalidating sector (PAGE) aligned range.

But for subpage support, we can have cases like btrfs_punch_hole_lock_range() which only passes sector aligned range in, and since sector size is smaller than page size, we can have offset == 0 while length < PAGE_SIZE and it's not the last page.

Further more, the page can have dirty range not covered by the invalidatepage range, causing the problem.

I'll update the commit message to explain the case more, and only put the fix into the subpage series, other than sending it out without subpage context.

Quoted 50 lines
Quoted 36 lines
---
  fs/btrfs/inode.c | 23 ++++++++++++++++-------
  1 file changed, 16 insertions(+), 7 deletions(-)
fs/btrfs/inode.c +8 -2
index eb493fbb65f9..872c5309b4ca 100644
@@ -8180,7 +8180,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
         int inode_evicting = inode->vfs_inode.i_state & I_FREEING;
         bool cleared_private2;
         bool found_ordered = false;
-       bool completed_ordered = false;
+       bool incompleted_ordered = false;
 
         /*
          * we have the page locked, so new writeback can't start,
@@ -8191,7 +8191,13 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
          */
         wait_on_page_writeback(page);
 
-       if (offset) {
+       /*
+        * The range doesn't cover the full page, just let btrfs_releasepage() to
+        * check if we can release the extent mapping.
+        * Any locked/pinned/logged extent map would prevent us freeing the
+        * extent mapping.
+        */
+       if (!(offset == 0 && length == PAGE_SIZE)) {
                 btrfs_releasepage(page, GFP_NOFS);
                 return;
         }

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page *page, unsigned int offset, · line 8218

Quoted 8 lines
fs/btrfs/inode.c +2 -3
@@ -8208,9 +8214,10 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                 end = min(page_end,
                           ordered->file_offset + ordered->num_bytes - 1);
                 /*
-                * IO on this page will never be started, so we need to account
-                * for any ordered extents now. Don't clear EXTENT_DELALLOC_NEW
-                * here, must leave that up for the ordered extent completion.
+                * IO on this ordered extent will never be started, so we need
+                * to account for any ordered extents now. Don't clear

So this comment update states that "IO on this ordered extent will never be started".
That is not true, unless some other patch not in misc-next changed something and I missed it (like splitting ordered extents).

Quoted 7 lines

If you have a 1M ordered extent, for file range [0, 1M[ for example, and then truncate the file to 512K or punch a hole for the range [512K, 1M[, then IO for the first 512K of the ordered extent is still done.

So I think what you wanted to say is more like "IO for this subpage will never be started ...".

This is indeed much better.

Quoted 32 lines
Quoted 17 lines
fs/btrfs/inode.c +4 -1
+                * EXTENT_DELALLOC_NEW here, must leave that up for the
+                * ordered extent completion.
                  */
                 if (!inode_evicting)
                         clear_extent_bit(tree, start, end,
@@ -8234,7 +8241,8 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                                                            start,
                                                            end - start + 1, 1)) {
                                 btrfs_finish_ordered_io(ordered);
-                               completed_ordered = true;
+                       } else {
+                               incompleted_ordered = true;
                         }
                 }
 

fs/btrfs/inode.c static void btrfs_invalidatepage(struct page *page, unsigned int offset, · line 8287

Quoted 5 lines
fs/btrfs/inode.c +1 -1
@@ -8276,7 +8284,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                  * is cleared if we don't delete, otherwise it can lead to
                  * corruptions if the i_size is extented later.
                  */
-               if (found_ordered && !completed_ordered)
+               if (found_ordered && incompleted_ordered)

I find this naming, "incompleted_ordered" confusing, I think "incompleted" is not even a valid english word.

What you mean is that if there is some ordered extent for the page range that we could not complete ourselves.
I would suggest naming it to "completed_all_ordered", initialize it to true and then set it to false when we can't complete an ordered extent ourselves.

Quoted 7 lines

Then it would just be "if (found_ordered && !completed_all_ordered) delete = false;".

Also, I haven't checked the other patchsets for subpage support, but from looking only at this patchset, I'm assuming we can't set ranges in the io tree smaller than page size, is that correct? Otherwise this would be calling clear_extent_bit for each subpage range.

For current subpage implementation, we have to support sector aligned writeback.

The requirement comes from data balance, we have to be able to write new data extents exactly the same size as the originals.

Thus here we support range smaller than page size for subpage. The extent io tree itself can support it without problem.

Thanks,
Qu

Quoted 19 lines

Thanks.

Quoted 14 lines
fs/btrfs/inode.c +1 -1
                         delete = false;
                 clear_extent_bit(tree, page_start, page_end, EXTENT_LOCKED |
                                  EXTENT_DELALLOC | EXTENT_UPTODATE |
@@ -8286,6 +8294,7 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
                 __btrfs_releasepage(page, GFP_NOFS);
         }
 
+       ClearPagePrivate2(page);
         ClearPageChecked(page);
         clear_page_extent_mapped(page);
  }
--

2.29.2