Skip to contentSkip to conversation

fs/btrfs: Fix raid6 qstripe kmap'ing

PATCH In review [email protected] 2 participants 6 messages Jan 28, 2021 – Feb 5, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fs/btrfs: Fix raid6 qstripe kmap'ing

From: Ira Weiny <[email protected]>

When a qstripe is required an extra page is allocated and mapped. There were 3 problems.

1) There is no reason to map the qstripe page more than 1 time if the number of bits set in rbio->dbitmap is greater than one.
2) There is no reason to map the parity page and unmap it each time through the loop.
3) There is no corresponding call of kunmap() for the qstripe page.

The page memory can continue to be reused with a single mapping on each iteration by raid6_call.gen_syndrome() without remapping. So map the page for the duration of the loop.

Similarly, improve the algorithm by mapping the parity page just 1 time.

Fixes 5a6ac9eacb49 ("Btrfs, raid56: support parity scrub on raid56")

To: Chris Mason <[email protected]>
To: Josef Bacik <[email protected]>
To: David Sterba <[email protected]>

---
This was found while replacing kmap() with kmap_local_page(). After this patch unwinding all the mappings becomes pretty straight forward.

I'm not exactly sure I've worded this commit message intelligently. Please forgive me if there is a better way to word it.

---
 fs/btrfs/raid56.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)
fs/btrfs/raid56.c +10 -11
index 93fbf87bdc8d..b8a39dad0f00 100644
@@ -2363,16 +2363,21 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 	SetPageUptodate(p_page);
 
 	if (has_qstripe) {
+		/* raid6, allocate and map temp space for the qstripe */
 		q_page = alloc_page(GFP_NOFS | __GFP_HIGHMEM);
 		if (!q_page) {
 			__free_page(p_page);
 			goto cleanup;
 		}
 		SetPageUptodate(q_page);
+		pointers[rbio->real_stripes] = kmap(q_page);
 	}
 
 	atomic_set(&rbio->error, 0);
 
+	/* map the parity stripe just once */
+	pointers[nr_data] = kmap(p_page);
+
 	for_each_set_bit(pagenr, rbio->dbitmap, rbio->stripe_npages) {
 		struct page *p;
 		void *parity;
@@ -2382,16 +2387,8 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 			pointers[stripe] = kmap(p);
 		}
 
-		/* then add the parity stripe */
-		pointers[stripe++] = kmap(p_page);
-
 		if (has_qstripe) {
-			/*
-			 * raid6, add the qstripe and call the
-			 * library function to fill in our p/q
-			 */
-			pointers[stripe++] = kmap(q_page);
-
+			/* raid6, call the library function to fill in our p/q */
 			raid6_call.gen_syndrome(rbio->real_stripes, PAGE_SIZE,
 						pointers);
 		} else {
@@ -2412,12 +2409,14 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
 
 		for (stripe = 0; stripe < nr_data; stripe++)
 			kunmap(page_in_rbio(rbio, stripe, pagenr, 0));
-		kunmap(p_page);
 	}
 
+	kunmap(p_page);
 	__free_page(p_page);
-	if (q_page)
+	if (q_page) {
+		kunmap(q_page);
 		__free_page(q_page);
+	}
 
 writeback:
 	/*
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
#1 lore raw
Quoted 24 lines · On Wed, Jan 27, 2021 at 10:15:03PM -0800, [email protected] wrote:

From: Ira Weiny <[email protected]>

When a qstripe is required an extra page is allocated and mapped. There were 3 problems.

1) There is no reason to map the qstripe page more than 1 time if the number of bits set in rbio->dbitmap is greater than one.
2) There is no reason to map the parity page and unmap it each time through the loop.
3) There is no corresponding call of kunmap() for the qstripe page.

The page memory can continue to be reused with a single mapping on each iteration by raid6_call.gen_syndrome() without remapping. So map the page for the duration of the loop.

Similarly, improve the algorithm by mapping the parity page just 1 time.

Fixes 5a6ac9eacb49 ("Btrfs, raid56: support parity scrub on raid56")

To: Chris Mason <[email protected]>
To: Josef Bacik <[email protected]>
To: David Sterba <[email protected]>

Quoted 6 lines

---
This was found while replacing kmap() with kmap_local_page(). After this patch unwinding all the mappings becomes pretty straight forward.

I'm not exactly sure I've worded this commit message intelligently. Please forgive me if there is a better way to word it.

Changelog is good, thanks. I've added stable tags as the missing unmap is a potential problem.

#2 lore raw
Quoted 29 lines · David Sterba
Quoted 28 lines · On Wed, Jan 27, 2021 at 10:15:03PM -0800, [email protected] wrote:

From: Ira Weiny <[email protected]>

When a qstripe is required an extra page is allocated and mapped. There were 3 problems.

1) There is no reason to map the qstripe page more than 1 time if the number of bits set in rbio->dbitmap is greater than one.
2) There is no reason to map the parity page and unmap it each time through the loop.
3) There is no corresponding call of kunmap() for the qstripe page.

The page memory can continue to be reused with a single mapping on each iteration by raid6_call.gen_syndrome() without remapping. So map the page for the duration of the loop.

Similarly, improve the algorithm by mapping the parity page just 1 time.

Fixes 5a6ac9eacb49 ("Btrfs, raid56: support parity scrub on raid56")

To: Chris Mason <[email protected]>
To: Josef Bacik <[email protected]>
To: David Sterba <[email protected]>

---
This was found while replacing kmap() with kmap_local_page(). After this patch unwinding all the mappings becomes pretty straight forward.

Quoted 5 lines
Quoted 2 lines

I'm not exactly sure I've worded this commit message intelligently. Please forgive me if there is a better way to word it.

Changelog is good, thanks. I've added stable tags as the missing unmap is a potential problem.

There are lots of tests faling, stack traces like below. I haven't seen anything obvious in the patch so that needs a closer look and for the time being I can't add the patch to for-next.

 BUG: kernel NULL pointer dereference, address:0000000000000000
 #PF: supervisor write access in kernel mode
 #PF: error_code(0x0002) - not-present page
 PGD 0 P4D 0
 Oops: 0002 [#1] PREEMPT SMP
 CPU: 2 PID: 17173 Comm: kworker/u8:5 Not tainted5.11.0-rc6-default+ #1422
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),BIOS rel-1.12.0-59-gc9ba527-rebuilt.opensuse.org 04/01/2014
 Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
 RIP: 0010:raid6_avx22_gen_syndrome+0x103/0x140 [raid6_pq]
 RSP: 0018:ffffa090042cfcf8 EFLAGS: 00010246
 RAX: ffff9e98e1848e80 RBX: ffff9e98d5849000 RCX:0000000000000020
 RDX: ffff9e98e32be000 RSI: 0000000000000000 RDI:ffff9e98e1848e80
 RBP: 0000000000000000 R08: 0000000000000000 R09:0000000000000001
 R10: ffff9e98e1848e90 R11: ffff9e98e1848e98 R12:0000000000001000
 R13: ffff9e98e1848e88 R14: 0000000000000005 R15:0000000000000002
 FS:  0000000000000000(0000) GS:ffff9e993da00000(0000)knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000000 CR3: 0000000023143003 CR4:0000000000170ea0
 Call Trace:
  finish_parity_scrub+0x47b/0x7a0 [btrfs]
  raid56_parity_scrub_stripe+0x24e/0x260 [btrfs]
  btrfs_work_helper+0xd5/0x1d0 [btrfs]
  process_one_work+0x262/0x5f0
  worker_thread+0x4e/0x300
  ? process_one_work+0x5f0/0x5f0
  kthread+0x151/0x170
  ? __kthread_bind_mask+0x60/0x60
author
#3 lore raw
Quoted 33 lines · David Sterba
Quoted 32 lines · David Sterba
Quoted 30 lines · On Wed, Jan 27, 2021 at 10:15:03PM -0800, [email protected] wrote:

From: Ira Weiny <[email protected]>

When a qstripe is required an extra page is allocated and mapped. There were 3 problems.

1) There is no reason to map the qstripe page more than 1 time if the number of bits set in rbio->dbitmap is greater than one.
2) There is no reason to map the parity page and unmap it each time through the loop.
3) There is no corresponding call of kunmap() for the qstripe page.

The page memory can continue to be reused with a single mapping on each iteration by raid6_call.gen_syndrome() without remapping. So map the page for the duration of the loop.

Similarly, improve the algorithm by mapping the parity page just 1 time.

Fixes 5a6ac9eacb49 ("Btrfs, raid56: support parity scrub on raid56")

To: Chris Mason <[email protected]>
To: Josef Bacik <[email protected]>
To: David Sterba <[email protected]>

---
This was found while replacing kmap() with kmap_local_page(). After this patch unwinding all the mappings becomes pretty straight forward.

I'm not exactly sure I've worded this commit message intelligently. Please forgive me if there is a better way to word it.

Quoted 6 lines
Quoted 2 lines

Changelog is good, thanks. I've added stable tags as the missing unmap is a potential problem.

There are lots of tests faling, stack traces like below. I haven't seen anything obvious in the patch so that needs a closer look and for the time being I can't add the patch to for-next.

:-(

I think I may have been off by 1 on the raid6 kmap...

Something like this should fix it...

fs/btrfs/raid56.c +1 -1
index b8a39dad0f00..dbf52f1a379d 100644
@@ -2370,7 +2370,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
                        goto cleanup;
                }
                SetPageUptodate(q_page);
-               pointers[rbio->real_stripes] = kmap(q_page);
+               pointers[rbio->real_stripes - 1] = kmap(q_page);
        }
 
        atomic_set(&rbio->error, 0);

Let me roll a new version.

Sorry,
Ira

Quoted 28 lines
 BUG: kernel NULL pointer dereference, address:0000000000000000
 #PF: supervisor write access in kernel mode
 #PF: error_code(0x0002) - not-present page
 PGD 0 P4D 0
 Oops: 0002 [#1] PREEMPT SMP
 CPU: 2 PID: 17173 Comm: kworker/u8:5 Not tainted5.11.0-rc6-default+ #1422
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),BIOS rel-1.12.0-59-gc9ba527-rebuilt.opensuse.org 04/01/2014
 Workqueue: btrfs-rmw btrfs_work_helper [btrfs]
 RIP: 0010:raid6_avx22_gen_syndrome+0x103/0x140 [raid6_pq]
 RSP: 0018:ffffa090042cfcf8 EFLAGS: 00010246
 RAX: ffff9e98e1848e80 RBX: ffff9e98d5849000 RCX:0000000000000020
 RDX: ffff9e98e32be000 RSI: 0000000000000000 RDI:ffff9e98e1848e80
 RBP: 0000000000000000 R08: 0000000000000000 R09:0000000000000001
 R10: ffff9e98e1848e90 R11: ffff9e98e1848e98 R12:0000000000001000
 R13: ffff9e98e1848e88 R14: 0000000000000005 R15:0000000000000002
 FS:  0000000000000000(0000) GS:ffff9e993da00000(0000)knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000000 CR3: 0000000023143003 CR4:0000000000170ea0
 Call Trace:
  finish_parity_scrub+0x47b/0x7a0 [btrfs]
  raid56_parity_scrub_stripe+0x24e/0x260 [btrfs]
  btrfs_work_helper+0xd5/0x1d0 [btrfs]
  process_one_work+0x262/0x5f0
  worker_thread+0x4e/0x300
  ? process_one_work+0x5f0/0x5f0
  kthread+0x151/0x170
  ? __kthread_bind_mask+0x60/0x60
#4 lore raw
Quoted 22 lines · Ira Weiny
Quoted 9 lines · David Sterba
Quoted 4 lines · David Sterba
Quoted 1 line · On Wed, Jan 27, 2021 at 10:15:03PM -0800, [email protected] wrote:

From: Ira Weiny <[email protected]>

Changelog is good, thanks. I've added stable tags as the missing unmap is a potential problem.

There are lots of tests faling, stack traces like below. I haven't seen anything obvious in the patch so that needs a closer look and for the time being I can't add the patch to for-next.

:-(

I think I may have been off by 1 on the raid6 kmap...

Something like this should fix it...

fs/btrfs/raid56.c +0 -0
index b8a39dad0f00..dbf52f1a379d 100644

fs/btrfs/raid56.c static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio, · line 2373

Quoted 5 lines
fs/btrfs/raid56.c +1 -1
@@ -2370,7 +2370,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
                        goto cleanup;
                }
                SetPageUptodate(q_page);
-               pointers[rbio->real_stripes] = kmap(q_page);
+               pointers[rbio->real_stripes - 1] = kmap(q_page);

Oh right and tests agree it works.

Quoted 5 lines
fs/btrfs/raid56.c +0 -0
        }
 
        atomic_set(&rbio->error, 0);

Let me roll a new version.

No need to, I'll fold the fixup. Thanks.

author
#5 lore raw
Quoted 31 lines · David Sterba
Quoted 22 lines · Ira Weiny
Quoted 9 lines · David Sterba
Quoted 4 lines · David Sterba
Quoted 1 line · On Wed, Jan 27, 2021 at 10:15:03PM -0800, [email protected] wrote:

From: Ira Weiny <[email protected]>

Changelog is good, thanks. I've added stable tags as the missing unmap is a potential problem.

There are lots of tests faling, stack traces like below. I haven't seen anything obvious in the patch so that needs a closer look and for the time being I can't add the patch to for-next.

:-(

I think I may have been off by 1 on the raid6 kmap...

Something like this should fix it...

fs/btrfs/raid56.c +0 -0
index b8a39dad0f00..dbf52f1a379d 100644

fs/btrfs/raid56.c static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio, · line 2373

Quoted 5 lines
fs/btrfs/raid56.c +1 -1
@@ -2370,7 +2370,7 @@ static noinline void finish_parity_scrub(struct btrfs_raid_bio *rbio,
                        goto cleanup;
                }
                SetPageUptodate(q_page);
-               pointers[rbio->real_stripes] = kmap(q_page);
+               pointers[rbio->real_stripes - 1] = kmap(q_page);

Oh right and tests agree it works.

Quoted 7 lines
Quoted 5 lines
fs/btrfs/raid56.c +0 -0
        }
 
        atomic_set(&rbio->error, 0);

Let me roll a new version.

No need to, I'll fold the fixup. Thanks.

Oh cool thanks! :-D
Ira