Skip to contentSkip to conversation

FAILED: patch "[PATCH] writeback: fix use-after-free in finish_writeback_work()" failed to apply to 5.3-stable tree

PATCH [email protected] 4 participants 4 messages Oct 12, 2019 – Oct 14, 2019

Subsystems (2)
Stable branch VFS
lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

The patch below does not apply to the 5.3-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From 8e00c4e9dd852f7a9bf12234fad65a2f2f93788f Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Sun, 6 Oct 2019 17:58:09 -0700
Subject: [PATCH] writeback: fix use-after-free in finish_writeback_work()

finish_writeback_work() reads @done->waitq after decrementing @done->cnt. However, once @done->cnt reaches zero, @done may be freed (from stack) at any moment and @done->waitq can contain something unrelated by the time finish_writeback_work() tries to read it. This led to the following crash.

"BUG: kernel NULL pointer dereference, address: 0000000000000002" #PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
PGD 0 P4D 0

  Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
  CPU: 40 PID: 555153 Comm: kworker/u98:50 Kdump: loaded Not tainted
  ...
  Workqueue: writeback wb_workfn (flush-btrfs-1)
  RIP: 0010:_raw_spin_lock_irqsave+0x10/0x30
  Code: 48 89 d8 5b c3 e8 50 db 6b ff eb f4 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 9c 5b fa 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 05 48 89 d8 5b c3 89 c6 e8 fe ca 6b ff eb f2 66 90
  RSP: 0018:ffffc90049b27d98 EFLAGS: 00010046
  RAX: 0000000000000000 RBX: 0000000000000246 RCX: 0000000000000000
  RDX: 0000000000000001 RSI: 0000000000000003 RDI: 0000000000000002
  RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000001
  R10: ffff889fff407600 R11: ffff88ba9395d740 R12: 000000000000e300
  R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000000000
  FS:  0000000000000000(0000) GS:ffff88bfdfa00000(0000) knlGS:0000000000000000
  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  CR2: 0000000000000002 CR3: 0000000002409005 CR4: 00000000001606e0
  Call Trace:
   __wake_up_common_lock+0x63/0xc0
   wb_workfn+0xd2/0x3e0
   process_one_work+0x1f5/0x3f0
   worker_thread+0x2d/0x3d0
   kthread+0x111/0x130
   ret_from_fork+0x1f/0x30

Fix it by reading and caching @done->waitq before decrementing @done->cnt.

Link http://lkml.kernel.org/r/[email protected] open here
Fixes 5b9cce4c7eb069 ("writeback: Generalize and expose wb_completion")
Signed-off-by Tejun Heo <[email protected]>
Debugged-by Chris Mason <[email protected]>
Reviewed-by Jens Axboe <[email protected]>
Cc Jan Kara <[email protected]>
Cc <[email protected]> [5.2+]
Signed-off-by Andrew Morton <[email protected]>
Signed-off-by Linus Torvalds <[email protected]>
fs/fs-writeback.c +7 -2
index 8aaa7eec7b74..e88421d9a48d 100644
@@ -164,8 +164,13 @@ static void finish_writeback_work(struct bdi_writeback *wb,
 
 	if (work->auto_free)
 		kfree(work);
-	if (done && atomic_dec_and_test(&done->cnt))
-		wake_up_all(done->waitq);
+	if (done) {
+		wait_queue_head_t *waitq = done->waitq;
+
+		/* @done can't be accessed after the following dec */
+		if (atomic_dec_and_test(&done->cnt))
+			wake_up_all(waitq);
+	}
 }
 
 static void wb_queue_work(struct bdi_writeback *wb,
maintainer
#1 lore raw
Quoted 50 lines · On Sat, Oct 12, 2019 at 10:17:58AM +0200, [email protected] wrote:

The patch below does not apply to the 5.3-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From 8e00c4e9dd852f7a9bf12234fad65a2f2f93788f Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Sun, 6 Oct 2019 17:58:09 -0700
Subject: [PATCH] writeback: fix use-after-free in finish_writeback_work()

finish_writeback_work() reads @done->waitq after decrementing @done->cnt. However, once @done->cnt reaches zero, @done may be freed (from stack) at any moment and @done->waitq can contain something unrelated by the time finish_writeback_work() tries to read it. This led to the following crash.

"BUG: kernel NULL pointer dereference, address: 0000000000000002" #PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
PGD 0 P4D 0

 Oops: 0002 [#1] SMP DEBUG_PAGEALLOC
 CPU: 40 PID: 555153 Comm: kworker/u98:50 Kdump: loaded Not tainted
 ...
 Workqueue: writeback wb_workfn (flush-btrfs-1)
 RIP: 0010:_raw_spin_lock_irqsave+0x10/0x30
 Code: 48 89 d8 5b c3 e8 50 db 6b ff eb f4 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 9c 5b fa 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 05 48 89 d8 5b c3 89 c6 e8 fe ca 6b ff eb f2 66 90
 RSP: 0018:ffffc90049b27d98 EFLAGS: 00010046
 RAX: 0000000000000000 RBX: 0000000000000246 RCX: 0000000000000000
 RDX: 0000000000000001 RSI: 0000000000000003 RDI: 0000000000000002
 RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000001
 R10: ffff889fff407600 R11: ffff88ba9395d740 R12: 000000000000e300
 R13: 0000000000000003 R14: 0000000000000000 R15: 0000000000000000
 FS:  0000000000000000(0000) GS:ffff88bfdfa00000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000002 CR3: 0000000002409005 CR4: 00000000001606e0
 Call Trace:
  __wake_up_common_lock+0x63/0xc0
  wb_workfn+0xd2/0x3e0
  process_one_work+0x1f5/0x3f0
  worker_thread+0x2d/0x3d0
  kthread+0x111/0x130
  ret_from_fork+0x1f/0x30
Quoted 5 lines

Fix it by reading and caching @done->waitq before decrementing @done->cnt.

Link http://lkml.kernel.org/r/[email protected] open here
Fixes 5b9cce4c7eb069 ("writeback: Generalize and expose wb_completion")

Hm... 5b9cce4c7eb069 went upstream during the 5.4 merge window, but:

Quoted 5 lines
Signed-off-by Tejun Heo <[email protected]>
Debugged-by Chris Mason <[email protected]>
Reviewed-by Jens Axboe <[email protected]>
Cc Jan Kara <[email protected]>
Cc <[email protected]> [5.2+]

This tag says that 8e00c4e9dd85 should be backported to 5.3.

Signature · 2 lines
Thanks,
Sasha
#2 lore raw
In reply to [email protected]
Quoted 5 lines · On Sat, Oct 12, 2019 at 1:18 AM <[email protected]> wrote:

The patch below does not apply to the 5.3-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

Hmm. It looks to me like 5.3 doesn't have the use-after-free problem, and that it was introduced by 5b9cce4c7eb0696558 ("writeback:
Generalize and expose wb_completion")

So I think the "Fixes" tag is right, and the "Cc: stable" is wrong.

Linus

#3 lore raw
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · On Sat, Oct 12, 2019 at 1:18 AM <[email protected]> wrote:

The patch below does not apply to the 5.3-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

Quoted 5 lines

Hmm. It looks to me like 5.3 doesn't have the use-after-free problem, and that it was introduced by 5b9cce4c7eb0696558 ("writeback:
Generalize and expose wb_completion")

So I think the "Fixes" tag is right, and the "Cc: stable" is wrong.

Yeap, sorry about that. I think I forgot --contains when running git-describe and mindlessly copied the output.

Thanks.

Signature · 1 line
tejun