Skip to contentSkip to conversation

btrfs: fix use-after-free in btrfs_encoded_read_endio

PATCH Superseded Johannes Thumshirn 5 participants 9 messages Nov 12, 2024 – Nov 25, 2024 Btrfs file system

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover btrfs: fix use-after-free in btrfs_encoded_read_endio

Shinichiro reported a occassional memory corruption in our CI system with btrfs/248 that lead to panics. He also managed to reproduce this corruption reliably on one host. See patch 1/2 for details on the corruption and the fix, patch 2/2 is a cleanup Damien suggested on top of the fix to make the code more obvious.

Changes to v1:
- Update commit message of patch 1/1
- Prevent double-free of 'priv' in case of io_uring in 2/2
- Use wait_for_completion_io() in 2/2
- Convert priv->pending from atomic_t to refcount_t calling it refs in 2/2

Link to v1:
https://lore.kernel.org/linux-btrfs/[email protected]

Johannes Thumshirn (2):
btrfs: fix use-after-free in btrfs_encoded_read_endio
btrfs: simplify waiting for encoded read endios

 fs/btrfs/inode.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/2 btrfs: fix use-after-free in btrfs_encoded_read_endio

From: Johannes Thumshirn <[email protected]>

Shinichiro reported the following use-after free that sometimes is happening in our CI system when running fstests' btrfs/284 on a TCMU runner device:

==================================================================

   BUG: KASAN: slab-use-after-free in lock_release+0x708/0x780
   Read of size 8 at addr ffff888106a83f18 by task kworker/u80:6/219

   CPU: 8 UID: 0 PID: 219 Comm: kworker/u80:6 Not tainted 6.12.0-rc6-kts+ #15
   Hardware name: Supermicro Super Server/X11SPi-TF, BIOS 3.3 02/21/2020
   Workqueue: btrfs-endio btrfs_end_bio_work [btrfs]
   Call Trace:
    <TASK>
    dump_stack_lvl+0x6e/0xa0
    ? lock_release+0x708/0x780
    print_report+0x174/0x505
    ? lock_release+0x708/0x780
    ? __virt_addr_valid+0x224/0x410
    ? lock_release+0x708/0x780
    kasan_report+0xda/0x1b0
    ? lock_release+0x708/0x780
    ? __wake_up+0x44/0x60
    lock_release+0x708/0x780
    ? __pfx_lock_release+0x10/0x10
    ? __pfx_do_raw_spin_lock+0x10/0x10
    ? lock_is_held_type+0x9a/0x110
    _raw_spin_unlock_irqrestore+0x1f/0x60
    __wake_up+0x44/0x60
    btrfs_encoded_read_endio+0x14b/0x190 [btrfs]
    btrfs_check_read_bio+0x8d9/0x1360 [btrfs]
    ? lock_release+0x1b0/0x780
    ? trace_lock_acquire+0x12f/0x1a0
    ? __pfx_btrfs_check_read_bio+0x10/0x10 [btrfs]
    ? process_one_work+0x7e3/0x1460
    ? lock_acquire+0x31/0xc0
    ? process_one_work+0x7e3/0x1460
    process_one_work+0x85c/0x1460
    ? __pfx_process_one_work+0x10/0x10
    ? assign_work+0x16c/0x240
    worker_thread+0x5e6/0xfc0
    ? __pfx_worker_thread+0x10/0x10
    kthread+0x2c3/0x3a0
    ? __pfx_kthread+0x10/0x10
    ret_from_fork+0x31/0x70
    ? __pfx_kthread+0x10/0x10
    ret_from_fork_asm+0x1a/0x30
    </TASK>

   Allocated by task 3661:
    kasan_save_stack+0x30/0x50
    kasan_save_track+0x14/0x30
    __kasan_kmalloc+0xaa/0xb0
    btrfs_encoded_read_regular_fill_pages+0x16c/0x6d0 [btrfs]
    send_extent_data+0xf0f/0x24a0 [btrfs]
    process_extent+0x48a/0x1830 [btrfs]
    changed_cb+0x178b/0x2ea0 [btrfs]
    btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
    _btrfs_ioctl_send+0x117/0x330 [btrfs]
    btrfs_ioctl+0x184a/0x60a0 [btrfs]
    __x64_sys_ioctl+0x12e/0x1a0
    do_syscall_64+0x95/0x180
    entry_SYSCALL_64_after_hwframe+0x76/0x7e

   Freed by task 3661:
    kasan_save_stack+0x30/0x50
    kasan_save_track+0x14/0x30
    kasan_save_free_info+0x3b/0x70
    __kasan_slab_free+0x4f/0x70
    kfree+0x143/0x490
    btrfs_encoded_read_regular_fill_pages+0x531/0x6d0 [btrfs]
    send_extent_data+0xf0f/0x24a0 [btrfs]
    process_extent+0x48a/0x1830 [btrfs]
    changed_cb+0x178b/0x2ea0 [btrfs]
    btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
    _btrfs_ioctl_send+0x117/0x330 [btrfs]
    btrfs_ioctl+0x184a/0x60a0 [btrfs]
    __x64_sys_ioctl+0x12e/0x1a0
    do_syscall_64+0x95/0x180
    entry_SYSCALL_64_after_hwframe+0x76/0x7e

   The buggy address belongs to the object at ffff888106a83f00
    which belongs to the cache kmalloc-rnd-07-96 of size 96
   The buggy address is located 24 bytes inside of
    freed 96-byte region [ffff888106a83f00, ffff888106a83f60)

   The buggy address belongs to the physical page:
   page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888106a83800 pfn:0x106a83
   flags: 0x17ffffc0000000(node=0|zone=2|lastcpupid=0x1fffff)
   page_type: f5(slab)
   raw: 0017ffffc0000000 ffff888100053680 ffffea0004917200 0000000000000004
   raw: ffff888106a83800 0000000080200019 00000001f5000000 0000000000000000
   page dumped because: kasan: bad access detected

   Memory state around the buggy address:
    ffff888106a83e00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
    ffff888106a83e80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
   >ffff888106a83f00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
                               ^
    ffff888106a83f80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
    ffff888106a84000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   ==================================================================

Further analyzing the trace and the crash dump's vmcore file shows that the wake_up() call in btrfs_encoded_read_endio() is calling wake_up() on the wait_queue that is in the private data passed to the end_io handler.

Commit 4ff47df40447 ("btrfs: move priv off stack in btrfs_encoded_read_regular_fill_pages()") moved 'struct btrfs_encoded_read_private' off the stack.

Before that commit one can see a corruption of the private data when analyzing the vmcore after a crash:

*(struct btrfs_encoded_read_private *)0xffff88815626eec8 = {
	.wait = (wait_queue_head_t){
		.lock = (spinlock_t){
			.rlock = (struct raw_spinlock){
				.raw_lock = (arch_spinlock_t){
					.val = (atomic_t){
						.counter = (int)-2005885696,
					},
					.locked = (u8)0,
					.pending = (u8)157,
					.locked_pending = (u16)40192,
					.tail = (u16)34928,
				},
				.magic = (unsigned int)536325682,
				.owner_cpu = (unsigned int)29,
				.owner = (void *)__SCT__tp_func_btrfs_transaction_commit+0x0 = 0x0,
				.dep_map = (struct lockdep_map){
					.key = (struct lock_class_key *)0xffff8881575a3b6c,
					.class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
					.name = (const char *)0xffff88815626f100 = "",
					.wait_type_outer = (u8)37,
					.wait_type_inner = (u8)178,
					.lock_type = (u8)154,
				},
			},
			.__padding = (u8 [24]){ 0, 157, 112, 136, 50, 174, 247, 31, 29 },
			.dep_map = (struct lockdep_map){
				.key = (struct lock_class_key *)0xffff8881575a3b6c,
				.class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
				.name = (const char *)0xffff88815626f100 = "",
				.wait_type_outer = (u8)37,
				.wait_type_inner = (u8)178,
				.lock_type = (u8)154,
			},
		},
		.head = (struct list_head){
			.next = (struct list_head *)0x112cca,
			.prev = (struct list_head *)0x47,
		},
	},
	.pending = (atomic_t){
		.counter = (int)-1491499288,
	},
	.status = (blk_status_t)130,
}

Here we can see several indicators of in-memory data corruption, e.g. the large negative atomic values of ->pending or ->wait->lock->rlock->raw_lock->val, as well as the bogus spinlock magic 0x1ff7ae32 (decimal 536325682 above) instead of 0xdead4ead or the bogus pointer values for ->wait->head.

To fix this, move the call to bio_put() before the atomic_test operation so the submitter side in btrfs_encoded_read_regular_fill_pages() is not woken up before the bio is cleaned up.

Also change atomic_dec_return() to atomic_dec_and_test() to fix the corruption, as atomic_dec_return() is defined as two instructions on x86_64, whereas atomic_dec_and_test() is defined as a single atomic operation. This can lead to a situation where counter value is already decremented but the if statement in btrfs_encoded_read_endio() is not completely processed, i.e. the 0 test has not completed. If another thread continues executing btrfs_encoded_read_regular_fill_pages() the atomic_dec_return() there can see an already updated ->pending counter and continues by freeing the private data. Continuing in the endio handler the test for 0 succeeds and the wait_queue is woken up, resulting in a use-after-free.

Fixes 1881fba89bd5 ("btrfs: add BTRFS_IOC_ENCODED_READ ioctl")
---
 fs/btrfs/inode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +2 -2
index 22b8e2764619..cb8b23a3e56b 100644
@@ -9089,7 +9089,8 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
 		 */
 		WRITE_ONCE(priv->status, bbio->bio.bi_status);
 	}
-	if (atomic_dec_return(&priv->pending) == 0) {
+	bio_put(&bbio->bio);
+	if (atomic_dec_and_test(&priv->pending)) {
 		int err = blk_status_to_errno(READ_ONCE(priv->status));
 
 		if (priv->uring_ctx) {
@@ -9099,7 +9100,6 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
 			wake_up(&priv->wait);
 		}
 	}
-	bio_put(&bbio->bio);
 }
 
 int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
Signature · 1 line
2.43.0
#7 lore raw

PATCH v2 2/2 btrfs: simplify waiting for encoded read endios

In reply to Johannes Thumshirn

From: Johannes Thumshirn <[email protected]>

Simplify the I/O completion path for encoded reads by using a completion instead of a wait_queue.

Furthermore skip taking an extra reference that is instantly dropped anyways and convert btrfs_encoded_read_private::pending into a refcount_t filed instead of atomic_t.

Freeing of the private data is now handled at a common place in btrfs_encoded_read_regular_fill_pages() and if btrfs_encoded_read_endio() is freeing the data in case it has an io_uring context associated, the btrfs_bio's private filed is NULLed to avoid a double free of the private data.

---
 fs/btrfs/inode.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)
fs/btrfs/inode.c +16 -15
index cb8b23a3e56b..3093905364e5 100644
@@ -9068,9 +9068,9 @@ static ssize_t btrfs_encoded_read_inline(
 }
 
 struct btrfs_encoded_read_private {
-	wait_queue_head_t wait;
+	struct completion done;
 	void *uring_ctx;
-	atomic_t pending;
+	refcount_t refs;
 	blk_status_t status;
 };
 
@@ -9090,14 +9090,15 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
 		WRITE_ONCE(priv->status, bbio->bio.bi_status);
 	}
 	bio_put(&bbio->bio);
-	if (atomic_dec_and_test(&priv->pending)) {
+	if (refcount_dec_and_test(&priv->refs)) {
 		int err = blk_status_to_errno(READ_ONCE(priv->status));
 
 		if (priv->uring_ctx) {
 			btrfs_uring_read_extent_endio(priv->uring_ctx, err);
+			bbio->private = NULL;
 			kfree(priv);
 		} else {
-			wake_up(&priv->wait);
+			complete(&priv->done);
 		}
 	}
 }
@@ -9116,8 +9117,8 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
 	if (!priv)
 		return -ENOMEM;
 
-	init_waitqueue_head(&priv->wait);
-	atomic_set(&priv->pending, 1);
+	init_completion(&priv->done);
+	refcount_set(&priv->refs, 1);
 	priv->status = 0;
 	priv->uring_ctx = uring_ctx;
 
@@ -9130,7 +9131,7 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
 		size_t bytes = min_t(u64, disk_io_size, PAGE_SIZE);
 
 		if (bio_add_page(&bbio->bio, pages[i], bytes, 0) < bytes) {
-			atomic_inc(&priv->pending);
+			refcount_inc(&priv->refs);
 			btrfs_submit_bbio(bbio, 0);
 
 			bbio = btrfs_bio_alloc(BIO_MAX_VECS, REQ_OP_READ, fs_info,
@@ -9145,26 +9146,26 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
 		disk_io_size -= bytes;
 	} while (disk_io_size);
 
-	atomic_inc(&priv->pending);
 	btrfs_submit_bbio(bbio, 0);
 
 	if (uring_ctx) {
-		if (atomic_dec_return(&priv->pending) == 0) {
+		if (bbio->private && refcount_read(&priv->refs) == 0) {
 			ret = blk_status_to_errno(READ_ONCE(priv->status));
 			btrfs_uring_read_extent_endio(uring_ctx, ret);
-			kfree(priv);
-			return ret;
+			goto out;
 		}
 
 		return -EIOCBQUEUED;
 	} else {
-		if (atomic_dec_return(&priv->pending) != 0)
-			io_wait_event(priv->wait, !atomic_read(&priv->pending));
+		wait_for_completion_io(&priv->done);
 		/* See btrfs_encoded_read_endio() for ordering. */
 		ret = blk_status_to_errno(READ_ONCE(priv->status));
-		kfree(priv);
-		return ret;
 	}
+
+out:
+	kfree(priv);
+	return ret;
+
 }
 
 ssize_t btrfs_encoded_read_regular(struct kiocb *iocb, struct iov_iter *iter,
Signature · 1 line
2.43.0
Reviewed-by on v2 1/2
#2 lore raw
In reply to Johannes Thumshirn
Quoted 168 lines · Johannes Thumshirn

From: Johannes Thumshirn <[email protected]>

Shinichiro reported the following use-after free that sometimes is happening in our CI system when running fstests' btrfs/284 on a TCMU runner device:

   ==================================================================
   BUG: KASAN: slab-use-after-free in lock_release+0x708/0x780
   Read of size 8 at addr ffff888106a83f18 by task kworker/u80:6/219

   CPU: 8 UID: 0 PID: 219 Comm: kworker/u80:6 Not tainted 6.12.0-rc6-kts+ #15
   Hardware name: Supermicro Super Server/X11SPi-TF, BIOS 3.3 02/21/2020
   Workqueue: btrfs-endio btrfs_end_bio_work [btrfs]
   Call Trace:
    <TASK>
    dump_stack_lvl+0x6e/0xa0
    ? lock_release+0x708/0x780
    print_report+0x174/0x505
    ? lock_release+0x708/0x780
    ? __virt_addr_valid+0x224/0x410
    ? lock_release+0x708/0x780
    kasan_report+0xda/0x1b0
    ? lock_release+0x708/0x780
    ? __wake_up+0x44/0x60
    lock_release+0x708/0x780
    ? __pfx_lock_release+0x10/0x10
    ? __pfx_do_raw_spin_lock+0x10/0x10
    ? lock_is_held_type+0x9a/0x110
    _raw_spin_unlock_irqrestore+0x1f/0x60
    __wake_up+0x44/0x60
    btrfs_encoded_read_endio+0x14b/0x190 [btrfs]
    btrfs_check_read_bio+0x8d9/0x1360 [btrfs]
    ? lock_release+0x1b0/0x780
    ? trace_lock_acquire+0x12f/0x1a0
    ? __pfx_btrfs_check_read_bio+0x10/0x10 [btrfs]
    ? process_one_work+0x7e3/0x1460
    ? lock_acquire+0x31/0xc0
    ? process_one_work+0x7e3/0x1460
    process_one_work+0x85c/0x1460
    ? __pfx_process_one_work+0x10/0x10
    ? assign_work+0x16c/0x240
    worker_thread+0x5e6/0xfc0
    ? __pfx_worker_thread+0x10/0x10
    kthread+0x2c3/0x3a0
    ? __pfx_kthread+0x10/0x10
    ret_from_fork+0x31/0x70
    ? __pfx_kthread+0x10/0x10
    ret_from_fork_asm+0x1a/0x30
    </TASK>

   Allocated by task 3661:
    kasan_save_stack+0x30/0x50
    kasan_save_track+0x14/0x30
    __kasan_kmalloc+0xaa/0xb0
    btrfs_encoded_read_regular_fill_pages+0x16c/0x6d0 [btrfs]
    send_extent_data+0xf0f/0x24a0 [btrfs]
    process_extent+0x48a/0x1830 [btrfs]
    changed_cb+0x178b/0x2ea0 [btrfs]
    btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
    _btrfs_ioctl_send+0x117/0x330 [btrfs]
    btrfs_ioctl+0x184a/0x60a0 [btrfs]
    __x64_sys_ioctl+0x12e/0x1a0
    do_syscall_64+0x95/0x180
    entry_SYSCALL_64_after_hwframe+0x76/0x7e

   Freed by task 3661:
    kasan_save_stack+0x30/0x50
    kasan_save_track+0x14/0x30
    kasan_save_free_info+0x3b/0x70
    __kasan_slab_free+0x4f/0x70
    kfree+0x143/0x490
    btrfs_encoded_read_regular_fill_pages+0x531/0x6d0 [btrfs]
    send_extent_data+0xf0f/0x24a0 [btrfs]
    process_extent+0x48a/0x1830 [btrfs]
    changed_cb+0x178b/0x2ea0 [btrfs]
    btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
    _btrfs_ioctl_send+0x117/0x330 [btrfs]
    btrfs_ioctl+0x184a/0x60a0 [btrfs]
    __x64_sys_ioctl+0x12e/0x1a0
    do_syscall_64+0x95/0x180
    entry_SYSCALL_64_after_hwframe+0x76/0x7e

   The buggy address belongs to the object at ffff888106a83f00
    which belongs to the cache kmalloc-rnd-07-96 of size 96
   The buggy address is located 24 bytes inside of
    freed 96-byte region [ffff888106a83f00, ffff888106a83f60)

   The buggy address belongs to the physical page:
   page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888106a83800 pfn:0x106a83
   flags: 0x17ffffc0000000(node=0|zone=2|lastcpupid=0x1fffff)
   page_type: f5(slab)
   raw: 0017ffffc0000000 ffff888100053680 ffffea0004917200 0000000000000004
   raw: ffff888106a83800 0000000080200019 00000001f5000000 0000000000000000
   page dumped because: kasan: bad access detected

   Memory state around the buggy address:
    ffff888106a83e00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
    ffff888106a83e80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
   >ffff888106a83f00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
                               ^
    ffff888106a83f80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
    ffff888106a84000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
   ==================================================================

Further analyzing the trace and the crash dump's vmcore file shows that the wake_up() call in btrfs_encoded_read_endio() is calling wake_up() on the wait_queue that is in the private data passed to the end_io handler.

Commit 4ff47df40447 ("btrfs: move priv off stack in btrfs_encoded_read_regular_fill_pages()") moved 'struct btrfs_encoded_read_private' off the stack.

Before that commit one can see a corruption of the private data when analyzing the vmcore after a crash:

*(struct btrfs_encoded_read_private *)0xffff88815626eec8 = {
        .wait = (wait_queue_head_t){
                .lock = (spinlock_t){
                        .rlock = (struct raw_spinlock){
                                .raw_lock = (arch_spinlock_t){
                                        .val = (atomic_t){
                                                .counter = (int)-2005885696,
                                        },
                                        .locked = (u8)0,
                                        .pending = (u8)157,
                                        .locked_pending = (u16)40192,
                                        .tail = (u16)34928,
                                },
                                .magic = (unsigned int)536325682,
                                .owner_cpu = (unsigned int)29,
                                .owner = (void *)__SCT__tp_func_btrfs_transaction_commit+0x0 = 0x0,
                                .dep_map = (struct lockdep_map){
                                        .key = (struct lock_class_key *)0xffff8881575a3b6c,
                                        .class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
                                        .name = (const char *)0xffff88815626f100 = "",
                                        .wait_type_outer = (u8)37,
                                        .wait_type_inner = (u8)178,
                                        .lock_type = (u8)154,
                                },
                        },
                        .__padding = (u8 [24]){ 0, 157, 112, 136, 50, 174, 247, 31, 29 },
                        .dep_map = (struct lockdep_map){
                                .key = (struct lock_class_key *)0xffff8881575a3b6c,
                                .class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
                                .name = (const char *)0xffff88815626f100 = "",
                                .wait_type_outer = (u8)37,
                                .wait_type_inner = (u8)178,
                                .lock_type = (u8)154,
                        },
                },
                .head = (struct list_head){
                        .next = (struct list_head *)0x112cca,
                        .prev = (struct list_head *)0x47,
                },
        },
        .pending = (atomic_t){
                .counter = (int)-1491499288,
        },
        .status = (blk_status_t)130,
}

Here we can see several indicators of in-memory data corruption, e.g. the large negative atomic values of ->pending or ->wait->lock->rlock->raw_lock->val, as well as the bogus spinlock magic 0x1ff7ae32 (decimal 536325682 above) instead of 0xdead4ead or the bogus pointer values for ->wait->head.

Quoted 3 lines

To fix this, move the call to bio_put() before the atomic_test operation so the submitter side in btrfs_encoded_read_regular_fill_pages() is not woken up before the bio is cleaned up.

This is the part I don't see what's the relation to the use-after-free problem on the private structure.
This seems like a cleanup that should be a separate patch with its own changelog.

Quoted 12 lines

Also change atomic_dec_return() to atomic_dec_and_test() to fix the corruption, as atomic_dec_return() is defined as two instructions on x86_64, whereas atomic_dec_and_test() is defined as a single atomic operation. This can lead to a situation where counter value is already decremented but the if statement in btrfs_encoded_read_endio() is not completely processed, i.e. the 0 test has not completed. If another thread continues executing btrfs_encoded_read_regular_fill_pages() the atomic_dec_return() there can see an already updated ->pending counter and continues by freeing the private data. Continuing in the endio handler the test for 0 succeeds and the wait_queue is woken up, resulting in a use-after-free.

This is the sort of explanation that should have been in v1. Basically, that the non-atomicity of atomic_dec_return() can make the waiter see the 0 value and free the private structure before the waker does a wake_up() against the private's wait queue.

So with bio_put() change in a separate patch:

(or with it but with an explanation on how this relates to the use-after-free, which I can't see)

Thanks.

Quoted 35 lines
Fixes 1881fba89bd5 ("btrfs: add BTRFS_IOC_ENCODED_READ ioctl")
---
 fs/btrfs/inode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +2 -3
index 22b8e2764619..cb8b23a3e56b 100644
@@ -9089,7 +9089,8 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
                 */
                WRITE_ONCE(priv->status, bbio->bio.bi_status);
        }
-       if (atomic_dec_return(&priv->pending) == 0) {
+       bio_put(&bbio->bio);
+       if (atomic_dec_and_test(&priv->pending)) {
                int err = blk_status_to_errno(READ_ONCE(priv->status));
 
                if (priv->uring_ctx) {
@@ -9099,7 +9100,6 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
                        wake_up(&priv->wait);
                }
        }
-       bio_put(&bbio->bio);
 }
 
 int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
--

2.43.0

#8 lore raw

Re: [PATCH v2 2/2] btrfs: simplify waiting for encoded read endios

In reply to Johannes Thumshirn
Quoted 39 lines · Johannes Thumshirn

From: Johannes Thumshirn <[email protected]>

Simplify the I/O completion path for encoded reads by using a completion instead of a wait_queue.

Furthermore skip taking an extra reference that is instantly dropped anyways and convert btrfs_encoded_read_private::pending into a refcount_t filed instead of atomic_t.

Freeing of the private data is now handled at a common place in btrfs_encoded_read_regular_fill_pages() and if btrfs_encoded_read_endio() is freeing the data in case it has an io_uring context associated, the btrfs_bio's private filed is NULLed to avoid a double free of the private data.

---
 fs/btrfs/inode.c | 31 ++++++++++++++++---------------
 1 file changed, 16 insertions(+), 15 deletions(-)
fs/btrfs/inode.c +2 -2
index cb8b23a3e56b..3093905364e5 100644
@@ -9068,9 +9068,9 @@ static ssize_t btrfs_encoded_read_inline(
 }
 
 struct btrfs_encoded_read_private {
-       wait_queue_head_t wait;
+       struct completion done;
        void *uring_ctx;
-       atomic_t pending;
+       refcount_t refs;
        blk_status_t status;
 };
 

fs/btrfs/inode.c static void btrfs_encoded_read_endio(struct btrfs_bio *bbio) · line 9098

Quoted 10 lines
fs/btrfs/inode.c +2 -1
@@ -9090,14 +9090,15 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
                WRITE_ONCE(priv->status, bbio->bio.bi_status);
        }
        bio_put(&bbio->bio);
-       if (atomic_dec_and_test(&priv->pending)) {
+       if (refcount_dec_and_test(&priv->refs)) {
                int err = blk_status_to_errno(READ_ONCE(priv->status));
 
                if (priv->uring_ctx) {
                        btrfs_uring_read_extent_endio(priv->uring_ctx, err);
+                       bbio->private = NULL;

Isn't this racy?

We decremented priv->refs to 0, the task at btrfs_encoded_read_regular_fill_pages() sees it as 0 and sees bbio->private still as non-NULL, does a kfree() on it and then we do it here again.

I.e. we should set bbio->private to NULL before decrementing, and possibly some barriers.

Thanks.

Quoted 66 lines
fs/btrfs/inode.c +12 -13
                        kfree(priv);
                } else {
-                       wake_up(&priv->wait);
+                       complete(&priv->done);
                }
        }
 }
@@ -9116,8 +9117,8 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
        if (!priv)
                return -ENOMEM;
 
-       init_waitqueue_head(&priv->wait);
-       atomic_set(&priv->pending, 1);
+       init_completion(&priv->done);
+       refcount_set(&priv->refs, 1);
        priv->status = 0;
        priv->uring_ctx = uring_ctx;
 
@@ -9130,7 +9131,7 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
                size_t bytes = min_t(u64, disk_io_size, PAGE_SIZE);
 
                if (bio_add_page(&bbio->bio, pages[i], bytes, 0) < bytes) {
-                       atomic_inc(&priv->pending);
+                       refcount_inc(&priv->refs);
                        btrfs_submit_bbio(bbio, 0);
 
                        bbio = btrfs_bio_alloc(BIO_MAX_VECS, REQ_OP_READ, fs_info,
@@ -9145,26 +9146,26 @@ int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
                disk_io_size -= bytes;
        } while (disk_io_size);
 
-       atomic_inc(&priv->pending);
        btrfs_submit_bbio(bbio, 0);
 
        if (uring_ctx) {
-               if (atomic_dec_return(&priv->pending) == 0) {
+               if (bbio->private && refcount_read(&priv->refs) == 0) {
                        ret = blk_status_to_errno(READ_ONCE(priv->status));
                        btrfs_uring_read_extent_endio(uring_ctx, ret);
-                       kfree(priv);
-                       return ret;
+                       goto out;
                }
 
                return -EIOCBQUEUED;
        } else {
-               if (atomic_dec_return(&priv->pending) != 0)
-                       io_wait_event(priv->wait, !atomic_read(&priv->pending));
+               wait_for_completion_io(&priv->done);
                /* See btrfs_encoded_read_endio() for ordering. */
                ret = blk_status_to_errno(READ_ONCE(priv->status));
-               kfree(priv);
-               return ret;
        }
+
+out:
+       kfree(priv);
+       return ret;
+
 }
 
 ssize_t btrfs_encoded_read_regular(struct kiocb *iocb, struct iov_iter *iter,
--

2.43.0

#3 lore raw
In reply to Filipe Manana
Quoted 4 lines · Filipe Manana

This is the part I don't see what's the relation to the use-after-free problem on the private structure.
This seems like a cleanup that should be a separate patch with its own changelog.

I need to re-test first, because AFAIR both the bio_put() and the atomic_dec_and_test() where needed to fix the bug on the test system.

#4 lore raw
In reply to Johannes Thumshirn
Quoted 167 lines · Johannes Thumshirn

From: Johannes Thumshirn <[email protected]>

Shinichiro reported the following use-after free that sometimes is happening in our CI system when running fstests' btrfs/284 on a TCMU runner device:

    ==================================================================
    BUG: KASAN: slab-use-after-free in lock_release+0x708/0x780
    Read of size 8 at addr ffff888106a83f18 by task kworker/u80:6/219

    CPU: 8 UID: 0 PID: 219 Comm: kworker/u80:6 Not tainted 6.12.0-rc6-kts+ #15
    Hardware name: Supermicro Super Server/X11SPi-TF, BIOS 3.3 02/21/2020
    Workqueue: btrfs-endio btrfs_end_bio_work [btrfs]
    Call Trace:
     <TASK>
     dump_stack_lvl+0x6e/0xa0
     ? lock_release+0x708/0x780
     print_report+0x174/0x505
     ? lock_release+0x708/0x780
     ? __virt_addr_valid+0x224/0x410
     ? lock_release+0x708/0x780
     kasan_report+0xda/0x1b0
     ? lock_release+0x708/0x780
     ? __wake_up+0x44/0x60
     lock_release+0x708/0x780
     ? __pfx_lock_release+0x10/0x10
     ? __pfx_do_raw_spin_lock+0x10/0x10
     ? lock_is_held_type+0x9a/0x110
     _raw_spin_unlock_irqrestore+0x1f/0x60
     __wake_up+0x44/0x60
     btrfs_encoded_read_endio+0x14b/0x190 [btrfs]
     btrfs_check_read_bio+0x8d9/0x1360 [btrfs]
     ? lock_release+0x1b0/0x780
     ? trace_lock_acquire+0x12f/0x1a0
     ? __pfx_btrfs_check_read_bio+0x10/0x10 [btrfs]
     ? process_one_work+0x7e3/0x1460
     ? lock_acquire+0x31/0xc0
     ? process_one_work+0x7e3/0x1460
     process_one_work+0x85c/0x1460
     ? __pfx_process_one_work+0x10/0x10
     ? assign_work+0x16c/0x240
     worker_thread+0x5e6/0xfc0
     ? __pfx_worker_thread+0x10/0x10
     kthread+0x2c3/0x3a0
     ? __pfx_kthread+0x10/0x10
     ret_from_fork+0x31/0x70
     ? __pfx_kthread+0x10/0x10
     ret_from_fork_asm+0x1a/0x30
     </TASK>

    Allocated by task 3661:
     kasan_save_stack+0x30/0x50
     kasan_save_track+0x14/0x30
     __kasan_kmalloc+0xaa/0xb0
     btrfs_encoded_read_regular_fill_pages+0x16c/0x6d0 [btrfs]
     send_extent_data+0xf0f/0x24a0 [btrfs]
     process_extent+0x48a/0x1830 [btrfs]
     changed_cb+0x178b/0x2ea0 [btrfs]
     btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
     _btrfs_ioctl_send+0x117/0x330 [btrfs]
     btrfs_ioctl+0x184a/0x60a0 [btrfs]
     __x64_sys_ioctl+0x12e/0x1a0
     do_syscall_64+0x95/0x180
     entry_SYSCALL_64_after_hwframe+0x76/0x7e

    Freed by task 3661:
     kasan_save_stack+0x30/0x50
     kasan_save_track+0x14/0x30
     kasan_save_free_info+0x3b/0x70
     __kasan_slab_free+0x4f/0x70
     kfree+0x143/0x490
     btrfs_encoded_read_regular_fill_pages+0x531/0x6d0 [btrfs]
     send_extent_data+0xf0f/0x24a0 [btrfs]
     process_extent+0x48a/0x1830 [btrfs]
     changed_cb+0x178b/0x2ea0 [btrfs]
     btrfs_ioctl_send+0x3bf9/0x5c20 [btrfs]
     _btrfs_ioctl_send+0x117/0x330 [btrfs]
     btrfs_ioctl+0x184a/0x60a0 [btrfs]
     __x64_sys_ioctl+0x12e/0x1a0
     do_syscall_64+0x95/0x180
     entry_SYSCALL_64_after_hwframe+0x76/0x7e

    The buggy address belongs to the object at ffff888106a83f00
     which belongs to the cache kmalloc-rnd-07-96 of size 96
    The buggy address is located 24 bytes inside of
     freed 96-byte region [ffff888106a83f00, ffff888106a83f60)

    The buggy address belongs to the physical page:
    page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff888106a83800 pfn:0x106a83
    flags: 0x17ffffc0000000(node=0|zone=2|lastcpupid=0x1fffff)
    page_type: f5(slab)
    raw: 0017ffffc0000000 ffff888100053680 ffffea0004917200 0000000000000004
    raw: ffff888106a83800 0000000080200019 00000001f5000000 0000000000000000
    page dumped because: kasan: bad access detected

    Memory state around the buggy address:
     ffff888106a83e00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
     ffff888106a83e80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
    >ffff888106a83f00: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
                                ^
     ffff888106a83f80: fa fb fb fb fb fb fb fb fb fb fb fb fc fc fc fc
     ffff888106a84000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    ==================================================================

Further analyzing the trace and the crash dump's vmcore file shows that the wake_up() call in btrfs_encoded_read_endio() is calling wake_up() on the wait_queue that is in the private data passed to the end_io handler.

Commit 4ff47df40447 ("btrfs: move priv off stack in btrfs_encoded_read_regular_fill_pages()") moved 'struct btrfs_encoded_read_private' off the stack.

Before that commit one can see a corruption of the private data when analyzing the vmcore after a crash:

*(struct btrfs_encoded_read_private *)0xffff88815626eec8 = {
	.wait = (wait_queue_head_t){
		.lock = (spinlock_t){
			.rlock = (struct raw_spinlock){
				.raw_lock = (arch_spinlock_t){
					.val = (atomic_t){
						.counter = (int)-2005885696,
					},
					.locked = (u8)0,
					.pending = (u8)157,
					.locked_pending = (u16)40192,
					.tail = (u16)34928,
				},
				.magic = (unsigned int)536325682,
				.owner_cpu = (unsigned int)29,
				.owner = (void *)__SCT__tp_func_btrfs_transaction_commit+0x0 = 0x0,
				.dep_map = (struct lockdep_map){
					.key = (struct lock_class_key *)0xffff8881575a3b6c,
					.class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
					.name = (const char *)0xffff88815626f100 = "",
					.wait_type_outer = (u8)37,
					.wait_type_inner = (u8)178,
					.lock_type = (u8)154,
				},
			},
			.__padding = (u8 [24]){ 0, 157, 112, 136, 50, 174, 247, 31, 29 },
			.dep_map = (struct lockdep_map){
				.key = (struct lock_class_key *)0xffff8881575a3b6c,
				.class_cache = (struct lock_class *[2]){ 0xffff8882a71985c0, 0xffffea00066f5d40 },
				.name = (const char *)0xffff88815626f100 = "",
				.wait_type_outer = (u8)37,
				.wait_type_inner = (u8)178,
				.lock_type = (u8)154,
			},
		},
		.head = (struct list_head){
			.next = (struct list_head *)0x112cca,
			.prev = (struct list_head *)0x47,
		},
	},
	.pending = (atomic_t){
		.counter = (int)-1491499288,
	},
	.status = (blk_status_t)130,
}

Here we can see several indicators of in-memory data corruption, e.g. the large negative atomic values of ->pending or ->wait->lock->rlock->raw_lock->val, as well as the bogus spinlock magic 0x1ff7ae32 (decimal 536325682 above) instead of 0xdead4ead or the bogus pointer values for ->wait->head.

Quoted 8 lines

To fix this, move the call to bio_put() before the atomic_test operation so the submitter side in btrfs_encoded_read_regular_fill_pages() is not woken up before the bio is cleaned up.

Also change atomic_dec_return() to atomic_dec_and_test() to fix the corruption, as atomic_dec_return() is defined as two instructions on x86_64, whereas atomic_dec_and_test() is defined as a single atomic operation.

This means we should not utilize "atomic_dec_return() == 0" as a way to do synchronization.

And unfortunately I'm also seeing other locations still utilizing the same patter inside btrfs_encoded_read_regular_fill_pages()

Shouldn't we also fix that call site even just for the sake of consistency?

Thanks,
Qu

Quoted 39 lines

This can lead to a situation where counter value is already decremented but the if statement in btrfs_encoded_read_endio() is not completely processed, i.e. the 0 test has not completed. If another thread continues executing btrfs_encoded_read_regular_fill_pages() the atomic_dec_return() there can see an already updated ->pending counter and continues by freeing the private data. Continuing in the endio handler the test for 0 succeeds and the wait_queue is woken up, resulting in a use-after-free.

Fixes 1881fba89bd5 ("btrfs: add BTRFS_IOC_ENCODED_READ ioctl")
---
  fs/btrfs/inode.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/inode.c +2 -2
index 22b8e2764619..cb8b23a3e56b 100644
@@ -9089,7 +9089,8 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
  		 */
  		WRITE_ONCE(priv->status, bbio->bio.bi_status);
  	}
-	if (atomic_dec_return(&priv->pending) == 0) {
+	bio_put(&bbio->bio);
+	if (atomic_dec_and_test(&priv->pending)) {
  		int err = blk_status_to_errno(READ_ONCE(priv->status));
 
  		if (priv->uring_ctx) {
@@ -9099,7 +9100,6 @@ static void btrfs_encoded_read_endio(struct btrfs_bio *bbio)
  			wake_up(&priv->wait);
  		}
  	}
-	bio_put(&bbio->bio);
  }
 
  int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode,
#5 lore raw
Quoted 11 lines · Qu Wenruo
Quoted 8 lines

To fix this, move the call to bio_put() before the atomic_test operation so the submitter side in btrfs_encoded_read_regular_fill_pages() is not woken up before the bio is cleaned up.

Also change atomic_dec_return() to atomic_dec_and_test() to fix the corruption, as atomic_dec_return() is defined as two instructions on x86_64, whereas atomic_dec_and_test() is defined as a single atomic operation.

This means we should not utilize "atomic_dec_return() == 0" as a way to do synchronization.

At least not for reference counting, hence recount_t doesn't even have an equivalent.

Quoted 4 lines

And unfortunately I'm also seeing other locations still utilizing the same patter inside btrfs_encoded_read_regular_fill_pages()

Shouldn't we also fix that call site even just for the sake of consistency?

I have no idea, TBH. The other user of atomic_dec_return() in btrfs is in delayed-inode.c:finish_one_item():

       /* atomic_dec_return implies a barrier */
       if ((atomic_dec_return(&delayed_root->items) <
            BTRFS_DELAYED_BACKGROUND || seq % BTRFS_DELAYED_BATCH == 0))
               cond_wake_up_nomb(&delayed_root->wait);

And that one looks safe in my eyes.

#6 lore raw
Quoted 24 lines · Johannes Thumshirn
Quoted 11 lines · Qu Wenruo
Quoted 8 lines

To fix this, move the call to bio_put() before the atomic_test operation so the submitter side in btrfs_encoded_read_regular_fill_pages() is not woken up before the bio is cleaned up.

Also change atomic_dec_return() to atomic_dec_and_test() to fix the corruption, as atomic_dec_return() is defined as two instructions on x86_64, whereas atomic_dec_and_test() is defined as a single atomic operation.

This means we should not utilize "atomic_dec_return() == 0" as a way to do synchronization.

At least not for reference counting, hence recount_t doesn't even have an equivalent.

Quoted 4 lines

And unfortunately I'm also seeing other locations still utilizing the same patter inside btrfs_encoded_read_regular_fill_pages()

Shouldn't we also fix that call site even just for the sake of consistency?

I have no idea, TBH. The other user of atomic_dec_return() in btrfs is in delayed-inode.c:finish_one_item():

Quoted 6 lines
       /* atomic_dec_return implies a barrier */
       if ((atomic_dec_return(&delayed_root->items) <
            BTRFS_DELAYED_BACKGROUND || seq % BTRFS_DELAYED_BATCH == 0))
               cond_wake_up_nomb(&delayed_root->wait);

And that one looks safe in my eyes.

A safe pattern where atomic_dec_return() is when once reaching 0 there will be no increment. Which is for example when setting the pending counter, possibyl adding e.g. pages one by one (atomic_inc) and then starting that. Each processed page then decrements the counter, once all are done the 0 will be there.

There are atomic_dec_return() that could be wrong, I haven't examined all of them but it seems to be always safer to use atomic_dec_and_test().