Skip to contentSkip to conversation

btrfs: trim: fix underflow in trim length to prevent access beyond device boundary

PATCH In review Qu Wenruo 4 participants 5 messages Jul 30, 2020 – Jul 31, 2020 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/1 btrfs: trim: fix underflow in trim length to prevent access beyond device boundary

[BUG]
The following script can lead to tons of beyond device boundary access:

mkfs.btrfs -f $dev -b 10G
mount $dev $mnt
trimfs $mnt
btrfs filesystem resize 1:-1G $mnt
trimfs $mnt

[CAUSE]
Since commit 929be17a9b49 ("btrfs: Switch btrfs_trim_free_extents to find_first_clear_extent_bit"), we try to avoid trimming ranges that's already trimmed.

So we check device->alloc_state by finding the first range which doesn't have CHUNK_TRIMMED and CHUNK_ALLOCATED not set.

But if we shrunk the device, that bits are not cleared, thus we could easily got a range starts beyond the shrunk device size.

This results the returned @start and @end are all beyond device size, then we call "end = min(end, device->total_bytes -1);" making @end smaller than device size.

Then finally we goes "len = end - start + 1", totally underflow the result, and lead to the beyond-device-boundary access.

[FIX]
This patch will fix the problem in two ways:
- Clear CHUNK_TRIMMED | CHUNK_ALLOCATED bits when shrinking device This is the root fix

- Add extra safe net when trimming free device extents We check if the returned range is already beyond current device boundary.

---
 fs/btrfs/extent-tree.c |  5 +++++
 fs/btrfs/volumes.c     | 12 ++++++++++++
 2 files changed, 17 insertions(+)
fs/btrfs/extent-tree.c +5 -0
index 61ede335f6c3..758f963feb96 100644
@@ -5667,6 +5667,11 @@ static int btrfs_trim_free_extents(struct btrfs_device *device, u64 *trimmed)
 		find_first_clear_extent_bit(&device->alloc_state, start,
 					    &start, &end,
 					    CHUNK_TRIMMED | CHUNK_ALLOCATED);
+		if (start >= device->total_bytes) {
+			mutex_unlock(&fs_info->chunk_mutex);
+			ret = 0;
+			break;
+		}
 
 		/* Ensure we skip the reserved area in the first 1M */
 		start = max_t(u64, start, SZ_1M);
fs/btrfs/volumes.c +12 -0
index 537ccf66ee20..906704c61a51 100644
@@ -4705,6 +4705,18 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	}
 
 	mutex_lock(&fs_info->chunk_mutex);
+	/*
+	 * Also clear any CHUNK_TRIMMED and CHUNK_ALLOCATED bits beyond the
+	 * current device boundary.
+	 */
+	ret = clear_extent_bits(&device->alloc_state, new_size, (u64)-1,
+				CHUNK_TRIMMED | CHUNK_ALLOCATED);
+	if (ret < 0) {
+		mutex_unlock(&fs_info->chunk_mutex);
+		btrfs_abort_transaction(trans, ret);
+		btrfs_end_transaction(trans);
+		goto done;
+	}
 	btrfs_device_set_disk_total_bytes(device, new_size);
 	if (list_empty(&device->post_commit_list))
 		list_add_tail(&device->post_commit_list,
Signature · 1 line
2.28.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 49 lines · Qu Wenruo

[BUG]
The following script can lead to tons of beyond device boundary access:

mkfs.btrfs -f $dev -b 10G
mount $dev $mnt
trimfs $mnt
btrfs filesystem resize 1:-1G $mnt
trimfs $mnt

[CAUSE]
Since commit 929be17a9b49 ("btrfs: Switch btrfs_trim_free_extents to find_first_clear_extent_bit"), we try to avoid trimming ranges that's already trimmed.

So we check device->alloc_state by finding the first range which doesn't have CHUNK_TRIMMED and CHUNK_ALLOCATED not set.

But if we shrunk the device, that bits are not cleared, thus we could easily got a range starts beyond the shrunk device size.

This results the returned @start and @end are all beyond device size, then we call "end = min(end, device->total_bytes -1);" making @end smaller than device size.

Then finally we goes "len = end - start + 1", totally underflow the result, and lead to the beyond-device-boundary access.

[FIX]
This patch will fix the problem in two ways:
- Clear CHUNK_TRIMMED | CHUNK_ALLOCATED bits when shrinking device This is the root fix

- Add extra safe net when trimming free device extents We check if the returned range is already beyond current device boundary.

---
 fs/btrfs/extent-tree.c |  5 +++++
 fs/btrfs/volumes.c     | 12 ++++++++++++
 2 files changed, 17 insertions(+)
fs/btrfs/extent-tree.c +0 -0
index 61ede335f6c3..758f963feb96 100644

fs/btrfs/extent-tree.c static int btrfs_trim_free_extents(struct btrfs_device *device, u64 *trimmed) · line 5674

Quoted 8 lines
fs/btrfs/extent-tree.c +5 -0
@@ -5667,6 +5667,11 @@ static int btrfs_trim_free_extents(struct btrfs_device *device, u64 *trimmed)
                find_first_clear_extent_bit(&device->alloc_state, start,
                                            &start, &end,
                                            CHUNK_TRIMMED | CHUNK_ALLOCATED);
+               if (start >= device->total_bytes) {
+                       mutex_unlock(&fs_info->chunk_mutex);
+                       ret = 0;
+                       break;
+               }

It's good to ensure we never trim beyond the end of the fs, to avoid corruption of whatever lies beyond it.
However we should have at least a WARN_ON / WARN_ON_ONCE here to help more easily detect other current or future bugs that lead to the same type of issue.

Other than that it looks good to me.
After that you can have,

Thanks.

Quoted 29 lines
fs/btrfs/extent-tree.c +0 -0
                /* Ensure we skip the reserved area in the first 1M */
                start = max_t(u64, start, SZ_1M);
fs/btrfs/volumes.c +12 -1
index 537ccf66ee20..906704c61a51 100644
@@ -4705,6 +4705,18 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
        }
 
        mutex_lock(&fs_info->chunk_mutex);
+       /*
+        * Also clear any CHUNK_TRIMMED and CHUNK_ALLOCATED bits beyond the
+        * current device boundary.
+        */
+       ret = clear_extent_bits(&device->alloc_state, new_size, (u64)-1,
+                               CHUNK_TRIMMED | CHUNK_ALLOCATED);
+       if (ret < 0) {
+               mutex_unlock(&fs_info->chunk_mutex);
+               btrfs_abort_transaction(trans, ret);
+               btrfs_end_transaction(trans);
+               goto done;
+       }
        btrfs_device_set_disk_total_bytes(device, new_size);
        if (list_empty(&device->post_commit_list))
                list_add_tail(&device->post_commit_list,
--

2.28.0

Signature · 3 lines
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”
#2 lore raw
Quoted 56 lines · Filipe Manana
Quoted 55 lines · Qu Wenruo

[BUG]
The following script can lead to tons of beyond device boundary access:

mkfs.btrfs -f $dev -b 10G
mount $dev $mnt
trimfs $mnt
btrfs filesystem resize 1:-1G $mnt
trimfs $mnt

[CAUSE]
Since commit 929be17a9b49 ("btrfs: Switch btrfs_trim_free_extents to find_first_clear_extent_bit"), we try to avoid trimming ranges that's already trimmed.

So we check device->alloc_state by finding the first range which doesn't have CHUNK_TRIMMED and CHUNK_ALLOCATED not set.

But if we shrunk the device, that bits are not cleared, thus we could easily got a range starts beyond the shrunk device size.

This results the returned @start and @end are all beyond device size, then we call "end = min(end, device->total_bytes -1);" making @end smaller than device size.

Then finally we goes "len = end - start + 1", totally underflow the result, and lead to the beyond-device-boundary access.

[FIX]
This patch will fix the problem in two ways:
- Clear CHUNK_TRIMMED | CHUNK_ALLOCATED bits when shrinking device This is the root fix

- Add extra safe net when trimming free device extents We check if the returned range is already beyond current device boundary.

---
 fs/btrfs/extent-tree.c |  5 +++++
 fs/btrfs/volumes.c     | 12 ++++++++++++
 2 files changed, 17 insertions(+)
fs/btrfs/extent-tree.c +3 -0
index 61ede335f6c3..758f963feb96 100644
@@ -5667,6 +5667,11 @@ static int btrfs_trim_free_extents(struct btrfs_device *device, u64 *trimmed)
                find_first_clear_extent_bit(&device->alloc_state, start,
                                            &start, &end,
                                            CHUNK_TRIMMED | CHUNK_ALLOCATED);
+               if (start >= device->total_bytes) {
+                       mutex_unlock(&fs_info->chunk_mutex);
+                       ret = 0;
Quoted 8 lines

fs/btrfs/extent-tree.c static int btrfs_trim_free_extents(struct btrfs_device *device, u64 *trimmed) · line 5674

Quoted 2 lines
fs/btrfs/extent-tree.c +2 -0
+                       break;
+               }

It's good to ensure we never trim beyond the end of the fs, to avoid corruption of whatever lies beyond it.
However we should have at least a WARN_ON / WARN_ON_ONCE here to help more easily detect other current or future bugs that lead to the same type of issue.

Since you're also on the idea of extra warning, it's definitely worthy then.

Thanks for the advice,
Qu

Quoted 39 lines

Other than that it looks good to me.
After that you can have,

Thanks.

Quoted 29 lines
fs/btrfs/extent-tree.c +0 -0
                /* Ensure we skip the reserved area in the first 1M */
                start = max_t(u64, start, SZ_1M);
fs/btrfs/volumes.c +12 -1
index 537ccf66ee20..906704c61a51 100644
@@ -4705,6 +4705,18 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
        }
 
        mutex_lock(&fs_info->chunk_mutex);
+       /*
+        * Also clear any CHUNK_TRIMMED and CHUNK_ALLOCATED bits beyond the
+        * current device boundary.
+        */
+       ret = clear_extent_bits(&device->alloc_state, new_size, (u64)-1,
+                               CHUNK_TRIMMED | CHUNK_ALLOCATED);
+       if (ret < 0) {
+               mutex_unlock(&fs_info->chunk_mutex);
+               btrfs_abort_transaction(trans, ret);
+               btrfs_end_transaction(trans);
+               goto done;
+       }
        btrfs_device_set_disk_total_bytes(device, new_size);
        if (list_empty(&device->post_commit_list))
                list_add_tail(&device->post_commit_list,
--

2.28.0

#3 lore raw
In reply to Qu Wenruo
Quoted 6 lines · Qu Wenruo
fs/btrfs/volumes.c +0 -0
@@ -4705,6 +4705,18 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	}
 
 	mutex_lock(&fs_info->chunk_mutex);

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) · line 4717

Quoted 10 lines
fs/btrfs/volumes.c +10 -0
+	/*
+	 * Also clear any CHUNK_TRIMMED and CHUNK_ALLOCATED bits beyond the
+	 * current device boundary.
+	 */
+	ret = clear_extent_bits(&device->alloc_state, new_size, (u64)-1,
+				CHUNK_TRIMMED | CHUNK_ALLOCATED);
+	if (ret < 0) {
+		mutex_unlock(&fs_info->chunk_mutex);
+		btrfs_abort_transaction(trans, ret);
+		btrfs_end_transaction(trans);

Can this be made more lightweight? If the device shrink succeeded, we now update only the device status, so failing the whole transaction here seems to be too much.

As the device size is being reduced, we could possibly update the last relevant entry in the alloc_state tree and delete everything after that, instead of clearing the bits.

#4 lore raw
Quoted 6 lines · David Sterba
Quoted 5 lines · Qu Wenruo
fs/btrfs/volumes.c +0 -0
@@ -4705,6 +4705,18 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	}
 
Quoted 15 lines

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) · line 4717

Quoted 11 lines
fs/btrfs/volumes.c +10 -0
 	mutex_lock(&fs_info->chunk_mutex);
+	/*
+	 * Also clear any CHUNK_TRIMMED and CHUNK_ALLOCATED bits beyond the
+	 * current device boundary.
+	 */
+	ret = clear_extent_bits(&device->alloc_state, new_size, (u64)-1,
+				CHUNK_TRIMMED | CHUNK_ALLOCATED);
+	if (ret < 0) {
+		mutex_unlock(&fs_info->chunk_mutex);
+		btrfs_abort_transaction(trans, ret);
+		btrfs_end_transaction(trans);

Can this be made more lightweight? If the device shrink succeeded, we now update only the device status, so failing the whole transaction here seems to be too much.

It can be lightweight by just ignoring the return value.

1. Currently clear_extent_bits() only return 0
As currently clear_extent_bits() uses BUG_ON() to handle the memory allocation error, and can only return 0.

2. We have the extra safenet to prevent problems
But we may want to add extra warning in the safenet itself, so we shouldn't completely rely on that.

3. Since we're clearing the only two utilized bits, it should not allocate mew memory
Thus it should always return 0.

So yep, we can safely ignore the return value.

Thanks,
Qu

Quoted 5 lines

As the device size is being reduced, we could possibly update the last relevant entry in the alloc_state tree and delete everything after that, instead of clearing the bits.