Skip to contentSkip to conversation

Btrfs: Allow shrinking close to used space

review Chris Ball 3 participants 4 messages Feb 25, 2009 – Mar 2, 2009 Btrfs file system

lore
Share & tools
lore

Chris Ball

author
#0 lore raw

Hi,

This patch (against experimental HEAD) attempts to make shrinking more robust, by only updating device size if we've succeeded in creating enough free space without any failures in btrfs_relocate_chunk().

Here's a log with my patch applied. The two things to note are that a near-limit shrink now works, and that a failed shrink (in this case, trying to shrink to less than the used space) no longer updates the device size erroneously:

http://dev.laptop.org/~cjb/btrfs/shrink-log

Please review carefully -- I'm still new to btrfs. The short version of the patch is:

* create a success path, as a break out of the while(1) relocating (rather than going to the "done" label).
* move the device size updating code into that path
* leave "path->reada = 2;" behind in the entry path, since path is used by the searching operation rather than the later resize.

Thanks!

fs/btrfs/volumes.c +26 -23
index 1316139..e2fa072 100644
@@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	if (!path)
 		return -ENOMEM;
 
-	trans = btrfs_start_transaction(root, 1);
-	if (!trans) {
-		ret = -ENOMEM;
-		goto done;
-	}
-
 	path->reada = 2;
 
-	lock_chunks(root);
-
-	device->total_bytes = new_size;
-	if (device->writeable)
-		device->fs_devices->total_rw_bytes -= diff;
-	ret = btrfs_update_device(trans, device);
-	if (ret) {
-		unlock_chunks(root);
-		btrfs_end_transaction(trans, root);
-		goto done;
-	}
-	WARN_ON(diff > old_total);
-	btrfs_set_super_total_bytes(super_copy, old_total - diff);
-	unlock_chunks(root);
-	btrfs_end_transaction(trans, root);
-
 	key.objectid = device->devid;
 	key.offset = (u64)-1;
 	key.type = BTRFS_DEV_EXTENT_KEY;
@@ -1867,7 +1845,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 		length = btrfs_dev_extent_length(l, dev_extent);
 
 		if (key.offset + length <= new_size)
-			goto done;
+			break;
 
 		chunk_tree = btrfs_dev_extent_chunk_tree(l, dev_extent);
 		chunk_objectid = btrfs_dev_extent_chunk_objectid(l, dev_extent);
@@ -1880,6 +1858,31 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 			goto done;
 	}
 
+	/*
+	 * We've succeeded in freeing up enough space and can now update
+	 * the device's size.
+	 */
+	trans = btrfs_start_transaction(root, 1);
+	if (!trans) {
+		ret = -ENOMEM;
+		goto done;
+	}
+
+	lock_chunks(root);
+	device->total_bytes = new_size;
+	if (device->writeable)
+		device->fs_devices->total_rw_bytes -= diff;
+	ret = btrfs_update_device(trans, device);
+	if (ret) {
+		unlock_chunks(root);
+		btrfs_end_transaction(trans, root);
+		goto done;
+	}
+	WARN_ON(diff > old_total);
+	btrfs_set_super_total_bytes(super_copy, old_total - diff);
+	unlock_chunks(root);
+	btrfs_end_transaction(trans, root);
+
 done:
 	btrfs_free_path(path);
 	return ret;
Signature · 1 line
Chris Ball   <[email protected]>
#1 lore raw
Quoted 37 lines · Chris Ball

Hi,

This patch (against experimental HEAD) attempts to make shrinking more robust, by only updating device size if we've succeeded in creating enough free space without any failures in btrfs_relocate_chunk().

Here's a log with my patch applied. The two things to note are that a near-limit shrink now works, and that a failed shrink (in this case, trying to shrink to less than the used space) no longer updates the device size erroneously:

http://dev.laptop.org/~cjb/btrfs/shrink-log

Please review carefully -- I'm still new to btrfs. The short version of the patch is:

* create a success path, as a break out of the while(1) relocating (rather than going to the "done" label).
* move the device size updating code into that path
* leave "path->reada = 2;" behind in the entry path, since path is used by the searching operation rather than the later resize.

Thanks!

fs/btrfs/volumes.c +0 -3
index 1316139..e2fa072 100644
@@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	if (!path)
 		return -ENOMEM;
 
-	trans = btrfs_start_transaction(root, 1);
-	if (!trans) {
-		ret = -ENOMEM;

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) · line 1830

Quoted 10 lines
fs/btrfs/volumes.c +0 -8
-		goto done;
-	}
-
 	path->reada = 2;
 
-	lock_chunks(root);
-
-	device->total_bytes = new_size;
-	if (device->writeable)
-		device->fs_devices->total_rw_bytes -= diff;

So I think you still want to do this part, to keep the allocator from actually allocating new space in the area we are trying to cull with the shrink, we just don't want to update the ondisk stuff just yet, so everything else can be moved to below the loop.

So this

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *device, u64 new_size) · line 1838

Quoted 8 lines
fs/btrfs/volumes.c +0 -8
-	ret = btrfs_update_device(trans, device);
-	if (ret) {
-		unlock_chunks(root);
-		btrfs_end_transaction(trans, root);
-		goto done;
-	}
-	WARN_ON(diff > old_total);
-	btrfs_set_super_total_bytes(super_copy, old_total - diff);

to here should all be moved below like you have it. Other than that it looks good. Thanks,

Josef

#2 lore raw

2009/2/26 Josef Bacik <[email protected]>:

Quoted 4 lines · Chris Ball
Quoted 3 lines

Hi,

This patch (against experimental HEAD) attempts to make shrinking mo=

re

Quoted 4 lines
Quoted 4 lines

robust, by only updating device size if we've succeeded in creating enough free space without any failures in btrfs_relocate_chunk().

Here's a log with my patch applied. =A0The two things to note are th=

at a

Quoted 7 lines
Quoted 7 lines

near-limit shrink now works, and that a failed shrink (in this case, trying to shrink to less than the used space) no longer updates the device size erroneously:

=A0 http://dev.laptop.org/~cjb/btrfs/shrink-log

Please review carefully -- I'm still new to btrfs. =A0The short vers=

ion of

Quoted 10 lines
Quoted 10 lines

the patch is:

* create a success path, as a break out of the while(1) relocating =A0 (rather than going to the "done" label).
* move the device size updating code into that path
* leave "path->reada =3D 2;" behind in the entry path, since path is =A0 used by the searching operation rather than the later resize.

Thanks!

Quoted 7 lines

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *=

Quoted 7 lines
fs/btrfs/volumes.c +0 -0
index 1316139..e2fa072 100644
@@ -1815,30 +1815,8 @@ int btrfs_shrink_device(struct btrfs_device *=

device, u64 new_size)

Quoted 11 lines
Quoted 11 lines
fs/btrfs/volumes.c +0 -6
 =A0 =A0 =A0 if (!path)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return -ENOMEM;
 
- =A0 =A0 trans =3D btrfs_start_transaction(root, 1);
- =A0 =A0 if (!trans) {
- =A0 =A0 =A0 =A0 =A0 =A0 ret =3D -ENOMEM;
- =A0 =A0 =A0 =A0 =A0 =A0 goto done;
- =A0 =A0 }
-
 =A0 =A0 =A0 path->reada =3D 2;
 
Quoted 5 lines

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *= · line 1830

Quoted 5 lines
fs/btrfs/volumes.c +0 -5
- =A0 =A0 lock_chunks(root);
-
- =A0 =A0 device->total_bytes =3D new_size;
- =A0 =A0 if (device->writeable)
- =A0 =A0 =A0 =A0 =A0 =A0 device->fs_devices->total_rw_bytes -=3D di=

ff;

Quoted 2 lines

So I think you still want to do this part, to keep the allocator from=

actually

Quoted 1 line

allocating new space in the area we are trying to cull with the shrin=

k, we just

Quoted 1 line

don't want to update the ondisk stuff just yet, so everything else ca=

n be moved

Quoted 12 lines

to below the loop.

So this

fs/btrfs/volumes.c int btrfs_shrink_device(struct btrfs_device *= · line 1838

Quoted 8 lines
fs/btrfs/volumes.c +0 -8
- =A0 =A0 ret =3D btrfs_update_device(trans, device);
- =A0 =A0 if (ret) {
- =A0 =A0 =A0 =A0 =A0 =A0 unlock_chunks(root);
- =A0 =A0 =A0 =A0 =A0 =A0 btrfs_end_transaction(trans, root);
- =A0 =A0 =A0 =A0 =A0 =A0 goto done;
- =A0 =A0 }
- =A0 =A0 WARN_ON(diff > old_total);
- =A0 =A0 btrfs_set_super_total_bytes(super_copy, old_total - diff);
Quoted 1 line

to here should all be moved below like you have it. =A0Other than tha=

t it looks

Quoted 2 lines

good. =A0Thanks,

This isn't working. we don't call btrfs_update_device here, but it can = be called
in other places. I think we should add a new field in btrfs_device to reflect the
on disk device size, and update it when shrinking succeeds.

Regards
Yan Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#3 lore raw

Hi,

Here's a new patch that incorporates these comments. We now update device->fs_devices->total_rw_bytes before the shrink, as Josef suggests, and create a new field in btrfs_device to store an on-disk size that is only updated on a successful shrink operation, as requested by Yan.

(Thanks, Yan and Josef, for the patient explanations.)

==
From: Chris Ball <[email protected]>

Btrfs: When shrinking, only update disk size on success

Previously, we updated a device's size prior to attempting a shrink operation. This patch moves the device resizing logic to only happen if the shrink completes successfully. In the process, it introduces a new field to btrfs_device -- disk_total_bytes -- to track the on-disk size.

---
 fs/btrfs/volumes.c |   35 ++++++++++++++++++++++++-----------
 fs/btrfs/volumes.h |    3 +++
 2 files changed, 27 insertions(+), 11 deletions(-)
fs/btrfs/volumes.c +24 -11
index 1316139..303b7d6 100644
@@ -1433,7 +1433,7 @@ static noinline int btrfs_update_device(struct btrfs_trans_handle *trans,
 	btrfs_set_device_io_align(leaf, dev_item, device->io_align);
 	btrfs_set_device_io_width(leaf, dev_item, device->io_width);
 	btrfs_set_device_sector_size(leaf, dev_item, device->sector_size);
-	btrfs_set_device_total_bytes(leaf, dev_item, device->total_bytes);
+	btrfs_set_device_total_bytes(leaf, dev_item, device->disk_total_bytes);
 	btrfs_set_device_bytes_used(leaf, dev_item, device->bytes_used);
 	btrfs_mark_buffer_dirty(leaf);
 
@@ -1828,14 +1828,6 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 	device->total_bytes = new_size;
 	if (device->writeable)
 		device->fs_devices->total_rw_bytes -= diff;
-	ret = btrfs_update_device(trans, device);
-	if (ret) {
-		unlock_chunks(root);
-		btrfs_end_transaction(trans, root);
-		goto done;
-	}
-	WARN_ON(diff > old_total);
-	btrfs_set_super_total_bytes(super_copy, old_total - diff);
 	unlock_chunks(root);
 	btrfs_end_transaction(trans, root);
 
@@ -1867,7 +1859,7 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 		length = btrfs_dev_extent_length(l, dev_extent);
 
 		if (key.offset + length <= new_size)
-			goto done;
+			break;
 
 		chunk_tree = btrfs_dev_extent_chunk_tree(l, dev_extent);
 		chunk_objectid = btrfs_dev_extent_chunk_objectid(l, dev_extent);
@@ -1880,6 +1872,26 @@ int btrfs_shrink_device(struct btrfs_device *device, u64 new_size)
 			goto done;
 	}
 
+	/* Shrinking succeeded, else we would be at "done". */
+	trans = btrfs_start_transaction(root, 1);
+	if (!trans) {
+		ret = -ENOMEM;
+		goto done;
+	}
+	lock_chunks(root);
+
+	device->disk_total_bytes = new_size;
+	/* Now btrfs_update_device() will change the on-disk size. */
+	ret = btrfs_update_device(trans, device);
+	if (ret) {
+		unlock_chunks(root);
+		btrfs_end_transaction(trans, root);
+		goto done;
+	}
+	WARN_ON(diff > old_total);
+	btrfs_set_super_total_bytes(super_copy, old_total - diff);
+	unlock_chunks(root);
+	btrfs_end_transaction(trans, root);
 done:
 	btrfs_free_path(path);
 	return ret;
@@ -2959,7 +2971,8 @@ static int fill_device_from_item(struct extent_buffer *leaf,
 	unsigned long ptr;
 
 	device->devid = btrfs_device_id(leaf, dev_item);
-	device->total_bytes = btrfs_device_total_bytes(leaf, dev_item);
+	device->disk_total_bytes = btrfs_device_total_bytes(leaf, dev_item);
+	device->total_bytes = device->disk_total_bytes;
 	device->bytes_used = btrfs_device_bytes_used(leaf, dev_item);
 	device->type = btrfs_device_type(leaf, dev_item);
 	device->io_align = btrfs_device_io_align(leaf, dev_item);
fs/btrfs/volumes.h +3 -0
index 86c44e9..bf1ba75 100644
@@ -52,6 +52,9 @@ struct btrfs_device {
 	/* size of the device */
 	u64 total_bytes;
 
+	/* size of the disk */
+	u64 disk_total_bytes;
+
 	/* bytes used */
 	u64 bytes_used;
 
Signature · 6 lines
1.6.1.3


- Chris.
-- 
Chris Ball   <[email protected]>