Skip to contentSkip to conversation

ext4: Add support for online resizing with bigalloc.

PATCH In review harshads 3 participants 4 messages Oct 25, 2017 – Oct 30, 2017 ext4 file system

lore
Share & tools
lore

harshads

author
#0 lore raw

PATCH 1/1 ext4: Add support for online resizing with bigalloc.

This patch adds support for online resizing on bigalloc file system by implementing EXT4_IOC_RESIZE_FS ioctl. Old resize interfaces (add block groups and extend last block group) are left untouched. Tests performed with cluster sizes of 1, 2, 4 and 8 blocks (of size 4k) per cluster. I will add these tests to xfstests.

---
 fs/ext4/ext4.h    |   4 +--
 fs/ext4/ioctl.c   |   6 ----
 fs/ext4/mballoc.c |  28 ++++++++-------
 fs/ext4/resize.c  | 104 ++++++++++++++++++++++++++++++++++--------------------
 4 files changed, 84 insertions(+), 58 deletions(-)
fs/ext4/ext4.h +2 -2
index b84aa1ca480a..53dcfd808567 100644
@@ -534,8 +534,8 @@ struct ext4_new_group_data {
 	__u64 inode_table;
 	__u32 blocks_count;
 	__u16 reserved_blocks;
-	__u16 unused;
-	__u32 free_blocks_count;
+	__u16 mdata_blocks;
+	__u32 free_clusters_count;
 };
 
 /* Indexes used to index group tables in ext4_new_group_data */
fs/ext4/ioctl.c +0 -6
index 28cc412852af..e165b06767ff 100644
@@ -702,12 +702,6 @@ group_add_out:
 		int err = 0, err2 = 0;
 		ext4_group_t o_group = EXT4_SB(sb)->s_groups_count;
 
-		if (ext4_has_feature_bigalloc(sb)) {
-			ext4_msg(sb, KERN_ERR,
-				 "Online resizing not (yet) supported with bigalloc");
-			return -EOPNOTSUPP;
-		}
-
 		if (copy_from_user(&n_blocks_count, (__u64 __user *)arg,
 				   sizeof(__u64))) {
 			return -EFAULT;
fs/ext4/mballoc.c +16 -12
index c1ab3ec30423..b3922e1c7eec 100644
@@ -4925,8 +4925,11 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb,
 	struct ext4_group_desc *desc;
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
 	struct ext4_buddy e4b;
-	int err = 0, ret, blk_free_count;
-	ext4_grpblk_t blocks_freed;
+	int err = 0, ret, free_clusters_count;
+	ext4_grpblk_t clusters_freed;
+	ext4_fsblk_t first_cluster = EXT4_B2C(sbi, block);
+	ext4_fsblk_t last_cluster = EXT4_B2C(sbi, block + count - 1);
+	unsigned long cluster_count = last_cluster - first_cluster + 1;
 
 	ext4_debug("Adding block(s) %llu-%llu\n", block, block + count - 1);
 
@@ -4938,8 +4941,8 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb,
 	 * Check to see if we are freeing blocks across a group
 	 * boundary.
 	 */
-	if (bit + count > EXT4_BLOCKS_PER_GROUP(sb)) {
-		ext4_warning(sb, "too much blocks added to group %u",
+	if (bit + cluster_count > EXT4_CLUSTERS_PER_GROUP(sb)) {
+		ext4_warning(sb, "too many blocks added to group %u",
 			     block_group);
 		err = -EINVAL;
 		goto error_return;
@@ -4985,14 +4988,14 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb,
 	if (err)
 		goto error_return;
 
-	for (i = 0, blocks_freed = 0; i < count; i++) {
+	for (i = 0, clusters_freed = 0; i < cluster_count; i++) {
 		BUFFER_TRACE(bitmap_bh, "clear bit");
 		if (!mb_test_bit(bit + i, bitmap_bh->b_data)) {
 			ext4_error(sb, "bit already cleared for block %llu",
 				   (ext4_fsblk_t)(block + i));
 			BUFFER_TRACE(bitmap_bh, "bit already cleared");
 		} else {
-			blocks_freed++;
+			clusters_freed++;
 		}
 	}
 
@@ -5006,19 +5009,20 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb,
 	 * them with group lock_held
 	 */
 	ext4_lock_group(sb, block_group);
-	mb_clear_bits(bitmap_bh->b_data, bit, count);
-	mb_free_blocks(NULL, &e4b, bit, count);
-	blk_free_count = blocks_freed + ext4_free_group_clusters(sb, desc);
-	ext4_free_group_clusters_set(sb, desc, blk_free_count);
+	mb_clear_bits(bitmap_bh->b_data, bit, cluster_count);
+	mb_free_blocks(NULL, &e4b, bit, cluster_count);
+	free_clusters_count = clusters_freed +
+		ext4_free_group_clusters(sb, desc);
+	ext4_free_group_clusters_set(sb, desc, free_clusters_count);
 	ext4_block_bitmap_csum_set(sb, block_group, desc, bitmap_bh);
 	ext4_group_desc_csum_set(sb, block_group, desc);
 	ext4_unlock_group(sb, block_group);
 	percpu_counter_add(&sbi->s_freeclusters_counter,
-			   EXT4_NUM_B2C(sbi, blocks_freed));
+			   clusters_freed);
 
 	if (sbi->s_log_groups_per_flex) {
 		ext4_group_t flex_group = ext4_flex_group(sbi, block_group);
-		atomic64_add(EXT4_NUM_B2C(sbi, blocks_freed),
+		atomic64_add(clusters_freed,
 			     &sbi->s_flex_groups[flex_group].free_clusters);
 	}
 
fs/ext4/resize.c +66 -38

272 lines not on this page · raw message

Signature · 1 line
2.15.0.rc0.271.g36b669edcc-goog
maintainer
#1 lore raw

(If you work for a company which supports enterprise distros, please keep reading. I have a question for you below.)

Quoted 7 lines · harshads

This patch adds support for online resizing on bigalloc file system by implementing EXT4_IOC_RESIZE_FS ioctl. Old resize interfaces (add block groups and extend last block group) are left untouched. Tests performed with cluster sizes of 1, 2, 4 and 8 blocks (of size 4k) per cluster. I will add these tests to xfstests.

Can you talk more about what sort of testing you have done? What file system sizes did you use?

I'm guessing you didn't test the old resize ioctls, because they are currently broken. I did a quick test because the code paths that are touched by this patch are also used by the old resize ioctls, and the following is failing:

mke2fs -t ext4 -Fq /dev/vdc 1G
mount /dev/vdc
export RESIZE2FS_KERNEL_VERSION=3.2.0
strace -o /tmp/st-$RESIZE2FS_KERNEL_VERSION resize2fs /dev/vdc 5G umount /dev/vdc

Quoted 1 line

From a quick check, and it looks like this broke sometime between 4.1

and 4.4. It works on 3.18 and 4.1, and but it failed for me in 4.4.

The 4.1 kernel was released in June 2015, and 4.4 kernel was released in January 2016. E2fsprogs v1.42 started using the new resize ioctl, which was released in November 2011. So if we take an enterprise distro which was initially released > 5-6 years ago, and it upgrades to a kernel newer than 4.1-4.4, the resize2fs from e2fsprogs 1.41.x (or earlier) will not be able to do online resize.

The question is whether or not we care at this point. We could use the observation that our lack of good testing, sometime in second half of 2015, we broke online resize backwards compatibility with e2fsprogs from before 2011, given that no one screamed --- perhaps we can just rip out support for the old resize ioctls entirely.

Or we can try to fix the old resize ioctl's, and then add better resize testing to xfstests.

What do the maintainers from the enterprise distro's think?

- Ted

#2 lore raw
Quoted 10 lines · Theodore Ts'o

(If you work for a company which supports enterprise distros, please keep reading. I have a question for you below.)

Quoted 6 lines · harshads

This patch adds support for online resizing on bigalloc file system by implementing EXT4_IOC_RESIZE_FS ioctl. Old resize interfaces (add block groups and extend last block group) are left untouched. Tests performed with cluster sizes of 1, 2, 4 and 8 blocks (of size 4k) per cluster. I will add these tests to xfstests.

Quoted 4 lines
Quoted 1 line

Can you talk more about what sort of testing you have done? What file system sizes did you use?

In my testing, I have tried to ensure that all different code paths that resize IOCTL triggers get executed. At a high level, these different code paths are - 1) extending last block group, 2) adding new block groups 3) conversion of file system to meta-bg. I performed tests with combinations of different cluster sizes (1, 2, 4, 8 block(s) per cluster) and different file system sizes. Following csv table lists all the tests that were performed and were successful with new resize IOCTL.

Cluster size (# 4K blocks), Original Size(# clusters), Final Size(# clusters) 1, 16384(64M), 24576(96M)
1, 24576(96M), 32767(128M)
1, 24576(96M), 32768(128M)
1, 24576(96M), 32769(128M)
1, 24576(96M), 49152(192M)
1, 24576(96M), 65536(256M)
1, 24576(96M), 491521(1.9G)
1, 24576(96M), 507904(~1.9G)
1, 24576(96M), 527488(2G)
1, 24576(96M), 5274880(20G)
2, 16384(128M), 24576(192M)
2, 24576(192M), 32767(256M)
2, 24576(192M), 32768(256M)
2, 24576(192M), 32769(256M)
2, 24576(192M), 49152(384M)
2, 24576(192M), 65536(512M)
2, 24576(192M), 491521(3.8G)
2, 24576(192M), 507904(3.8G)
2, 24576(192M), 527488(4G)
2, 24576(192M), 5274880(40G)
4, 16384(256M), 24576(384M)
4, 24576(384M), 32767(512M)
4, 24576(384M), 32768(512M)
4, 24576(384M), 32769(512M)
4, 24576(384M), 49152(768M)
4, 24576(384M), 65536(1G)
4, 24576(384M), 491521(7.6G)
4, 24576(384M), 507904(7.6G)
4, 24576(384M), 527488(8G)
4, 24576(384M), 5274880(80G)
8, 16384(512M), 24576(768M)
8, 24576(768M), 32767(1G)
8, 24576(768M), 32768(1G)
8, 24576(768M), 32769(1G)
8, 24576(768M), 49152(1.5G)
8, 24576(768M), 65536(2G)
8, 24576(768M), 491521(15.2G)
8, 24576(768M), 507904(15.2G)
8, 24576(768M), 527488(16G)
8, 24576(768M), 5274880(160G)

Quoted 5 lines

I'm guessing you didn't test the old resize ioctls, because they are currently broken. I did a quick test because the code paths that are touched by this patch are also used by the old resize ioctls, and the following is failing:

Yes, you are right. Also in the latest kernel, there are code-paths that are present only for the old resize IOCTLs and I have not touch them yet. Let's see what others have to say and then I can fix those if required.

Thanks,
Harshad.

Quoted 29 lines

mke2fs -t ext4 -Fq /dev/vdc 1G
mount /dev/vdc
export RESIZE2FS_KERNEL_VERSION=3.2.0
strace -o /tmp/st-$RESIZE2FS_KERNEL_VERSION resize2fs /dev/vdc 5G umount /dev/vdc

From a quick check, and it looks like this broke sometime between 4.1 and 4.4. It works on 3.18 and 4.1, and but it failed for me in 4.4.

The 4.1 kernel was released in June 2015, and 4.4 kernel was released in January 2016. E2fsprogs v1.42 started using the new resize ioctl, which was released in November 2011. So if we take an enterprise distro which was initially released > 5-6 years ago, and it upgrades to a kernel newer than 4.1-4.4, the resize2fs from e2fsprogs 1.41.x (or earlier) will not be able to do online resize.

The question is whether or not we care at this point. We could use the observation that our lack of good testing, sometime in second half of 2015, we broke online resize backwards compatibility with e2fsprogs from before 2011, given that no one screamed --- perhaps we can just rip out support for the old resize ioctls entirely.

Or we can try to fix the old resize ioctl's, and then add better resize testing to xfstests.

What do the maintainers from the enterprise distro's think?

- Ted

#3 lore raw
Quoted 73 lines · Harshad Shirwadkar
Quoted 10 lines · Theodore Ts'o

(If you work for a company which supports enterprise distros, please keep reading. I have a question for you below.)

Quoted 6 lines · harshads

This patch adds support for online resizing on bigalloc file system by implementing EXT4_IOC_RESIZE_FS ioctl. Old resize interfaces (add block groups and extend last block group) are left untouched. Tests performed with cluster sizes of 1, 2, 4 and 8 blocks (of size 4k) per cluster. I will add these tests to xfstests.

Quoted 4 lines
Quoted 1 line

Can you talk more about what sort of testing you have done? What file system sizes did you use?

In my testing, I have tried to ensure that all different code paths that resize IOCTL triggers get executed. At a high level, these different code paths are - 1) extending last block group, 2) adding new block groups 3) conversion of file system to meta-bg. I performed tests with combinations of different cluster sizes (1, 2, 4, 8 block(s) per cluster) and different file system sizes. Following csv table lists all the tests that were performed and were successful with new resize IOCTL.

Cluster size (# 4K blocks), Original Size(# clusters), Final Size(# clusters) 1, 16384(64M), 24576(96M)
1, 24576(96M), 32767(128M)
1, 24576(96M), 32768(128M)
1, 24576(96M), 32769(128M)
1, 24576(96M), 49152(192M)
1, 24576(96M), 65536(256M)
1, 24576(96M), 491521(1.9G)
1, 24576(96M), 507904(~1.9G)
1, 24576(96M), 527488(2G)
1, 24576(96M), 5274880(20G)
2, 16384(128M), 24576(192M)
2, 24576(192M), 32767(256M)
2, 24576(192M), 32768(256M)
2, 24576(192M), 32769(256M)
2, 24576(192M), 49152(384M)
2, 24576(192M), 65536(512M)
2, 24576(192M), 491521(3.8G)
2, 24576(192M), 507904(3.8G)
2, 24576(192M), 527488(4G)
2, 24576(192M), 5274880(40G)
4, 16384(256M), 24576(384M)
4, 24576(384M), 32767(512M)
4, 24576(384M), 32768(512M)
4, 24576(384M), 32769(512M)
4, 24576(384M), 49152(768M)
4, 24576(384M), 65536(1G)
4, 24576(384M), 491521(7.6G)
4, 24576(384M), 507904(7.6G)
4, 24576(384M), 527488(8G)
4, 24576(384M), 5274880(80G)
8, 16384(512M), 24576(768M)
8, 24576(768M), 32767(1G)
8, 24576(768M), 32768(1G)
8, 24576(768M), 32769(1G)
8, 24576(768M), 49152(1.5G)
8, 24576(768M), 65536(2G)
8, 24576(768M), 491521(15.2G)
8, 24576(768M), 507904(15.2G)
8, 24576(768M), 527488(16G)
8, 24576(768M), 5274880(160G)

Quoted 5 lines

I'm guessing you didn't test the old resize ioctls, because they are currently broken. I did a quick test because the code paths that are touched by this patch are also used by the old resize ioctls, and the following is failing:

Quoted 30 lines

Yes, you are right. Also in the latest kernel, there are code-paths that are present only for the old resize IOCTLs and I have not touch them yet. Let's see what others have to say and then I can fix those if required.

Thanks,
Harshad.

Quoted 22 lines

mke2fs -t ext4 -Fq /dev/vdc 1G
mount /dev/vdc
export RESIZE2FS_KERNEL_VERSION=3.2.0
strace -o /tmp/st-$RESIZE2FS_KERNEL_VERSION resize2fs /dev/vdc 5G umount /dev/vdc

From a quick check, and it looks like this broke sometime between 4.1 and 4.4. It works on 3.18 and 4.1, and but it failed for me in 4.4.

The 4.1 kernel was released in June 2015, and 4.4 kernel was released in January 2016. E2fsprogs v1.42 started using the new resize ioctl, which was released in November 2011. So if we take an enterprise distro which was initially released > 5-6 years ago, and it upgrades to a kernel newer than 4.1-4.4, the resize2fs from e2fsprogs 1.41.x (or earlier) will not be able to do online resize.

The question is whether or not we care at this point. We could use the observation that our lack of good testing, sometime in second half of 2015, we broke online resize backwards compatibility with e2fsprogs from before 2011, given that no one screamed --- perhaps we can just rip out support for the old resize ioctls entirely.

How about deprecate the old resize ioctls if you have any of the new features that landed in e2fsprogs after online-resize2fs gained the ability to call the new resize ioctl?

In other words, if your e2fsprogs is new enough to know about bigalloc, it's new enough to call the new resize ioctl, so we can have the old ioctl return EOPNOTSUPP if bigalloc is enabled.

--D

Quoted 7 lines
Quoted 7 lines

Or we can try to fix the old resize ioctl's, and then add better resize testing to xfstests.

What do the maintainers from the enterprise distro's think?

- Ted