Skip to contentSkip to conversation

Btrfs: only allocate necessary space when relocating a data block group

PATCH In review [email protected] 3 participants 5 messages Jun 9, 2020 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 3/3 Btrfs: only allocate necessary space when relocating a data block group

From: Filipe Manana <[email protected]>

When relocating a data block group we group extents from the block group into a cluster and when the cluster reaches a certain number of extents we do the relocation.

The first step is reserving data space and we try to reserve more space than we need if the block group is not full, when there are gaps between the collected extents. What happens is we attempt to reserve an amount of space that corresponds to the different between the end offset of the last extent and the start offset of the first extent. This can cause us to fail with -ENOSPC even when we have enough free space for relocating all the extents. We should skip space reservation for any gaps which always exist for block groups that are not full. Non full block groups are the ones which are useful to relocate, therefore a common use case.

So fix this by tracking the total number of bytes used for all the extents in the cluster and then reserving an amount of space that matches exactly the sum of the sizes of all the collected extents.

---
 fs/btrfs/relocation.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)
fs/btrfs/relocation.c +9 -7
index 11d156995446..7ec75229d86f 100644
@@ -108,6 +108,7 @@ struct tree_block {
 struct file_extent_cluster {
 	u64 start;
 	u64 end;
+	u64 total_bytes;
 	u64 boundary[MAX_EXTENTS];
 	unsigned int nr;
 };
@@ -2583,14 +2584,14 @@ int prealloc_file_extent_cluster(struct inode *inode,
 	int nr = 0;
 	int ret = 0;
 	u64 prealloc_start = cluster->start - offset;
-	u64 prealloc_end = cluster->end - offset;
 	u64 cur_offset;
+	u64 allocated = 0;
 
 	BUG_ON(cluster->start != cluster->boundary[0]);
 	inode_lock(inode);
 
 	ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode),
-					      prealloc_end + 1 - prealloc_start);
+					      cluster->total_bytes);
 	if (ret)
 		goto out;
 
@@ -2604,21 +2605,19 @@ int prealloc_file_extent_cluster(struct inode *inode,
 
 		lock_extent(&BTRFS_I(inode)->io_tree, start, end);
 		num_bytes = end + 1 - start;
-		if (cur_offset < start)
-			btrfs_free_reserved_data_space_noquota(inode,
-						       start - cur_offset);
 		ret = btrfs_prealloc_file_range(inode, 0, start,
 						num_bytes, num_bytes,
 						end + 1, &alloc_hint);
 		cur_offset = end + 1;
+		allocated += num_bytes;
 		unlock_extent(&BTRFS_I(inode)->io_tree, start, end);
 		if (ret)
 			break;
 		nr++;
 	}
-	if (cur_offset < prealloc_end)
+	if (allocated < cluster->total_bytes)
 		btrfs_free_reserved_data_space_noquota(inode,
-					       prealloc_end + 1 - cur_offset);
+				       cluster->total_bytes - allocated);
 out:
 	inode_unlock(inode);
 	return ret;
@@ -2809,6 +2808,7 @@ int relocate_data_extent(struct inode *inode, struct btrfs_key *extent_key,
 		if (ret)
 			return ret;
 		cluster->nr = 0;
+		cluster->total_bytes = 0;
 	}
 
 	if (!cluster->nr)
@@ -2818,12 +2818,14 @@ int relocate_data_extent(struct inode *inode, struct btrfs_key *extent_key,
 	cluster->end = extent_key->objectid + extent_key->offset - 1;
 	cluster->boundary[cluster->nr] = extent_key->objectid;
 	cluster->nr++;
+	cluster->total_bytes += extent_key->offset;
 
 	if (cluster->nr >= MAX_EXTENTS) {
 		ret = relocate_file_extent_cluster(inode, cluster);
 		if (ret)
 			return ret;
 		cluster->nr = 0;
+		cluster->total_bytes = 0;
 	}
 	return 0;
 }
Signature · 1 line
2.11.0
#1 lore raw
Quoted 5 lines · On 9.06.20 г. 13:19 ч., [email protected] wrote:

From: Filipe Manana <[email protected]>

When relocating a data block group we group extents from the block group into a cluster and when the cluster reaches a certain number of extents we do the relocation.

We don't reserve more space because the cluster is guaranteed to contain contiguous extents, namely in relocate_data_extent we have:

if (cluster->nr > 0 && extent_key->objectid != cluster->end + 1) {
ret = relocate_file_extent_cluster(inode, cluster); if (ret)

      return ret;

cluster->nr = 0;
}

Cluster->end points to the end offset of the last added extent and the check above ensures that the one which is currently added is also contiguous. So relocate_file_extent_cluster is always called with contiguous range of data extents, which might actually be less than MAX_EXTENTS.

As a matter of fact this is a rather hard requirement since prealloc_file_extent_cluster assumes that since when it iterates the extents in the cluster it does:

if (nr + 1 < cluster->nr)

   end = cluster->boundary[nr + 1] - 1 - offset;

else

   end = cluster->end - offset;

If those extents weren't contiguous we'd be preallocating more space in the data reloc inode.

#2 lore raw
Quoted 33 lines · Nikolay Borisov
Quoted 5 lines · On 9.06.20 г. 13:19 ч., [email protected] wrote:

From: Filipe Manana <[email protected]>

When relocating a data block group we group extents from the block group into a cluster and when the cluster reaches a certain number of extents we do the relocation.

We don't reserve more space because the cluster is guaranteed to contain contiguous extents, namely in relocate_data_extent we have:

if (cluster->nr > 0 && extent_key->objectid != cluster->end + 1)

{
   ret = relocate_file_extent_cluster(inode, cluster);
   if (ret)
      return ret;
   cluster->nr = 0;
}

Cluster->end points to the end offset of the last added extent and the check above ensures that the one which is currently added is also contiguous. So relocate_file_extent_cluster is always called with contiguous range of data extents, which might actually be less than MAX_EXTENTS.

As a matter of fact this is a rather hard requirement since prealloc_file_extent_cluster assumes that since when it iterates the extents in the cluster it does:

if (nr + 1 < cluster->nr)

Quoted 8 lines
   end = cluster->boundary[nr + 1] - 1 - offset;

else

   end = cluster->end - offset;

If those extents weren't contiguous we'd be preallocating more space in the data reloc inode.

Yes, thanks.
I was biased by the gap detection at prealloc_file_extent_cluster() and missed that, which in this case can't happen.

So lets drop this.

#3 lore raw
Quoted 40 lines · Filipe Manana
Quoted 39 lines · Nikolay Borisov
Quoted 5 lines · On 9.06.20 г. 13:19 ч., [email protected] wrote:

From: Filipe Manana <[email protected]>

When relocating a data block group we group extents from the block group into a cluster and when the cluster reaches a certain number of extents we do the relocation.

We don't reserve more space because the cluster is guaranteed to contain contiguous extents, namely in relocate_data_extent we have:

if (cluster->nr > 0 && extent_key->objectid != cluster->end + 1)

{
   ret = relocate_file_extent_cluster(inode, cluster);
   if (ret)
      return ret;
   cluster->nr = 0;
}

Cluster->end points to the end offset of the last added extent and the check above ensures that the one which is currently added is also contiguous. So relocate_file_extent_cluster is always called with contiguous range of data extents, which might actually be less than MAX_EXTENTS.

As a matter of fact this is a rather hard requirement since prealloc_file_extent_cluster assumes that since when it iterates the extents in the cluster it does:

if (nr + 1 < cluster->nr)

   end = cluster->boundary[nr + 1] - 1 - offset;

else

   end = cluster->end - offset;
Quoted 9 lines
Quoted 2 lines

If those extents weren't contiguous we'd be preallocating more space in the data reloc inode.

Yes, thanks.
I was biased by the gap detection at prealloc_file_extent_cluster() and missed that, which in this case can't happen.

So lets drop this.

Thinking a bit more about this the following check is redundant, since extents are guaranteed to be contiguous:

if (cur_offset < start)

btrfs_free_reserved_data_space(inode, data_reserved,

                                        cur_offset, start - cur_offset);

This was added as part of 18513091af948 but IMO it's bogus since clustered data extent reloc landed much earlier, in 0257bb82d21bedff26541bcf12f1461c23f9ed61. A good cleanup would be to simply remove this check.

#4 lore raw
In reply to [email protected]
Thank you for the patch! Perhaps something to improve:

Hi,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on kdave/for-next]
[also build test WARNING on v5.7 next-20200608]
[cannot apply to btrfs/next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/fdmanana-kernel-org/Btrfs-remove-the-start-argument-from-btrfs_free_reserved_data_space_noquota/20200609-182248
base:   https://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git for-next
config: nios2-allyesconfig (attached as .config)
compiler: nios2-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=nios2

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>, old ones prefixed by <<):

In file included from fs/btrfs/relocation.c:13:
fs/btrfs/ctree.h:2216:8: warning: type qualifiers ignored on function return type [-Wignored-qualifiers] 2216 | size_t __const btrfs_get_num_csums(void);
| ^~~~~~~
fs/btrfs/relocation.c: In function 'prealloc_file_extent_cluster':

>> fs/btrfs/relocation.c:2587:6: warning: variable 'cur_offset' set but not used [-Wunused-but-set-variable]

2587 | u64 cur_offset;
| ^~~~~~~~~~

vim +/cur_offset +2587 fs/btrfs/relocation.c

5d4f98a28c7d33 Yan Zheng      2009-06-10  2574  
efa56464562991 Yan, Zheng     2010-05-16  2575  static noinline_for_stack
efa56464562991 Yan, Zheng     2010-05-16  2576  int prealloc_file_extent_cluster(struct inode *inode,
efa56464562991 Yan, Zheng     2010-05-16  2577  				 struct file_extent_cluster *cluster)
efa56464562991 Yan, Zheng     2010-05-16  2578  {
efa56464562991 Yan, Zheng     2010-05-16  2579  	u64 alloc_hint = 0;
efa56464562991 Yan, Zheng     2010-05-16  2580  	u64 start;
efa56464562991 Yan, Zheng     2010-05-16  2581  	u64 end;
efa56464562991 Yan, Zheng     2010-05-16  2582  	u64 offset = BTRFS_I(inode)->index_cnt;
efa56464562991 Yan, Zheng     2010-05-16  2583  	u64 num_bytes;
efa56464562991 Yan, Zheng     2010-05-16  2584  	int nr = 0;
efa56464562991 Yan, Zheng     2010-05-16  2585  	int ret = 0;
dcb40c196fc85c Wang Xiaoguang 2016-07-25  2586  	u64 prealloc_start = cluster->start - offset;
18513091af9483 Wang Xiaoguang 2016-07-25 @2587  	u64 cur_offset;
eb4d9ef455539b Filipe Manana  2020-06-09  2588  	u64 allocated = 0;
efa56464562991 Yan, Zheng     2010-05-16  2589  
efa56464562991 Yan, Zheng     2010-05-16  2590  	BUG_ON(cluster->start != cluster->boundary[0]);
5955102c9984fa Al Viro        2016-01-22  2591  	inode_lock(inode);
efa56464562991 Yan, Zheng     2010-05-16  2592  
8fa95e39f0371b Filipe Manana  2020-06-09  2593  	ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode),
eb4d9ef455539b Filipe Manana  2020-06-09  2594  					      cluster->total_bytes);
efa56464562991 Yan, Zheng     2010-05-16  2595  	if (ret)
efa56464562991 Yan, Zheng     2010-05-16  2596  		goto out;
efa56464562991 Yan, Zheng     2010-05-16  2597  
18513091af9483 Wang Xiaoguang 2016-07-25  2598  	cur_offset = prealloc_start;
efa56464562991 Yan, Zheng     2010-05-16  2599  	while (nr < cluster->nr) {
efa56464562991 Yan, Zheng     2010-05-16  2600  		start = cluster->boundary[nr] - offset;
efa56464562991 Yan, Zheng     2010-05-16  2601  		if (nr + 1 < cluster->nr)
efa56464562991 Yan, Zheng     2010-05-16  2602  			end = cluster->boundary[nr + 1] - 1 - offset;
efa56464562991 Yan, Zheng     2010-05-16  2603  		else
efa56464562991 Yan, Zheng     2010-05-16  2604  			end = cluster->end - offset;
efa56464562991 Yan, Zheng     2010-05-16  2605  
d0082371cf086e Jeff Mahoney   2012-03-01  2606  		lock_extent(&BTRFS_I(inode)->io_tree, start, end);
efa56464562991 Yan, Zheng     2010-05-16  2607  		num_bytes = end + 1 - start;
efa56464562991 Yan, Zheng     2010-05-16  2608  		ret = btrfs_prealloc_file_range(inode, 0, start,
efa56464562991 Yan, Zheng     2010-05-16  2609  						num_bytes, num_bytes,
efa56464562991 Yan, Zheng     2010-05-16  2610  						end + 1, &alloc_hint);
18513091af9483 Wang Xiaoguang 2016-07-25  2611  		cur_offset = end + 1;
eb4d9ef455539b Filipe Manana  2020-06-09  2612  		allocated += num_bytes;
d0082371cf086e Jeff Mahoney   2012-03-01  2613  		unlock_extent(&BTRFS_I(inode)->io_tree, start, end);
efa56464562991 Yan, Zheng     2010-05-16  2614  		if (ret)
efa56464562991 Yan, Zheng     2010-05-16  2615  			break;
efa56464562991 Yan, Zheng     2010-05-16  2616  		nr++;
efa56464562991 Yan, Zheng     2010-05-16  2617  	}
eb4d9ef455539b Filipe Manana  2020-06-09  2618  	if (allocated < cluster->total_bytes)
8fa95e39f0371b Filipe Manana  2020-06-09  2619  		btrfs_free_reserved_data_space_noquota(inode,
eb4d9ef455539b Filipe Manana  2020-06-09  2620  				       cluster->total_bytes - allocated);
efa56464562991 Yan, Zheng     2010-05-16  2621  out:
5955102c9984fa Al Viro        2016-01-22  2622  	inode_unlock(inode);
efa56464562991 Yan, Zheng     2010-05-16  2623  	return ret;
efa56464562991 Yan, Zheng     2010-05-16  2624  }
efa56464562991 Yan, Zheng     2010-05-16  2625  

:::::: The code at line 2587 was first introduced by commit
:::::: 18513091af9483ba84328d42092bd4d42a3c958f btrfs: update btrfs_space_info's bytes_may_use timely

:::::: TO: Wang Xiaoguang <[email protected]> :::::: CC: Chris Mason <[email protected]>

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org