Skip to contentSkip to conversation

common/rc: add _require_blocks_in_file helper

PATCH In review Joanne Chang 4 participants 8 messages Jan 8 – Feb 9

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 common/rc: add _require_blocks_in_file helper

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

---
v2 -> v3:
- added explanation for using constants to commit message

v1 -> v2:
- changed title from "generic/735: disable for f2fs"
- used a new helper function instead of _exclude_fs as Christoph Hellwig suggested

 common/rc         | 16 ++++++++++++++++
 tests/generic/735 |  1 +
 2 files changed, 17 insertions(+)
common/rc +16 -0
index c3cdc220..e92f4854 100644
@@ -6120,6 +6120,22 @@ _require_file_attr_special()
 	fi
 }
 
+# Require filesystem to accomodate enough blocks in a file
+_require_blocks_in_file()
+{
+	local blocks=$1
+
+	case $FSTYP in
+	f2fs)
+		if [ $blocks -gt 1057053439 ]; then
+			_notrun "$blocks blocks per file not supported on $FSTYP"
+		fi
+		;;
+	*)
+		;;
+	esac
+}
+
 ################################################################################
 # make sure this script returns success
 /bin/true
tests/generic/735 +1 -0
index 9bbdf3a1..2fbf125c 100755
@@ -22,6 +22,7 @@ fi
 _require_odirect
 _require_xfs_io_command "falloc"
 _require_xfs_io_command "finsert"
+_require_blocks_in_file $(( (1 << 32) - 1 ))
 
 dev_size=$((80 * 1024 * 1024))
 _scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
Signature · 1 line
2.52.0.351.gbe84eed79e-goog
#1 lore raw
Quoted 8 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Not to mention trying to create a file with 1,057,053,439 blocks allocated to it would probably take forever.

Hang on, we're talking about iblocks (aka the number of blocks allocated to this inode), not the maximum file size in blocks, right?

If so, then maybe this function and its comments should s/blocks/iblocks/? Or am I confused? ;)

--D

Quoted 57 lines

---
v2 -> v3:
- added explanation for using constants to commit message

v1 -> v2:
- changed title from "generic/735: disable for f2fs"
- used a new helper function instead of _exclude_fs as Christoph Hellwig suggested

 common/rc         | 16 ++++++++++++++++
 tests/generic/735 |  1 +
 2 files changed, 17 insertions(+)
common/rc +16 -0
index c3cdc220..e92f4854 100644
@@ -6120,6 +6120,22 @@ _require_file_attr_special()
 	fi
 }
 
+# Require filesystem to accomodate enough blocks in a file
+_require_blocks_in_file()
+{
+	local blocks=$1
+
+	case $FSTYP in
+	f2fs)
+		if [ $blocks -gt 1057053439 ]; then
+			_notrun "$blocks blocks per file not supported on $FSTYP"
+		fi
+		;;
+	*)
+		;;
+	esac
+}
+
 ################################################################################
 # make sure this script returns success
 /bin/true
tests/generic/735 +1 -0
index 9bbdf3a1..2fbf125c 100755
@@ -22,6 +22,7 @@ fi
 _require_odirect
 _require_xfs_io_command "falloc"
 _require_xfs_io_command "finsert"
+_require_blocks_in_file $(( (1 << 32) - 1 ))
 
 dev_size=$((80 * 1024 * 1024))
 _scratch_mkfs_sized $dev_size >>$seqres.full 2>&1
Signature · 1 line
2.52.0.351.gbe84eed79e-goog
#2 lore raw
Quoted 13 lines · Darrick J. Wong
Quoted 8 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Not to mention trying to create a file with 1,057,053,439 blocks allocated to it would probably take forever.

Quoted 7 lines

Hang on, we're talking about iblocks (aka the number of blocks allocated to this inode), not the maximum file size in blocks, right?

If so, then maybe this function and its comments should s/blocks/iblocks/? Or am I confused? ;)

--D

If I understand correctly, generic/735 creates a large logical file, but the actual physical block allocation is much smaller. Also, the F2FS limitation is about how many blocks the inode can address, no matter if the blocks are actually allocated.

So I believe the requirement is about the maximum file size in blocks, not the number of blocks actually allocated. Does it make sense to keep the name, or do you think another term would be clearer? I appreciate your thoughts on this.

Best regards,
Joanne

#3 lore raw
Quoted 27 lines · Joanne Chang
Quoted 13 lines · Darrick J. Wong
Quoted 8 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Not to mention trying to create a file with 1,057,053,439 blocks allocated to it would probably take forever.

Quoted 7 lines

Hang on, we're talking about iblocks (aka the number of blocks allocated to this inode), not the maximum file size in blocks, right?

If so, then maybe this function and its comments should s/blocks/iblocks/? Or am I confused? ;)

--D

If I understand correctly, generic/735 creates a large logical file, but the actual physical block allocation is much smaller. Also, the F2FS limitation is about how many blocks the inode can address, no matter if the blocks are actually allocated.

Quoted 4 lines

So I believe the requirement is about the maximum file size in blocks, not the number of blocks actually allocated. Does it make sense to keep the name, or do you think another term would be clearer? I appreciate your thoughts on this.

Hi Darrick,

I think Joanne's explanation makes sense, if you don't have more review points on it, I'll merge this patch.

Thanks,
Zorro

Quoted 4 lines

Best regards,
Joanne

#4 lore raw
Quoted 33 lines · Zorro Lang
Quoted 27 lines · Joanne Chang
Quoted 13 lines · Darrick J. Wong
Quoted 8 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Not to mention trying to create a file with 1,057,053,439 blocks allocated to it would probably take forever.

Quoted 7 lines

Hang on, we're talking about iblocks (aka the number of blocks allocated to this inode), not the maximum file size in blocks, right?

If so, then maybe this function and its comments should s/blocks/iblocks/? Or am I confused? ;)

--D

If I understand correctly, generic/735 creates a large logical file, but the actual physical block allocation is much smaller. Also, the F2FS limitation is about how many blocks the inode can address, no matter if the blocks are actually allocated.

Quoted 4 lines

So I believe the requirement is about the maximum file size in blocks, not the number of blocks actually allocated. Does it make sense to keep the name, or do you think another term would be clearer? I appreciate your thoughts on this.

Quoted 4 lines

Hi Darrick,

I think Joanne's explanation makes sense, if you don't have more review points on it, I'll merge this patch.

Agpgth, I didn't notice this reply! My apologies! :(

Ok, since this is a limit on the maximum logical file block number, _require_max_file_range_blocks, perhaps?

--D

Quoted 9 lines

Thanks,
Zorro

Quoted 4 lines

Best regards,
Joanne

#5 lore raw
Quoted 36 lines · Darrick J. Wong
Quoted 35 lines · Zorro Lang
Quoted 27 lines · Joanne Chang
Quoted 13 lines · Darrick J. Wong
Quoted 8 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Not to mention trying to create a file with 1,057,053,439 blocks allocated to it would probably take forever.

Quoted 7 lines

Hang on, we're talking about iblocks (aka the number of blocks allocated to this inode), not the maximum file size in blocks, right?

If so, then maybe this function and its comments should s/blocks/iblocks/? Or am I confused? ;)

--D

If I understand correctly, generic/735 creates a large logical file, but the actual physical block allocation is much smaller. Also, the F2FS limitation is about how many blocks the inode can address, no matter if the blocks are actually allocated.

Quoted 4 lines

So I believe the requirement is about the maximum file size in blocks, not the number of blocks actually allocated. Does it make sense to keep the name, or do you think another term would be clearer? I appreciate your thoughts on this.

Hi Darrick,

Quoted 7 lines
Quoted 2 lines

I think Joanne's explanation makes sense, if you don't have more review points on it, I'll merge this patch.

Agpgth, I didn't notice this reply! My apologies! :(

Ok, since this is a limit on the maximum logical file block number, _require_max_file_range_blocks, perhaps?

Thanks Darrick! I'm good with this, if Joanne agrees with this too, I can help to make this change locally to avoid asking for a new patch.

Thanks,
Zorro

Quoted 13 lines

--D

Quoted 9 lines

Thanks,
Zorro

Quoted 4 lines

Best regards,
Joanne

#6 lore raw
Quoted 7 lines · Zorro Lang
Quoted 2 lines · Darrick J. Wong

Ok, since this is a limit on the maximum logical file block number, _require_max_file_range_blocks, perhaps?

Thanks Darrick! I'm good with this, if Joanne agrees with this too, I can help to make this change locally to avoid asking for a new patch.

I agree too. Thanks Darrick for the suggestion, and thanks Zorro for your help.

Best regards,
Joanne

Quoted 5 lines

Thanks,
Zorro

Quoted 2 lines

--D

Reviewed-by on v3 1/1
#7 lore raw
In reply to Joanne Chang
Quoted 10 lines · Joanne Chang

generic/735 attempts to create a file with nearly 2^32 blocks. However, some filesystems have a maximum file block limit below this threshold. For instance, F2FS is limited to approximately 2^30 blocks due to the capacity of the inode. So add _require_blocks_in_file helper to skip the test in such cases.

The helper uses a hardcoded constant instead of a programmatic method, so that bugs which affect the maximum file size are not masked.

Thanks,