Skip to contentSkip to conversation

ext4/307: Calculate donor size to avoid failures for 64k bs

PATCH Superseded Ojaswin Mujoo 3 participants 7 messages Feb 2 – Feb 24

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ext4/307: Calculate donor size to avoid failures for 64k bs

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

While we are at it, add some new lines to make the code bit more readable.

---
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0
Tested-by on v1 1/1
#1 lore raw
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 5 lines

While we are at it, add some new lines to make the code bit more readable.

Thanks for the patch Ojaswin. Tested successfully on my setup.

Quoted 78 lines
---
  tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
  1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
  # Import common functions.
  . ./common/filter
  . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
  # Disable all sync operations to get higher load
  FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
  _workout()
  {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
  	echo ""
  	echo "Run fsstress"
  	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
  	echo "fsstress $args" >> $seqres.full
  	_run_fsstress $args
+
  	find $out -type f > $out.list
  	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
  	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
  	echo "Perform compacting"
  	cat $out.list | run_check $here/src/e4compact \
  		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
  	echo "Check data"
  	run_check md5sum -c $out.md5sum
  }
@@ -41,7 +74,7 @@ _require_scratch
  _require_defrag
  _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
  _scratch_mount
 
  _workout
#2 lore raw
In reply to Ojaswin Mujoo
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 6 lines

While we are at it, add some new lines to make the code bit more readable.

---

Hi,

Thanks for your patch, but this patch is conflict with v2026.01.27 :
42c2ccaf ("ext4/307: allocate donor file size dynamically")

both of you were trying to fix the donor file size issue. Please confirm if that commit helps you. If you still need further change, please rebase your patch on v2026.01.27.

Thanks,
Zorro

Quoted 80 lines
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0
#3 lore raw
Quoted 24 lines · Zorro Lang
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 6 lines

While we are at it, add some new lines to make the code bit more readable.

---

Quoted 8 lines

Hi,

Thanks for your patch, but this patch is conflict with v2026.01.27 :
42c2ccaf ("ext4/307: allocate donor file size dynamically")

both of you were trying to fix the donor file size issue. Please confirm if that commit helps you. If you still need further change, please rebase your patch on v2026.01.27.

Hi Zorro,

thanks for pointing this out. Weird that I didn't see this when I rbeased on master. I'll test the patch and get back to you.

Regards,
ojaswin

Quoted 85 lines

Thanks,
Zorro

Quoted 80 lines
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0
#4 lore raw
Quoted 30 lines · Ojaswin Mujoo
Quoted 29 lines · Zorro Lang
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 6 lines

While we are at it, add some new lines to make the code bit more readable.

---

Hi,

Thanks for your patch, but this patch is conflict with v2026.01.27 :
42c2ccaf ("ext4/307: allocate donor file size dynamically")

Quoted 8 lines
Quoted 3 lines

both of you were trying to fix the donor file size issue. Please confirm if that commit helps you. If you still need further change, please rebase your patch on v2026.01.27.

Hi Zorro,

thanks for pointing this out. Weird that I didn't see this when I rbeased on master. I'll test the patch and get back to you.

You can always rebase to "for-next" branch to get latest version, or check "patches-in-queue" scratch branch. The master branch is lagging behind :)

Quoted 89 lines

Regards,
ojaswin

Quoted 85 lines

Thanks,
Zorro

Quoted 80 lines
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0
#5 lore raw
Quoted 35 lines · Zorro Lang
Quoted 34 lines · Ojaswin Mujoo
Quoted 32 lines · Zorro Lang
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 6 lines

While we are at it, add some new lines to make the code bit more readable.

---

Hi,

Thanks for your patch, but this patch is conflict with v2026.01.27 :
42c2ccaf ("ext4/307: allocate donor file size dynamically")

both of you were trying to fix the donor file size issue. Please confirm if that commit helps you. If you still need further change, please rebase your patch on v2026.01.27.

Quoted 7 lines
Quoted 4 lines

Hi Zorro,

thanks for pointing this out. Weird that I didn't see this when I rbeased on master. I'll test the patch and get back to you.

You can always rebase to "for-next" branch to get latest version, or check "patches-in-queue" scratch branch. The master branch is lagging behind :)

Ahh got it, thanks :)

Regards,
ojaswin

Quoted 91 lines
Quoted 90 lines

Regards,
ojaswin

Quoted 85 lines

Thanks,
Zorro

Quoted 80 lines
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0
#6 lore raw
In reply to Zorro Lang
Quoted 29 lines · Zorro Lang
Quoted 16 lines · Ojaswin Mujoo

Hard coding donor size to 250M causes the following failure in e4compat

e4compact.c:68: do_defrag_range: Assertion `donor->length >= len' failed.
./common/rc: line 4616: 1583182 Aborted (core dumped)

The issue is that e4compat asserts that the stat.st_size of all files combined shouldn't be more than that of the donor file. In case of 64k blocksize, fsstress often creates sparse files which have >3G size even though their disk utilization is <100M. Since donor file is not big enough, this trips e4compat causing the failure.

Fix this by dynamically calculating the donor file size based on the size of all files. Also, make some changes to avoid future ENOSPC like reducing the -n passed to fsstress to keep the size around 2G and making sure we use the whole SCRATCH disk instead of 500M.

Quoted 6 lines

While we are at it, add some new lines to make the code bit more readable.

---

Hi,

Thanks for your patch, but this patch is conflict with v2026.01.27 :
42c2ccaf ("ext4/307: allocate donor file size dynamically")

Quoted 6 lines

both of you were trying to fix the donor file size issue. Please confirm if that commit helps you. If you still need further change, please rebase your patch on v2026.01.27.

Thanks,
Zorro

Hi Zorro,

Sorry for the delay but we tested this again with 64k bs ext4 but the test still fails. The issue is that e4compat.c essentially asserts:

(sum of stat.st_size of all files) <= donor stat.st_size

But in the merged patch we are using du -sm which gives the blocks used instead of the filesize.

Below are the numbers for 64k blocksize run:

du -sm <fsstress dir>:
132 // 132M

/* sum of stat.st_size of all files */
find <fsstress dir> -type f -printf '%s\n' | awk '{s+=$1} END {print s}' 5110092115 // ~4G

stat -c %s donor
138412032 // 132M

Since donor is still smaller than the sum of sizes of all the files, e4compat eventually hits an assert() causing the crash.

The patch shared in this thread takes care of this. However, it has also got me wondering whether the right fix would be to instead fix e4compat to compare blocks used instead of file sizes. I've not looked at the code closely yet but I don't see a reason why we should rely on donor being as big as the sum of sizes, when all we need is that the blocks used by each file should fit in the donor size.

What do you think?

Regards,
ojaswin

Quoted 82 lines
Quoted 81 lines
 tests/ext4/307 | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
tests/ext4/307 +37 -4
index 1f0e42ca..7e7fb9c8 100755
@@ -12,24 +12,57 @@ _begin_fstest auto ioctl rw defrag prealloc
 # Import common functions.
 . ./common/filter
 . ./common/defrag
+
+# Get the total of stat.st_size of each file to determine the
+# size of donor file. We use st_size here instead of blocks used
+# because thats what e4compat.c checks against.
+get_bytes_used() {
+	local filelist="$1"
+	local total=0
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
+	while IFS= read -r f; do
+		echo -n "File: $f: " >> $seqres.full
+		[[ -z "$f" ]] && continue
+
+		if [[ -f "$f" ]]; then
+			local bytes
+			bytes=$(stat -c %s "$f")
+			total=$((total + bytes))
+			echo $bytes >> $seqres.full
+		fi
+	done < "$filelist"
+
+	echo "Total bytes: $total" >> $seqres.full
+	echo $total
+}
+
 # Disable all sync operations to get higher load
 FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0"
 _workout()
 {
+	local blksz=$(_get_block_size $SCRATCH_MNT)
+
 	echo ""
 	echo "Run fsstress"
 	out=$SCRATCH_MNT/fsstress.$$
-	args=`_scale_fsstress_args -p4 -n999 -f setattr=1 -d $out`
+	args=`_scale_fsstress_args -p4 -n500 -f setattr=1 -d $out`
 	echo "fsstress $args" >> $seqres.full
 	_run_fsstress $args
+
 	find $out -type f > $out.list
 	cat $out.list | xargs  md5sum > $out.md5sum
-	usage=`du -sch $out | tail -n1 | gawk '{ print $1 }'`
+
+	bytes=`get_bytes_used $out.list`
+	echo "Total bytes used: $bytes" >> $seqres.full
+
 	echo "Allocate donor file"
-	$XFS_IO_PROG -c "falloc 0 250M" -f $SCRATCH_MNT/donor | _filter_xfs_io
+	$XFS_IO_PROG -c "falloc 0 $bytes" -f $SCRATCH_MNT/donor | _filter_xfs_io
+
 	echo "Perform compacting"
 	cat $out.list | run_check $here/src/e4compact \
 		-i -v -f $SCRATCH_MNT/donor  >> $seqres.full 2>&1
+
 	echo "Check data"
 	run_check md5sum -c $out.md5sum
 }
@@ -41,7 +74,7 @@ _require_scratch
 _require_defrag
 _require_xfs_io_command "falloc"
 
-_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount
 
 _workout
Signature · 1 line
2.52.0