Skip to contentSkip to conversation

common/dmlogwrites: add _require_log_writes_sized helper

PATCH Superseded Disha Goel 2 participants 7 messages Apr 27 – May 8 Btrfs file system

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Disha Goel

author
#0 lore raw

PATCH v2 1/2 common/dmlogwrites: add _require_log_writes_sized helper

Add a new helper function _require_log_writes_sized() to check if the LOGWRITES_DEV meets a minimum size requirement.

This is useful for tests that use dm-log-writes with additional space requirements, such as creating LVM snapshots during log replay or tests that generate large amounts of logged I/O operations.

The function takes a size parameter in KB, calls _require_log_writes() internally, and skips the test if LOGWRITES_DEV is smaller than the required size.

v2:
- Renamed function from _require_logwrites_size to _require_log_writes_sized
- Added documentation comment clarifying the parameter is in KB
- Made function call _require_log_writes() internally to avoid redundant calls in test code

---
 common/dmlogwrites | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
common/dmlogwrites +15 -0
index a27e1966..0925b333 100644
@@ -14,6 +14,21 @@ _require_log_writes()
 	_require_test_program "log-writes/replay-log"
 }
 
+# Require a log writes device of a minimum size
+# $1: minimum size in KB
+_require_log_writes_sized()
+{
+        local size=$1
+
+        [ $# -eq 1 ] || _fail "_require_log_writes_sized: expected size param"
+
+        _require_log_writes
+
+        local devsize=$(_get_device_size $LOGWRITES_DEV)
+        [ $devsize -lt $1 ] && \
+                _notrun "LOGWRITES_DEV too small, ${devsize}KB < ${size}KB"
+}
+
 # Starting from v4.15-rc1, DAX support was added to dm-log-writes, but note
 # that it doesn't track the data that we write via the mmap(), so we can't do
 # any data integrity checking. We can only verify that the metadata writes for
Signature · 1 line
2.45.1
author
#1 lore raw

PATCH v2 2/2 btrfs/291: fix state transition logic and add size requirement

This patch fixes two issues in btrfs/291:

1. Add minimum 9GB LOGWRITES_DEV size requirement
The test creates LVM snapshots at each FUA point during replay, requiring significant space. Use _require_log_writes_sized to ensure sufficient space is available before running the test.

2. Fix state transition logic for verity enablement
The original test assumed orphan items would always be created during verity enablement (state 0->1 transition). However, in some cases verity completes without creating orphan items, causing the test to fail with "expected to reach verity done state".

Fix by transitioning to state 1 when either orphan items exist OR merkle items appear, handling both verity enablement paths. Also improve state 1 validation to only check for cleared merkle items when measurement actually fails.

The test now correctly handles verity enablement with or without orphan items while maintaining crash consistency validation.

v2:
- Use _require_log_writes_sized instead of separate _require_log_writes and _require_logwrites_size calls

---
 tests/btrfs/291 | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
tests/btrfs/291 +12 -5
index 122aeaa5..ac1d192d 100755
@@ -36,7 +36,7 @@ _cleanup()
 _require_scratch
 _require_test
 _require_loop
-_require_log_writes
+_require_log_writes_sized $((9 * 1024 * 1024))
 _require_dm_target snapshot
 _require_command $LVM_PROG lvm
 _require_scratch_verity
@@ -129,9 +129,14 @@ do
 	_udev_wait /dev/mapper/$vgname-$snapname
 
 	orphan=$(count_item $snap_dev ORPHAN)
-	[ $state -eq 0 ] && [ $orphan -gt 0 ] && state=1
-
 	pre_mount=$(count_merkle_items $snap_dev)
+
+	if [ $state -eq 0 ]; then
+		if [ $orphan -gt 0 ] || [ $pre_mount -gt 0 ]; then
+			state=1
+		fi
+	fi
+
 	_mount $snap_dev $SCRATCH_MNT || _fail "mount failed at entry $cur"
 	fsverity measure $SCRATCH_MNT/fsv >>$seqres.full 2>&1
 	measured=$?
@@ -143,8 +148,10 @@ do
 	echo "entry: $cur, state: $state, orphan: $orphan, pre_mount: $pre_mount, post_mount: $post_mount" >> $seqres.full
 
 	if [ $state -eq 1 ]; then
-		[ $post_mount -eq 0 ] || \
-			_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		if [ $measured -ne 0 ]; then
+			[ $post_mount -eq 0 ] || \
+				_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		fi
 	fi
 	if [ $state -eq 2 ]; then
 		[ $pre_mount -gt 0 ] || \
Signature · 1 line
2.45.1
#2 lore raw

Re: [PATCH v2 2/2] btrfs/291: fix state transition logic and add size requirement

Quoted 7 lines · Disha Goel

This patch fixes two issues in btrfs/291:

1. Add minimum 9GB LOGWRITES_DEV size requirement
The test creates LVM snapshots at each FUA point during replay, requiring significant space. Use _require_log_writes_sized to ensure sufficient space is available before running the test.

It also depends on the SCRATCH_DEV size; the fix fails if SCRATCH_DEV is
10G.

[  173.478493] BTRFS error (device dm-3): device total_bytes should be
at most 9663676416 but found 10737418240
[  173.482763] BTRFS error (device dm-3): failed to read chunk tree: -22
[  173.485457] BTRFS error (device dm-3): open_ctree failed: -22

Thanks.

Quoted 66 lines

2. Fix state transition logic for verity enablement
The original test assumed orphan items would always be created during verity enablement (state 0->1 transition). However, in some cases verity completes without creating orphan items, causing the test to fail with "expected to reach verity done state".

Fix by transitioning to state 1 when either orphan items exist OR merkle items appear, handling both verity enablement paths. Also improve state 1 validation to only check for cleared merkle items when measurement actually fails.

The test now correctly handles verity enablement with or without orphan items while maintaining crash consistency validation.

v2:
- Use _require_log_writes_sized instead of separate _require_log_writes and _require_logwrites_size calls

---
 tests/btrfs/291 | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
tests/btrfs/291 +12 -5
index 122aeaa5..ac1d192d 100755
@@ -36,7 +36,7 @@ _cleanup()
 _require_scratch
 _require_test
 _require_loop
-_require_log_writes
+_require_log_writes_sized $((9 * 1024 * 1024))
 _require_dm_target snapshot
 _require_command $LVM_PROG lvm
 _require_scratch_verity
@@ -129,9 +129,14 @@ do
 	_udev_wait /dev/mapper/$vgname-$snapname
 
 	orphan=$(count_item $snap_dev ORPHAN)
-	[ $state -eq 0 ] && [ $orphan -gt 0 ] && state=1
-
 	pre_mount=$(count_merkle_items $snap_dev)
+
+	if [ $state -eq 0 ]; then
+		if [ $orphan -gt 0 ] || [ $pre_mount -gt 0 ]; then
+			state=1
+		fi
+	fi
+
 	_mount $snap_dev $SCRATCH_MNT || _fail "mount failed at entry $cur"
 	fsverity measure $SCRATCH_MNT/fsv >>$seqres.full 2>&1
 	measured=$?
@@ -143,8 +148,10 @@ do
 	echo "entry: $cur, state: $state, orphan: $orphan, pre_mount: $pre_mount, post_mount: $post_mount" >> $seqres.full
 
 	if [ $state -eq 1 ]; then
-		[ $post_mount -eq 0 ] || \
-			_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		if [ $measured -ne 0 ]; then
+			[ $post_mount -eq 0 ] || \
+				_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		fi
 	fi
 	if [ $state -eq 2 ]; then
 		[ $pre_mount -gt 0 ] || \
author
#3 lore raw

Re: [PATCH v2 2/2] btrfs/291: fix state transition logic and add size requirement

Quoted 12 lines · Anand Jain
Quoted 7 lines · Disha Goel

This patch fixes two issues in btrfs/291:

1. Add minimum 9GB LOGWRITES_DEV size requirement
The test creates LVM snapshots at each FUA point during replay, requiring significant space. Use _require_log_writes_sized to ensure sufficient space is available before running the test.

It also depends on the SCRATCH_DEV size; the fix fails if SCRATCH_DEV is
10G.

Quoted 7 lines
[  173.478493] BTRFS error (device dm-3): device total_bytes should be
at most 9663676416 but found 10737418240
[  173.482763] BTRFS error (device dm-3): failed to read chunk tree: -22
[  173.485457] BTRFS error (device dm-3): open_ctree failed: -22

Thanks.

Thanks for the detailed review! I've addressed all the feedback in v3.

I've tested the fix with multiple SCRATCH_DEV sizes and confirmed it works correctly.

Quoted 68 lines
Quoted 67 lines

2. Fix state transition logic for verity enablement
The original test assumed orphan items would always be created during verity enablement (state 0->1 transition). However, in some cases verity completes without creating orphan items, causing the test to fail with "expected to reach verity done state".

Fix by transitioning to state 1 when either orphan items exist OR merkle items appear, handling both verity enablement paths. Also improve state 1 validation to only check for cleared merkle items when measurement actually fails.

The test now correctly handles verity enablement with or without orphan items while maintaining crash consistency validation.

v2:
- Use _require_log_writes_sized instead of separate _require_log_writes and _require_logwrites_size calls

---
  tests/btrfs/291 | 17 ++++++++++++-----
  1 file changed, 12 insertions(+), 5 deletions(-)
tests/btrfs/291 +12 -5
index 122aeaa5..ac1d192d 100755
@@ -36,7 +36,7 @@ _cleanup()
  _require_scratch
  _require_test
  _require_loop
-_require_log_writes
+_require_log_writes_sized $((9 * 1024 * 1024))
  _require_dm_target snapshot
  _require_command $LVM_PROG lvm
  _require_scratch_verity
@@ -129,9 +129,14 @@ do
  	_udev_wait /dev/mapper/$vgname-$snapname
 
  	orphan=$(count_item $snap_dev ORPHAN)
-	[ $state -eq 0 ] && [ $orphan -gt 0 ] && state=1
-
  	pre_mount=$(count_merkle_items $snap_dev)
+
+	if [ $state -eq 0 ]; then
+		if [ $orphan -gt 0 ] || [ $pre_mount -gt 0 ]; then
+			state=1
+		fi
+	fi
+
  	_mount $snap_dev $SCRATCH_MNT || _fail "mount failed at entry $cur"
  	fsverity measure $SCRATCH_MNT/fsv >>$seqres.full 2>&1
  	measured=$?
@@ -143,8 +148,10 @@ do
  	echo "entry: $cur, state: $state, orphan: $orphan, pre_mount: $pre_mount, post_mount: $post_mount" >> $seqres.full
 
  	if [ $state -eq 1 ]; then
-		[ $post_mount -eq 0 ] || \
-			_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		if [ $measured -ne 0 ]; then
+			[ $post_mount -eq 0 ] || \
+				_fail "mount failed to clear under-construction merkle items pre: $pre_mount, post: $post_mount at entry $cur";
+		fi
  	fi
  	if [ $state -eq 2 ]; then
  		[ $pre_mount -gt 0 ] || \
Signature · 2 lines
Regards,
Disha
#5 lore raw
Quoted 7 lines · Anand Jain

looks good.

Thanks

sorry, not yet; I'm canceling the RVB.
Details below.

Quoted 2 lines

v2:
- Renamed function from _require_logwrites_size to

_require_log_writes_sized

Quoted 3 lines

- Added documentation comment clarifying the parameter is in KB
- Made function call _require_log_writes() internally to avoid redundant calls in test code

This change log should be below the Signed-off-by "---"

Quoted 4 lines

---

... v2 changes here...

Quoted 8 lines
 common/dmlogwrites | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
common/dmlogwrites +0 -0
index a27e1966..0925b333 100644

common/dmlogwrites _require_log_writes() · line 23

Quoted 10 lines
common/dmlogwrites +7 -0
@@ -14,6 +14,21 @@ _require_log_writes()
 	_require_test_program "log-writes/replay-log"
 }
 
+# Require a log writes device of a minimum size
+# $1: minimum size in KB
+_require_log_writes_sized()
+{
+        local size=$1
+
+        [ $# -eq 1 ] || _fail "_require_log_writes_sized: expected

size param"

common/dmlogwrites _require_log_writes() · line 27

Quoted 4 lines
common/dmlogwrites +4 -0
+
+        _require_log_writes
+
+        local devsize=$(_get_device_size $LOGWRITES_DEV)

Please keep all declarations at the beginning of the function.

common/dmlogwrites _require_log_writes() · line 28

Quoted 1 line
common/dmlogwrites +1 -0
+        [ $devsize -lt $1 ] && \

use $size instead of $1 and

$devsize and $size units are different, 512 sectors vs KiB

common/dmlogwrites _require_log_writes() · line 29

Quoted 1 line
common/dmlogwrites +1 -0
+                _notrun "LOGWRITES_DEV too small, ${devsize}KB <

${size}KB"

common/dmlogwrites _require_log_writes() · line 31

Quoted 2 lines
common/dmlogwrites +2 -0
+}
+

$devsize is not in KB and $size is in KiB

#6 lore raw
Quoted 60 lines · Anand Jain
Quoted 7 lines · Anand Jain

looks good.

Thanks

sorry, not yet; I'm canceling the RVB.
Details below.

Quoted 2 lines

v2:
- Renamed function from _require_logwrites_size to

_require_log_writes_sized

Quoted 3 lines

- Added documentation comment clarifying the parameter is in KB
- Made function call _require_log_writes() internally to avoid redundant calls in test code

This change log should be below the Signed-off-by "---"

Quoted 4 lines

---

... v2 changes here...

Quoted 8 lines
 common/dmlogwrites | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
common/dmlogwrites +0 -0
index a27e1966..0925b333 100644

common/dmlogwrites _require_log_writes() · line 23

Quoted 10 lines
common/dmlogwrites +7 -0
@@ -14,6 +14,21 @@ _require_log_writes()
 	_require_test_program "log-writes/replay-log"
 }
 
+# Require a log writes device of a minimum size
+# $1: minimum size in KB
+_require_log_writes_sized()
+{
+        local size=$1
+
+        [ $# -eq 1 ] || _fail "_require_log_writes_sized: expected

size param"

common/dmlogwrites _require_log_writes() · line 27

Quoted 4 lines
common/dmlogwrites +4 -0
+
+        _require_log_writes
+
+        local devsize=$(_get_device_size $LOGWRITES_DEV)

Please keep all declarations at the beginning of the function.

Quoted 7 lines

common/dmlogwrites _require_log_writes() · line 28

Quoted 1 line
common/dmlogwrites +1 -0
+        [ $devsize -lt $1 ] && \

use $size instead of $1 and

$devsize and $size units are different, 512 sectors vs KiB

My bad, devsize is also in KiB. So this is fine.

Quoted 7 lines

common/dmlogwrites _require_log_writes() · line 29

Quoted 2 lines
common/dmlogwrites +1 -0
+                _notrun "LOGWRITES_DEV too small, ${devsize}KB <

${size}KB"

common/dmlogwrites _require_log_writes() · line 31

Quoted 2 lines
common/dmlogwrites +2 -0
+}
+

$devsize is not in KB and $size is in KiB

This is fine as well.

Thanks, Anand