Skip to contentSkip to conversation

xfs: new helper grows the mounted scratch XFS filesystem

PATCH In review Zorro Lang 2 participants 4 messages May 10, 2018 – May 11, 2018

lore
Share & tools
lore

Zorro Lang

author
#0 lore raw

PATCH v2 1/1 xfs: new helper grows the mounted scratch XFS filesystem

When test on large SCRATCH_DEV, grow a small XFS to huge size is a horrible thing (e.g grow 128m to 500T). So add a helper named _scratch_xfs_growfs() to do below things:

1) If a 'size' argument is specified, the filesystem will not be grown larger than that size (depend on the device size).
2) else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will not be grown larger than 10x the current fs size.
3) else the scratch filesystem will be expanded to fit the scratch device.

Signed-off-by Zorro Lang <[email protected]>

---

Hi,

Thanks Eryu and Darrick help to review this patch.

V2 has below changes:
1) Change function name _scratch_xfs_growfs_limited to _scratch_xfs_growfs
2) Change the comment about _scratch_xfs_growfs
3) Change local variable limit_size to max_size

Thanks,
Zorro

 common/xfs    | 36 ++++++++++++++++++++++++++++++++++++
 tests/xfs/002 |  2 +-
 tests/xfs/127 |  2 +-
 tests/xfs/233 |  2 +-
 4 files changed, 39 insertions(+), 3 deletions(-)
common/xfs +36 -0
index e0bc3f43..a6fe5bf9 100644
@@ -721,3 +721,39 @@ _require_xfs_db_write_array()
 	rm -f $TEST_DIR/$seq.img
 	[ $supported -eq 0 ] && _notrun "xfs_db write can't support array"
 }
+
+# Usage: _scratch_xfs_growfs [max_size_in_bytes]
+#
+# If max_size_in_bytes is specified, the filesystem will *not* be grown larger
+# than that size (depend on the device size).
+# Else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will
+# not be grown larger than 10x the current fs size.
+# Else the scratch filesystem will be expanded to fit the scratch device.
+_scratch_xfs_growfs()
+{
+	local max_size="$1"
+	local option=""
+
+	if [ "$LARGE_SCRATCH_DEV" = "yes" -o -n "$max_size" ]; then
+		local tmp=`mktemp -u`
+		xfs_info $SCRATCH_MNT | _filter_mkfs > /dev/null 2>$tmp.info
+		. $tmp.info
+		rm -f $tmp.info
+
+		local fs_size=$((dbsize * dblocks))
+		local dev_size_kb=`_get_device_size $SCRATCH_DEV`
+
+		# default max_size is 10 times current fs size.
+		if [ -z "$max_size" ]; then
+			max_size=$((fs_size * 10))
+		fi
+		# don't limit growfs size if device size is smaller
+		if [ $((dev_size_kb * 1024)) -gt $max_size ]; then
+			option="-D $((max_size / dbsize))"
+		else
+			option=""
+		fi
+	fi
+
+	$XFS_GROWFS_PROG $option $SCRATCH_MNT
+}
tests/xfs/002 +1 -1
index 741117be..a77a8719 100755
@@ -68,7 +68,7 @@ _scratch_xfs_db -x -c "sb 2" -c "type data" -c "write fill 0xff 224 4"
 _scratch_mount
 
 # This should pass
-$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full 2>&1 || _fail "growfs failed"
+_scratch_xfs_growfs >> $seqres.full 2>&1 || _fail "growfs failed"
 
 # success, all done
 status=0
tests/xfs/127 +1 -1
index 9df99904..272ea959 100755
@@ -61,7 +61,7 @@ _cp_reflink $testdir/original $testdir/copy1
 _cp_reflink $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 | _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more reflink copies"
tests/xfs/233 +1 -1
index e61c444d..f3689bc9 100755
@@ -59,7 +59,7 @@ cp -p $testdir/original $testdir/copy1
 cp -p $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 |  _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more copies"
Signature · 1 line
2.14.3
#1 lore raw
Quoted 46 lines · Zorro Lang

When test on large SCRATCH_DEV, grow a small XFS to huge size is a horrible thing (e.g grow 128m to 500T). So add a helper named _scratch_xfs_growfs() to do below things:

1) If a 'size' argument is specified, the filesystem will not be grown larger than that size (depend on the device size).
2) else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will not be grown larger than 10x the current fs size.
3) else the scratch filesystem will be expanded to fit the scratch device.

Signed-off-by Zorro Lang <[email protected]>

---

Hi,

Thanks Eryu and Darrick help to review this patch.

V2 has below changes:
1) Change function name _scratch_xfs_growfs_limited to _scratch_xfs_growfs
2) Change the comment about _scratch_xfs_growfs
3) Change local variable limit_size to max_size

Thanks,
Zorro

 common/xfs    | 36 ++++++++++++++++++++++++++++++++++++
 tests/xfs/002 |  2 +-
 tests/xfs/127 |  2 +-
 tests/xfs/233 |  2 +-
 4 files changed, 39 insertions(+), 3 deletions(-)
common/xfs +6 -0
index e0bc3f43..a6fe5bf9 100644
@@ -721,3 +721,39 @@ _require_xfs_db_write_array()
 	rm -f $TEST_DIR/$seq.img
 	[ $supported -eq 0 ] && _notrun "xfs_db write can't support array"
 }
+
+# Usage: _scratch_xfs_growfs [max_size_in_bytes]
+#
+# If max_size_in_bytes is specified, the filesystem will *not* be grown larger
+# than that size (depend on the device size).
+# Else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will

common/xfs _require_xfs_db_write_array() · line 739

Quoted 10 lines
common/xfs +10 -0
+# not be grown larger than 10x the current fs size.
+# Else the scratch filesystem will be expanded to fit the scratch device.
+_scratch_xfs_growfs()
+{
+	local max_size="$1"
+	local option=""
+
+	if [ "$LARGE_SCRATCH_DEV" = "yes" -o -n "$max_size" ]; then
+		local tmp=`mktemp -u`
+		xfs_info $SCRATCH_MNT | _filter_mkfs > /dev/null 2>$tmp.info

$XFS_INFO_PROG, not xfs_info (?)

Otherwise mostly looks fine to me.

--D

Quoted 66 lines
common/xfs +20 -0
+		. $tmp.info
+		rm -f $tmp.info
+
+		local fs_size=$((dbsize * dblocks))
+		local dev_size_kb=`_get_device_size $SCRATCH_DEV`
+
+		# default max_size is 10 times current fs size.
+		if [ -z "$max_size" ]; then
+			max_size=$((fs_size * 10))
+		fi
+		# don't limit growfs size if device size is smaller
+		if [ $((dev_size_kb * 1024)) -gt $max_size ]; then
+			option="-D $((max_size / dbsize))"
+		else
+			option=""
+		fi
+	fi
+
+	$XFS_GROWFS_PROG $option $SCRATCH_MNT
+}
tests/xfs/002 +1 -1
index 741117be..a77a8719 100755
@@ -68,7 +68,7 @@ _scratch_xfs_db -x -c "sb 2" -c "type data" -c "write fill 0xff 224 4"
 _scratch_mount
 
 # This should pass
-$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full 2>&1 || _fail "growfs failed"
+_scratch_xfs_growfs >> $seqres.full 2>&1 || _fail "growfs failed"
 
 # success, all done
 status=0
tests/xfs/127 +1 -1
index 9df99904..272ea959 100755
@@ -61,7 +61,7 @@ _cp_reflink $testdir/original $testdir/copy1
 _cp_reflink $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 | _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more reflink copies"
tests/xfs/233 +1 -1
index e61c444d..f3689bc9 100755
@@ -59,7 +59,7 @@ cp -p $testdir/original $testdir/copy1
 cp -p $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 |  _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more copies"
Signature · 6 lines
2.14.3

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author
#2 lore raw
Quoted 60 lines · Darrick J. Wong
Quoted 46 lines · Zorro Lang

When test on large SCRATCH_DEV, grow a small XFS to huge size is a horrible thing (e.g grow 128m to 500T). So add a helper named _scratch_xfs_growfs() to do below things:

1) If a 'size' argument is specified, the filesystem will not be grown larger than that size (depend on the device size).
2) else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will not be grown larger than 10x the current fs size.
3) else the scratch filesystem will be expanded to fit the scratch device.

Signed-off-by Zorro Lang <[email protected]>

---

Hi,

Thanks Eryu and Darrick help to review this patch.

V2 has below changes:
1) Change function name _scratch_xfs_growfs_limited to _scratch_xfs_growfs
2) Change the comment about _scratch_xfs_growfs
3) Change local variable limit_size to max_size

Thanks,
Zorro

 common/xfs    | 36 ++++++++++++++++++++++++++++++++++++
 tests/xfs/002 |  2 +-
 tests/xfs/127 |  2 +-
 tests/xfs/233 |  2 +-
 4 files changed, 39 insertions(+), 3 deletions(-)
common/xfs +6 -0
index e0bc3f43..a6fe5bf9 100644
@@ -721,3 +721,39 @@ _require_xfs_db_write_array()
 	rm -f $TEST_DIR/$seq.img
 	[ $supported -eq 0 ] && _notrun "xfs_db write can't support array"
 }
+
+# Usage: _scratch_xfs_growfs [max_size_in_bytes]
+#
+# If max_size_in_bytes is specified, the filesystem will *not* be grown larger
+# than that size (depend on the device size).
+# Else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will

common/xfs _require_xfs_db_write_array() · line 739

Quoted 10 lines
common/xfs +10 -0
+# not be grown larger than 10x the current fs size.
+# Else the scratch filesystem will be expanded to fit the scratch device.
+_scratch_xfs_growfs()
+{
+	local max_size="$1"
+	local option=""
+
+	if [ "$LARGE_SCRATCH_DEV" = "yes" -o -n "$max_size" ]; then
+		local tmp=`mktemp -u`
+		xfs_info $SCRATCH_MNT | _filter_mkfs > /dev/null 2>$tmp.info

$XFS_INFO_PROG, not xfs_info (?)

Quoted 1 line

Otherwise mostly looks fine to me.

There's not $XFS_INFO_PROG in xfstests for now.

Quoted 73 lines

--D

Quoted 66 lines
common/xfs +20 -0
+		. $tmp.info
+		rm -f $tmp.info
+
+		local fs_size=$((dbsize * dblocks))
+		local dev_size_kb=`_get_device_size $SCRATCH_DEV`
+
+		# default max_size is 10 times current fs size.
+		if [ -z "$max_size" ]; then
+			max_size=$((fs_size * 10))
+		fi
+		# don't limit growfs size if device size is smaller
+		if [ $((dev_size_kb * 1024)) -gt $max_size ]; then
+			option="-D $((max_size / dbsize))"
+		else
+			option=""
+		fi
+	fi
+
+	$XFS_GROWFS_PROG $option $SCRATCH_MNT
+}
tests/xfs/002 +1 -1
index 741117be..a77a8719 100755
@@ -68,7 +68,7 @@ _scratch_xfs_db -x -c "sb 2" -c "type data" -c "write fill 0xff 224 4"
 _scratch_mount
 
 # This should pass
-$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full 2>&1 || _fail "growfs failed"
+_scratch_xfs_growfs >> $seqres.full 2>&1 || _fail "growfs failed"
 
 # success, all done
 status=0
tests/xfs/127 +1 -1
index 9df99904..272ea959 100755
@@ -61,7 +61,7 @@ _cp_reflink $testdir/original $testdir/copy1
 _cp_reflink $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 | _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more reflink copies"
tests/xfs/233 +1 -1
index e61c444d..f3689bc9 100755
@@ -59,7 +59,7 @@ cp -p $testdir/original $testdir/copy1
 cp -p $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 |  _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more copies"
Signature · 6 lines
2.14.3

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 61 lines · Zorro Lang
Quoted 60 lines · Darrick J. Wong
Quoted 46 lines · Zorro Lang

When test on large SCRATCH_DEV, grow a small XFS to huge size is a horrible thing (e.g grow 128m to 500T). So add a helper named _scratch_xfs_growfs() to do below things:

1) If a 'size' argument is specified, the filesystem will not be grown larger than that size (depend on the device size).
2) else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will not be grown larger than 10x the current fs size.
3) else the scratch filesystem will be expanded to fit the scratch device.

Signed-off-by Zorro Lang <[email protected]>

---

Hi,

Thanks Eryu and Darrick help to review this patch.

V2 has below changes:
1) Change function name _scratch_xfs_growfs_limited to _scratch_xfs_growfs
2) Change the comment about _scratch_xfs_growfs
3) Change local variable limit_size to max_size

Thanks,
Zorro

 common/xfs    | 36 ++++++++++++++++++++++++++++++++++++
 tests/xfs/002 |  2 +-
 tests/xfs/127 |  2 +-
 tests/xfs/233 |  2 +-
 4 files changed, 39 insertions(+), 3 deletions(-)
common/xfs +6 -0
index e0bc3f43..a6fe5bf9 100644
@@ -721,3 +721,39 @@ _require_xfs_db_write_array()
 	rm -f $TEST_DIR/$seq.img
 	[ $supported -eq 0 ] && _notrun "xfs_db write can't support array"
 }
+
+# Usage: _scratch_xfs_growfs [max_size_in_bytes]
+#
+# If max_size_in_bytes is specified, the filesystem will *not* be grown larger
+# than that size (depend on the device size).
+# Else if LARGE_SCRATCH_DEV = yes (--large-fs option), the filesystem will

common/xfs _require_xfs_db_write_array() · line 739

Quoted 10 lines
common/xfs +10 -0
+# not be grown larger than 10x the current fs size.
+# Else the scratch filesystem will be expanded to fit the scratch device.
+_scratch_xfs_growfs()
+{
+	local max_size="$1"
+	local option=""
+
+	if [ "$LARGE_SCRATCH_DEV" = "yes" -o -n "$max_size" ]; then
+		local tmp=`mktemp -u`
+		xfs_info $SCRATCH_MNT | _filter_mkfs > /dev/null 2>$tmp.info

$XFS_INFO_PROG, not xfs_info (?)

Quoted 3 lines
Quoted 1 line

Otherwise mostly looks fine to me.

There's not $XFS_INFO_PROG in xfstests for now.

DOH, sorry, it exists in my tree as a patch that I've been holding onto for xfsprogs 4.17. Ignore me for now.

--D

Quoted 73 lines
Quoted 73 lines

--D

Quoted 66 lines
common/xfs +20 -0
+		. $tmp.info
+		rm -f $tmp.info
+
+		local fs_size=$((dbsize * dblocks))
+		local dev_size_kb=`_get_device_size $SCRATCH_DEV`
+
+		# default max_size is 10 times current fs size.
+		if [ -z "$max_size" ]; then
+			max_size=$((fs_size * 10))
+		fi
+		# don't limit growfs size if device size is smaller
+		if [ $((dev_size_kb * 1024)) -gt $max_size ]; then
+			option="-D $((max_size / dbsize))"
+		else
+			option=""
+		fi
+	fi
+
+	$XFS_GROWFS_PROG $option $SCRATCH_MNT
+}
tests/xfs/002 +1 -1
index 741117be..a77a8719 100755
@@ -68,7 +68,7 @@ _scratch_xfs_db -x -c "sb 2" -c "type data" -c "write fill 0xff 224 4"
 _scratch_mount
 
 # This should pass
-$XFS_GROWFS_PROG $SCRATCH_MNT >> $seqres.full 2>&1 || _fail "growfs failed"
+_scratch_xfs_growfs >> $seqres.full 2>&1 || _fail "growfs failed"
 
 # success, all done
 status=0
tests/xfs/127 +1 -1
index 9df99904..272ea959 100755
@@ -61,7 +61,7 @@ _cp_reflink $testdir/original $testdir/copy1
 _cp_reflink $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 | _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more reflink copies"
tests/xfs/233 +1 -1
index e61c444d..f3689bc9 100755
@@ -59,7 +59,7 @@ cp -p $testdir/original $testdir/copy1
 cp -p $testdir/copy1 $testdir/copy2
 
 echo "Grow fs"
-$XFS_GROWFS_PROG $SCRATCH_MNT 2>&1 |  _filter_growfs >> $seqres.full
+_scratch_xfs_growfs 2>&1 |  _filter_growfs >> $seqres.full
 _scratch_cycle_mount
 
 echo "Create more copies"
Signature · 6 lines
2.14.3

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe fstests" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html