Skip to contentSkip to conversation

xfs: test reproducible builds

PATCH Superseded Luca Di Maio 3 participants 7 messages Dec 15, 2025 – Dec 17, 2025

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 xfs: test reproducible builds

With the addition of the -p populate option, SOURCE_DATE_EPOCH and DETERMINISTIC_SEED support, it is possible to create fully reproducible pre-populated filesystems. We should test them here.

v1 -> v2:
- Changed test group from parent to mkfs
- Fixed PROTO_DIR to point to a new dir
- Populate PROTO_DIR with relevant file types
- Move from md5sum to sha256sum
v2 -> v3
- Properly check if mkfs.xfs supports SOURCE_DATE_EPOCH and DETERMINISTIC_SEED
- use fsstress program to generate the PROTO_DIR content
- simplify test output
v3 -> v4
- Add _cleanup function

---
 tests/xfs/841     | 171 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/841.out |   3 +
 2 files changed, 174 insertions(+)
 create mode 100755 tests/xfs/841
 create mode 100644 tests/xfs/841.out
tests/xfs/841 new file +171 -0
new file mode 100755
index 00000000..a75f5879
@@ -0,0 +1,171 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chainguard, Inc. All Rights Reserved.
+#
+# FS QA Test No. 841
+#
+# Test that XFS filesystems created with reproducibility options produce
+# identical images across multiple runs. This verifies that the combination
+# of SOURCE_DATE_EPOCH, DETERMINISTIC_SEED, and -m uuid= options result in
+# bit-for-bit reproducible filesystem images.
+
+. ./common/preamble
+_begin_fstest auto quick mkfs
+
+# Image file settings
+IMG_SIZE="512M"
+IMG_FILE="$TEST_DIR/xfs_reproducible_test.img"
+PROTO_DIR="$TEST_DIR/proto"
+
+# Fixed values for reproducibility
+FIXED_UUID="12345678-1234-1234-1234-123456789abc"
+FIXED_EPOCH="1234567890"
+
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}
+
+# Check if mkfs.xfs supports required options
+_check_mkfs_xfs_options()
+{
+	local check_img="$TEST_DIR/mkfs_check.img"
+	truncate -s 64M "$check_img" || return 1
+
+	# Check -m uuid support
+	$MKFS_XFS_PROG -m uuid=00000000-0000-0000-0000-000000000000 \
+		-N "$check_img" &> /dev/null
+	local uuid_support=$?
+
+	# Check -p support (protofile/directory population)
+	$MKFS_XFS_PROG 2>&1 | grep populate &> /dev/null
+	local proto_support=$?
+
+	grep -q SOURCE_DATE_EPOCH "$MKFS_XFS_PROG"
+	local reproducible_support=$?
+
+	rm -f "$check_img"
+
+	if [ $uuid_support -ne 0 ]; then
+		_notrun "mkfs.xfs does not support -m uuid= option"
+	fi
+	if [ $proto_support -ne 0 ]; then
+		_notrun "mkfs.xfs does not support -p option for directory population"
+	fi
+	if [ $reproducible_support -ne 0 ]; then
+		_notrun "mkfs.xfs does not support env options for reproducibility"
+	fi
+}
+
+# Create a prototype directory with all file types supported by mkfs.xfs -p
+_create_proto_dir()
+{
+	rm -rf "$PROTO_DIR"
+	mkdir -p "$PROTO_DIR"
+
+	$FSSTRESS_PROG -d $PROTO_DIR -s 1 $F -n 2000 -p 2 -z \
+		-f creat=15 \
+		-f mkdir=8 \
+		-f write=15 \
+		-f truncate=5 \
+		-f symlink=8 \
+		-f link=8 \
+		-f setfattr=12 \
+		-f chown=3 \
+		-f rename=5 \
+		-f unlink=2 \
+		-f rmdir=1
+
+
+	# FIFO (named pipe)
+	mkfifo "$PROTO_DIR/fifo"
+
+	# Unix socket
+	$here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true
+
+	# Block device (requires root)
+	mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true
+
+	# Character device (requires root)
+	mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true
+}
+
+_require_test
+_check_mkfs_xfs_options
+
+# Create XFS filesystem with full reproducibility options
+# Uses -p to populate from directory during mkfs (no mount needed)
+_mkfs_xfs_reproducible()
+{
+	local img=$1
+
+	# Create fresh image file
+	rm -f "$img"
+	truncate -s $IMG_SIZE "$img" || return 1
+
+	# Set environment variables for reproducibility:
+	# - SOURCE_DATE_EPOCH: fixes all inode timestamps to this value
+	# - DETERMINISTIC_SEED: uses fixed seed (0x53454544) instead of
+	#   getrandom()
+	#
+	# mkfs.xfs options:
+	# - -m uuid=: fixed filesystem UUID
+	# - -p dir: populate filesystem from directory during creation
+	SOURCE_DATE_EPOCH=$FIXED_EPOCH \
+	DETERMINISTIC_SEED=1 \
+	$MKFS_XFS_PROG \
+		-f \
+		-m uuid=$FIXED_UUID \
+		-p "$PROTO_DIR" \
+		"$img" >> $seqres.full 2>&1
+
+	return $?
+}
+
+# Compute hash of the image file
+_hash_image()
+{
+	sha256sum "$1" | awk '{print $1}'
+}
+
+# Run a single reproducibility test iteration
+_run_iteration()
+{
+	local iteration=$1
+
+	echo "Iteration $iteration: Creating filesystem with -p $PROTO_DIR" >> $seqres.full
+	if ! _mkfs_xfs_reproducible "$IMG_FILE"; then
+		echo "mkfs.xfs failed" >> $seqres.full
+		return 1
+	fi
+
+	local hash=$(_hash_image "$IMG_FILE")
+	echo "Iteration $iteration: Hash = $hash" >> $seqres.full
+
+	echo $hash
+}
+
+# Create the prototype directory with various file types
+_create_proto_dir
+
+echo "Test: XFS reproducible filesystem image creation"
+
+# Run three iterations
+hash1=$(_run_iteration 1)
+[ -z "$hash1" ] && _fail "Iteration 1 failed"
+
+hash2=$(_run_iteration 2)
+[ -z "$hash2" ] && _fail "Iteration 2 failed"
+
+hash3=$(_run_iteration 3)
+[ -z "$hash3" ] && _fail "Iteration 3 failed"
+
+# Verify all hashes match
+if [ "$hash1" = "$hash2" ] && [ "$hash2" = "$hash3" ]; then
+	echo "All filesystem images are identical."
+else
+	echo "ERROR: Filesystem images differ!"
+fi
+
+# success, all done
+status=0
+exit
tests/xfs/841.out new file +3 -0
new file mode 100644
index 00000000..3bdfbfda
@@ -0,0 +1,3 @@
+QA output created by 841
+Test: XFS reproducible filesystem image creation
+All filesystem images are identical.
Signature · 1 line
2.51.0
#1 lore raw
Quoted 47 lines · Luca Di Maio

With the addition of the -p populate option, SOURCE_DATE_EPOCH and DETERMINISTIC_SEED support, it is possible to create fully reproducible pre-populated filesystems. We should test them here.

v1 -> v2:
- Changed test group from parent to mkfs
- Fixed PROTO_DIR to point to a new dir
- Populate PROTO_DIR with relevant file types
- Move from md5sum to sha256sum
v2 -> v3
- Properly check if mkfs.xfs supports SOURCE_DATE_EPOCH and DETERMINISTIC_SEED
- use fsstress program to generate the PROTO_DIR content
- simplify test output
v3 -> v4
- Add _cleanup function

---
 tests/xfs/841     | 171 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/841.out |   3 +
 2 files changed, 174 insertions(+)
 create mode 100755 tests/xfs/841
 create mode 100644 tests/xfs/841.out
tests/xfs/841 new file +16 -0
new file mode 100755
index 00000000..a75f5879
@@ -0,0 +1,171 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Chainguard, Inc. All Rights Reserved.
+#
+# FS QA Test No. 841
+#
+# Test that XFS filesystems created with reproducibility options produce
+# identical images across multiple runs. This verifies that the combination
+# of SOURCE_DATE_EPOCH, DETERMINISTIC_SEED, and -m uuid= options result in
+# bit-for-bit reproducible filesystem images.
+
+. ./common/preamble
+_begin_fstest auto quick mkfs
+
+# Image file settings
+IMG_SIZE="512M"

tests/xfs/841 line 26

Quoted 10 lines
tests/xfs/841 new file +10 -0
+IMG_FILE="$TEST_DIR/xfs_reproducible_test.img"
+PROTO_DIR="$TEST_DIR/proto"
+
+# Fixed values for reproducibility
+FIXED_UUID="12345678-1234-1234-1234-123456789abc"
+FIXED_EPOCH="1234567890"
+
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}

After test specific cleanup, this needs to call _generic_cleanup() to handle all the internal test state cleanup requirements.

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
Quoted 6 lines · Dave Chinner

tests/xfs/841 line 26

Quoted 3 lines
tests/xfs/841 new file +3 -0
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}

After test specific cleanup, this needs to call _generic_cleanup() to handle all the internal test state cleanup requirements.

There's no such thing as _generic_cleanup, and none of the _cleanup()-using tests that I've looked at recently hooks into any kind of generic cleanup routine.

#3 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Dave Chinner

tests/xfs/841 line 26

Quoted 3 lines
tests/xfs/841 new file +3 -0
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}

After test specific cleanup, this needs to call _generic_cleanup() to handle all the internal test state cleanup requirements.

There's no such thing as _generic_cleanup, and none of the _cleanup()-using tests that I've looked at recently hooks into any kind of generic cleanup routine.

Oh, I forgot that was one of the huge cleanups I have sitting around in my local fstests tree. i.e. making all the tests in fstests do custom _cleanup() operations consistently and sanely.

It factors the _cleanup() function in common/preamble into _generic_cleanup(), then fixes all the custom test _cleanup() functions to do the custom test cleanup then call _generic_cleanup() to do the rest.

The patch fixes a -lot- of random test cleanup failures when cancelling a test run with ctrl-c because custom cleanup functions are rarely tested....

Not a small patch, though, and likely needs to be updated because I haven't touched it in a while:

278 files changed, 326 insertions(+), 1037 deletions(-)

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw
In reply to Christoph Hellwig
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Dave Chinner

tests/xfs/841 line 26

Quoted 3 lines
tests/xfs/841 new file +3 -0
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}

After test specific cleanup, this needs to call _generic_cleanup() to handle all the internal test state cleanup requirements.

There's no such thing as _generic_cleanup, and none of the _cleanup()-using tests that I've looked at recently hooks into any kind of generic cleanup routine.

I forgot to mention: the lack of _generic_cleanup() doesn't mean my review comment should be ignored - the new custom _cleanup() function above still needs to do the relevant generic cleanup work that is done in common/preamble::_cleanup()...

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#5 lore raw
Quoted 14 lines · Dave Chinner
Quoted 9 lines · Christoph Hellwig
Quoted 7 lines · Dave Chinner

tests/xfs/841 line 26

Quoted 3 lines
tests/xfs/841 new file +3 -0
+_cleanup() {
+	rm -r -f "$PROTO_DIR" "$IMG_FILE"
+}

After test specific cleanup, this needs to call _generic_cleanup()
to handle all the internal test state cleanup requirements.

Quoted 3 lines

There's no such thing as _generic_cleanup, and none of the _cleanup()-using tests that I've looked at recently hooks into any kind of generic cleanup routine.

Quoted 6 lines

I forgot to mention: the lack of _generic_cleanup() doesn't mean my review comment should be ignored - the new custom _cleanup() function above still needs to do the relevant generic cleanup work that is done in common/preamble::_cleanup()...

-Dave.

Ack, incoming thanks for the review
L.

Reviewed-by on v4 1/1
#6 lore raw
In reply to Luca Di Maio

[adding the fstests list where this really should be sent. Sorry for not noticing earlier.]

Quoted 3 lines · Luca Di Maio

With the addition of the -p populate option, SOURCE_DATE_EPOCH and DETERMINISTIC_SEED support, it is possible to create fully reproducible pre-populated filesystems. We should test them here.

Looks good: