xfs: test reproducible builds
PATCH v6 1/1 xfs: test reproducible builds
With the addition of the -p populate option, SOURCE_DATE_EPOCH and DETERMINISTIC_SEED support, it is possible to create fully reproducible pre-populated filesystems. We should test them here.
v1 -> v2:
- Changed test group from parent to mkfs
- Fixed PROTO_DIR to point to a new dir
- Populate PROTO_DIR with relevant file types
- Move from md5sum to sha256sum
v2 -> v3
- Properly check if mkfs.xfs supports SOURCE_DATE_EPOCH and DETERMINISTIC_SEED
- use fsstress program to generate the PROTO_DIR content
- simplify test output
v3 -> v4
- Add _cleanup function
v4 -> v5
- copy _cleanup from common/preamble
v5 -> v6
- remove debug typo/leftover
--- tests/xfs/841 | 173 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/841.out | 3 + 2 files changed, 176 insertions(+) create mode 100755 tests/xfs/841 create mode 100644 tests/xfs/841.out
tests/xfs/841 new file +173 -0
@@ -0,0 +1,173 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Chainguard, Inc. All Rights Reserved. +# +# FS QA Test No. 841 +# +# Test that XFS filesystems created with reproducibility options produce +# identical images across multiple runs. This verifies that the combination +# of SOURCE_DATE_EPOCH, DETERMINISTIC_SEED, and -m uuid= options result in +# bit-for-bit reproducible filesystem images. + +. ./common/preamble +_begin_fstest auto quick mkfs + +# Image file settings +IMG_SIZE="512M" +IMG_FILE="$TEST_DIR/xfs_reproducible_test.img" +PROTO_DIR="$TEST_DIR/proto" + +# Fixed values for reproducibility +FIXED_UUID="12345678-1234-1234-1234-123456789abc" +FIXED_EPOCH="1234567890" + +_cleanup() { + cd / + command -v _kill_fsstress &>/dev/null && _kill_fsstress + rm -r -f $tmp.* "$PROTO_DIR" "$IMG_FILE" +} + +# Check if mkfs.xfs supports required options +_check_mkfs_xfs_options() +{ + local check_img="$TEST_DIR/mkfs_check.img" + truncate -s 64M "$check_img" || return 1 + + # Check -m uuid support + $MKFS_XFS_PROG -m uuid=00000000-0000-0000-0000-000000000000 \ + -N "$check_img" &> /dev/null + local uuid_support=$? + + # Check -p support (protofile/directory population) + $MKFS_XFS_PROG 2>&1 | grep populate &> /dev/null + local proto_support=$? + + grep -q SOURCE_DATE_EPOCH "$MKFS_XFS_PROG" + local reproducible_support=$? + + rm -f "$check_img" + + if [ $uuid_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -m uuid= option" + fi + if [ $proto_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -p option for directory population" + fi + if [ $reproducible_support -ne 0 ]; then + _notrun "mkfs.xfs does not support env options for reproducibility" + fi +} + +# Create a prototype directory with all file types supported by mkfs.xfs -p +_create_proto_dir() +{ + rm -rf "$PROTO_DIR" + mkdir -p "$PROTO_DIR" + + $FSSTRESS_PROG -d $PROTO_DIR -s 1 -n 2000 -p 2 -z \ + -f creat=15 \ + -f mkdir=8 \ + -f write=15 \ + -f truncate=5 \ + -f symlink=8 \ + -f link=8 \ + -f setfattr=12 \ + -f chown=3 \ + -f rename=5 \ + -f unlink=2 \ + -f rmdir=1 + + + # FIFO (named pipe) + mkfifo "$PROTO_DIR/fifo" + + # Unix socket + $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true + + # Block device (requires root) + mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + + # Character device (requires root) + mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true +} + +_require_test +_check_mkfs_xfs_options + +# Create XFS filesystem with full reproducibility options +# Uses -p to populate from directory during mkfs (no mount needed) +_mkfs_xfs_reproducible() +{ + local img=$1 + + # Create fresh image file + rm -f "$img" + truncate -s $IMG_SIZE "$img" || return 1 + + # Set environment variables for reproducibility: + # - SOURCE_DATE_EPOCH: fixes all inode timestamps to this value + # - DETERMINISTIC_SEED: uses fixed seed (0x53454544) instead of + # getrandom() + # + # mkfs.xfs options: + # - -m uuid=: fixed filesystem UUID + # - -p dir: populate filesystem from directory during creation + SOURCE_DATE_EPOCH=$FIXED_EPOCH \ + DETERMINISTIC_SEED=1 \ + $MKFS_XFS_PROG \ + -f \ + -m uuid=$FIXED_UUID \ + -p "$PROTO_DIR" \ + "$img" >> $seqres.full 2>&1 + + return $? +} + +# Compute hash of the image file +_hash_image() +{ + sha256sum "$1" | awk '{print $1}' +} + +# Run a single reproducibility test iteration +_run_iteration() +{ + local iteration=$1 + + echo "Iteration $iteration: Creating filesystem with -p $PROTO_DIR" >> $seqres.full + if ! _mkfs_xfs_reproducible "$IMG_FILE"; then + echo "mkfs.xfs failed" >> $seqres.full + return 1 + fi + + local hash=$(_hash_image "$IMG_FILE") + echo "Iteration $iteration: Hash = $hash" >> $seqres.full + + echo $hash +} + +# Create the prototype directory with various file types +_create_proto_dir + +echo "Test: XFS reproducible filesystem image creation" + +# Run three iterations +hash1=$(_run_iteration 1) +[ -z "$hash1" ] && _fail "Iteration 1 failed" + +hash2=$(_run_iteration 2) +[ -z "$hash2" ] && _fail "Iteration 2 failed" + +hash3=$(_run_iteration 3) +[ -z "$hash3" ] && _fail "Iteration 3 failed" + +# Verify all hashes match +if [ "$hash1" = "$hash2" ] && [ "$hash2" = "$hash3" ]; then + echo "All filesystem images are identical." +else + echo "ERROR: Filesystem images differ!" +fi + +# success, all done +status=0 +exit
tests/xfs/841.out new file +3 -0
@@ -0,0 +1,3 @@ +QA output created by 841 +Test: XFS reproducible filesystem image creation +All filesystem images are identical.
Signature · 1 line
2.51.0
Still looks good:
Quoted 21 lines · Luca Di Maio
With the addition of the
-ppopulate option,SOURCE_DATE_EPOCHandDETERMINISTIC_SEEDsupport, it is possible to create fully reproducible pre-populated filesystems. We should test them here.v1 -> v2:
- Changed test group from parent to mkfs
- FixedPROTO_DIRto point to a new dir
- PopulatePROTO_DIRwith relevant file types
- Move from md5sum to sha256sum
v2 -> v3
- Properly check if mkfs.xfs supportsSOURCE_DATE_EPOCHandDETERMINISTIC_SEED
- use fsstress program to generate thePROTO_DIRcontent
- simplify test output
v3 -> v4
- Add_cleanupfunction
v4 -> v5
- copy_cleanupfrom common/preamble
v5 -> v6
- remove debug typo/leftover
Quoted 1 line
Signed-off-by Luca Di Maio <[email protected]>
Looks good to me still
--D
Quoted 199 lines
--- tests/xfs/841 | 173 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/841.out | 3 + 2 files changed, 176 insertions(+) create mode 100755 tests/xfs/841 create mode 100644 tests/xfs/841.outtests/xfs/841 new file +173
-0@@ -0,0 +1,173 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Chainguard, Inc. All Rights Reserved. +# +# FS QA Test No. 841 +# +# Test that XFS filesystems created with reproducibility options produce +# identical images across multiple runs. This verifies that the combination +# of SOURCE_DATE_EPOCH, DETERMINISTIC_SEED, and -m uuid= options result in +# bit-for-bit reproducible filesystem images. + +. ./common/preamble +_begin_fstest auto quick mkfs + +# Image file settings +IMG_SIZE="512M" +IMG_FILE="$TEST_DIR/xfs_reproducible_test.img" +PROTO_DIR="$TEST_DIR/proto" + +# Fixed values for reproducibility +FIXED_UUID="12345678-1234-1234-1234-123456789abc" +FIXED_EPOCH="1234567890" + +_cleanup() { + cd / + command -v _kill_fsstress &>/dev/null && _kill_fsstress + rm -r -f $tmp.* "$PROTO_DIR" "$IMG_FILE" +} + +# Check if mkfs.xfs supports required options +_check_mkfs_xfs_options() +{ + local check_img="$TEST_DIR/mkfs_check.img" + truncate -s 64M "$check_img" || return 1 + + # Check -m uuid support + $MKFS_XFS_PROG -m uuid=00000000-0000-0000-0000-000000000000 \ + -N "$check_img" &> /dev/null + local uuid_support=$? + + # Check -p support (protofile/directory population) + $MKFS_XFS_PROG 2>&1 | grep populate &> /dev/null + local proto_support=$? + + grep -q SOURCE_DATE_EPOCH "$MKFS_XFS_PROG" + local reproducible_support=$? + + rm -f "$check_img" + + if [ $uuid_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -m uuid= option" + fi + if [ $proto_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -p option for directory population" + fi + if [ $reproducible_support -ne 0 ]; then + _notrun "mkfs.xfs does not support env options for reproducibility" + fi +} + +# Create a prototype directory with all file types supported by mkfs.xfs -p +_create_proto_dir() +{ + rm -rf "$PROTO_DIR" + mkdir -p "$PROTO_DIR" + + $FSSTRESS_PROG -d $PROTO_DIR -s 1 -n 2000 -p 2 -z \ + -f creat=15 \ + -f mkdir=8 \ + -f write=15 \ + -f truncate=5 \ + -f symlink=8 \ + -f link=8 \ + -f setfattr=12 \ + -f chown=3 \ + -f rename=5 \ + -f unlink=2 \ + -f rmdir=1 + + + # FIFO (named pipe) + mkfifo "$PROTO_DIR/fifo" + + # Unix socket + $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true + + # Block device (requires root) + mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + + # Character device (requires root) + mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true +} + +_require_test +_check_mkfs_xfs_options + +# Create XFS filesystem with full reproducibility options +# Uses -p to populate from directory during mkfs (no mount needed) +_mkfs_xfs_reproducible() +{ + local img=$1 + + # Create fresh image file + rm -f "$img" + truncate -s $IMG_SIZE "$img" || return 1 + + # Set environment variables for reproducibility: + # - SOURCE_DATE_EPOCH: fixes all inode timestamps to this value + # - DETERMINISTIC_SEED: uses fixed seed (0x53454544) instead of + # getrandom() + # + # mkfs.xfs options: + # - -m uuid=: fixed filesystem UUID + # - -p dir: populate filesystem from directory during creation + SOURCE_DATE_EPOCH=$FIXED_EPOCH \ + DETERMINISTIC_SEED=1 \ + $MKFS_XFS_PROG \ + -f \ + -m uuid=$FIXED_UUID \ + -p "$PROTO_DIR" \ + "$img" >> $seqres.full 2>&1 + + return $? +} + +# Compute hash of the image file +_hash_image() +{ + sha256sum "$1" | awk '{print $1}' +} + +# Run a single reproducibility test iteration +_run_iteration() +{ + local iteration=$1 + + echo "Iteration $iteration: Creating filesystem with -p $PROTO_DIR" >> $seqres.full + if ! _mkfs_xfs_reproducible "$IMG_FILE"; then + echo "mkfs.xfs failed" >> $seqres.full + return 1 + fi + + local hash=$(_hash_image "$IMG_FILE") + echo "Iteration $iteration: Hash = $hash" >> $seqres.full + + echo $hash +} + +# Create the prototype directory with various file types +_create_proto_dir + +echo "Test: XFS reproducible filesystem image creation" + +# Run three iterations +hash1=$(_run_iteration 1) +[ -z "$hash1" ] && _fail "Iteration 1 failed" + +hash2=$(_run_iteration 2) +[ -z "$hash2" ] && _fail "Iteration 2 failed" + +hash3=$(_run_iteration 3) +[ -z "$hash3" ] && _fail "Iteration 3 failed" + +# Verify all hashes match +if [ "$hash1" = "$hash2" ] && [ "$hash2" = "$hash3" ]; then + echo "All filesystem images are identical." +else + echo "ERROR: Filesystem images differ!" +fi + +# success, all done +status=0 +exittests/xfs/841.out new file +3
-0@@ -0,0 +1,3 @@ +QA output created by 841 +Test: XFS reproducible filesystem image creation +All filesystem images are identical.Signature · 1 line
2.51.0
Quoted 5 lines · Luca Di Maio
With the addition of the
-ppopulate option,SOURCE_DATE_EPOCHandDETERMINISTIC_SEEDsupport, it is possible to create fully reproducible pre-populated filesystems. We should test them here.
Quoted 16 lines
v1 -> v2:
- Changed test group from parent to mkfs
- FixedPROTO_DIRto point to a new dir
- PopulatePROTO_DIRwith relevant file types
- Move from md5sum to sha256sum
v2 -> v3
- Properly check if mkfs.xfs supportsSOURCE_DATE_EPOCHandDETERMINISTIC_SEED
- use fsstress program to generate thePROTO_DIRcontent
- simplify test output
v3 -> v4
- Add_cleanupfunction
v4 -> v5
- copy_cleanupfrom common/preamble
v5 -> v6
- remove debug typo/leftover
I think it's not good to be commit log, better to move this part ->
Quoted 3 lines
Signed-off-by Luca Di Maio <[email protected]>---
-> here
Quoted 80 lines
tests/xfs/841 | 173 ++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/841.out | 3 + 2 files changed, 176 insertions(+) create mode 100755 tests/xfs/841 create mode 100644 tests/xfs/841.outtests/xfs/841 new file +68
-0@@ -0,0 +1,173 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Chainguard, Inc. All Rights Reserved. +# +# FS QA Test No. 841 +# +# Test that XFS filesystems created with reproducibility options produce +# identical images across multiple runs. This verifies that the combination +# of SOURCE_DATE_EPOCH, DETERMINISTIC_SEED, and -m uuid= options result in +# bit-for-bit reproducible filesystem images. + +. ./common/preamble +_begin_fstest auto quick mkfs + +# Image file settings +IMG_SIZE="512M" +IMG_FILE="$TEST_DIR/xfs_reproducible_test.img" +PROTO_DIR="$TEST_DIR/proto" + +# Fixed values for reproducibility +FIXED_UUID="12345678-1234-1234-1234-123456789abc" +FIXED_EPOCH="1234567890" + +_cleanup() { + cd / + command -v _kill_fsstress &>/dev/null && _kill_fsstress + rm -r -f $tmp.* "$PROTO_DIR" "$IMG_FILE" +} + +# Check if mkfs.xfs supports required options +_check_mkfs_xfs_options() +{ + local check_img="$TEST_DIR/mkfs_check.img" + truncate -s 64M "$check_img" || return 1 + + # Check -m uuid support + $MKFS_XFS_PROG -m uuid=00000000-0000-0000-0000-000000000000 \ + -N "$check_img" &> /dev/null + local uuid_support=$? + + # Check -p support (protofile/directory population) + $MKFS_XFS_PROG 2>&1 | grep populate &> /dev/null + local proto_support=$? + + grep -q SOURCE_DATE_EPOCH "$MKFS_XFS_PROG" + local reproducible_support=$? + + rm -f "$check_img" + + if [ $uuid_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -m uuid= option" + fi + if [ $proto_support -ne 0 ]; then + _notrun "mkfs.xfs does not support -p option for directory population" + fi + if [ $reproducible_support -ne 0 ]; then + _notrun "mkfs.xfs does not support env options for reproducibility" + fi +} + +# Create a prototype directory with all file types supported by mkfs.xfs -p +_create_proto_dir() +{ + rm -rf "$PROTO_DIR" + mkdir -p "$PROTO_DIR" + + $FSSTRESS_PROG -d $PROTO_DIR -s 1 -n 2000 -p 2 -z \ + -f creat=15 \
tests/xfs/841 line 78
Quoted 10 lines
tests/xfs/841 new file +10
-0+ -f mkdir=8 \ + -f write=15 \ + -f truncate=5 \ + -f symlink=8 \ + -f link=8 \ + -f setfattr=12 \ + -f chown=3 \ + -f rename=5 \ + -f unlink=2 \ + -f rmdir=1
To match the _kill_fsstress, when I merge this patch I'll change this part to:
FSSTRESS_ARGS=`_scale_fsstress_args -d $PROTO_DIR -s 1 -n 2000 -p 2 -z
-f creat=15 \
-f mkdir=8 \
-f write=15 \
-f truncate=5 \
-f symlink=8 \
-f link=8 \
-f setfattr=12 \
-f chown=3 \
-f rename=5 \
-f unlink=2 \
-f rmdir=1`
_run_fsstress $FSSTRESS_ARGSThanks,
Zorro
Quoted 107 lines
tests/xfs/841 new file +95
-0+ + + # FIFO (named pipe) + mkfifo "$PROTO_DIR/fifo" + + # Unix socket + $here/src/af_unix "$PROTO_DIR/socket" 2> /dev/null || true + + # Block device (requires root) + mknod "$PROTO_DIR/blockdev" b 1 0 2> /dev/null || true + + # Character device (requires root) + mknod "$PROTO_DIR/chardev" c 1 3 2> /dev/null || true +} + +_require_test +_check_mkfs_xfs_options + +# Create XFS filesystem with full reproducibility options +# Uses -p to populate from directory during mkfs (no mount needed) +_mkfs_xfs_reproducible() +{ + local img=$1 + + # Create fresh image file + rm -f "$img" + truncate -s $IMG_SIZE "$img" || return 1 + + # Set environment variables for reproducibility: + # - SOURCE_DATE_EPOCH: fixes all inode timestamps to this value + # - DETERMINISTIC_SEED: uses fixed seed (0x53454544) instead of + # getrandom() + # + # mkfs.xfs options: + # - -m uuid=: fixed filesystem UUID + # - -p dir: populate filesystem from directory during creation + SOURCE_DATE_EPOCH=$FIXED_EPOCH \ + DETERMINISTIC_SEED=1 \ + $MKFS_XFS_PROG \ + -f \ + -m uuid=$FIXED_UUID \ + -p "$PROTO_DIR" \ + "$img" >> $seqres.full 2>&1 + + return $? +} + +# Compute hash of the image file +_hash_image() +{ + sha256sum "$1" | awk '{print $1}' +} + +# Run a single reproducibility test iteration +_run_iteration() +{ + local iteration=$1 + + echo "Iteration $iteration: Creating filesystem with -p $PROTO_DIR" >> $seqres.full + if ! _mkfs_xfs_reproducible "$IMG_FILE"; then + echo "mkfs.xfs failed" >> $seqres.full + return 1 + fi + + local hash=$(_hash_image "$IMG_FILE") + echo "Iteration $iteration: Hash = $hash" >> $seqres.full + + echo $hash +} + +# Create the prototype directory with various file types +_create_proto_dir + +echo "Test: XFS reproducible filesystem image creation" + +# Run three iterations +hash1=$(_run_iteration 1) +[ -z "$hash1" ] && _fail "Iteration 1 failed" + +hash2=$(_run_iteration 2) +[ -z "$hash2" ] && _fail "Iteration 2 failed" + +hash3=$(_run_iteration 3) +[ -z "$hash3" ] && _fail "Iteration 3 failed" + +# Verify all hashes match +if [ "$hash1" = "$hash2" ] && [ "$hash2" = "$hash3" ]; then + echo "All filesystem images are identical." +else + echo "ERROR: Filesystem images differ!" +fi + +# success, all done +status=0 +exittests/xfs/841.out new file +3
-0@@ -0,0 +1,3 @@ +QA output created by 841 +Test: XFS reproducible filesystem image creation +All filesystem images are identical.Signature · 1 line
2.51.0
Quoted 9 lines · Zorro Lang
On Thu, Jan 08, 2026 at 03:22:22PM +0100, Luca Di Maio wrote:
I think it's not good to be commit log, better to move this part ->Quoted 3 lines
Signed-off-by Luca Di Maio <[email protected]>---
-> here
Thanks Zorro, sent a v7 with the correction
L.
Quoted 5 lines · Luca Di Maio
Quoted 4 lines · Zorro Lang
On Thu, Jan 08, 2026 at 03:22:22PM +0100, Luca Di Maio wrote:
I think it's not good to be commit log, better to move this part ->Quoted 1 line
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Signed-off-by Luca Di Maio <[email protected]>---
-> here
Thanks Zorro, sent a v7 with the correction
I've merged v6 into "patches-in-queue" branch with a bit changes, will merge into for-next branch after testing, feel free to check it:
https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git?h=patches-in-queue
Quoted 3 lines
L.