btrfs: exercise NOCOMPRESS range and EOF handling
PATCH 1/3 btrfs: exercise NOCOMPRESS range and EOF handling
In reply to a message not in the archiveCheck that a bounded --nocomp request rewrites compressed extents in the selected range while distant untouched regions still point to their original compressed storage.
Cover a partial final sector, zero length, a length extending past EOF, start at EOF, and the empty-file no-op. Preserve contents and sizes across the conversion and a remount.
Add shared FIEMAP and integrity helpers for the NOCOMPRESS tests. Require full mapping coverage and inspect every overlapping extent so holes and partly converted ranges cannot pass. Include a small ioctl helper to probe kernel support and check errno without btrfs-progs translation.
--- .gitignore | 1 + common/btrfs_nocomp | 134 ++++++++++++++++++++++++++++++++++++++ src/Makefile | 2 +- src/btrfs_defrag_nocomp.c | 63 ++++++++++++++++++ tests/btrfs/355 | 80 +++++++++++++++++++++++ tests/btrfs/355.out | 2 + 6 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 common/btrfs_nocomp create mode 100644 src/btrfs_defrag_nocomp.c create mode 100755 tests/btrfs/355 create mode 100644 tests/btrfs/355.out
.gitignore +1 -0
@@ -216,6 +216,7 @@ tags /src/truncate /src/t_btrfs_received_uuid_ioctl /src/btrfs_ioctl +/src/btrfs_defrag_nocomp # Symlinked files /tests/generic/035.out
common/btrfs_nocomp new file +134 -0
134 lines not on this page · raw message
src/Makefile +1 -1
@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint fs-monitor btrfs_ioctl + rw_hint fs-monitor btrfs_ioctl btrfs_defrag_nocomp EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \
src/btrfs_defrag_nocomp.c new file +63 -0
63 lines not on this page · raw message
tests/btrfs/355 new file +80 -0
80 lines not on this page · raw message
tests/btrfs/355.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 355 +Silence is golden
Signature · 1 line
2.43.0
PATCH 2/3 btrfs: check NOCOMPRESS preserves holes and preallocation
In reply to a message not in the archivePlace compressed data islands around holes and preallocated extents, including a KEEP_SIZE reservation beyond EOF. Defragment the file with --nocomp and require every data island to become uncompressed.
Check that each hole stays unmapped and both reservations retain their complete physical mappings and UNWRITTEN state. Verify the layout, contents, and file size again after remounting with compression enabled. This extends btrfs/257 to NOCOMPRESS and compressed sparse files.
--- tests/btrfs/356 | 71 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/356.out | 2 ++ 2 files changed, 73 insertions(+) create mode 100755 tests/btrfs/356 create mode 100644 tests/btrfs/356.out
tests/btrfs/356 new file +71 -0
@@ -0,0 +1,71 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# +# FS QA Test 356 +# +# Verify NOCOMPRESS skips holes and preallocated extents. +# +. ./common/preamble +_begin_fstest auto quick defrag compress prealloc fiemap + +. ./common/filter +. ./common/btrfs_nocomp + +_nc_require +_require_xfs_io_command falloc -k +_scratch_mkfs -s 4k >> "$seqres.full" 2>&1 || _fail "mkfs failed" +_scratch_mount -o compress=zlib +_nc_probe + +file="$SCRATCH_MNT/islands" +_nc_write "$file" 0 131072 0x31 +_nc_write "$file" 524288 131072 0x32 +_nc_write "$file" 1048576 131072 0x33 +for offset in 0 524288 1048576; do + _nc_check_range "$file" "$offset" 131072 compressed +done +$XFS_IO_PROG -c 'falloc 256K 128K' -c 'falloc -k 1280K 128K' -c fsync \ + "$file" >> "$seqres.full" 2>&1 || _fail "fallocate failed" +[ "$(stat -c %s "$file")" = 1179648 ] || _fail "fallocate extended EOF" +_nc_save_file "$file" + +check_layout() +{ + local offset + + for offset in 262144 1310720; do + _nc_check_range "$file" "$offset" 131072 prealloc + _nc_mapping "$file" "$offset" 131072 > "$tmp.prealloc-$offset-now" + cmp -s "$tmp.prealloc-$offset" "$tmp.prealloc-$offset-now" || + _fail "prealloc mapping changed at $offset" + done + _nc_check_range "$file" 131072 131072 hole + _nc_check_range "$file" 393216 131072 hole + _nc_check_range "$file" 655360 393216 hole + _nc_check_range "$file" 1179648 131072 hole +} + +# Include the KEEP_SIZE reservation beyond EOF in the baseline. FIEMAP +# reports preallocation there even though there are no bytes to read. +for offset in 262144 1310720; do + _nc_check_range "$file" "$offset" 131072 prealloc + _nc_mapping "$file" "$offset" 131072 > "$tmp.prealloc-$offset" +done +check_layout +_nc_defrag "$file" +for offset in 0 524288 1048576; do + _nc_check_range "$file" "$offset" 131072 regular +done +check_layout +_nc_verify_files + +_scratch_cycle_mount compress=zlib +_nc_verify_files +check_layout +for offset in 0 524288 1048576; do + _nc_check_range "$file" "$offset" 131072 regular +done + +echo "Silence is golden" +_exit 0
tests/btrfs/356.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 356 +Silence is golden
Signature · 1 line
2.43.0
PATCH 3/3 btrfs: cover NOCOMPRESS inline eligibility and short tails
In reply to a message not in the archiveCheck that --nocomp preserves a compressed inline extent while it remains eligible under max_inline, including after a remount. Lower max_inline to zero and require the same extent to become regular, uncompressed data.
Exercise one-byte, sector-minus-one, sector-plus-one, and maximum-compressed-extent-plus-one file sizes. Check every mapped byte range for regular uncompressed data after defrag, and verify all file contents, sizes, and converted mappings again after a remount.
Use FIEMAP INLINE and ENCODED flags instead of parsing dump-tree output, and skip configurations without inline extent creation support.
--- tests/btrfs/357 | 59 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/357.out | 2 ++ 2 files changed, 61 insertions(+) create mode 100755 tests/btrfs/357 create mode 100644 tests/btrfs/357.out
tests/btrfs/357 new file +59 -0
@@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# +# FS QA Test 357 +# +# Verify NOCOMPRESS inline eligibility and short EOF tails. +# +. ./common/preamble +_begin_fstest auto quick defrag compress fiemap + +. ./common/filter +. ./common/btrfs_nocomp + +_nc_require +_require_btrfs_inline_extents_creation +_scratch_mkfs -s 4k >> "$seqres.full" 2>&1 || _fail "mkfs failed" +_scratch_mount -o compress=zlib,max_inline=4096 +_nc_probe + +file="$SCRATCH_MNT/inline" +_nc_write "$file" 0 2048 0x34 +_nc_save_file "$file" +_nc_check_range "$file" 0 2048 inline +_nc_defrag "$file" +_nc_check_range "$file" 0 2048 inline +_nc_verify_files +_scratch_cycle_mount compress=zlib,max_inline=4096 +_nc_check_range "$file" 0 2048 inline +_nc_verify_files + +# Lowering max_inline makes the same extent eligible for conversion. +_scratch_cycle_mount compress=zlib,max_inline=0 +_nc_defrag "$file" +_nc_check_range "$file" 0 2048 regular +_nc_verify_files + +for length in 1 4095 4097 131073; do + f="$SCRATCH_MNT/tail-$length" + _nc_write "$f" 0 "$length" 0x36 + _nc_save_file "$f" + # One-sector files cannot save a sector through compression. The larger + # fixtures must contain compressed data before testing conversion. + if [ "$length" -gt 4096 ]; then + _nc_check_range "$f" 0 4096 compressed + fi + _nc_defrag "$f" + _nc_check_range "$f" 0 "$length" regular +done +_nc_verify_files +_scratch_cycle_mount compress=zlib,max_inline=0 +_nc_verify_files +_nc_check_range "$file" 0 2048 regular +for length in 1 4095 4097 131073; do + _nc_check_range "$SCRATCH_MNT/tail-$length" 0 "$length" regular +done + +echo "Silence is golden" +_exit 0
tests/btrfs/357.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 357 +Silence is golden
Signature · 1 line
2.43.0
Quoted 44 lines · Dhruva G
Check that a bounded --nocomp request rewrites compressed extents in the selected range while distant untouched regions still point to their original compressed storage.
Cover a partial final sector, zero length, a length extending past EOF, start at EOF, and the empty-file no-op. Preserve contents and sizes across the conversion and a remount.Add shared FIEMAP and integrity helpers for the NOCOMPRESS tests. Require full mapping coverage and inspect every overlapping extent so holes and partly converted ranges cannot pass. Include a small ioctl helper to probe kernel support and check errno without btrfs-progs translation.
Signed-off-by Dhruva Gole <[email protected]>--- .gitignore | 1 + common/btrfs_nocomp | 134 ++++++++++++++++++++++++++++++++++++++ src/Makefile | 2 +- src/btrfs_defrag_nocomp.c | 63 ++++++++++++++++++ tests/btrfs/355 | 80 +++++++++++++++++++++++ tests/btrfs/355.out | 2 + 6 files changed, 281 insertions(+), 1 deletion(-)create mode 100644 common/btrfs_nocomp
create mode 100644src/btrfs_defrag_nocomp.c
create mode 100755 tests/btrfs/355
create mode 100644 tests/btrfs/355.out.gitignore +1
-0@@ -216,6 +216,7 @@ tags /src/truncate /src/t_btrfs_received_uuid_ioctl /src/btrfs_ioctl +/src/btrfs_defrag_nocomp # Symlinked files /tests/generic/035.outcommon/btrfs_nocomp new file +0
-0
common/btrfs_nocomp line 3
Quoted 3 lines
common/btrfs_nocomp new file +3
-0@@ -0,0 +1,134 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# Helpers for BTRFS_DEFRAG_RANGE_NOCOMPRESS tests.
I don't like a dedicated file just do all those.
Why not put them into common/btrfs?
common/btrfs_nocomp line 5
Quoted 2 lines
common/btrfs_nocomp new file +2
-0+ +_nc_require()
And "nc" is too generic, _require_btrfs_defrag_nocompress() would make more sense.
All the namings here look just some temporary scaffolding, not for generic purposes.
Quoted 16 lines
common/btrfs_nocomp new file +16
-0+{ + _require_scratch + _require_test_program btrfs_defrag_nocomp + _require_xfs_io_command fiemap ranged + _require_btrfs_command filesystem defragment --nocomp + _require_btrfs_no_nodatacow + _require_btrfs_no_nodatasum + _require_btrfs_no_autodefrag + _require_btrfs_support_sectorsize 4096 + _wants_kernel_commit 009b2056cb25 \ + "btrfs: defrag: add flag to force no-compression" +} + +_nc_write() +{ + $XFS_IO_PROG -f -c "pwrite -S $4 -b 128K $2 $3" -c fsync "$1" \
common/btrfs_nocomp line 31
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ >> "$seqres.full" 2>&1 || _fail "write failed: $1" +} + +_nc_probe() +{ + local result + + _nc_write "$SCRATCH_MNT/probe" 0 131072 0x5a + result=$($here/src/btrfs_defrag_nocomp "$SCRATCH_MNT/probe" 0 131072) || + _fail "NOCOMPRESS probe failed"
Why we need such dedicated probe program while the kernel is already rejecting unknown flags with ENOTTY?
And I think this probing should also be done inside the _require_btrfs_defrag_nocompress().
Quoted 19 lines
common/btrfs_nocomp new file +19
-0+ case "$result" in + OK) ;; + EOPNOTSUPP|ENOTTY) _notrun "kernel does not support NOCOMPRESS" ;; + *) _fail "unexpected NOCOMPRESS probe result: $result" ;; + esac + rm -f "$SCRATCH_MNT/probe" +} + +_nc_defrag() +{ + local file="${@: -1}" + + $BTRFS_UTIL_PROG filesystem defragment --nocomp "$@" \ + >> "$seqres.full" 2>&1 || _fail "NOCOMPRESS defrag failed" + $XFS_IO_PROG -c fsync "$file" >> "$seqres.full" 2>&1 || + _fail "fsync after defrag failed" +} + +_nc_map()
common/btrfs_nocomp line 60
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+{ + $XFS_IO_PROG -c "fiemap -v $2 $3" "$1" > "$tmp.nc-map" \ + 2>> "$seqres.full" || _fail "fiemap failed: $1" + cat "$tmp.nc-map" >> "$seqres.full" +} + +# FIEMAP prints logical offsets in 512-byte units. Check every overlapping +# extent and complete coverage, so a hole or a partly converted range fails. +# The caller supplies byte offsets and lengths, including a partial EOF sector. +_nc_check_range()
This looks a little overkilled to me.
For regular hole/compression flags, what about filtering out the non-hole logical bytenr (e.g. replace them with 'XXX' or 'regular'), then output to the stdout, and use golden output to verify?
Quoted 45 lines · Dhruva G
Check that a bounded --nocomp request rewrites compressed extents in the selected range while distant untouched regions still point to their original compressed storage.
Cover a partial final sector, zero length, a length extending past EOF, start at EOF, and the empty-file no-op. Preserve contents and sizes across the conversion and a remount.Add shared FIEMAP and integrity helpers for the NOCOMPRESS tests. Require full mapping coverage and inspect every overlapping extent so holes and partly converted ranges cannot pass. Include a small ioctl helper to probe kernel support and check errno without btrfs-progs translation.
Signed-off-by Dhruva Gole <[email protected]>--- .gitignore | 1 + common/btrfs_nocomp | 134 ++++++++++++++++++++++++++++++++++++++ src/Makefile | 2 +- src/btrfs_defrag_nocomp.c | 63 ++++++++++++++++++ tests/btrfs/355 | 80 +++++++++++++++++++++++ tests/btrfs/355.out | 2 + 6 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 common/btrfs_nocomp create mode 100644 src/btrfs_defrag_nocomp.c create mode 100755 tests/btrfs/355 create mode 100644 tests/btrfs/355.out.gitignore +1
-0@@ -216,6 +216,7 @@ tags /src/truncate /src/t_btrfs_received_uuid_ioctl /src/btrfs_ioctl +/src/btrfs_defrag_nocomp # Symlinked files /tests/generic/035.outcommon/btrfs_nocomp new file +0
-0
common/btrfs_nocomp line 6
Quoted 6 lines
common/btrfs_nocomp new file +6
-0@@ -0,0 +1,134 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# Helpers for BTRFS_DEFRAG_RANGE_NOCOMPRESS tests. + +_nc_require() +{
Such a odd name because:
1. All our require functions are named as _require_something, but here the logic is inverted.
2. "nc" is not really intuitive, is rather cryptic, specially taking into account this is specific for the no compress option for defrag - there's no indication in the name this is defrag related.
Quoted 7 lines
common/btrfs_nocomp new file +7
-0+ _require_scratch + _require_test_program btrfs_defrag_nocomp + _require_xfs_io_command fiemap ranged + _require_btrfs_command filesystem defragment --nocomp + _require_btrfs_no_nodatacow + _require_btrfs_no_nodatasum + _require_btrfs_no_autodefrag
common/btrfs_nocomp line 23
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ _require_btrfs_support_sectorsize 4096 + _wants_kernel_commit 009b2056cb25 \ + "btrfs: defrag: add flag to force no-compression" +} + +_nc_write() +{ + $XFS_IO_PROG -f -c "pwrite -S $4 -b 128K $2 $3" -c fsync "$1" \ + >> "$seqres.full" 2>&1 || _fail "write failed: $1" +}
A useless wrapper. Just make the tests call XFS_IO_PROG, this does not add any value at all.
Quoted 5 lines
common/btrfs_nocomp new file +5
-0+ +_nc_probe() +{ + local result +
common/btrfs_nocomp line 38
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ _nc_write "$SCRATCH_MNT/probe" 0 131072 0x5a + result=$($here/src/btrfs_defrag_nocomp "$SCRATCH_MNT/probe" 0 131072) || + _fail "NOCOMPRESS probe failed" + case "$result" in + OK) ;; + EOPNOTSUPP|ENOTTY) _notrun "kernel does not support NOCOMPRESS" ;; + *) _fail "unexpected NOCOMPRESS probe result: $result" ;; + esac + rm -f "$SCRATCH_MNT/probe" +}
Too complex, we usually just call the user space utility on a simple scenario that should never fail and if it fails we skip the test saying the feature is not supported by the kernel.
And we do it in a _require_something function as well.
common/btrfs_nocomp line 48
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ +_nc_defrag() +{ + local file="${@: -1}" + + $BTRFS_UTIL_PROG filesystem defragment --nocomp "$@" \ + >> "$seqres.full" 2>&1 || _fail "NOCOMPRESS defrag failed" + $XFS_IO_PROG -c fsync "$file" >> "$seqres.full" 2>&1 || + _fail "fsync after defrag failed" +}
Another pointless wrapper.
I also dislike this practice of adding a _fail for everything. Just let the tests fail with a golden output mismatch.
common/btrfs_nocomp line 55
Quoted 7 lines
common/btrfs_nocomp new file +7
-0+ +_nc_map() +{ + $XFS_IO_PROG -c "fiemap -v $2 $3" "$1" > "$tmp.nc-map" \ + 2>> "$seqres.full" || _fail "fiemap failed: $1" + cat "$tmp.nc-map" >> "$seqres.full" +}
Yet another wrapper.
Quoted 34 lines
common/btrfs_nocomp new file +34
-0+ +# FIEMAP prints logical offsets in 512-byte units. Check every overlapping +# extent and complete coverage, so a hole or a partly converted range fails. +# The caller supplies byte offsets and lengths, including a partial EOF sector. +_nc_check_range() +{ + _nc_map "$1" "$2" "$3" + $AWK_PROG -v start="$2" -v len="$3" -v mode="$4" ' + BEGIN { + first = int(start / 512) + end = int((start + len + 511) / 512) + next_block = first + } + $1 ~ /^[0-9]+:$/ && $5 ~ /^0x/ { + gsub(/[^0-9]/, " ", $2) + split($2, logical) + lo = logical[1] + 0 + hi = logical[2] + 1 + if (hi <= first || lo >= end) + next + if (mode == "hole") { + bad = 1 + next + } + flags = strtonum($5) + encoded = and(flags, 0x8) != 0 + unwritten = and(flags, 0x800) != 0 + inline = and(flags, 0x200) != 0 + if (and(flags, 0x6) || lo > next_block) + bad = 1 + if (mode == "compressed" && (!encoded || unwritten || inline)) + bad = 1 + if (mode == "regular" && (encoded || unwritten || inline)) + bad = 1
common/btrfs_nocomp line 99
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ if (mode == "inline" && (!encoded || unwritten || !inline)) + bad = 1 + if (mode == "prealloc" && (encoded || !unwritten || inline)) + bad = 1 + if (hi > next_block) + next_block = hi + } + END { exit bad || (mode != "hole" && next_block < end) } + ' "$tmp.nc-map" || _fail "unexpected $4 mapping: $1 ($2, $3)" +}
Why do we need this level of complexity?
Adding an awk script to process fiemap output and detect unexpected results is overly complex.
Why not do what we do in every other test that checks the extent layout of a file?
Simply call fiemap in the test and add the expected output to the golden output file.
That's simple, does the work and it's the preferred way to do things in fstests.
Quoted 4 lines
common/btrfs_nocomp new file +4
-0+ +# Ignore flags such as LAST, which may change when another range is rewritten. +_nc_mapping() +{
common/btrfs_nocomp line 113
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ _nc_map "$1" "$2" "$3" + _filter_xfs_io_fiemap < "$tmp.nc-map" | \ + $AWK_PROG '{ print $1, $2, $3, $4 }' +} + +_nc_save_file() +{ + md5sum "$1" >> "$tmp.nc-checksums" || _fail "checksum failed: $1" + stat -c '%s %n' "$1" >> "$tmp.nc-sizes" || _fail "stat failed: $1" +}
This name "save_file" is totally confusing.
Quoted 11 lines
common/btrfs_nocomp new file +11
-0+ +_nc_verify_files() +{ + local size file actual + + md5sum -c "$tmp.nc-checksums" >> "$seqres.full" 2>&1 || + _fail "file contents changed" + while read -r size file; do + actual=$(stat -c %s "$file") || _fail "stat failed: $file" + [ "$actual" = "$size" ] || _fail "file size changed: $file" + done < "$tmp.nc-sizes"
common/btrfs_nocomp line 134
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+} + +_nc_raw_expect() +{ + local expected="$1" result + shift + result=$($here/src/btrfs_defrag_nocomp "$@") || _fail "raw ioctl failed" + [ "$result" = "$expected" ] || + _fail "expected $expected, got $result" +}
This is all stuff we can do in the tests by calling md5sum and placing the expected digest in the golden output file, like we do in many other tests.
Quoted 19 lines
src/Makefile +1
-1@@ -36,7 +36,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail \ detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe \ uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault min_dio_alignment \ - rw_hint fs-monitor btrfs_ioctl + rw_hint fs-monitor btrfs_ioctl btrfs_defrag_nocomp EXTRA_EXECS = dmerror fill2attr fill2fs fill2fs_check scaleread.sh \ btrfs_crc32c_forged_name.py popdir.pl popattr.py \src/btrfs_defrag_nocomp.c new file +0
-0
src/btrfs_defrag_nocomp.c line 3
Quoted 3 lines
src/btrfs_defrag_nocomp.c new file +3
-0@@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) Meta Platforms, Inc. and affiliates. +/* Issue NOCOMPRESS requests without userspace range/error translation. */
Why do we need a C program?
Why not just have the tests call btrfs-progs: btrfs filesystem defragment --nocomp ....
This patchset adds an insane level of complexity for no good reason, making the tests hard to read and maintain.
I can't see why just calling btrfs-progs, fiemap and md5sum directly in the tests and then placing the expected output in the golden file is not enough...
Thanks.
Quoted 157 lines
src/btrfs_defrag_nocomp.c new file +60
-0+#include <errno.h> +#include <fcntl.h> +#include <linux/btrfs.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <unistd.h> + +#ifndef BTRFS_DEFRAG_RANGE_NOCOMPRESS +#define BTRFS_DEFRAG_RANGE_NOCOMPRESS (1ULL << 3) +#endif + +static unsigned long long number (const char *s) +{ +char *end; +unsigned long long value; + + errno =0 ; + value =strtoull (s, &end,0 ); +if (errno || *end || end == s || *s =='-' ) { +fprintf (stderr,"invalid number: %s\n" , s); +exit (2 ); + } +return value; +} + +int main (int argc,char **argv) +{ +struct btrfs_ioctl_defrag_range_args range = {0 }; +int fd, err =0 ; +const char *result; + +if (argc !=4 ) { +fprintf (stderr,"usage: %s file start len\n" , argv[0 ]); +return 2 ; + } + range.flags =BTRFS_DEFRAG_RANGE_NOCOMPRESS ; + range.start =number (argv[2 ]); + range.len =number (argv[3 ]); + fd =open (argv[1 ],O_RDONLY |O_CLOEXEC ); +if (fd <0 ) { +perror ("open" ); +return 2 ; + } +if (ioctl (fd,BTRFS_IOC_DEFRAG_RANGE , &range) <0 ) + err = errno; +close (fd); +switch (err) { +case 0 : result ="OK" ;break ; +case EINVAL : result ="EINVAL" ;break ; +case EOPNOTSUPP : result ="EOPNOTSUPP" ;break ; +case ENOTTY : result ="ENOTTY" ;break ; +default : +fprintf (stderr,"unexpected errno %d (%s)\n" , err,strerror (err)); +return 1 ; + } +puts (result); +return 0 ; +}tests/btrfs/355 new file +80
-0@@ -0,0 +1,80 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# +# FS QA Test 355 +# +# Verify bounded NOCOMPRESS conversion and EOF range handling. +# +. ./common/preamble +_begin_fstest auto quick defrag compress fiemap + +. ./common/filter +. ./common/btrfs_nocomp + +_nc_require +_scratch_mkfs -s 4k >> "$seqres.full" 2>&1 || _fail "mkfs failed" +_scratch_mount -o compress=zlib +_nc_probe + +file="$SCRATCH_MNT/range" +_nc_write "$file" 0 4194304 0x5a +_nc_save_file "$file" +_nc_check_range "$file" 0 4194304 compressed +_nc_mapping "$file" 0 131072 > "$tmp.prefix" +_nc_mapping "$file" 4063232 131072 > "$tmp.suffix" + +_nc_defrag -s 1M -l 512K "$file" +_nc_check_range "$file" 1048576 524288 regular +_nc_check_range "$file" 0 1048576 compressed +_nc_check_range "$file" 1572864 2621440 compressed +_nc_mapping "$file" 0 131072 > "$tmp.prefix-after" +_nc_mapping "$file" 4063232 131072 > "$tmp.suffix-after" +cmp -s "$tmp.prefix" "$tmp.prefix-after" || _fail "prefix mapping changed" +cmp -s "$tmp.suffix" "$tmp.suffix-after" || _fail "suffix mapping changed" + +# The two-sector tail can be compressed, unlike a single-sector tail on a +# 4K filesystem. First convert a one-byte range, then use an unaligned start +# and end the request at EOF, one byte into the final 4K sector. +tail="$SCRATCH_MNT/partial" +_nc_write "$tail" 0 135169 0x39 +_nc_save_file "$tail" +_nc_check_range "$tail" 131072 4097 compressed +_nc_defrag -s 128K -l 1 "$tail" +_nc_check_range "$tail" 131072 4096 regular +_nc_defrag -s 131073 -l 4096 "$tail" +_nc_check_range "$tail" 131072 4097 regular + +# Zero length means to EOF; a length extending past EOF must be clamped. +for mode in zero huge; do + f="$SCRATCH_MNT/$mode" + _nc_write "$f" 0 262144 0x38 + _nc_save_file "$f" + _nc_check_range "$f" 0 262144 compressed + case "$mode" in + zero) _nc_defrag -s 128K -l 0 "$f" ;; + huge) _nc_defrag -s 128K -l 16M "$f" ;; + esac + _nc_check_range "$f" 0 131072 compressed + _nc_check_range "$f" 131072 131072 regular +done + +# A nonempty file rejects start == EOF, but an empty file is a no-op. +_nc_raw_expect EINVAL "$file" 4194304 4096 +touch "$SCRATCH_MNT/empty" || _fail "create empty file failed" +_nc_save_file "$SCRATCH_MNT/empty" +_nc_raw_expect OK "$SCRATCH_MNT/empty" 0 0 +_nc_verify_files +_scratch_cycle_mount compress=zlib +_nc_verify_files +_nc_check_range "$file" 1048576 524288 regular +_nc_check_range "$file" 0 1048576 compressed +_nc_check_range "$file" 1572864 2621440 compressed +_nc_check_range "$tail" 131072 4097 regular +for mode in zero huge; do + _nc_check_range "$SCRATCH_MNT/$mode" 0 131072 compressed + _nc_check_range "$SCRATCH_MNT/$mode" 131072 131072 regular +done + +echo "Silence is golden" +_exit 0tests/btrfs/355.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 355 +Silence is golden --2.43.0
Quoted 54 lines · Qu Wenruo
Quoted 44 lines · Dhruva G
Check that a bounded --nocomp request rewrites compressed extents in the selected range while distant untouched regions still point to their original compressed storage.
Cover a partial final sector, zero length, a length extending past EOF, start at EOF, and the empty-file no-op. Preserve contents and sizes across the conversion and a remount.Add shared FIEMAP and integrity helpers for the NOCOMPRESS tests. Require full mapping coverage and inspect every overlapping extent so holes and partly converted ranges cannot pass. Include a small ioctl helper to probe kernel support and check errno without btrfs-progs translation.
Signed-off-by Dhruva Gole <[email protected]>--- .gitignore | 1 + common/btrfs_nocomp | 134 ++++++++++++++++++++++++++++++++++++++ src/Makefile | 2 +- src/btrfs_defrag_nocomp.c | 63 ++++++++++++++++++ tests/btrfs/355 | 80 +++++++++++++++++++++++ tests/btrfs/355.out | 2 + 6 files changed, 281 insertions(+), 1 deletion(-)create mode 100644 common/btrfs_nocomp
create mode 100644src/btrfs_defrag_nocomp.c
create mode 100755 tests/btrfs/355
create mode 100644 tests/btrfs/355.out.gitignore +1
-0@@ -216,6 +216,7 @@ tags /src/truncate /src/t_btrfs_received_uuid_ioctl /src/btrfs_ioctl +/src/btrfs_defrag_nocomp # Symlinked files /tests/generic/035.outcommon/btrfs_nocomp new file +0
-0common/btrfs_nocomp line 3
Quoted 3 lines
common/btrfs_nocomp new file +3
-0@@ -0,0 +1,134 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) Meta Platforms, Inc. and affiliates. +# Helpers for BTRFS_DEFRAG_RANGE_NOCOMPRESS tests.I don't like a dedicated file just do all those.
Why not put them into common/btrfs?
Quoted 8 lines
common/btrfs_nocomp line 5
Quoted 2 lines
common/btrfs_nocomp new file +2
-0+ +_nc_require()And "nc" is too generic,
_require_btrfs_defrag_nocompress()would make more sense.All the namings here look just some temporary scaffolding, not for generic purposes.
Agreed. I'll move the shared helpers into common/btrfs and use more descriptive names in v2.
Quoted 31 lines
Quoted 17 lines
common/btrfs_nocomp new file +16
-0+{ + _require_scratch + _require_test_program btrfs_defrag_nocomp + _require_xfs_io_command fiemap ranged + _require_btrfs_command filesystem defragment --nocomp + _require_btrfs_no_nodatacow + _require_btrfs_no_nodatasum + _require_btrfs_no_autodefrag + _require_btrfs_support_sectorsize 4096 + _wants_kernel_commit 009b2056cb25 \ + "btrfs: defrag: add flag to force no-compression" +} + +_nc_write() +{ + $XFS_IO_PROG -f -c "pwrite -S $4 -b 128K $2 $3" -c fsync "$1" \common/btrfs_nocomp line 31
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+ >> "$seqres.full" 2>&1 || _fail "write failed: $1" +} + +_nc_probe() +{ + local result + + _nc_write "$SCRATCH_MNT/probe" 0 131072 0x5a + result=$($here/src/btrfs_defrag_nocomp "$SCRATCH_MNT/probe" 0 131072) || + _fail "NOCOMPRESS probe failed"Why we need such dedicated probe program while the kernel is already rejecting unknown flags with ENOTTY?
Quoted 2 lines
And I think this probing should also be done inside the
_require_btrfs_defrag_nocompress().
Agreed. I'll do this in v2.
Quoted 32 lines
Quoted 20 lines
common/btrfs_nocomp new file +19
-0+ case "$result" in + OK) ;; + EOPNOTSUPP|ENOTTY) _notrun "kernel does not support NOCOMPRESS" ;; + *) _fail "unexpected NOCOMPRESS probe result: $result" ;; + esac + rm -f "$SCRATCH_MNT/probe" +} + +_nc_defrag() +{ + local file="${@: -1}" + + $BTRFS_UTIL_PROG filesystem defragment --nocomp "$@" \ + >> "$seqres.full" 2>&1 || _fail "NOCOMPRESS defrag failed" + $XFS_IO_PROG -c fsync "$file" >> "$seqres.full" 2>&1 || + _fail "fsync after defrag failed" +} + +_nc_map()common/btrfs_nocomp line 60
Quoted 10 lines
common/btrfs_nocomp new file +10
-0+{ + $XFS_IO_PROG -c "fiemap -v $2 $3" "$1" > "$tmp.nc-map" \ + 2>> "$seqres.full" || _fail "fiemap failed: $1" + cat "$tmp.nc-map" >> "$seqres.full" +} + +# FIEMAP prints logical offsets in 512-byte units. Check every overlapping +# extent and complete coverage, so a hole or a partly converted range fails. +# The caller supplies byte offsets and lengths, including a partial EOF sector. +_nc_check_range()This looks a little overkilled to me.
Quoted 3 lines
Quoted 1 line
For regular hole/compression flags, what about filtering out the
non-hole logical bytenr (e.g. replace them with 'XXX' or 'regular'), then output to the stdout, and use golden output to verify?
For patch 3, the test also needs to distinguish compressed inline extents from compressed non-inline extents. _filter_fiemap_flags 1 retains the encoded flag, but currently filters out the inline flag.
Would you prefer a small opt-in extension to retain the inline flag, or a test-local filter? With the extension, existing callers would retain their current output, and the inline test could use:_filter_fiemap_flags 1 1.
Thanks for reviewing and suggestions!
Regards,
Dhruva Gole
Quoted 1 line · Dhruva G
On 12-10-2026 03:30, Qu Wenruo wrote:
[...]
Quoted 10 lines
Quoted 4 lines
This looks a little overkilled to me.
Quoted 1 line
For regular hole/compression flags, what about filtering out the
non-hole logical bytenr (e.g. replace them with 'XXX' or 'regular'), then output to the stdout, and use golden output to verify?
For patch 3, the test also needs to distinguish compressed inline extents from compressed non-inline extents.
_filter_fiemap_flags1 retains the encoded flag, but currently filters out the inline flag.Would you prefer a small opt-in extension to retain the inline flag, or a test-local filter?
I think for now a local filter would be good enough if there is only one test case requiring this.
But in the long run, we want a filter that only filters out the bytenr. Maybe something like _filter_fiemap_keep_flags().
Quoted 2 lines
With the extension, existing callers would retain their current output, and the inline test could use:
_filter_fiemap_flags1 1.
More than one boolean parameters in a row is never a good idea to me.
Thus I'd prefer a dedicated filter.
Thanks,
Qu
Quoted 5 lines
Thanks for reviewing and suggestions!
Regards,
Dhruva Gole