common/populate: avoid test directory space exhaustion from metadump caching
PATCH 1/2 common/populate: avoid test directory space exhaustion from metadump caching
Previously, caching the populated scratch device metadump was done unconditionally. Saving these large metadumps can exhaust the test directory's available space, leading to subsequent test failures due to lack of space.
Introduce a space check in the caching path. By default, only save the populated metadump if the space occupied on the scratch device is less than 40% of the space available on the test device.
Also introduce the "FORCECACHE" environment variable. When the user sets FORCECACHE=1, bypass this space check and force caching the metadump.
Without patch:
./check -g auto ... Failed 46 of 1282 tests
With patch:
./check -g auto ... Failed 36 of 1282 tests FORCECACHE=1 ./check -g auto ... Failed 46 of 1282 tests
--- check | 6 ++++++ common/populate | 20 +++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-)
check +6 -0
@@ -44,6 +44,12 @@ export QA_CHECK_FS=${QA_CHECK_FS:=true} # number of diff lines from a failed test, 0 for whole output export DIFF_LENGTH=${DIFF_LENGTH:=10} +# Some tests use _scratch_populate_cached() style helpers to populate +# SCRATCH device once and cache the metadump in TEST device so subsequent +# calls can be faster. The caching is optional based on how much space +# TEST device has, however we can force the caching using FORCECACHE=1 +export FORCECACHE="${FORCECACHE:-}" + # by default don't output timestamps timestamp=${TIMESTAMP:=false} . common/exit
common/populate +19 -1
@@ -1177,6 +1177,24 @@ _scratch_populate_cached() { ;; esac - _scratch_populate_save_metadump "${POPULATE_METADUMP}" && \ + if [ "$FORCECACHE" = 1 ]; then + # FORCECACHE is explicitly 1 - always save the metadump + _scratch_populate_save_metadump "${POPULATE_METADUMP}" && \ echo "${meta_descr}" > "${populate_metadump_descr}" + else + # FORCECACHE is not 1 (or not provided) - only save if + # scratch device occupied data is less than 40% of the + # space available in the test device + test_avail=$(_get_available_space $TEST_DIR) + scratch_total=$(_get_total_space $SCRATCH_MNT) + scratch_avail=$(_get_available_space $SCRATCH_MNT) + scratch_occupied=$((scratch_total-scratch_avail)) + if [ "$scratch_occupied" -lt \ + "$((test_avail * 40 / 100))" ]; then + _scratch_populate_save_metadump \ + "${POPULATE_METADUMP}" && \ + echo "${meta_descr}" > \ + "${populate_metadump_descr}" + fi + fi }
Signature · 1 line
2.56.0
PATCH 2/2 xfs/601: skip test if TEST_DEV has insufficient space
Currently, the test fails with no clear message when TEST_DEV does not have enough free space to hold the copy img of the scratch device.
Check that TEST_DEV has enough free space before proceeding and skip the test with a descriptive message if it does not.
--- tests/xfs/601 | 12 ++++++++++++ 1 file changed, 12 insertions(+)
tests/xfs/601 +12 -0
@@ -34,6 +34,18 @@ _xfs_skip_offline_rebuild echo "Format and populate" _scratch_populate_cached nofill > $seqres.full 2>&1 +# Check if TEST_DEV has enough free space for SCRATCH_DEV copy img +test_avail=$(_get_available_space $TEST_DIR) +_scratch_mount +scratch_total=$(_get_total_space $SCRATCH_MNT) +scratch_avail=$(_get_available_space $SCRATCH_MNT) +scratch_occupied=$((scratch_total-scratch_avail)) +_scratch_unmount + +if [ "$((test_avail * 100))" -le "$((scratch_occupied * 110))" ]; then + _notrun "$TEST_DEV too small" +fi + mkdir -p $testdir copy_file=$testdir/copy.img
Signature · 1 line
2.56.0