Skip to contentSkip to conversation

overlay: add test for lowerdir mount option parsing

PATCH Superseded Amir Goldstein 2 participants 3 messages Oct 23, 2023

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 overlay: add test for lowerdir mount option parsing

Check parsing and display of spaces and escaped colons and commans in lowerdir mount option.

This is a regression test for two bugs introduced in v6.5 with the conversion to new mount api.

There is another regression of new mount api related to libmount parsing of escaped commas, but this needs a fix in libmount - this test only verifies the fixes in the kernel, so it uses LIBMOUNT_FORCE_MOUNT2=always to force mount(2) and kernel pasring of the comma separated options list.

---

Zorro,

I've fixed the libmount issue by splitting the combined test cases to two test cases that corresspond to the two kernel fix commits.

The first test case (lowerdir_spaces) is agnostic to libmount version.

The second test case (lowerdir_commas) explicitly opts-in to mount(2) syscall.

ATM, using LIBMOUNT_FORCE_MOUNT2=always, as the second test cases does, would be our recommended workaround for the escaped commas regression in v6.5, until libmount gets a fix to detect overlayfs escaped commas.

Thanks,
Amir.

 tests/overlay/083     | 71 +++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/083.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/overlay/083
 create mode 100644 tests/overlay/083.out
tests/overlay/083 new file +71 -0
new file mode 100755
index 00000000..0f434951
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test 083
+#
+# Test regressions in parsing and display of special chars in mount options.
+#
+# The following kernel commits from v6.5 introduced regressions:
+#  b36a5780cb44 ("ovl: modify layer parameter parsing")
+#  1784fbc2ed9c ("ovl: port to new mount api")
+#
+
+. ./common/preamble
+_begin_fstest auto quick mount
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs overlay
+_fixed_by_kernel_commit 32db51070850 \
+	"ovl: fix regression in showing lowerdir mount option"
+_fixed_by_kernel_commit c34706acf40b \
+	"ovl: fix regression in parsing of mount options with escaped comma"
+
+# _overlay_check_* helpers do not handle special chars well
+_require_scratch_nocheck
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create lowerdirs with special characters
+lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with  spaces"
+lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons"
+lowerdir_commas="$OVL_BASE_SCRATCH_MNT/lower3,with,,commas"
+lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons"
+lowerdir_commas_esc="$OVL_BASE_SCRATCH_MNT/lower3\,with\,\,commas"
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+mkdir -p "$lowerdir_spaces" "$lowerdir_colons" "$lowerdir_commas"
+
+# _overlay_mount_* helpers do not handle special chars well, so execute mount directly.
+# if escaped colons are not parsed correctly, mount will fail.
+$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \
+	-o"upperdir=$upperdir,workdir=$workdir" \
+	-o"lowerdir=$lowerdir_colons_esc:$lowerdir_spaces" \
+	2>&1 | tee -a $seqres.full
+
+# if spaces are not escaped when showing mount options,
+# mount command will not show the word 'spaces' after the spaces
+$MOUNT_PROG -t overlay | grep ovl_esc_test  | tee -a $seqres.full | grep -v spaces && \
+	echo "ERROR: escaped spaces truncated from lowerdir mount option"
+$UMOUNT_PROG $SCRATCH_MNT
+
+# Kernel commit c34706acf40b fixes parsing of mount options with escaped comma
+# when the mount options string is provided via data argument to mount(2) syscall.
+# With libmount >= 2.39, libmount itself will try to split the comma separated
+# options list provided to mount(8) commnad line and call fsconfig(2) for each
+# mount option seperately.  Since libmount does not obay to overlayfs escaped
+# commas format, it will call fsconfig(2) with the wrong path (i.e. ".../lower3")
+# and this test will fail, but the failure would indicate a libmount issue, not
+# a kernel issue.  Therefore, force libmount to use mount(2) syscall, so we only
+# test the kernel fix.
+LIBMOUNT_FORCE_MOUNT2=always $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_DEV $SCRATCH_MNT \
+	-o"upperdir=$upperdir,workdir=$workdir,lowerdir=$lowerdir_commas_esc" 2>> $seqres.full || \
+	echo "ERROR: incorrect parsing of escaped comma in lowerdir mount option"
+
+echo "Silence is golden"
+status=0
+exit
tests/overlay/083.out new file +2 -0
new file mode 100644
index 00000000..0beba309
@@ -0,0 +1,2 @@
+QA output created by 083
+Silence is golden
Signature · 1 line
2.34.1
#1 lore raw
Quoted 100 lines · Amir Goldstein

Check parsing and display of spaces and escaped colons and commans in lowerdir mount option.

This is a regression test for two bugs introduced in v6.5 with the conversion to new mount api.

There is another regression of new mount api related to libmount parsing of escaped commas, but this needs a fix in libmount - this test only verifies the fixes in the kernel, so it uses LIBMOUNT_FORCE_MOUNT2=always to force mount(2) and kernel pasring of the comma separated options list.

---

Zorro,

I've fixed the libmount issue by splitting the combined test cases to two test cases that corresspond to the two kernel fix commits.

The first test case (lowerdir_spaces) is agnostic to libmount version.

The second test case (lowerdir_commas) explicitly opts-in to mount(2) syscall.

ATM, using LIBMOUNT_FORCE_MOUNT2=always, as the second test cases does, would be our recommended workaround for the escaped commas regression in v6.5, until libmount gets a fix to detect overlayfs escaped commas.

Thanks,
Amir.

 tests/overlay/083     | 71 +++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/083.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/overlay/083
 create mode 100644 tests/overlay/083.out
tests/overlay/083 new file +57 -0
new file mode 100755
index 00000000..0f434951
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test 083
+#
+# Test regressions in parsing and display of special chars in mount options.
+#
+# The following kernel commits from v6.5 introduced regressions:
+#  b36a5780cb44 ("ovl: modify layer parameter parsing")
+#  1784fbc2ed9c ("ovl: port to new mount api")
+#
+
+. ./common/preamble
+_begin_fstest auto quick mount
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs overlay
+_fixed_by_kernel_commit 32db51070850 \
+	"ovl: fix regression in showing lowerdir mount option"
+_fixed_by_kernel_commit c34706acf40b \
+	"ovl: fix regression in parsing of mount options with escaped comma"
+
+# _overlay_check_* helpers do not handle special chars well
+_require_scratch_nocheck
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create lowerdirs with special characters
+lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with  spaces"
+lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons"
+lowerdir_commas="$OVL_BASE_SCRATCH_MNT/lower3,with,,commas"
+lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons"
+lowerdir_commas_esc="$OVL_BASE_SCRATCH_MNT/lower3\,with\,\,commas"
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+mkdir -p "$lowerdir_spaces" "$lowerdir_colons" "$lowerdir_commas"
+
+# _overlay_mount_* helpers do not handle special chars well, so execute mount directly.
+# if escaped colons are not parsed correctly, mount will fail.
+$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \
+	-o"upperdir=$upperdir,workdir=$workdir" \
+	-o"lowerdir=$lowerdir_colons_esc:$lowerdir_spaces" \
+	2>&1 | tee -a $seqres.full
+
+# if spaces are not escaped when showing mount options,
+# mount command will not show the word 'spaces' after the spaces
+$MOUNT_PROG -t overlay | grep ovl_esc_test  | tee -a $seqres.full | grep -v spaces && \
+	echo "ERROR: escaped spaces truncated from lowerdir mount option"
+$UMOUNT_PROG $SCRATCH_MNT
+
+# Kernel commit c34706acf40b fixes parsing of mount options with escaped comma
+# when the mount options string is provided via data argument to mount(2) syscall.

tests/overlay/083 line 67

Quoted 10 lines
tests/overlay/083 new file +10 -0
+# With libmount >= 2.39, libmount itself will try to split the comma separated
+# options list provided to mount(8) commnad line and call fsconfig(2) for each
+# mount option seperately.  Since libmount does not obay to overlayfs escaped
+# commas format, it will call fsconfig(2) with the wrong path (i.e. ".../lower3")
+# and this test will fail, but the failure would indicate a libmount issue, not
+# a kernel issue.  Therefore, force libmount to use mount(2) syscall, so we only
+# test the kernel fix.
+LIBMOUNT_FORCE_MOUNT2=always $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_DEV $SCRATCH_MNT \
+	-o"upperdir=$upperdir,workdir=$workdir,lowerdir=$lowerdir_commas_esc" 2>> $seqres.full || \
+	echo "ERROR: incorrect parsing of escaped comma in lowerdir mount option"

Hi Amir, Please check, this part still fails as:

# ./check -overlay overlay/083
FSTYP         -- overlay
PLATFORM      -- Linux/x86_64 hp-dl380pg8-01 6.6.0-rc6-mainline+ #7 SMP PREEMPT_DYNAMIC Thu Oct 19 22:34:28 CST 2023
MKFS_OPTIONS  -- /mnt/scratch
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /mnt/scratch /mnt/scratch/ovl-mnt

overlay/083 0s ... - output mismatch (see /root/git/xfstests/results//overlay/083.out.bad)
--- tests/overlay/083.out 2023-10-23 21:18:46.765059777 +0800
+++ /root/git/xfstests/results//overlay/083.out.bad 2023-10-23 21:51:39.030561512 +0800
@@ -1,2 +1,3 @@
QA output created by 083
+ERROR: incorrect parsing of escaped comma in lowerdir mount option
Silence is golden
...
(Run 'diff -u /root/git/xfstests/tests/overlay/083.out /root/git/xfstests/results//overlay/083.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
32db51070850 ovl: fix regression in showing lowerdir mount option

HINT: You _MAY_ be missing kernel fix:
c34706acf40b ovl: fix regression in parsing of mount options with escaped comma

Ran: overlay/083
Failures: overlay/083
Failed 1 of 1 tests

# cat results/overlay/083.full 
ovl_esc_test on /mnt/scratch/ovl-mnt type overlay (rw,relatime,seclabel,lowerdir=/mnt/scratch/lower2\:with\:\:colons:/mnt/scratch/lower1 with  spaces,upperdir=/mnt/scratch/ovl-upper,workdir=/mnt/scratch/ovl-work,uuid=on)
mount: /mnt/scratch/ovl-mnt: mount(2) system call failed: Stale file handle.
       dmesg(1) may have more information after failed mount system call.
# dmesg
[341033.046302] run fstests overlay/083 at 2023-10-23 22:00:11
[341033.458188] XFS (loop0): Mounting V5 Filesystem 23e1781e-f5ce-4c2b-8801-3b586f459ee8
[341033.464410] XFS (loop0): Ending clean mount
[341033.506666] overlayfs: failed to verify origin (/lower3,with,,commas, ino=16908385, err=-116)
[341033.507626] overlayfs: failed to verify upper root origin
[341033.540865] XFS (loop1): Unmounting Filesystem 58e81b68-81dd-48cd-ab23-ce30ec77e689
[341033.554488] XFS (loop0): Unmounting Filesystem 23e1781e-f5ce-4c2b-8801-3b586f459ee8
Quoted 15 lines
tests/overlay/083 new file +4 -0
+
+echo "Silence is golden"
+status=0
+exit
tests/overlay/083.out new file +2 -0
new file mode 100644
index 00000000..0beba309
@@ -0,0 +1,2 @@
+QA output created by 083
+Silence is golden
Signature · 1 line
2.34.1
#2 lore raw
Quoted 146 lines · Zorro Lang
Quoted 100 lines · Amir Goldstein

Check parsing and display of spaces and escaped colons and commans in lowerdir mount option.

This is a regression test for two bugs introduced in v6.5 with the conversion to new mount api.

There is another regression of new mount api related to libmount parsing of escaped commas, but this needs a fix in libmount - this test only verifies the fixes in the kernel, so it uses LIBMOUNT_FORCE_MOUNT2=always to force mount(2) and kernel pasring of the comma separated options list.

---

Zorro,

I've fixed the libmount issue by splitting the combined test cases to two test cases that corresspond to the two kernel fix commits.

The first test case (lowerdir_spaces) is agnostic to libmount version.

The second test case (lowerdir_commas) explicitly opts-in to mount(2) syscall.

ATM, using LIBMOUNT_FORCE_MOUNT2=always, as the second test cases does, would be our recommended workaround for the escaped commas regression in v6.5, until libmount gets a fix to detect overlayfs escaped commas.

Thanks,
Amir.

 tests/overlay/083     | 71 +++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/083.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/overlay/083
 create mode 100644 tests/overlay/083.out
tests/overlay/083 new file +57 -0
new file mode 100755
index 00000000..0f434951
@@ -0,0 +1,71 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test 083
+#
+# Test regressions in parsing and display of special chars in mount options.
+#
+# The following kernel commits from v6.5 introduced regressions:
+#  b36a5780cb44 ("ovl: modify layer parameter parsing")
+#  1784fbc2ed9c ("ovl: port to new mount api")
+#
+
+. ./common/preamble
+_begin_fstest auto quick mount
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs overlay
+_fixed_by_kernel_commit 32db51070850 \
+     "ovl: fix regression in showing lowerdir mount option"
+_fixed_by_kernel_commit c34706acf40b \
+     "ovl: fix regression in parsing of mount options with escaped comma"
+
+# _overlay_check_* helpers do not handle special chars well
+_require_scratch_nocheck
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create lowerdirs with special characters
+lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with  spaces"
+lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons"
+lowerdir_commas="$OVL_BASE_SCRATCH_MNT/lower3,with,,commas"
+lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons"
+lowerdir_commas_esc="$OVL_BASE_SCRATCH_MNT/lower3\,with\,\,commas"
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+mkdir -p "$lowerdir_spaces" "$lowerdir_colons" "$lowerdir_commas"
+
+# _overlay_mount_* helpers do not handle special chars well, so execute mount directly.
+# if escaped colons are not parsed correctly, mount will fail.
+$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \
+     -o"upperdir=$upperdir,workdir=$workdir" \
+     -o"lowerdir=$lowerdir_colons_esc:$lowerdir_spaces" \
+     2>&1 | tee -a $seqres.full
+
+# if spaces are not escaped when showing mount options,
+# mount command will not show the word 'spaces' after the spaces
+$MOUNT_PROG -t overlay | grep ovl_esc_test  | tee -a $seqres.full | grep -v spaces && \
+     echo "ERROR: escaped spaces truncated from lowerdir mount option"
+$UMOUNT_PROG $SCRATCH_MNT
+
+# Kernel commit c34706acf40b fixes parsing of mount options with escaped comma
+# when the mount options string is provided via data argument to mount(2) syscall.

tests/overlay/083 line 67

Quoted 10 lines
tests/overlay/083 new file +10 -0
+# With libmount >= 2.39, libmount itself will try to split the comma separated
+# options list provided to mount(8) commnad line and call fsconfig(2) for each
+# mount option seperately.  Since libmount does not obay to overlayfs escaped
+# commas format, it will call fsconfig(2) with the wrong path (i.e. ".../lower3")
+# and this test will fail, but the failure would indicate a libmount issue, not
+# a kernel issue.  Therefore, force libmount to use mount(2) syscall, so we only
+# test the kernel fix.
+LIBMOUNT_FORCE_MOUNT2=always $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_DEV $SCRATCH_MNT \
+     -o"upperdir=$upperdir,workdir=$workdir,lowerdir=$lowerdir_commas_esc" 2>> $seqres.full || \
+     echo "ERROR: incorrect parsing of escaped comma in lowerdir mount option"

Hi Amir, Please check, this part still fails as:

# ./check -overlay overlay/083
FSTYP         -- overlay
PLATFORM      -- Linux/x86_64 hp-dl380pg8-01 6.6.0-rc6-mainline+ #7 SMP PREEMPT_DYNAMIC Thu Oct 19 22:34:28 CST 2023
MKFS_OPTIONS  -- /mnt/scratch
MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /mnt/scratch /mnt/scratch/ovl-mnt

overlay/083 0s ... - output mismatch (see /root/git/xfstests/results//overlay/083.out.bad)
--- tests/overlay/083.out 2023-10-23 21:18:46.765059777 +0800
+++ /root/git/xfstests/results//overlay/083.out.bad 2023-10-23 21:51:39.030561512 +0800
@@ -1,2 +1,3 @@
QA output created by 083
+ERROR: incorrect parsing of escaped comma in lowerdir mount option
Silence is golden
...
(Run 'diff -u /root/git/xfstests/tests/overlay/083.out /root/git/xfstests/results//overlay/083.out.bad' to see the entire diff)

HINT: You _MAY_ be missing kernel fix:
32db51070850 ovl: fix regression in showing lowerdir mount option

HINT: You _MAY_ be missing kernel fix:
c34706acf40b ovl: fix regression in parsing of mount options with escaped comma

Ran: overlay/083
Failures: overlay/083
Failed 1 of 1 tests

# cat results/overlay/083.full
ovl_esc_test on /mnt/scratch/ovl-mnt type overlay (rw,relatime,seclabel,lowerdir=/mnt/scratch/lower2\:with\:\:colons:/mnt/scratch/lower1 with  spaces,upperdir=/mnt/scratch/ovl-upper,workdir=/mnt/scratch/ovl-work,uuid=on)
mount: /mnt/scratch/ovl-mnt: mount(2) system call failed: Stale file handle.
       dmesg(1) may have more information after failed mount system call.
Quoted 9 lines
# dmesg
[341033.046302] run fstests overlay/083 at 2023-10-23 22:00:11
[341033.458188] XFS (loop0): Mounting V5 Filesystem 23e1781e-f5ce-4c2b-8801-3b586f459ee8
[341033.464410] XFS (loop0): Ending clean mount
[341033.506666] overlayfs: failed to verify origin (/lower3,with,,commas, ino=16908385, err=-116)
[341033.507626] overlayfs: failed to verify upper root origin
[341033.540865] XFS (loop1): Unmounting Filesystem 58e81b68-81dd-48cd-ab23-ce30ec77e689
[341033.554488] XFS (loop0): Unmounting Filesystem 23e1781e-f5ce-4c2b-8801-3b586f459ee8

Oh, you must have CONFIG_OVERLAY_FS_INDEX=y.
I did not check this config.

I will send a fix soon.

Thanks,
Amir.