Skip to contentSkip to conversation

Tests for fanotify and overlayfs

PATCH In review Amir Goldstein 4 participants 9 messages Sep 3, 2023 – Sep 15, 2023 Library code

lore
Share & tools
lore
#0 lore raw

cover Tests for fanotify and overlayfs

Hi Petr,

Kernel 6.5 and upcoming kernel 6.6 have several improvements to the integration of fanotify and overlayfs.

Kernel 6.5 includes the fix to generate fanotify events on watched overlayfs underlying filesystem (tst_variant 1).

Kernel 6.5 also added support for the new AT_HANDLE_FID flag, which is used in this test.

Kernel 6.6 brings overlayfs support for AT_HANDLE_FID and with it better support for watching overlayfs with fanotify (tst_variant 2).

tst_variant 2 is duely skipped on kernel 6.5 and tst_variant 1 is skipped on older kernels.

Since the overlayfs changes are not in a release kernel yet, and since this is not a new test that can go to staging tests, I will leave it to you to decide how and when you want to merge these new test variants.

Thanks,
Amir.

Amir Goldstein (3):
fanotify13: Test watching overlayfs upper fs
fanotify13: Test watching overlayfs with FAN_REPORT_FID
fanotify13: Test unique overlayfs fsid

 lib/tst_fs_setup.c                            |  2 +-
 testcases/kernel/syscalls/fanotify/fanotify.h | 36 ++++++++-
 .../kernel/syscalls/fanotify/fanotify13.c     | 77 +++++++++++++++++--
 3 files changed, 105 insertions(+), 10 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/3 fanotify13: Test watching overlayfs upper fs

Run a test variant with overlayfs (over all supported fs) when watching the upper fs.

This is a regression test for kernel fix bc2473c90fca
("ovl: enable fsnotify events on underlying real files"), from kernel 6.5, which is not likely to be backported to older kernels.

To avoid waiting for events that won't arrive when testing old kernels, require that kernel supports encoding fid with new flag AT_HADNLE_FID, also merged to 6.5 and not likely to be backported to older kernels.

---
 lib/tst_fs_setup.c                            |  2 +-
 testcases/kernel/syscalls/fanotify/fanotify.h | 21 +++++++
 .../kernel/syscalls/fanotify/fanotify13.c     | 62 +++++++++++++++++--
 3 files changed, 79 insertions(+), 6 deletions(-)
lib/tst_fs_setup.c +1 -1
index 6b93483de..30673670f 100644
@@ -42,7 +42,7 @@ int mount_overlay(const char *file, const int lineno, int skip)
 			tst_res_(file, lineno, TINFO,
 				TST_FS_SETUP_OVERLAYFS_MSG);
 		}
-	} else {
+	} else if (skip) {
 		tst_brk_(file, lineno, TBROK | TERRNO,
 			"overlayfs mount failed");
 	}
testcases/kernel/syscalls/fanotify/fanotify.h +21 -0
index 51078103e..75a081dc9 100644
@@ -72,6 +72,10 @@ static inline int safe_fanotify_mark(const char *file, const int lineno,
 #define MAX_HANDLE_SZ		128
 #endif
 
+#ifndef AT_HANDLE_FID
+#define AT_HANDLE_FID		0x200
+#endif
+
 /*
  * Helper function used to obtain fsid and file_handle for a given path.
  * Used by test files correlated to FAN_REPORT_FID functionality.
@@ -260,10 +264,27 @@ static inline int fanotify_mark_supported_by_kernel(uint64_t flag)
 	return rval;
 }
 
+static inline int fanotify_handle_supported_by_kernel(int flag)
+{
+	/*
+	 * On Kernel that does not support AT_HANDLE_FID this will result
+	 * with EINVAL. On older kernels, this will result in EBADF.
+	 */
+	if (name_to_handle_at(-1, "", NULL, NULL, AT_EMPTY_PATH | flag)) {
+		if (errno == EINVAL)
+			return -1;
+	}
+	return 0;
+}
+
 #define REQUIRE_MARK_TYPE_SUPPORTED_BY_KERNEL(mark_type) \
 	fanotify_init_flags_err_msg(#mark_type, __FILE__, __LINE__, tst_brk_, \
 				    fanotify_mark_supported_by_kernel(mark_type))
 
+#define REQUIRE_HANDLE_TYPE_SUPPORTED_BY_KERNEL(handle_type) \
+	fanotify_init_flags_err_msg(#handle_type, __FILE__, __LINE__, tst_brk_, \
+				    fanotify_handle_supported_by_kernel(handle_type))
+
 #define REQUIRE_FANOTIFY_EVENTS_SUPPORTED_ON_FS(init_flags, mark_type, mask, fname) do { \
 	if (mark_type)							\
 		REQUIRE_MARK_TYPE_SUPPORTED_BY_KERNEL(mark_type);	\
testcases/kernel/syscalls/fanotify/fanotify13.c +57 -5

119 lines not on this page · raw message

Signature · 1 line
2.34.1
#2 lore raw

Re: [LTP] [PATCH 1/3] fanotify13: Test watching overlayfs upper fs

Hello Amir,

Quoted 10 lines · Amir Goldstein

Run a test variant with overlayfs (over all supported fs) when watching the upper fs.

This is a regression test for kernel fix bc2473c90fca
("ovl: enable fsnotify events on underlying real files"), from kernel 6.5, which is not likely to be backported to older kernels.

To avoid waiting for events that won't arrive when testing old kernels, require that kernel supports encoding fid with new flag AT_HADNLE_FID, also merged to 6.5 and not likely to be backported to older kernels.

Unfortunately Petr's not here at the moment.

I guess this first patch doesn't require 6.6? So it could be merged independently without further considerations for what makes it into 6.6?

Signature · 2 lines
Thank you,
Richard.
#3 lore raw

Re: [LTP] [PATCH 1/3] fanotify13: Test watching overlayfs upper fs

Quoted 12 lines · Richard Palethorpe

Hello Amir,

Quoted 7 lines · Amir Goldstein

Run a test variant with overlayfs (over all supported fs) when watching the upper fs.

This is a regression test for kernel fix bc2473c90fca
("ovl: enable fsnotify events on underlying real files"), from kernel 6.5, which is not likely to be backported to older kernels.

Quoted 9 lines
Quoted 3 lines

To avoid waiting for events that won't arrive when testing old kernels, require that kernel supports encoding fid with new flag AT_HADNLE_FID, also merged to 6.5 and not likely to be backported to older kernels.

Unfortunately Petr's not here at the moment.

I guess this first patch doesn't require 6.6? So it could be merged independently without further considerations for what makes it into 6.6?

Yes that is correct.

Thanks,
Amir.

Reviewed-by
#4 lore raw

Re: [LTP] [PATCH 1/3] fanotify13: Test watching overlayfs upper fs

Hi Amir, Richie,

Quoted 17 lines · Richard Palethorpe
Quoted 15 lines

Hello Amir,

Quoted 10 lines · Amir Goldstein

Run a test variant with overlayfs (over all supported fs) when watching the upper fs.

This is a regression test for kernel fix bc2473c90fca
("ovl: enable fsnotify events on underlying real files"), from kernel 6.5, which is not likely to be backported to older kernels.

To avoid waiting for events that won't arrive when testing old kernels, require that kernel supports encoding fid with new flag AT_HADNLE_FID, also merged to 6.5 and not likely to be backported to older kernels.

Quoted 7 lines
Quoted 4 lines

Unfortunately Petr's not here at the moment.

I guess this first patch doesn't require 6.6? So it could be merged independently without further considerations for what makes it into 6.6?

Yes that is correct.

I'm finally back :).

I'll ask Cyril this patch to be merged before LTP release (final day today before git freeze).

Kind regards,
Petr

Quoted 2 lines

Thanks,
Amir.

Reviewed-by on v1 1/3
#5 lore raw

Re: [PATCH 1/3] fanotify13: Test watching overlayfs upper fs

In reply to Amir Goldstein

Hi!

Quoted 11 lines
---
 lib/tst_fs_setup.c                            |  2 +-
 testcases/kernel/syscalls/fanotify/fanotify.h | 21 +++++++
 .../kernel/syscalls/fanotify/fanotify13.c     | 62 +++++++++++++++++--
 3 files changed, 79 insertions(+), 6 deletions(-)
lib/tst_fs_setup.c +0 -0
index 6b93483de..30673670f 100644

lib/tst_fs_setup.c int mount_overlay(const char *file, const int lineno, int skip) · line 48

Quoted 8 lines
lib/tst_fs_setup.c +1 -1
@@ -42,7 +42,7 @@ int mount_overlay(const char *file, const int lineno, int skip)
 			tst_res_(file, lineno, TINFO,
 				TST_FS_SETUP_OVERLAYFS_MSG);
 		}
-	} else {
+	} else if (skip) {
 		tst_brk_(file, lineno, TBROK | TERRNO,
 			"overlayfs mount failed");
 	}

The skip flag should be called strict, at least that is what we usually name it, but that is very minor.

...

lib/tst_fs_setup.c

Quoted 13 lines

static struct tst_test test = {

lib/tst_fs_setup.c +2 -1
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(test_cases),
+	.test_variants = 2,
 	.setup = do_setup,
 	.cleanup = do_cleanup,
 	.needs_root = 1,
 	.mount_device = 1,
-	.mntpoint = MOUNT_PATH,
+	.mntpoint = OVL_BASE_MNTPOINT,
 	.all_filesystems = 1,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "c285a2f01d69"},

The git hash for the regression test with variant=1 should have been added here.

The rest looks good to me. With the two minor issues fixed:

@Peter Vorel Feel free to push the patch with the two fixes applied.

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw

Re: [PATCH 1/3] fanotify13: Test watching overlayfs upper fs

Hi all,

...

Quoted 28 lines

lib/tst_fs_setup.c int mount_overlay(const char *file, const int lineno, int skip) · line 48

Quoted 5 lines
lib/tst_fs_setup.c +1 -1
-	} else {
+	} else if (skip) {
 		tst_brk_(file, lineno, TBROK | TERRNO,
 			"overlayfs mount failed");
 	}

The skip flag should be called strict, at least that is what we usually name it, but that is very minor.

...

testcases/kernel/syscalls/fanotify/fanotify13.c static void do_cleanup(void) · line 354

Quoted 13 lines

static struct tst_test test = {

testcases/kernel/syscalls/fanotify/fanotify13.c +2 -1
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(test_cases),
+	.test_variants = 2,
 	.setup = do_setup,
 	.cleanup = do_cleanup,
 	.needs_root = 1,
 	.mount_device = 1,
-	.mntpoint = MOUNT_PATH,
+	.mntpoint = OVL_BASE_MNTPOINT,
 	.all_filesystems = 1,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "c285a2f01d69"},

The git hash for the regression test with variant=1 should have been added here.

Quoted 5 lines

The rest looks good to me. With the two minor issues fixed:

@Peter Vorel Feel free to push the patch with the two fixes applied.

Thanks for spotting both issues, fixed and merged.

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#7 lore raw

PATCH 2/3 fanotify13: Test watching overlayfs with FAN_REPORT_FID

In reply to Amir Goldstein

Run a test variant watching overlayfs (over all supported fs) and reporting events with fid.

This requires overlayfs support for AT_HANDLE_FID.

---
 testcases/kernel/syscalls/fanotify/fanotify.h   | 15 +++++++++++----
 testcases/kernel/syscalls/fanotify/fanotify13.c |  7 +++++--
 2 files changed, 16 insertions(+), 6 deletions(-)
testcases/kernel/syscalls/fanotify/fanotify.h +11 -4
index 75a081dc9..eea1dad91 100644
@@ -93,6 +93,11 @@ static inline void fanotify_get_fid(const char *path, __kernel_fsid_t *fsid,
 
 	if (name_to_handle_at(AT_FDCWD, path, handle, &mount_id, 0) == -1) {
 		if (errno == EOPNOTSUPP) {
+			/* Try to request non-decodeable fid instead */
+			if (name_to_handle_at(AT_FDCWD, path, handle, &mount_id,
+					      AT_HANDLE_FID) == 0)
+				return;
+
 			tst_brk(TCONF,
 				"filesystem %s does not support file handles",
 				tst_device->fs_type);
@@ -179,6 +184,7 @@ static inline int fanotify_events_supported_by_kernel(uint64_t mask,
  * @return  0: fanotify supported both in kernel and on tested filesystem
  * @return -1: @flags not supported in kernel
  * @return -2: @flags not supported on tested filesystem (tested if @fname is not NULL)
+ * @return -3: @flags not supported on overlayfs (tested if @fname == OVL_MNT)
  */
 static inline int fanotify_init_flags_supported_on_fs(unsigned int flags, const char *fname)
 {
@@ -199,7 +205,7 @@ static inline int fanotify_init_flags_supported_on_fs(unsigned int flags, const
 
 	if (fname && fanotify_mark(fd, FAN_MARK_ADD, FAN_ACCESS, AT_FDCWD, fname) < 0) {
 		if (errno == ENODEV || errno == EOPNOTSUPP || errno == EXDEV) {
-			rval = -2;
+			rval = strcmp(fname, OVL_MNT) ? -2 : -3;
 		} else {
 			tst_brk(TBROK | TERRNO,
 				"fanotify_mark (%d, FAN_MARK_ADD, ..., AT_FDCWD, %s) failed",
@@ -226,10 +232,11 @@ static inline void fanotify_init_flags_err_msg(const char *flags_str,
 	if (fail == -1)
 		res_func(file, lineno, TCONF,
 			 "%s not supported in kernel?", flags_str);
-	if (fail == -2)
+	if (fail == -2 || fail == -3)
 		res_func(file, lineno, TCONF,
-			 "%s not supported on %s filesystem",
-			 flags_str, tst_device->fs_type);
+			 "%s not supported on %s%s filesystem",
+			 flags_str, fail == -3 ? "overlayfs over " : "",
+			 tst_device->fs_type);
 }
 
 #define FANOTIFY_INIT_FLAGS_ERR_MSG(flags, fail) \
testcases/kernel/syscalls/fanotify/fanotify13.c +5 -2
index adba41453..5c1d287d7 100644
@@ -287,6 +287,7 @@ static void do_setup(void)
 	 * Bind mount to either base fs or to overlayfs over base fs:
 	 * Variant #0: watch base fs - open files on base fs
 	 * Variant #1: watch upper fs - open files on overlayfs
+	 * Variant #2: watch overlayfs - open files on overlayfs
 	 *
 	 * Variant #1 tests a bug whose fix bc2473c90fca ("ovl: enable fsnotify
 	 * events on underlying real files") in kernel 6.5 is not likely to be
@@ -294,11 +295,13 @@ static void do_setup(void)
 	 * To avoid waiting for events that won't arrive when testing old kernels,
 	 * require that kernel supports encoding fid with new flag AT_HADNLE_FID,
 	 * also merged to 6.5 and not likely to be backported to older kernels.
+	 * Variant #2 tests overlayfs watch with FAN_REPORT_FID, which also
+	 * requires kernel with support for AT_HADNLE_FID.
 	 */
 	if (tst_variant) {
 		REQUIRE_HANDLE_TYPE_SUPPORTED_BY_KERNEL(AT_HANDLE_FID);
 		ovl_mounted = TST_MOUNT_OVERLAY();
-		mnt = OVL_UPPER;
+		mnt = tst_variant == 1 ? OVL_UPPER : OVL_MNT;
 	} else {
 		mnt = OVL_BASE_MNTPOINT;
 
@@ -343,7 +346,7 @@ static void do_cleanup(void)
 static struct tst_test test = {
 	.test = do_test,
 	.tcnt = ARRAY_SIZE(test_cases),
-	.test_variants = 2,
+	.test_variants = 3,
 	.setup = do_setup,
 	.cleanup = do_cleanup,
 	.needs_root = 1,
Signature · 1 line
2.34.1
#8 lore raw

PATCH 3/3 fanotify13: Test unique overlayfs fsid

In reply to Amir Goldstein

With feature uuid=auto, overlayfs gets an fsid which is different than the base fs fsid.

---
 testcases/kernel/syscalls/fanotify/fanotify13.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
testcases/kernel/syscalls/fanotify/fanotify13.c +12 -0
index 5c1d287d7..3bb9eb1df 100644
@@ -96,6 +96,7 @@ static int fanotify_fd;
 static int filesystem_mark_unsupported;
 static char events_buf[BUF_SIZE];
 static struct event_t event_set[EVENT_MAX];
+static struct fanotify_fid_t base_fs_fid;
 
 static void create_objects(void)
 {
@@ -113,8 +114,19 @@ static void get_object_stats(void)
 {
 	unsigned int i;
 
+	fanotify_save_fid(OVL_BASE_MNTPOINT, &base_fs_fid);
 	for (i = 0; i < ARRAY_SIZE(objects); i++)
 		fanotify_save_fid(objects[i].path, &objects[i].fid);
+
+	/* Variant #2: watching overlayfs - expect fsid != base fs fsid */
+	if (ovl_mounted && tst_variant == 2 &&
+	    memcmp(&objects[0].fid.fsid, &base_fs_fid.fsid,
+		   sizeof(base_fs_fid.fsid)) == 0) {
+		tst_res(TFAIL,
+			"overlayfs fsid is the same as stat.f_fsid that was "
+			"obtained via statfs(2) on the base fs");
+	}
+
 }
 
 static int setup_marks(unsigned int fd, struct test_case_t *tc)
Signature · 1 line
2.34.1