Skip to contentSkip to conversation

fs: add ->f_real() file operation

PATCH Superseded Amir Goldstein 3 participants 6 messages Nov 18 – Nov 20

Subsystems (8)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Amir Goldstein

author maintainer
#0 lore raw

PATCH 1/1 fs: add ->f_real() file operation

sync_file_range(2) and cachestat(2) operate on file->f_mapping. For a file on a stacking filesystem, such as overlayfs, the data and its page cache are in the real (backing) file, so on a stacked file sync_file_range() writes back nothing and cachestat() reports nothing.

Add a ->f_real() file operation, modelled after ->d_real(), and a file_real() helper in backing-file.h that returns the backing file holding the data of a stacked file, or the file itself. file_real() is a no-op without CONFIG_FS_STACK; otherwise its slow path in fs/backing-file.c calls ->f_real() and verifies that it returned a backing file. The real file is owned by the stacked file, so it is valid for as long as the caller holds the stacked file.

file_real() is only meant for inspecting and writing back the page cache of the data. Operations that need the credentials or permission checks of the stacking filesystem need a dedicated file operation, like ->fadvise().

Use file_real() in sync_file_range() (also used by io_uring) and in cachestat(), and implement ->f_real() for overlayfs.

---

Christian,

Posting the POC as proper patch after Ding has approved that it works. I hope that this is what you had in mind.

Thanks,
Amir.

Changes since POC:
- Document ->f_real() in vfs.rst and locking.rst
- Add Link, Tested-by and Cc tags

POC: https://lore.kernel.org/r/CAOQ4uxg5baphOjo+_A=3nZBmFEmMV7KOiv_W=rjEVYV41eTEaA@mail.gmail.com/

 Documentation/filesystems/locking.rst |    1 +
 Documentation/filesystems/vfs.rst     |   17 +++++++++++++++++
 fs/backing-file.c                     |   12 ++++++++++++
 fs/overlayfs/file.c                   |   11 +++++++++++
 fs/sync.c                             |    8 ++++++++
 include/linux/backing-file.h          |   26 ++++++++++++++++++++++++++
 include/linux/fs.h                    |    1 +
 mm/filemap.c                          |    9 ++++++++-
 8 files changed, 84 insertions(+), 1 deletion(-)
Documentation/filesystems/locking.rst +1 -0
@@ -485,6 +485,7 @@ prototypes::
 			struct file *file_out, loff_t pos_out,
 			loff_t len, unsigned int remap_flags);
 	int (*fadvise)(struct file *, loff_t, loff_t, int);
+	struct file *(*f_real)(const struct file *);
 
 locking rules:
 	All may block.
Documentation/filesystems/vfs.rst +17 -0
@@ -1038,6 +1038,7 @@ This describes how the VFS can manipulat
 					   struct file *file_out, loff_t pos_out,
 					   loff_t len, unsigned int remap_flags);
 		int (*fadvise)(struct file *, loff_t, loff_t, int);
+		struct file *(*f_real)(const struct file *);
 		int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags);
 		int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *,
 					unsigned int poll_flags);
@@ -1162,6 +1163,22 @@ otherwise noted.
 ``fadvise``
 	possibly called by the fadvise64() system call.
 
+``f_real``
+	stacking filesystems (e.g. overlayfs) whose file data is accessed
+	through a backing file implement this method to return that backing
+	file.  It is called through file_real() by system calls that operate
+	on the page cache of the data, such as sync_file_range(2) and
+	cachestat(2).
+
+	The returned file must be a backing file (FMODE_BACKING) or the file
+	itself.  It is owned by the file and must remain valid for as long
+	as the caller holds a reference to the file.
+
+	The returned file is only used to inspect and write back the page
+	cache of the data.  Operations that need the credentials or the
+	permission checks of the stacking filesystem need a dedicated file
+	operation (e.g. ``fadvise``).
+
 ``mmap_prepare``
 	Called by the mmap(2) system call. Allows a VFS to set up a
 	file-backed memory mapping, most notably establishing relevant
include/linux/fs.h +1 -0
@@ -1960,6 +1960,7 @@ struct file_operations {
 				   struct file *file_out, loff_t pos_out,
 				   loff_t len, unsigned int remap_flags);
 	int (*fadvise)(struct file *, loff_t, loff_t, int);
+	struct file *(*f_real)(const struct file *);
 	int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags);
 	int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *,
 				unsigned int poll_flags);
include/linux/backing-file.h +26 -0
@@ -40,5 +40,31 @@ ssize_t backing_file_splice_write(struct
 				  struct backing_file_ctx *ctx);
 int backing_file_mmap(struct file *file, struct vm_area_struct *vma,
 		      struct backing_file_ctx *ctx);
+struct file *__file_real(struct file *file);
+
+/**
+ * file_real - Return the real file hosting the data
+ * @file: the file to query
+ *
+ * If @file is on a stacking filesystem (e.g. overlayfs) whose data is
+ * accessed through a backing file, return that backing file.
+ * Otherwise return @file itself.
+ *
+ * The returned file is owned by @file and is valid for as long as the
+ * caller holds a reference to @file.
+ *
+ * The real file is only meant to be used for inspecting and writing
+ * back the page cache of the data (e.g. sync_file_range(2), cachestat(2)).
+ * It must not be used for operations that require the credentials or the
+ * permission checks of the stacking filesystem; those need a dedicated
+ * file operation (e.g. ->fadvise()).
+ */
+static inline struct file *file_real(struct file *file)
+{
+	if (!IS_ENABLED(CONFIG_FS_STACK) || likely(!file->f_op->f_real))
+		return file;
+
+	return __file_real(file);
+}
 
 #endif /* _LINUX_BACKING_FILE_H */
fs/backing-file.c +12 -0
@@ -102,6 +102,18 @@ struct file *backing_tmpfile_open(const
 }
 EXPORT_SYMBOL(backing_tmpfile_open);
 
+/* Slow path of file_real() for a file with ->f_real() */
+struct file *__file_real(struct file *file)
+{
+	struct file *real_file = file->f_op->f_real(file);
+
+	if (!IS_ERR(real_file) && real_file != file &&
+	    WARN_ON_ONCE(!(real_file->f_mode & FMODE_BACKING)))
+		return ERR_PTR(-EIO);
+
+	return real_file;
+}
+
 struct backing_aio {
 	struct kiocb iocb;
 	refcount_t ref;
fs/sync.c +8 -0
@@ -7,6 +7,7 @@
 #include <linux/kernel.h>
 #include <linux/file.h>
 #include <linux/fs.h>
+#include <linux/backing-file.h>
 #include <linux/slab.h>
 #include <linux/export.h>
 #include <linux/namei.h>
@@ -269,6 +270,13 @@ int sync_file_range(struct file *file, l
 	if (!S_ISREG(i_mode) && !S_ISBLK(i_mode) && !S_ISDIR(i_mode))
 		goto out;
 
+	/* The page cache of a stacked file is in its real file */
+	file = file_real(file);
+	if (IS_ERR(file)) {
+		ret = PTR_ERR(file);
+		goto out;
+	}
+
 	mapping = file->f_mapping;
 	ret = 0;
 	if (flags & SYNC_FILE_RANGE_WAIT_BEFORE) {
mm/filemap.c +8 -1
@@ -31,6 +31,7 @@
 #include <linux/hash.h>
 #include <linux/writeback.h>
 #include <linux/backing-dev.h>
+#include <linux/backing-file.h>
 #include <linux/folio_batch.h>
 #include <linux/security.h>
 #include <linux/cpuset.h>
@@ -4782,6 +4783,7 @@ SYSCALL_DEFINE4(cachestat, unsigned int,
 		struct cachestat __user *, cstat, unsigned int, flags)
 {
 	CLASS(fd, f)(fd);
+	struct file *real_file;
 	struct address_space *mapping;
 	struct cachestat_range csr;
 	struct cachestat cs;
@@ -4804,11 +4806,16 @@ SYSCALL_DEFINE4(cachestat, unsigned int,
 	if (flags != 0)
 		return -EINVAL;
 
+	/* The page cache of a stacked file is in its real file */
+	real_file = file_real(fd_file(f));
+	if (IS_ERR(real_file))
+		return PTR_ERR(real_file);
+
 	first_index = csr.off >> PAGE_SHIFT;
 	last_index =
 		csr.len == 0 ? ULONG_MAX : (csr.off + csr.len - 1) >> PAGE_SHIFT;
 	memset(&cs, 0, sizeof(struct cachestat));
-	mapping = fd_file(f)->f_mapping;
+	mapping = real_file->f_mapping;
 	filemap_cachestat(mapping, first_index, last_index, &cs);
 
 	if (copy_to_user(cstat, &cs, sizeof(struct cachestat)))
fs/overlayfs/file.c +11 -0
@@ -195,6 +195,16 @@ static struct file *ovl_real_file(const
 	return ovl_real_file_path(file, &realpath);
 }
 
+/*
+ * The real file that holds the page cache of the data: the upper file after
+ * copy up, otherwise the lower data file.  After copy up of a file that was
+ * opened from lower, this opens the upper file, just like a write would.
+ */
+static struct file *ovl_f_real(const struct file *file)
+{
+	return ovl_real_file(file);
+}
+
 static int ovl_open(struct inode *inode, struct file *file)
 {
 	struct dentry *dentry = file_dentry(file);
@@ -656,6 +666,7 @@ const struct file_operations ovl_file_op
 	.mmap		= ovl_mmap,
 	.fallocate	= ovl_fallocate,
 	.fadvise	= ovl_fadvise,
+	.f_real		= ovl_f_real,
 	.flush		= ovl_flush,
 	.splice_read    = ovl_splice_read,
 	.splice_write   = ovl_splice_write,
#1 lore raw
Quoted 13 lines

sync_file_range(2) and cachestat(2) operate on file->f_mapping. For a file on a stacking filesystem, such as overlayfs, the data and its page cache are in the real (backing) file, so on a stacked file sync_file_range() writes back nothing and cachestat() reports nothing.

Add a ->f_real() file operation, modelled after ->d_real(), and a file_real() helper in backing-file.h that returns the backing file holding the data of a stacked file, or the file itself. file_real() is a no-op without CONFIG_FS_STACK; otherwise its slow path in fs/backing-file.c calls ->f_real() and verifies that it returned a backing file. The real file is owned by the stacked file, so it is valid for as long as the caller holds the stacked file.

Quoted 7 lines

file_real() is only meant for inspecting and writing back the page cache of the data. Operations that need the credentials or permission checks of the stacking filesystem need a dedicated file operation, like ->fadvise().

Use file_real() in sync_file_range() (also used by io_uring) and in cachestat(), and implement ->f_real() for overlayfs.

Approach looks good.

Would you mind taking the cachestat selftest patch together with it:
https://lore.kernel.org/linux-fsdevel/[email protected]/

I will test it on your patch and update with results.

Quoted 62 lines
Documentation/filesystems/locking.rst +1 -0
index f7363ad770c76..db9ba2a8c13e7 100644
@@ -485,6 +485,7 @@ file_operations
 			struct file *file_out, loff_t pos_out,
 			loff_t len, unsigned int remap_flags);
 	int (*fadvise)(struct file *, loff_t, loff_t, int);
+	struct file *(*f_real)(const struct file *);
 
 locking rules:
 	All may block.
Documentation/filesystems/vfs.rst +17 -0
index 1824462a43711..e4ce15545244b 100644
@@ -1038,6 +1038,7 @@ This describes how the VFS can manipulate an open file.  As of kernel
 					   struct file *file_out, loff_t pos_out,
 					   loff_t len, unsigned int remap_flags);
 		int (*fadvise)(struct file *, loff_t, loff_t, int);
+		struct file *(*f_real)(const struct file *);
 		int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags);
 		int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *,
 					unsigned int poll_flags);
@@ -1162,6 +1163,22 @@ otherwise noted.
 ``fadvise``
 	possibly called by the fadvise64() system call.
 
+``f_real``
+	stacking filesystems (e.g. overlayfs) whose file data is accessed
+	through a backing file implement this method to return that backing
+	file.  It is called through file_real() by system calls that operate
+	on the page cache of the data, such as sync_file_range(2) and
+	cachestat(2).
+
+	The returned file must be a backing file (FMODE_BACKING) or the file
+	itself.  It is owned by the file and must remain valid for as long
+	as the caller holds a reference to the file.
+
+	The returned file is only used to inspect and write back the page
+	cache of the data.  Operations that need the credentials or the
+	permission checks of the stacking filesystem need a dedicated file
+	operation (e.g. ``fadvise``).
+
 ``mmap_prepare``
 	Called by the mmap(2) system call. Allows a VFS to set up a
 	file-backed memory mapping, most notably establishing relevant
fs/backing-file.c +0 -0
index c601b27f0fd35..1e0e85515b4c7 100644
@@ -102,6 +102,18 @@ struct file *backing_tmpfile_open(const struct file *user_file, int flags,
 }
 EXPORT_SYMBOL(backing_tmpfile_open);
 

fs/backing-file.c struct file *backing_tmpfile_open(const struct file *user_file, int flags, · line 114

Quoted 10 lines
fs/backing-file.c +10 -0
+/* Slow path of file_real() for a file with ->f_real() */
+struct file *__file_real(struct file *file)
+{
+	struct file *real_file = file->f_op->f_real(file);
+
+	if (!IS_ERR(real_file) && real_file != file &&
+	    WARN_ON_ONCE(!(real_file->f_mode & FMODE_BACKING)))
+		return ERR_PTR(-EIO);
+
+	return real_file;

This will go one level only AFAICS and in case of nested overlay we would not get to the real file.

Signature · 1 line
Pavel Tikhomirov <[email protected]>
author maintainer
#2 lore raw
Quoted 22 lines · Pavel Tikhomirov
Quoted 14 lines

sync_file_range(2) and cachestat(2) operate on file->f_mapping. For a file on a stacking filesystem, such as overlayfs, the data and its page cache are in the real (backing) file, so on a stacked file sync_file_range() writes back nothing and cachestat() reports nothing.

Add a ->f_real() file operation, modelled after ->d_real(), and a file_real() helper in backing-file.h that returns the backing file holding the data of a stacked file, or the file itself. file_real() is a no-op without CONFIG_FS_STACK; otherwise its slow path in fs/backing-file.c calls ->f_real() and verifies that it returned a backing file. The real file is owned by the stacked file, so it is valid for as long as the caller holds the stacked file.

Quoted 7 lines

file_real() is only meant for inspecting and writing back the page cache of the data. Operations that need the credentials or permission checks of the stacking filesystem need a dedicated file operation, like ->fadvise().

Use file_real() in sync_file_range() (also used by io_uring) and in cachestat(), and implement ->f_real() for overlayfs.

Quoted 7 lines

Approach looks good.

Would you mind taking the cachestat selftest patch together with it:
https://lore.kernel.org/linux-fsdevel/[email protected]/

I will test it on your patch and update with results.

Since you found a bug, do you mind adding the nested ovl test case as well as lower file test case and re-post your test?

Merging your test is independent of merging my fix, but Christian can take them together when you have approved both work.

Quoted 62 lines
Quoted 62 lines
Documentation/filesystems/locking.rst +1 -0
index f7363ad770c76..db9ba2a8c13e7 100644
@@ -485,6 +485,7 @@ file_operations
                      struct file *file_out, loff_t pos_out,
                      loff_t len, unsigned int remap_flags);
      int (*fadvise)(struct file *, loff_t, loff_t, int);
+     struct file *(*f_real)(const struct file *);
 
 locking rules:
      All may block.
Documentation/filesystems/vfs.rst +17 -0
index 1824462a43711..e4ce15545244b 100644
@@ -1038,6 +1038,7 @@ This describes how the VFS can manipulate an open file.  As of kernel
                                         struct file *file_out, loff_t pos_out,
                                         loff_t len, unsigned int remap_flags);
              int (*fadvise)(struct file *, loff_t, loff_t, int);
+             struct file *(*f_real)(const struct file *);
              int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags);
              int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *,
                                      unsigned int poll_flags);
@@ -1162,6 +1163,22 @@ otherwise noted.
 ``fadvise``
      possibly called by the fadvise64() system call.
 
+``f_real``
+     stacking filesystems (e.g. overlayfs) whose file data is accessed
+     through a backing file implement this method to return that backing
+     file.  It is called through file_real() by system calls that operate
+     on the page cache of the data, such as sync_file_range(2) and
+     cachestat(2).
+
+     The returned file must be a backing file (FMODE_BACKING) or the file
+     itself.  It is owned by the file and must remain valid for as long
+     as the caller holds a reference to the file.
+
+     The returned file is only used to inspect and write back the page
+     cache of the data.  Operations that need the credentials or the
+     permission checks of the stacking filesystem need a dedicated file
+     operation (e.g. ``fadvise``).
+
 ``mmap_prepare``
      Called by the mmap(2) system call. Allows a VFS to set up a
      file-backed memory mapping, most notably establishing relevant
fs/backing-file.c +0 -0
index c601b27f0fd35..1e0e85515b4c7 100644
@@ -102,6 +102,18 @@ struct file *backing_tmpfile_open(const struct file *user_file, int flags,
 }
 EXPORT_SYMBOL(backing_tmpfile_open);
 
Quoted 13 lines

fs/backing-file.c struct file *backing_tmpfile_open(const · line 114

Quoted 10 lines
fs/backing-file.c +10 -0
+/* Slow path of file_real() for a file with ->f_real() */
+struct file *__file_real(struct file *file)
+{
+     struct file *real_file = file->f_op->f_real(file);
+
+     if (!IS_ERR(real_file) && real_file != file &&
+         WARN_ON_ONCE(!(real_file->f_mode & FMODE_BACKING)))
+             return ERR_PTR(-EIO);
+
+     return real_file;

This will go one level only AFAICS and in case of nested overlay we would not get to the real file.

Good catch! but the bug is not here it is in ovl_f_real() Please test with

static struct file *ovl_f_real(const struct file *file) {
- return ovl_real_file(file);
+ struct file *real_file = ovl_real_file(file);
+
+ if (IS_ERR(real_file))
+ return real_file;
+
+ /* Handle recursion into stacked lower fs */
+ return file_real(real_file);
}

When you have tested this please report back and I will post v2.

Thanks,
Amir.

#3 lore raw

PATCH 1/1 selftests/cachestat: add overlayfs test cases

Run cachestat() on overlayfs files in four variants. The file is written either to the lower dir of an overlayfs or through its mount, so its page cache is in the lower or in the upper dir. Then it is opened read-only, so it is not copied up, and checked:

- through the overlayfs mount: the page cache is one stacking level down;
- through a second overlayfs mount, which has the first one's merged dir as its lower dir: the page cache is two stacking levels down.

Overlayfs does not cache file data in its own inodes, so cachestat() sees the pages only if it is forwarded to the real file.

Add a check_cached argument to test_cachestat() to check the pages of a file the test did not write itself, and bump NR_TESTS to the actual number of tests (it was 9 while ten were already run).

The test rework and the changelog were drafted with the help of an AI coding assistant (Claude); the result was reviewed and tested by me.

---
v2:
- Rework of patch 3/3 of v1 on top of Amir's ->f_real() patch; the ->cachestat() file operation from patches 1-2 is replaced by it.
- Add lower file and nested overlayfs cases, and a check_cached argument to test_cachestat().
v1: https://lore.kernel.org/r/[email protected]/

The nested cases pass only with Amir's ovl_f_real() recursion fix
(https://lore.kernel.org/r/CAOQ4uxjHwtZEHu9xMM6zg9niSgNdLXhAu65jA8Ai_SKVg-W2nw@mail.gmail.com/)
plus EXPORT_SYMBOL(__file_real) when overlayfs is a module; on Amir's v1 alone they fail with "Cached: 0".

---
 .../selftests/cachestat/test_cachestat.c      | 172 +++++++++++++++++-
 1 file changed, 163 insertions(+), 9 deletions(-)
tools/testing/selftests/cachestat/test_cachestat.c +163 -9
index 542cd09cb4434..af8fa21c90ca1 100644
@@ -4,27 +4,42 @@
 
 #include <stdio.h>
 #include <stdbool.h>
+#include <stdlib.h>
 #include <linux/kernel.h>
 #include <linux/magic.h>
 #include <linux/mman.h>
 #include <sys/mman.h>
+#include <sys/mount.h>
 #include <sys/shm.h>
+#include <sys/stat.h>
 #include <sys/syscall.h>
 #include <sys/vfs.h>
 #include <unistd.h>
 #include <string.h>
 #include <fcntl.h>
 #include <errno.h>
+#include <limits.h>
 
 #include "kselftest.h"
 
-#define NR_TESTS	9
+#define NR_TESTS	14
 
 static const char * const dev_files[] = {
 	"/dev/zero", "/dev/null", "/dev/urandom",
 	"/proc/version", "/proc"
 };
 
+static const struct {
+	bool lower;
+	bool nested;
+	const char *desc;
+} ovl_tests[] = {
+	{ false, false, "an overlayfs file" },
+	{ false, true, "a nested overlayfs file" },
+	{ true, false, "an overlayfs lower file" },
+	{ true, true, "a nested overlayfs lower file" },
+};
+
 void print_cachestat(struct cachestat *cs)
 {
 	ksft_print_msg(
@@ -117,10 +132,14 @@ static bool is_on_tmpfs(int fd)
  * Open/create the file at filename, (optionally) write random data to it
  * (exactly num_pages), then test the cachestat syscall on this file.
  *
+ * If write_random == true or check_cached == true, check that all num_pages
+ * pages of the file are cached or evicted.
+ *
  * If test_fsync == true, fsync the file, then check the number of dirty
  * pages.
  */
-static int test_cachestat(const char *filename, bool write_random, bool create,
+static int test_cachestat(const char *filename, bool write_random,
+			  bool check_cached, bool create,
 			  bool test_fsync, unsigned long num_pages,
 			  int open_flags, mode_t open_mode)
 {
@@ -161,7 +180,7 @@ static int test_cachestat(const char *filename, bool write_random, bool create,
 	} else {
 		print_cachestat(&cs);
 
-		if (write_random) {
+		if (write_random || check_cached) {
 			if (cs.nr_cache + cs.nr_evicted != num_pages) {
 				ksft_print_msg(
 					"Total number of cached and evicted pages is off.\n");
@@ -294,6 +313,122 @@ bool run_cachestat_test(enum file_type type)
 	return ret;
 }
 
+static int mount_overlayfs(const char *lowerdir, const char *upperdir,
+			   const char *workdir, const char *merged)
+{
+	char opts[4 * PATH_MAX];
+
+	snprintf(opts, sizeof(opts), "lowerdir=%s,upperdir=%s,workdir=%s",
+		 lowerdir, upperdir, workdir);
+	if (mount("overlay", merged, "overlay", 0, opts)) {
+		ksft_print_msg("Unable to mount overlayfs (need root?): %s\n",
+			       strerror(errno));
+		return KSFT_SKIP;
+	}
+
+	return KSFT_PASS;
+}
+
+/*
+ * Write a file to an overlayfs, either directly to its lower dir (if lower)
+ * or through the overlayfs mount, so that its page cache is in the lower or
+ * upper dir. Then open the file read-only, so it is not copied up, through the
+ * overlayfs mount or, if nested, through a second overlayfs mount on top of it
+ * (using the first one's merged dir as its lower dir), and run cachestat on
+ * it. Overlayfs does not cache file data in its own inodes, so cachestat sees
+ * the pages only if it is forwarded one (or two, if nested) stacking levels
+ * down, to the page cache belonging to the real file.
+ */
+static int run_cachestat_overlayfs_test(bool lower, bool nested)
+{
+	char tmpl[] = "/tmp/cachestat_ovl.XXXXXX";
+	char lowerdir[PATH_MAX];
+	char upper1[PATH_MAX], work1[PATH_MAX], merged1[PATH_MAX];
+	char upper2[PATH_MAX], work2[PATH_MAX], merged2[PATH_MAX];
+	char cmd[PATH_MAX + 16], file[PATH_MAX + 16];
+	const char *write_path, *read_path;
+	char *base;
+	int ret;
+
+	base = mkdtemp(tmpl);
+	if (!base) {
+		ksft_print_msg("Unable to create overlayfs base dir: %s\n",
+			       strerror(errno));
+		return KSFT_FAIL;
+	}
+
+	snprintf(lowerdir, sizeof(lowerdir), "%s/lower", base);
+	snprintf(upper1, sizeof(upper1), "%s/upper1", base);
+	snprintf(work1, sizeof(work1), "%s/work1", base);
+	snprintf(merged1, sizeof(merged1), "%s/merged1", base);
+	snprintf(upper2, sizeof(upper2), "%s/upper2", base);
+	snprintf(work2, sizeof(work2), "%s/work2", base);
+	snprintf(merged2, sizeof(merged2), "%s/merged2", base);
+
+	if (mkdir(lowerdir, 0755) || mkdir(upper1, 0755) ||
+	    mkdir(work1, 0755) || mkdir(merged1, 0755) ||
+	    mkdir(upper2, 0755) || mkdir(work2, 0755) ||
+	    mkdir(merged2, 0755)) {
+		ksft_print_msg("Unable to create overlayfs dirs: %s\n",
+			       strerror(errno));
+		ret = KSFT_FAIL;
+		goto cleanup;
+	}
+
+	if (lower)
+		write_path = lowerdir;
+	else
+		write_path = merged1;
+
+	if (nested)
+		read_path = merged2;
+	else
+		read_path = merged1;
+
+	snprintf(file, sizeof(file), "%s/cachestat", write_path);
+
+	/* Write the lower file before mounting overlayfs over it */
+	if (lower) {
+		ret = test_cachestat(file, true, false, false, false, 4,
+				     O_CREAT | O_RDWR, 0600);
+		if (ret != KSFT_PASS)
+			goto cleanup;
+	}
+
+	ret = mount_overlayfs(lowerdir, upper1, work1, merged1);
+	if (ret != KSFT_PASS)
+		goto cleanup;
+
+	if (!lower) {
+		ret = test_cachestat(file, true, false, false, false, 4,
+				     O_CREAT | O_RDWR, 0600);
+		if (ret != KSFT_PASS)
+			goto umount;
+	}
+
+	if (nested) {
+		ret = mount_overlayfs(merged1, upper2, work2, merged2);
+		if (ret != KSFT_PASS)
+			goto umount;
+	}
+
+	snprintf(file, sizeof(file), "%s/cachestat", read_path);
+	/* Look at that page cache through the overlayfs layer(s) */
+	ret = test_cachestat(file, false, true, false, false, 4,
+			     O_RDONLY, 0);
+
+	if (nested)
+		umount(merged2);
+umount:
+	umount(merged1);
+cleanup:
+	/* Best-effort recursive cleanup of the temporary tree. */
+	snprintf(cmd, sizeof(cmd), "rm -rf %s", base);
+	if (system(cmd))
+		ksft_print_msg("Unable to clean up %s\n", base);
+	return ret;
+}
+
 int main(void)
 {
 	int ret;
@@ -317,8 +452,8 @@ int main(void)
 	for (int i = 0; i < 5; i++) {
 		const char *dev_filename = dev_files[i];
 
-		if (test_cachestat(dev_filename, false, false, false,
-			4, O_RDONLY, 0400) == KSFT_PASS)
+		if (test_cachestat(dev_filename, false, false, false, false,
+				   4, O_RDONLY, 0400) == KSFT_PASS)
 			ksft_test_result_pass("cachestat works with %s\n", dev_filename);
 		else {
 			ksft_test_result_fail("cachestat fails with %s\n", dev_filename);
@@ -326,16 +461,16 @@ int main(void)
 		}
 	}
 
-	if (test_cachestat("tmpfilecachestat", true, true,
-		false, 4, O_CREAT | O_RDWR, 0600) == KSFT_PASS)
+	if (test_cachestat("tmpfilecachestat", true, false, true,
+			   false, 4, O_CREAT | O_RDWR, 0600) == KSFT_PASS)
 		ksft_test_result_pass("cachestat works with a normal file\n");
 	else {
 		ksft_test_result_fail("cachestat fails with normal file\n");
 		ret = 1;
 	}
 
-	switch (test_cachestat("tmpfilecachestat", true, true,
-		true, 4, O_CREAT | O_RDWR, 0600)) {
+	switch (test_cachestat("tmpfilecachestat", true, false, true,
+			       true, 4, O_CREAT | O_RDWR, 0600)) {
 	case KSFT_FAIL:
 		ksft_test_result_fail("cachestat fsync fails with normal file\n");
 		ret = KSFT_FAIL;
@@ -361,5 +496,24 @@ int main(void)
 		ksft_test_result_fail("cachestat fails with a mmap file\n");
 		ret = 1;
 	}
+
+	for (int i = 0; i < ARRAY_SIZE(ovl_tests); i++) {
+		const char *desc = ovl_tests[i].desc;
+
+		switch (run_cachestat_overlayfs_test(ovl_tests[i].lower,
+						     ovl_tests[i].nested)) {
+		case KSFT_FAIL:
+			ksft_test_result_fail("cachestat fails with %s\n", desc);
+			ret = 1;
+			break;
+		case KSFT_PASS:
+			ksft_test_result_pass("cachestat works with %s\n", desc);
+			break;
+		case KSFT_SKIP:
+			ksft_test_result_skip("overlayfs not available for %s\n",
+					      desc);
+			break;
+		}
+	}
 	return ret;
 }
Signature · 1 line
2.56.0
author maintainer
#4 lore raw

Re: [PATCH] selftests/cachestat: add overlayfs test cases

Quoted 31 lines · Pavel Tikhomirov

Run cachestat() on overlayfs files in four variants. The file is written either to the lower dir of an overlayfs or through its mount, so its page cache is in the lower or in the upper dir. Then it is opened read-only, so it is not copied up, and checked:

- through the overlayfs mount: the page cache is one stacking level down;
- through a second overlayfs mount, which has the first one's merged dir as its lower dir: the page cache is two stacking levels down.

Overlayfs does not cache file data in its own inodes, so cachestat() sees the pages only if it is forwarded to the real file.

Add a check_cached argument to test_cachestat() to check the pages of a file the test did not write itself, and bump NR_TESTS to the actual number of tests (it was 9 while ten were already run).

The test rework and the changelog were drafted with the help of an AI coding assistant (Claude); the result was reviewed and tested by me.

---
v2:
- Rework of patch 3/3 of v1 on top of Amir's ->f_real() patch; the ->cachestat() file operation from patches 1-2 is replaced by it.
- Add lower file and nested overlayfs cases, and a check_cached argument to test_cachestat().
v1: https://lore.kernel.org/r/[email protected]/

Quoted 4 lines

The nested cases pass only with Amir's ovl_f_real() recursion fix
(https://lore.kernel.org/r/CAOQ4uxjHwtZEHu9xMM6zg9niSgNdLXhAu65jA8Ai_SKVg-W2nw@mail.gmail.com/)
plus EXPORT_SYMBOL(__file_real) when overlayfs is a module; on Amir's v1 alone they fail with "Cached: 0".

Posted v2 with your Tested-by:

https://lore.kernel.org/linux-fsdevel/[email protected]/

Christian,

Please apply Pavel's test along with the file_real() patch.

Thanks,
Amir.

Quoted 260 lines
---
 .../selftests/cachestat/test_cachestat.c      | 172 +++++++++++++++++-
 1 file changed, 163 insertions(+), 9 deletions(-)
tools/testing/selftests/cachestat/test_cachestat.c +163 -10
index 542cd09cb4434..af8fa21c90ca1 100644
@@ -4,27 +4,42 @@
 
 #include <stdio.h>
 #include <stdbool.h>
+#include <stdlib.h>
 #include <linux/kernel.h>
 #include <linux/magic.h>
 #include <linux/mman.h>
 #include <sys/mman.h>
+#include <sys/mount.h>
 #include <sys/shm.h>
+#include <sys/stat.h>
 #include <sys/syscall.h>
 #include <sys/vfs.h>
 #include <unistd.h>
 #include <string.h>
 #include <fcntl.h>
 #include <errno.h>
+#include <limits.h>
 
 #include "kselftest.h"
 
-#define NR_TESTS       9
+#define NR_TESTS       14
 
 static const char * const dev_files[] = {
        "/dev/zero", "/dev/null", "/dev/urandom",
        "/proc/version", "/proc"
 };
 
+static const struct {
+       bool lower;
+       bool nested;
+       const char *desc;
+} ovl_tests[] = {
+       { false, false, "an overlayfs file" },
+       { false, true, "a nested overlayfs file" },
+       { true, false, "an overlayfs lower file" },
+       { true, true, "a nested overlayfs lower file" },
+};
+
 void print_cachestat(struct cachestat *cs)
 {
        ksft_print_msg(
@@ -117,10 +132,14 @@ static bool is_on_tmpfs(int fd)
  * Open/create the file at filename, (optionally) write random data to it
  * (exactly num_pages), then test the cachestat syscall on this file.
  *
+ * If write_random == true or check_cached == true, check that all num_pages
+ * pages of the file are cached or evicted.
+ *
  * If test_fsync == true, fsync the file, then check the number of dirty
  * pages.
  */
-static int test_cachestat(const char *filename, bool write_random, bool create,
+static int test_cachestat(const char *filename, bool write_random,
+                         bool check_cached, bool create,
                          bool test_fsync, unsigned long num_pages,
                          int open_flags, mode_t open_mode)
 {
@@ -161,7 +180,7 @@ static int test_cachestat(const char *filename, bool write_random, bool create,
        } else {
                print_cachestat(&cs);
 
-               if (write_random) {
+               if (write_random || check_cached) {
                        if (cs.nr_cache + cs.nr_evicted != num_pages) {
                                ksft_print_msg(
                                        "Total number of cached and evicted pages is off.\n");
@@ -294,6 +313,122 @@ bool run_cachestat_test(enum file_type type)
        return ret;
 }
 
+static int mount_overlayfs(const char *lowerdir, const char *upperdir,
+                          const char *workdir, const char *merged)
+{
+       char opts[4 * PATH_MAX];
+
+       snprintf(opts, sizeof(opts), "lowerdir=%s,upperdir=%s,workdir=%s",
+                lowerdir, upperdir, workdir);
+       if (mount("overlay", merged, "overlay", 0, opts)) {
+               ksft_print_msg("Unable to mount overlayfs (need root?): %s\n",
+                              strerror(errno));
+               return KSFT_SKIP;
+       }
+
+       return KSFT_PASS;
+}
+
+/*
+ * Write a file to an overlayfs, either directly to its lower dir (if lower)
+ * or through the overlayfs mount, so that its page cache is in the lower or
+ * upper dir. Then open the file read-only, so it is not copied up, through the
+ * overlayfs mount or, if nested, through a second overlayfs mount on top of it
+ * (using the first one's merged dir as its lower dir), and run cachestat on
+ * it. Overlayfs does not cache file data in its own inodes, so cachestat sees
+ * the pages only if it is forwarded one (or two, if nested) stacking levels
+ * down, to the page cache belonging to the real file.
+ */
+static int run_cachestat_overlayfs_test(bool lower, bool nested)
+{
+       char tmpl[] = "/tmp/cachestat_ovl.XXXXXX";
+       char lowerdir[PATH_MAX];
+       char upper1[PATH_MAX], work1[PATH_MAX], merged1[PATH_MAX];
+       char upper2[PATH_MAX], work2[PATH_MAX], merged2[PATH_MAX];
+       char cmd[PATH_MAX + 16], file[PATH_MAX + 16];
+       const char *write_path, *read_path;
+       char *base;
+       int ret;
+
+       base = mkdtemp(tmpl);
+       if (!base) {
+               ksft_print_msg("Unable to create overlayfs base dir: %s\n",
+                              strerror(errno));
+               return KSFT_FAIL;
+       }
+
+       snprintf(lowerdir, sizeof(lowerdir), "%s/lower", base);
+       snprintf(upper1, sizeof(upper1), "%s/upper1", base);
+       snprintf(work1, sizeof(work1), "%s/work1", base);
+       snprintf(merged1, sizeof(merged1), "%s/merged1", base);
+       snprintf(upper2, sizeof(upper2), "%s/upper2", base);
+       snprintf(work2, sizeof(work2), "%s/work2", base);
+       snprintf(merged2, sizeof(merged2), "%s/merged2", base);
+
+       if (mkdir(lowerdir, 0755) || mkdir(upper1, 0755) ||
+           mkdir(work1, 0755) || mkdir(merged1, 0755) ||
+           mkdir(upper2, 0755) || mkdir(work2, 0755) ||
+           mkdir(merged2, 0755)) {
+               ksft_print_msg("Unable to create overlayfs dirs: %s\n",
+                              strerror(errno));
+               ret = KSFT_FAIL;
+               goto cleanup;
+       }
+
+       if (lower)
+               write_path = lowerdir;
+       else
+               write_path = merged1;
+
+       if (nested)
+               read_path = merged2;
+       else
+               read_path = merged1;
+
+       snprintf(file, sizeof(file), "%s/cachestat", write_path);
+
+       /* Write the lower file before mounting overlayfs over it */
+       if (lower) {
+               ret = test_cachestat(file, true, false, false, false, 4,
+                                    O_CREAT | O_RDWR, 0600);
+               if (ret != KSFT_PASS)
+                       goto cleanup;
+       }
+
+       ret = mount_overlayfs(lowerdir, upper1, work1, merged1);
+       if (ret != KSFT_PASS)
+               goto cleanup;
+
+       if (!lower) {
+               ret = test_cachestat(file, true, false, false, false, 4,
+                                    O_CREAT | O_RDWR, 0600);
+               if (ret != KSFT_PASS)
+                       goto umount;
+       }
+
+       if (nested) {
+               ret = mount_overlayfs(merged1, upper2, work2, merged2);
+               if (ret != KSFT_PASS)
+                       goto umount;
+       }
+
+       snprintf(file, sizeof(file), "%s/cachestat", read_path);
+       /* Look at that page cache through the overlayfs layer(s) */
+       ret = test_cachestat(file, false, true, false, false, 4,
+                            O_RDONLY, 0);
+
+       if (nested)
+               umount(merged2);
+umount:
+       umount(merged1);
+cleanup:
+       /* Best-effort recursive cleanup of the temporary tree. */
+       snprintf(cmd, sizeof(cmd), "rm -rf %s", base);
+       if (system(cmd))
+               ksft_print_msg("Unable to clean up %s\n", base);
+       return ret;
+}
+
 int main(void)
 {
        int ret;
@@ -317,8 +452,8 @@ int main(void)
        for (int i = 0; i < 5; i++) {
                const char *dev_filename = dev_files[i];
 
-               if (test_cachestat(dev_filename, false, false, false,
-                       4, O_RDONLY, 0400) == KSFT_PASS)
+               if (test_cachestat(dev_filename, false, false, false, false,
+                                  4, O_RDONLY, 0400) == KSFT_PASS)
                        ksft_test_result_pass("cachestat works with %s\n", dev_filename);
                else {
                        ksft_test_result_fail("cachestat fails with %s\n", dev_filename);
@@ -326,16 +461,16 @@ int main(void)
                }
        }
 
-       if (test_cachestat("tmpfilecachestat", true, true,
-               false, 4, O_CREAT | O_RDWR, 0600) == KSFT_PASS)
+       if (test_cachestat("tmpfilecachestat", true, false, true,
+                          false, 4, O_CREAT | O_RDWR, 0600) == KSFT_PASS)
                ksft_test_result_pass("cachestat works with a normal file\n");
        else {
                ksft_test_result_fail("cachestat fails with normal file\n");
                ret = 1;
        }
 
-       switch (test_cachestat("tmpfilecachestat", true, true,
-               true, 4, O_CREAT | O_RDWR, 0600)) {
+       switch (test_cachestat("tmpfilecachestat", true, false, true,
+                              true, 4, O_CREAT | O_RDWR, 0600)) {
        case KSFT_FAIL:
                ksft_test_result_fail("cachestat fsync fails with normal file\n");
                ret = KSFT_FAIL;
@@ -361,5 +496,24 @@ int main(void)
                ksft_test_result_fail("cachestat fails with a mmap file\n");
                ret = 1;
        }
+
+       for (int i = 0; i < ARRAY_SIZE(ovl_tests); i++) {
+               const char *desc = ovl_tests[i].desc;
+
+               switch (run_cachestat_overlayfs_test(ovl_tests[i].lower,
+                                                    ovl_tests[i].nested)) {
+               case KSFT_FAIL:
+                       ksft_test_result_fail("cachestat fails with %s\n", desc);
+                       ret = 1;
+                       break;
+               case KSFT_PASS:
+                       ksft_test_result_pass("cachestat works with %s\n", desc);
+                       break;
+               case KSFT_SKIP:
+                       ksft_test_result_skip("overlayfs not available for %s\n",
+                                             desc);
+                       break;
+               }
+       }
        return ret;
 }
--

2.56.0

maintainer Applied
#5 lore raw

Re: (subset) [PATCH] selftests/cachestat: add overlayfs test cases

In reply to Pavel Tikhomirov
Quoted 11 lines · Pavel Tikhomirov

Run cachestat() on overlayfs files in four variants. The file is written either to the lower dir of an overlayfs or through its mount, so its page cache is in the lower or in the upper dir. Then it is opened read-only, so it is not copied up, and checked:

- through the overlayfs mount: the page cache is one stacking level down;
- through a second overlayfs mount, which has the first one's merged dir as its lower dir: the page cache is two stacking levels down.

[...]

Applied to the vfs-7.5.misc branch of the vfs/vfs.git tree. Patches in the vfs-7.5.misc branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs-7.5.misc

[1/1] selftests/cachestat: add overlayfs test cases

      https://git.kernel.org/vfs/vfs/c/ab5ec75b9f6d