Skip to contentSkip to conversation

Overlayfs: constant st_ino/d_ino for non-samefs

PATCH Superseded Amir Goldstein 3 participants 26 messages Nov 7, 2017 – Mar 23, 2018 Overlay filesystem

v8 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Amir Goldstein

author maintainer
#0 lore raw

v8 cover Overlayfs: constant st_ino/d_ino for non-samefs

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
[2] https://github.com/amir73il/xfstests/commits/overlayfs-devel

Amir Goldstein (8):
ovl: move include of ovl_entry.h into overlayfs.h
ovl: allocate anonymous devs for lowerdirs
ovl: factor out ovl_map_dev_ino() helper
ovl: return anonymous st_dev for lower inodes
ovl: relax same fs constraint for constant st_ino
ovl: constant st_ino for non-samefs with xino
ovl: consistent d_ino for non-samefs with xino
ovl: add support for 'xino' mount option

Chandan Rajendra (1):
ovl: re-structure overlay lower layers in-memory

 fs/overlayfs/copy_up.c   |   1 -
 fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
 fs/overlayfs/namei.c     |  42 +++++++--------
 fs/overlayfs/overlayfs.h |   9 ++--
 fs/overlayfs/ovl_entry.h |  19 ++++++-
 fs/overlayfs/readdir.c   |  49 +++++++++++++++---
 fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
 fs/overlayfs/util.c      |  39 ++++++++++++--
 8 files changed, 277 insertions(+), 102 deletions(-)
Signature · 1 line
2.7.4
author maintainer
#1 lore raw

PATCH v8 1/9 ovl: move include of ovl_entry.h into overlayfs.h

Most overlayfs c files already explicitly include ovl_entry.h to use overlay entry struct definitions and upcoming changes are going to require even more c files to include this header.

All overlayfs c files include overlayfs.h and overlayfs.h itself refers to some structs defined in ovl_entry.h, so it seems more logic to include ovl_entry.h from overlayfs.h than from c files.

---
 fs/overlayfs/copy_up.c   | 1 -
 fs/overlayfs/inode.c     | 1 -
 fs/overlayfs/namei.c     | 1 -
 fs/overlayfs/overlayfs.h | 1 +
 fs/overlayfs/super.c     | 1 -
 fs/overlayfs/util.c      | 1 -
 6 files changed, 1 insertion(+), 5 deletions(-)
fs/overlayfs/copy_up.c +0 -1
index c441f9387a1b..9311d183b532 100644
@@ -22,7 +22,6 @@
 #include <linux/ratelimit.h>
 #include <linux/exportfs.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 #define OVL_COPY_UP_CHUNK_SIZE (1 << 20)
 
fs/overlayfs/inode.c +0 -1
index 2949448d0c7d..b772e5a2a730 100644
@@ -14,7 +14,6 @@
 #include <linux/posix_acl.h>
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 {
fs/overlayfs/namei.c +0 -1
index a12dc10bf726..505a4b8902fc 100644
@@ -15,7 +15,6 @@
 #include <linux/mount.h>
 #include <linux/exportfs.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 struct ovl_lookup_data {
 	struct qstr name;
fs/overlayfs/overlayfs.h +1 -0
index d53157ccf0d7..1cf3bdd193a4 100644
@@ -9,6 +9,7 @@
 
 #include <linux/kernel.h>
 #include <linux/uuid.h>
+#include "ovl_entry.h"
 
 enum ovl_path_type {
 	__OVL_PATH_UPPER	= (1 << 0),
fs/overlayfs/super.c +0 -1
index f5738e96a052..8702803ba328 100644
@@ -18,7 +18,6 @@
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 MODULE_AUTHOR("Miklos Szeredi <[email protected]>");
 MODULE_DESCRIPTION("Overlay filesystem");
fs/overlayfs/util.c +0 -1
index 51ca8bd16009..9158d17bb320 100644
@@ -17,7 +17,6 @@
 #include <linux/namei.h>
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 int ovl_want_write(struct dentry *dentry)
 {
Signature · 1 line
2.7.4
author maintainer
#2 lore raw

PATCH v8 2/9 ovl: re-structure overlay lower layers in-memory

In reply to Amir Goldstein

From: Chandan Rajendra <[email protected]>

Define new structures to represent overlay instance lower layers and overlay merge dir lower layers to make room for storing more per layer information in-memory.

Instead of keeping the fs instance lower layers in an array of struct vfsmount, keep them in an array of new struct ovl_layer, that has a pointer to struct vfsmount.

Instead of keeping the dentry lower layers in an array of struct path, keep them in an array of new struct ovl_path, that has a pointer to struct dentry and to struct ovl_layer.

Store fs root layer index in struct layer to get from non-root dir layer object to root dir layer layer object.

[amir: split re-structure from anonymous bdev patch
store layer index in layer struct]

---
 fs/overlayfs/namei.c     | 41 ++++++++++++++-----------------
 fs/overlayfs/overlayfs.h |  5 ++--
 fs/overlayfs/ovl_entry.h | 15 ++++++++++--
 fs/overlayfs/readdir.c   |  4 +--
 fs/overlayfs/super.c     | 63 +++++++++++++++++++++++++++---------------------
 fs/overlayfs/util.c      | 14 ++++++++++-
 6 files changed, 84 insertions(+), 58 deletions(-)
fs/overlayfs/namei.c +18 -23
index 505a4b8902fc..7b89f5944e04 100644
@@ -285,16 +285,15 @@ static int ovl_lookup_layer(struct dentry *base, struct ovl_lookup_data *d,
 
 
 static int ovl_check_origin(struct dentry *upperdentry,
-			    struct path *lowerstack, unsigned int numlower,
-			    struct path **stackp, unsigned int *ctrp)
+			    struct ovl_path *lower, unsigned int numlower,
+			    struct ovl_path **stackp, unsigned int *ctrp)
 {
 	struct vfsmount *mnt;
 	struct dentry *origin = NULL;
 	int i;
 
-
 	for (i = 0; i < numlower; i++) {
-		mnt = lowerstack[i].mnt;
+		mnt = lower[i].layer->mnt;
 		origin = ovl_get_origin(upperdentry, mnt);
 		if (IS_ERR(origin))
 			return PTR_ERR(origin);
@@ -308,12 +307,12 @@ static int ovl_check_origin(struct dentry *upperdentry,
 
 	BUG_ON(*ctrp);
 	if (!*stackp)
-		*stackp = kmalloc(sizeof(struct path), GFP_KERNEL);
+		*stackp = kmalloc(sizeof(struct ovl_path), GFP_KERNEL);
 	if (!*stackp) {
 		dput(origin);
 		return -ENOMEM;
 	}
-	**stackp = (struct path) { .dentry = origin, .mnt = mnt };
+	**stackp = (struct ovl_path){.dentry = origin, .layer = lower[i].layer};
 	*ctrp = 1;
 
 	return 0;
@@ -383,13 +382,13 @@ int ovl_verify_origin(struct dentry *dentry, struct vfsmount *mnt,
  * OVL_XATTR_ORIGIN and that origin file handle can be decoded to lower path.
  * Return 0 on match, -ESTALE on mismatch or stale origin, < 0 on error.
  */
-int ovl_verify_index(struct dentry *index, struct path *lowerstack,
+int ovl_verify_index(struct dentry *index, struct ovl_path *lower,
 		     unsigned int numlower)
 {
 	struct ovl_fh *fh = NULL;
 	size_t len;
-	struct path origin = { };
-	struct path *stack = &origin;
+	struct ovl_path origin = { };
+	struct ovl_path *stack = &origin;
 	unsigned int ctr = 0;
 	int err;
 
@@ -428,7 +427,7 @@ int ovl_verify_index(struct dentry *index, struct path *lowerstack,
 	if (err)
 		goto fail;
 
-	err = ovl_check_origin(index, lowerstack, numlower, &stack, &ctr);
+	err = ovl_check_origin(index, lower, numlower, &stack, &ctr);
 	if (!err && !ctr)
 		err = -ESTALE;
 	if (err)
@@ -567,7 +566,8 @@ int ovl_path_next(int idx, struct dentry *dentry, struct path *path)
 		idx++;
 	}
 	BUG_ON(idx > oe->numlower);
-	*path = oe->lowerstack[idx - 1];
+	path->dentry = oe->lowerstack[idx - 1].dentry;
+	path->mnt = oe->lowerstack[idx - 1].layer->mnt;
 
 	return (idx < oe->numlower) ? idx + 1 : -1;
 }
@@ -580,7 +580,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
 	struct ovl_entry *poe = dentry->d_parent->d_fsdata;
 	struct ovl_entry *roe = dentry->d_sb->s_root->d_fsdata;
-	struct path *stack = NULL;
+	struct ovl_path *stack = NULL;
 	struct dentry *upperdir, *upperdentry = NULL;
 	struct dentry *index = NULL;
 	unsigned int ctr = 0;
@@ -645,17 +645,17 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 
 	if (!d.stop && poe->numlower) {
 		err = -ENOMEM;
-		stack = kcalloc(ofs->numlower, sizeof(struct path),
+		stack = kcalloc(ofs->numlower, sizeof(struct ovl_path),
 				GFP_KERNEL);
 		if (!stack)
 			goto out_put_upper;
 	}
 
 	for (i = 0; !d.stop && i < poe->numlower; i++) {
-		struct path lowerpath = poe->lowerstack[i];
+		struct ovl_path lower = poe->lowerstack[i];
 
 		d.last = i == poe->numlower - 1;
-		err = ovl_lookup_layer(lowerpath.dentry, &d, &this);
+		err = ovl_lookup_layer(lower.dentry, &d, &this);
 		if (err)
 			goto out_put;
 
@@ -663,7 +663,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			continue;
 
 		stack[ctr].dentry = this;
-		stack[ctr].mnt = lowerpath.mnt;
+		stack[ctr].layer = lower.layer;
 		ctr++;
 
 		if (d.stop)
@@ -671,13 +671,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 
 		if (d.redirect && d.redirect[0] == '/' && poe != roe) {
 			poe = roe;
-
 			/* Find the current layer on the root dentry */
-			for (i = 0; i < poe->numlower; i++)
-				if (poe->lowerstack[i].mnt == lowerpath.mnt)
-					break;
-			if (WARN_ON(i == poe->numlower))
-				break;
+			i = lower.layer->idx - 1;
 		}
 	}
 
@@ -699,7 +694,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		goto out_put;
 
 	oe->opaque = upperopaque;
-	memcpy(oe->lowerstack, stack, sizeof(struct path) * ctr);
+	memcpy(oe->lowerstack, stack, sizeof(struct ovl_path) * ctr);
 	dentry->d_fsdata = oe;
 
 	if (upperdentry)
fs/overlayfs/overlayfs.h +3 -2
index 1cf3bdd193a4..6d43e0f3d6f5 100644
@@ -203,6 +203,7 @@ void ovl_path_lower(struct dentry *dentry, struct path *path);
 enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path);
 struct dentry *ovl_dentry_upper(struct dentry *dentry);
 struct dentry *ovl_dentry_lower(struct dentry *dentry);
+struct ovl_layer *ovl_layer_lower(struct dentry *dentry);
 struct dentry *ovl_dentry_real(struct dentry *dentry);
 struct dentry *ovl_i_dentry_upper(struct inode *inode);
 struct inode *ovl_inode_upper(struct inode *inode);
@@ -251,7 +252,7 @@ static inline bool ovl_is_impuredir(struct dentry *dentry)
 /* namei.c */
 int ovl_verify_origin(struct dentry *dentry, struct vfsmount *mnt,
 		      struct dentry *origin, bool is_upper, bool set);
-int ovl_verify_index(struct dentry *index, struct path *lowerstack,
+int ovl_verify_index(struct dentry *index, struct ovl_path *lower,
 		     unsigned int numlower);
 int ovl_get_index_name(struct dentry *origin, struct qstr *name);
 int ovl_path_next(int idx, struct dentry *dentry, struct path *path);
@@ -268,7 +269,7 @@ int ovl_check_d_type_supported(struct path *realpath);
 void ovl_workdir_cleanup(struct inode *dir, struct vfsmount *mnt,
 			 struct dentry *dentry, int level);
 int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
-			 struct path *lowerstack, unsigned int numlower);
+			 struct ovl_path *lower, unsigned int numlower);
 
 /* inode.c */
 int ovl_set_nlink_upper(struct dentry *dentry);
fs/overlayfs/ovl_entry.h +13 -2
index 25d9b5adcd42..7573f98ddec1 100644
@@ -17,11 +17,22 @@ struct ovl_config {
 	bool index;
 };
 
+struct ovl_layer {
+	struct vfsmount *mnt;
+	/* Index of this layer in mount (upper == 0) */
+	int idx;
+};
+
+struct ovl_path {
+	struct ovl_layer *layer;
+	struct dentry *dentry;
+};
+
 /* private information held for overlayfs's superblock */
 struct ovl_fs {
 	struct vfsmount *upper_mnt;
 	unsigned numlower;
-	struct vfsmount **lower_mnt;
+	struct ovl_layer *lower_layers;
 	/* workbasedir is the path at workdir= mount option */
 	struct dentry *workbasedir;
 	/* workdir is the 'work' directory under workbasedir */
@@ -52,7 +63,7 @@ struct ovl_entry {
 		struct rcu_head rcu;
 	};
 	unsigned numlower;
-	struct path lowerstack[];
+	struct ovl_path lowerstack[];
 };
 
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
fs/overlayfs/readdir.c +2 -2
index 333de85d6de4..4f2479b6ef91 100644
@@ -1019,7 +1019,7 @@ void ovl_workdir_cleanup(struct inode *dir, struct vfsmount *mnt,
 }
 
 int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
-			 struct path *lowerstack, unsigned int numlower)
+			 struct ovl_path *lower, unsigned int numlower)
 {
 	int err;
 	struct dentry *index = NULL;
@@ -1054,7 +1054,7 @@ int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
 			index = NULL;
 			break;
 		}
-		err = ovl_verify_index(index, lowerstack, numlower);
+		err = ovl_verify_index(index, lower, numlower);
 		/* Cleanup stale and orphan index entries */
 		if (err && (err == -ESTALE || err == -ENOENT))
 			err = ovl_cleanup(dir, index);
fs/overlayfs/super.c +35 -28

136 lines not on this page · raw message

fs/overlayfs/util.c +13 -1
index 9158d17bb320..3c3b026c189b 100644
@@ -124,7 +124,12 @@ void ovl_path_lower(struct dentry *dentry, struct path *path)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
 
-	*path = oe->numlower ? oe->lowerstack[0] : (struct path) { };
+	if (oe->numlower) {
+		path->mnt = oe->lowerstack[0].layer->mnt;
+		path->dentry = oe->lowerstack[0].dentry;
+	} else {
+		*path = (struct path) { };
+	}
 }
 
 enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path)
@@ -151,6 +156,13 @@ struct dentry *ovl_dentry_lower(struct dentry *dentry)
 	return oe->numlower ? oe->lowerstack[0].dentry : NULL;
 }
 
+struct ovl_layer *ovl_layer_lower(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	return oe->numlower ? oe->lowerstack[0].layer : NULL;
+}
+
 struct dentry *ovl_dentry_real(struct dentry *dentry)
 {
 	return ovl_dentry_upper(dentry) ?: ovl_dentry_lower(dentry);
Signature · 1 line
2.7.4
author maintainer
#3 lore raw

PATCH v8 3/9 ovl: allocate anonymous devs for lowerdirs

In reply to Amir Goldstein

Generate unique values of st_dev per lower layer for non-samefs overlay mount. The unique values are obtained by allocating anonymous bdevs for each of the lowerdirs in the overlayfs instance.

The anonymous bdev is going to be returned by stat(2) for lowerdir non-dir entries in non-samefs case.

[amir: split from ovl_getattr() and re-structure patches]

---
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/super.c     | 18 ++++++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)
fs/overlayfs/ovl_entry.h +1 -0
index 7573f98ddec1..63ef5ae9275b 100644
@@ -21,6 +21,7 @@ struct ovl_layer {
 	struct vfsmount *mnt;
 	/* Index of this layer in mount (upper == 0) */
 	int idx;
+	dev_t pseudo_dev;
 };
 
 struct ovl_path {
fs/overlayfs/super.c +16 -2
index ceca3a9ac27f..0b8fe8dca187 100644
@@ -219,8 +219,10 @@ static void ovl_put_super(struct super_block *sb)
 	if (ufs->upper_mnt && ufs->upperdir_locked)
 		ovl_inuse_unlock(ufs->upper_mnt->mnt_root);
 	mntput(ufs->upper_mnt);
-	for (i = 0; i < ufs->numlower; i++)
+	for (i = 0; i < ufs->numlower; i++) {
 		mntput(ufs->lower_layers[i].mnt);
+		free_anon_bdev(ufs->lower_layers[i].pseudo_dev);
+	}
 	kfree(ufs->lower_layers);
 
 	kfree(ufs->config.lowerdir);
@@ -1032,11 +1034,19 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 		goto out_put_workdir;
 	for (i = 0; i < numlower; i++) {
 		struct vfsmount *mnt;
+		dev_t dev;
+
+		err = get_anon_bdev(&dev);
+		if (err) {
+			pr_err("overlayfs: failed to get anonymous bdev for lowerpath\n");
+			goto out_put_lower_layers;
+		}
 
 		mnt = clone_private_mount(&stack[i]);
 		err = PTR_ERR(mnt);
 		if (IS_ERR(mnt)) {
 			pr_err("overlayfs: failed to clone lowerpath\n");
+			free_anon_bdev(dev);
 			goto out_put_lower_layers;
 		}
 		/*
@@ -1047,6 +1057,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 
 		ufs->lower_layers[ufs->numlower].mnt = mnt;
 		ufs->lower_layers[ufs->numlower].idx = i + 1;
+		ufs->lower_layers[ufs->numlower].pseudo_dev = dev;
 		ufs->numlower++;
 
 		/* Check if all lower layers are on same sb */
@@ -1161,8 +1172,11 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 out_free_oe:
 	kfree(oe);
 out_put_lower_layers:
-	for (i = 0; i < ufs->numlower; i++)
+	for (i = 0; i < ufs->numlower; i++) {
+		if (ufs->lower_layers[i].mnt)
+			free_anon_bdev(ufs->lower_layers[i].pseudo_dev);
 		mntput(ufs->lower_layers[i].mnt);
+	}
 	kfree(ufs->lower_layers);
 out_put_workdir:
 	dput(ufs->workdir);
Signature · 1 line
2.7.4
author maintainer
#4 lore raw

PATCH v8 4/9 ovl: factor out ovl_map_dev_ino() helper

In reply to Amir Goldstein

A helper for ovl_getattr() to map the values of st_dev and st_ino according to constant st_ino rules.

---
 fs/overlayfs/inode.c | 52 ++++++++++++++++++++++++++++++++++------------------
 1 file changed, 34 insertions(+), 18 deletions(-)
fs/overlayfs/inode.c +34 -18
index b772e5a2a730..6c54ecff56ee 100644
@@ -58,6 +58,34 @@ int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 	return err;
 }
 
+static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
+{
+	bool samefs = ovl_same_sb(dentry->d_sb);
+
+	if (samefs) {
+		/*
+		 * When all layers are on the same fs, all real inode
+		 * number are unique, so we use the overlay st_dev,
+		 * which is friendly to du -x.
+		 */
+		stat->dev = dentry->d_sb->s_dev;
+	} else if (S_ISDIR(dentry->d_inode->i_mode)) {
+		/*
+		 * Always use the overlay st_dev for directories, so 'find
+		 * -xdev' will scan the entire overlay mount and won't cross the
+		 * overlay mount boundaries.
+		 *
+		 * If not all layers are on the same fs the pair {real st_ino;
+		 * overlay st_dev} is not unique, so use the non persistent
+		 * overlay st_ino for directories.
+		 */
+		stat->dev = dentry->d_sb->s_dev;
+		stat->ino = dentry->d_inode->i_ino;
+	}
+
+	return 0;
+}
+
 int ovl_getattr(const struct path *path, struct kstat *stat,
 		u32 request_mask, unsigned int flags)
 {
@@ -75,13 +103,10 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 		goto out;
 
 	/*
-	 * When all layers are on the same fs, all real inode number are
-	 * unique, so we use the overlay st_dev, which is friendly to du -x.
-	 *
-	 * We also use st_ino of the copy up origin, if we know it.
+	 * When all layers are on the same fs, we use st_ino of copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
 	 *
-	 * If filesystem supports NFS export ops, this also guaranties
+	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
 	if (ovl_same_sb(dentry->d_sb)) {
@@ -108,21 +133,12 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 			    ovl_test_flag(OVL_INDEX, d_inode(dentry)))
 				stat->ino = lowerstat.ino;
 		}
-		stat->dev = dentry->d_sb->s_dev;
-	} else if (is_dir) {
-		/*
-		 * If not all layers are on the same fs the pair {real st_ino;
-		 * overlay st_dev} is not unique, so use the non persistent
-		 * overlay st_ino.
-		 *
-		 * Always use the overlay st_dev for directories, so 'find
-		 * -xdev' will scan the entire overlay mount and won't cross the
-		 * overlay mount boundaries.
-		 */
-		stat->dev = dentry->d_sb->s_dev;
-		stat->ino = dentry->d_inode->i_ino;
 	}
 
+	err = ovl_map_dev_ino(dentry, stat);
+	if (err)
+		goto out;
+
 	/*
 	 * It's probably not worth it to count subdirs to get the
 	 * correct link count.  nlink=1 seems to pacify 'find' and
Signature · 1 line
2.7.4
author maintainer
#5 lore raw

PATCH v8 5/9 ovl: return anonymous st_dev for lower inodes

In reply to Amir Goldstein

For non-samefs setup, to make sure that st_dev/st_ino pair is unique across the system, we return a unique anonymous st_dev for stat(2) of lower layer inode.

A following patch is going to fix constant st_dev/st_ino across copy up by returning origin st_dev/st_ino for copied up objects.

If the st_dev/st_ino for copied up object would have been the same as that of the real underlying lower file, running diff on underlying lower file and overlay copied up file would result in diff reporting that the 2 files are equal when in fact, they may have different content.

[amir: split from allocate anonymous bdev patch
simplify get pseudo dev code]

---
 fs/overlayfs/inode.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
fs/overlayfs/inode.c +10 -0
index 6c54ecff56ee..29b0adc77ebe 100644
@@ -15,6 +15,7 @@
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
 
+
 int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 {
 	int err;
@@ -60,6 +61,7 @@ int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 
 static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 {
+	struct ovl_layer *lower_layer = ovl_layer_lower(dentry);
 	bool samefs = ovl_same_sb(dentry->d_sb);
 
 	if (samefs) {
@@ -81,6 +83,14 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 		 */
 		stat->dev = dentry->d_sb->s_dev;
 		stat->ino = dentry->d_inode->i_ino;
+	} else if (lower_layer) {
+		/*
+		 * For non-samefs setup, if we cannot map all layers st_ino
+		 * to a unified address space, we need to make sure that st_dev
+		 * is unique per layer. Upper layer uses real st_dev and lower
+		 * layers use the unique anonymous bdev.
+		 */
+		stat->dev = lower_layer->pseudo_dev;
 	}
 
 	return 0;
Signature · 1 line
2.7.4
author maintainer
#6 lore raw

PATCH v8 6/9 ovl: relax same fs constraint for constant st_ino

In reply to Amir Goldstein

For the case of all layers not on the same fs, return the copy up origin inode st_dev/st_ino for non-dir from stat(2).

This guaranties constant st_dev/st_ino for non-dir across copy up. Like the same fs case, st_ino of non-dir is also persistent.

If the st_dev/st_ino for copied up object would have been the same as that of the real underlying lower file, running diff on underlying lower file and overlay copied up file would result in diff reporting that the two files are equal when in fact, they may have different content.

Therefore, unlike the same fs case, st_dev is not uniform across all overlay object, which is less friednly to du -x.

---
 fs/overlayfs/inode.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
fs/overlayfs/inode.c +6 -3
index 29b0adc77ebe..80a4b61a1149 100644
@@ -104,6 +104,7 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 	struct path realpath;
 	const struct cred *old_cred;
 	bool is_dir = S_ISDIR(dentry->d_inode->i_mode);
+	bool samefs = ovl_same_sb(dentry->d_sb);
 	int err;
 
 	type = ovl_path_real(dentry, &realpath);
@@ -113,13 +114,13 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 		goto out;
 
 	/*
-	 * When all layers are on the same fs, we use st_ino of copy up origin.
+	 * For non-dir or same fs, we use st_ino of the copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
 	 *
 	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
-	if (ovl_same_sb(dentry->d_sb)) {
+	if (!is_dir || samefs) {
 		if (OVL_TYPE_ORIGIN(type)) {
 			struct kstat lowerstat;
 			u32 lowermask = STATX_INO | (!is_dir ? STATX_NLINK : 0);
@@ -130,7 +131,9 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 			if (err)
 				goto out;
 
-			WARN_ON_ONCE(stat->dev != lowerstat.dev);
+			if (samefs)
+				WARN_ON_ONCE(stat->dev != lowerstat.dev);
+
 			/*
 			 * Lower hardlinks may be broken on copy up to different
 			 * upper files, so we cannot use the lower origin st_ino
Signature · 1 line
2.7.4
author maintainer
#7 lore raw

PATCH v8 7/9 ovl: constant st_ino for non-samefs with xino

In reply to Amir Goldstein

When overlay layers are not all on the same fs, but all inode numbers of underlying fs are not using 'xino' high bits, use the high bits to partition the overlay st_ino address space. The high bits hold the layer index (upper is 0). This way overlay inode numbers are unique and all inodes use overlay st_dev. Inode numbers are also persistent for a given layer configuration.

Currently, our only indication for available high ino bits is from a filesystem that supports file handles and uses the default encode_fh() operation, which encodes a 32bit inode number.

---
 fs/overlayfs/inode.c     | 23 ++++++++++++++++++++++-
 fs/overlayfs/overlayfs.h |  3 ++-
 fs/overlayfs/ovl_entry.h |  2 ++
 fs/overlayfs/super.c     | 30 +++++++++++++++++++++++-------
 fs/overlayfs/util.c      | 24 +++++++++++++++++++++---
 5 files changed, 70 insertions(+), 12 deletions(-)
fs/overlayfs/inode.c +22 -1
index 80a4b61a1149..521c4d21eb7b 100644
@@ -63,6 +63,7 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 {
 	struct ovl_layer *lower_layer = ovl_layer_lower(dentry);
 	bool samefs = ovl_same_sb(dentry->d_sb);
+	int xinobits = ovl_xino_bits(dentry->d_sb);
 
 	if (samefs) {
 		/*
@@ -71,6 +72,24 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 		 * which is friendly to du -x.
 		 */
 		stat->dev = dentry->d_sb->s_dev;
+	} else if (xinobits) {
+		/*
+		 * All inode numbers of underlying fs should not be using the
+		 * high xinobits, so we use high xinobits to partition the
+		 * overlay st_ino address space. The high bits holds the layer
+		 * index (upper is 0). This way overlay inode numbers are unique
+		 * and all inodes use overlay st_dev. Inode numbers are also
+		 * persistent for a given layer configuration.
+		 */
+		if (stat->ino >> (64 - xinobits)) {
+			pr_warn_ratelimited("overlayfs: inode number too big (%pd2, ino=%llu, xinobits=%d)\n",
+					    dentry, stat->ino, xinobits);
+			return -EOVERFLOW;
+		}
+
+		if (lower_layer)
+			stat->ino |= ((u64)lower_layer->idx) << (64 - xinobits);
+		stat->dev = dentry->d_sb->s_dev;
 	} else if (S_ISDIR(dentry->d_inode->i_mode)) {
 		/*
 		 * Always use the overlay st_dev for directories, so 'find
@@ -116,11 +135,13 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 	/*
 	 * For non-dir or same fs, we use st_ino of the copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
+	 * With xino feature and non-samefs, we use st_ino of the copy up
+	 * origin masked with high bits that represent the layer id.
 	 *
 	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
-	if (!is_dir || samefs) {
+	if (!is_dir || samefs || ovl_xino_bits(dentry->d_sb)) {
 		if (OVL_TYPE_ORIGIN(type)) {
 			struct kstat lowerstat;
 			u32 lowermask = STATX_INO | (!is_dir ? STATX_NLINK : 0);
fs/overlayfs/overlayfs.h +2 -1
index 6d43e0f3d6f5..29569bf1cc6e 100644
@@ -192,7 +192,8 @@ void ovl_drop_write(struct dentry *dentry);
 struct dentry *ovl_workdir(struct dentry *dentry);
 const struct cred *ovl_override_creds(struct super_block *sb);
 struct super_block *ovl_same_sb(struct super_block *sb);
-bool ovl_can_decode_fh(struct super_block *sb);
+int ovl_xino_bits(struct super_block *sb);
+int ovl_can_decode_fh(struct super_block *sb);
 struct dentry *ovl_indexdir(struct super_block *sb);
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
 bool ovl_dentry_remote(struct dentry *dentry);
fs/overlayfs/ovl_entry.h +2 -0
index 63ef5ae9275b..226cf94838a3 100644
@@ -52,6 +52,8 @@ struct ovl_fs {
 	/* Did we take the inuse lock? */
 	bool upperdir_locked;
 	bool workdir_locked;
+	/* Inode numbers in all layers do not use the high xino_bits */
+	int xino_bits;
 };
 
 /* private information held for every overlayfs dentry */
fs/overlayfs/super.c +23 -7
index 0b8fe8dca187..ddedfca9305c 100644
@@ -17,6 +17,7 @@
 #include <linux/statfs.h>
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
+#include <linux/exportfs.h>
 #include "overlayfs.h"
 
 MODULE_AUTHOR("Miklos Szeredi <[email protected]>");
@@ -629,6 +630,7 @@ static int ovl_check_namelen(struct path *path, struct ovl_fs *ofs,
 static int ovl_lower_dir(const char *name, struct path *path,
 			 struct ovl_fs *ofs, int *stack_depth, bool *remote)
 {
+	int fh_type;
 	int err;
 
 	err = ovl_mount_dir_noesc(name, path);
@@ -648,10 +650,14 @@ static int ovl_lower_dir(const char *name, struct path *path,
 	 * The inodes index feature needs to encode and decode file
 	 * handles, so it requires that all layers support them.
 	 */
-	if (ofs->config.index && !ovl_can_decode_fh(path->dentry->d_sb)) {
+	fh_type = ovl_can_decode_fh(path->dentry->d_sb);
+	if (ofs->config.index && !fh_type) {
 		ofs->config.index = false;
 		pr_warn("overlayfs: fs on '%s' does not support file handles, falling back to index=off.\n", name);
 	}
+	/* Check if lower fs has 32bit inode numbers */
+	if (fh_type != FILEID_INO32_GEN)
+		ofs->xino_bits = 0;
 
 	return 0;
 
@@ -842,6 +848,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	unsigned int i;
 	bool remote = false;
 	struct cred *cred;
+	int fh_type;
 	int err;
 
 	err = -ENOMEM;
@@ -961,6 +968,8 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 		goto out_put_lowerpath;
 	}
 
+	/* Assume underlaying fs uses 32bit inodes unless proven otherwise */
+	ufs->xino_bits = 32;
 	if (ufs->config.upperdir) {
 		ufs->upper_mnt = clone_private_mount(&upperpath);
 		err = PTR_ERR(ufs->upper_mnt);
@@ -1017,14 +1026,17 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 			} else {
 				vfs_removexattr(ufs->workdir, OVL_XATTR_OPAQUE);
 			}
+		}
 
-			/* Check if upper/work fs supports file handles */
-			if (ufs->config.index &&
-			    !ovl_can_decode_fh(ufs->workdir->d_sb)) {
-				ufs->config.index = false;
-				pr_warn("overlayfs: upper fs does not support file handles, falling back to index=off.\n");
-			}
+		/* Check if upper fs supports file handles */
+		fh_type = ovl_can_decode_fh(ufs->upper_mnt->mnt_sb);
+		if (ufs->config.index && !fh_type) {
+			ufs->config.index = false;
+			pr_warn("overlayfs: upper fs does not support file handles, falling back to index=off.\n");
 		}
+		/* Check if upper fs has 32bit inode numbers */
+		if (fh_type != FILEID_INO32_GEN)
+			ufs->xino_bits = 0;
 	}
 
 	err = -ENOMEM;
@@ -1073,6 +1085,10 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb != ufs->same_sb)
 		ufs->same_sb = NULL;
 
+	/* When all layers on same fs, overlay can use real inode numbers */
+	if (ufs->same_sb)
+		ufs->xino_bits = 0;
+
 	err = -ENOMEM;
 	oe = ovl_alloc_entry(numlower);
 	if (!oe)
fs/overlayfs/util.c +21 -3
index 3c3b026c189b..255ae337fc46 100644
@@ -50,10 +50,28 @@ struct super_block *ovl_same_sb(struct super_block *sb)
 	return ofs->same_sb;
 }
 
-bool ovl_can_decode_fh(struct super_block *sb)
+int ovl_xino_bits(struct super_block *sb)
 {
-	return (sb->s_export_op && sb->s_export_op->fh_to_dentry &&
-		!uuid_is_null(&sb->s_uuid));
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	return ofs->xino_bits;
+}
+
+/*
+ * Check if underlying fs supports file handles and try to determine encoding
+ * type, in order to deduce maximum inode number used by fs.
+ *
+ * Return 0 if file handles are not supported.
+ * Return 1 (FILEID_INO32_GEN) if fs uses the default 32bit inode encoding.
+ * Return -1 if fs uses a non default encoding with unknown inode size.
+ */
+int ovl_can_decode_fh(struct super_block *sb)
+{
+	if (!sb->s_export_op || !sb->s_export_op->fh_to_dentry ||
+	    uuid_is_null(&sb->s_uuid))
+		return 0;
+
+	return sb->s_export_op->encode_fh ? -1 : FILEID_INO32_GEN;
 }
 
 struct dentry *ovl_indexdir(struct super_block *sb)
Signature · 1 line
2.7.4
author maintainer
#8 lore raw

PATCH v8 8/9 ovl: consistent d_ino for non-samefs with xino

In reply to Amir Goldstein

When overlay layers are not all on the same fs, but all inode numbers of underlying fs do not use the high 'xino' bits, overlay st_ino values are constant and persistent.

In that case, relax non-samefs constraint for consistent d_ino and always iterate non-merge dir using ovl_fill_real() actor so we can remap lower inode numbers to unique layer range.

---
 fs/overlayfs/readdir.c | 45 +++++++++++++++++++++++++++++++++++++++------
 1 file changed, 39 insertions(+), 6 deletions(-)
fs/overlayfs/readdir.c +39 -6
index 4f2479b6ef91..92a6dcdbdaf7 100644
@@ -120,6 +120,10 @@ static bool ovl_calc_d_ino(struct ovl_readdir_data *rdd,
 	if (!rdd->dentry)
 		return false;
 
+	/* Always recalc d_ino when remapping lower inode numbers */
+	if (ovl_xino_bits(rdd->dentry->d_sb))
+		return true;
+
 	/* Always recalc d_ino for parent */
 	if (strcmp(p->name, "..") == 0)
 		return true;
@@ -440,6 +444,19 @@ static struct ovl_dir_cache *ovl_cache_get(struct dentry *dentry)
 	return cache;
 }
 
+/* Map inode number to lower layer unique range */
+static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int idx,
+			       const char *name, int namelen)
+{
+	if (ino >> (64 - xinobits)) {
+		pr_warn_ratelimited("overlayfs: d_ino too big (%.*s, ino=%llu, xinobits=%d)\n",
+				    namelen, name, ino, xinobits);
+		return ino;
+	}
+
+	return ino | ((u64)idx) << (64 - xinobits);
+}
+
 /*
  * Set d_ino for upper entries. Non-upper entries should always report
  * the uppermost real inode ino and should not call this function.
@@ -457,9 +474,10 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p)
 	struct dentry *this = NULL;
 	enum ovl_path_type type;
 	u64 ino = p->real_ino;
+	int xinobits = ovl_xino_bits(dir->d_sb);
 	int err = 0;
 
-	if (!ovl_same_sb(dir->d_sb))
+	if (!ovl_same_sb(dir->d_sb) && !xinobits)
 		goto out;
 
 	if (p->name[0] == '.') {
@@ -496,6 +514,10 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p)
 
 		WARN_ON_ONCE(dir->d_sb->s_dev != stat.dev);
 		ino = stat.ino;
+	} else if (xinobits && !OVL_TYPE_UPPER(type)) {
+		ino = ovl_remap_lower_ino(ino, xinobits,
+					  ovl_layer_lower(this)->idx,
+					  p->name, p->len);
 	}
 
 out:
@@ -616,6 +638,8 @@ struct ovl_readdir_translate {
 	struct ovl_dir_cache *cache;
 	struct dir_context ctx;
 	u64 parent_ino;
+	int layer_idx;
+	int xinobits;
 };
 
 static int ovl_fill_real(struct dir_context *ctx, const char *name,
@@ -626,14 +650,17 @@ static int ovl_fill_real(struct dir_context *ctx, const char *name,
 		container_of(ctx, struct ovl_readdir_translate, ctx);
 	struct dir_context *orig_ctx = rdt->orig_ctx;
 
-	if (rdt->parent_ino && strcmp(name, "..") == 0)
+	if (rdt->parent_ino && strcmp(name, "..") == 0) {
 		ino = rdt->parent_ino;
-	else if (rdt->cache) {
+	} else if (rdt->cache) {
 		struct ovl_cache_entry *p;
 
 		p = ovl_cache_entry_find(&rdt->cache->root, name, namelen);
 		if (p)
 			ino = p->ino;
+	} else if (rdt->xinobits) {
+		ino = ovl_remap_lower_ino(ino, rdt->xinobits, rdt->layer_idx,
+					  name, namelen);
 	}
 
 	return orig_ctx->actor(orig_ctx, name, namelen, offset, ino, d_type);
@@ -644,11 +671,16 @@ static int ovl_iterate_real(struct file *file, struct dir_context *ctx)
 	int err;
 	struct ovl_dir_file *od = file->private_data;
 	struct dentry *dir = file->f_path.dentry;
+	struct ovl_layer *lower_layer = ovl_layer_lower(dir);
 	struct ovl_readdir_translate rdt = {
 		.ctx.actor = ovl_fill_real,
 		.orig_ctx = ctx,
+		.xinobits = ovl_xino_bits(dir->d_sb),
 	};
 
+	if (rdt.xinobits && lower_layer)
+		rdt.layer_idx = lower_layer->idx;
+
 	if (OVL_TYPE_MERGE(ovl_path_type(dir->d_parent))) {
 		struct kstat stat;
 		struct path statpath = file->f_path;
@@ -688,9 +720,10 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)
 		 * dir is impure then need to adjust d_ino for copied up
 		 * entries.
 		 */
-		if (ovl_same_sb(dentry->d_sb) &&
-		    (ovl_test_flag(OVL_IMPURE, d_inode(dentry)) ||
-		     OVL_TYPE_MERGE(ovl_path_type(dentry->d_parent)))) {
+		if (ovl_xino_bits(dentry->d_sb) ||
+		    (ovl_same_sb(dentry->d_sb) &&
+		     (ovl_test_flag(OVL_IMPURE, d_inode(dentry)) ||
+		      OVL_TYPE_MERGE(ovl_path_type(dentry->d_parent))))) {
 			return ovl_iterate_real(file, ctx);
 		}
 		return iterate_dir(od->realfile, ctx);
Signature · 1 line
2.7.4
author maintainer
#9 lore raw

PATCH v8 9/9 ovl: add support for 'xino' mount option

In reply to Amir Goldstein

With mount option 'xino', mounter declares that there are enough free high bits in underlying fs to hold the layer id. If overlayfs does encounter underlying inodes using the high xino bits reserved for layer id, stat(2) will return -EOVERLFLOW.

The mount option name 'xino' goes after a similar meaning mount option of aufs, but in overlayfs case, the mapping is stateless.

An example for a use case of 'xino' is when upper/lower is on an xfs filesystem mounted with mount option 'inode32' or on an xfs filesystem that is not huge. If xfs filesystem AG count is limited by device size to under 2^(31-x), then it is safe to use x high blocks for the overlay layer id.

---
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/super.c     | 19 ++++++++++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)
fs/overlayfs/ovl_entry.h +1 -0
index 226cf94838a3..13140028f2bb 100644
@@ -15,6 +15,7 @@ struct ovl_config {
 	bool default_permissions;
 	bool redirect_dir;
 	bool index;
+	bool xino;
 };
 
 struct ovl_layer {
fs/overlayfs/super.c +18 -1
index ddedfca9305c..3f41ddf6bd0d 100644
@@ -308,6 +308,8 @@ static int ovl_show_options(struct seq_file *m, struct dentry *dentry)
 	if (ufs->config.index != ovl_index_def)
 		seq_printf(m, ",index=%s",
 			   ufs->config.index ? "on" : "off");
+	if (ufs->config.xino)
+		seq_puts(m, ",xino");
 	return 0;
 }
 
@@ -341,6 +343,7 @@ enum {
 	OPT_REDIRECT_DIR_OFF,
 	OPT_INDEX_ON,
 	OPT_INDEX_OFF,
+	OPT_XINO,
 	OPT_ERR,
 };
 
@@ -353,6 +356,7 @@ static const match_table_t ovl_tokens = {
 	{OPT_REDIRECT_DIR_OFF,		"redirect_dir=off"},
 	{OPT_INDEX_ON,			"index=on"},
 	{OPT_INDEX_OFF,			"index=off"},
+	{OPT_XINO,			"xino"},
 	{OPT_ERR,			NULL}
 };
 
@@ -433,6 +437,10 @@ static int ovl_parse_opt(char *opt, struct ovl_config *config)
 			config->index = false;
 			break;
 
+		case OPT_XINO:
+			config->xino = true;
+			break;
+
 		default:
 			pr_err("overlayfs: unrecognized mount option \"%s\" or missing value\n", p);
 			return -EINVAL;
@@ -1085,9 +1093,18 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb != ufs->same_sb)
 		ufs->same_sb = NULL;
 
-	/* When all layers on same fs, overlay can use real inode numbers */
+	/*
+	 * When all layers on same fs, overlay can use real inode numbers.
+	 * With mount option 'xino', mounter declares that there are enough
+	 * free high bits in underlying fs to hold the layer id.
+	 * If overlayfs does encounter underlying inodes using the high xino
+	 * bits reserved for layer id, stat(2) will return -EOVERLFLOW.
+	 */
+	BUILD_BUG_ON(ilog2(OVL_MAX_STACK) > 31);
 	if (ufs->same_sb)
 		ufs->xino_bits = 0;
+	else if (ufs->config.xino && !ufs->xino_bits)
+		ufs->xino_bits = ilog2(ufs->numlower) + 1;
 
 	err = -ENOMEM;
 	oe = ovl_alloc_entry(numlower);
Signature · 1 line
2.7.4
#10 lore raw
In reply to Amir Goldstein

Hi Miklos,

These patches apply on top of which tree/branch. I am trying linus's latest tree and these fail to apply.

Vivek

Quoted 58 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
[2] https://github.com/amir73il/xfstests/commits/overlayfs-devel

Amir Goldstein (8):
ovl: move include of ovl_entry.h into overlayfs.h
ovl: allocate anonymous devs for lowerdirs
ovl: factor out ovl_map_dev_ino() helper
ovl: return anonymous st_dev for lower inodes
ovl: relax same fs constraint for constant st_ino
ovl: constant st_ino for non-samefs with xino
ovl: consistent d_ino for non-samefs with xino
ovl: add support for 'xino' mount option

Chandan Rajendra (1):
ovl: re-structure overlay lower layers in-memory

 fs/overlayfs/copy_up.c   |   1 -
 fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
 fs/overlayfs/namei.c     |  42 +++++++--------
 fs/overlayfs/overlayfs.h |   9 ++--
 fs/overlayfs/ovl_entry.h |  19 ++++++-
 fs/overlayfs/readdir.c   |  49 +++++++++++++++---
 fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
 fs/overlayfs/util.c      |  39 ++++++++++++--
 8 files changed, 277 insertions(+), 102 deletions(-)
Signature · 1 line
2.7.4
author maintainer
#11 lore raw
Quoted 5 lines · Vivek Goyal

Hi Miklos,

These patches apply on top of which tree/branch. I am trying linus's latest tree and these fail to apply.

Hi Vivek,

Sorry, I forgot to mention, those patches are based on top of the series "Overlayfs fixes for exposed whiteouts" posted on Nov 1 and one lockdep fix patch posted on Oct 27.

You can fetch them all from my tree. I write the branch below [1] just forgot to refer to it in the text...

FYI, there are 3 more patches on my overlayfs-devel branch [3], which I intend to post for v4.15. They are complementary patches to the "fixes for exposed whiteouts" series.

Amir.

Quoted 29 lines · Amir Goldstein
Quoted 27 lines

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 5 lines
Quoted 5 lines

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
[2] https://github.com/amir73il/xfstests/commits/overlayfs-devel

[3] https://github.com/amir73il/linux/commits/overlayfs-devel

Quoted 26 lines
Quoted 26 lines

Amir Goldstein (8):
ovl: move include of ovl_entry.h into overlayfs.h
ovl: allocate anonymous devs for lowerdirs
ovl: factor out ovl_map_dev_ino() helper
ovl: return anonymous st_dev for lower inodes
ovl: relax same fs constraint for constant st_ino
ovl: constant st_ino for non-samefs with xino
ovl: consistent d_ino for non-samefs with xino
ovl: add support for 'xino' mount option

Chandan Rajendra (1):
ovl: re-structure overlay lower layers in-memory

 fs/overlayfs/copy_up.c   |   1 -
 fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
 fs/overlayfs/namei.c     |  42 +++++++--------
 fs/overlayfs/overlayfs.h |   9 ++--
 fs/overlayfs/ovl_entry.h |  19 ++++++-
 fs/overlayfs/readdir.c   |  49 +++++++++++++++---
 fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
 fs/overlayfs/util.c      |  39 ++++++++++++--
 8 files changed, 277 insertions(+), 102 deletions(-)

--
2.7.4

maintainer
#12 lore raw
In reply to Amir Goldstein
Quoted 20 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Quoted 6 lines

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now pushed out to overlayfs-next, hopefully not invalidating too much of your work.

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

I'll try and look at the other series posted by you and Vivek to see what else would fit in to 4.15.

Thanks,
Miklos

author maintainer
#13 lore raw
Quoted 28 lines · Miklos Szeredi
Quoted 20 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Quoted 6 lines

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Quoted 2 lines

pushed out to overlayfs-next, hopefully not invalidating too much of your work.

Not a problem. most of the work on v8 was on the last patches.
I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.

Quoted 4 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Sure. your call.

Quoted 2 lines

I'll try and look at the other series posted by you and Vivek to see what else would fit in to 4.15.

OK. I need to address a small comment by Zhangyi on restoring origin fh. Will push a fix today.

Thanks,
Amir.

author maintainer
#14 lore raw
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 8 lines
Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)

[unmount of xfs vdf]

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

Quoted 11 lines
Quoted 2 lines

pushed out to overlayfs-next, hopefully not invalidating too much of your work.

Not a problem. most of the work on v8 was on the last patches.
I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.

Quoted 4 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Sure. your call.

Quoted 2 lines

I'll try and look at the other series posted by you and Vivek to see what else would fit in to 4.15.

OK. I need to address a small comment by Zhangyi on restoring origin fh. Will push a fix today.

Pushed rebased and fixes version to:
https://github.com/amir73il/linux/commits/ovl-whiteouts

Thanks,
Amir.

author maintainer
#15 lore raw
Quoted 39 lines · Amir Goldstein
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 8 lines
Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Quoted 5 lines

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount

Fixed by "fix free of ERR_PTR oe"

Quoted 9 lines

overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)

[unmount of xfs vdf]

Fixed by "fix double path_put() on error"

Quoted 3 lines

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Amir.

maintainer
#16 lore raw
Quoted 58 lines · Amir Goldstein
Quoted 39 lines · Amir Goldstein
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 8 lines
Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Quoted 5 lines

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount

Fixed by "fix free of ERR_PTR oe"

Quoted 9 lines

overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)

[unmount of xfs vdf]

Quoted 7 lines

Fixed by "fix double path_put() on error"

Quoted 3 lines

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Thanks for testing and looking at the issues. I've now set up xfstests as well in my test environment...

Force pushed branch with fixes to overlayfs-next.

Thanks,
Miklos

author maintainer
#17 lore raw
Quoted 61 lines · Miklos Szeredi
Quoted 60 lines · Amir Goldstein
Quoted 39 lines · Amir Goldstein
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 8 lines
Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Quoted 5 lines

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount

Fixed by "fix free of ERR_PTR oe"

Quoted 9 lines

overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)

[unmount of xfs vdf]

Fixed by "fix double path_put() on error"

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Thanks for testing and looking at the issues. I've now set up xfstests as well in my test environment...

You may want to try out Ted's kvm-xfstests if you haven't already. I've set it up to also run unionmount testsuite tests from within xfstests:

https://github.com/amir73il/xfstests-bld/commits/overlayfs-devel

Quoted 2 lines

Force pushed branch with fixes to overlayfs-next.

Looks good.
Will test.

Amir.

author maintainer
#18 lore raw
Quoted 75 lines · Amir Goldstein
Quoted 61 lines · Miklos Szeredi
Quoted 60 lines · Amir Goldstein
Quoted 39 lines · Amir Goldstein
Quoted 29 lines · Amir Goldstein
Quoted 28 lines · Miklos Szeredi
Quoted 26 lines · Amir Goldstein

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino, but this is because of a test bug
- With the test bug fix available at [2] test passes with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename" because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Quoted 8 lines
Quoted 2 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack(). IMO should add ufs->remote boolean because it is interesting anyway and set s_d_op in ovl_fill_super() itself.

Quoted 5 lines

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount

Fixed by "fix free of ERR_PTR oe"

Quoted 9 lines

overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)

[unmount of xfs vdf]

Fixed by "fix double path_put() on error"

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Thanks for testing and looking at the issues. I've now set up xfstests as well in my test environment...

You may want to try out Ted's kvm-xfstests if you haven't already. I've set it up to also run unionmount testsuite tests from within xfstests:

Quoted 8 lines

https://github.com/amir73il/xfstests-bld/commits/overlayfs-devel

Quoted 2 lines

Force pushed branch with fixes to overlayfs-next.

Looks good.
Will test.

Tested. All good.

Did you get a chance to look at the 2 cleanup patches on overlayfs-devel:

3c964f63efcf ovl: factor out ovl_map_dev_ino() helper 8e97d9da4f2f ovl: store layer index in ovl_layer

Do you want me to post them proper?
A variant of them was posted within the v8 series.

BTW, there is a locking order issue for nested overlay I am working on which I intend to post later today. I hope it can make it to v4.15 as well.

Thanks,
Amir.

author maintainer
#19 lore raw
In reply to Amir Goldstein
Quoted 2 lines · Amir Goldstein
Quoted 1 line · Amir Goldstein

On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <[email protected]> wrote:

[...]

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

I the meantime I went and committed v7 (with the noted changes) and based my cleanup for ovl_fill_super() on top of that. So that's now pushed out to overlayfs-next, hopefully not invalidating too much of your work.

Not a problem. most of the work on v8 was on the last patches.
I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.

And finally, those are the cleanup patches from v8:
42448ca8681a ovl: factor out ovl_map_dev_ino() helper 52a325cd6b41 ovl: store layer index in ovl_layer

Can be found at the tip of:
https://github.com/amir73il/linux/commits/overlayfs-devel

The xino patches are also rebased and tested on:
https://github.com/amir73il/linux/commits/ovl-xino

In case you change your mind about this merge cycle.

Thanks,
Amir.

author maintainer
#20 lore raw
In reply to Miklos Szeredi
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

Vivek has already written an xfstest for the middle layer opaque issue. Eddie has already tested the NFSv3 DT_UNKNOWN issue as
well as the ovl-xino branch that fixes NFSv3 DT_UNKNOWN for the non samefs case.

You may want to pick at least the first and third patches from ovl-fixes to v4.16.

Thanks,
Amir.

maintainer
#21 lore raw
Quoted 8 lines · Amir Goldstein
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

I'm a bit confused about this last one. It's in Vivek's lookup fixes series as well but in a slightly different form. Which one should I be looking at?

Thanks,
Miklois

author maintainer
#22 lore raw
Quoted 18 lines · Miklos Szeredi
Quoted 8 lines · Amir Goldstein
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

Quoted 4 lines

I'm a bit confused about this last one. It's in Vivek's lookup fixes series as well but in a slightly different form. Which one should I be looking at?

Vivek's patch is the later version (so v2).
There are 2 differences between v1 and v2:
1. Vivek's commit message is more elaborate, so should take it.
2. Vivek's patch sets only d->stop and not d->opaque This difference is purely semantic, because d->opaque is ignored in ovl_lookup() for anything but the upper layer.

I am fine with the semantic change, but wasn't sure if you had other meaning in mind w.r.t d->opaque and metadata going forward.

Thanks,
Amir.

maintainer
#23 lore raw
Quoted 31 lines · Amir Goldstein
Quoted 18 lines · Miklos Szeredi
Quoted 8 lines · Amir Goldstein
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

Quoted 4 lines

I'm a bit confused about this last one. It's in Vivek's lookup fixes series as well but in a slightly different form. Which one should I be looking at?

Vivek's patch is the later version (so v2).
There are 2 differences between v1 and v2:
1. Vivek's commit message is more elaborate, so should take it.
2. Vivek's patch sets only d->stop and not d->opaque This difference is purely semantic, because d->opaque is ignored in ovl_lookup() for anything but the upper layer.

Quoted 3 lines

I am fine with the semantic change, but wasn't sure if you had other meaning in mind w.r.t d->opaque and metadata going forward.

I think clearing opaque it is the correct thing to do even if it doesn't have any effect.

Thanks,
Miklos

#24 lore raw
Quoted 32 lines · Miklos Szeredi
Quoted 31 lines · Amir Goldstein
Quoted 18 lines · Miklos Szeredi
Quoted 8 lines · Amir Goldstein
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

Quoted 4 lines

I'm a bit confused about this last one. It's in Vivek's lookup fixes series as well but in a slightly different form. Which one should I be looking at?

Vivek's patch is the later version (so v2).
There are 2 differences between v1 and v2:
1. Vivek's commit message is more elaborate, so should take it.
2. Vivek's patch sets only d->stop and not d->opaque This difference is purely semantic, because d->opaque is ignored in ovl_lookup() for anything but the upper layer.

Quoted 6 lines
Quoted 3 lines

I am fine with the semantic change, but wasn't sure if you had other meaning in mind w.r.t d->opaque and metadata going forward.

I think clearing opaque it is the correct thing to do even if it doesn't have any effect.

Hi Miklos,

I am fine with anything. I was just trying to make it semantically more clear. That is when ovl_lookup_layer() is called, what do fields in ovl_data{} reflect. I felt that "->opaque" should reflect the property of last element of the path. And if we stick to it, then we should not clear d->opauque when absolute redirect is found.

But I am not particular about it and it should not affect existing metacopy patches I am working on. So I am fine with whatever you think is better thing to do.

Vivek

maintainer
#25 lore raw
Quoted 40 lines · Vivek Goyal
Quoted 32 lines · Miklos Szeredi
Quoted 31 lines · Amir Goldstein
Quoted 18 lines · Miklos Szeredi
Quoted 8 lines · Amir Goldstein
Quoted 1 line · Miklos Szeredi

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <[email protected]> wrote:

[...]

Quoted 3 lines

The -oxino patches are interesting, but maybe we should leave them brewing for another cycle. Do you agree?

Quoted 8 lines

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and absolute path redirects

Quoted 4 lines

I'm a bit confused about this last one. It's in Vivek's lookup fixes series as well but in a slightly different form. Which one should I be looking at?

Vivek's patch is the later version (so v2).
There are 2 differences between v1 and v2:
1. Vivek's commit message is more elaborate, so should take it.
2. Vivek's patch sets only d->stop and not d->opaque This difference is purely semantic, because d->opaque is ignored in ovl_lookup() for anything but the upper layer.

Quoted 6 lines
Quoted 3 lines

I am fine with the semantic change, but wasn't sure if you had other meaning in mind w.r.t d->opaque and metadata going forward.

I think clearing opaque it is the correct thing to do even if it doesn't have any effect.

Quoted 7 lines

Hi Miklos,

I am fine with anything. I was just trying to make it semantically more clear. That is when ovl_lookup_layer() is called, what do fields in ovl_data{} reflect. I felt that "->opaque" should reflect the property of last element of the path. And if we stick to it, then we should not clear d->opauque when absolute redirect is found.

Right. In the light of the rest of your patchset it makes sense.

Thanks,
Miklos