Skip to contentSkip to conversation

ovl: new mount options lowerdir+,datadir+

PATCH In review Amir Goldstein 2 participants 10 messages Oct 30, 2023 Overlay filesystem

lore
Share & tools
lore

Amir Goldstein

author maintainer
#0 lore raw

cover ovl: new mount options lowerdir+,datadir+

Miklos,

As discussed, here are the patches for the new mount options.

- Only string format is supported
- Legacy lowerdir= cannot be mixed with new lowerdir+,datadir+
- lowerdir+,datadir+ are not escaped
- lowerdir,upperdir,workdir are escaped as always

I did not find a good reason to change escaping of upperdir,workdir. We can skip escaping when we add support for path format.

Wrote some xfstests [1].

Thanks,
Amir.

[1] https://github.com/amir73il/xfstests/commits/ovl-lowerdir-add

Amir Goldstein (4):
ovl: remove unused code in lowerdir param parsing
ovl: store and show the user provided lowerdir mount option
ovl: refactor layer parsing helpers
ovl: add support for appending lowerdirs one by one

 fs/overlayfs/params.c | 317 ++++++++++++++++++++++--------------------
 fs/overlayfs/params.h |   1 +
 fs/overlayfs/super.c  |   5 +-
 3 files changed, 173 insertions(+), 150 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH 1/4 ovl: remove unused code in lowerdir param parsing

Commit beae836e9c61 ("ovl: temporarily disable appending lowedirs") removed the ability to append lowerdirs with syntax lowerdir=":<path>". Remove leftover code and comments that are irrelevant with lowerdir append mode disabled.

---
 fs/overlayfs/params.c | 95 ++++++++-----------------------------------
 1 file changed, 16 insertions(+), 79 deletions(-)
fs/overlayfs/params.c +16 -79
index f6ff23fd101c..0059cc405159 100644
@@ -346,7 +346,7 @@ static void ovl_parse_param_drop_lowerdir(struct ovl_fs_context *ctx)
 /*
  * Parse lowerdir= mount option:
  *
- * (1) lowerdir=/lower1:/lower2:/lower3::/data1::/data2
+ * e.g.: lowerdir=/lower1:/lower2:/lower3::/data1::/data2
  *     Set "/lower1", "/lower2", and "/lower3" as lower layers and
  *     "/data1" and "/data2" as data lower layers. Any existing lower
  *     layers are replaced.
@@ -356,9 +356,9 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	int err;
 	struct ovl_fs_context *ctx = fc->fs_private;
 	struct ovl_fs_context_layer *l;
-	char *dup = NULL, *dup_iter;
+	char *dup = NULL, *iter;
 	ssize_t nr_lower = 0, nr = 0, nr_data = 0;
-	bool append = false, data_layer = false;
+	bool data_layer = false;
 
 	/*
 	 * Ensure we're backwards compatible with mount(2)
@@ -366,10 +366,10 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	 */
 
 	/* drop all existing lower layers */
-	if (!*name) {
-		ovl_parse_param_drop_lowerdir(ctx);
+	ovl_parse_param_drop_lowerdir(ctx);
+
+	if (!*name)
 		return 0;
-	}
 
 	if (*name == ':') {
 		pr_err("cannot append lower layer");
@@ -385,36 +385,11 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	if (nr_lower < 0)
 		goto out_err;
 
-	if ((nr_lower > OVL_MAX_STACK) ||
-	    (append && (size_add(ctx->nr, nr_lower) > OVL_MAX_STACK))) {
+	if (nr_lower > OVL_MAX_STACK) {
 		pr_err("too many lower directories, limit is %d\n", OVL_MAX_STACK);
 		goto out_err;
 	}
 
-	if (!append)
-		ovl_parse_param_drop_lowerdir(ctx);
-
-	/*
-	 * (1) append
-	 *
-	 * We want nr <= nr_lower <= capacity We know nr > 0 and nr <=
-	 * capacity. If nr == 0 this wouldn't be append. If nr +
-	 * nr_lower is <= capacity then nr <= nr_lower <= capacity
-	 * already holds. If nr + nr_lower exceeds capacity, we realloc.
-	 *
-	 * (2) replace
-	 *
-	 * Ensure we're backwards compatible with mount(2) which allows
-	 * "lowerdir=/a:/b:/c,lowerdir=/d:/e:/f" causing the last
-	 * specified lowerdir mount option to win.
-	 *
-	 * We want nr <= nr_lower <= capacity We know either (i) nr == 0
-	 * or (ii) nr > 0. We also know nr_lower > 0. The capacity
-	 * could've been changed multiple times already so we only know
-	 * nr <= capacity. If nr + nr_lower > capacity we realloc,
-	 * otherwise nr <= nr_lower <= capacity holds already.
-	 */
-	nr_lower += ctx->nr;
 	if (nr_lower > ctx->capacity) {
 		err = -ENOMEM;
 		l = krealloc_array(ctx->lower, nr_lower, sizeof(*ctx->lower),
@@ -426,41 +401,17 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 		ctx->capacity = nr_lower;
 	}
 
-	/*
-	 *   (3) By (1) and (2) we know nr <= nr_lower <= capacity.
-	 *   (4) If ctx->nr == 0 => replace
-	 *       We have verified above that the lowerdir mount option
-	 *       isn't an append, i.e., the lowerdir mount option
-	 *       doesn't start with ":" or "::".
-	 * (4.1) The lowerdir mount options only contains regular lower
-	 *       layers ":".
-	 *       => Nothing to verify.
-	 * (4.2) The lowerdir mount options contains regular ":" and
-	 *       data "::" layers.
-	 *       => We need to verify that data lower layers "::" aren't
-	 *          followed by regular ":" lower layers
-	 *   (5) If ctx->nr > 0 => append
-	 *       We know that there's at least one regular layer
-	 *       otherwise we would've failed when parsing the previous
-	 *       lowerdir mount option.
-	 * (5.1) The lowerdir mount option is a regular layer ":" append
-	 *       => We need to verify that no data layers have been
-	 *          specified before.
-	 * (5.2) The lowerdir mount option is a data layer "::" append
-	 *       We know that there's at least one regular layer or
-	 *       other data layers. => There's nothing to verify.
-	 */
-	dup_iter = dup;
-	for (nr = ctx->nr; nr < nr_lower; nr++) {
-		l = &ctx->lower[nr];
+	iter = dup;
+	l = ctx->lower;
+	for (nr = 0; nr < nr_lower; nr++, l++) {
 		memset(l, 0, sizeof(*l));
 
-		err = ovl_mount_dir(dup_iter, &l->path, false);
+		err = ovl_mount_dir(iter, &l->path, false);
 		if (err)
 			goto out_put;
 
 		err = -ENOMEM;
-		l->name = kstrdup(dup_iter, GFP_KERNEL_ACCOUNT);
+		l->name = kstrdup(iter, GFP_KERNEL_ACCOUNT);
 		if (!l->name)
 			goto out_put;
 
@@ -472,8 +423,8 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 			break;
 
 		err = -EINVAL;
-		dup_iter = strchr(dup_iter, '\0') + 1;
-		if (*dup_iter) {
+		iter = strchr(iter, '\0') + 1;
+		if (*iter) {
 			/*
 			 * This is a regular layer so we require that
 			 * there are no data layers.
@@ -489,7 +440,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 
 		/* This is a data lower layer. */
 		data_layer = true;
-		dup_iter++;
+		iter++;
 	}
 	ctx->nr = nr_lower;
 	ctx->nr_data += nr_data;
@@ -497,21 +448,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	return 0;
 
 out_put:
-	/*
-	 * We know nr >= ctx->nr < nr_lower. If we failed somewhere
-	 * we want to undo until nr == ctx->nr. This is correct for
-	 * both ctx->nr == 0 and ctx->nr > 0.
-	 */
-	for (; nr >= ctx->nr; nr--) {
-		l = &ctx->lower[nr];
-		kfree(l->name);
-		l->name = NULL;
-		path_put(&l->path);
-
-		/* don't overflow */
-		if (nr == 0)
-			break;
-	}
+	ovl_parse_param_drop_lowerdir(ctx);
 
 out_err:
 	kfree(dup);
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH 2/4 ovl: store and show the user provided lowerdir mount option

In reply to Amir Goldstein

We are about to add new mount options for adding lowerdir one by one, but those mount options will not support escaping.

For the existing case, where lowerdir mount option is provided as a colon separated list, store the user provided (possibly escaped) string and display it as is when showing the lowerdir mount option.

---
 fs/overlayfs/params.c | 46 +++++++++++++++++++++----------------------
 fs/overlayfs/params.h |  1 +
 fs/overlayfs/super.c  |  5 ++++-
 3 files changed, 28 insertions(+), 24 deletions(-)
fs/overlayfs/params.c +23 -23
index 0059cc405159..0bf754a69e91 100644
@@ -332,12 +332,18 @@ static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,
 	return 0;
 }
 
-static void ovl_parse_param_drop_lowerdir(struct ovl_fs_context *ctx)
+static void ovl_reset_lowerdirs(struct ovl_fs_context *ctx)
 {
-	for (size_t nr = 0; nr < ctx->nr; nr++) {
-		path_put(&ctx->lower[nr].path);
-		kfree(ctx->lower[nr].name);
-		ctx->lower[nr].name = NULL;
+	struct ovl_fs_context_layer *l = ctx->lower;
+
+	// Reset old user provided lowerdir string
+	kfree(ctx->lowerdir_all);
+	ctx->lowerdir_all = NULL;
+
+	for (size_t nr = 0; nr < ctx->nr; nr++, l++) {
+		path_put(&l->path);
+		kfree(l->name);
+		l->name = NULL;
 	}
 	ctx->nr = 0;
 	ctx->nr_data = 0;
@@ -366,7 +372,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	 */
 
 	/* drop all existing lower layers */
-	ovl_parse_param_drop_lowerdir(ctx);
+	ovl_reset_lowerdirs(ctx);
 
 	if (!*name)
 		return 0;
@@ -376,6 +382,11 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 		return -EINVAL;
 	}
 
+	// Store user provided lowerdir string to show in mount options
+	ctx->lowerdir_all = kstrdup(name, GFP_KERNEL);
+	if (!ctx->lowerdir_all)
+		return -ENOMEM;
+
 	dup = kstrdup(name, GFP_KERNEL);
 	if (!dup)
 		return -ENOMEM;
@@ -448,7 +459,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	return 0;
 
 out_put:
-	ovl_parse_param_drop_lowerdir(ctx);
+	ovl_reset_lowerdirs(ctx);
 
 out_err:
 	kfree(dup);
@@ -554,7 +565,7 @@ static int ovl_get_tree(struct fs_context *fc)
 
 static inline void ovl_fs_context_free(struct ovl_fs_context *ctx)
 {
-	ovl_parse_param_drop_lowerdir(ctx);
+	ovl_reset_lowerdirs(ctx);
 	path_put(&ctx->upper);
 	path_put(&ctx->work);
 	kfree(ctx->lower);
@@ -870,24 +881,13 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry)
 {
 	struct super_block *sb = dentry->d_sb;
 	struct ovl_fs *ofs = OVL_FS(sb);
-	size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer;
+	char **lowerdirs = ofs->config.lowerdirs;
 
 	/*
-	 * lowerdirs[] starts from offset 1, then
-	 * >= 0 regular lower layers prefixed with : and
-	 * >= 0 data-only lower layers prefixed with ::
-	 *
-	 * we need to escase comma and space like seq_show_option() does and
-	 * we also need to escape the colon separator from lowerdir paths.
+	 * lowerdirs[0] holds the colon separated list that user provided
+	 * with lowerdir mount option.
 	 */
-	seq_puts(m, ",lowerdir=");
-	for (nr = 1; nr < ofs->numlayer; nr++) {
-		if (nr > 1)
-			seq_putc(m, ':');
-		if (nr >= nr_merged_lower)
-			seq_putc(m, ':');
-		seq_escape(m, ofs->config.lowerdirs[nr], ":, \t\n\\");
-	}
+	seq_show_option(m, "lowerdir", lowerdirs[0]);
 	if (ofs->config.upperdir) {
 		seq_show_option(m, "upperdir", ofs->config.upperdir);
 		seq_show_option(m, "workdir", ofs->config.workdir);
fs/overlayfs/params.h +1 -0
index 8750da68ab2a..c96d93982021 100644
@@ -32,6 +32,7 @@ struct ovl_fs_context {
 	size_t nr_data;
 	struct ovl_opt_set set;
 	struct ovl_fs_context_layer *lower;
+	char *lowerdir_all; /* user provided lowerdir string */
 };
 
 int ovl_init_fs_context(struct fs_context *fc);
fs/overlayfs/super.c +4 -1
index 60bd7331e20f..26bb429c78dc 100644
@@ -1320,8 +1320,11 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc)
 	ofs->layers = layers;
 	/*
 	 * Layer 0 is reserved for upper even if there's no upper.
-	 * For consistency, config.lowerdirs[0] is NULL.
+	 * config.lowerdirs[0] is used for storing the user provided colon
+	 * separated lowerdir string.
 	 */
+	ofs->config.lowerdirs[0] = ctx->lowerdir_all;
+	ctx->lowerdir_all = NULL;
 	ofs->numlayer = 1;
 
 	sb->s_stack_depth = 0;
Signature · 1 line
2.34.1
author maintainer
#3 lore raw

PATCH 3/4 ovl: refactor layer parsing helpers

In reply to Amir Goldstein

In preparation for new mount options to add lowerdirs one by one, generalize ovl_parse_param_upperdir() into helper ovl_parse_layer() with bool @upper argument that will be false for adding lower layers.

---
 fs/overlayfs/params.c | 116 ++++++++++++++++++++++--------------------
 1 file changed, 62 insertions(+), 54 deletions(-)
fs/overlayfs/params.c +62 -54
index 0bf754a69e91..9a9238eac730 100644
@@ -43,7 +43,7 @@ module_param_named(metacopy, ovl_metacopy_def, bool, 0644);
 MODULE_PARM_DESC(metacopy,
 		 "Default to on or off for the metadata only copy up feature");
 
-enum {
+enum ovl_opt {
 	Opt_lowerdir,
 	Opt_upperdir,
 	Opt_workdir,
@@ -238,19 +238,8 @@ static int ovl_mount_dir_noesc(const char *name, struct path *path)
 		pr_err("failed to resolve '%s': %i\n", name, err);
 		goto out;
 	}
-	err = -EINVAL;
-	if (ovl_dentry_weird(path->dentry)) {
-		pr_err("filesystem on '%s' not supported\n", name);
-		goto out_put;
-	}
-	if (!d_is_dir(path->dentry)) {
-		pr_err("'%s' not a directory\n", name);
-		goto out_put;
-	}
 	return 0;
 
-out_put:
-	path_put_init(path);
 out:
 	return err;
 }
@@ -268,7 +257,7 @@ static void ovl_unescape(char *s)
 	}
 }
 
-static int ovl_mount_dir(const char *name, struct path *path, bool upper)
+static int ovl_mount_dir(const char *name, struct path *path)
 {
 	int err = -ENOMEM;
 	char *tmp = kstrdup(name, GFP_KERNEL);
@@ -276,60 +265,81 @@ static int ovl_mount_dir(const char *name, struct path *path, bool upper)
 	if (tmp) {
 		ovl_unescape(tmp);
 		err = ovl_mount_dir_noesc(tmp, path);
-
-		if (!err && upper && path->dentry->d_flags & DCACHE_OP_REAL) {
-			pr_err("filesystem on '%s' not supported as upperdir\n",
-			       tmp);
-			path_put_init(path);
-			err = -EINVAL;
-		}
 		kfree(tmp);
 	}
 	return err;
 }
 
-static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,
-				    bool workdir)
+static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
+			       enum ovl_opt layer, const char *name, bool upper)
 {
-	int err;
-	struct ovl_fs *ofs = fc->s_fs_info;
-	struct ovl_config *config = &ofs->config;
-	struct ovl_fs_context *ctx = fc->fs_private;
-	struct path path;
-	char *dup;
+	if (ovl_dentry_weird(path->dentry))
+		return invalfc(fc, "filesystem on %s not supported", name);
 
-	err = ovl_mount_dir(name, &path, true);
-	if (err)
-		return err;
+	if (!d_is_dir(path->dentry))
+		return invalfc(fc, "%s is not a directory", name);
 
 	/*
 	 * Check whether upper path is read-only here to report failures
 	 * early. Don't forget to recheck when the superblock is created
 	 * as the mount attributes could change.
 	 */
-	if (__mnt_is_readonly(path.mnt)) {
-		path_put(&path);
-		return -EINVAL;
+	if (upper) {
+		if (path->dentry->d_flags & DCACHE_OP_REAL)
+			return invalfc(fc, "filesystem not supported as %s", name);
+		if (__mnt_is_readonly(path->mnt))
+			return invalfc(fc, "%s is read-only", name);
 	}
+	return 0;
+}
 
-	dup = kstrdup(name, GFP_KERNEL);
-	if (!dup) {
-		path_put(&path);
-		return -ENOMEM;
-	}
+static void ovl_add_layer(struct fs_context *fc, enum ovl_opt layer,
+			 struct path *path, char **pname)
+{
+	struct ovl_fs *ofs = fc->s_fs_info;
+	struct ovl_config *config = &ofs->config;
+	struct ovl_fs_context *ctx = fc->fs_private;
 
-	if (workdir) {
-		kfree(config->workdir);
-		config->workdir = dup;
-		path_put(&ctx->work);
-		ctx->work = path;
-	} else {
-		kfree(config->upperdir);
-		config->upperdir = dup;
-		path_put(&ctx->upper);
-		ctx->upper = path;
+	switch (layer) {
+	case Opt_workdir:
+		swap(config->workdir, *pname);
+		swap(ctx->work, *path);
+		break;
+	case Opt_upperdir:
+		swap(config->upperdir, *pname);
+		swap(ctx->upper, *path);
+		break;
+	default:
+		WARN_ON(1);
 	}
-	return 0;
+}
+
+static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
+			   enum ovl_opt layer, bool upper)
+{
+	char *name = kstrdup(param->string, GFP_KERNEL);
+	struct path path;
+	int err;
+
+	if (!name)
+		return -ENOMEM;
+
+	err = ovl_mount_dir(name, &path);
+	if (err)
+		goto out_free;
+
+	err = ovl_mount_dir_check(fc, &path, layer, param->key, upper);
+	if (err)
+		goto out_put;
+
+	/* Store the user provided path string in ctx to show in mountinfo */
+	ovl_add_layer(fc, layer, &path, &name);
+
+out_put:
+	path_put(&path);
+out_free:
+	kfree(name);
+	return err;
 }
 
 static void ovl_reset_lowerdirs(struct ovl_fs_context *ctx)
@@ -417,7 +427,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc)
 	for (nr = 0; nr < nr_lower; nr++, l++) {
 		memset(l, 0, sizeof(*l));
 
-		err = ovl_mount_dir(iter, &l->path, false);
+		err = ovl_mount_dir(iter, &l->path);
 		if (err)
 			goto out_put;
 
@@ -505,10 +515,8 @@ static int ovl_parse_param(struct fs_context *fc, struct fs_parameter *param)
 		err = ovl_parse_param_lowerdir(param->string, fc);
 		break;
 	case Opt_upperdir:
-		fallthrough;
 	case Opt_workdir:
-		err = ovl_parse_param_upperdir(param->string, fc,
-					       (Opt_workdir == opt));
+		err = ovl_parse_layer(fc, param, opt, true);
 		break;
 	case Opt_default_permissions:
 		config->default_permissions = true;
Signature · 1 line
2.34.1
maintainer
#4 lore raw

Re: [PATCH 3/4] ovl: refactor layer parsing helpers

Quoted 28 lines · Amir Goldstein

In preparation for new mount options to add lowerdirs one by one, generalize ovl_parse_param_upperdir() into helper ovl_parse_layer() with bool @upper argument that will be false for adding lower layers.

---
 fs/overlayfs/params.c | 116 ++++++++++++++++++++++--------------------
 1 file changed, 62 insertions(+), 54 deletions(-)
fs/overlayfs/params.c +1 -1
index 0bf754a69e91..9a9238eac730 100644
@@ -43,7 +43,7 @@ module_param_named(metacopy, ovl_metacopy_def, bool, 0644);
 MODULE_PARM_DESC(metacopy,
                 "Default to on or off for the metadata only copy up feature");
 
-enum {
+enum ovl_opt {
        Opt_lowerdir,
        Opt_upperdir,
        Opt_workdir,
@@ -238,19 +238,8 @@ static int ovl_mount_dir_noesc(const char *name, struct path *path)
                pr_err("failed to resolve '%s': %i\n", name, err);
                goto out;

fs/overlayfs/params.c static int ovl_mount_dir_noesc(const char *name, struct path *path) · line 249

Quoted 10 lines
fs/overlayfs/params.c +0 -9
        }
-       err = -EINVAL;
-       if (ovl_dentry_weird(path->dentry)) {
-               pr_err("filesystem on '%s' not supported\n", name);
-               goto out_put;
-       }
-       if (!d_is_dir(path->dentry)) {
-               pr_err("'%s' not a directory\n", name);
-               goto out_put;
-       }

This will lose the check for lowerdir, no?

Quoted 41 lines
fs/overlayfs/params.c +3 -16
        return 0;
 
-out_put:
-       path_put_init(path);
 out:
        return err;
 }
@@ -268,7 +257,7 @@ static void ovl_unescape(char *s)
        }
 }
 
-static int ovl_mount_dir(const char *name, struct path *path, bool upper)
+static int ovl_mount_dir(const char *name, struct path *path)
 {
        int err = -ENOMEM;
        char *tmp = kstrdup(name, GFP_KERNEL);
@@ -276,60 +265,81 @@ static int ovl_mount_dir(const char *name, struct path *path, bool upper)
        if (tmp) {
                ovl_unescape(tmp);
                err = ovl_mount_dir_noesc(tmp, path);
-
-               if (!err && upper && path->dentry->d_flags & DCACHE_OP_REAL) {
-                       pr_err("filesystem on '%s' not supported as upperdir\n",
-                              tmp);
-                       path_put_init(path);
-                       err = -EINVAL;
-               }
                kfree(tmp);
        }
        return err;
 }
 
-static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,
-                                   bool workdir)
+static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
+                              enum ovl_opt layer, const char *name, bool upper)
 {
-       int err;
-       struct ovl_fs *ofs = fc->s_fs_info;
-       struct ovl_config *config = &ofs->config;
-       struct ovl_fs_context *ctx = fc->fs_private;

fs/overlayfs/params.c static int ovl_mount_dir(const char *name, struct path *path, bool upper) · line 280

Quoted 10 lines
fs/overlayfs/params.c +4 -5
-       struct path path;
-       char *dup;
+       if (ovl_dentry_weird(path->dentry))
+               return invalfc(fc, "filesystem on %s not supported", name);
 
-       err = ovl_mount_dir(name, &path, true);
-       if (err)
-               return err;
+       if (!d_is_dir(path->dentry))
+               return invalfc(fc, "%s is not a directory", name);

This can result in:

overlay: lowerdir+ is not a directory

Which is somewhat confusing. Not sure how mount/libmount will present such option error messages, as that does not currently work.

So the kernel could be really nice about it and tell the user which
lowerdir (layer index). But libmount could also indicate which
option failed, in which case indicating the layer would not be needed.
OTOH when using the legacy API we do need to tell the user whether
it was upperdir or workdir, but that doesn't affect lowerdir+. So
some compromise and negotiation with util-linux devs is needed.

Thanks,
Miklos

author maintainer
#5 lore raw

Re: [PATCH 3/4] ovl: refactor layer parsing helpers

Quoted 34 lines · Miklos Szeredi
Quoted 32 lines · Amir Goldstein

In preparation for new mount options to add lowerdirs one by one, generalize ovl_parse_param_upperdir() into helper ovl_parse_layer() with bool @upper argument that will be false for adding lower layers.

---
 fs/overlayfs/params.c | 116 ++++++++++++++++++++++--------------------
 1 file changed, 62 insertions(+), 54 deletions(-)
fs/overlayfs/params.c +1 -4
index 0bf754a69e91..9a9238eac730 100644
@@ -43,7 +43,7 @@ module_param_named(metacopy, ovl_metacopy_def, bool, 0644);
 MODULE_PARM_DESC(metacopy,
                 "Default to on or off for the metadata only copy up feature");
 
-enum {
+enum ovl_opt {
        Opt_lowerdir,
        Opt_upperdir,
        Opt_workdir,
@@ -238,19 +238,8 @@ static int ovl_mount_dir_noesc(const char *name, struct path *path)
                pr_err("failed to resolve '%s': %i\n", name, err);
                goto out;
        }
-       err = -EINVAL;
-       if (ovl_dentry_weird(path->dentry)) {
-               pr_err("filesystem on '%s' not supported\n", name);
Quoted 9 lines

fs/overlayfs/params.c static int ovl_mount_dir_noesc(const char *name, struct path *path) · line 249

Quoted 6 lines
fs/overlayfs/params.c +0 -6
-               goto out_put;
-       }
-       if (!d_is_dir(path->dentry)) {
-               pr_err("'%s' not a directory\n", name);
-               goto out_put;
-       }

This will lose the check for lowerdir, no?

oops. I guess I'll need to add a test case...

Quoted 59 lines
Quoted 41 lines
fs/overlayfs/params.c +3 -16
        return 0;
 
-out_put:
-       path_put_init(path);
 out:
        return err;
 }
@@ -268,7 +257,7 @@ static void ovl_unescape(char *s)
        }
 }
 
-static int ovl_mount_dir(const char *name, struct path *path, bool upper)
+static int ovl_mount_dir(const char *name, struct path *path)
 {
        int err = -ENOMEM;
        char *tmp = kstrdup(name, GFP_KERNEL);
@@ -276,60 +265,81 @@ static int ovl_mount_dir(const char *name, struct path *path, bool upper)
        if (tmp) {
                ovl_unescape(tmp);
                err = ovl_mount_dir_noesc(tmp, path);
-
-               if (!err && upper && path->dentry->d_flags & DCACHE_OP_REAL) {
-                       pr_err("filesystem on '%s' not supported as upperdir\n",
-                              tmp);
-                       path_put_init(path);
-                       err = -EINVAL;
-               }
                kfree(tmp);
        }
        return err;
 }
 
-static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,
-                                   bool workdir)
+static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
+                              enum ovl_opt layer, const char *name, bool upper)
 {
-       int err;
-       struct ovl_fs *ofs = fc->s_fs_info;
-       struct ovl_config *config = &ofs->config;
-       struct ovl_fs_context *ctx = fc->fs_private;

fs/overlayfs/params.c static int ovl_mount_dir(const char *name, struct path *path, bool upper) · line 280

Quoted 10 lines
fs/overlayfs/params.c +4 -5
-       struct path path;
-       char *dup;
+       if (ovl_dentry_weird(path->dentry))
+               return invalfc(fc, "filesystem on %s not supported", name);
 
-       err = ovl_mount_dir(name, &path, true);
-       if (err)
-               return err;
+       if (!d_is_dir(path->dentry))
+               return invalfc(fc, "%s is not a directory", name);

This can result in:

overlay: lowerdir+ is not a directory

Which is somewhat confusing. Not sure how mount/libmount will present such option error messages, as that does not currently work.

Quoted 7 lines

So the kernel could be really nice about it and tell the user which
lowerdir (layer index). But libmount could also indicate which
option failed, in which case indicating the layer would not be needed.
OTOH when using the legacy API we do need to tell the user whether
it was upperdir or workdir, but that doesn't affect lowerdir+. So
some compromise and negotiation with util-linux devs is needed.

What a mess. I prefer to restore the old pr_err messages with the pathname for now, because they are more likely to help the users fix the problem.

We could sort it better when we add support for path parameters. At least with path params, we would know that it is not legacy mount API.

Thanks,
Amir.

author maintainer
#6 lore raw

PATCH 4/4 ovl: add support for appending lowerdirs one by one

In reply to Amir Goldstein

Add new mount options lowerdir+ and datadir+ that can be used to add layers to lower layers stack one by one.

Unlike the legacy lowerdir mount option, special characters (i.e. colons and cammas) are not unescaped with these new mount options.

The new mount options can be repeated to compose a large stack of lower layers, but they may not be mixed with the lagacy lowerdir mount option, because for displaying lower layers in mountinfo, we do not want to mix escaped with unescaped lower layers path syntax.

Similar to data-only layer rules with the lowerdir mount option, the datadir+ option must follow at least one lowerdir+ option and the lowerdir+ option must not follow the datadir+ option.

If the legacy lowerdir mount option follows lowerdir+ and datadir+ mount options, it overrides them. Sepcifically, calling:

  fsconfig(FSCONFIG_SET_STRING, "lowerdir", "", 0);

can be used to reset previously setup lower layers.

---
 fs/overlayfs/params.c | 78 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 76 insertions(+), 2 deletions(-)
fs/overlayfs/params.c +76 -2
index 9a9238eac730..1c390e93d060 100644
@@ -45,6 +45,8 @@ MODULE_PARM_DESC(metacopy,
 
 enum ovl_opt {
 	Opt_lowerdir,
+	Opt_lowerdir_add,
+	Opt_datadir_add,
 	Opt_upperdir,
 	Opt_workdir,
 	Opt_default_permissions,
@@ -140,8 +142,11 @@ static int ovl_verity_mode_def(void)
 #define fsparam_string_empty(NAME, OPT) \
 	__fsparam(fs_param_is_string, NAME, OPT, fs_param_can_be_empty, NULL)
 
+
 const struct fs_parameter_spec ovl_parameter_spec[] = {
 	fsparam_string_empty("lowerdir",    Opt_lowerdir),
+	fsparam_string("lowerdir+",         Opt_lowerdir_add),
+	fsparam_string("datadir+",          Opt_datadir_add),
 	fsparam_string("upperdir",          Opt_upperdir),
 	fsparam_string("workdir",           Opt_workdir),
 	fsparam_flag("default_permissions", Opt_default_permissions),
@@ -273,6 +278,8 @@ static int ovl_mount_dir(const char *name, struct path *path)
 static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
 			       enum ovl_opt layer, const char *name, bool upper)
 {
+	struct ovl_fs_context *ctx = fc->fs_private;
+
 	if (ovl_dentry_weird(path->dentry))
 		return invalfc(fc, "filesystem on %s not supported", name);
 
@@ -289,16 +296,44 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
 			return invalfc(fc, "filesystem not supported as %s", name);
 		if (__mnt_is_readonly(path->mnt))
 			return invalfc(fc, "%s is read-only", name);
+	} else {
+		if (ctx->lowerdir_all)
+			return invalfc(fc, "%s cannot follow lowerdir mount option", name);
+		if (ctx->nr_data && layer == Opt_lowerdir_add)
+			return invalfc(fc, "regular lower layers cannot follow data layers");
+		if (ctx->nr == OVL_MAX_STACK)
+			return invalfc(fc, "too many lower directories, limit is %d",
+				       OVL_MAX_STACK);
 	}
 	return 0;
 }
 
+static int ovl_ctx_realloc_lower(struct fs_context *fc)
+{
+	struct ovl_fs_context *ctx = fc->fs_private;
+	struct ovl_fs_context_layer *l;
+	size_t nr;
+
+	if (ctx->nr < ctx->capacity)
+		return 0;
+
+	nr = min(max(4096 / sizeof(*l), ctx->capacity * 2), (size_t) OVL_MAX_STACK);
+	l = krealloc_array(ctx->lower, nr, sizeof(*l), GFP_KERNEL_ACCOUNT);
+	if (!l)
+		return -ENOMEM;
+
+	ctx->lower = l;
+	ctx->capacity = nr;
+	return 0;
+}
+
 static void ovl_add_layer(struct fs_context *fc, enum ovl_opt layer,
 			 struct path *path, char **pname)
 {
 	struct ovl_fs *ofs = fc->s_fs_info;
 	struct ovl_config *config = &ofs->config;
 	struct ovl_fs_context *ctx = fc->fs_private;
+	struct ovl_fs_context_layer *l;
 
 	switch (layer) {
 	case Opt_workdir:
@@ -309,6 +344,16 @@ static void ovl_add_layer(struct fs_context *fc, enum ovl_opt layer,
 		swap(config->upperdir, *pname);
 		swap(ctx->upper, *path);
 		break;
+	case Opt_datadir_add:
+		ctx->nr_data++;
+		fallthrough;
+	case Opt_lowerdir_add:
+		WARN_ON(ctx->nr >= ctx->capacity);
+		l = &ctx->lower[ctx->nr++];
+		memset(l, 0, sizeof(*l));
+		swap(l->name, *pname);
+		swap(l->path, *path);
+		break;
 	default:
 		WARN_ON(1);
 	}
@@ -324,7 +369,10 @@ static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
 	if (!name)
 		return -ENOMEM;
 
-	err = ovl_mount_dir(name, &path);
+	if (upper)
+		err = ovl_mount_dir(name, &path);
+	else
+		err = ovl_mount_dir_noesc(name, &path);
 	if (err)
 		goto out_free;
 
@@ -332,6 +380,12 @@ static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
 	if (err)
 		goto out_put;
 
+	if (!upper) {
+		err = ovl_ctx_realloc_lower(fc);
+		if (err)
+			goto out_put;
+	}
+
 	/* Store the user provided path string in ctx to show in mountinfo */
 	ovl_add_layer(fc, layer, &path, &name);
 
@@ -514,6 +568,10 @@ static int ovl_parse_param(struct fs_context *fc, struct fs_parameter *param)
 	case Opt_lowerdir:
 		err = ovl_parse_param_lowerdir(param->string, fc);
 		break;
+	case Opt_lowerdir_add:
+	case Opt_datadir_add:
+		err = ovl_parse_layer(fc, param, opt, false);
+		break;
 	case Opt_upperdir:
 	case Opt_workdir:
 		err = ovl_parse_layer(fc, param, opt, true);
@@ -889,13 +947,29 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry)
 {
 	struct super_block *sb = dentry->d_sb;
 	struct ovl_fs *ofs = OVL_FS(sb);
+	size_t nr, nr_merged_lower, nr_lower = 0;
 	char **lowerdirs = ofs->config.lowerdirs;
 
 	/*
 	 * lowerdirs[0] holds the colon separated list that user provided
 	 * with lowerdir mount option.
+	 * lowerdirs[1..numlayer] hold the lowerdir paths that were added
+	 * using the lowerdir+ and datadir+ mount options.
+	 * For now, we do not allow mixing the legacy lowerdir mount option
+	 * with the new lowerdir+ and datadir+ mount options.
 	 */
-	seq_show_option(m, "lowerdir", lowerdirs[0]);
+	if (lowerdirs[0]) {
+		seq_show_option(m, "lowerdir", lowerdirs[0]);
+	} else {
+		nr_lower = ofs->numlayer;
+		nr_merged_lower = nr_lower - ofs->numdatalayer;
+	}
+	for (nr = 1; nr < nr_lower; nr++) {
+		if (nr < nr_merged_lower)
+			seq_show_option(m, "lowerdir+", lowerdirs[nr]);
+		else
+			seq_show_option(m, "datadir+", lowerdirs[nr]);
+	}
 	if (ofs->config.upperdir) {
 		seq_show_option(m, "upperdir", ofs->config.upperdir);
 		seq_show_option(m, "workdir", ofs->config.workdir);
Signature · 1 line
2.34.1
author maintainer
#7 lore raw

Re: [PATCH 4/4] ovl: add support for appending lowerdirs one by one

Quoted 152 lines · Amir Goldstein

Add new mount options lowerdir+ and datadir+ that can be used to add layers to lower layers stack one by one.

Unlike the legacy lowerdir mount option, special characters (i.e. colons and cammas) are not unescaped with these new mount options.

The new mount options can be repeated to compose a large stack of lower layers, but they may not be mixed with the lagacy lowerdir mount option, because for displaying lower layers in mountinfo, we do not want to mix escaped with unescaped lower layers path syntax.

Similar to data-only layer rules with the lowerdir mount option, the datadir+ option must follow at least one lowerdir+ option and the lowerdir+ option must not follow the datadir+ option.

If the legacy lowerdir mount option follows lowerdir+ and datadir+ mount options, it overrides them. Sepcifically, calling:

  fsconfig(FSCONFIG_SET_STRING, "lowerdir", "", 0);

can be used to reset previously setup lower layers.

---
 fs/overlayfs/params.c | 78 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 76 insertions(+), 2 deletions(-)
fs/overlayfs/params.c +55 -1
index 9a9238eac730..1c390e93d060 100644
@@ -45,6 +45,8 @@ MODULE_PARM_DESC(metacopy,
 
 enum ovl_opt {
        Opt_lowerdir,
+       Opt_lowerdir_add,
+       Opt_datadir_add,
        Opt_upperdir,
        Opt_workdir,
        Opt_default_permissions,
@@ -140,8 +142,11 @@ static int ovl_verity_mode_def(void)
 #define fsparam_string_empty(NAME, OPT) \
        __fsparam(fs_param_is_string, NAME, OPT, fs_param_can_be_empty, NULL)
 
+
 const struct fs_parameter_spec ovl_parameter_spec[] = {
        fsparam_string_empty("lowerdir",    Opt_lowerdir),
+       fsparam_string("lowerdir+",         Opt_lowerdir_add),
+       fsparam_string("datadir+",          Opt_datadir_add),
        fsparam_string("upperdir",          Opt_upperdir),
        fsparam_string("workdir",           Opt_workdir),
        fsparam_flag("default_permissions", Opt_default_permissions),
@@ -273,6 +278,8 @@ static int ovl_mount_dir(const char *name, struct path *path)
 static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
                               enum ovl_opt layer, const char *name, bool upper)
 {
+       struct ovl_fs_context *ctx = fc->fs_private;
+
        if (ovl_dentry_weird(path->dentry))
                return invalfc(fc, "filesystem on %s not supported", name);
 
@@ -289,16 +296,44 @@ static int ovl_mount_dir_check(struct fs_context *fc, const struct path *path,
                        return invalfc(fc, "filesystem not supported as %s", name);
                if (__mnt_is_readonly(path->mnt))
                        return invalfc(fc, "%s is read-only", name);
+       } else {
+               if (ctx->lowerdir_all)
+                       return invalfc(fc, "%s cannot follow lowerdir mount option", name);
+               if (ctx->nr_data && layer == Opt_lowerdir_add)
+                       return invalfc(fc, "regular lower layers cannot follow data layers");
+               if (ctx->nr == OVL_MAX_STACK)
+                       return invalfc(fc, "too many lower directories, limit is %d",
+                                      OVL_MAX_STACK);
        }
        return 0;
 }
 
+static int ovl_ctx_realloc_lower(struct fs_context *fc)
+{
+       struct ovl_fs_context *ctx = fc->fs_private;
+       struct ovl_fs_context_layer *l;
+       size_t nr;
+
+       if (ctx->nr < ctx->capacity)
+               return 0;
+
+       nr = min(max(4096 / sizeof(*l), ctx->capacity * 2), (size_t) OVL_MAX_STACK);
+       l = krealloc_array(ctx->lower, nr, sizeof(*l), GFP_KERNEL_ACCOUNT);
+       if (!l)
+               return -ENOMEM;
+
+       ctx->lower = l;
+       ctx->capacity = nr;
+       return 0;
+}
+
 static void ovl_add_layer(struct fs_context *fc, enum ovl_opt layer,
                         struct path *path, char **pname)
 {
        struct ovl_fs *ofs = fc->s_fs_info;
        struct ovl_config *config = &ofs->config;
        struct ovl_fs_context *ctx = fc->fs_private;
+       struct ovl_fs_context_layer *l;
 
        switch (layer) {
        case Opt_workdir:
@@ -309,6 +344,16 @@ static void ovl_add_layer(struct fs_context *fc, enum ovl_opt layer,
                swap(config->upperdir, *pname);
                swap(ctx->upper, *path);
                break;
+       case Opt_datadir_add:
+               ctx->nr_data++;
+               fallthrough;
+       case Opt_lowerdir_add:
+               WARN_ON(ctx->nr >= ctx->capacity);
+               l = &ctx->lower[ctx->nr++];
+               memset(l, 0, sizeof(*l));
+               swap(l->name, *pname);
+               swap(l->path, *path);
+               break;
        default:
                WARN_ON(1);
        }
@@ -324,7 +369,10 @@ static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
        if (!name)
                return -ENOMEM;
 
-       err = ovl_mount_dir(name, &path);
+       if (upper)
+               err = ovl_mount_dir(name, &path);
+       else
+               err = ovl_mount_dir_noesc(name, &path);
        if (err)
                goto out_free;
 
@@ -332,6 +380,12 @@ static int ovl_parse_layer(struct fs_context *fc, struct fs_parameter *param,
        if (err)
                goto out_put;
 
+       if (!upper) {
+               err = ovl_ctx_realloc_lower(fc);
+               if (err)
+                       goto out_put;
+       }
+
        /* Store the user provided path string in ctx to show in mountinfo */
        ovl_add_layer(fc, layer, &path, &name);
 

fs/overlayfs/params.c static int ovl_parse_param(struct fs_context *fc, struct fs_parameter *param) · line 577

Quoted 10 lines
fs/overlayfs/params.c +4 -0
@@ -514,6 +568,10 @@ static int ovl_parse_param(struct fs_context *fc, struct fs_parameter *param)
        case Opt_lowerdir:
                err = ovl_parse_param_lowerdir(param->string, fc);
                break;
+       case Opt_lowerdir_add:
+       case Opt_datadir_add:
+               err = ovl_parse_layer(fc, param, opt, false);
+               break;
        case Opt_upperdir:
        case Opt_workdir:
                err = ovl_parse_layer(fc, param, opt, true);

Sorry, ovl_parse_layer() doesn't need bool upper arg. Your POC didn't have it.

I removed that and pushed to overlayfs-next.

Thanks,
Amir.

maintainer
#8 lore raw
In reply to Amir Goldstein
Quoted 5 lines · Amir Goldstein

Miklos,

As discussed, here are the patches for the new mount options.

Quoted 7 lines

- Only string format is supported
- Legacy lowerdir= cannot be mixed with new lowerdir+,datadir+
- lowerdir+,datadir+ are not escaped
- lowerdir,upperdir,workdir are escaped as always

I did not find a good reason to change escaping of upperdir,workdir. We can skip escaping when we add support for path format.

Looks good, other than the minor error reporting issue.

Thanks,
Miklos

author maintainer
#9 lore raw
Quoted 12 lines · Miklos Szeredi
Quoted 10 lines · Amir Goldstein

Miklos,

As discussed, here are the patches for the new mount options.

- Only string format is supported
- Legacy lowerdir= cannot be mixed with new lowerdir+,datadir+
- lowerdir+,datadir+ are not escaped
- lowerdir,upperdir,workdir are escaped as always

Quoted 5 lines
Quoted 2 lines

I did not find a good reason to change escaping of upperdir,workdir. We can skip escaping when we add support for path format.

Looks good, other than the minor error reporting issue.

Pushed fix to overlayfs-next and added some documentation.

Thanks,
Amir.