ovl: fix regression in showing lowerdir mount option
Subsystems (3)
Amir Goldstein
maintainerPATCH 1/1 ovl: fix regression in showing lowerdir mount option
Before commit b36a5780cb44 ("ovl: modify layer parameter parsing"), spaces and commas in lowerdir mount option value used to be escaped using seq_show_option().
In current upstream, when lowerdir value has a space, it is not escaped in /proc/mounts, e.g.:
none /mnt overlay rw,relatime,lowerdir=l l,upperdir=u,workdir=w 0 0
which results in broken output of the mount utility:
none on /mnt type overlay (rw,relatime,lowerdir=l)
Store the original lowerdir mount options before unescaping and show them using the same escaping used for seq_show_option() in addition to escaping the colon separator character.
--- fs/overlayfs/params.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-)
fs/overlayfs/params.c +23 -15
@@ -164,7 +164,8 @@ static ssize_t ovl_parse_param_split_lowerdirs(char *str)for (s = d = str;; s++, d++) {if (*s =='\\' ) { - s++; +/* keep esc chars in split lowerdir */ + *d++ = *s++; }else if (*s ==':' ) {bool next_colon = (*(s +1 ) ==':' );
@@ -239,7 +240,7 @@ static void ovl_unescape(char *s) } } -static int ovl_mount_dir (const char *name,struct path *path) +static int ovl_mount_dir (const char *name,struct path *path,bool upper) {int err = -ENOMEM ;char *tmp =kstrdup (name,GFP_KERNEL );
@@ -248,7 +249,7 @@ static int ovl_mount_dir(const char *name, struct path *path)ovl_unescape (tmp); err =ovl_mount_dir_noesc (tmp, path); -if (!err && path->dentry->d_flags &DCACHE_OP_REAL ) { +if (!err && upper && path->dentry->d_flags &DCACHE_OP_REAL ) {pr_err ("filesystem on '%s' not supported as upperdir\n" , tmp);path_put_init (path);
@@ -269,7 +270,7 @@ static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,struct path path;char *dup; - err =ovl_mount_dir (name, &path); + err =ovl_mount_dir (name, &path, true);if (err)return err;
@@ -472,7 +473,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) l = &ctx->lower[nr];memset (l,0 ,sizeof (*l)); - err =ovl_mount_dir_noesc (dup_iter, &l->path); + err =ovl_mount_dir (dup_iter, &l->path, false);if (err) goto out_put;
@@ -950,16 +951,23 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry)struct super_block *sb = dentry->d_sb;struct ovl_fs *ofs =OVL_FS (sb);size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer; -char **lowerdatadirs = &ofs->config.lowerdirs[nr_merged_lower]; - -/* lowerdirs[] starts from offset 1 */ -seq_printf (m,",lowerdir=%s" , ofs->config.lowerdirs[1 ]); -/* dump regular lower layers */ -for (nr =2 ; nr < nr_merged_lower; nr++) -seq_printf (m,":%s" , ofs->config.lowerdirs[nr]); -/* dump data lower layers */ -for (nr =0 ; nr < ofs->numdatalayer; nr++) -seq_printf (m,"::%s" , lowerdatadirs[nr]); + +/* +* lowerdirs[] starts from offset 1, then +* >= 0 regular lower layers prefixed with : and +* >= 0 data-only lower layers prefixed with :: +* +* we need to escase comma and space like seq_show_option() does and +* we also need to escape the colon separator from lowerdir paths. +*/ +seq_puts (m,",lowerdir=" ); +for (nr =1 ; nr < ofs->numlayer; nr++) { +if (nr >1 ) +seq_putc (m,':' ); +if (nr >= nr_merged_lower) +seq_putc (m,':' ); +seq_escape (m, ofs->config.lowerdirs[nr],":,= \t\n\\" ); + }if (ofs->config.upperdir) {seq_show_option (m,"upperdir" , ofs->config.upperdir);seq_show_option (m,"workdir" , ofs->config.workdir);
Signature · 1 line
2.34.1
Christian Brauner
maintainerQuoted 13 lines · Amir Goldstein
Before commit
b36a5780cb44("ovl: modify layer parameter parsing"), spaces and commas in lowerdir mount option value used to be escaped usingseq_show_option().In current upstream, when lowerdir value has a space, it is not escaped in
/proc/mounts, e.g.:none /mnt overlay rw,relatime,lowerdir=l l,upperdir=u,workdir=w 0 0
which results in broken output of the mount utility:
none on /mnt type overlay (rw,relatime,lowerdir=l)
Quoted 7 lines
Store the original lowerdir mount options before unescaping and show them using the same escaping used for
seq_show_option()in addition to escaping the colon separator character.Fixes b36a5780cb44 ("ovl: modify layer parameter parsing")Signed-off-by Amir Goldstein <[email protected]>---
Seems good to me.
Amir Goldstein
maintainerQuoted 19 lines · Christian Brauner
Quoted 17 lines · Amir Goldstein
Before commit
b36a5780cb44("ovl: modify layer parameter parsing"), spaces and commas in lowerdir mount option value used to be escaped usingseq_show_option().In current upstream, when lowerdir value has a space, it is not escaped in
/proc/mounts, e.g.:none /mnt overlay rw,relatime,lowerdir=l l,upperdir=u,workdir=w 0 0
which results in broken output of the mount utility:
none on /mnt type overlay (rw,relatime,lowerdir=l)
Store the original lowerdir mount options before unescaping and show them using the same escaping used for
seq_show_option()in addition to escaping the colon separator character.
Quoted 5 lines
Quoted 3 lines
Fixes b36a5780cb44 ("ovl: modify layer parameter parsing")Signed-off-by Amir Goldstein <[email protected]>---
Seems good to me.
FYI, test pushed to
https://github.com/amir73il/xfstests/commits/overlayfs-devel
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 94 lines · Amir Goldstein
Before commit
b36a5780cb44("ovl: modify layer parameter parsing"), spaces and commas in lowerdir mount option value used to be escaped usingseq_show_option().In current upstream, when lowerdir value has a space, it is not escaped in
/proc/mounts, e.g.:none /mnt overlay rw,relatime,lowerdir=l l,upperdir=u,workdir=w 0 0
which results in broken output of the mount utility:
none on /mnt type overlay (rw,relatime,lowerdir=l)
Store the original lowerdir mount options before unescaping and show them using the same escaping used for
seq_show_option()in addition to escaping the colon separator character.Fixes b36a5780cb44 ("ovl: modify layer parameter parsing")Signed-off-by Amir Goldstein <[email protected]>--- fs/overlayfs/params.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-)fs/overlayfs/params.c +12
-15@@ -164,7 +164,8 @@ static ssize_t ovl_parse_param_split_lowerdirs(char *str)for (s = d = str;; s++, d++) {if (*s =='\\' ) { - s++; +/* keep esc chars in split lowerdir */ + *d++ = *s++; }else if (*s ==':' ) {bool next_colon = (*(s +1 ) ==':' );@@ -239,7 +240,7 @@ static void ovl_unescape(char *s) } } -static int ovl_mount_dir (const char *name,struct path *path) +static int ovl_mount_dir (const char *name,struct path *path,bool upper) {int err = -ENOMEM ;char *tmp =kstrdup (name,GFP_KERNEL );@@ -248,7 +249,7 @@ static int ovl_mount_dir(const char *name, struct path *path)ovl_unescape (tmp); err =ovl_mount_dir_noesc (tmp, path); -if (!err && path->dentry->d_flags &DCACHE_OP_REAL ) { +if (!err && upper && path->dentry->d_flags &DCACHE_OP_REAL ) {pr_err ("filesystem on '%s' not supported as upperdir\n" , tmp);path_put_init (path);@@ -269,7 +270,7 @@ static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,struct path path;char *dup; - err =ovl_mount_dir (name, &path); + err =ovl_mount_dir (name, &path, true);if (err)return err;@@ -472,7 +473,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) l = &ctx->lower[nr];memset (l,0 ,sizeof (*l)); - err =ovl_mount_dir_noesc (dup_iter, &l->path); + err =ovl_mount_dir (dup_iter, &l->path, false);if (err) goto out_put;@@ -950,16 +951,23 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry)struct super_block *sb = dentry->d_sb;struct ovl_fs *ofs =OVL_FS (sb);size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer; -char **lowerdatadirs = &ofs->config.lowerdirs[nr_merged_lower]; - -/* lowerdirs[] starts from offset 1 */ -seq_printf (m,",lowerdir=%s" , ofs->config.lowerdirs[1 ]); -/* dump regular lower layers */ -for (nr =2 ; nr < nr_merged_lower; nr++) -seq_printf (m,":%s" , ofs->config.lowerdirs[nr]); -/* dump data lower layers */ -for (nr =0 ; nr < ofs->numdatalayer; nr++) -seq_printf (m,"::%s" , lowerdatadirs[nr]); + +/* +* lowerdirs[] starts from offset 1, then +* >= 0 regular lower layers prefixed with : and +* >= 0 data-only lower layers prefixed with :: +*
fs/overlayfs/params.c int ovl_show_options(struct seq_file *m, struct dentry *dentry) · line 969
Quoted 10 lines
fs/overlayfs/params.c +10
-0+* we need to escase comma and space like seq_show_option() does and +* we also need to escape the colon separator from lowerdir paths. +*/ +seq_puts (m,",lowerdir=" ); +for (nr =1 ; nr < ofs->numlayer; nr++) { +if (nr >1 ) +seq_putc (m,':' ); +if (nr >= nr_merged_lower) +seq_putc (m,':' ); +seq_escape (m, ofs->config.lowerdirs[nr],":,= \t\n\\" );
This is too eager. Just need to escape what seq_show_option() would
escape, which is comma and whitespace. The '=' is not need escaped
in values only in keys (and that likely never triggers). Colons should have stayed escaped as "\:", so no point in adding another level of escape.
Yes, this two level escape is pretty confusing, considering that commas are escaped on both levels if using the old API. When using the new API commas need not be escaped, but can be, since the same
unescaping is done. Not a serious issue as backslash in filenames is
basically nonexistent, but an inconsistency nonetheless.
Following choices exist:
1) should the redundant escaping be left in mountinfo?
2) should FSCONFIG_SET_STRING accept escaped commas?
3) should unescaped commas on FSCONFIG_SET_STRING (and FSCONFIG_SET_PATH) be double escaped in mountinfo?
Currently it's yes, yes, no. I'm fine with leaving things as they are, but at least the documentation should be clear on what should happen.
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 102 lines · Miklos Szeredi
Quoted 100 lines · Amir Goldstein
Before commit
b36a5780cb44("ovl: modify layer parameter parsing"), spaces and commas in lowerdir mount option value used to be escaped usingseq_show_option().In current upstream, when lowerdir value has a space, it is not escaped in
/proc/mounts, e.g.:none /mnt overlay rw,relatime,lowerdir=l l,upperdir=u,workdir=w 0 0
which results in broken output of the mount utility:
none on /mnt type overlay (rw,relatime,lowerdir=l)
Store the original lowerdir mount options before unescaping and show them using the same escaping used for
seq_show_option()in addition to escaping the colon separator character.Fixes b36a5780cb44 ("ovl: modify layer parameter parsing")Signed-off-by Amir Goldstein <[email protected]>--- fs/overlayfs/params.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-)fs/overlayfs/params.c +18
-15@@ -164,7 +164,8 @@ static ssize_t ovl_parse_param_split_lowerdirs(char *str)for (s = d = str;; s++, d++) {if (*s =='\\' ) { - s++; +/* keep esc chars in split lowerdir */ + *d++ = *s++; }else if (*s ==':' ) {bool next_colon = (*(s +1 ) ==':' );@@ -239,7 +240,7 @@ static void ovl_unescape(char *s) } } -static int ovl_mount_dir (const char *name,struct path *path) +static int ovl_mount_dir (const char *name,struct path *path,bool upper) {int err = -ENOMEM ;char *tmp =kstrdup (name,GFP_KERNEL );@@ -248,7 +249,7 @@ static int ovl_mount_dir(const char *name, struct path *path)ovl_unescape (tmp); err =ovl_mount_dir_noesc (tmp, path); -if (!err && path->dentry->d_flags &DCACHE_OP_REAL ) { +if (!err && upper && path->dentry->d_flags &DCACHE_OP_REAL ) {pr_err ("filesystem on '%s' not supported as upperdir\n" , tmp);path_put_init (path);@@ -269,7 +270,7 @@ static int ovl_parse_param_upperdir(const char *name, struct fs_context *fc,struct path path;char *dup; - err =ovl_mount_dir (name, &path); + err =ovl_mount_dir (name, &path, true);if (err)return err;@@ -472,7 +473,7 @@ static int ovl_parse_param_lowerdir(const char *name, struct fs_context *fc) l = &ctx->lower[nr];memset (l,0 ,sizeof (*l)); - err =ovl_mount_dir_noesc (dup_iter, &l->path); + err =ovl_mount_dir (dup_iter, &l->path, false);if (err) goto out_put;@@ -950,16 +951,23 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry)struct super_block *sb = dentry->d_sb;struct ovl_fs *ofs =OVL_FS (sb);size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer; -char **lowerdatadirs = &ofs->config.lowerdirs[nr_merged_lower]; - -/* lowerdirs[] starts from offset 1 */ -seq_printf (m,",lowerdir=%s" , ofs->config.lowerdirs[1 ]); -/* dump regular lower layers */ -for (nr =2 ; nr < nr_merged_lower; nr++) -seq_printf (m,":%s" , ofs->config.lowerdirs[nr]); -/* dump data lower layers */ -for (nr =0 ; nr < ofs->numdatalayer; nr++) -seq_printf (m,"::%s" , lowerdatadirs[nr]); + +/* +* lowerdirs[] starts from offset 1, then +* >= 0 regular lower layers prefixed with : and +* >= 0 data-only lower layers prefixed with :: +* +* we need to escase comma and space like seq_show_option() does and +* we also need to escape the colon separator from lowerdir paths. +*/ +seq_puts (m,",lowerdir=" ); +for (nr =1 ; nr < ofs->numlayer; nr++) { +if (nr >1 )
Quoted 8 lines
fs/overlayfs/params.c int ovl_show_options(struct seq_file *m, struct dentry *dentry) · line 969
Quoted 4 lines
fs/overlayfs/params.c +4
-0+seq_putc (m,':' ); +if (nr >= nr_merged_lower) +seq_putc (m,':' ); +seq_escape (m, ofs->config.lowerdirs[nr],":,= \t\n\\" );This is too eager. Just need to escape what
seq_show_option()would
escape, which is comma and whitespace. The '=' is not need escaped
in values only in keys (and that likely never triggers).
Right. I will remove =.
Quoted 2 lines
Colons should have stayed escaped as "\:", so no point in adding another level of escape.
Yeh, unless the colon are not escaped because they were set via new api. In this case, I would like to escape them in mountinfo (e.g. "a:b" => "a\072b"). This way, libmount would be able to parse mountinfo correctly in the future and learn how to iterate lowerdirs, even before fsinfo().
I am not even sure if fsinfo() is going to be able to iterate lowerdirs, so escaping colons may be needed there as well.
Quoted 6 lines
Yes, this two level escape is pretty confusing, considering that commas are escaped on both levels if using the old API. When using the new API commas need not be escaped, but can be, since the same
unescaping is done. Not a serious issue as backslash in filenames is
basically nonexistent, but an inconsistency nonetheless.
In general, I think we should be conservative and if escaping in mountinfo doesn't hurt we should not change it.
Applications should consume mountinfo via libmount and libmount already unescapes the seq_show_option() format.
Quoted 4 lines
Following choices exist:
1) should the redundant escaping be left in mountinfo?
Absolutely yes. I don't think it is redundant at all. It may be redundant in fsinfo(), when lowerdirs can be iterated? but mountinfo output is a single monolothic string even if lowerdirs were set individually by new mount api.
Quoted 3 lines
2) should
FSCONFIG_SET_STRINGaccept escaped commas?
Well, it already does.
If you are considering to change that retroactively then my argument is that IMO userspace needs to be able to pass in \: in lowerdirs and see it in mountinto (and mount command output) as long as libmount does not know how to split lowerdirs by itself. Otherwise, replaying the options from mountinfo into libmount will not work correctly.
Quoted 3 lines
3) should unescaped commas on
FSCONFIG_SET_STRING(andFSCONFIG_SET_PATH) be double escaped in mountinfo?
Too much escaping in the sentence. I could not parse it ;)
For example, if "a:b" is set via FSCONFIG_SET_{STRING,PATH}
IMO mount info should output "a\072b", for the aforementioned reasons. If by "double escaped" you meant "a\:b" or "a\134\072b", then I don't think this is necessary.
Quoted 3 lines
Currently it's yes, yes, no. I'm fine with leaving things as they are, but at least the documentation should be clear on what should happen.
OK.
How is that:
Documentation/filesystems/overlayfs.rst +0 -0
@@ -339,6 +339,18 @@ The specified lower directories will be stacked
beginning from the
rightmost one and going left. In the above example lower1 will be the top, lower2 the middle and lower3 the bottom layer.
+Note: directory names containing colons can be provided as lower layer by +escaping the colons with a single backslash. For example:
+
+ mount -t overlay overlay -olowerdir=/a\:lower\:\:dir /merged +
+Since kernel version v6.5, directory names containing colons can also +be provided as lower layer using the fsconfig syscall from new mount api:
+
+ fsconfig(fs_fd, FSCONFIG_SET_STRING, "lowerdir", "/a:lower::dir", 0); +
+In the latter case, colons in lower layer directory names will be escaped +as an octal characters (\072) when displayed in /proc/self/mountinfo.
The wording of the last sentence above is somewhat legalish -
De facto, since v6.5,y, we will escape ":" to "\072" no matter via which api we got it and regardless of whether it is also escaped with "\:" or not. but we only need to commit to this rule for unescaped colons via new mount api.
I will add this documentation to the fix patch and retain escaping of ":".
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 9 lines · Amir Goldstein
Quoted 3 lines · Miklos Szeredi
This is too eager. Just need to escape what
seq_show_option()would
escape, which is comma and whitespace. The '=' is not need escaped
in values only in keys (and that likely never triggers).Right. I will remove =.
Quoted 7 lines
Quoted 2 lines
Colons should have stayed escaped as "\:", so no point in adding another level of escape.
Yeh, unless the colon are not escaped because they were set via new api. In this case, I would like to escape them in mountinfo (e.g. "a:b" => "a\072b"). This way, libmount would be able to parse mountinfo correctly in the future and learn how to iterate lowerdirs, even before
fsinfo().
I'm not sure that will work. libmount will just remove the \072 escaping from the whole lowerdir= option and we'll be left with the
the unescaped sequence. I think it's better to add the old "\:"
escaping in case it was not added originally.
Quoted 2 lines
I am not even sure if
fsinfo()is going to be able to iterate lowerdirs, so escaping colons may be needed there as well.
Right.
Quoted 10 lines
Quoted 5 lines
Yes, this two level escape is pretty confusing, considering that commas are escaped on both levels if using the old API. When using the new API commas need not be escaped, but can be, since the same
unescaping is done. Not a serious issue as backslash in filenames is
basically nonexistent, but an inconsistency nonetheless.In general, I think we should be conservative and if escaping in mountinfo doesn't hurt we should not change it.
Applications should consume mountinfo via libmount and libmount already unescapes theseq_show_option()format.
Yes, it's like a network stack with various levels of encoding and decoding. The escaping on the mountinfo level is there so that fields
and options can properly be separated. The escaping on the old mount
API had a similar purpose but using a different encoding. Shouldn't
we make this consistent, so that only one level of escaping is done and at the right level?
Quoted 8 lines
Quoted 3 lines
Following choices exist:
1) should the redundant escaping be left in mountinfo?
Absolutely yes. I don't think it is redundant at all. It may be redundant in
fsinfo(), when lowerdirs can be iterated? but mountinfo output is a single monolothic string even if lowerdirs were set individually by new mount api.
No. What I mean by redundant is that a layer named "x,y" is encoded
as e.g. upperdir=x\,y when mounting on the old API. Mountinfo it
will look like "upperdir=x\134\054y". The \134 is totally redundant and not needed to separate the options, just remains there historically from the input encoding.
Quoted 10 lines
Quoted 2 lines
2) should
FSCONFIG_SET_STRINGaccept escaped commas?Well, it already does.
If you are considering to change that retroactively then my argument is that IMO userspace needs to be able to pass in \: in lowerdirs and see it in mountinto (and mount command output) as long as libmount does not know how to split lowerdirs by itself. Otherwise, replaying the options from mountinfo into libmount will not work correctly.
If we are not strict about the encoding/decoding rules then something can break anyway.
Quoted 10 lines
Quoted 4 lines
3) should unescaped commas on
FSCONFIG_SET_STRING(andFSCONFIG_SET_PATH) be double escaped in mountinfo?Too much escaping in the sentence. I could not parse it ;)
For example, if "a:b" is set via FSCONFIG_SET_{STRING,PATH}
IMO mount info should output "a\072b", for the aforementioned reasons. If by "double escaped" you meant "a\:b" or "a\134\072b", then I don't think this is necessary.
Agree double escaping being unncessary, not sure I agree about needing to escape ':' on the mountinfo level.
Quoted 28 lines
Quoted 4 lines
Currently it's yes, yes, no. I'm fine with leaving things as they are, but at least the documentation should be clear on what should happen.
OK.
How is that:Documentation/filesystems/overlayfs.rst +0
-0@@ -339,6 +339,18 @@ The specified lower directories will be stackedbeginning from the
rightmost one and going left. In the above example lower1 will be the top, lower2 the middle and lower3 the bottom layer.Documentation/filesystems/overlayfs.rst +12
-0+Note: directory names containing colons can be provided as lower layer by +escaping the colons with a single backslash. For example: + + mount -t overlay overlay -olowerdir=/a\:lower\:\:dir /merged + +Since kernel version v6.5, directory names containing colons can also +be provided as lower layer using the fsconfig syscall from new mount api: + + fsconfig(fs_fd, FSCONFIG_SET_STRING, "lowerdir", "/a:lower::dir", 0); + +In the latter case, colons in lower layer directory names will be escaped +as an octal characters (\072) when displayed in /proc/self/mountinfo.
Quoted 4 lines
The wording of the last sentence above is somewhat legalish -
De facto, since v6.5,y, we will escape ":" to "\072" no matter via which api we got it and regardless of whether it is also escaped with "\:" or not. but we only need to commit to this rule for unescaped colons via new mount api.
See my above argument against escaping colons on the mountinfo level.
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 19 lines · Miklos Szeredi
Quoted 9 lines · Amir Goldstein
Quoted 3 lines · Miklos Szeredi
This is too eager. Just need to escape what
seq_show_option()would
escape, which is comma and whitespace. The '=' is not need escaped
in values only in keys (and that likely never triggers).Right. I will remove =.
Quoted 7 lines
Quoted 2 lines
Colons should have stayed escaped as "\:", so no point in adding another level of escape.
Yeh, unless the colon are not escaped because they were set via new api. In this case, I would like to escape them in mountinfo (e.g. "a:b" => "a\072b"). This way, libmount would be able to parse mountinfo correctly in the future and learn how to iterate lowerdirs, even before
fsinfo().
Quoted 5 lines
I'm not sure that will work. libmount will just remove the \072 escaping from the whole lowerdir= option and we'll be left with the
the unescaped sequence. I think it's better to add the old "\:"
escaping in case it was not added originally.
It is better to add "\:" if we want to be super nice and abide by the unspoken rule that the mount options parsed by libmount can be fed into mount(2) syscall or back into libmount to get the same overlayfs config.
But I don't think it is a must because this rule is for overlayfs created with mount(2) syscall or libmount, which knows how to split commas and use new mount api, but it doesn't know how to split lowerdir colons.
If in the future, libmount will learn how to split lowerdir colons and use new mount api to configure them one by one, then in the same future, libmount will be able to distinguish between single escaped colons (\072) and unescaped colons, and we do not need to add "\:" for making this possible.
Quoted 32 lines
Quoted 2 lines
I am not even sure if
fsinfo()is going to be able to iterate lowerdirs, so escaping colons may be needed there as well.Right.
Quoted 10 lines
Quoted 5 lines
Yes, this two level escape is pretty confusing, considering that commas are escaped on both levels if using the old API. When using the new API commas need not be escaped, but can be, since the same
unescaping is done. Not a serious issue as backslash in filenames is
basically nonexistent, but an inconsistency nonetheless.In general, I think we should be conservative and if escaping in mountinfo doesn't hurt we should not change it.
Applications should consume mountinfo via libmount and libmount already unescapes theseq_show_option()format.Yes, it's like a network stack with various levels of encoding and decoding. The escaping on the mountinfo level is there so that fields
and options can properly be separated. The escaping on the old mount
API had a similar purpose but using a different encoding. Shouldn't
we make this consistent, so that only one level of escaping is done and at the right level?Quoted 8 lines
Quoted 3 lines
Following choices exist:
1) should the redundant escaping be left in mountinfo?
Absolutely yes. I don't think it is redundant at all. It may be redundant in
fsinfo(), when lowerdirs can be iterated? but mountinfo output is a single monolothic string even if lowerdirs were set individually by new mount api.
Quoted 6 lines
No. What I mean by redundant is that a layer named "x,y" is encoded
as e.g. upperdir=x\,y when mounting on the old API. Mountinfo it
will look like "upperdir=x\134\054y". The \134 is totally redundant and not needed to separate the options, just remains there historically from the input encoding.
It is not only historical.
In the present, even with most recent libmount, the mount(8) tool e.g. mount -t overlay, displays the options (... upperdir=x\,y) and then the user can replay those options.
It is true that libmount could have displayed "upperdir=x\054y" as (... upperdir="x,y"), which is the escaping for commas that libmount supports, but:
1. libmount does not do that
2. those options could be fed back into libmount but not directly to
mount(2)
Quoted 55 lines
Quoted 10 lines
Quoted 2 lines
2) should
FSCONFIG_SET_STRINGaccept escaped commas?Well, it already does.
If you are considering to change that retroactively then my argument is that IMO userspace needs to be able to pass in \: in lowerdirs and see it in mountinto (and mount command output) as long as libmount does not know how to split lowerdirs by itself. Otherwise, replaying the options from mountinfo into libmount will not work correctly.If we are not strict about the encoding/decoding rules then something can break anyway.
Quoted 10 lines
Quoted 4 lines
3) should unescaped commas on
FSCONFIG_SET_STRING(andFSCONFIG_SET_PATH) be double escaped in mountinfo?Too much escaping in the sentence. I could not parse it ;)
For example, if "a:b" is set via FSCONFIG_SET_{STRING,PATH}
IMO mount info should output "a\072b", for the aforementioned reasons. If by "double escaped" you meant "a\:b" or "a\134\072b", then I don't think this is necessary.Agree double escaping being unncessary, not sure I agree about needing to escape ':' on the mountinfo level.
Quoted 28 lines
Quoted 4 lines
Currently it's yes, yes, no. I'm fine with leaving things as they are, but at least the documentation should be clear on what should happen.
OK.
How is that:Documentation/filesystems/overlayfs.rst +0
-0@@ -339,6 +339,18 @@ The specified lower directories will be stackedbeginning from the
rightmost one and going left. In the above example lower1 will be the top, lower2 the middle and lower3 the bottom layer.Documentation/filesystems/overlayfs.rst +12
-0+Note: directory names containing colons can be provided as lower layer by +escaping the colons with a single backslash. For example: + + mount -t overlay overlay -olowerdir=/a\:lower\:\:dir /merged + +Since kernel version v6.5, directory names containing colons can also +be provided as lower layer using the fsconfig syscall from new mount api: + + fsconfig(fs_fd, FSCONFIG_SET_STRING, "lowerdir", "/a:lower::dir", 0); + +In the latter case, colons in lower layer directory names will be escaped +as an octal characters (\072) when displayed in /proc/self/mountinfo.
Quoted 7 lines
Quoted 4 lines
The wording of the last sentence above is somewhat legalish -
De facto, since v6.5,y, we will escape ":" to "\072" no matter via which api we got it and regardless of whether it is also escaped with "\:" or not. but we only need to commit to this rule for unescaped colons via new mount api.See my above argument against escaping colons on the mountinfo level.
See counter arguments.
My goal is to get as much of the parsing/escaping logic done by libmount going forward, so I'd rather not add new escaping logic to new use cases if we can avoid it.
At the moment, I would like to get some minimal fix to the 6.5 regression merged while 6.5.y is still getting fixes.
If we cannot reach consensus within this short timeframe, then I think that the smallest and least controversial fix is to escape colons once as \072 as the current patch does.
We can always remove this escaping later if we find that it is not needed no harm done. If you like, I can remove the commitment to mountinfo format from documentation for the fix patch?
Being extra nice and displaying "a\,b" and "a\:b" for new users that used new mount api explicitly to pass "a,b" and "a:b" is possible, but it is not a 6.5 regression fix, so we can take the time to decide if we want to do that or not.
OK?
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 6 lines · Amir Goldstein
Being extra nice and displaying "a\,b" and "a\:b" for new users that used new mount api explicitly to pass "a,b" and "a:b" is possible, but it is not a 6.5 regression fix, so we can take the time to decide if we want to do that or not.
OK?
Yeah, I'm mostly convinced by your arguments.
The other issue that has to be decided pretty quickly is whether to leave the append mode (starting lowerdir with ':') or remove it now and replace it in the next cycle with a mode that doesn't play games with separators (i.e. adding layers one at a time).
The only argument I can see for the current append is that it's possible to add multiple layers at once, but it's also redundant once we add the one-by-one append mode and I'm not convinced that it's worth the complexity and the mess with having to escape separators.
What do you think?
Thanks,
Miklos
Quoted 3 lines
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 17 lines · Miklos Szeredi
Quoted 6 lines · Amir Goldstein
Being extra nice and displaying "a\,b" and "a\:b" for new users that used new mount api explicitly to pass "a,b" and "a:b" is possible, but it is not a 6.5 regression fix, so we can take the time to decide if we want to do that or not.
OK?
Yeah, I'm mostly convinced by your arguments.
The other issue that has to be decided pretty quickly is whether to leave the append mode (starting lowerdir with ':') or remove it now and replace it in the next cycle with a mode that doesn't play games with separators (i.e. adding layers one at a time).
Quoted 6 lines
The only argument I can see for the current append is that it's possible to add multiple layers at once, but it's also redundant once we add the one-by-one append mode and I'm not convinced that it's worth the complexity and the mess with having to escape separators.
What do you think?
I think that we can leave the string-append mode, but disallow ':' and '\' within an appended string, so we only support adding lowerdirs one at a time and no support for special chars in file name, so escaping is moot.
This way, we do not complicate things and leave the functionality intact. Sure, it's going to be redundant once we add support for FSCONFIG_SET_PATH*, but the added code to strip the ":" or "::" prefix for FSCONFIG_SET_STRING is not really complicated.
I can add this patch if you agree (without all the possible code cleanup).
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 30 lines · Amir Goldstein
Quoted 17 lines · Miklos Szeredi
Quoted 6 lines · Amir Goldstein
Being extra nice and displaying "a\,b" and "a\:b" for new users that used new mount api explicitly to pass "a,b" and "a:b" is possible, but it is not a 6.5 regression fix, so we can take the time to decide if we want to do that or not.
OK?
Yeah, I'm mostly convinced by your arguments.
The other issue that has to be decided pretty quickly is whether to leave the append mode (starting lowerdir with ':') or remove it now and replace it in the next cycle with a mode that doesn't play games with separators (i.e. adding layers one at a time).
Quoted 6 lines
The only argument I can see for the current append is that it's possible to add multiple layers at once, but it's also redundant once we add the one-by-one append mode and I'm not convinced that it's worth the complexity and the mess with having to escape separators.
What do you think?
I think that we can leave the string-append mode, but disallow ':' and '\' within an appended string, so we only support adding lowerdirs one at a time and no support for special chars in file name, so escaping is moot.
Quoted 6 lines
This way, we do not complicate things and leave the functionality intact. Sure, it's going to be redundant once we add support for FSCONFIG_SET_PATH*, but the added code to strip the ":" or "::" prefix for
FSCONFIG_SET_STRINGis not really complicated.I can add this patch if you agree (without all the possible code cleanup).
So what I figured is that a separate key for the append interface is better, otherwise data only and plain lower layers can't be differentiated.
So my PoC is using "lowerdir+" and "datadir+" for this. I think this might be a good convention going forward if any other filesystem needs such append mode options.
If you can code this up quickly, that's good. I can have a go at it on Monday, but my PoC patch needs splitting up and so it's not ready for 6.6.
Thanks,
Miklos
Miklos Szeredi
maintainerQuoted 3 lines · Miklos Szeredi
If you can code this up quickly, that's good. I can have a go at it on Monday, but my PoC patch needs splitting up and so it's not ready for 6.6.
Attaching my current patch (against your 3 patches).
Thanks,
Miklos
fs/overlayfs/params.c +189 -279
@@ -43,8 +43,10 @@ module_param_named(metacopy, ovl_metacopMODULE_PARM_DESC (metacopy,"Default to on or off for the metadata only copy up feature" ); -enum { +enum ovl_opt { Opt_lowerdir, + Opt_lowerdir_add, + Opt_datadir_add, Opt_upperdir, Opt_workdir, Opt_default_permissions,
@@ -140,10 +142,13 @@ static int ovl_verity_mode_def(void)#define fsparam_string_empty (NAME ,OPT ) \ __fsparam(fs_param_is_string, NAME, OPT, fs_param_can_be_empty, NULL) +const struct fs_parameter_spec ovl_parameter_spec[] = {fsparam_string_empty ("lowerdir" , Opt_lowerdir), -fsparam_string ("upperdir" , Opt_upperdir), -fsparam_string ("workdir" , Opt_workdir), +fsparam_path ("lowerdir+" , Opt_lowerdir_add), +fsparam_path ("datadir+" , Opt_datadir_add), +fsparam_path ("upperdir" , Opt_upperdir), +fsparam_path ("workdir" , Opt_workdir),fsparam_flag ("default_permissions" , Opt_default_permissions),fsparam_enum ("redirect_dir" , Opt_redirect_dir, ovl_parameter_redirect_dir),fsparam_enum ("index" , Opt_index, ovl_parameter_bool),
@@ -225,36 +230,6 @@ static ssize_t ovl_parse_param_split_lowreturn nr_layers; } -static int ovl_mount_dir_noesc (const char *name,struct path *path) -{ -int err = -EINVAL ; - -if (!*name) { -pr_err ("empty lowerdir\n" ); - goto out; - } - err =kern_path (name,LOOKUP_FOLLOW , path); -if (err) { -pr_err ("failed to resolve '%s': %i\n" , name, err); - goto out; - } - err = -EINVAL ; -if (ovl_dentry_weird (path->dentry)) { -pr_err ("filesystem on '%s' not supported\n" , name); - goto out_put; - } -if (!d_is_dir (path->dentry)) { -pr_err ("'%s' not a directory\n" , name); - goto out_put; - } -return 0 ; - -out_put: -path_put_init (path); -out: -return err; -} -static void ovl_unescape (char *s) {char *d = s;
@@ -268,70 +243,179 @@ static void ovl_unescape(char *s) } } -static int ovl_mount_dir (const char *name,struct path *path,bool upper) +static int ovl_mount_dir_check (struct fs_context *fc,const struct path *path, +enum ovl_opt layer,const char *name) { -int err = -ENOMEM ; -char *tmp =kstrdup (name,GFP_KERNEL ); +struct ovl_fs_context *ctx = fc->fs_private; -if (tmp) { -ovl_unescape (tmp); - err =ovl_mount_dir_noesc (tmp, path); +if (ovl_dentry_weird (path->dentry)) +return invalfc (fc,"filesystem on %s not supported" , name); -if (!err && upper && path->dentry->d_flags &DCACHE_OP_REAL ) { -pr_err ("filesystem on '%s' not supported as upperdir\n" , - tmp); -path_put_init (path); - err = -EINVAL ; +if (!d_is_dir (path->dentry)) +return invalfc (fc,"%s is not a directory" , name); + +if (layer == Opt_upperdir || layer == Opt_workdir) { +if (path->dentry->d_flags &DCACHE_OP_REAL ) +return invalfc (fc,"filesystem not supported as %s" , name); +/* +* Check whether upper path is read-only here to report failures +* early. Don't forget to recheck when the superblock is created +* as the mount attributes could change. +*/ +if (__mnt_is_readonly (path->mnt)) +return invalfc (fc,"%s is read-only" , name); + }else { +if (ctx->nr ==OVL_MAX_STACK ) +return invalfc (fc,"too many lower directories, limit is %d" ,OVL_MAX_STACK ); +if (ctx->nr_data && layer == Opt_lowerdir_add) +return invalfc (fc,"regular lower layers cannot follow data layers" ); + } +return 0 ; +} + +static char *escape_colons (char *s,char *p) +{ +char *ret = s; + +for (;;) { +char c = *p++; +if (c !=':' ) { + *s++ = c; +if (!c) +return ret; + }else if (s +2 > p) { +return ERR_PTR (-ENAMETOOLONG ); + }else { + *s++ ='\\' ; + *s++ = c; } + } +} + +static char *ovl_layer_path (struct fs_context *fc,const struct path *path, +bool lower,const char *name) +{ +char *tmp, *p =NULL ; + + tmp =kmalloc (PATH_MAX ,GFP_KERNEL ); +if (tmp) { + p =d_path (path, tmp,PATH_MAX ); +if (!IS_ERR (p) && lower) + p =escape_colons (tmp, p); +if (!IS_ERR (p)) + p =kstrdup (p,GFP_KERNEL );kfree (tmp); } -return err; +if (IS_ERR (p)) +errorfc (fc,"failed to generate %s pathname" , name); + +return p ?:ERR_PTR (-ENOMEM ); } -static int ovl_parse_param_upperdir (const char *name,struct fs_context *fc, -bool workdir) +static int ovl_ctx_realloc_lower (struct fs_context *fc) +{ +struct ovl_fs_context *ctx = fc->fs_private; +struct ovl_fs_context_layer *l; +size_t nr; + +if (ctx->nr < ctx->capacity) +return 0 ; + + nr =min (max (4096 /sizeof (*l), ctx->capacity *2 ), (size_t )OVL_MAX_STACK ); + l =krealloc_array (ctx->lower, nr,sizeof (*l),GFP_KERNEL_ACCOUNT ); +if (!l) +return -ENOMEM ; + + ctx->lower = l; + ctx->capacity = nr; +return 0 ; +} + +static void ovl_add_layer (struct fs_context *fc,enum ovl_opt layer, +struct path *path,char **pp) { -int err;struct ovl_fs *ofs = fc->s_fs_info;struct ovl_config *config = &ofs->config;struct ovl_fs_context *ctx = fc->fs_private; +struct ovl_fs_context_layer *l; + +switch (layer) { +case Opt_workdir: +swap (config->workdir, *pp); +swap (ctx->work, *path); +break ; +case Opt_upperdir: +swap (config->upperdir, *pp); +swap (ctx->upper, *path); +break ; +case Opt_datadir_add: + ctx->nr_data++; + fallthrough; +case Opt_lowerdir_add: +WARN_ON (ctx->nr >= ctx->capacity); + l = &ctx->lower[ctx->nr++]; +memset (l,0 ,sizeof (*l)); +swap (l->name, *pp); +swap (l->path, *path); +break ; +default : +WARN_ON (1 ); + } +} + +static int ovl_parse_layer (struct fs_context *fc,struct fs_parameter *param, +enum ovl_opt layer) +{ +char *pathname =NULL , *tmp =kstrdup (param->string,GFP_KERNEL ); +bool lower = (layer == Opt_lowerdir_add || layer == Opt_datadir_add);struct path path; -char *dup; +int err; - err =ovl_mount_dir (name, &path, true); +if (param->type == fs_value_is_string) { +if (!tmp) +return -ENOMEM ; +/* look up with unescaped value */ +ovl_unescape (tmp); +swap (param->string, tmp); + } + err =fs_lookup_param (fc, param, false,LOOKUP_FOLLOW , &path); +if (param->type == fs_value_is_string) +swap (param->string, tmp); +kfree (tmp);if (err)return err; + err =ovl_mount_dir_check (fc, &path, layer, param->key); +if (err) + goto put_path; +/* -* Check whether upper path is read-only here to report failures -* early. Don't forget to recheck when the superblock is created -* as the mount attributes could change. +* If configuring with FSCONFIG_SET_PATH/FSCONFIG_SET_PATH_EMPTY, then +* need to generate a path for mount option display. */ -if (__mnt_is_readonly (path.mnt)) { -path_put (&path); -return -EINVAL ; - } +if (param->type == fs_value_is_string) +swap (param->string, pathname); - dup =kstrdup (name,GFP_KERNEL ); -if (!dup) { -path_put (&path); -return -ENOMEM ; - } - -if (workdir) { -kfree (config->workdir); - config->workdir = dup; -path_put (&ctx->work); - ctx->work = path; - }else { -kfree (config->upperdir); - config->upperdir = dup; -path_put (&ctx->upper); - ctx->upper = path; - } -return 0 ; +if (!pathname) { + pathname =ovl_layer_path (fc, &path, lower, param->key); + err =PTR_ERR (pathname); +if (IS_ERR (pathname)) + goto put_path; + +pr_info ("Generated pathname: <%s>\n" , pathname); + } + err =0 ; +if (lower) + err =ovl_ctx_realloc_lower (fc); +if (!err) +ovl_add_layer (fc, layer, &path, &pathname); +kfree (pathname); +put_path: +path_put (&path); +return err; } +static void ovl_parse_param_drop_lowerdir (struct ovl_fs_context *ctx) {for (size_t nr =0 ; nr < ctx->nr; nr++) {
@@ -344,225 +428,51 @@ static void ovl_parse_param_drop_lowerdi }/* -* Parse lowerdir= mount option: +* Parse lowerdir= mount option in old monolithic style: * * (1) lowerdir=/lower1:/lower2:/lower3::/data1::/data2 * Set "/lower1", "/lower2", and "/lower3" as lower layers and * "/data1" and "/data2" as data lower layers. Any existing lower * layers are replaced. -* (2) lowerdir=:/lower4 -* Append "/lower4" to current stack of lower layers. This requires -* that there already is at least one lower layer configured. -* (3) lowerdir=::/lower5 -* Append data "/lower5" as data lower layer. This requires that -* there's at least one regular lower layer present. */ -static int ovl_parse_param_lowerdir (const char *name,struct fs_context *fc) +static int ovl_parse_lower_layers (struct fs_context *fc, +struct fs_parameter *param) { -int err; +int err =0 ;struct ovl_fs_context *ctx = fc->fs_private; -struct ovl_fs_context_layer *l; -char *dup =NULL , *dup_iter; -ssize_t nr_lower =0 , nr =0 , nr_data =0 ; -bool append = false, data_layer = false; - -/* -* Ensure we're backwards compatible with mount(2) -* by allowing relative paths. -*/ - -/* drop all existing lower layers */ -if (!*name) { -ovl_parse_param_drop_lowerdir (ctx); -return 0 ; - } - -if (strncmp (name,"::" ,2 ) ==0 ) { -/* -* This is a data layer. -* There must be at least one regular lower layer -* specified. -*/ -if (ctx->nr ==0 ) { -pr_err ("data lower layers without regular lower layers not allowed" ); -return -EINVAL ; - } - -/* Skip the leading "::". */ - name +=2 ; - data_layer = true; -/* -* A data layer is automatically an append as there -* must've been at least one regular lower layer. -*/ - append = true; - }else if (*name ==':' ) { -/* -* This is a regular lower layer. -* If users want to append a layer enforce that they -* have already specified a first layer before. It's -* better to be strict. -*/ -if (ctx->nr ==0 ) { -pr_err ("cannot append layer if no previous layer has been specified" ); -return -EINVAL ; - } - -/* -* Once a sequence of data layers has started regular -* lower layers are forbidden. -*/ -if (ctx->nr_data >0 ) { -pr_err ("regular lower layers cannot follow data lower layers" ); -return -EINVAL ; - } - -/* Skip the leading ":". */ - name++; - append = true; - } +char *iter; +ssize_t nr_lower, nr; +enum ovl_opt layer = Opt_lowerdir_add; - dup =kstrdup (name,GFP_KERNEL ); -if (!dup) -return -ENOMEM ; +ovl_parse_param_drop_lowerdir (ctx); - err = -EINVAL ; - nr_lower =ovl_parse_param_split_lowerdirs (dup); + nr_lower =ovl_parse_param_split_lowerdirs (param->string);if (nr_lower <0 ) - goto out_err; +return nr_lower; -if ((nr_lower >OVL_MAX_STACK ) || - (append && (size_add (ctx->nr, nr_lower) >OVL_MAX_STACK ))) { -pr_err ("too many lower directories, limit is %d\n" ,OVL_MAX_STACK ); - goto out_err; - } - -if (!append) -ovl_parse_param_drop_lowerdir (ctx); + iter = param->string; +for (nr =0 ; nr < nr_lower; nr++) { +struct fs_parameter p = { + .key = (layer == Opt_lowerdir_add) ?"lowerdir" :"datadir" , + .type = fs_value_is_string, + .string =kstrdup (iter,GFP_KERNEL ), + }; -/* -* (1) append -* -* We want nr <= nr_lower <= capacity We know nr > 0 and nr <= -* capacity. If nr == 0 this wouldn't be append. If nr + -* nr_lower is <= capacity then nr <= nr_lower <= capacity -* already holds. If nr + nr_lower exceeds capacity, we realloc. -* -* (2) replace -* -* Ensure we're backwards compatible with mount(2) which allows -* "lowerdir=/a:/b:/c,lowerdir=/d:/e:/f" causing the last -* specified lowerdir mount option to win. -* -* We want nr <= nr_lower <= capacity We know either (i) nr == 0 -* or (ii) nr > 0. We also know nr_lower > 0. The capacity -* could've been changed multiple times already so we only know -* nr <= capacity. If nr + nr_lower > capacity we realloc, -* otherwise nr <= nr_lower <= capacity holds already. -*/ - nr_lower += ctx->nr; -if (nr_lower > ctx->capacity) { - err = -ENOMEM ; - l =krealloc_array (ctx->lower, nr_lower,sizeof (*ctx->lower), -GFP_KERNEL_ACCOUNT ); -if (!l) - goto out_err; +if (!p.string) +return -ENOMEM ; - ctx->lower = l; - ctx->capacity = nr_lower; - } - -/* -* (3) By (1) and (2) we know nr <= nr_lower <= capacity. -* (4) If ctx->nr == 0 => replace -* We have verified above that the lowerdir mount option -* isn't an append, i.e., the lowerdir mount option -* doesn't start with ":" or "::". -* (4.1) The lowerdir mount options only contains regular lower -* layers ":". -* => Nothing to verify. -* (4.2) The lowerdir mount options contains regular ":" and -* data "::" layers. -* => We need to verify that data lower layers "::" aren't -* followed by regular ":" lower layers -* (5) If ctx->nr > 0 => append -* We know that there's at least one regular layer -* otherwise we would've failed when parsing the previous -* lowerdir mount option. -* (5.1) The lowerdir mount option is a regular layer ":" append -* => We need to verify that no data layers have been -* specified before. -* (5.2) The lowerdir mount option is a data layer "::" append -* We know that there's at least one regular layer or -* other data layers. => There's nothing to verify. -*/ - dup_iter = dup; -for (nr = ctx->nr; nr < nr_lower; nr++) { - l = &ctx->lower[nr]; -memset (l,0 ,sizeof (*l)); - - err =ovl_mount_dir (dup_iter, &l->path, false); + err =ovl_parse_layer (fc, &p, layer); +kfree (p.string);if (err) - goto out_put; - - err = -ENOMEM ; - l->name =kstrdup (dup_iter,GFP_KERNEL_ACCOUNT ); -if (!l->name) - goto out_put; - -if (data_layer) - nr_data++; - -/* Calling strchr() again would overrun. */ -if ((nr +1 ) == nr_lower)break ; - err = -EINVAL ; - dup_iter =strchr (dup_iter,'\0' ) +1 ; -if (*dup_iter) { -/* -* This is a regular layer so we require that -* there are no data layers. -*/ -if ((ctx->nr_data + nr_data) >0 ) { -pr_err ("regular lower layers cannot follow data lower layers" ); - goto out_put; - } - - data_layer = false; -continue ; + layer = Opt_lowerdir_add; + iter =strchr (iter,'\0' ) +1 ; +if (!*iter) { + layer = Opt_datadir_add; + iter++; } - -/* This is a data lower layer. */ - data_layer = true; - dup_iter++; - } - ctx->nr = nr_lower; - ctx->nr_data += nr_data; -kfree (dup); -return 0 ; - -out_put: -/* -* We know nr >= ctx->nr < nr_lower. If we failed somewhere -* we want to undo until nr == ctx->nr. This is correct for -* both ctx->nr == 0 and ctx->nr > 0. -*/ -for (; nr >= ctx->nr; nr--) { - l = &ctx->lower[nr]; -kfree (l->name); - l->name =NULL ; -path_put (&l->path); - -/* don't overflow */ -if (nr ==0 ) -break ; } - -out_err: -kfree (dup); - -/* Intentionally don't realloc to a smaller size. */ return err; }
@@ -600,13 +510,13 @@ static int ovl_parse_param(struct fs_conswitch (opt) {case Opt_lowerdir: - err =ovl_parse_param_lowerdir (param->string, fc); + err =ovl_parse_lower_layers (fc, param);break ; +case Opt_lowerdir_add: +case Opt_datadir_add:case Opt_upperdir: - fallthrough;case Opt_workdir: - err =ovl_parse_param_upperdir (param->string, fc, - (Opt_workdir == opt)); + err =ovl_parse_layer (fc, param, opt);break ;case Opt_default_permissions: config->default_permissions = true;
fs/fs_parser.c +3 -1
@@ -150,6 +150,7 @@ int fs_lookup_param(struct fs_context *fstruct filename *f;bool put_f;int ret; +int dfd =AT_FDCWD ;switch (param->type) {case fs_value_is_string:
@@ -160,13 +161,14 @@ int fs_lookup_param(struct fs_context *fbreak ;case fs_value_is_filename: f = param->name; + dfd = param->dirfd; put_f = false;break ;default :return invalf (fc,"%s: not usable as path" , param->key); } - ret =filename_lookup (param->dirfd, f, flags, _path,NULL ); + ret =filename_lookup (dfd, f, flags, _path,NULL );if (ret <0 ) {errorf (fc,"%s: Lookup failure for '%s'" , param->key, f->name); goto out;
Amir Goldstein
maintainerQuoted 9 lines · Miklos Szeredi
Quoted 3 lines · Miklos Szeredi
If you can code this up quickly, that's good. I can have a go at it on Monday, but my PoC patch needs splitting up and so it's not ready for 6.6.
Attaching my current patch (against your 3 patches).
That's a nice patch, but it's quite big for rc6 ;-)
I'll just go ahead and write a patch for ovl-fixes to disallow the ':' prefix and we can get your patch into shape for 6.7.
I plan to send ovl-fixes to Linus tomorrow.
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 18 lines · Miklos Szeredi
Quoted 3 lines · Miklos Szeredi
If you can code this up quickly, that's good. I can have a go at it on Monday, but my PoC patch needs splitting up and so it's not ready for 6.6.
Attaching my current patch (against your 3 patches).
(quoted diff) +8
-0+static char *escape_colons(char *s, char *p) +{ + char *ret = s; + + for (;;) { + char c = *p++; + if (c != ':') { + *s++ = c;
Review
Quoted 10 lines
(quoted diff) +10
-0+ if (!c) + return ret; + } else if (s + 2 > p) { + return ERR_PTR(-ENAMETOOLONG); + } else { + *s++ = '\\'; + *s++ = c; + } + } +}
I think it is a bad idea going down this rabbit hole. This escaping is incorrect for an already escaped input (e.g. "lowerdir=/a\:b") - it would escape the \ instead of :
I think that playing more escaping games is the opposite of what the explicit path params want to achieve.
Suggesting instead:
fs/overlayfs/params.c +0 -0
@@ -933,23 +933,27 @@ int ovl_show_options(struct seq_file *m, struct
dentry *dentry)
{
struct super_block *sb = dentry->d_sb;
struct ovl_fs *ofs = OVL_FS(sb);
- size_t nr, nr_merged_lower = ofs->numlayer - ofs->numdatalayer;
+ size_t nr, nr_merged_lower, nr_added_lower = 0;
+ const char **lowerdirs = ofs->config.lowerdirs; /*
- * lowerdirs[] starts from offset 1, then
- * >= 0 regular lower layers prefixed with : and
- * >= 0 data-only lower layers prefixed with ::
- *
- * we need to escase comma and space like seq_show_option() does and
- * we also need to escape the colon separator from lowerdir paths.
+ * lowerdirs[0] holds the colon separated list if user provided
+ * it with legacy lowerdir= mount option. Otherwise, we use the
+ * options lowerdir+ and datadir+ to show the separated lowerdirs
+ * that are stored in lowerdirs[1..numlayer].
*/- seq_puts(m, ",lowerdir=");
- for (nr = 1; nr < ofs->numlayer; nr++) {
- if (nr > 1)
- seq_putc(m, ':');
- if (nr >= nr_merged_lower)
- seq_putc(m, ':');
- seq_escape(m, ofs->config.lowerdirs[nr], ":, \t\n\\");
+ if (lowerdirs[0]) {
+ seq_show_option(m, "lowerdir", lowerdirs[0]);
+ } else {
+ nr_added_lower = ofs->numlayer - 1; + nr_merged_lower = nr_added_lower - ofs->numdatalayer; + lowerdirs++;
+ }
+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) {
+ if (nr < nr_merged_lower)
+ seq_show_option(m, "lowerdir+", *lowerdirs);
+ else
+ seq_show_option(m, "datadir+", *lowerdirs);
}
if (ofs->config.upperdir) {
seq_show_option(m, "upperdir", ofs->config.upperdir);---
This implies that we do not allow mixing legacy lowerdir= with new lowerdir+/datadir+.
I don't see why mixing would be needed.
Thanks,
Amir.
Miklos Szeredi
maintainerReview
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);
Good.
I did some testing and it turns out libmount still regresses on 6.6-rc6 for the escaped comma case. The reason is that libmount doesn't understand escaping of commas, hence the '-oupper=upper\,1' will result in two fsconfig() calls: 'upper=upper\' and '1'. Prior to 6.5 these were nicely reconstructed into the original 'upper=upper\,1' by legacy_parse_param().
The same reconstructing could be done by ovl_parse_param() when detecting an option ending with a backslash. But I guess we only need to do this if there's a report of such a regression.
But this raises the question: shouldn't we turn off comma unescaping, since it's useless?
Going further, unescaping in general for upperdir and workdir could be turned off (lowerdir+ and datadir+ are naturally escape free), leaving only unescaping in lowerdir?
Yes, that also has the potential to regress something out there. But it also has the potential to clean up the interface further if no such
regression happens. And I don't think we need to hurry, the 6.7
cycle would be good for experimenting.
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 9 lines · Miklos Szeredi
Review
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);
Quoted 9 lines
Good.
I did some testing and it turns out libmount still regresses on 6.6-rc6 for the escaped comma case. The reason is that libmount doesn't understand escaping of commas, hence the '-oupper=upper\,1' will result in two
fsconfig()calls: 'upper=upper\' and '1'. Prior to 6.5 these were nicely reconstructed into the original 'upper=upper\,1' bylegacy_parse_param().
Technically, I think this is a libmount regression, not a kernel regression. Since libmount 2.39, libmount will split the commas differently than overlayfs always did.
If a user using libmount 2.39 upgrades to kernel 6.5 and reports a regression, we can tell that user to opt-out of new mount api via LIBMOUNT_FORCE_MOUNT2=never.
Quoted 4 lines
The same reconstructing could be done by
ovl_parse_param()when detecting an option ending with a backslash. But I guess we only need to do this if there's a report of such a regression.
I really hope we will not need to do that.
Quoted 3 lines
But this raises the question: shouldn't we turn off comma unescaping, since it's useless?
Not sure I understand what this means.
Quoted 9 lines
Going further, unescaping in general for upperdir and workdir could be turned off (lowerdir+ and datadir+ are naturally escape free), leaving only unescaping in lowerdir?
Yes, that also has the potential to regress something out there. But it also has the potential to clean up the interface further if no such
regression happens. And I don't think we need to hurry, the 6.7
cycle would be good for experimenting.
I am ok with doing this experiment in 6.7.
I saw that your patch converted upperdir/workdir to path type without unescaping. Don't we need to keep the support for string type params for ->parse_monolothic()?
Or is this the reason for the change in fs_lookup_param()?
Anyway, please make sure to split the patch that changes upperdir/workdir params to be without unescaping, so if we get regression reports in the future, it will be easy to revert this change independently from lowerdir+,datadir+.
BTW, can lowerdir+,datadir+ be provided to mount(8) via command line? or does that require a libmount change? If they work with mount(8), I can write up an fstests.
FYI, I rebased overlayfs-next over 6.6-rc6.
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 21 lines · Amir Goldstein
Quoted 9 lines · Miklos Szeredi
Review
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);Quoted 9 lines
Good.
I did some testing and it turns out libmount still regresses on 6.6-rc6 for the escaped comma case. The reason is that libmount doesn't understand escaping of commas, hence the '-oupper=upper\,1' will result in two
fsconfig()calls: 'upper=upper\' and '1'. Prior to 6.5 these were nicely reconstructed into the original 'upper=upper\,1' bylegacy_parse_param().
Quoted 3 lines
Technically, I think this is a libmount regression, not a kernel regression. Since libmount 2.39, libmount will split the commas differently than overlayfs always did.
Ah, but it's not a regression after all, since the kernel un-split the same commas until 6.5, so there was no way the libmount devs would
have observed any regression in overlayfs mount. But arguing about
which component is the cause of the regression is not very productive. Indeed libmount can be fixed parse overlayfs options the same way as the kernel parsed them before 6.5, which is probably a much better fix, than a kernel one.
Karel, is doing such filesystem specific option handling feasible?
If so, then for overlayfs please please pass an un-escaped (\char -> char) string to fsconfig for "upperdir=" and "workdir=" options.
Quoted 5 lines
I am ok with doing this experiment in 6.7.
I saw that your patch converted upperdir/workdir to path type without unescaping. Don't we need to keep the support for string type params for ->parse_monolothic()?
Yes.
Quoted 1 line
Or is this the reason for the change in
fs_lookup_param()?
That just a bug. It already supported the string type, and in that case lookup must be done against AT_FDCWD and not against zero fd (which is what it previously did).
Thanks,
Miklos
Quoted 12 lines · Miklos Szeredi
Quoted 11 lines · Amir Goldstein
Quoted 9 lines · Miklos Szeredi
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Good.
I did some testing and it turns out libmount still regresses on 6.6-rc6 for the escaped comma case. The reason is that libmount doesn't understand escaping of commas, hence the '-oupper=upper\,1' will result in two
fsconfig()calls: 'upper=upper\' and '1'. Prior to 6.5 these were nicely reconstructed into the original 'upper=upper\,1' bylegacy_parse_param().
Yes, libmount does not interpret '\,' in any way, it's just comma after a char :-)
Quoted 4 lines
Quoted 4 lines
Technically, I think this is a libmount regression, not a kernel regression. Since libmount 2.39, libmount will split the commas differently than overlayfs always did.
The difference is that old libmount versions do not split the string; it only removes well-known non-kernel stuff and flags from the string. However, the rest of the string remains unmodified. This means that "upper=aaa,bbb" comprises two options for the old libmount, but because it neither reorders nor splits it, it is sent unmodified to the mount(2) syscall.
The new libmount works with mount options differently. It keeps them parsed in memory (in a struct libmnt_optlist), and this list is used for fsconfig().
Quoted 9 lines
Ah, but it's not a regression after all, since the kernel un-split the same commas until 6.5, so there was no way the libmount devs would
have observed any regression in overlayfs mount. But arguing about
which component is the cause of the regression is not very productive. Indeed libmount can be fixed parse overlayfs options the same way as the kernel parsed them before 6.5, which is probably a much better fix, than a kernel one.Karel, is doing such filesystem specific option handling feasible?
For decade we have support for commas in mount option due to creativity of SELinux developers:
foo,context="aaa,bbb,ccc",bar
is valid mount options string and libmount will ignore commas within " " and split it to foo, bar, and context=.
The current util-linux git (and old 6.2 kernel):
# strace -e fsopen,fsconfig ./mount -t overlay overlay -o 'lowerdir="/tmp/test-lower,",upperdir=/tmp/test-upper,workdir=/tmp/test-work' /tmp/test
fsopen("overlay", FSOPEN_CLOEXEC) = 3
fsconfig(3, FSCONFIG_SET_STRING, "source", "overlay", 0) = 0
fsconfig(3, FSCONFIG_SET_STRING, "lowerdir", "/tmp/test-lower,", 0) = -1 EINVAL (Invalid argument)You can see "/tmp/test-lower,".
Maybe all we need is to improve mount(8) docs to force people use "" for paths when used in mount options.
Anyway, I think we can improve libmount to ignore \, as non-separator. The question is what the rest of the userspace universe, because fstab is interpreted on many places ...
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Amir Goldstein
maintainerQuoted 67 lines · Karel Zak
Quoted 12 lines · Miklos Szeredi
Quoted 11 lines · Amir Goldstein
Quoted 9 lines · Miklos Szeredi
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Good.
I did some testing and it turns out libmount still regresses on 6.6-rc6 for the escaped comma case. The reason is that libmount doesn't understand escaping of commas, hence the '-oupper=upper\,1' will result in two
fsconfig()calls: 'upper=upper\' and '1'. Prior to 6.5 these were nicely reconstructed into the original 'upper=upper\,1' bylegacy_parse_param().Yes, libmount does not interpret '\,' in any way, it's just comma after a char :-)
Quoted 4 lines
Quoted 4 lines
Technically, I think this is a libmount regression, not a kernel regression. Since libmount 2.39, libmount will split the commas differently than overlayfs always did.
The difference is that old libmount versions do not split the string; it only removes well-known non-kernel stuff and flags from the string. However, the rest of the string remains unmodified. This means that "upper=aaa,bbb" comprises two options for the old libmount, but because it neither reorders nor splits it, it is sent unmodified to the mount(2) syscall.
The new libmount works with mount options differently. It keeps them parsed in memory (in a struct
libmnt_optlist), and this list is used forfsconfig().Quoted 9 lines
Ah, but it's not a regression after all, since the kernel un-split the same commas until 6.5, so there was no way the libmount devs would
have observed any regression in overlayfs mount. But arguing about
which component is the cause of the regression is not very productive. Indeed libmount can be fixed parse overlayfs options the same way as the kernel parsed them before 6.5, which is probably a much better fix, than a kernel one.Karel, is doing such filesystem specific option handling feasible?
For decade we have support for commas in mount option due to creativity of SELinux developers:
foo,context="aaa,bbb,ccc",bar
is valid mount options string and libmount will ignore commas within " " and split it to foo, bar, and context=.
The current util-linux git (and old 6.2 kernel):
# strace -e fsopen,fsconfig ./mount -t overlay overlay -o 'lowerdir="/tmp/test-lower,",upperdir=/tmp/test-upper,workdir=/tmp/test-work' /tmp/test fsopen("overlay", FSOPEN_CLOEXEC) = 3 fsconfig(3, FSCONFIG_SET_STRING, "source", "overlay", 0) = 0 fsconfig(3, FSCONFIG_SET_STRING, "lowerdir", "/tmp/test-lower,", 0) = -1 EINVAL (Invalid argument)
Quoted 5 lines
You can see "
/tmp/test-lower,".Maybe all we need is to improve mount(8) docs to force people use "" for paths when used in mount options.
Even if people would read documentation ;-)
I don't think that would have helped in this case,
because with old libmount and old kernel, overlayfs does not parse lowerdir="/tmp/test-lower," correctly - it requires lowerdir="/tmp/test-lower\,"
Quoted 3 lines
Anyway, I think we can improve libmount to ignore \, as non-separator. The question is what the rest of the userspace universe, because fstab is interpreted on many places ...
The other option is to require overlayfs users to opt-in to new mount api (LIBMOUNT_FORCE_MOUNT2=always) and otherwise default to
the old mount api or
instead of improving libmount parsing of \,
maybe just detect it and default to the old mount api
just for overlayfs or generally?
All-in-all, those cases are probably rare, so I think the minimal solution with as little room for complication should be applied. Detecting \, and defaulting to old mount api seems to meet this criteria?
BTW, smb3_fs_context_parse_monolithic() also has special handling of comma parsing in kernel (,, is a non-separator comma), so smb3 (cifs) may also require special treatment when choosing old/new mount api.
Thanks,
Amir.
Quoted 8 lines · Miklos Szeredi
Ah, but it's not a regression after all, since the kernel un-split the same commas until 6.5, so there was no way the libmount devs would
have observed any regression in overlayfs mount. But arguing about
which component is the cause of the regression is not very productive. Indeed libmount can be fixed parse overlayfs options the same way as the kernel parsed them before 6.5, which is probably a much better fix, than a kernel one.
Quoted 4 lines
Karel, is doing such filesystem specific option handling feasible?
If so, then for overlayfs please please pass an un-escaped (\char -> char) string to fsconfig for "upperdir=" and "workdir=" options.
Committed to the libmount:
https://github.com/util-linux/util-linux/commit/f6c29efa929cb8c741591ab38061e7921d53a997
will be in util-linux v2.40 and in v2.39.3
It's implemented for all filesystems, not exception for overlayfs.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Amir Goldstein
maintainerQuoted 15 lines · Karel Zak
Quoted 8 lines · Miklos Szeredi
Ah, but it's not a regression after all, since the kernel un-split the same commas until 6.5, so there was no way the libmount devs would
have observed any regression in overlayfs mount. But arguing about
which component is the cause of the regression is not very productive. Indeed libmount can be fixed parse overlayfs options the same way as the kernel parsed them before 6.5, which is probably a much better fix, than a kernel one.Quoted 4 lines
Karel, is doing such filesystem specific option handling feasible?
If so, then for overlayfs please please pass an un-escaped (\char -> char) string to fsconfig for "upperdir=" and "workdir=" options.
Quoted 6 lines
Committed to the libmount:
https://github.com/util-linux/util-linux/commit/f6c29efa929cb8c741591ab38061e7921d53a997will be in util-linux v2.40 and in v2.39.3
It's implemented for all filesystems, not exception for overlayfs.
Nice!
Note that this code:
if (*p == ',' && (p == optstr0 || *(p - 1) != '\\'))
stop = p; /* terminate the option item */
...
strrem(p, '\\');
value = p;Will not split either of the following:
b\,a\,r
b\\,a\,r
while legacy overlayfs does split the , when followed by \\
IMO what you did is good enough for all practical cases and there is no need to go down the rabbit hole of proper escaping (unless bugs are reported).
Just wanted to point out the difference.
Thanks,
Amir.
Amir Goldstein
maintainerQuoted 11 lines · Miklos Szeredi
Review
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);Good.
And if we are going to show lowerdir+/datadir+ in a comma separated string, we might as well also support them with FSCONFIG_SET_STRING as long as they don't need escaping.
I think this may even be more important than supporting path params just to restore the feature that was retroactively disabled in 6.5.y. We can later add FSCONFIG_SET_PATH support for all those params.
I can take on writing the string params patch based on your POC, including the fstests, which are quite simple to do in bash for string params.
Thanks,
Amir.
Miklos Szeredi
maintainerQuoted 11 lines · Amir Goldstein
Quoted 9 lines · Miklos Szeredi
Quoted 5 lines · Amir Goldstein
(quoted diff) +5
-0+ for (nr = 0; nr < nr_added_lower; nr++, lowerdirs++) { + if (nr < nr_merged_lower) + seq_show_option(m, "lowerdir+", *lowerdirs); + else + seq_show_option(m, "datadir+", *lowerdirs);
Quoted 6 lines
Quoted 2 lines
Good.
And if we are going to show lowerdir+/datadir+ in a comma separated string, we might as well also support them with
FSCONFIG_SET_STRINGas long as they don't need escaping.
My patch does support SET_STRING. The difference is that string
param will be shown in mountinfo unchanged, while a path param will
be canonicalized. It's interesting, because using string params and
legacy interfaces can hide the location of the layers. OTOH they
allow for less cluttered mountinfo. So I guess this difference can
be an advantage and a disadvantage too, depending on your point of view, but it's probably not something we want to change.
Quoted 3 lines
I think this may even be more important than supporting path params just to restore the feature that was retroactively disabled in 6.5.y.
Yeah, that would be a good way to split this up.
Quoted 4 lines
We can later add
FSCONFIG_SET_PATHsupport for all those params.I can take on writing the string params patch based on your POC, including the fstests, which are quite simple to do in bash for string params.
Okay, thanks.
Miklos