Support nested overlayfs mounts
Subsystems (2)
v3 cover Support nested overlayfs mounts
There are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmpfs), then you cannot currently store a lowerdir on the rootfs, becasue the inner overlayfs will eat all the whiteouts and overlay xattrs. This means you can't e.g. store on the rootfs a prepared container image for use with overlayfs.
This patch series adds support for nesting of overlayfs mounts by escaping the problematic features and unescaping them when exposing to the overlayfs user.
This series is also available here:
https://github.com/alexlarsson/linux/tree/ovl-nesting
And xfstest to test it is available here:
https://github.com/alexlarsson/xfstests/tree/overlayfs-nesting
The overlay/083 test checks both xattr escaping, the new whiteouts as well as actual nesting of overlayfs.
Note that this series breaks the overlay/026 test which validates that writing overlay.* xattrs is not supported, but it now is. I'm not sure if we should fix this test to not fail, or if we should make this an opt-in mount feature.
Changes since v2:
* Uses a new approach for escaping whiteouts with a regular file with an overlay.whiteout xattr in a lower directory with an overlay.whiteouts xattr.
Changes since v1:
* Moved all xattr handling to xattr.c
* Made creation of escaped whiteouts atomic
Alexander Larsson (6):
ovl: Move xattr support to new xattrs.c file
ovl: Add OVL_XATTR_TRUSTED/USER_PREFIX_LEN macros
ovl: Support escaped overlay.* xattrs
ovl: Add an alternative type of whiteout
ovl: Handle escaped xwhiteouts across layers
ovl: Add documentation on nesting of overlayfs mounts
Documentation/filesystems/overlayfs.rst | 23 ++ fs/overlayfs/Makefile | 2 +- fs/overlayfs/dir.c | 4 +- fs/overlayfs/inode.c | 124 ---------- fs/overlayfs/namei.c | 15 +- fs/overlayfs/overlayfs.h | 42 +++- fs/overlayfs/readdir.c | 27 +- fs/overlayfs/super.c | 67 +---- fs/overlayfs/util.c | 40 +++ fs/overlayfs/xattrs.c | 312 ++++++++++++++++++++++++ 10 files changed, 443 insertions(+), 213 deletions(-) create mode 100644 fs/overlayfs/xattrs.c
Signature · 1 line
2.41.0
Amir Goldstein
maintainerQuoted 11 lines · Alexander Larsson
There are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmpfs), then you cannot currently store a lowerdir on the rootfs, becasue the inner overlayfs will eat all the whiteouts and overlay xattrs. This means you can't e.g. store on the rootfs a prepared container image for use with overlayfs.
This patch series adds support for nesting of overlayfs mounts by escaping the problematic features and unescaping them when exposing to the overlayfs user.
Quoted 9 lines
This series is also available here:
https://github.com/alexlarsson/linux/tree/ovl-nestingAnd xfstest to test it is available here:
https://github.com/alexlarsson/xfstests/tree/overlayfs-nestingThe overlay/083 test checks both xattr escaping, the new whiteouts as well as actual nesting of overlayfs.
This test look good. Left minor comments in github.
Missing test for the perplexing "escaped xwhiteouts across layers" use case.
Quoted 4 lines
Note that this series breaks the overlay/026 test which validates that writing overlay.* xattrs is not supported, but it now is. I'm not sure if we should fix this test to not fail, or if we should make this an opt-in mount feature.
I think we don't need an opt-in mount option, but would like to know what Miklos thinks?
We can fix the test this way:
setfattr trusted.overlayfsrz
verify success
getfattr trusted.overlayfsrz
verify success AND that xattr was not escaped
setfattr trusted.overlay.fsz
verify failure OR that xattr was escaped
getfattr trusted.overlay.fsz
verify failure OR that xattr was escaped
Thanks,
Amir.
Quoted 35 lines
Changes since v2:
* Uses a new approach for escaping whiteouts with a regular file with an overlay.whiteout xattr in a lower directory with an overlay.whiteouts xattr.Changes since v1:
* Moved all xattr handling to
xattr.c
* Made creation of escaped whiteouts atomicAlexander Larsson (6):
ovl: Move xattr support to newxattrs.cfile
ovl: Add OVL_XATTR_TRUSTED/USER_PREFIX_LEN macros
ovl: Support escaped overlay.* xattrs
ovl: Add an alternative type of whiteout
ovl: Handle escaped xwhiteouts across layers
ovl: Add documentation on nesting of overlayfs mountsDocumentation/filesystems/overlayfs.rst | 23 ++ fs/overlayfs/Makefile | 2 +- fs/overlayfs/dir.c | 4 +- fs/overlayfs/inode.c | 124 ---------- fs/overlayfs/namei.c | 15 +- fs/overlayfs/overlayfs.h | 42 +++- fs/overlayfs/readdir.c | 27 +- fs/overlayfs/super.c | 67 +---- fs/overlayfs/util.c | 40 +++ fs/overlayfs/xattrs.c | 312 ++++++++++++++++++++++++ 10 files changed, 443 insertions(+), 213 deletions(-) create mode 100644 fs/overlayfs/xattrs.c--
2.41.0
A message that is not in the archive (sent off-list or not yet received): CAL7ro1Fm72xndy1C0zZbScPtgjGXucZ3rip0cO4VutSu0Jy4-A@mail.gmail.com
Amir Goldstein
maintainerQuoted 52 lines · Alexander Larsson
Quoted 19 lines · Amir Goldstein
Quoted 17 lines · Alexander Larsson
There are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmpfs), then you cannot currently store a lowerdir on the rootfs, becasue the inner overlayfs will eat all the whiteouts and overlay xattrs. This means you can't e.g. store on the rootfs a prepared container image for use with overlayfs.
This patch series adds support for nesting of overlayfs mounts by escaping the problematic features and unescaping them when exposing to the overlayfs user.
This series is also available here:
https://github.com/alexlarsson/linux/tree/ovl-nestingAnd xfstest to test it is available here:
https://github.com/alexlarsson/xfstests/tree/overlayfs-nestingQuoted 6 lines
Quoted 3 lines
The overlay/083 test checks both xattr escaping, the new whiteouts as well as actual nesting of overlayfs.
This test look good. Left minor comments in github.
Missing test for the perplexing "escaped xwhiteouts across layers" use case.It's kinda hidden, but the creation of $middir/mid actually does test this.
The whiteouts xattr is set on $lowerdir/mid, and this is shadowed by the $middir/mid, so
if that change is removed from the tree, then the "ls" indo_test_xwhiteout" breaks due to the readdir not seeing the xwhiteouts on the mid dir when $lowerdir and $middir are merged.We could perhaps point it out more clearly though.
Quoted 15 lines
Quoted 4 lines
Note that this series breaks the overlay/026 test which validates that writing overlay.* xattrs is not supported, but it now is. I'm not sure if we should fix this test to not fail, or if we should make this an opt-in mount feature.
I think we don't need an opt-in mount option, but would like to know what Miklos thinks?
We can fix the test this way:
setfattr trusted.overlayfsrz
verify success
getfattr trusted.overlayfsrz
verify success AND that xattr was not escaped
Quoted 7 lines
Quoted 4 lines
setfattr trusted.overlay.fsz
verify failure OR that xattr was escaped
getfattr trusted.overlay.fsz
verify failure OR that xattr was escapedYes, it's fixable in a new xfstest. My question is more about how much we care about older xfstest versions showing a regression.
Don't worry about that - it happens quite often when tests are encoding expectations that is not a SPEC and we change behavior.
Yes it can be a nuisance to testers, but testers should update to recent xfstests
or at least check what changed in recent xfstests when they see a regression.
If there is an actual application that breaks because of the behavior change it is a different story, but I doubt there is.
Thanks,
Amir.
PATCH v3 1/6 ovl: Move xattr support to new xattrs.c file
In reply to Alexander LarssonThis moves the code from super.c and inode.c, and makes ovl_xattr_get/set() static.
This is in preparation for doing more work on xattrs support.
--- fs/overlayfs/Makefile | 2 +- fs/overlayfs/inode.c | 124 ------------------------ fs/overlayfs/overlayfs.h | 18 ++-- fs/overlayfs/super.c | 65 +------------ fs/overlayfs/xattrs.c | 198 +++++++++++++++++++++++++++++++++++++++ 5 files changed, 209 insertions(+), 198 deletions(-) create mode 100644 fs/overlayfs/xattrs.c
fs/overlayfs/Makefile +1 -1
@@ -6,4 +6,4 @@ obj-$(CONFIG_OVERLAY_FS) += overlay.o overlay-objs := super.o namei.o util.o inode.o file.o dir.o readdir.o \ - copy_up.o export.o params.o + copy_up.o export.o params.o xattrs.o
fs/overlayfs/inode.c +0 -124
130 lines not on this page · raw message
fs/overlayfs/overlayfs.h +9 -9
@@ -684,17 +684,8 @@ int ovl_set_nlink_lower(struct dentry *dentry);unsigned int ovl_get_nlink (struct ovl_fs *ofs,struct dentry *lowerdentry,struct dentry *upperdentry,unsigned int fallback); -int ovl_setattr (struct mnt_idmap *idmap,struct dentry *dentry, -struct iattr *attr); -int ovl_getattr (struct mnt_idmap *idmap,const struct path *path, -struct kstat *stat,u32 request_mask,unsigned int flags);int ovl_permission (struct mnt_idmap *idmap,struct inode *inode,int mask); -int ovl_xattr_set (struct dentry *dentry,struct inode *inode,const char *name, -const void *value,size_t size,int flags); -int ovl_xattr_get (struct dentry *dentry,struct inode *inode,const char *name, -void *value,size_t size); -ssize_t ovl_listxattr (struct dentry *dentry,char *list,size_t size);#ifdef CONFIG_FS_POSIX_ACL struct posix_acl *do_ovl_get_acl(struct mnt_idmap *idmap,
@@ -830,3 +821,12 @@ static inline bool ovl_force_readonly(struct ovl_fs *ofs) {return (!ovl_upper_mnt (ofs) || !ofs->workdir); } + +/* xattr.c */ + +const struct xattr_handler **ovl_xattr_handlers (struct ovl_fs *ofs); +int ovl_setattr (struct mnt_idmap *idmap,struct dentry *dentry, +struct iattr *attr); +int ovl_getattr (struct mnt_idmap *idmap,const struct path *path, +struct kstat *stat,u32 request_mask,unsigned int flags); +ssize_t ovl_listxattr (struct dentry *dentry,char *list,size_t size);
fs/overlayfs/super.c +1 -64
77 lines not on this page · raw message
fs/overlayfs/xattrs.c new file +198 -0
198 lines not on this page · raw message
Signature · 1 line
2.41.0
PATCH v3 2/6 ovl: Add OVL_XATTR_TRUSTED/USER_PREFIX_LEN macros
In reply to Alexander LarssonThese match the ones for e.g. XATTR_TRUSTED_PREFIX_LEN.
--- fs/overlayfs/overlayfs.h | 2 ++ fs/overlayfs/xattrs.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-)
fs/overlayfs/overlayfs.h +2 -0
@@ -28,7 +28,9 @@ enum ovl_path_type {#define OVL_XATTR_NAMESPACE "overlay."#define OVL_XATTR_TRUSTED_PREFIX XATTR_TRUSTED_PREFIX OVL_XATTR_NAMESPACE +#define OVL_XATTR_TRUSTED_PREFIX_LEN (sizeof(OVL_XATTR_TRUSTED_PREFIX) - 1)#define OVL_XATTR_USER_PREFIX XATTR_USER_PREFIX OVL_XATTR_NAMESPACE +#define OVL_XATTR_USER_PREFIX_LEN (sizeof(OVL_XATTR_USER_PREFIX) - 1)enum ovl_xattr {OVL_XATTR_OPAQUE ,
fs/overlayfs/xattrs.c +2 -2
@@ -10,10 +10,10 @@ bool ovl_is_private_xattr(struct super_block *sb, const char *name)if (ofs->config.userxattr)return strncmp (name,OVL_XATTR_USER_PREFIX , -sizeof (OVL_XATTR_USER_PREFIX ) -1 ) ==0 ; +OVL_XATTR_USER_PREFIX_LEN ) ==0 ;else return strncmp (name,OVL_XATTR_TRUSTED_PREFIX , -sizeof (OVL_XATTR_TRUSTED_PREFIX ) -1 ) ==0 ; +OVL_XATTR_TRUSTED_PREFIX_LEN ) ==0 ; } static int ovl_xattr_set(struct dentry *dentry, struct inode *inode, const char *name,
Signature · 1 line
2.41.0
PATCH v3 3/6 ovl: Support escaped overlay.* xattrs
In reply to Alexander LarssonThere are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmps), then you cannot currently store a lowerdir on the rootfs. This means you can't e.g. store on the rootfs a prepared container image for use using overlayfs.
To work around this, we introduce an escapment mechanism for overlayfs xattrs. Whenever the lower/upper dir has a xattr named overlay.overlay.XYZ, we list it as overlay.XYZ in listxattrs, and when the user calls getxattr or setxattr on overlay.XYZ, we apply to overlay.overlay.XYZ in the backing directories.
This allows storing any kind of overlay xattrs in a overlayfs mount that can be used as a lowerdir in another mount. It is possible to stack this mechanism multiple times, such that overlay.overlay.overlay.XYZ will survive two levels of overlay mounts, however this is not all that useful in practice because of stack depth limitations of overlayfs mounts.
--- fs/overlayfs/overlayfs.h | 7 ++++ fs/overlayfs/xattrs.c | 78 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 81 insertions(+), 4 deletions(-)
fs/overlayfs/overlayfs.h +7 -0
@@ -32,6 +32,13 @@ enum ovl_path_type {#define OVL_XATTR_USER_PREFIX XATTR_USER_PREFIX OVL_XATTR_NAMESPACE#define OVL_XATTR_USER_PREFIX_LEN (sizeof(OVL_XATTR_USER_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_PREFIX OVL_XATTR_NAMESPACE +#define OVL_XATTR_ESCAPE_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX OVL_XATTR_TRUSTED_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_TRUSTED_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_USER_PREFIX OVL_XATTR_USER_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_USER_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_USER_PREFIX) - 1) +enum ovl_xattr {OVL_XATTR_OPAQUE ,OVL_XATTR_REDIRECT ,
fs/overlayfs/xattrs.c +74 -4
128 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v3 3/6] ovl: Support escaped overlay.* xattrs
Quoted 8 lines · Alexander Larsson
There are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmps), then you cannot currently store a lowerdir on the rootfs. This means you can't e.g. store on the rootfs a prepared container image for use using overlayfs.
Quoted 5 lines
To work around this, we introduce an escapment mechanism for overlayfs xattrs. Whenever the lower/upper dir has a xattr named
overlay.overlay.XYZ, we list it as overlay.XYZ in listxattrs, and when the user calls getxattr or setxattr onoverlay.XYZ, we apply tooverlay.overlay.XYZin the backing directories.
Please use ""
Quoted 9 lines
This allows storing any kind of overlay xattrs in a overlayfs mount that can be used as a lowerdir in another mount. It is possible to stack this mechanism multiple times, such that overlay.overlay.overlay.XYZ will survive two levels of overlay mounts, however this is not all that useful in practice because of stack depth limitations of overlayfs mounts.
Signed-off-by Alexander Larsson <[email protected]>
With comment in ovl_listxattr() below fixed you may add:
Quoted 81 lines
--- fs/overlayfs/overlayfs.h | 7 ++++ fs/overlayfs/xattrs.c | 78 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 81 insertions(+), 4 deletions(-)fs/overlayfs/overlayfs.h +7
-0@@ -32,6 +32,13 @@ enum ovl_path_type {#define OVL_XATTR_USER_PREFIX XATTR_USER_PREFIX OVL_XATTR_NAMESPACE#define OVL_XATTR_USER_PREFIX_LEN (sizeof(OVL_XATTR_USER_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_PREFIX OVL_XATTR_NAMESPACE +#define OVL_XATTR_ESCAPE_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX OVL_XATTR_TRUSTED_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_TRUSTED_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_USER_PREFIX OVL_XATTR_USER_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_USER_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_USER_PREFIX) - 1) +enum ovl_xattr {OVL_XATTR_OPAQUE ,OVL_XATTR_REDIRECT ,fs/overlayfs/xattrs.c +19
-2@@ -4,6 +4,18 @@#include <linux/xattr.h> #include "overlayfs.h" +bool ovl_is_escaped_xattr (struct super_block *sb,const char *name) +{ +struct ovl_fs *ofs = sb->s_fs_info; + +if (ofs->config.userxattr) +return strncmp (name,OVL_XATTR_ESCAPE_USER_PREFIX , +OVL_XATTR_ESCAPE_USER_PREFIX_LEN ) ==0 ; +else +return strncmp (name,OVL_XATTR_ESCAPE_TRUSTED_PREFIX , +OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN -1 ) ==0 ; +} +bool ovl_is_private_xattr (struct super_block *sb,const char *name) {struct ovl_fs *ofs =OVL_FS (sb);@@ -82,8 +94,8 @@ static int ovl_xattr_get(struct dentry *dentry, struct inode *inode, const charstatic bool ovl_can_list (struct super_block *sb,const char *s) { -/* Never list private (.overlay) */ -if (ovl_is_private_xattr (sb, s)) +/* Never list non-escaped private (.overlay) */ +if (ovl_is_private_xattr (sb, s) && !ovl_is_escaped_xattr (sb, s))return false;/* List all non-trusted xattrs */ @@ -97,10 +109,12 @@ static bool ovl_can_list(struct super_block *sb, const char *s)ssize_t ovl_listxattr (struct dentry *dentry,char *list,size_t size) {struct dentry *realdentry =ovl_dentry_real (dentry); +struct ovl_fs *ofs =OVL_FS (dentry->d_sb);ssize_t res;size_t len;char *s;const struct cred *old_cred; +size_t prefix_len; old_cred =ovl_override_creds (dentry->d_sb); res =vfs_listxattr (realdentry, list, size);@@ -108,6 +122,9 @@ ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)if (res <=0 || size ==0 )return res; + prefix_len = ofs->config.userxattr ? +OVL_XATTR_USER_PREFIX_LEN :OVL_XATTR_TRUSTED_PREFIX_LEN ; +/* filter out private xattrs */ for (s = list, len = res; len;) {size_t slen =strnlen (s, len) +1 ;
fs/overlayfs/xattrs.c ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size) · line 145
Quoted 9 lines
fs/overlayfs/xattrs.c +6
-0@@ -120,6 +137,12 @@ ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)if (!ovl_can_list (dentry->d_sb, s)) { res -= slen;memmove (s, s + slen, len); + }else if (ovl_is_escaped_xattr (dentry->d_sb, s)) { +memmove (s + prefix_len, + s + prefix_len +OVL_XATTR_ESCAPE_PREFIX_LEN , + slen - (prefix_len +OVL_XATTR_ESCAPE_PREFIX_LEN ) + len); + res -=OVL_XATTR_ESCAPE_PREFIX_LEN ; + s += slen -OVL_XATTR_ESCAPE_PREFIX_LEN ;
It's a bit hard to follow. how about:
res -= OVL_XATTR_ESCAPE_PREFIX_LEN;name_len = slen - prefix_len - OVL_XATTR_ESCAPE_PREFIX_LEN; s += prefix_len;memmove(s, s + OVL_XATTR_ESCAPE_PREFIX_LEN, name_len + len); s += OVL_XATTR_ESCAPE_PREFIX_LEN + name_len;
Quoted 75 lines
fs/overlayfs/xattrs.c +49
-3}else { s += slen; }@@ -128,11 +151,47 @@ ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)return res; } +static char *ovl_xattr_escape_name (const char *prefix,const char *name) +{ +size_t prefix_len =strlen (prefix); +size_t name_len =strlen (name); +size_t escaped_len; +char *escaped, *s; + + escaped_len = prefix_len +OVL_XATTR_ESCAPE_PREFIX_LEN + name_len; +if (escaped_len >XATTR_NAME_MAX ) +return ERR_PTR (-EOPNOTSUPP ); + + escaped =kmalloc (escaped_len +1 ,GFP_KERNEL ); +if (escaped ==NULL ) +return ERR_PTR (-ENOMEM ); + + s = escaped; +memcpy (s, prefix, prefix_len); + s += prefix_len; +memcpy (s,OVL_XATTR_ESCAPE_PREFIX ,OVL_XATTR_ESCAPE_PREFIX_LEN ); + s +=OVL_XATTR_ESCAPE_PREFIX_LEN ; +memcpy (s, name, name_len +1 ); + +return escaped; +} +static int ovl_own_xattr_get (const struct xattr_handler *handler,struct dentry *dentry,struct inode *inode,const char *name,void *buffer,size_t size) { -return -EOPNOTSUPP ; +char *escaped; +int r; + + escaped =ovl_xattr_escape_name (handler->prefix, name); +if (IS_ERR (escaped)) +return PTR_ERR (escaped); + + r =ovl_xattr_get (dentry, inode, escaped, buffer, size); + +kfree (escaped); + +return r; } static int ovl_own_xattr_set(const struct xattr_handler *handler,@@ -141,7 +200,18 @@ static int ovl_own_xattr_set(const struct xattr_handler *handler, const char *name, const void *value, size_t size, int flags) { - return -EOPNOTSUPP; + char *escaped; +int r; + + escaped =ovl_xattr_escape_name (handler->prefix, name); +if (IS_ERR (escaped)) +return PTR_ERR (escaped); + + r =ovl_xattr_set (dentry, inode, escaped, value, size, flags); + +kfree (escaped); + +return r; } static int ovl_other_xattr_get(const struct xattr_handler *handler, --2.41.0
A message that is not in the archive (sent off-list or not yet received): CAL7ro1FMVdFPu5WqxHCoGTcpt8P532z02sx1Ngtf6BW61WPDGg@mail.gmail.com
Amir Goldstein
maintainerRe: [PATCH v3 3/6] ovl: Support escaped overlay.* xattrs
Quoted 20 lines · Alexander Larsson
Quoted 16 lines · Amir Goldstein
Quoted 13 lines · Alexander Larsson
There are cases where you want to use an overlayfs mount as a lowerdir for another overlayfs mount. For example, if the system rootfs is on overlayfs due to composefs, or to make it volatile (via tmps), then you cannot currently store a lowerdir on the rootfs. This means you can't e.g. store on the rootfs a prepared container image for use using overlayfs.
To work around this, we introduce an escapment mechanism for overlayfs xattrs. Whenever the lower/upper dir has a xattr named
overlay.overlay.XYZ, we list it as overlay.XYZ in listxattrs, and when the user calls getxattr or setxattr onoverlay.XYZ, we apply tooverlay.overlay.XYZin the backing directories.
Quoted 7 lines
Quoted 1 line
Please use ""
I like backquotes, because the github/gitlab UIs show commit messages as markdown, and there it renders the quoted part monospace, whereas " is just a regular character with no special rendering. But I changed it.
Well. It's not critical to me. just thought it would be more consistent with comment and doc style.
Quoted 108 lines
Quoted 108 lines
Quoted 10 lines
This allows storing any kind of overlay xattrs in a overlayfs mount that can be used as a lowerdir in another mount. It is possible to stack this mechanism multiple times, such that overlay.overlay.overlay.XYZ will survive two levels of overlay mounts, however this is not all that useful in practice because of stack depth limitations of overlayfs mounts.
Signed-off-by Alexander Larsson <[email protected]>With comment in
ovl_listxattr()below fixed you may add:Reviewed-by Amir Goldstein <[email protected]>Quoted 81 lines
--- fs/overlayfs/overlayfs.h | 7 ++++ fs/overlayfs/xattrs.c | 78 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 81 insertions(+), 4 deletions(-)fs/overlayfs/overlayfs.h +7
-0@@ -32,6 +32,13 @@ enum ovl_path_type {#define OVL_XATTR_USER_PREFIX XATTR_USER_PREFIX OVL_XATTR_NAMESPACE#define OVL_XATTR_USER_PREFIX_LEN (sizeof(OVL_XATTR_USER_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_PREFIX OVL_XATTR_NAMESPACE +#define OVL_XATTR_ESCAPE_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX OVL_XATTR_TRUSTED_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_TRUSTED_PREFIX) - 1) +#define OVL_XATTR_ESCAPE_USER_PREFIX OVL_XATTR_USER_PREFIX OVL_XATTR_ESCAPE_PREFIX +#define OVL_XATTR_ESCAPE_USER_PREFIX_LEN (sizeof(OVL_XATTR_ESCAPE_USER_PREFIX) - 1) +enum ovl_xattr {OVL_XATTR_OPAQUE ,OVL_XATTR_REDIRECT ,fs/overlayfs/xattrs.c +19
-2@@ -4,6 +4,18 @@#include <linux/xattr.h> #include "overlayfs.h" +bool ovl_is_escaped_xattr (struct super_block *sb,const char *name) +{ +struct ovl_fs *ofs = sb->s_fs_info; + +if (ofs->config.userxattr) +return strncmp (name,OVL_XATTR_ESCAPE_USER_PREFIX , +OVL_XATTR_ESCAPE_USER_PREFIX_LEN ) ==0 ; +else +return strncmp (name,OVL_XATTR_ESCAPE_TRUSTED_PREFIX , +OVL_XATTR_ESCAPE_TRUSTED_PREFIX_LEN -1 ) ==0 ; +} +bool ovl_is_private_xattr (struct super_block *sb,const char *name) {struct ovl_fs *ofs =OVL_FS (sb);@@ -82,8 +94,8 @@ static int ovl_xattr_get(struct dentry *dentry, struct inode *inode, const charstatic bool ovl_can_list (struct super_block *sb,const char *s) { -/* Never list private (.overlay) */ -if (ovl_is_private_xattr (sb, s)) +/* Never list non-escaped private (.overlay) */ +if (ovl_is_private_xattr (sb, s) && !ovl_is_escaped_xattr (sb, s))return false;/* List all non-trusted xattrs */ @@ -97,10 +109,12 @@ static bool ovl_can_list(struct super_block *sb, const char *s)ssize_t ovl_listxattr (struct dentry *dentry,char *list,size_t size) {struct dentry *realdentry =ovl_dentry_real (dentry); +struct ovl_fs *ofs =OVL_FS (dentry->d_sb);ssize_t res;size_t len;char *s;const struct cred *old_cred; +size_t prefix_len; old_cred =ovl_override_creds (dentry->d_sb); res =vfs_listxattr (realdentry, list, size);@@ -108,6 +122,9 @@ ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)if (res <=0 || size ==0 )return res; + prefix_len = ofs->config.userxattr ? +OVL_XATTR_USER_PREFIX_LEN :OVL_XATTR_TRUSTED_PREFIX_LEN ; +/* filter out private xattrs */ for (s = list, len = res; len;) {size_t slen =strnlen (s, len) +1 ;fs/overlayfs/xattrs.c ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size) · line 145
Quoted 9 lines
fs/overlayfs/xattrs.c +6
-0@@ -120,6 +137,12 @@ ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)if (!ovl_can_list (dentry->d_sb, s)) { res -= slen;memmove (s, s + slen, len); + }else if (ovl_is_escaped_xattr (dentry->d_sb, s)) { +memmove (s + prefix_len, + s + prefix_len +OVL_XATTR_ESCAPE_PREFIX_LEN , + slen - (prefix_len +OVL_XATTR_ESCAPE_PREFIX_LEN ) + len); + res -=OVL_XATTR_ESCAPE_PREFIX_LEN ; + s += slen -OVL_XATTR_ESCAPE_PREFIX_LEN ;It's a bit hard to follow. how about:
Quoted 9 lines
Quoted 6 lines
res -=
OVL_XATTR_ESCAPE_PREFIX_LEN;name_len= slen -prefix_len-OVL_XATTR_ESCAPE_PREFIX_LEN; s +=prefix_len;memmove(s, s + OVL_XATTR_ESCAPE_PREFIX_LEN, name_len + len); s +=OVL_XATTR_ESCAPE_PREFIX_LEN+name_len;The last line needs to be just s+=
name_len, but yeah, this looks nicer.
Cool. no need to post v4 just for this.
You can push to your branch and once we have feedback from Miklos we can take it from there.
Thanks,
Amir.
PATCH v3 4/6 ovl: Add an alternative type of whiteout
In reply to Alexander LarssonAn xattr whiteout (called "xwhiteout" in the code) is a reguar file of zero size with the "overlay.whiteout" xattr set. A file like this in a directory with the "overlay.whiteouts" xattrs set will be treated the same way as a regular whiteout.
The "overlay.whiteouts" directory xattr is used in order to efficiently handle overlay checks in readdir(), as we only need to checks xattrs in affected directories.
The advantage of this kind of whiteout is that they can be escaped using the standard overlay xattr escaping mechanism. So, a file with a "overlay.overlay.whiteout" xattr would be unescaped to "overlay.whiteout", which could then be consumed by another overlayfs as a whiteout.
Overlayfs itself doesn't create whiteouts like this, but a userspace mechanism could use this alternative mechanism to convert images that may contain whiteouts to be used with overlayfs.
Note: To work as a whiteout for both regular overlayfs mounts as well as userxattr mounts both the user.overlay and the trusted.overlay xattr will need to be created. So, for example, when constructing the lowerdir for a "trusted.*" mount is should contain whiteouts with both "trusted.overlay.overlay.whiteout" and "user.overlay.whiteout" set. These will be unescaped to "trusted.overlay.whiteout" and "user.overlay.whiteout", which will work like a whiteout for both kinds of overlayfs mounts.
--- fs/overlayfs/dir.c | 4 ++-- fs/overlayfs/namei.c | 15 ++++++++++----- fs/overlayfs/overlayfs.h | 15 +++++++++++++++ fs/overlayfs/readdir.c | 27 ++++++++++++++++++++------- fs/overlayfs/super.c | 2 +- fs/overlayfs/util.c | 40 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 88 insertions(+), 15 deletions(-)
fs/overlayfs/dir.c +2 -2
@@ -477,7 +477,7 @@ static int ovl_create_over_whiteout(struct dentry *dentry, struct inode *inode, goto out_unlock; err = -ESTALE ; -if (d_is_negative (upper) || !IS_WHITEOUT (d_inode (upper))) +if (d_is_negative (upper) || !ovl_upper_is_whiteout (ofs, upper)) goto out_dput; newdentry =ovl_create_temp (ofs, workdir, cattr);
@@ -1219,7 +1219,7 @@ static int ovl_rename(struct mnt_idmap *idmap, struct inode *olddir, } }else {if (!d_is_negative (newdentry)) { -if (!new_opaque || !ovl_is_whiteout (newdentry)) +if (!new_opaque || !ovl_upper_is_whiteout (ofs, newdentry)) goto out_dput; }else {if (flags &RENAME_EXCHANGE )
fs/overlayfs/namei.c +10 -5
@@ -251,7 +251,10 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, err = -EREMOTE ; goto out_err; } -if (ovl_is_whiteout (this)) { + + path.dentry = this; + path.mnt = d->mnt; +if (ovl_path_is_whiteout (OVL_FS (d->sb), &path)) { d->stop = d->opaque = true; goto put_and_out; }
@@ -264,8 +267,6 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, goto put_and_out; } - path.dentry = this; - path.mnt = d->mnt;if (!d_can_lookup (this)) {if (d->is_dir || !last_element) { d->stop = true;
@@ -438,7 +439,7 @@ int ovl_check_origin_fh(struct ovl_fs *ofs, struct ovl_fh *fh, bool connected,else if (IS_ERR (origin))return PTR_ERR (origin); -if (upperdentry && !ovl_is_whiteout (upperdentry) && +if (upperdentry && !ovl_upper_is_whiteout (ofs, upperdentry) &&inode_wrong_type (d_inode (upperdentry),d_inode (origin)->i_mode)) goto invalid;
@@ -1383,7 +1384,11 @@ bool ovl_lower_positive(struct dentry *dentry)break ; } }else { - positive = !ovl_is_whiteout (this); +struct path path = { + .dentry = this, + .mnt = parentpath->layer->mnt, + }; + positive = !ovl_path_is_whiteout (OVL_FS (dentry->d_sb), &path); done = true;dput (this); }
fs/overlayfs/overlayfs.h +15 -0
@@ -49,6 +49,8 @@ enum ovl_xattr {OVL_XATTR_UUID ,OVL_XATTR_METACOPY ,OVL_XATTR_PROTATTR , +OVL_XATTR_XWHITEOUT , + OVL_XATTR_XWHITEOUTS, }; enum ovl_inode_flag {
@@ -469,6 +471,7 @@ void ovl_inode_update(struct inode *inode, struct dentry *upperdentry);void ovl_dir_modified (struct dentry *dentry,bool impurity);u64 ovl_inode_version_get (struct inode *inode);bool ovl_is_whiteout (struct dentry *dentry); +bool ovl_path_is_whiteout (struct ovl_fs *ofs,const struct path *path);struct file *ovl_path_open (const struct path *path,int flags);int ovl_copy_up_start (struct dentry *dentry,int flags);void ovl_copy_up_end (struct dentry *dentry);
@@ -476,9 +479,21 @@ bool ovl_already_copied_up(struct dentry *dentry, int flags);bool ovl_path_check_dir_xattr (struct ovl_fs *ofs,const struct path *path,enum ovl_xattr ox);bool ovl_path_check_origin_xattr (struct ovl_fs *ofs,const struct path *path); +bool ovl_path_check_xwhiteout_xattr (struct ovl_fs *ofs,const struct path *path); +bool ovl_path_check_xwhiteouts_xattr (struct ovl_fs *ofs,const struct path *path);bool ovl_init_uuid_xattr (struct super_block *sb,struct ovl_fs *ofs,const struct path *upperpath); +static inline bool ovl_upper_is_whiteout (struct ovl_fs *ofs, +struct dentry *upperdentry) +{ +struct path upperpath = { + .dentry = upperdentry, + .mnt =ovl_upper_mnt (ofs), + }; +return ovl_path_is_whiteout (ofs, &upperpath); +} +static inline bool ovl_check_origin_xattr (struct ovl_fs *ofs,struct dentry *upperdentry) {
fs/overlayfs/readdir.c +20 -7
@@ -25,6 +25,7 @@ struct ovl_cache_entry {struct ovl_cache_entry *next_maybe_whiteout;bool is_upper;bool is_whiteout; +bool check_xwhiteout;char name[]; };
@@ -47,6 +48,7 @@ struct ovl_readdir_data {int err;bool is_upper;bool d_type_supported; +bool in_xwhiteouts_dir; };struct ovl_dir_file {
@@ -162,6 +164,8 @@ static struct ovl_cache_entry *ovl_cache_entry_new(struct ovl_readdir_data *rdd, p->ino =0 ; p->is_upper = rdd->is_upper; p->is_whiteout = false; +/* Defer check for overlay.whiteout to ovl_iterate() */ + p->check_xwhiteout = rdd->in_xwhiteouts_dir && d_type ==DT_REG ;if (d_type ==DT_CHR ) { p->next_maybe_whiteout = rdd->first_maybe_whiteout;
@@ -301,6 +305,8 @@ static inline int ovl_dir_read(const struct path *realpath,if (IS_ERR (realfile))return PTR_ERR (realfile); + rdd->in_xwhiteouts_dir = rdd->dentry && +ovl_path_check_xwhiteouts_xattr (OVL_FS (rdd->dentry->d_sb), realpath); rdd->first_maybe_whiteout =NULL ; rdd->ctx.pos =0 ; do {
@@ -447,7 +453,7 @@ static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int fsid, }/* -* Set d_ino for upper entries. Non-upper entries should always report +* Set d_ino for upper entries if needed. Non-upper entries should always report * the uppermost real inode ino and should not call this function. * * When not all layer are on same fs, report real ino also for upper.
@@ -455,8 +461,11 @@ static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int fsid,* When all layers are on the same fs, and upper has a reference to * copy up origin, call vfs_getattr() on the overlay entry to make * sure that d_ino will be consistent with st_ino from stat(2). +* +* Also checks the overlay.whiteout xattr by doing a full lookup which will return +* negative in this case. */ -static int ovl_cache_update_ino (const struct path *path,struct ovl_cache_entry *p) +static int ovl_cache_update (const struct path *path,struct ovl_cache_entry *p,bool update_ino) {struct dentry *dir = path->dentry;
@@ -467,7 +476,7 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entryint xinobits =ovl_xino_bits (ofs);int err =0 ; -if (!ovl_same_dev (ofs)) +if (!ovl_same_dev (ofs) && !p->check_xwhiteout) goto out;if (p->name[0 ] =='.' ) {
@@ -481,6 +490,7 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entry goto get; } } +/* This checks also for xwhiteouts */ this =lookup_one (mnt_idmap (path->mnt), p->name, dir, p->len);if (IS_ERR_OR_NULL (this) || !this->d_inode) {/* Mark a stale entry */
@@ -494,6 +504,9 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entry } get: +if (!ovl_same_dev (ofs) || !update_ino) + goto out; + type =ovl_path_type (this);if (OVL_TYPE_ORIGIN (type)) {struct kstat stat;
@@ -572,7 +585,7 @@ static int ovl_dir_read_impure(const struct path *path, struct list_head *list,list_for_each_entry_safe (p, n, list, l_node) {if (strcmp (p->name,"." ) !=0 &&strcmp (p->name,".." ) !=0 ) { - err =ovl_cache_update_ino (path, p); + err =ovl_cache_update (path, p, true);if (err)return err; }
@@ -778,13 +791,13 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)while (od->cursor != &od->cache->entries) { p = list_entry(od->cursor, struct ovl_cache_entry, l_node);if (!p->is_whiteout) { -if (!p->ino) { - err =ovl_cache_update_ino (&file->f_path, p); +if (!p->ino || p->check_xwhiteout) { + err =ovl_cache_update (&file->f_path, p, !p->ino);if (err) goto out; } } -/* ovl_cache_update_ino() sets is_whiteout on stale entry */ +/* ovl_cache_update() sets is_whiteout on stale entry */ if (!p->is_whiteout) {if (!dir_emit (ctx, p->name, p->len, p->ino, p->type))break ;
fs/overlayfs/super.c +1 -1
@@ -579,7 +579,7 @@ static int ovl_check_rename_whiteout(struct ovl_fs *ofs)if (IS_ERR (whiteout)) goto cleanup_temp; - err =ovl_is_whiteout (whiteout); + err =ovl_upper_is_whiteout (ofs, whiteout);/* Best effort cleanup of whiteout and temp file */ if (err)
fs/overlayfs/util.c +40 -0
@@ -575,6 +575,16 @@ bool ovl_is_whiteout(struct dentry *dentry)return inode &&IS_WHITEOUT (inode); } +/* +* Use this over ovl_is_whiteout for upper and lower files, as it also +* handles overlay.whiteout xattr whiteout files. +*/ +bool ovl_path_is_whiteout (struct ovl_fs *ofs,const struct path *path) +{ +return ovl_is_whiteout (path->dentry) || +ovl_path_check_xwhiteout_xattr (ofs, path); +} +struct file *ovl_path_open (const struct path *path,int flags) {struct inode *inode =d_inode (path->dentry);
@@ -676,6 +686,32 @@ bool ovl_path_check_origin_xattr(struct ovl_fs *ofs, const struct path *path)return false; } +bool ovl_path_check_xwhiteout_xattr (struct ovl_fs *ofs,const struct path *path) +{ +struct dentry *dentry = path->dentry; +int res; + +/* xattr.whiteout must be a zero size regular file */ +if (!d_is_reg (dentry) ||i_size_read (d_inode (dentry)) !=0 ) +return false; + + res =ovl_path_getxattr (ofs, path,OVL_XATTR_XWHITEOUT ,NULL ,0 ); +return res >=0 ; +} + +bool ovl_path_check_xwhiteouts_xattr (struct ovl_fs *ofs,const struct path *path) +{ +struct dentry *dentry = path->dentry; +int res; + +/* xattr.whiteouts must be a directory */ +if (!d_is_dir (dentry)) +return false; + + res =ovl_path_getxattr (ofs, path,OVL_XATTR_XWHITEOUTS ,NULL ,0 ); +return res >=0 ; +} +/* * Load persistent uuid from xattr into s_uuid if found, or store a new * random generated value in s_uuid and in xattr.
@@ -760,6 +796,8 @@ bool ovl_path_check_dir_xattr(struct ovl_fs *ofs, const struct path *path,#define OVL_XATTR_UUID_POSTFIX "uuid"#define OVL_XATTR_METACOPY_POSTFIX "metacopy"#define OVL_XATTR_PROTATTR_POSTFIX "protattr" +#define OVL_XATTR_XWHITEOUT_POSTFIX "whiteout" +#define OVL_XATTR_XWHITEOUTS_POSTFIX "whiteouts"#define OVL_XATTR_TAB_ENTRY (x) \ [x] = { [false] = OVL_XATTR_TRUSTED_PREFIX x ## _POSTFIX, \
@@ -775,6 +813,8 @@ const char *const ovl_xattr_table[][2] = {OVL_XATTR_TAB_ENTRY (OVL_XATTR_UUID ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_METACOPY ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_PROTATTR ), +OVL_XATTR_TAB_ENTRY (OVL_XATTR_XWHITEOUT ), + OVL_XATTR_TAB_ENTRY(OVL_XATTR_XWHITEOUTS), }; int ovl_check_setxattr(struct ovl_fs *ofs, struct dentry *upperdentry,
Signature · 1 line
2.41.0
Re: [PATCH v3 4/6] ovl: Add an alternative type of whiteout
Quoted 20 lines · Alexander Larsson
An xattr whiteout (called "xwhiteout" in the code) is a reguar file of zero size with the "overlay.whiteout" xattr set. A file like this in a directory with the "overlay.whiteouts" xattrs set will be treated the same way as a regular whiteout.
The "overlay.whiteouts" directory xattr is used in order to efficiently handle overlay checks in
readdir(), as we only need to checks xattrs in affected directories.The advantage of this kind of whiteout is that they can be escaped using the standard overlay xattr escaping mechanism. So, a file with a "overlay.overlay.whiteout" xattr would be unescaped to "overlay.whiteout", which could then be consumed by another overlayfs as a whiteout.
Overlayfs itself doesn't create whiteouts like this, but a userspace mechanism could use this alternative mechanism to convert images that may contain whiteouts to be used with overlayfs.
Quoted 8 lines
Note: To work as a whiteout for both regular overlayfs mounts as well as userxattr mounts both the user.overlay and the trusted.overlay xattr will need to be created. So, for example, when constructing the lowerdir for a "trusted.*" mount is should contain whiteouts with both "trusted.overlay.overlay.whiteout" and "user.overlay.whiteout" set. These will be unescaped to "trusted.overlay.whiteout" and "user.overlay.whiteout", which will work like a whiteout for both kinds of overlayfs mounts.
I find this paragraph to be over complicated.
In the context of this commit, it should be enough to say that
"To work as a whiteout for both regular overlayfs mounts as well
as userxattr mounts both the "user.overlay.whiteout*" and the "trusted.overlay.whiteout*" xattrs will need to be created."
How these would look inside an escaped lowerdir in the underlying filesystem if the layer is composed on an overlayfs mount is way out of context here IMO.
The paragraph above that explains that the use case for the alternative whiteout in escaped overlayfs is all the mention that's needed IMO.
Otherwise, you may add:
Quoted 324 lines
Signed-off-by Alexander Larsson <[email protected]>--- fs/overlayfs/dir.c | 4 ++-- fs/overlayfs/namei.c | 15 ++++++++++----- fs/overlayfs/overlayfs.h | 15 +++++++++++++++ fs/overlayfs/readdir.c | 27 ++++++++++++++++++++------- fs/overlayfs/super.c | 2 +- fs/overlayfs/util.c | 40 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 88 insertions(+), 15 deletions(-)fs/overlayfs/dir.c +2
-2@@ -477,7 +477,7 @@ static int ovl_create_over_whiteout(struct dentry *dentry, struct inode *inode, goto out_unlock; err = -ESTALE ; -if (d_is_negative (upper) || !IS_WHITEOUT (d_inode (upper))) +if (d_is_negative (upper) || !ovl_upper_is_whiteout (ofs, upper)) goto out_dput; newdentry =ovl_create_temp (ofs, workdir, cattr);@@ -1219,7 +1219,7 @@ static int ovl_rename(struct mnt_idmap *idmap, struct inode *olddir, } }else {if (!d_is_negative (newdentry)) { -if (!new_opaque || !ovl_is_whiteout (newdentry)) +if (!new_opaque || !ovl_upper_is_whiteout (ofs, newdentry)) goto out_dput; }else {if (flags &RENAME_EXCHANGE )fs/overlayfs/namei.c +10
-5@@ -251,7 +251,10 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, err = -EREMOTE ; goto out_err; } -if (ovl_is_whiteout (this)) { + + path.dentry = this; + path.mnt = d->mnt; +if (ovl_path_is_whiteout (OVL_FS (d->sb), &path)) { d->stop = d->opaque = true; goto put_and_out; }@@ -264,8 +267,6 @@ static int ovl_lookup_single(struct dentry *base, struct ovl_lookup_data *d, goto put_and_out; } - path.dentry = this; - path.mnt = d->mnt;if (!d_can_lookup (this)) {if (d->is_dir || !last_element) { d->stop = true;@@ -438,7 +439,7 @@ int ovl_check_origin_fh(struct ovl_fs *ofs, struct ovl_fh *fh, bool connected,else if (IS_ERR (origin))return PTR_ERR (origin); -if (upperdentry && !ovl_is_whiteout (upperdentry) && +if (upperdentry && !ovl_upper_is_whiteout (ofs, upperdentry) &&inode_wrong_type (d_inode (upperdentry),d_inode (origin)->i_mode)) goto invalid;@@ -1383,7 +1384,11 @@ bool ovl_lower_positive(struct dentry *dentry)break ; } }else { - positive = !ovl_is_whiteout (this); +struct path path = { + .dentry = this, + .mnt = parentpath->layer->mnt, + }; + positive = !ovl_path_is_whiteout (OVL_FS (dentry->d_sb), &path); done = true;dput (this); }fs/overlayfs/overlayfs.h +15
-0@@ -49,6 +49,8 @@ enum ovl_xattr {OVL_XATTR_UUID ,OVL_XATTR_METACOPY ,OVL_XATTR_PROTATTR , +OVL_XATTR_XWHITEOUT , + OVL_XATTR_XWHITEOUTS, }; enum ovl_inode_flag {@@ -469,6 +471,7 @@ void ovl_inode_update(struct inode *inode, struct dentry *upperdentry);void ovl_dir_modified (struct dentry *dentry,bool impurity);u64 ovl_inode_version_get (struct inode *inode);bool ovl_is_whiteout (struct dentry *dentry); +bool ovl_path_is_whiteout (struct ovl_fs *ofs,const struct path *path);struct file *ovl_path_open (const struct path *path,int flags);int ovl_copy_up_start (struct dentry *dentry,int flags);void ovl_copy_up_end (struct dentry *dentry);@@ -476,9 +479,21 @@ bool ovl_already_copied_up(struct dentry *dentry, int flags);bool ovl_path_check_dir_xattr (struct ovl_fs *ofs,const struct path *path,enum ovl_xattr ox);bool ovl_path_check_origin_xattr (struct ovl_fs *ofs,const struct path *path); +bool ovl_path_check_xwhiteout_xattr (struct ovl_fs *ofs,const struct path *path); +bool ovl_path_check_xwhiteouts_xattr (struct ovl_fs *ofs,const struct path *path);bool ovl_init_uuid_xattr (struct super_block *sb,struct ovl_fs *ofs,const struct path *upperpath); +static inline bool ovl_upper_is_whiteout (struct ovl_fs *ofs, +struct dentry *upperdentry) +{ +struct path upperpath = { + .dentry = upperdentry, + .mnt =ovl_upper_mnt (ofs), + }; +return ovl_path_is_whiteout (ofs, &upperpath); +} +static inline bool ovl_check_origin_xattr (struct ovl_fs *ofs,struct dentry *upperdentry) {fs/overlayfs/readdir.c +20
-7@@ -25,6 +25,7 @@ struct ovl_cache_entry {struct ovl_cache_entry *next_maybe_whiteout;bool is_upper;bool is_whiteout; +bool check_xwhiteout;char name[]; };@@ -47,6 +48,7 @@ struct ovl_readdir_data {int err;bool is_upper;bool d_type_supported; +bool in_xwhiteouts_dir; };struct ovl_dir_file {@@ -162,6 +164,8 @@ static struct ovl_cache_entry *ovl_cache_entry_new(struct ovl_readdir_data *rdd, p->ino =0 ; p->is_upper = rdd->is_upper; p->is_whiteout = false; +/* Defer check for overlay.whiteout to ovl_iterate() */ + p->check_xwhiteout = rdd->in_xwhiteouts_dir && d_type ==DT_REG ;if (d_type ==DT_CHR ) { p->next_maybe_whiteout = rdd->first_maybe_whiteout;@@ -301,6 +305,8 @@ static inline int ovl_dir_read(const struct path *realpath,if (IS_ERR (realfile))return PTR_ERR (realfile); + rdd->in_xwhiteouts_dir = rdd->dentry && +ovl_path_check_xwhiteouts_xattr (OVL_FS (rdd->dentry->d_sb), realpath); rdd->first_maybe_whiteout =NULL ; rdd->ctx.pos =0 ; do {@@ -447,7 +453,7 @@ static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int fsid, }/* -* Set d_ino for upper entries. Non-upper entries should always report +* Set d_ino for upper entries if needed. Non-upper entries should always report * the uppermost real inode ino and should not call this function. * * When not all layer are on same fs, report real ino also for upper. @@ -455,8 +461,11 @@ static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int fsid,* When all layers are on the same fs, and upper has a reference to * copy up origin, call vfs_getattr() on the overlay entry to make * sure that d_ino will be consistent with st_ino from stat(2). +* +* Also checks the overlay.whiteout xattr by doing a full lookup which will return +* negative in this case. */ -static int ovl_cache_update_ino (const struct path *path,struct ovl_cache_entry *p) +static int ovl_cache_update (const struct path *path,struct ovl_cache_entry *p,bool update_ino) {struct dentry *dir = path->dentry;@@ -467,7 +476,7 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entryint xinobits =ovl_xino_bits (ofs);int err =0 ; -if (!ovl_same_dev (ofs)) +if (!ovl_same_dev (ofs) && !p->check_xwhiteout) goto out;if (p->name[0 ] =='.' ) {@@ -481,6 +490,7 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entry goto get; } } +/* This checks also for xwhiteouts */ this =lookup_one (mnt_idmap (path->mnt), p->name, dir, p->len);if (IS_ERR_OR_NULL (this) || !this->d_inode) {/* Mark a stale entry */ @@ -494,6 +504,9 @@ static int ovl_cache_update_ino(const struct path *path, struct ovl_cache_entry } get: +if (!ovl_same_dev (ofs) || !update_ino) + goto out; + type =ovl_path_type (this);if (OVL_TYPE_ORIGIN (type)) {struct kstat stat;@@ -572,7 +585,7 @@ static int ovl_dir_read_impure(const struct path *path, struct list_head *list,list_for_each_entry_safe (p, n, list, l_node) {if (strcmp (p->name,"." ) !=0 &&strcmp (p->name,".." ) !=0 ) { - err =ovl_cache_update_ino (path, p); + err =ovl_cache_update (path, p, true);if (err)return err; }@@ -778,13 +791,13 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)while (od->cursor != &od->cache->entries) { p = list_entry(od->cursor, struct ovl_cache_entry, l_node);if (!p->is_whiteout) { -if (!p->ino) { - err =ovl_cache_update_ino (&file->f_path, p); +if (!p->ino || p->check_xwhiteout) { + err =ovl_cache_update (&file->f_path, p, !p->ino);if (err) goto out; } } -/* ovl_cache_update_ino() sets is_whiteout on stale entry */ +/* ovl_cache_update() sets is_whiteout on stale entry */ if (!p->is_whiteout) {if (!dir_emit (ctx, p->name, p->len, p->ino, p->type))break ;fs/overlayfs/super.c +1
-1@@ -579,7 +579,7 @@ static int ovl_check_rename_whiteout(struct ovl_fs *ofs)if (IS_ERR (whiteout)) goto cleanup_temp; - err =ovl_is_whiteout (whiteout); + err =ovl_upper_is_whiteout (ofs, whiteout);/* Best effort cleanup of whiteout and temp file */ if (err)fs/overlayfs/util.c +40
-1@@ -575,6 +575,16 @@ bool ovl_is_whiteout(struct dentry *dentry)return inode &&IS_WHITEOUT (inode); } +/* +* Use this over ovl_is_whiteout for upper and lower files, as it also +* handles overlay.whiteout xattr whiteout files. +*/ +bool ovl_path_is_whiteout (struct ovl_fs *ofs,const struct path *path) +{ +return ovl_is_whiteout (path->dentry) || +ovl_path_check_xwhiteout_xattr (ofs, path); +} +struct file *ovl_path_open (const struct path *path,int flags) {struct inode *inode =d_inode (path->dentry);@@ -676,6 +686,32 @@ bool ovl_path_check_origin_xattr(struct ovl_fs *ofs, const struct path *path)return false; } +bool ovl_path_check_xwhiteout_xattr (struct ovl_fs *ofs,const struct path *path) +{ +struct dentry *dentry = path->dentry; +int res; + +/* xattr.whiteout must be a zero size regular file */ +if (!d_is_reg (dentry) ||i_size_read (d_inode (dentry)) !=0 ) +return false; + + res =ovl_path_getxattr (ofs, path,OVL_XATTR_XWHITEOUT ,NULL ,0 ); +return res >=0 ; +} + +bool ovl_path_check_xwhiteouts_xattr (struct ovl_fs *ofs,const struct path *path) +{ +struct dentry *dentry = path->dentry; +int res; + +/* xattr.whiteouts must be a directory */ +if (!d_is_dir (dentry)) +return false; + + res =ovl_path_getxattr (ofs, path,OVL_XATTR_XWHITEOUTS ,NULL ,0 ); +return res >=0 ; +} +/* * Load persistent uuid from xattr into s_uuid if found, or store a new * random generated value in s_uuid and in xattr. @@ -760,6 +796,8 @@ bool ovl_path_check_dir_xattr(struct ovl_fs *ofs, const struct path *path,#define OVL_XATTR_UUID_POSTFIX "uuid"#define OVL_XATTR_METACOPY_POSTFIX "metacopy"#define OVL_XATTR_PROTATTR_POSTFIX "protattr" +#define OVL_XATTR_XWHITEOUT_POSTFIX "whiteout" +#define OVL_XATTR_XWHITEOUTS_POSTFIX "whiteouts"#define OVL_XATTR_TAB_ENTRY (x) \ [x] = { [false] = OVL_XATTR_TRUSTED_PREFIX x ## _POSTFIX, \@@ -775,6 +813,8 @@ const char *const ovl_xattr_table[][2] = {OVL_XATTR_TAB_ENTRY (OVL_XATTR_UUID ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_METACOPY ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_PROTATTR ), +OVL_XATTR_TAB_ENTRY (OVL_XATTR_XWHITEOUT ), + OVL_XATTR_TAB_ENTRY(OVL_XATTR_XWHITEOUTS), }; int ovl_check_setxattr(struct ovl_fs *ofs, struct dentry *upperdentry, --2.41.0
PATCH v3 5/6 ovl: Handle escaped xwhiteouts across layers
In reply to Alexander LarssonWe use the "overlay.whiteouts" to mark any directory in a lower dir that may contain "overlay.whiteout" files (xwhiteouts). This works even if other layers overlap that directory in a mount.
For example, take these files in three layers:
layer2/dir/ - overlay.whiteouts
layer2/dir/file - overlay.whiteout
layer1/dir/
layer1/dir/file
An overlayfs mount with -o lowerdir=layer2:layer1 would have a whiteout in layer2.
However, suppose you wanted to put this inside an overlayfs layer (say "layerA"). I.e. you want to escape the whiteouts above so that when the new layer is mounted using overlayfs the mount shows the above content. The natural approach is to just take each layer and escape it:
layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/file
This initially seems to work, however if there is another lowerdir (say "layerB") that overlaps the xwhiteouts dir, then this runs into problem:
layerB/layer2/dir/ - **NO overlay.overlay.whiteouts ** layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/file
If you mount this with -o lowerdir=layerB:layerA, then in the final mount, there will be no overlay.whiteouts xattrs on the "layer2/dir" merged directory, because the topmost lower dir xattrs win.
We would like this to work as is, to avoid having layer escaping depend on other layers. So, to fix this up we special case the reading of escaped "overlay.whiteouts" xattrs by looking in all layers for the first hit.
--- fs/overlayfs/xattrs.c | 46 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-)
fs/overlayfs/xattrs.c +45 -1
@@ -92,6 +92,25 @@ static int ovl_xattr_get(struct dentry *dentry, struct inode *inode, const charreturn res; } +static int ovl_xattr_get_first (struct dentry *dentry,struct inode *inode,const char *name, +void *value,size_t size) +{ +const struct cred *old_cred; +struct path realpath; +int idx, next; +int res = -ENODATA ; + + old_cred =ovl_override_creds (dentry->d_sb); +for (idx =0 ; idx !=-1 ; idx = next) { + next =ovl_path_next (idx, dentry, &realpath); + res =vfs_getxattr (mnt_idmap (realpath.mnt), realpath.dentry, name, value, size); +if (res != -ENODATA && res != -EOPNOTSUPP ) +break ; + } +revert_creds (old_cred); +return res; +} +static bool ovl_can_list (struct super_block *sb,const char *s) {/* Never list non-escaped private (.overlay) */
@@ -176,6 +195,18 @@ static char *ovl_xattr_escape_name(const char *prefix, const char *name)return escaped; } + +static int str_ends_with (const char *s,const char *sub) +{ +int slen =strlen (s); +int sublen =strlen (sub); + +if (sublen > slen) +return 0 ; + +return !memcmp (s + slen - sublen, sub, sublen); +} +static int ovl_own_xattr_get (const struct xattr_handler *handler,struct dentry *dentry,struct inode *inode,const char *name,void *buffer,size_t size)
@@ -187,7 +218,20 @@ static int ovl_own_xattr_get(const struct xattr_handler *handler,if (IS_ERR (escaped))return PTR_ERR (escaped); - r =ovl_xattr_get (dentry, inode, escaped, buffer, size); +/* +* Escaped "overlay.whiteouts" directories need to be combined across layers. +* For example, if a lower layer contains an escaped "overlay.whiteout" +* its parent directory will be marked with an escaped "overlay.whiteouts". +* The merged directory will contain a (now non-escaped) whiteout, and its +* parent should therefore be marked too. However, if a layer above the marked +* one has covers the same directory but without whiteouts the covering directory +* would not be marged, and thus the merged directory would not be marked. +*/ +if (d_is_dir (dentry) && +str_ends_with (escaped,"overlay.whiteouts" )) + r =ovl_xattr_get_first (dentry, inode, escaped, buffer, size); +else + r =ovl_xattr_get (dentry, inode, escaped, buffer, size);kfree (escaped);
Signature · 1 line
2.41.0
Amir Goldstein
maintainerRe: [PATCH v3 5/6] ovl: Handle escaped xwhiteouts across layers
Quoted 34 lines · Alexander Larsson
We use the "overlay.whiteouts" to mark any directory in a lower dir that may contain "overlay.whiteout" files (xwhiteouts). This works even if other layers overlap that directory in a mount.
For example, take these files in three layers:
layer2/dir/ - overlay.whiteouts
layer2/dir/file - overlay.whiteout
layer1/dir/
layer1/dir/fileAn overlayfs mount with -o lowerdir=layer2:layer1 would have a whiteout in layer2.
However, suppose you wanted to put this inside an overlayfs layer (say "layerA"). I.e. you want to escape the whiteouts above so that when the new layer is mounted using overlayfs the mount shows the above content. The natural approach is to just take each layer and escape it:
layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/fileThis initially seems to work, however if there is another lowerdir (say "layerB") that overlaps the xwhiteouts dir, then this runs into problem:
layerB/layer2/dir/ - **NO overlay.overlay.whiteouts ** layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/file
Quoted 8 lines
If you mount this with -o lowerdir=layerB:layerA, then in the final mount, there will be no overlay.whiteouts xattrs on the "layer2/dir" merged directory, because the topmost lower dir xattrs win.
We would like this to work as is, to avoid having layer escaping depend on other layers. So, to fix this up we special case the reading of escaped "overlay.whiteouts" xattrs by looking in all layers for the first hit.
I have a few issues with this special casing:
1. Miklos did not speak his opinion yet
2. I don't like special casing by suffix nor special casing a single xattr
3. I can think of example like this:
layerB/layer2/dir/file3
layerB/layer2/dir/ - **NO overlay.overlay.opaque ** layerA/layer2/dir/ - overlay.overlay.opaque layerA/layer2/dir/file2
layerA/layer1/dir/
layerA/layer1/dir/file1
LayerB doesn't have layer1 so it does not know that opaque is needed, therefore opaque needs to be merged as well in this case.
Can we employ the logic of ovl_xattr_get_first() for any escaped xattr prefix on a merge dir?
Thanks,
Amir.
Quoted 79 lines
Signed-off-by Alexander Larsson <[email protected]>--- fs/overlayfs/xattrs.c | 46 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-)fs/overlayfs/xattrs.c +45
-2@@ -92,6 +92,25 @@ static int ovl_xattr_get(struct dentry *dentry, struct inode *inode, const charreturn res; } +static int ovl_xattr_get_first (struct dentry *dentry,struct inode *inode,const char *name, +void *value,size_t size) +{ +const struct cred *old_cred; +struct path realpath; +int idx, next; +int res = -ENODATA ; + + old_cred =ovl_override_creds (dentry->d_sb); +for (idx =0 ; idx !=-1 ; idx = next) { + next =ovl_path_next (idx, dentry, &realpath); + res =vfs_getxattr (mnt_idmap (realpath.mnt), realpath.dentry, name, value, size); +if (res != -ENODATA && res != -EOPNOTSUPP ) +break ; + } +revert_creds (old_cred); +return res; +} +static bool ovl_can_list (struct super_block *sb,const char *s) {/* Never list non-escaped private (.overlay) */ @@ -176,6 +195,18 @@ static char *ovl_xattr_escape_name(const char *prefix, const char *name)return escaped; } + +static int str_ends_with (const char *s,const char *sub) +{ +int slen =strlen (s); +int sublen =strlen (sub); + +if (sublen > slen) +return 0 ; + +return !memcmp (s + slen - sublen, sub, sublen); +} +static int ovl_own_xattr_get (const struct xattr_handler *handler,struct dentry *dentry,struct inode *inode,const char *name,void *buffer,size_t size)@@ -187,7 +218,20 @@ static int ovl_own_xattr_get(const struct xattr_handler *handler,if (IS_ERR (escaped))return PTR_ERR (escaped); - r =ovl_xattr_get (dentry, inode, escaped, buffer, size); +/* +* Escaped "overlay.whiteouts" directories need to be combined across layers. +* For example, if a lower layer contains an escaped "overlay.whiteout" +* its parent directory will be marked with an escaped "overlay.whiteouts". +* The merged directory will contain a (now non-escaped) whiteout, and its +* parent should therefore be marked too. However, if a layer above the marked +* one has covers the same directory but without whiteouts the covering directory +* would not be marged, and thus the merged directory would not be marked. +*/ +if (d_is_dir (dentry) && +str_ends_with (escaped,"overlay.whiteouts" )) + r =ovl_xattr_get_first (dentry, inode, escaped, buffer, size); +else + r =ovl_xattr_get (dentry, inode, escaped, buffer, size);kfree (escaped); --2.41.0
A message that is not in the archive (sent off-list or not yet received): CAL7ro1F2vxqp5gwvRzavZ6QRVw57+RHxv=iHJk-Z9X3_ibg21w@mail.gmail.com
Amir Goldstein
maintainerRe: [PATCH v3 5/6] ovl: Handle escaped xwhiteouts across layers
Quoted 88 lines · Alexander Larsson
Quoted 45 lines · Amir Goldstein
Quoted 34 lines · Alexander Larsson
We use the "overlay.whiteouts" to mark any directory in a lower dir that may contain "overlay.whiteout" files (xwhiteouts). This works even if other layers overlap that directory in a mount.
For example, take these files in three layers:
layer2/dir/ - overlay.whiteouts
layer2/dir/file - overlay.whiteout
layer1/dir/
layer1/dir/fileAn overlayfs mount with -o lowerdir=layer2:layer1 would have a whiteout in layer2.
However, suppose you wanted to put this inside an overlayfs layer (say "layerA"). I.e. you want to escape the whiteouts above so that when the new layer is mounted using overlayfs the mount shows the above content. The natural approach is to just take each layer and escape it:
layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/fileThis initially seems to work, however if there is another lowerdir (say "layerB") that overlaps the xwhiteouts dir, then this runs into problem:
layerB/layer2/dir/ - **NO overlay.overlay.whiteouts ** layerA/layer2/dir/ - overlay.overlay.whiteouts layerA/layer2/dir/file - overlay.overlay.whiteout layerA/layer1/dir/
layerA/layer1/dir/fileQuoted 8 lines
If you mount this with -o lowerdir=layerB:layerA, then in the final mount, there will be no overlay.whiteouts xattrs on the "layer2/dir" merged directory, because the topmost lower dir xattrs win.
We would like this to work as is, to avoid having layer escaping depend on other layers. So, to fix this up we special case the reading of escaped "overlay.whiteouts" xattrs by looking in all layers for the first hit.
Quoted 4 lines
I have a few issues with this special casing:
1. Miklos did not speak his opinion yet2. I don't like special casing by suffix nor special casing a single xattr
For the suffix part, that is because we need to do the same for multiple escaped xattrs.
Like, if you have "overlay.overlay.overlay.whiteouts", which gets unescaped to "overlay.overlay.whiteouts" in a mount, you run into the same behaviour:
layerB/layer2/dir/file3
layerB/layer2/dir/ - **NO overlay.overlay.overlay.whiteout ** layerA/layer2/dir/ - overlay.overlay.overlay.whiteout layerA/layer2/dir/file2
layerA/layer1/dir/
layerA/layer1/dir/file1If you were to mount B:A over the 2:1 mount, and you getxattr(overlay.overlay.whiteout) on mnt/dir you would get ENODATA, unless you also special-cased the multiple escaped xattrs. So if you wanted to use this mount to nest yet another level it would not work.
But, that does speak to the general question that maybe even more xattrs need this kind of treatment.
Quoted 9 lines
3. I can think of example like this:
layerB/layer2/dir/file3
layerB/layer2/dir/ - **NO overlay.overlay.opaque ** layerA/layer2/dir/ - overlay.overlay.opaque layerA/layer2/dir/file2
layerA/layer1/dir/
layerA/layer1/dir/file1Quoted 3 lines
LayerB doesn't have layer1 so it does not know that opaque is needed, therefore opaque needs to be merged as well in this case.
I think you may be right here.
My original thought was when using the traditional mechanism for creating lower dirs, i.e. by make overlayfs create an upper and then reuse it as lower, we would not run into this situation. Because when overlayfs copy-ups "layer2/dir" to layerB it would copy the xattrs from the lower dir, so you wouldn't end up with NO overlay.overlay.opaque.
Quoted 8 lines
However,
copy_up.ctreats the escaped xattrs asovl_is_private_xattr, so they don't get copied up, even in this setup.Quoted 3 lines
Can we employ the logic of
ovl_xattr_get_first()for any escaped xattr prefix on a merge dir?We could either make escaped xattrs be copied up,
Seems to me that they should be copied up.
A copy up should not hide xattrs that are visible before copy up.
Quoted 3 lines
or we could use
ovl_xattr_get_first()for all escaped xattrs.
I'm not sure the latter is right though. What if you *wanted* to drop the opaque mark in the new image version (ie. layerB). If we always usedovl_xattr_get_first()then it would not be possible.
I must admit that I personally cannot follow all the user cases that we want to cover and I think that the opaque example above demonstrates that there is no one answer that fits all use cases.
Quoted 2 lines
Maybe it's just a fact of life that escaping xattrs like this means you just *have* to care about lower layers when escaping a layer, and we should stop trying to make it not so. However, if we do that, then we should make
ovl_copy_xattr()copy up escaped xattrs.
Not sure what you are suggesting.
If you are suggesting to drop this patch and fix the xattr escaping patch to copy up escaped xattr then I completely agree.
Thanks,
Amir.
PATCH v3 6/6 ovl: Add documentation on nesting of overlayfs mounts
In reply to Alexander Larsson--- Documentation/filesystems/overlayfs.rst | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
Documentation/filesystems/overlayfs.rst +23 -0
@@ -492,6 +492,29 @@ directory tree on the same or different underlying filesystem, and even to a different machine. With the "inodes index" feature, trying to mount the copied layers will fail the verification of the lower root file handle. +Nesting overlayfs mounts +------------------------ + +It is possible to use a lower directory that is stored on an overlayfs +mount. For regular files this does not need any special care. However, files +that have overlayfs attributes, such as whiteouts or "overlay.*" xattrs will be +interpreted by the underlying overlayfs mount and stripped out. In order to +allow the second overlayfs mount to see the attributes they must be escaped. + +Overlayfs specific xattrs are escaped by using a special prefix of +"overlay.overlay.". So, a file with a "trusted.overlay.overlay.metacopy" xattr +in the lower dir will be exposed as a regular file with a +"trusted.overlay.metacopy" xattr in the overlayfs mount. This can be nested by +repeating the prefix multiple time, as each instance only removes one prefix. + +A lower dir with a regular whiteout will always be handled by the overlayfs +mount, so to support storing an effective whiteout file in an overlayfs mount an +alternative form of whiteout is supported. This form is a regular, zero-size +file with the "overlay.whiteout" xattr set, inside a directory with the +"overlay.whiteouts" xattr set. Such whiteouts are never created by overlayfs, +but can be used by userspace tools (like containers) that generate lower layers. +These alternative whiteouts can be escaped using the standard xattr escape +mechanism in order to properly nest to any depth. Non-standard behavior ---------------------
Signature · 1 line
2.41.0
Re: [PATCH v3 6/6] ovl: Add documentation on nesting of overlayfs mounts
Quoted 2 lines · Alexander Larsson
Signed-off-by Alexander Larsson <[email protected]>
Quoted 41 lines
--- Documentation/filesystems/overlayfs.rst | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)Documentation/filesystems/overlayfs.rst +23
-1@@ -492,6 +492,29 @@ directory tree on the same or different underlying filesystem, and even to a different machine. With the "inodes index" feature, trying to mount the copied layers will fail the verification of the lower root file handle. +Nesting overlayfs mounts +------------------------ + +It is possible to use a lower directory that is stored on an overlayfs +mount. For regular files this does not need any special care. However, files +that have overlayfs attributes, such as whiteouts or "overlay.*" xattrs will be +interpreted by the underlying overlayfs mount and stripped out. In order to +allow the second overlayfs mount to see the attributes they must be escaped. + +Overlayfs specific xattrs are escaped by using a special prefix of +"overlay.overlay.". So, a file with a "trusted.overlay.overlay.metacopy" xattr +in the lower dir will be exposed as a regular file with a +"trusted.overlay.metacopy" xattr in the overlayfs mount. This can be nested by +repeating the prefix multiple time, as each instance only removes one prefix. + +A lower dir with a regular whiteout will always be handled by the overlayfs +mount, so to support storing an effective whiteout file in an overlayfs mount an +alternative form of whiteout is supported. This form is a regular, zero-size +file with the "overlay.whiteout" xattr set, inside a directory with the +"overlay.whiteouts" xattr set. Such whiteouts are never created by overlayfs, +but can be used by userspace tools (like containers) that generate lower layers. +These alternative whiteouts can be escaped using the standard xattr escape +mechanism in order to properly nest to any depth. Non-standard behavior --------------------- --2.41.0