Overlayfs SHUTDOWN ioctl
Subsystems (2)
Amir Goldstein
maintainercover Overlayfs SHUTDOWN ioctl
Miklos,
I was trying to think of a way forward w.r.t container runtime mount leaks and overlayfs mount failures, see [1][2][3].
It does not seem reasonable to expect they will fix all the mount leaks and that new ones will not show up. It's a hard problem to solve.
I posted a fix patch to mitigate the recent regression with index=off [2], but it does not seem reasonable to hold index=on feature hostage for eternity or until all mount leaks are fixed.
The proposal I have come up with is to provide an API for containers to shutdown the instance before unmount, so the leaked mounts become "zombies" with no ability to do any harm beyond hogging resources.
The SHUTDOWN ioctl, used by xfs/ext4/f2fs to stop any access to underlying blockdev is implemented in overlayfs to stop any access to underlying layers. The real objects are still referenced, but no vfs API can be called on underlying layers.
Naturally, SHUTDOWN releases the inuse locks to mitigate the mount failures.
I wrote an xfstest to verify SHUTDOWN solves the mount leak issue [5].
Thoughts?
Thanks,
Amir.
[1] https://github.com/containers/libpod/issues/3540
[2] https://github.com/moby/moby/issues/39475
[3] https://github.com/coreos/linux/pull/339
[4] https://github.com/amir73il/linux/commits/ovl-overlap-detect-regression-fix
[5] https://github.com/amir73il/xfstests/commit/a56d5560e404cc8052a8e47850676364b5e8c76c
Amir Goldstein (4):
ovl: support [S|G]ETFLAGS ioctl for directories
ovl: use generic vfs_ioc_setflags_prepare() helper
ovl: add pre/post access hooks to underlying layers
ovl: add support for SHUTDOWN ioctl
fs/overlayfs/copy_up.c | 10 +- fs/overlayfs/dir.c | 26 +++-- fs/overlayfs/file.c | 200 ++++++++++++++++++++++++++------------- fs/overlayfs/inode.c | 64 +++++++++---- fs/overlayfs/namei.c | 15 ++- fs/overlayfs/overlayfs.h | 15 ++- fs/overlayfs/ovl_entry.h | 7 ++ fs/overlayfs/readdir.c | 17 +++- fs/overlayfs/super.c | 9 +- fs/overlayfs/util.c | 75 +++++++++++++-- 10 files changed, 318 insertions(+), 120 deletions(-)
Signature · 1 line
2.17.1
Amir Goldstein
maintainerPATCH 1/4 ovl: support [S|G]ETFLAGS ioctl for directories
[S|G]ETFLAGS and FS[S|G]ETXATTR ioctls are applicable to both files and directories, so add ioctl operations to dir as well.
ifdef away compat ioctl implementation to conform to standard practice.
With this change, xfstest generic/079 which tests these ioctls on files and directories passes.
--- fs/overlayfs/file.c | 10 ++++++---- fs/overlayfs/overlayfs.h | 2 ++ fs/overlayfs/readdir.c | 4 ++++ 3 files changed, 12 insertions(+), 4 deletions(-)
fs/overlayfs/file.c +6 -4
@@ -502,7 +502,7 @@ static long ovl_ioctl_set_fsxflags(struct file *file, unsigned int cmd, ovl_fsxflags_to_iflags(fa.fsx_xflags)); } -static long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg) +long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {long ret;
@@ -527,8 +527,8 @@ static long ovl_ioctl(struct file *file, unsigned int cmd, unsigned long arg)return ret; } -static long ovl_compat_ioctl (struct file *file,unsigned int cmd, -unsigned long arg) +#ifdef CONFIG_COMPAT +long ovl_compat_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {switch (cmd) {case FS_IOC32_GETFLAGS :
@@ -545,6 +545,7 @@ static long ovl_compat_ioctl(struct file *file, unsigned int cmd,return ovl_ioctl (file, cmd, arg); } +#endif enum ovl_copyop {OVL_COPY ,
@@ -646,8 +647,9 @@ const struct file_operations ovl_file_operations = { .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, .unlocked_ioctl = ovl_ioctl, +#ifdef CONFIG_COMPAT .compat_ioctl = ovl_compat_ioctl, - +#endif .copy_file_range = ovl_copy_file_range, .remap_file_range = ovl_remap_file_range, };
fs/overlayfs/overlayfs.h +2 -0
@@ -416,6 +416,8 @@ struct dentry *ovl_create_temp(struct dentry *workdir, struct ovl_cattr *attr);/* file.c */ extern const struct file_operations ovl_file_operations; +long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg); +long ovl_compat_ioctl (struct file *file,unsigned int cmd,unsigned long arg);/* copy_up.c */ int ovl_copy_up (struct dentry *dentry);
fs/overlayfs/readdir.c +4 -0
@@ -907,6 +907,10 @@ const struct file_operations ovl_dir_operations = { .llseek = ovl_dir_llseek, .fsync = ovl_dir_fsync, .release = ovl_dir_release, + .unlocked_ioctl = ovl_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ovl_compat_ioctl, +#endif };int ovl_check_empty_dir (struct dentry *dentry,struct list_head *list)
Signature · 1 line
2.17.1
Amir Goldstein
maintainerRe: [PATCH 1/4] ovl: support [S|G]ETFLAGS ioctl for directories
Quoted 77 lines · Amir Goldstein
[S|G]ETFLAGS and FS[S|G]ETXATTR ioctls are applicable to both files and directories, so add ioctl operations to dir as well.
ifdef away compat ioctl implementation to conform to standard practice.
With this change, xfstest generic/079 which tests these ioctls on files and directories passes.
Signed-off-by Amir Goldstein <[email protected]>--- fs/overlayfs/file.c | 10 ++++++---- fs/overlayfs/overlayfs.h | 2 ++ fs/overlayfs/readdir.c | 4 ++++ 3 files changed, 12 insertions(+), 4 deletions(-)fs/overlayfs/file.c +6
-4@@ -502,7 +502,7 @@ static long ovl_ioctl_set_fsxflags(struct file *file, unsigned int cmd, ovl_fsxflags_to_iflags(fa.fsx_xflags)); } -static long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg) +long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {long ret;@@ -527,8 +527,8 @@ static long ovl_ioctl(struct file *file, unsigned int cmd, unsigned long arg)return ret; } -static long ovl_compat_ioctl (struct file *file,unsigned int cmd, -unsigned long arg) +#ifdef CONFIG_COMPAT +long ovl_compat_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {switch (cmd) {case FS_IOC32_GETFLAGS :@@ -545,6 +545,7 @@ static long ovl_compat_ioctl(struct file *file, unsigned int cmd,return ovl_ioctl (file, cmd, arg); } +#endif enum ovl_copyop {OVL_COPY ,@@ -646,8 +647,9 @@ const struct file_operations ovl_file_operations = { .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, .unlocked_ioctl = ovl_ioctl, +#ifdef CONFIG_COMPAT .compat_ioctl = ovl_compat_ioctl, - +#endif .copy_file_range = ovl_copy_file_range, .remap_file_range = ovl_remap_file_range, };fs/overlayfs/overlayfs.h +2
-0@@ -416,6 +416,8 @@ struct dentry *ovl_create_temp(struct dentry *workdir, struct ovl_cattr *attr);/* file.c */ extern const struct file_operations ovl_file_operations; +long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg); +long ovl_compat_ioctl (struct file *file,unsigned int cmd,unsigned long arg);/* copy_up.c */ int ovl_copy_up (struct dentry *dentry);fs/overlayfs/readdir.c +0
-0
fs/overlayfs/readdir.c const struct file_operations ovl_dir_operations = { · line 915
Quoted 9 lines
fs/overlayfs/readdir.c +4
-0@@ -907,6 +907,10 @@ const struct file_operations ovl_dir_operations = { .llseek = ovl_dir_llseek, .fsync = ovl_dir_fsync, .release = ovl_dir_release, + .unlocked_ioctl = ovl_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ovl_compat_ioctl, +#endif };
Big self NACK!!!
Cannot call ovl_ioctl => ovl_real_ioctl => ovl_real_fdget with a directory. If we do this need to implement ovl_dir_ioctl and refactor the ioctl helpers.
Sorry,
Amir.
Amir Goldstein
maintainerPATCH 2/4 ovl: use generic vfs_ioc_setflags_prepare() helper
In reply to Amir GoldsteinCanonalize to ioctl FS_* flags instead of inode S_* flags.
Note that we do not call the helper vfs_ioc_fssetxattr_check() for FS_IOC_FSSETXATTR ioctl. The reason is that underlying filesystem will perform all the checks. We only need to perform the capability check before overriding credentials.
--- fs/overlayfs/file.c | 62 ++++++++++++++++++++++----------------------- 1 file changed, 30 insertions(+), 32 deletions(-)
fs/overlayfs/file.c +30 -32
@@ -406,12 +406,28 @@ static long ovl_real_ioctl(struct file *file, unsigned int cmd,return ret; } +static unsigned int ovl_iflags_to_fsflags (unsigned int iflags) +{ +unsigned int flags =0 ; + +if (iflags &S_SYNC ) + flags |=FS_SYNC_FL ; +if (iflags &S_APPEND ) + flags |=FS_APPEND_FL ; +if (iflags &S_IMMUTABLE ) + flags |=FS_IMMUTABLE_FL ; +if (iflags &S_NOATIME ) + flags |=FS_NOATIME_FL ; + +return flags; +} +static long ovl_ioctl_set_flags (struct file *file,unsigned int cmd, -unsigned long arg,unsigned int iflags) +unsigned long arg,unsigned int flags) {long ret;struct inode *inode =file_inode (file); -unsigned int old_iflags; +unsigned int oldflags;if (!inode_owner_or_capable (inode))return -EACCES ;
@@ -423,10 +439,9 @@ static long ovl_ioctl_set_flags(struct file *file, unsigned int cmd,inode_lock (inode);/* Check the capability before cred override */ - ret = -EPERM ; - old_iflags =READ_ONCE (inode->i_flags); -if (((iflags ^ old_iflags) & (S_APPEND |S_IMMUTABLE )) && - !capable (CAP_LINUX_IMMUTABLE )) + oldflags =ovl_iflags_to_fsflags (READ_ONCE (inode->i_flags)); + ret =vfs_ioc_setflags_prepare (inode, oldflags, flags); +if (ret) goto unlock; ret =ovl_maybe_copy_up (file_dentry (file),O_WRONLY );
@@ -445,22 +460,6 @@ static long ovl_ioctl_set_flags(struct file *file, unsigned int cmd, } -static unsigned int ovl_fsflags_to_iflags (unsigned int flags) -{ -unsigned int iflags =0 ; - -if (flags &FS_SYNC_FL ) - iflags |=S_SYNC ; -if (flags &FS_APPEND_FL ) - iflags |=S_APPEND ; -if (flags &FS_IMMUTABLE_FL ) - iflags |=S_IMMUTABLE ; -if (flags &FS_NOATIME_FL ) - iflags |=S_NOATIME ; - -return iflags; -} -static long ovl_ioctl_set_fsflags (struct file *file,unsigned int cmd,unsigned long arg) {
@@ -469,24 +468,23 @@ static long ovl_ioctl_set_fsflags(struct file *file, unsigned int cmd,if (get_user (flags, (int __user *) arg))return -EFAULT ; -return ovl_ioctl_set_flags (file, cmd, arg, -ovl_fsflags_to_iflags (flags)); +return ovl_ioctl_set_flags (file, cmd, arg, flags); } -static unsigned int ovl_fsxflags_to_iflags (unsigned int xflags) +static unsigned int ovl_fsxflags_to_fsflags (unsigned int xflags) { -unsigned int iflags =0 ; +unsigned int flags =0 ;if (xflags &FS_XFLAG_SYNC ) - iflags |=S_SYNC ; + flags |=FS_SYNC_FL ;if (xflags &FS_XFLAG_APPEND ) - iflags |=S_APPEND ; + flags |=FS_APPEND_FL ;if (xflags &FS_XFLAG_IMMUTABLE ) - iflags |=S_IMMUTABLE ; + flags |=FS_IMMUTABLE_FL ;if (xflags &FS_XFLAG_NOATIME ) - iflags |=S_NOATIME ; + flags |=FS_NOATIME_FL ; -return iflags; +return flags; } static long ovl_ioctl_set_fsxflags(struct file *file, unsigned int cmd,
@@ -499,7 +497,7 @@ static long ovl_ioctl_set_fsxflags(struct file *file, unsigned int cmd,return -EFAULT ;return ovl_ioctl_set_flags (file, cmd, arg, -ovl_fsxflags_to_iflags (fa.fsx_xflags)); +ovl_fsxflags_to_fsflags (fa.fsx_xflags)); }long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg)
Signature · 1 line
2.17.1
Amir Goldstein
maintainerPATCH 3/4 ovl: add pre/post access hooks to underlying layers
In reply to Amir Goldsteinwrite access to underlying layers is surrounded with ovl_want_write()/ ovl_drop_write() pair and error may be returned when write access is denied.
read/lookup access to underlying layers is also surrounded with ovl_override_creds()/revert_creds(), but access cannot be denied.
Change all call sites to use ovl_override_creds()/ovl_revert_creds() pair and allow error to be returned when access to underlying layers is denied.
This is a prep patch for adding SHUTDOWN ioctl support.
--- fs/overlayfs/copy_up.c | 10 +++-- fs/overlayfs/dir.c | 26 +++++++---- fs/overlayfs/file.c | 94 +++++++++++++++++++++++++++------------- fs/overlayfs/inode.c | 64 +++++++++++++++++++-------- fs/overlayfs/namei.c | 15 ++++--- fs/overlayfs/overlayfs.h | 3 +- fs/overlayfs/readdir.c | 13 ++++-- fs/overlayfs/util.c | 26 ++++++++--- 8 files changed, 174 insertions(+), 77 deletions(-)
fs/overlayfs/copy_up.c +7 -3
@@ -850,10 +850,14 @@ static int ovl_copy_up_one(struct dentry *parent, struct dentry *dentry,int ovl_copy_up_flags (struct dentry *dentry,int flags) { -int err =0 ; -const struct cred *old_cred =ovl_override_creds (dentry->d_sb); +int err; +const struct cred *old_cred;bool disconnected = (dentry->d_flags &DCACHE_DISCONNECTED ); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) +return err; +/* * With NFS export, copy up can get called for a disconnected non-dir. * In this case, we will copy up lower inode to index dir without
@@ -886,7 +890,7 @@ int ovl_copy_up_flags(struct dentry *dentry, int flags)dput (parent);dput (next); } -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);return err; }
fs/overlayfs/dir.c +18 -8
@@ -544,7 +544,9 @@ static int ovl_create_or_link(struct dentry *dentry, struct inode *inode,if (err)return err; - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) +return err;/* * When linking a file with copy up origin into a new parent, mark the
@@ -579,7 +581,7 @@ static int ovl_create_or_link(struct dentry *dentry, struct inode *inode, err =ovl_create_over_whiteout (dentry, inode, attr); } out_revert_creds: -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);return err; }
@@ -653,9 +655,12 @@ static int ovl_set_link_redirect(struct dentry *dentry)const struct cred *old_cred;int err; - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) +return err; + err =ovl_set_redirect (dentry, false); -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);return err; }
@@ -846,12 +851,15 @@ static int ovl_do_remove(struct dentry *dentry, bool is_dir)if (err) goto out_drop_write; - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) + goto out_drop_write; +if (!lower_positive) err =ovl_remove_upper (dentry, is_dir, &list);else err =ovl_remove_and_whiteout (dentry, &list); -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);if (!err) {if (is_dir)clear_nlink (dentry->d_inode);
@@ -1096,7 +1104,9 @@ static int ovl_rename(struct inode *olddir, struct dentry *old, update_nlink = true; } - old_cred =ovl_override_creds (old->d_sb); + err =ovl_override_creds (old->d_sb, &old_cred); +if (err) + goto out_drop_write;if (!list_empty (&list)) { opaquedir =ovl_clear_empty (new, &list);
@@ -1221,7 +1231,7 @@ static int ovl_rename(struct inode *olddir, struct dentry *old, out_unlock:unlock_rename (new_upperdir, old_upperdir); out_revert_creds: -revert_creds (old_cred); +ovl_revert_creds (old->d_sb, old_cred);if (update_nlink)ovl_nlink_end (new); out_drop_write:
fs/overlayfs/file.c +64 -30
226 lines not on this page · raw message
fs/overlayfs/inode.c +45 -19
149 lines not on this page · raw message
fs/overlayfs/namei.c +10 -5
@@ -836,7 +836,10 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,if (dentry->d_name.len > ofs->namelen)return ERR_PTR (-ENAMETOOLONG ); - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) +return ERR_PTR (err); + upperdir =ovl_dentry_upper (dentry->d_parent);if (upperdir) { err =ovl_lookup_layer (upperdir, &d, &upperdentry);
@@ -1074,7 +1077,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry, goto out_free_oe; } -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);if (origin_path) {dput (origin_path->dentry);kfree (origin_path);
@@ -1101,7 +1104,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,kfree (upperredirect); out:kfree (d.redirect); -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);return ERR_PTR (err); }
@@ -1125,7 +1128,9 @@ bool ovl_lower_positive(struct dentry *dentry)if (!ovl_dentry_upper (dentry))return true; - old_cred =ovl_override_creds (dentry->d_sb); +if (ovl_override_creds (dentry->d_sb, &old_cred)) +return false; +/* Positive upper -> have to look up lower to see whether it exists */ for (i =0 ; !done && !positive && i < poe->numlower; i++) {struct dentry *this;
@@ -1155,7 +1160,7 @@ bool ovl_lower_positive(struct dentry *dentry)dput (this); } } -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);return positive; }
fs/overlayfs/overlayfs.h +2 -1
@@ -204,7 +204,8 @@ static inline bool ovl_open_flags_need_copy_up(int flags)int ovl_want_write (struct dentry *dentry);void ovl_drop_write (struct dentry *dentry);struct dentry *ovl_workdir (struct dentry *dentry); -const struct cred *ovl_override_creds (struct super_block *sb); +int ovl_override_creds (struct super_block *sb,const struct cred **old_cred); +void ovl_revert_creds (struct super_block *sb,const struct cred *old_cred);struct super_block *ovl_same_sb (struct super_block *sb);int ovl_can_decode_fh (struct super_block *sb);struct dentry *ovl_indexdir (struct super_block *sb);
fs/overlayfs/readdir.c +9 -4
@@ -271,7 +271,9 @@ static int ovl_check_whiteouts(struct dentry *dir, struct ovl_readdir_data *rdd)struct dentry *dentry;const struct cred *old_cred; - old_cred =ovl_override_creds (rdd->dentry->d_sb); + err =ovl_override_creds (rdd->dentry->d_sb, &old_cred); +if (err) +return err; err =down_write_killable (&dir->d_inode->i_rwsem);if (!err) {
@@ -286,7 +288,7 @@ static int ovl_check_whiteouts(struct dentry *dir, struct ovl_readdir_data *rdd) }inode_unlock (dir->d_inode); } -revert_creds (old_cred); +ovl_revert_creds (rdd->dentry->d_sb, old_cred);return err; }
@@ -920,9 +922,12 @@ int ovl_check_empty_dir(struct dentry *dentry, struct list_head *list)struct rb_root root =RB_ROOT ;const struct cred *old_cred; - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) +return err; + err =ovl_dir_read_merged (dentry, list, &root); -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred);if (err)return err;
fs/overlayfs/util.c +19 -7
@@ -33,11 +33,17 @@ struct dentry *ovl_workdir(struct dentry *dentry)return ofs->workdir; } -const struct cred *ovl_override_creds (struct super_block *sb) +int ovl_override_creds (struct super_block *sb,const struct cred **old_cred) {struct ovl_fs *ofs = sb->s_fs_info; -return override_creds (ofs->creator_cred); + *old_cred =override_creds (ofs->creator_cred); +return 0 ; +} + +void ovl_revert_creds (struct super_block *sb,const struct cred *old_cred) +{ +revert_creds (old_cred); }struct super_block *ovl_same_sb (struct super_block *sb)
@@ -783,7 +789,10 @@ int ovl_nlink_start(struct dentry *dentry)if (d_is_dir (dentry) || !ovl_test_flag (OVL_INDEX , inode)) goto out; - old_cred =ovl_override_creds (dentry->d_sb); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (err) + goto out; +/* * The overlay inode nlink should be incremented/decremented IFF the * upper operation succeeds, along with nlink change of upper inode.
@@ -791,7 +800,7 @@ int ovl_nlink_start(struct dentry *dentry)* value relative to the upper inode nlink in an upper inode xattr. */ err =ovl_set_nlink_upper (dentry); -revert_creds (old_cred); +ovl_revert_creds (dentry->d_sb, old_cred); out:if (err)
@@ -806,10 +815,13 @@ void ovl_nlink_end(struct dentry *dentry)if (ovl_test_flag (OVL_INDEX , inode) && inode->i_nlink ==0 ) {const struct cred *old_cred; +int err; - old_cred =ovl_override_creds (dentry->d_sb); -ovl_cleanup_index (dentry); -revert_creds (old_cred); + err =ovl_override_creds (dentry->d_sb, &old_cred); +if (!err) { +ovl_cleanup_index (dentry); +ovl_revert_creds (dentry->d_sb, old_cred); + } }ovl_inode_unlock (inode);
Signature · 1 line
2.17.1
Amir Goldstein
maintainerPATCH 4/4 ovl: add support for SHUTDOWN ioctl
In reply to Amir GoldsteinKeep accounting of active users accessing underlying layers. On SHUTDOWN ioctl, deny new users from accessing underlying layers.
After SHUTDOWN ioctl, when active users count drops to zero, we release the inuse exclusive locks on upperdir and workdir, because no user can access those dirs from this instance anymore.
This will allow container runtimes to issue the SHUTDOWN ioctl before unmounting overlayfs to mitigate overlayfs mount failures that result from mount leaks of old overlayfs mounts.
--- fs/overlayfs/file.c | 34 ++++++++++++++++++++++++++++ fs/overlayfs/overlayfs.h | 10 +++++++- fs/overlayfs/ovl_entry.h | 7 ++++++ fs/overlayfs/super.c | 9 +++----- fs/overlayfs/util.c | 49 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 102 insertions(+), 7 deletions(-)
fs/overlayfs/file.c +34 -0
@@ -530,6 +530,33 @@ static long ovl_ioctl_set_fsxflags(struct file *file, unsigned int cmd, ovl_fsxflags_to_fsflags(fa.fsx_xflags)); } +static int ovl_ioctl_shutdown (struct super_block *sb,unsigned long arg) +{ +struct ovl_fs *ofs = sb->s_fs_info; +__u32 flags; + +if (!capable (CAP_SYS_ADMIN )) +return -EPERM ; + +if (get_user (flags, (__u32 __user *)arg)) +return -EFAULT ; + +if (flags !=OVL_SHUTDOWN_FLAGS_NOSYNC ) +return -EINVAL ; + +down_write (&sb->s_umount); + +if (!ofs->goingdown) { +pr_info ("overlayfs: shutdown requested\n" ); + ofs->goingdown = true; +ovl_drop_active (ofs); + } + +up_write (&sb->s_umount); + +return 0 ; +} +long ovl_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {long ret;
@@ -548,6 +575,10 @@ long ovl_ioctl(struct file *file, unsigned int cmd, unsigned long arg) ret =ovl_ioctl_set_fsxflags (file, cmd, arg);break ; +case OVL_IOC_SHUTDOWN : + ret =ovl_ioctl_shutdown (file_inode (file)->i_sb, arg); +break ; +default : ret = -ENOTTY ; }
@@ -567,6 +598,9 @@ long ovl_compat_ioctl(struct file *file, unsigned int cmd, unsigned long arg) cmd =FS_IOC_SETFLAGS ;break ; +case OVL_IOC_SHUTDOWN : +break ; +default :return -ENOIOCTLCMD ; }
fs/overlayfs/overlayfs.h +9 -1
@@ -28,6 +28,14 @@ enum ovl_path_type {#define OVL_XATTR_UPPER OVL_XATTR_PREFIX "upper"#define OVL_XATTR_METACOPY OVL_XATTR_PREFIX "metacopy" +/* +* Should be same as XFS_IOC_GOINGDOWN. +* We only support the NOSYNC flag. +*/ +#define OVL_IOC_SHUTDOWN _IOR('X', 125, __u32) +#define OVL_SHUTDOWN_FLAGS_NOSYNC 0x2 + +enum ovl_inode_flag {/* Pure upper dir that may contain non pure upper entries */ OVL_IMPURE ,
@@ -201,6 +209,7 @@ static inline bool ovl_open_flags_need_copy_up(int flags) }/* util.c */ +void ovl_drop_active (struct ovl_fs *ofs);int ovl_want_write (struct dentry *dentry);void ovl_drop_write (struct dentry *dentry);struct dentry *ovl_workdir (struct dentry *dentry);
@@ -301,7 +310,6 @@ static inline void ovl_inode_unlock(struct inode *inode)mutex_unlock (&OVL_I (inode)->lock); } -/* namei.c */ int ovl_check_fh_len (struct ovl_fh *fh,int fh_len); struct dentry *ovl_decode_real_fh(struct ovl_fh *fh, struct vfsmount *mnt,
fs/overlayfs/ovl_entry.h +7 -0
@@ -71,6 +71,13 @@ struct ovl_fs {struct inode *indexdir_trap;/* Inode numbers in all layers do not use the high xino_bits */ unsigned int xino_bits; +/* +* Number of users currently accessing underlying layers (+1) +* When zero, access to underlying layers is denied. +*/ +atomic_t active; +/* Filesystem will shutdown when the last active reference drops */ +bool goingdown; };/* private information held for every overlayfs dentry */
fs/overlayfs/super.c +3 -6
@@ -212,17 +212,14 @@ static void ovl_free_fs(struct ovl_fs *ofs) {unsigned i; +ovl_drop_active (ofs);iput (ofs->workbasedir_trap);iput (ofs->indexdir_trap);iput (ofs->workdir_trap);iput (ofs->upperdir_trap);dput (ofs->indexdir);dput (ofs->workdir); -if (ofs->workdir_locked) -ovl_inuse_unlock (ofs->workbasedir);dput (ofs->workbasedir); -if (ofs->upperdir_locked) -ovl_inuse_unlock (ofs->upper_mnt->mnt_root);mntput (ofs->upper_mnt);for (i =0 ; i < ofs->numlower; i++) {iput (ofs->lower_layers[i].trap);
@@ -237,8 +234,6 @@ static void ovl_free_fs(struct ovl_fs *ofs)kfree (ofs->config.upperdir);kfree (ofs->config.workdir);kfree (ofs->config.redirect_mode); -if (ofs->creator_cred) -put_cred (ofs->creator_cred);kfree (ofs); }
@@ -1570,6 +1565,8 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)if (!ofs) goto out; +atomic_set (&ofs->active,1 ); + ofs->creator_cred = cred =prepare_creds ();if (!cred) goto out_err;
fs/overlayfs/util.c +49 -0
77 lines not on this page · raw message
Signature · 1 line
2.17.1
Amir Goldstein
maintainerQuoted 23 lines · Amir Goldstein
Miklos,
I was trying to think of a way forward w.r.t container runtime mount leaks and overlayfs mount failures, see [1][2][3].
It does not seem reasonable to expect they will fix all the mount leaks and that new ones will not show up. It's a hard problem to solve.
I posted a fix patch to mitigate the recent regression with index=off [2], but it does not seem reasonable to hold index=on feature hostage for eternity or until all mount leaks are fixed.
The proposal I have come up with is to provide an API for containers to shutdown the instance before unmount, so the leaked mounts become "zombies" with no ability to do any harm beyond hogging resources.
The SHUTDOWN ioctl, used by xfs/ext4/f2fs to stop any access to underlying blockdev is implemented in overlayfs to stop any access to underlying layers. The real objects are still referenced, but no vfs API can be called on underlying layers.
Quoted 6 lines
Naturally, SHUTDOWN releases the inuse locks to mitigate the mount failures.
I wrote an xfstest to verify SHUTDOWN solves the mount leak issue [5].
Thoughts?
I had one thought myself:
On kernel < v4.19 a SHUTDOWN ioctl on an overlayfs file will have a completely different consequence - the underlying fs would shutdown! So we have several options:
1. Allow SHUTDOWN only on dir inode (it's usually executed on the mount root anyway)
2. Introduce a new SHUTDOWN flag that existing no fs (kernel < v4.19) supports
3. Use one of the new mount APIs to request "umount & shutdown sb"
4. Other ideas?
I personally like the compromise of option #1, but I have not spent time studying option #3 which could be better.
Quoted 31 lines
Thanks,
Amir.[1] https://github.com/containers/libpod/issues/3540
[2] https://github.com/moby/moby/issues/39475
[3] https://github.com/coreos/linux/pull/339
[4] https://github.com/amir73il/linux/commits/ovl-overlap-detect-regression-fix
[5] https://github.com/amir73il/xfstests/commit/a56d5560e404cc8052a8e47850676364b5e8c76cAmir Goldstein (4):
ovl: support [S|G]ETFLAGS ioctl for directories
ovl: use genericvfs_ioc_setflags_prepare()helper
ovl: add pre/post access hooks to underlying layers
ovl: add support for SHUTDOWN ioctlfs/overlayfs/copy_up.c | 10 +- fs/overlayfs/dir.c | 26 +++-- fs/overlayfs/file.c | 200 ++++++++++++++++++++++++++------------- fs/overlayfs/inode.c | 64 +++++++++---- fs/overlayfs/namei.c | 15 ++- fs/overlayfs/overlayfs.h | 15 ++- fs/overlayfs/ovl_entry.h | 7 ++ fs/overlayfs/readdir.c | 17 +++- fs/overlayfs/super.c | 9 +- fs/overlayfs/util.c | 75 +++++++++++++-- 10 files changed, 318 insertions(+), 120 deletions(-)--
2.17.1