implement containerized syncfs for overlayfs
Subsystems (3)
v4 cover implement containerized syncfs for overlayfs
Current syncfs(2) syscall on overlayfs just calls sync_filesystem() on upper_sb to synchronize whole dirty inodes in upper filesystem regardless of the overlay ownership of the inode. In the use case of container, when multiple containers using the same underlying upper filesystem, it has some shortcomings as below.
(1) Performance
Synchronization is probably heavy because it actually syncs unnecessary inodes for target overlayfs.
(2) Interference
Unplanned synchronization will probably impact IO performance of unrelated container processes on the other overlayfs.
This series try to implement containerized syncfs for overlayfs so that only sync target dirty upper inodes which are belong to specific overlayfs instance. By doing this, it is able to reduce cost of synchronization and will not seriously impact IO performance of unrelated processes.
v1->v2:
- Mark overlayfs' inode dirty itself instead of adding notification mechanism to vfs inode.
v2->v3:
- Introduce overlayfs' extra syncfs wait list to wait target upper inodes in ->sync_fs.
v3->v4:
- Using wait_sb_inodes() to wait syncing upper inodes.
- Mark overlay inode dirty only when having upper inode and VM_SHARED flag in ovl_mmap().
- Check upper i_state after checking upper mmap state
in ovl_write_inode.
Chengguang Xu (9):
ovl: setup overlayfs' private bdi
ovl: implement ->writepages operation
ovl: implement overlayfs' ->evict_inode operation
ovl: mark overlayfs' inode dirty on modification
ovl: mark overlayfs' inode dirty on shared mmap
ovl: implement overlayfs' ->write_inode operation
ovl: cache dirty overlayfs' inode
fs: export wait_sb_inodes()
ovl: implement containerized syncfs for overlayfs
fs/fs-writeback.c | 3 +- fs/overlayfs/file.c | 3 ++ fs/overlayfs/inode.c | 15 ++++++++++ fs/overlayfs/overlayfs.h | 4 +++ fs/overlayfs/super.c | 63 ++++++++++++++++++++++++++++++++++++--- fs/overlayfs/util.c | 14 +++++++++ include/linux/writeback.h | 1 + 7 files changed, 98 insertions(+), 5 deletions(-)
Signature · 1 line
2.26.2
RFC PATCH v4 1/9 ovl: setup overlayfs' private bdi
Setup overlayfs' private bdi so that we can collect overlayfs' own dirty inodes.
--- fs/overlayfs/super.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/overlayfs/super.c +4 -0
@@ -1869,6 +1869,10 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)if (!ofs) goto out; + err =super_setup_bdi (sb); +if (err) + goto out_err; + ofs->creator_cred = cred =prepare_creds ();if (!cred) goto out_err;
Signature · 1 line
2.26.2
RFC PATCH v4 2/9 ovl: implement ->writepages operation
In reply to Chengguang XuImplement overlayfs' ->writepages operation so that we can sync dirty data/metadata to upper filesystem.
--- fs/overlayfs/inode.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/overlayfs/inode.c +14 -0
@@ -11,6 +11,7 @@#include <linux/posix_acl.h> #include <linux/ratelimit.h> #include <linux/fiemap.h> +#include <linux/writeback.h> #include "overlayfs.h"
@@ -516,7 +517,20 @@ static const struct inode_operations ovl_special_inode_operations = { .update_time = ovl_update_time, }; +static int ovl_writepages (struct address_space *mapping, +struct writeback_control *wbc) +{ +struct inode *inode = mapping->host; +struct ovl_fs *ofs = inode->i_sb->s_fs_info; +struct inode *upper =ovl_inode_upper (inode); + +if (!ovl_should_sync (ofs)) +return 0 ; +return sync_inode (upper, wbc); +} +static const struct address_space_operations ovl_aops = { + .writepages = ovl_writepages,/* For O_DIRECT dentry_open() checks f_mapping->a_ops->direct_IO */ .direct_IO = noop_direct_IO, };
Signature · 1 line
2.26.2
RFC PATCH v4 3/9 ovl: implement overlayfs' ->evict_inode operation
In reply to Chengguang XuImplement overlayfs' ->evict_inode operation,
so that we can clear all inode dirty flags.
--- fs/overlayfs/super.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/overlayfs/super.c +7 -0
@@ -390,11 +390,18 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data)return ret; } +static void ovl_evict_inode (struct inode *inode) +{ + inode->i_state &= ~I_DIRTY_ALL ; +clear_inode (inode); +} +static const struct super_operations ovl_super_operations = { .alloc_inode = ovl_alloc_inode, .free_inode = ovl_free_inode, .destroy_inode = ovl_destroy_inode, .drop_inode = generic_delete_inode, + .evict_inode = ovl_evict_inode, .put_super = ovl_put_super, .sync_fs = ovl_sync_fs, .statfs = ovl_statfs,
Signature · 1 line
2.26.2
RFC PATCH v4 4/9 ovl: mark overlayfs' inode dirty on modification
In reply to Chengguang XuMark overlayfs' inode dirty on modification so that we can recognize target inodes during syncfs.
--- fs/overlayfs/inode.c | 1 + fs/overlayfs/overlayfs.h | 4 ++++ fs/overlayfs/util.c | 14 ++++++++++++++ 3 files changed, 19 insertions(+)
fs/overlayfs/inode.c +1 -0
@@ -468,6 +468,7 @@ int ovl_update_time(struct inode *inode, struct timespec64 *ts, int flags)if (upperpath.dentry) {touch_atime (&upperpath); inode->i_atime =d_inode (upperpath.dentry)->i_atime; +ovl_mark_inode_dirty (inode); } }return 0 ;
fs/overlayfs/overlayfs.h +4 -0
@@ -247,6 +247,7 @@ static inline bool ovl_open_flags_need_copy_up(int flags) }/* util.c */ +void ovl_mark_inode_dirty (struct inode *inode);int ovl_want_write (struct dentry *dentry);void ovl_drop_write (struct dentry *dentry);struct dentry *ovl_workdir (struct dentry *dentry);
@@ -472,6 +473,9 @@ static inline void ovl_copyattr(struct inode *from, struct inode *to) to->i_mtime = from->i_mtime; to->i_ctime = from->i_ctime;i_size_write (to,i_size_read (from)); + +if (ovl_inode_upper (to) && from->i_state &I_DIRTY_ALL ) +ovl_mark_inode_dirty (to); }static inline void ovl_copyflags (struct inode *from,struct inode *to)
fs/overlayfs/util.c +14 -0
@@ -950,3 +950,17 @@ char *ovl_get_redirect_xattr(struct ovl_fs *ofs, struct dentry *dentry,kfree (buf);return ERR_PTR (res); } + +/* +* We intentionally add I_DIRTY_SYNC flag regardless dirty flag +* of upper inode so that we have chance to invoke ->write_inode +* to re-dirty overlayfs' inode during writeback process. +*/ +void ovl_mark_inode_dirty (struct inode *inode) +{ +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =I_DIRTY_SYNC ; + + iflag |= upper->i_state &I_DIRTY_ALL ; +__mark_inode_dirty (inode, iflag); +}
Signature · 1 line
2.26.2
Miklos Szeredi
maintainerRe: [RFC PATCH v4 4/9] ovl: mark overlayfs' inode dirty on modification
Quoted 36 lines · Chengguang Xu
Mark overlayfs' inode dirty on modification so that we can recognize target inodes during syncfs.
Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/inode.c | 1 + fs/overlayfs/overlayfs.h | 4 ++++ fs/overlayfs/util.c | 14 ++++++++++++++ 3 files changed, 19 insertions(+)fs/overlayfs/inode.c +1
-0@@ -468,6 +468,7 @@ int ovl_update_time(struct inode *inode, struct timespec64 *ts, int flags)if (upperpath.dentry) {touch_atime (&upperpath); inode->i_atime =d_inode (upperpath.dentry)->i_atime; +ovl_mark_inode_dirty (inode); } }return 0 ;fs/overlayfs/overlayfs.h +1
-0@@ -247,6 +247,7 @@ static inline bool ovl_open_flags_need_copy_up(int flags) }/* util.c */ +void ovl_mark_inode_dirty (struct inode *inode);int ovl_want_write (struct dentry *dentry);void ovl_drop_write (struct dentry *dentry);struct dentry *ovl_workdir (struct dentry *dentry);
fs/overlayfs/overlayfs.h static inline void ovl_copyattr(struct inode *from, struct inode *to) · line 479
Quoted 7 lines
fs/overlayfs/overlayfs.h +3
-0@@ -472,6 +473,9 @@ static inline void ovl_copyattr(struct inode *from, struct inode *to) to->i_mtime = from->i_mtime; to->i_ctime = from->i_ctime;i_size_write (to,i_size_read (from)); + +if (ovl_inode_upper (to) && from->i_state &I_DIRTY_ALL ) +ovl_mark_inode_dirty (to); }
Okay, ovl_copyattr() certainly seems a good place to copy dirtyness as well.
What I'm fearing is that it does not cover all the places where underlying inode can be dirtied. This really needs an audit of all filesystem modifying operations.
Quoted 28 lines
fs/overlayfs/overlayfs.h +0
-0static inline void ovl_copyflags (struct inode *from,struct inode *to)fs/overlayfs/util.c +14
-1@@ -950,3 +950,17 @@ char *ovl_get_redirect_xattr(struct ovl_fs *ofs, struct dentry *dentry,kfree (buf);return ERR_PTR (res); } + +/* +* We intentionally add I_DIRTY_SYNC flag regardless dirty flag +* of upper inode so that we have chance to invoke ->write_inode +* to re-dirty overlayfs' inode during writeback process. +*/ +void ovl_mark_inode_dirty (struct inode *inode) +{ +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =I_DIRTY_SYNC ; + + iflag |= upper->i_state &I_DIRTY_ALL ; +__mark_inode_dirty (inode, iflag); +} --2.26.2
Re: [RFC PATCH v4 4/9] ovl: mark overlayfs' inode dirty on modification
---- 在 星期五, 2021-04-09 21:45:28 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 47 lines · Chengguang Xu
Quoted 36 lines
Mark overlayfs' inode dirty on modification so that we can recognize target inodes during syncfs.
Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/inode.c | 1 + fs/overlayfs/overlayfs.h | 4 ++++ fs/overlayfs/util.c | 14 ++++++++++++++ 3 files changed, 19 insertions(+)fs/overlayfs/inode.c +1
-0@@ -468,6 +468,7 @@ int ovl_update_time(struct inode *inode, struct timespec64 *ts, int flags)if (upperpath.dentry) {touch_atime (&upperpath); inode->i_atime =d_inode (upperpath.dentry)->i_atime; +ovl_mark_inode_dirty (inode); } }return 0 ;fs/overlayfs/overlayfs.h +1
-0@@ -247,6 +247,7 @@ static inline bool ovl_open_flags_need_copy_up(int flags) }/* util.c */ +void ovl_mark_inode_dirty (struct inode *inode);int ovl_want_write (struct dentry *dentry);void ovl_drop_write (struct dentry *dentry);struct dentry *ovl_workdir (struct dentry *dentry);fs/overlayfs/overlayfs.h static inline void ovl_copyattr(struct inode *from, struct inode *to) · line 479
Quoted 7 lines
fs/overlayfs/overlayfs.h +3
-0@@ -472,6 +473,9 @@ static inline void ovl_copyattr(struct inode *from, struct inode *to) to->i_mtime = from->i_mtime; to->i_ctime = from->i_ctime;i_size_write (to,i_size_read (from)); + +if (ovl_inode_upper (to) && from->i_state &I_DIRTY_ALL ) +ovl_mark_inode_dirty (to); }Okay,
ovl_copyattr()certainly seems a good place to copy dirtyness as well.
Quoted 3 lines
What I'm fearing is that it does not cover all the places where underlying inode can be dirtied. This really needs an audit of all filesystem modifying operations.
You are right. Let me fix it in next version.
Thanks,
Chengguang
Quoted 29 lines
Quoted 28 lines
fs/overlayfs/overlayfs.h +0
-0static inline void ovl_copyflags (struct inode *from,struct inode *to)fs/overlayfs/util.c +14
-1@@ -950,3 +950,17 @@ char *ovl_get_redirect_xattr(struct ovl_fs *ofs, struct dentry *dentry,kfree (buf);return ERR_PTR (res); } + +/* +* We intentionally add I_DIRTY_SYNC flag regardless dirty flag +* of upper inode so that we have chance to invoke ->write_inode +* to re-dirty overlayfs' inode during writeback process. +*/ +void ovl_mark_inode_dirty (struct inode *inode) +{ +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =I_DIRTY_SYNC ; + + iflag |= upper->i_state &I_DIRTY_ALL ; +__mark_inode_dirty (inode, iflag); +} --2.26.2
RFC PATCH v4 5/9 ovl: mark overlayfs' inode dirty on shared mmap
In reply to Chengguang XuOverlayfs cannot be notified when mmapped area gets dirty, so we need to proactively mark inode dirty in ->mmap operation.
--- fs/overlayfs/file.c | 3 +++ 1 file changed, 3 insertions(+)
fs/overlayfs/file.c +3 -0
@@ -486,6 +486,9 @@ static int ovl_mmap(struct file *file, struct vm_area_struct *vma)/* Drop reference count from new vm_file value */ fput (realfile); }else { +if (ovl_inode_upper (file_inode (file)) && + vma->vm_flags &VM_SHARED ) +ovl_mark_inode_dirty (file_inode (file));/* Drop reference count from previous vm_file value */ fput (file); }
Signature · 1 line
2.26.2
RFC PATCH v4 6/9 ovl: implement overlayfs' ->write_inode operation
In reply to Chengguang XuImplement overlayfs' ->write_inode to sync dirty data and redirty overlayfs' inode if necessary.
--- fs/overlayfs/super.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
fs/overlayfs/super.c +30 -0
@@ -390,6 +390,35 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data)return ret; } +static int ovl_write_inode (struct inode *inode, +struct writeback_control *wbc) +{ +struct ovl_fs *ofs = inode->i_sb->s_fs_info; +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =0 ; +int ret =0 ; + +if (!upper) +return 0 ; + +if (!ovl_should_sync (ofs)) +return 0 ; + +if (upper->i_sb->s_op->write_inode) + ret = upper->i_sb->s_op->write_inode (inode, wbc); + +if (mapping_writably_mapped (upper->i_mapping) || +mapping_tagged (upper->i_mapping,PAGECACHE_TAG_WRITEBACK )) + iflag |=I_DIRTY_PAGES ; + + iflag |= upper->i_state &I_DIRTY_ALL ; + +if (iflag) +ovl_mark_inode_dirty (inode); + +return ret; +} +static void ovl_evict_inode (struct inode *inode) { inode->i_state &= ~I_DIRTY_ALL ;
@@ -402,6 +431,7 @@ static const struct super_operations ovl_super_operations = { .destroy_inode = ovl_destroy_inode, .drop_inode = generic_delete_inode, .evict_inode = ovl_evict_inode, + .write_inode = ovl_write_inode, .put_super = ovl_put_super, .sync_fs = ovl_sync_fs, .statfs = ovl_statfs,
Signature · 1 line
2.26.2
Miklos Szeredi
maintainerRe: [RFC PATCH v4 6/9] ovl: implement overlayfs' ->write_inode operation
Quoted 29 lines · Chengguang Xu
Implement overlayfs' ->write_inode to sync dirty data and redirty overlayfs' inode if necessary.
Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/super.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)fs/overlayfs/super.c +12
-0@@ -390,6 +390,35 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data)return ret; } +static int ovl_write_inode (struct inode *inode, +struct writeback_control *wbc) +{ +struct ovl_fs *ofs = inode->i_sb->s_fs_info; +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =0 ; +int ret =0 ; + +if (!upper) +return 0 ; + +if (!ovl_should_sync (ofs))
fs/overlayfs/super.c static int ovl_remount(struct super_block *sb, int *flags, char *data) · line 414
Quoted 10 lines
fs/overlayfs/super.c +10
-0+return 0 ; + +if (upper->i_sb->s_op->write_inode) + ret = upper->i_sb->s_op->write_inode (inode, wbc); + +if (mapping_writably_mapped (upper->i_mapping) || +mapping_tagged (upper->i_mapping,PAGECACHE_TAG_WRITEBACK )) + iflag |=I_DIRTY_PAGES ; + + iflag |= upper->i_state &I_DIRTY_ALL ;
How is I_DIRTY_SYNC added/removed from the overlay inode?
Thanks,
Miklos
Re: [RFC PATCH v4 6/9] ovl: implement overlayfs' ->write_inode operation
---- 在 星期五, 2021-04-09 21:49:07 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 34 lines · Chengguang Xu
Quoted 33 lines
Implement overlayfs' ->write_inode to sync dirty data and redirty overlayfs' inode if necessary.
Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/super.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)fs/overlayfs/super.c +16
-0@@ -390,6 +390,35 @@ static int ovl_remount(struct super_block *sb, int *flags, char *data)return ret; } +static int ovl_write_inode (struct inode *inode, +struct writeback_control *wbc) +{ +struct ovl_fs *ofs = inode->i_sb->s_fs_info; +struct inode *upper =ovl_inode_upper (inode); +unsigned long iflag =0 ; +int ret =0 ; + +if (!upper) +return 0 ; + +if (!ovl_should_sync (ofs)) +return 0 ; + +if (upper->i_sb->s_op->write_inode) + ret = upper->i_sb->s_op->write_inode (inode, wbc);
Quoted 9 lines
fs/overlayfs/super.c static int ovl_remount(struct super_block *sb, int *flags, char *data) · line 414
Quoted 6 lines
fs/overlayfs/super.c +6
-0+ +if (mapping_writably_mapped (upper->i_mapping) || +mapping_tagged (upper->i_mapping,PAGECACHE_TAG_WRITEBACK )) + iflag |=I_DIRTY_PAGES ; + + iflag |= upper->i_state &I_DIRTY_ALL ;How is
I_DIRTY_SYNCadded/removed from the overlay inode?
generally, I_DIRTY_SYNC is added to overlay inode when the operation dirties upper inode, I'll check all those places and call ovl_mark_inode_dirty() to do it. After writeback if upper inode becomes clean status, we will remove I_DIRTY_SYNC from overlay inode.
One exception is mmaped file, it will always keep I_DIRTY_SYNC until to be evicted.
Thanks,
Chengguang
RFC PATCH v4 7/9 ovl: cache dirty overlayfs' inode
In reply to Chengguang XuNow drop overlayfs' inode will sync dirty data,
so we change to only drop clean inode.
--- fs/overlayfs/super.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
fs/overlayfs/super.c +10 -1
@@ -425,11 +425,20 @@ static void ovl_evict_inode(struct inode *inode)clear_inode (inode); } +static int ovl_drop_inode (struct inode *inode) +{ +struct inode *upper =ovl_inode_upper (inode); + +if (!upper || !(inode->i_state &I_DIRTY_ALL )) +return 1 ; +return generic_drop_inode (inode); +} +static const struct super_operations ovl_super_operations = { .alloc_inode = ovl_alloc_inode, .free_inode = ovl_free_inode, .destroy_inode = ovl_destroy_inode, - .drop_inode = generic_delete_inode, + .drop_inode = ovl_drop_inode, .evict_inode = ovl_evict_inode, .write_inode = ovl_write_inode, .put_super = ovl_put_super,
Signature · 1 line
2.26.2
Miklos Szeredi
maintainerRe: [RFC PATCH v4 7/9] ovl: cache dirty overlayfs' inode
Quoted 3 lines · Chengguang Xu
Now drop overlayfs' inode will sync dirty data,
so we change to only drop clean inode.
I don't understand what happens here. Please add more explanation.
Thanks,
Miklos
Re: [RFC PATCH v4 7/9] ovl: cache dirty overlayfs' inode
---- 在 星期五, 2021-04-09 21:50:35 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 6 lines · Chengguang Xu
Quoted 3 lines
Now drop overlayfs' inode will sync dirty data,
so we change to only drop clean inode.I don't understand what happens here. Please add more explanation.
In iput_final(), clean overlayfs inode will directly drop as the same as before, dirty overlayfs inode will keep in the cache to wait writeback to sync dirty data and then add to lru list to wait reclaim.
The purpose of doing this is to keep compatible behavior with original one, because without this series, dropping overlayfs inode will not trigger syncing underlying dirty inode.
Thanks,
Chengguang
Miklos Szeredi
maintainerRe: [RFC PATCH v4 7/9] ovl: cache dirty overlayfs' inode
Quoted 9 lines · Chengguang Xu
---- 在 星期五, 2021-04-09 21:50:35 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 6 lines · Chengguang Xu
Quoted 3 lines
Now drop overlayfs' inode will sync dirty data,
so we change to only drop clean inode.I don't understand what happens here. Please add more explanation.
Quoted 7 lines
In
iput_final(), clean overlayfs inode will directly drop as the same as before, dirty overlayfs inode will keep in the cache to wait writeback to sync dirty data and then add to lru list to wait reclaim.The purpose of doing this is to keep compatible behavior with original one, because without this series, dropping overlayfs inode will not trigger syncing underlying dirty inode.
I get it now. Can you please update the patch header with this description?
Thanks,
Miklos
Re: [RFC PATCH v4 7/9] ovl: cache dirty overlayfs' inode
---- 在 星期二, 2021-04-13 16:43:27 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 14 lines · Chengguang Xu
Quoted 13 lines
---- 在 星期五, 2021-04-09 21:50:35 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 6 lines · Chengguang Xu
Quoted 3 lines
Now drop overlayfs' inode will sync dirty data,
so we change to only drop clean inode.I don't understand what happens here. Please add more explanation.
In
iput_final(), clean overlayfs inode will directly drop as the same as before, dirty overlayfs inode will keep in the cache to wait writeback to sync dirty data and then add to lru list to wait reclaim.
Quoted 6 lines
Quoted 3 lines
The purpose of doing this is to keep compatible behavior with original one, because without this series, dropping overlayfs inode will not trigger syncing underlying dirty inode.
I get it now. Can you please update the patch header with this description?
No problem, I'll update commit log in next version.
Thanks,
Chengguang
RFC PATCH v4 8/9 fs: export wait_sb_inodes()
In reply to Chengguang XuIn order to wait syncing upper inodes we need to call wait_sb_inodes() in overlayfs' ->sync_fs.
--- fs/fs-writeback.c | 3 ++- include/linux/writeback.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
fs/fs-writeback.c +2 -1
@@ -2365,7 +2365,7 @@ EXPORT_SYMBOL(__mark_inode_dirty);* completed by the time we have gained the lock and waited for all IO that is * in progress regardless of the order callers are granted the lock. */ -static void wait_sb_inodes (struct super_block *sb) +void wait_sb_inodes (struct super_block *sb) {LIST_HEAD (sync_list);
@@ -2449,6 +2449,7 @@ static void wait_sb_inodes(struct super_block *sb)rcu_read_unlock ();mutex_unlock (&sb->s_sync_lock); } +EXPORT_SYMBOL (wait_sb_inodes);static void __writeback_inodes_sb_nr (struct super_block *sb,unsigned long nr,enum wb_reason reason,bool skip_if_busy)
include/linux/writeback.h +1 -0
@@ -198,6 +198,7 @@ void wakeup_flusher_threads_bdi(struct backing_dev_info *bdi, enum wb_reason reason);void inode_wait_for_writeback (struct inode *inode);void inode_io_list_del (struct inode *inode); +void wait_sb_inodes (struct super_block *sb);/* writeback.h requires fs.h; it, too, is not included from here. */ static inline void wait_on_inode (struct inode *inode)
Signature · 1 line
2.26.2
RFC PATCH v4 9/9 ovl: implement containerized syncfs for overlayfs
In reply to Chengguang XuNow overlayfs can only sync dirty inode during syncfs, so remove unnecessary sync_filesystem() on upper file system.
--- fs/overlayfs/super.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
fs/overlayfs/super.c +8 -3
@@ -15,6 +15,8 @@#include <linux/seq_file.h> #include <linux/posix_acl_xattr.h> #include <linux/exportfs.h> +#include <linux/blkdev.h> +#include <linux/writeback.h> #include "overlayfs.h" MODULE_AUTHOR ("Miklos Szeredi <[email protected]>" );
@@ -270,8 +272,7 @@ static int ovl_sync_fs(struct super_block *sb, int wait)* Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC). * All the super blocks will be iterated, including upper_sb. * -* If this is a syncfs(2) call, then we do need to call -* sync_filesystem() on upper_sb, but enough if we do it when being +* if this is a syncfs(2) call, it will be enough we do it when being * called with wait == 1. */ if (!wait)
@@ -280,7 +281,11 @@ static int ovl_sync_fs(struct super_block *sb, int wait) upper_sb =ovl_upper_mnt (ofs)->mnt_sb;down_read (&upper_sb->s_umount); - ret =sync_filesystem (upper_sb); +wait_sb_inodes (upper_sb); +if (upper_sb->s_op->sync_fs) + ret = upper_sb->s_op->sync_fs (upper_sb, wait); +if (!ret) + ret =sync_blockdev (upper_sb->s_bdev);up_read (&upper_sb->s_umount);return ret;
Signature · 1 line
2.26.2
Miklos Szeredi
maintainerRe: [RFC PATCH v4 9/9] ovl: implement containerized syncfs for overlayfs
Quoted 34 lines · Chengguang Xu
Now overlayfs can only sync dirty inode during syncfs, so remove unnecessary
sync_filesystem()on upper file system.Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/super.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)fs/overlayfs/super.c +3
-2@@ -15,6 +15,8 @@#include <linux/seq_file.h> #include <linux/posix_acl_xattr.h> #include <linux/exportfs.h> +#include <linux/blkdev.h> +#include <linux/writeback.h> #include "overlayfs.h" MODULE_AUTHOR ("Miklos Szeredi <[email protected]>" );@@ -270,8 +272,7 @@ static int ovl_sync_fs(struct super_block *sb, int wait)* Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC). * All the super blocks will be iterated, including upper_sb. * -* If this is a syncfs(2) call, then we do need to call -* sync_filesystem() on upper_sb, but enough if we do it when being +* if this is a syncfs(2) call, it will be enough we do it when being * called with wait == 1. */ if (!wait)
fs/overlayfs/super.c static int ovl_sync_fs(struct super_block *sb, int wait) · line 288
Quoted 9 lines
fs/overlayfs/super.c +5
-1@@ -280,7 +281,11 @@ static int ovl_sync_fs(struct super_block *sb, int wait) upper_sb =ovl_upper_mnt (ofs)->mnt_sb;down_read (&upper_sb->s_umount); - ret =sync_filesystem (upper_sb); +wait_sb_inodes (upper_sb); +if (upper_sb->s_op->sync_fs) + ret = upper_sb->s_op->sync_fs (upper_sb, wait); +if (!ret) + ret =sync_blockdev (upper_sb->s_bdev);
Should this instead be __sync_blockdev(..., wait)?
Thanks,
Miklos
Re: [RFC PATCH v4 9/9] ovl: implement containerized syncfs for overlayfs
---- 在 星期五, 2021-04-09 21:51:26 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 37 lines · Chengguang Xu
Quoted 36 lines
Now overlayfs can only sync dirty inode during syncfs, so remove unnecessary
sync_filesystem()on upper file system.Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/super.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)fs/overlayfs/super.c +3
-2@@ -15,6 +15,8 @@#include <linux/seq_file.h> #include <linux/posix_acl_xattr.h> #include <linux/exportfs.h> +#include <linux/blkdev.h> +#include <linux/writeback.h> #include "overlayfs.h" MODULE_AUTHOR ("Miklos Szeredi <[email protected]>" );@@ -270,8 +272,7 @@ static int ovl_sync_fs(struct super_block *sb, int wait)* Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC). * All the super blocks will be iterated, including upper_sb. * -* If this is a syncfs(2) call, then we do need to call -* sync_filesystem() on upper_sb, but enough if we do it when being +* if this is a syncfs(2) call, it will be enough we do it when being * called with wait == 1. */ if (!wait)@@ -280,7 +281,11 @@ static int ovl_sync_fs(struct super_block *sb, int wait) upper_sb =ovl_upper_mnt (ofs)->mnt_sb;
Quoted 9 lines
fs/overlayfs/super.c static int ovl_sync_fs(struct super_block *sb, int wait) · line 288
Quoted 7 lines
fs/overlayfs/super.c +5
-1down_read (&upper_sb->s_umount); - ret =sync_filesystem (upper_sb); +wait_sb_inodes (upper_sb); +if (upper_sb->s_op->sync_fs) + ret = upper_sb->s_op->sync_fs (upper_sb, wait); +if (!ret) + ret =sync_blockdev (upper_sb->s_bdev);Should this instead be
__sync_blockdev(..., wait)?
I don't remember why we skipped the case of (wait == 0) here, just guess it's not worth to export internal function __sync_blockdev() to modules, do you prefer to call __sync_blockdev() and handle both nowait and wait cases?
Thanks,
Chengguang
Miklos Szeredi
maintainerRe: [RFC PATCH v4 9/9] ovl: implement containerized syncfs for overlayfs
Quoted 49 lines · Chengguang Xu
---- 在 星期五, 2021-04-09 21:51:26 Miklos Szeredi <[email protected]> 撰写 ----
Quoted 37 lines · Chengguang Xu
Quoted 36 lines
Now overlayfs can only sync dirty inode during syncfs, so remove unnecessary
sync_filesystem()on upper file system.Signed-off-by Chengguang Xu <[email protected]>--- fs/overlayfs/super.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)fs/overlayfs/super.c +3
-2@@ -15,6 +15,8 @@#include <linux/seq_file.h> #include <linux/posix_acl_xattr.h> #include <linux/exportfs.h> +#include <linux/blkdev.h> +#include <linux/writeback.h> #include "overlayfs.h" MODULE_AUTHOR ("Miklos Szeredi <[email protected]>" );@@ -270,8 +272,7 @@ static int ovl_sync_fs(struct super_block *sb, int wait)* Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC). * All the super blocks will be iterated, including upper_sb. * -* If this is a syncfs(2) call, then we do need to call -* sync_filesystem() on upper_sb, but enough if we do it when being +* if this is a syncfs(2) call, it will be enough we do it when being * called with wait == 1. */ if (!wait)@@ -280,7 +281,11 @@ static int ovl_sync_fs(struct super_block *sb, int wait) upper_sb =ovl_upper_mnt (ofs)->mnt_sb;Quoted 9 lines
fs/overlayfs/super.c static int ovl_sync_fs(struct super_block *sb, int wait) · line 288
Quoted 7 lines
fs/overlayfs/super.c +5
-1down_read (&upper_sb->s_umount); - ret =sync_filesystem (upper_sb); +wait_sb_inodes (upper_sb); +if (upper_sb->s_op->sync_fs) + ret = upper_sb->s_op->sync_fs (upper_sb, wait); +if (!ret) + ret =sync_blockdev (upper_sb->s_bdev);Should this instead be
__sync_blockdev(..., wait)?
Quoted 3 lines
I don't remember why we skipped the case of (wait == 0) here, just guess it's not worth to export internal function
__sync_blockdev()to modules, do you prefer to call__sync_blockdev()and handle both nowait and wait cases?
Possibly it would make most sense to export the "->sync_fs() + __sync_blockdev()" part of __sync_filesystem() as a separate helper.
Thanks,
Miklos
---- 在 星期五, 2020-11-13 14:55:46 Chengguang Xu <[email protected]> 撰写 ----
Quoted 10 lines
Current syncfs(2) syscall on overlayfs just calls
sync_filesystem()onupper_sbto synchronize whole dirty inodes in upper filesystem regardless of the overlay ownership of the inode. In the use case of container, when multiple containers using the same underlying upper filesystem, it has some shortcomings as below.(1) Performance
Synchronization is probably heavy because it actually syncs unnecessary inodes for target overlayfs.
Quoted 8 lines
(2) Interference
Unplanned synchronization will probably impact IO performance of unrelated container processes on the other overlayfs.This series try to implement containerized syncfs for overlayfs so that only sync target dirty upper inodes which are belong to specific overlayfs instance. By doing this, it is able to reduce cost of synchronization and will not seriously impact IO performance of unrelated processes.
Hi Miklos,
I think this version has addressed all previous issues and comments from Jack and Amir. Have you got time to review this patch series?
Thanks,
Chengguang
Miklos Szeredi
maintainerQuoted 21 lines · Chengguang Xu
---- 在 星期五, 2020-11-13 14:55:46 Chengguang Xu <[email protected]> 撰写 ----
Quoted 10 lines
Current syncfs(2) syscall on overlayfs just calls
sync_filesystem()onupper_sbto synchronize whole dirty inodes in upper filesystem regardless of the overlay ownership of the inode. In the use case of container, when multiple containers using the same underlying upper filesystem, it has some shortcomings as below.(1) Performance
Synchronization is probably heavy because it actually syncs unnecessary inodes for target overlayfs.Quoted 8 lines
(2) Interference
Unplanned synchronization will probably impact IO performance of unrelated container processes on the other overlayfs.This series try to implement containerized syncfs for overlayfs so that only sync target dirty upper inodes which are belong to specific overlayfs instance. By doing this, it is able to reduce cost of synchronization and will not seriously impact IO performance of unrelated processes.
Quoted 4 lines
Hi Miklos,
I think this version has addressed all previous issues and comments from Jack and Amir. Have you got time to review this patch series?
Hopefully yes.
I'm really keen to finish off the unprivileged overlay patches first. Will test and post a new version shortly.
Thanks,
Miklos
Quoted 20 lines · Chengguang Xu
---- 在 星期五, 2020-11-13 14:55:46 Chengguang Xu <[email protected]> 撰写 ----
Quoted 10 lines
Current syncfs(2) syscall on overlayfs just calls
sync_filesystem()onupper_sbto synchronize whole dirty inodes in upper filesystem regardless of the overlay ownership of the inode. In the use case of container, when multiple containers using the same underlying upper filesystem, it has some shortcomings as below.(1) Performance
Synchronization is probably heavy because it actually syncs unnecessary inodes for target overlayfs.Quoted 8 lines
(2) Interference
Unplanned synchronization will probably impact IO performance of unrelated container processes on the other overlayfs.This series try to implement containerized syncfs for overlayfs so that only sync target dirty upper inodes which are belong to specific overlayfs instance. By doing this, it is able to reduce cost of synchronization and will not seriously impact IO performance of unrelated processes.
Quoted 4 lines
Hi Miklos,
I think this version has addressed all previous issues and comments from Jack and Amir. Have you got time to review this patch series?
Yes, the patches now look good to me. Feel free to add:
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR