erofs: reuse superblock for file-backed mounts
Subsystems (3)
PATCH v2 1/1 erofs: reuse superblock for file-backed mounts
When the same file-backed image is mounted multiple times (via path or fd) with the superblock_share mount option and matching domain_id, reuse the existing superblock instead of creating a new one. This allows multiple mounts of the same image to share in-kernel data structures more efficiently.
Superblock sharing requires both superblock_share and domain_id to be specified, following the same pattern as inode_share. The backing file is identified by its inode and fsoffset. If mount options conflict, a separate superblock is created transparently as a fallback. Remount is not allowed on shared superblocks.
Tested by mounting a 177M Fedora EROFS image 20 times with full traversal:
#!/bin/sh
IMG=${1:-/root/fedora.erofs}
N=20
DIR=$(mktemp -d)
trap "umount $DIR/m* 2>/dev/null; rm -rf $DIR" EXIT
sync; echo 3 > /proc/sys/vm/drop_caches
INODES_BEFORE=$(grep erofs_inode /proc/slabinfo | awk '{print $2}')
MEM_BEFORE=$(grep ^Slab: /proc/meminfo | awk '{print $2}')
for i in $(seq 1 $N); do
mkdir $DIR/m$i
mount -t erofs -o domain_id=test,superblock_share "$IMG" $DIR/m$i
find $DIR/m$i > /dev/null
done
echo "Superblocks: $(grep $DIR /proc/self/mountinfo | \
awk '{print $3}' | sort -u | wc -l)"
echo "erofs_inode delta: +$(( $(grep erofs_inode /proc/slabinfo | \
awk '{print $2}') - INODES_BEFORE ))"
echo "Slab delta: +$(( $(grep ^Slab: /proc/meminfo | \
awk '{print $2}') - MEM_BEFORE )) kB"unpatched kernel:
Superblocks: 20
erofs_inode delta: +46368
Slab delta: +39952 kB
0.08user 0.82system 0:00.98elapsed
patched kernel:
Superblocks: 1
erofs_inode delta: +2044
Slab delta: +1088 kB
0.07user 0.21system 0:00.31elapsed
The time difference shows that sharing the superblock also benefits the page cache and inode cache, as subsequent mounts of the same image avoid re-reading the backing file. This is particularly useful for container hosts running multiple containers from the same base image.
---
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
Needs: https://lore.kernel.org/linux-fsdevel/[email protected]/
Documentation/filesystems/erofs.rst | 16 ++++++- fs/erofs/internal.h | 1 + fs/erofs/super.c | 72 +++++++++++++++++++++++++++-- 3 files changed, 83 insertions(+), 6 deletions(-)
Documentation/filesystems/erofs.rst +14 -2
@@ -131,12 +131,17 @@ device=%s Specify a path to an extra device to be used together. directio (For file-backed mounts) Use direct I/O to access backing files, and asynchronous I/O will be enabled if supported. domain_id=%s Specify a trusted domain ID. Filesystems sharing the same - domain ID can share page cache across mounts when inode - page sharing is enabled. (not shown in mountinfo output) + domain ID can share page cache across mounts when + inode_share is enabled, and share superblocks when + superblock_share is enabled. (not shown in mountinfo + output) fsoffset=%llu Specify block-aligned filesystem offset for the primary device. inode_share Enable inode page sharing for this filesystem. Inodes with identical content within the same domain ID can share the page cache. +superblock_share Share the superblock when the same backing file is mounted + more than once with the same domain_id and compatible + options. Remount is not allowed on shared superblocks. =================== ========================================================= File-backed mounts
@@ -154,6 +159,13 @@ Only regular files are accepted as backing files; to mount an image that resides on a block device, use the traditional block device mount path instead. +When domain_id and superblock_share are both specified and the same +backing file (identified by inode and fsoffset) is mounted more than +once with the same domain_id and compatible mount options, the kernel +reuses the existing superblock instead of creating a new one. + +Remount is not allowed on shared superblocks. + Sysfs Entries =============
fs/erofs/internal.h +1 -0
@@ -156,6 +156,7 @@ struct erofs_sb_info {#define EROFS_MOUNT_DAX_NEVER 0x00000080#define EROFS_MOUNT_DIRECT_IO 0x00000100#define EROFS_MOUNT_INODE_SHARE 0x00000200 +#define EROFS_MOUNT_SUPERBLOCK_SHARE 0x00000400#define clear_opt (opt, option) ((opt)->mount_opt &= ~EROFS_MOUNT_##option)#define set_opt (opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)
fs/erofs/super.c +68 -4
126 lines not on this page · raw message
Signature · 1 line
2.55.0
Gao Xiang
maintainerHi Giuseppe,
Quoted 2 lines · Giuseppe Scrivano
When the same file-backed image is mounted multiple times (via path or fd) with the
superblock_sharemount option and matchingdomain_id, reuse
First, I still need to get vfs maintainer opinions for "the sharing superblock feature for file-backed mounts"
I don't think it needs to be bothered with "domain_id" since "domain_id" represents a trusted zone so a security concept for data sourcing, and here I think you should just share the same filesystem (with the same backing files) and the same option, so I don't think "domain_id" is really useful.
Quoted 5 lines
the existing superblock instead of creating a new one. This allows multiple mounts of the same image to share in-kernel data structures more efficiently.
Superblock sharing requires both
superblock_shareanddomain_idto be
But I think a "superblock_share" yes/no mount option may be useful.
Quoted 71 lines
specified, following the same pattern as
inode_share. The backing file is identified by its inode and fsoffset. If mount options conflict, a separate superblock is created transparently as a fallback. Remount is not allowed on shared superblocks.Tested by mounting a 177M Fedora EROFS image 20 times with full traversal:
#!/bin/sh IMG=${1:-/root/fedora.erofs} N=20 DIR=$(mktemp -d) trap "umount $DIR/m* 2>/dev/null; rm -rf $DIR" EXIT sync; echo 3 > /proc/sys/vm/drop_caches INODES_BEFORE=$(grep erofs_inode /proc/slabinfo | awk '{print $2}') MEM_BEFORE=$(grep ^Slab: /proc/meminfo | awk '{print $2}') for i in $(seq 1 $N); do mkdir $DIR/m$i mount -t erofs -o domain_id=test,superblock_share "$IMG" $DIR/m$i find $DIR/m$i > /dev/null done echo "Superblocks: $(grep $DIR /proc/self/mountinfo | \ awk '{print $3}' | sort -u | wc -l)" echo "erofs_inode delta: +$(( $(grep erofs_inode /proc/slabinfo | \ awk '{print $2}') - INODES_BEFORE ))" echo "Slab delta: +$(( $(grep ^Slab: /proc/meminfo | \ awk '{print $2}') - MEM_BEFORE )) kB"unpatched kernel:
Superblocks: 20
erofs_inodedelta: +46368
Slab delta: +39952 kB
0.08user 0.82system 0:00.98elapsedpatched kernel:
Superblocks: 1
erofs_inodedelta: +2044
Slab delta: +1088 kB
0.07user 0.21system 0:00.31elapsedThe time difference shows that sharing the superblock also benefits the page cache and inode cache, as subsequent mounts of the same image avoid re-reading the backing file. This is particularly useful for container hosts running multiple containers from the same base image.
Signed-off-by Giuseppe Scrivano <[email protected]>---
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
Needs: https://lore.kernel.org/linux-fsdevel/[email protected]/Documentation/filesystems/erofs.rst | 16 ++++++- fs/erofs/internal.h | 1 + fs/erofs/super.c | 72 +++++++++++++++++++++++++++-- 3 files changed, 83 insertions(+), 6 deletions(-)Documentation/filesystems/erofs.rst +1
-2@@ -131,12 +131,17 @@ device=%s Specify a path to an extra device to be used together. directio (For file-backed mounts) Use direct I/O to access backing files, and asynchronous I/O will be enabled if supported. domain_id=%s Specify a trusted domain ID. Filesystems sharing the same - domain ID can share page cache across mounts when inode - page sharing is enabled. (not shown in mountinfo output) + domain ID can share page cache across mounts when
Documentation/filesystems/erofs.rst device=%s Specify a path to an extra device to be used together. · line 144
Quoted 10 lines
Documentation/filesystems/erofs.rst +6
-0+ inode_share is enabled, and share superblocks when + superblock_share is enabled. (not shown in mountinfo + output) fsoffset=%llu Specify block-aligned filesystem offset for the primary device. inode_share Enable inode page sharing for this filesystem. Inodes with identical content within the same domain ID can share the page cache. +superblock_share Share the superblock when the same backing file is mounted + more than once with the same domain_id and compatible + options. Remount is not allowed on shared superblocks.
As I commented above.
Quoted 90 lines
Documentation/filesystems/erofs.rst +7
-0=================== ========================================================= File-backed mounts@@ -154,6 +159,13 @@ Only regular files are accepted as backing files; to mount an image that resides on a block device, use the traditional block device mount path instead. +When domain_id and superblock_share are both specified and the same +backing file (identified by inode and fsoffset) is mounted more than +once with the same domain_id and compatible mount options, the kernel +reuses the existing superblock instead of creating a new one. + +Remount is not allowed on shared superblocks. + Sysfs Entries =============fs/erofs/internal.h +1
-0@@ -156,6 +156,7 @@ struct erofs_sb_info {#define EROFS_MOUNT_DAX_NEVER 0x00000080#define EROFS_MOUNT_DIRECT_IO 0x00000100#define EROFS_MOUNT_INODE_SHARE 0x00000200 +#define EROFS_MOUNT_SUPERBLOCK_SHARE 0x00000400#define clear_opt (opt, option) ((opt)->mount_opt &= ~EROFS_MOUNT_##option)#define set_opt (opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)fs/erofs/super.c +16
-2@@ -386,7 +386,7 @@ static void erofs_default_options(struct erofs_sb_info *sbi)enum { Opt_user_xattr, Opt_acl, Opt_cache_strategy, Opt_dax, Opt_dax_enum, Opt_device, Opt_domain_id, Opt_directio, Opt_fsoffset, Opt_inode_share, - Opt_source, + Opt_source, Opt_superblock_share, };static const struct constant_table erofs_param_cache_strategy[] = {@@ -415,6 +415,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = {fsparam_u64 ("fsoffset" , Opt_fsoffset),fsparam_flag ("inode_share" , Opt_inode_share),fsparam_file_or_string ("source" , Opt_source), + fsparam_flag("superblock_share", Opt_superblock_share), {} };@@ -536,7 +537,8 @@ static int erofs_fc_parse_param(struct fs_context *fc, ++sbi->devs->extra_devices;break ;case Opt_domain_id: -if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE )) { +if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE ) && + !IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) {errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); }else {kfree_sensitive (sbi->domain_id);@@ -560,6 +562,12 @@ static int erofs_fc_parse_param(struct fs_context *fc,else set_opt (&sbi->opt,INODE_SHARE );break ; +case Opt_superblock_share: +if (!IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) +errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); +else +set_opt (&sbi->opt,SUPERBLOCK_SHARE ); +break ;case Opt_source:return erofs_fc_parse_source (fc, param); }@@ -659,6 +667,10 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)errorfc (fc,"domain_id is needed when inode_ishare is on" );return -EINVAL ; } +if (!sbi->domain_id &&test_opt (&sbi->opt,SUPERBLOCK_SHARE )) { +errorfc (fc,"domain_id is needed when superblock_share is on" ); +return -EINVAL ; + }if (test_opt (&sbi->opt,DAX_ALWAYS ) &&test_opt (&sbi->opt,INODE_SHARE )) {errorfc (fc,"FSDAX is not allowed when inode_ishare is on" );return -EINVAL ;@@ -788,6 +800,55 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)return 0 ; } +static int erofs_fc_test_file_super (struct super_block *sb, +struct fs_context *fc)
fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 814
Quoted 10 lines
fs/erofs/super.c +10
-0+{ +struct erofs_sb_info *sbi =EROFS_SB (sb); +struct erofs_sb_info *new_sbi = fc->s_fs_info; + +if (sb->s_iflags &SB_I_RETIRED ) +return 0 ; +if (!sbi->dif0.file || !new_sbi->dif0.file) +return 0 ; +if (!test_opt (&new_sbi->opt,SUPERBLOCK_SHARE )) +return 0 ;
I think it'd be better to just memcmp(sbi->opt, new_sbi->opt, ..) directly.
fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 817
Quoted 3 lines
fs/erofs/super.c +3
-0+if (!sbi->domain_id || !new_sbi->domain_id || +strcmp (sbi->domain_id, new_sbi->domain_id)) +return 0 ;
And also you need to ensure the devs has the same backing files too.
Quoted 6 lines
fs/erofs/super.c +6
-0+return file_inode (sbi->dif0.file) ==file_inode (new_sbi->dif0.file) && + sbi->dif0.fsoff == new_sbi->dif0.fsoff && + sbi->opt.mount_opt == new_sbi->opt.mount_opt && + sbi->opt.cache_strategy == new_sbi->opt.cache_strategy; +} +
fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 833
Quoted 10 lines
fs/erofs/super.c +10
-0+static int erofs_fc_get_tree_file (struct fs_context *fc) +{ +struct erofs_sb_info *sbi = fc->s_fs_info; +struct super_block *sb; +int err; + +if (!test_opt (&sbi->opt,SUPERBLOCK_SHARE )) +return get_tree_nodev (fc, erofs_fc_fill_super); + + sb =sget_fc (fc, erofs_fc_test_file_super, set_anon_super_fc);
Especially the usage above, I need acks from vfs maintainers first.
Thanks,
Gao Xiang
Hi Gao,
thanks for the review!
Quoted 6 lines · Gao Xiang
Hi Giuseppe,
Quoted 2 lines · Giuseppe Scrivano
When the same file-backed image is mounted multiple times (via path or fd) with the
superblock_sharemount option and matchingdomain_id, reuse
Quoted 8 lines
First, I still need to get vfs maintainer opinions for "the sharing superblock feature for file-backed mounts"
I don't think it needs to be bothered with "
domain_id" since "domain_id" represents a trusted zone so a security concept for data sourcing, and here I think you should just share the same filesystem (with the same backing files) and the same option, so I don't think "domain_id" is really useful.
I wanted to be conservative and let it behave as inode_share, but I agree it is not needed, I'll drop this.
Quoted 196 lines
Quoted 5 lines
the existing superblock instead of creating a new one. This allows multiple mounts of the same image to share in-kernel data structures more efficiently.
Superblock sharing requires both
superblock_shareanddomain_idto beBut I think a "
superblock_share" yes/no mount option may be useful.Quoted 71 lines
specified, following the same pattern as
inode_share. The backing file is identified by its inode and fsoffset. If mount options conflict, a separate superblock is created transparently as a fallback. Remount is not allowed on shared superblocks.Tested by mounting a 177M Fedora EROFS image 20 times with full traversal:
#!/bin/sh IMG=${1:-/root/fedora.erofs} N=20 DIR=$(mktemp -d) trap "umount $DIR/m* 2>/dev/null; rm -rf $DIR" EXIT sync; echo 3 > /proc/sys/vm/drop_caches INODES_BEFORE=$(grep erofs_inode /proc/slabinfo | awk '{print $2}') MEM_BEFORE=$(grep ^Slab: /proc/meminfo | awk '{print $2}') for i in $(seq 1 $N); do mkdir $DIR/m$i mount -t erofs -o domain_id=test,superblock_share "$IMG" $DIR/m$i find $DIR/m$i > /dev/null done echo "Superblocks: $(grep $DIR /proc/self/mountinfo | \ awk '{print $3}' | sort -u | wc -l)" echo "erofs_inode delta: +$(( $(grep erofs_inode /proc/slabinfo | \ awk '{print $2}') - INODES_BEFORE ))" echo "Slab delta: +$(( $(grep ^Slab: /proc/meminfo | \ awk '{print $2}') - MEM_BEFORE )) kB"unpatched kernel:
Superblocks: 20
erofs_inodedelta: +46368
Slab delta: +39952 kB
0.08user 0.82system 0:00.98elapsedpatched kernel:
Superblocks: 1
erofs_inodedelta: +2044
Slab delta: +1088 kB
0.07user 0.21system 0:00.31elapsedThe time difference shows that sharing the superblock also benefits the page cache and inode cache, as subsequent mounts of the same image avoid re-reading the backing file. This is particularly useful for container hosts running multiple containers from the same base image.
Signed-off-by Giuseppe Scrivano <[email protected]>---
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
Needs: https://lore.kernel.org/linux-fsdevel/[email protected]/Documentation/filesystems/erofs.rst | 16 ++++++- fs/erofs/internal.h | 1 + fs/erofs/super.c | 72 +++++++++++++++++++++++++++-- 3 files changed, 83 insertions(+), 6 deletions(-)Documentation/filesystems/erofs.rst +1
-2@@ -131,12 +131,17 @@ device=%s Specify a path to an extra device to be used together. directio (For file-backed mounts) Use direct I/O to access backing files, and asynchronous I/O will be enabled if supported. domain_id=%s Specify a trusted domain ID. Filesystems sharing the same - domain ID can share page cache across mounts when inode - page sharing is enabled. (not shown in mountinfo output) + domain ID can share page cache across mounts whenDocumentation/filesystems/erofs.rst device=%s Specify a path to an extra device to be used together. · line 144
Quoted 10 lines
Documentation/filesystems/erofs.rst +6
-0+ inode_share is enabled, and share superblocks when + superblock_share is enabled. (not shown in mountinfo + output) fsoffset=%llu Specify block-aligned filesystem offset for the primary device. inode_share Enable inode page sharing for this filesystem. Inodes with identical content within the same domain ID can share the page cache. +superblock_share Share the superblock when the same backing file is mounted + more than once with the same domain_id and compatible + options. Remount is not allowed on shared superblocks.As I commented above.
Quoted 90 lines
Documentation/filesystems/erofs.rst +7
-0=================== ========================================================= File-backed mounts@@ -154,6 +159,13 @@ Only regular files are accepted as backing files; to mount an image that resides on a block device, use the traditional block device mount path instead. +When domain_id and superblock_share are both specified and the same +backing file (identified by inode and fsoffset) is mounted more than +once with the same domain_id and compatible mount options, the kernel +reuses the existing superblock instead of creating a new one. + +Remount is not allowed on shared superblocks. + Sysfs Entries =============fs/erofs/internal.h +1
-0@@ -156,6 +156,7 @@ struct erofs_sb_info {#define EROFS_MOUNT_DAX_NEVER 0x00000080#define EROFS_MOUNT_DIRECT_IO 0x00000100#define EROFS_MOUNT_INODE_SHARE 0x00000200 +#define EROFS_MOUNT_SUPERBLOCK_SHARE 0x00000400#define clear_opt (opt, option) ((opt)->mount_opt &= ~EROFS_MOUNT_##option)#define set_opt (opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)fs/erofs/super.c +16
-2@@ -386,7 +386,7 @@ static void erofs_default_options(struct erofs_sb_info *sbi)enum { Opt_user_xattr, Opt_acl, Opt_cache_strategy, Opt_dax, Opt_dax_enum, Opt_device, Opt_domain_id, Opt_directio, Opt_fsoffset, Opt_inode_share, - Opt_source, + Opt_source, Opt_superblock_share, };static const struct constant_table erofs_param_cache_strategy[] = {@@ -415,6 +415,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = {fsparam_u64 ("fsoffset" , Opt_fsoffset),fsparam_flag ("inode_share" , Opt_inode_share),fsparam_file_or_string ("source" , Opt_source), + fsparam_flag("superblock_share", Opt_superblock_share), {} };@@ -536,7 +537,8 @@ static int erofs_fc_parse_param(struct fs_context *fc, ++sbi->devs->extra_devices;break ;case Opt_domain_id: -if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE )) { +if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE ) && + !IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) {errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); }else {kfree_sensitive (sbi->domain_id);@@ -560,6 +562,12 @@ static int erofs_fc_parse_param(struct fs_context *fc,else set_opt (&sbi->opt,INODE_SHARE );break ; +case Opt_superblock_share: +if (!IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) +errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); +else +set_opt (&sbi->opt,SUPERBLOCK_SHARE ); +break ;case Opt_source:return erofs_fc_parse_source (fc, param); }@@ -659,6 +667,10 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)errorfc (fc,"domain_id is needed when inode_ishare is on" );return -EINVAL ; } +if (!sbi->domain_id &&test_opt (&sbi->opt,SUPERBLOCK_SHARE )) { +errorfc (fc,"domain_id is needed when superblock_share is on" ); +return -EINVAL ; + }if (test_opt (&sbi->opt,DAX_ALWAYS ) &&test_opt (&sbi->opt,INODE_SHARE )) {errorfc (fc,"FSDAX is not allowed when inode_ishare is on" );return -EINVAL ;@@ -788,6 +800,55 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)return 0 ; } +static int erofs_fc_test_file_super (struct super_block *sb, +struct fs_context *fc)fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 814
Quoted 10 lines
fs/erofs/super.c +10
-0+{ +struct erofs_sb_info *sbi =EROFS_SB (sb); +struct erofs_sb_info *new_sbi = fc->s_fs_info; + +if (sb->s_iflags &SB_I_RETIRED ) +return 0 ; +if (!sbi->dif0.file || !new_sbi->dif0.file) +return 0 ; +if (!test_opt (&new_sbi->opt,SUPERBLOCK_SHARE )) +return 0 ;I think it'd be better to just
memcmp(sbi->opt, new_sbi->opt, ..)directly.
Quoted 5 lines
fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 817
Quoted 3 lines
fs/erofs/super.c +3
-0+if (!sbi->domain_id || !new_sbi->domain_id || +strcmp (sbi->domain_id, new_sbi->domain_id)) +return 0 ;And also you need to ensure the devs has the same backing files too.
are you fine if we just skip mounts with extra_devs? It can be relaxed in future if needed. Otherwise we would need to open them before we do these checks to be sure we are referring to the same files.
Thanks,
Giuseppe
Quoted 21 lines
Quoted 6 lines
fs/erofs/super.c +6
-0+return file_inode (sbi->dif0.file) ==file_inode (new_sbi->dif0.file) && + sbi->dif0.fsoff == new_sbi->dif0.fsoff && + sbi->opt.mount_opt == new_sbi->opt.mount_opt && + sbi->opt.cache_strategy == new_sbi->opt.cache_strategy; +} +fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 833
Quoted 10 lines
fs/erofs/super.c +10
-0+static int erofs_fc_get_tree_file (struct fs_context *fc) +{ +struct erofs_sb_info *sbi = fc->s_fs_info; +struct super_block *sb; +int err; + +if (!test_opt (&sbi->opt,SUPERBLOCK_SHARE )) +return get_tree_nodev (fc, erofs_fc_fill_super); + + sb =sget_fc (fc, erofs_fc_test_file_super, set_anon_super_fc);Especially the usage above, I need acks from vfs maintainers first.
Thanks,
Gao Xiang
Gao Xiang
maintainerQuoted 15 lines · Giuseppe Scrivano
Hi Gao,
thanks for the review!
Quoted 9 lines · Gao Xiang
Hi Giuseppe,
Quoted 2 lines · Giuseppe Scrivano
When the same file-backed image is mounted multiple times (via path or fd) with the
superblock_sharemount option and matchingdomain_id, reuseFirst, I still need to get vfs maintainer opinions for "the sharing superblock feature for file-backed mounts"
Quoted 8 lines
Quoted 5 lines
I don't think it needs to be bothered with "
domain_id" since "domain_id" represents a trusted zone so a security concept for data sourcing, and here I think you should just share the same filesystem (with the same backing files) and the same option, so I don't think "domain_id" is really useful.I wanted to be conservative and let it behave as
inode_share, but I agree it is not needed, I'll drop this.
It doesn't need to be aligned with "inode_share", but you remind me a bit, let's exclude this feature with "inode_share" for now (since composefs doesn't need it) and let's handle more coupled features later.
Quoted 203 lines
Quoted 197 lines
Quoted 6 lines
the existing superblock instead of creating a new one. This allows multiple mounts of the same image to share in-kernel data structures more efficiently.
Superblock sharing requires both
superblock_shareanddomain_idto beBut I think a "
superblock_share" yes/no mount option may be useful.Quoted 71 lines
specified, following the same pattern as
inode_share. The backing file is identified by its inode and fsoffset. If mount options conflict, a separate superblock is created transparently as a fallback. Remount is not allowed on shared superblocks.Tested by mounting a 177M Fedora EROFS image 20 times with full traversal:
#!/bin/sh IMG=${1:-/root/fedora.erofs} N=20 DIR=$(mktemp -d) trap "umount $DIR/m* 2>/dev/null; rm -rf $DIR" EXIT sync; echo 3 > /proc/sys/vm/drop_caches INODES_BEFORE=$(grep erofs_inode /proc/slabinfo | awk '{print $2}') MEM_BEFORE=$(grep ^Slab: /proc/meminfo | awk '{print $2}') for i in $(seq 1 $N); do mkdir $DIR/m$i mount -t erofs -o domain_id=test,superblock_share "$IMG" $DIR/m$i find $DIR/m$i > /dev/null done echo "Superblocks: $(grep $DIR /proc/self/mountinfo | \ awk '{print $3}' | sort -u | wc -l)" echo "erofs_inode delta: +$(( $(grep erofs_inode /proc/slabinfo | \ awk '{print $2}') - INODES_BEFORE ))" echo "Slab delta: +$(( $(grep ^Slab: /proc/meminfo | \ awk '{print $2}') - MEM_BEFORE )) kB"unpatched kernel:
Superblocks: 20
erofs_inodedelta: +46368
Slab delta: +39952 kB
0.08user 0.82system 0:00.98elapsedpatched kernel:
Superblocks: 1
erofs_inodedelta: +2044
Slab delta: +1088 kB
0.07user 0.21system 0:00.31elapsedThe time difference shows that sharing the superblock also benefits the page cache and inode cache, as subsequent mounts of the same image avoid re-reading the backing file. This is particularly useful for container hosts running multiple containers from the same base image.
Signed-off-by Giuseppe Scrivano <[email protected]>---
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
Needs: https://lore.kernel.org/linux-fsdevel/[email protected]/Documentation/filesystems/erofs.rst | 16 ++++++- fs/erofs/internal.h | 1 + fs/erofs/super.c | 72 +++++++++++++++++++++++++++-- 3 files changed, 83 insertions(+), 6 deletions(-)Documentation/filesystems/erofs.rst +1
-2@@ -131,12 +131,17 @@ device=%s Specify a path to an extra device to be used together. directio (For file-backed mounts) Use direct I/O to access backing files, and asynchronous I/O will be enabled if supported. domain_id=%s Specify a trusted domain ID. Filesystems sharing the same - domain ID can share page cache across mounts when inode - page sharing is enabled. (not shown in mountinfo output) + domain ID can share page cache across mounts whenDocumentation/filesystems/erofs.rst device=%s Specify a path to an extra device to be used together. · line 144
Quoted 10 lines
Documentation/filesystems/erofs.rst +6
-0+ inode_share is enabled, and share superblocks when + superblock_share is enabled. (not shown in mountinfo + output) fsoffset=%llu Specify block-aligned filesystem offset for the primary device. inode_share Enable inode page sharing for this filesystem. Inodes with identical content within the same domain ID can share the page cache. +superblock_share Share the superblock when the same backing file is mounted + more than once with the same domain_id and compatible + options. Remount is not allowed on shared superblocks.As I commented above.
Quoted 90 lines
Documentation/filesystems/erofs.rst +7
-0=================== ========================================================= File-backed mounts@@ -154,6 +159,13 @@ Only regular files are accepted as backing files; to mount an image that resides on a block device, use the traditional block device mount path instead. +When domain_id and superblock_share are both specified and the same +backing file (identified by inode and fsoffset) is mounted more than +once with the same domain_id and compatible mount options, the kernel +reuses the existing superblock instead of creating a new one. + +Remount is not allowed on shared superblocks. + Sysfs Entries =============fs/erofs/internal.h +1
-0@@ -156,6 +156,7 @@ struct erofs_sb_info {#define EROFS_MOUNT_DAX_NEVER 0x00000080#define EROFS_MOUNT_DIRECT_IO 0x00000100#define EROFS_MOUNT_INODE_SHARE 0x00000200 +#define EROFS_MOUNT_SUPERBLOCK_SHARE 0x00000400#define clear_opt (opt, option) ((opt)->mount_opt &= ~EROFS_MOUNT_##option)#define set_opt (opt, option) ((opt)->mount_opt |= EROFS_MOUNT_##option)fs/erofs/super.c +16
-2@@ -386,7 +386,7 @@ static void erofs_default_options(struct erofs_sb_info *sbi)enum { Opt_user_xattr, Opt_acl, Opt_cache_strategy, Opt_dax, Opt_dax_enum, Opt_device, Opt_domain_id, Opt_directio, Opt_fsoffset, Opt_inode_share, - Opt_source, + Opt_source, Opt_superblock_share, };static const struct constant_table erofs_param_cache_strategy[] = {@@ -415,6 +415,7 @@ static const struct fs_parameter_spec erofs_fs_parameters[] = {fsparam_u64 ("fsoffset" , Opt_fsoffset),fsparam_flag ("inode_share" , Opt_inode_share),fsparam_file_or_string ("source" , Opt_source), + fsparam_flag("superblock_share", Opt_superblock_share), {} };@@ -536,7 +537,8 @@ static int erofs_fc_parse_param(struct fs_context *fc, ++sbi->devs->extra_devices;break ;case Opt_domain_id: -if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE )) { +if (!IS_ENABLED (CONFIG_EROFS_FS_PAGE_CACHE_SHARE ) && + !IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) {errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); }else {kfree_sensitive (sbi->domain_id);@@ -560,6 +562,12 @@ static int erofs_fc_parse_param(struct fs_context *fc,else set_opt (&sbi->opt,INODE_SHARE );break ; +case Opt_superblock_share: +if (!IS_ENABLED (CONFIG_EROFS_FS_BACKED_BY_FILE )) +errorfc (fc,"%s option not supported" , erofs_fs_parameters[opt].name); +else +set_opt (&sbi->opt,SUPERBLOCK_SHARE ); +break ;case Opt_source:return erofs_fc_parse_source (fc, param); }@@ -659,6 +667,10 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)errorfc (fc,"domain_id is needed when inode_ishare is on" );return -EINVAL ; } +if (!sbi->domain_id &&test_opt (&sbi->opt,SUPERBLOCK_SHARE )) { +errorfc (fc,"domain_id is needed when superblock_share is on" ); +return -EINVAL ; + }if (test_opt (&sbi->opt,DAX_ALWAYS ) &&test_opt (&sbi->opt,INODE_SHARE )) {errorfc (fc,"FSDAX is not allowed when inode_ishare is on" );return -EINVAL ;@@ -788,6 +800,55 @@ static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc)return 0 ; } +static int erofs_fc_test_file_super (struct super_block *sb, +struct fs_context *fc)fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 814
Quoted 10 lines
fs/erofs/super.c +10
-0+{ +struct erofs_sb_info *sbi =EROFS_SB (sb); +struct erofs_sb_info *new_sbi = fc->s_fs_info; + +if (sb->s_iflags &SB_I_RETIRED ) +return 0 ; +if (!sbi->dif0.file || !new_sbi->dif0.file) +return 0 ; +if (!test_opt (&new_sbi->opt,SUPERBLOCK_SHARE )) +return 0 ;I think it'd be better to just
memcmp(sbi->opt, new_sbi->opt, ..)directly.Quoted 5 lines
fs/erofs/super.c static int erofs_fc_fill_super(struct super_block *sb, struct fs_context *fc) · line 817
Quoted 3 lines
fs/erofs/super.c +3
-0+if (!sbi->domain_id || !new_sbi->domain_id || +strcmp (sbi->domain_id, new_sbi->domain_id)) +return 0 ;And also you need to ensure the devs has the same backing files too.
Quoted 4 lines
are you fine if we just skip mounts with
extra_devs? It can be relaxed in future if needed. Otherwise we would need to open them before we do these checks to be sure we are referring to the same files.
Sorry I totally missed that, that is fine, but if there is a dedicated "superblock_inode" feature, I hope it could be errored out other than silent fallback.
Also it'd be better to have a code comment that it doesn't cover mounts with extra_devs so that we could handle them in the follow-ups.
Thanks,
Gao Xiang
Quoted 3 lines
Thanks,
Giuseppe