Report overlayfs file ids with fanotify
Subsystems (2)
Amir Goldstein
maintainerv2 cover Report overlayfs file ids with fanotify
Miklos,
This is the second part of the work to support fanotify reporting of events with file ids on overlayfs. The fanotify_event_info_fid struct reported with fanotify events has an object file handle and an fsid, so fanotify requires that filesystems can encode file handles and have a non-zero f_fsid.
The first part [1] that was merged to v6.5-rc1, relaxed the fanotify requirements for filesystems to support reporting events with fid to require only the ->encode_fh() export operation.
Patch 1 changes overlayfs export_operations to meet the new requirements with the default overlay configurations (i.e. no need for nfs_export=on), thus, allowing an fanotify watch with FAN_REPORT_FID on overlayfs. There are LTS tests [2] for fanotify(FAN_REPORT_FID) + overlayfs.
Patches 2-4 are not strcitly needed to support reporting fanotify events with fid, because overlayfs already reports a non-zero f_fsid, it's just not a unique fsid. So before allowing to report events with overlayfs fids, I wanted to fix overlayfs fsid to be more unique.
I wanted to implement a persistent and unique fsid for overlayfs. I wanted it to be the default behavior, but needed to avoid breaking applications that rely on an existing overlayfs fsid to persist. I came up with a solution that is described in patch 4 (uuid=auto) that meets all the requirement above.
There is an xfstest to test the persistent-unique fsid feature [3].
This patch set has been in overlayfs-next for soaking since v6.5-rc1. If you have any reservations that we will not be able to resolve in time for 6.6, especially regarding the on-disk format and backward compat, we could also merge only patch 1 and leave the fsid patches for later.
Thanks,
Amir.
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://github.com/amir73il/ltp/commits/ovl_encode_fid
[3] https://github.com/amir73il/xfstests/commits/ovl_fsid
Amir Goldstein (4):
ovl: support encoding non-decodable file handles
ovl: add support for unique fsid per instance
ovl: store persistent uuid/fsid with uuid=on
ovl: auto generate uuid for new overlay filesystems
Documentation/filesystems/overlayfs.rst | 25 ++++++++++ fs/overlayfs/copy_up.c | 2 +- fs/overlayfs/export.c | 26 ++++++++--- fs/overlayfs/inode.c | 2 +- fs/overlayfs/namei.c | 5 +- fs/overlayfs/overlayfs.h | 22 +++++++++ fs/overlayfs/ovl_entry.h | 3 +- fs/overlayfs/params.c | 31 +++++++++++-- fs/overlayfs/super.c | 29 ++++++++++-- fs/overlayfs/util.c | 61 +++++++++++++++++++++++++ 10 files changed, 186 insertions(+), 20 deletions(-)
Signature · 1 line
2.34.1
Amir Goldstein
maintainerPATCH v2 1/4 ovl: support encoding non-decodable file handles
When all layers support file handles, we support encoding non-decodable file handles (a.k.a. fid) even with nfs_export=off.
When file handles do not need to be decoded, we do not need to copy up redirected lower directories on encode, and we encode also non-indexed upper with lower file handle, so fid will not change on copy up.
This enables reporting fanotify events with file handles on overlayfs with default config/mount options.
--- fs/overlayfs/export.c | 26 ++++++++++++++++++++------ fs/overlayfs/inode.c | 2 +- fs/overlayfs/overlayfs.h | 1 + fs/overlayfs/ovl_entry.h | 1 + fs/overlayfs/super.c | 9 +++++++++ 5 files changed, 32 insertions(+), 7 deletions(-)
fs/overlayfs/export.c +20 -6
@@ -174,28 +174,37 @@ static int ovl_connect_layer(struct dentry *dentry)* U = upper file handle * L = lower file handle * -* (*) Connecting an overlay dir from real lower dentry is not always +* (*) Decoding a connected overlay dir from real lower dentry is not always * possible when there are redirects in lower layers and non-indexed merge dirs. * To mitigate those case, we may copy up the lower dir ancestor before encode -* a lower dir file handle. +* of a decodable file handle for non-upper dir. * * Return 0 for upper file handle, > 0 for lower file handle or < 0 on error. */ static int ovl_check_encode_origin (struct dentry *dentry) {struct ovl_fs *ofs = dentry->d_sb->s_fs_info; +bool decodable = ofs->config.nfs_export; + +/* Lower file handle for non-upper non-decodable */ +if (!ovl_dentry_upper (dentry) && !decodable) +return 0 ;/* Upper file handle for pure upper */ if (!ovl_dentry_lower (dentry))return 0 ;/* -* Upper file handle for non-indexed upper. -* * Root is never indexed, so if there's an upper layer, encode upper for * root. */ -if (ovl_dentry_upper (dentry) && +if (dentry == dentry->d_sb->s_root) +return 0 ; + +/* +* Upper decodable file handle for non-indexed upper. +*/ +if (ovl_dentry_upper (dentry) && decodable && !ovl_test_flag (OVL_INDEX ,d_inode (dentry)))return 0 ;
@@ -205,7 +214,7 @@ static int ovl_check_encode_origin(struct dentry *dentry)* ovl_connect_layer() will try to make origin's layer "connected" by * copying up a "connectable" ancestor. */ -if (d_is_dir (dentry) &&ovl_upper_mnt (ofs)) +if (d_is_dir (dentry) &&ovl_upper_mnt (ofs) && decodable)return ovl_connect_layer (dentry);/* Lower file handle for indexed and non-upper dir/non-dir */
@@ -876,3 +885,8 @@ const struct export_operations ovl_export_operations = { .get_name = ovl_get_name, .get_parent = ovl_get_parent, }; + +/* encode_fh() encodes non-decodable file handles with nfs_export=off */ +const struct export_operations ovl_export_fid_operations = { + .encode_fh = ovl_encode_fh, +};
fs/overlayfs/inode.c +1 -1
@@ -1311,7 +1311,7 @@ static bool ovl_hash_bylower(struct super_block *sb, struct dentry *upper,return false;/* No, if non-indexed upper with NFS export */ -if (sb->s_export_op && upper) +if (ofs->config.nfs_export && upper)return false;/* Otherwise, hash by lower inode for fsnotify */
fs/overlayfs/overlayfs.h +1 -0
@@ -799,6 +799,7 @@ int ovl_set_origin(struct ovl_fs *ofs, struct dentry *lower,/* export.c */ extern const struct export_operations ovl_export_operations; +extern const struct export_operations ovl_export_fid_operations;/* super.c */ int ovl_fill_super (struct super_block *sb,struct fs_context *fc);
fs/overlayfs/ovl_entry.h +1 -0
@@ -82,6 +82,7 @@ struct ovl_fs {const struct cred *creator_cred;bool tmpfile;bool noxattr; +bool nofh;/* Did we take the inuse lock? */ bool upperdir_locked;bool workdir_locked;
fs/overlayfs/super.c +9 -0
@@ -400,6 +400,7 @@ static int ovl_lower_dir(const char *name, struct path *path,pr_warn ("fs on '%s' does not support file handles, falling back to index=off,nfs_export=off.\n" , name); } + ofs->nofh |= !fh_type;/* * Decoding origin file handle is required for persistent st_ino. * Without persistent st_ino, xino=auto falls back to xino=off.
@@ -818,6 +819,7 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs, ofs->config.index = false;pr_warn ("upper fs does not support file handles, falling back to index=off.\n" ); } + ofs->nofh |= !fh_type;/* Check if upper fs has 32bit inode numbers */ if (fh_type !=FILEID_INO32_GEN )
@@ -1452,8 +1454,15 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc) ofs->config.nfs_export = false; } +/* +* Support encoding decodable file handles with nfs_export=on +* and encoding non-decodable file handles with nfs_export=off +* if all layers support file handles. +*/ if (ofs->config.nfs_export) sb->s_export_op = &ovl_export_operations; +else if (!ofs->nofh) + sb->s_export_op = &ovl_export_fid_operations;/* Never override disk quota limits or use reserved space */ cap_lower (cred->cap_effective,CAP_SYS_RESOURCE );
Signature · 1 line
2.34.1
Amir Goldstein
maintainerPATCH v2 2/4 ovl: add support for unique fsid per instance
In reply to Amir GoldsteinThe legacy behavior of ovl_statfs() reports the f_fsid filled by underlying upper fs. This fsid is not unique among overlayfs instances on the same upper fs.
With mount option uuid=on, generate a non-persistent uuid per overlayfs instance and use it as the seed for f_fsid, similar to tmpfs.
This is useful for reporting fanotify events with fid info from different instances of overlayfs over the same upper fs.
The old behavior of null uuid and upper fs fsid is retained with the mount option uuid=null, which is the default.
The mount option uuid=off that disables uuid checks in underlying layers also retains the legacy behavior.
--- Documentation/filesystems/overlayfs.rst | 16 ++++++++++++++++ fs/overlayfs/copy_up.c | 2 +- fs/overlayfs/namei.c | 5 +++-- fs/overlayfs/overlayfs.h | 16 ++++++++++++++++ fs/overlayfs/ovl_entry.h | 2 +- fs/overlayfs/params.c | 25 +++++++++++++++++++++---- fs/overlayfs/super.c | 16 +++++++++++----- 7 files changed, 69 insertions(+), 13 deletions(-)
Documentation/filesystems/overlayfs.rst +16 -0
@@ -657,6 +657,22 @@ can be useful in case the underlying disk is copied and the UUID of this copy is changed. This is only applicable if all lower/upper/work directories are on the same filesystem, otherwise it will fallback to normal behaviour. + +UUID and fsid +------------- + +The UUID of overlayfs instance itself and the fsid reported by statfs(2) are +controlled by the "uuid" mount option, which supports these values: + +- "null": (default) + UUID of overlayfs is null. fsid is taken from upper most filesystem. +- "off": + UUID of overlayfs is null. fsid is taken from upper most filesystem. + UUID of underlying layers is ignored. +- "on": + UUID of overlayfs is generated and used to report a unique fsid. + + Volatile mount --------------
fs/overlayfs/copy_up.c +1 -1
@@ -416,7 +416,7 @@ struct ovl_fh *ovl_encode_real_fh(struct ovl_fs *ofs, struct dentry *real,if (is_upper) fh->fb.flags |=OVL_FH_FLAG_PATH_UPPER ; fh->fb.len =sizeof (fh->fb) + buflen; -if (ofs->config.uuid) +if (ovl_origin_uuid (ofs)) fh->fb.uuid = *uuid;return fh;
fs/overlayfs/namei.c +3 -2
@@ -171,8 +171,9 @@ struct dentry *ovl_decode_real_fh(struct ovl_fs *ofs, struct ovl_fh *fh,* layer where file handle will be decoded. * In case of uuid=off option just make sure that stored uuid is null. */ -if (ofs->config.uuid ? !uuid_equal (&fh->fb.uuid, &mnt->mnt_sb->s_uuid) : - !uuid_is_null (&fh->fb.uuid)) +if (ovl_origin_uuid (ofs) ? + !uuid_equal (&fh->fb.uuid, &mnt->mnt_sb->s_uuid) : + !uuid_is_null (&fh->fb.uuid))return NULL ; bytes = (fh->fb.len - offsetof(struct ovl_fb , fid));
fs/overlayfs/overlayfs.h +16 -0
@@ -67,6 +67,12 @@ enum { OVL_REDIRECT_ON, }; +enum { +OVL_UUID_OFF , +OVL_UUID_NULL , +OVL_UUID_ON , +}; +enum {OVL_XINO_OFF ,OVL_XINO_AUTO ,
@@ -534,6 +540,16 @@ static inline bool ovl_redirect_dir(struct ovl_fs *ofs)return ofs->config.redirect_mode ==OVL_REDIRECT_ON ; } +static inline bool ovl_origin_uuid (struct ovl_fs *ofs) +{ +return ofs->config.uuid !=OVL_UUID_OFF ; +} + +static inline bool ovl_has_fsid (struct ovl_fs *ofs) +{ +return ofs->config.uuid ==OVL_UUID_ON ; +} +/* * With xino=auto, we do best effort to keep all inodes on same st_dev and * d_ino consistent with st_ino.
fs/overlayfs/ovl_entry.h +1 -1
@@ -12,7 +12,7 @@ struct ovl_config {int redirect_mode;int verity_mode;bool index; -bool uuid; +int uuid;bool nfs_export;int xino;bool metacopy;
fs/overlayfs/params.c +21 -4
@@ -65,6 +65,23 @@ static const struct constant_table ovl_parameter_bool[] = { {} }; +static const struct constant_table ovl_parameter_uuid[] = { + {"off" ,OVL_UUID_OFF }, + {"null" ,OVL_UUID_NULL }, + {"on" ,OVL_UUID_ON }, + {} +}; + +static const char *ovl_uuid_mode (struct ovl_config *config) +{ +return ovl_parameter_uuid[config->uuid].name; +} + +static int ovl_uuid_def (void ) +{ +return OVL_UUID_NULL ; +} +static const struct constant_table ovl_parameter_xino[] = { {"off" ,OVL_XINO_OFF }, {"auto" ,OVL_XINO_AUTO },
@@ -129,7 +146,7 @@ const struct fs_parameter_spec ovl_parameter_spec[] = {fsparam_flag ("default_permissions" , Opt_default_permissions),fsparam_enum ("redirect_dir" , Opt_redirect_dir, ovl_parameter_redirect_dir),fsparam_enum ("index" , Opt_index, ovl_parameter_bool), -fsparam_enum ("uuid" , Opt_uuid, ovl_parameter_bool), +fsparam_enum ("uuid" , Opt_uuid, ovl_parameter_uuid),fsparam_enum ("nfs_export" , Opt_nfs_export, ovl_parameter_bool),fsparam_flag ("userxattr" , Opt_userxattr), fsparam_enum("xino", Opt_xino, ovl_parameter_xino),
@@ -701,7 +718,7 @@ int ovl_init_fs_context(struct fs_context *fc) ofs->config.redirect_mode =ovl_redirect_mode_def (); ofs->config.index = ovl_index_def; - ofs->config.uuid = true; + ofs->config.uuid =ovl_uuid_def (); ofs->config.nfs_export = ovl_nfs_export_def; ofs->config.xino =ovl_xino_def (); ofs->config.metacopy = ovl_metacopy_def;
@@ -947,8 +964,8 @@ int ovl_show_options(struct seq_file *m, struct dentry *dentry) ovl_redirect_mode(&ofs->config));if (ofs->config.index != ovl_index_def)seq_printf (m,",index=%s" , ofs->config.index ?"on" :"off" ); -if (!ofs->config.uuid) -seq_puts (m,",uuid=off" ); +if (ofs->config.uuid !=ovl_uuid_def ()) +seq_printf (m,",uuid=%s" ,ovl_uuid_mode (&ofs->config));if (ofs->config.nfs_export != ovl_nfs_export_def)seq_printf (m,",nfs_export=%s" , ofs->config.nfs_export ?"on" :"off" );
fs/overlayfs/super.c +11 -5
@@ -242,8 +242,9 @@ static int ovl_sync_fs(struct super_block *sb, int wait)*/ static int ovl_statfs (struct dentry *dentry,struct kstatfs *buf) { -struct ovl_fs *ofs = dentry->d_sb->s_fs_info; -struct dentry *root_dentry = dentry->d_sb->s_root; +struct super_block *sb = dentry->d_sb; +struct ovl_fs *ofs =OVL_FS (sb); +struct dentry *root_dentry = sb->s_root;struct path path;int err;
@@ -253,6 +254,8 @@ static int ovl_statfs(struct dentry *dentry, struct kstatfs *buf)if (!err) { buf->f_namelen = ofs->namelen; buf->f_type =OVERLAYFS_SUPER_MAGIC ; +if (ovl_has_fsid (ofs)) + buf->f_fsid =uuid_to_fsid (sb->s_uuid.b); }return err;
@@ -1421,9 +1424,12 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc)if (!ovl_upper_mnt (ofs)) sb->s_flags |=SB_RDONLY ; -if (!ofs->config.uuid && ofs->numfs >1 ) { -pr_warn ("The uuid=off requires a single fs for lower and upper, falling back to uuid=on.\n" ); - ofs->config.uuid = true; +if (!ovl_origin_uuid (ofs) && ofs->numfs >1 ) { +pr_warn ("The uuid=off requires a single fs for lower and upper, falling back to uuid=null.\n" ); + ofs->config.uuid =OVL_UUID_NULL ; + }else if (ovl_has_fsid (ofs)) { +/* Use per instance uuid/fsid */ +uuid_gen (&sb->s_uuid); }if (!ovl_force_readonly (ofs) && ofs->config.index) {
Signature · 1 line
2.34.1
Amir Goldstein
maintainerPATCH v2 3/4 ovl: store persistent uuid/fsid with uuid=on
In reply to Amir GoldsteinWith uuid=on, store a persistent uuid in xattr on the upper dir to give the overlayfs instance a persistent identifier.
This also makes f_fsid persistent and more reliable for reporting fid info in fanotify events.
uuid=on is not supported on non-upper overlayfs or with upper fs that does not support xattrs.
--- Documentation/filesystems/overlayfs.rst | 3 ++ fs/overlayfs/overlayfs.h | 3 ++ fs/overlayfs/params.c | 5 ++++ fs/overlayfs/super.c | 10 +++++-- fs/overlayfs/util.c | 39 +++++++++++++++++++++++++ 5 files changed, 57 insertions(+), 3 deletions(-)
Documentation/filesystems/overlayfs.rst +3 -0
@@ -671,6 +671,9 @@ controlled by the "uuid" mount option, which supports these values: UUID of underlying layers is ignored. - "on": UUID of overlayfs is generated and used to report a unique fsid. + UUID is stored in xattr "trusted.overlay.uuid", making overlayfs fsid + unique and persistent. This option requires an overlayfs with upper + filesystem that supports xattrs. Volatile mount
fs/overlayfs/overlayfs.h +3 -0
@@ -37,6 +37,7 @@ enum ovl_xattr {OVL_XATTR_IMPURE ,OVL_XATTR_NLINK ,OVL_XATTR_UPPER , +OVL_XATTR_UUID ,OVL_XATTR_METACOPY , OVL_XATTR_PROTATTR, };
@@ -465,6 +466,8 @@ bool ovl_already_copied_up(struct dentry *dentry, int flags);bool ovl_path_check_dir_xattr (struct ovl_fs *ofs,const struct path *path,enum ovl_xattr ox);bool ovl_path_check_origin_xattr (struct ovl_fs *ofs,const struct path *path); +bool ovl_init_uuid_xattr (struct super_block *sb,struct ovl_fs *ofs, +const struct path *upperpath);static inline bool ovl_check_origin_xattr (struct ovl_fs *ofs,struct dentry *upperdentry)
fs/overlayfs/params.c +5 -0
@@ -801,6 +801,11 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx, config->ovl_volatile = false; } +if (!config->upperdir && config->uuid ==OVL_UUID_ON ) { +pr_info ("option \"uuid=on\" requires an upper fs, falling back to uuid=null.\n" ); + config->uuid =OVL_UUID_NULL ; + } +/* Resolve verity -> metacopy dependency */ if (config->verity_mode && !config->metacopy) {/* Don't allow explicit specified conflicting combinations */
fs/overlayfs/super.c +7 -3
@@ -777,6 +777,10 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs, ofs->config.index = false;pr_warn ("...falling back to index=off.\n" ); } +if (ovl_has_fsid (ofs)) { + ofs->config.uuid =OVL_UUID_NULL ; +pr_warn ("...falling back to uuid=null.\n" ); + }/* * xattr support is required for persistent st_ino. * Without persistent st_ino, xino=auto falls back to xino=off.
@@ -1427,9 +1431,9 @@ int ovl_fill_super(struct super_block *sb, struct fs_context *fc)if (!ovl_origin_uuid (ofs) && ofs->numfs >1 ) {pr_warn ("The uuid=off requires a single fs for lower and upper, falling back to uuid=null.\n" ); ofs->config.uuid =OVL_UUID_NULL ; - }else if (ovl_has_fsid (ofs)) { -/* Use per instance uuid/fsid */ -uuid_gen (&sb->s_uuid); + }else if (ovl_has_fsid (ofs) &&ovl_upper_mnt (ofs)) { +/* Use per instance persistent uuid/fsid */ +ovl_init_uuid_xattr (sb, ofs, &ctx->upper); }if (!ovl_force_readonly (ofs) && ofs->config.index) {
fs/overlayfs/util.c +39 -0
@@ -676,6 +676,43 @@ bool ovl_path_check_origin_xattr(struct ovl_fs *ofs, const struct path *path)return false; } +/* +* Load persistent uuid from xattr into s_uuid if found, or store a new +* random generated value in s_uuid and in xattr. +*/ +bool ovl_init_uuid_xattr (struct super_block *sb,struct ovl_fs *ofs, +const struct path *upperpath) +{ +bool set = false; +int res; + +/* Try to load existing persistent uuid */ + res =ovl_path_getxattr (ofs, upperpath,OVL_XATTR_UUID , sb->s_uuid.b, +UUID_SIZE ); +if (res ==UUID_SIZE ) +return true; + +if (res != -ENODATA ) + goto fail; + +/* Generate overlay instance uuid */ +uuid_gen (&sb->s_uuid); + +/* Try to store persistent uuid */ + set = true; + res =ovl_setxattr (ofs, upperpath->dentry,OVL_XATTR_UUID , sb->s_uuid.b, +UUID_SIZE ); +if (res ==0 ) +return true; + +fail: +memset (sb->s_uuid.b,0 ,UUID_SIZE ); + ofs->config.uuid =OVL_UUID_NULL ; +pr_warn ("failed to %s uuid (%pd2, err=%i); falling back to uuid=null.\n" , + set ?"set" :"get" , upperpath->dentry, res); +return false; +} +bool ovl_path_check_dir_xattr (struct ovl_fs *ofs,const struct path *path,enum ovl_xattr ox) {
@@ -698,6 +735,7 @@ bool ovl_path_check_dir_xattr(struct ovl_fs *ofs, const struct path *path,#define OVL_XATTR_IMPURE_POSTFIX "impure"#define OVL_XATTR_NLINK_POSTFIX "nlink"#define OVL_XATTR_UPPER_POSTFIX "upper" +#define OVL_XATTR_UUID_POSTFIX "uuid"#define OVL_XATTR_METACOPY_POSTFIX "metacopy"#define OVL_XATTR_PROTATTR_POSTFIX "protattr"
@@ -712,6 +750,7 @@ const char *const ovl_xattr_table[][2] = {OVL_XATTR_TAB_ENTRY (OVL_XATTR_IMPURE ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_NLINK ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_UPPER ), +OVL_XATTR_TAB_ENTRY (OVL_XATTR_UUID ),OVL_XATTR_TAB_ENTRY (OVL_XATTR_METACOPY ), OVL_XATTR_TAB_ENTRY(OVL_XATTR_PROTATTR), };
Signature · 1 line
2.34.1
Amir Goldstein
maintainerPATCH v2 4/4 ovl: auto generate uuid for new overlay filesystems
In reply to Amir GoldsteinAdd a new mount option uuid=auto, which is the default.
If a persistent UUID xattr is found it is used.
Otherwise, an existing ovelrayfs with copied up subdirs in upper dir that was never mounted with uuid=on retains the null UUID.
A new overlayfs with no copied up subdirs, generates the persistent UUID on first mount.
--- Documentation/filesystems/overlayfs.rst | 8 +++++++- fs/overlayfs/overlayfs.h | 4 +++- fs/overlayfs/params.c | 3 ++- fs/overlayfs/util.c | 22 ++++++++++++++++++++++ 4 files changed, 34 insertions(+), 3 deletions(-)
Documentation/filesystems/overlayfs.rst +7 -1
@@ -664,7 +664,7 @@ UUID and fsid The UUID of overlayfs instance itself and the fsid reported by statfs(2) are controlled by the "uuid" mount option, which supports these values: -- "null": (default) +- "null": UUID of overlayfs is null. fsid is taken from upper most filesystem. - "off": UUID of overlayfs is null. fsid is taken from upper most filesystem.
@@ -674,6 +674,12 @@ controlled by the "uuid" mount option, which supports these values: UUID is stored in xattr "trusted.overlay.uuid", making overlayfs fsid unique and persistent. This option requires an overlayfs with upper filesystem that supports xattrs. +- "auto": (default) + UUID is taken from xattr "trusted.overlay.uuid" if it exists. + Upgrade to "uuid=on" on first time mount of new overlay filesystem that + meets the prerequites. + Downgrade to "uuid=null" for existing overlay filesystems that were never + mounted with "uuid=on". Volatile mount
fs/overlayfs/overlayfs.h +3 -1
@@ -71,6 +71,7 @@ enum {enum {OVL_UUID_OFF ,OVL_UUID_NULL , +OVL_UUID_AUTO ,OVL_UUID_ON , };
@@ -550,7 +551,8 @@ static inline bool ovl_origin_uuid(struct ovl_fs *ofs)static inline bool ovl_has_fsid (struct ovl_fs *ofs) { -return ofs->config.uuid ==OVL_UUID_ON ; +return ofs->config.uuid ==OVL_UUID_ON || + ofs->config.uuid ==OVL_UUID_AUTO ; }/*
fs/overlayfs/params.c +2 -1
@@ -68,6 +68,7 @@ static const struct constant_table ovl_parameter_bool[] = {static const struct constant_table ovl_parameter_uuid[] = { {"off" ,OVL_UUID_OFF }, {"null" ,OVL_UUID_NULL }, + {"auto" ,OVL_UUID_AUTO }, {"on" ,OVL_UUID_ON }, {} };
@@ -79,7 +80,7 @@ static const char *ovl_uuid_mode(struct ovl_config *config)static int ovl_uuid_def (void ) { -return OVL_UUID_NULL ; +return OVL_UUID_AUTO ; }static const struct constant_table ovl_parameter_xino[] = {
fs/overlayfs/util.c +22 -0
@@ -695,6 +695,28 @@ bool ovl_init_uuid_xattr(struct super_block *sb, struct ovl_fs *ofs,if (res != -ENODATA ) goto fail; +/* +* With uuid=auto, if uuid xattr is found, it will be used. +* If uuid xattrs is not found, generate a persistent uuid only on mount +* of new overlays where upper root dir is not yet marked as impure. +* An upper dir is marked as impure on copy up or lookup of its subdirs. +*/ +if (ofs->config.uuid ==OVL_UUID_AUTO ) { + res =ovl_path_getxattr (ofs, upperpath,OVL_XATTR_IMPURE ,NULL , +0 ); +if (res >0 ) { +/* Any mount of old overlay - downgrade to uuid=null */ + ofs->config.uuid =OVL_UUID_NULL ; +return true; + }else if (res == -ENODATA ) { +/* First mount of new overlay - upgrade to uuid=on */ + ofs->config.uuid =OVL_UUID_ON ; + }else if (res <0 ) { + goto fail; + } + + } +/* Generate overlay instance uuid */ uuid_gen (&sb->s_uuid);
Signature · 1 line
2.34.1
Amir Goldstein
maintainerQuoted 30 lines · Amir Goldstein
Miklos,
This is the second part of the work to support fanotify reporting of events with file ids on overlayfs. The
fanotify_event_info_fidstruct reported with fanotify events has an object file handle and an fsid, so fanotify requires that filesystems can encode file handles and have a non-zerof_fsid.The first part [1] that was merged to v6.5-rc1, relaxed the fanotify requirements for filesystems to support reporting events with fid to require only the ->encode_fh() export operation.
Patch 1 changes overlayfs
export_operationsto meet the new requirements with the default overlay configurations (i.e. no need for nfs_export=on), thus, allowing an fanotify watch withFAN_REPORT_FIDon overlayfs. There are LTS tests [2] forfanotify(FAN_REPORT_FID)+ overlayfs.Patches 2-4 are not strcitly needed to support reporting fanotify events with fid, because overlayfs already reports a non-zero
f_fsid, it's just not a unique fsid. So before allowing to report events with overlayfs fids, I wanted to fix overlayfs fsid to be more unique.I wanted to implement a persistent and unique fsid for overlayfs. I wanted it to be the default behavior, but needed to avoid breaking applications that rely on an existing overlayfs fsid to persist. I came up with a solution that is described in patch 4 (uuid=auto) that meets all the requirement above.
There is an xfstest to test the persistent-unique fsid feature [3].
Quoted 5 lines
This patch set has been in overlayfs-next for soaking since v6.5-rc1. If you have any reservations that we will not be able to resolve in time for 6.6, especially regarding the on-disk format and backward compat, we could also merge only patch 1 and leave the fsid patches for later.
Hi Miklos,
Any comments on this series (which is currently on overlayfs-next)?
Please let me know if you want me to take care of the 6.6 PR or if you intend to prepare it yourself.
Other candidates for 6.6 include:
- ovl lock re-ordering
https://lore.kernel.org/linux-unionfs/[email protected]/
- CONFIG_OVERLAY_FS_DEBUG
https://lore.kernel.org/linux-unionfs/[email protected]/
Thanks,
Amir.
Quoted 27 lines
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://github.com/amir73il/ltp/commits/ovl_encode_fid
[3] https://github.com/amir73il/xfstests/commits/ovl_fsidAmir Goldstein (4):
ovl: support encoding non-decodable file handles
ovl: add support for unique fsid per instance
ovl: store persistent uuid/fsid with uuid=on
ovl: auto generate uuid for new overlay filesystemsDocumentation/filesystems/overlayfs.rst | 25 ++++++++++ fs/overlayfs/copy_up.c | 2 +- fs/overlayfs/export.c | 26 ++++++++--- fs/overlayfs/inode.c | 2 +- fs/overlayfs/namei.c | 5 +- fs/overlayfs/overlayfs.h | 22 +++++++++ fs/overlayfs/ovl_entry.h | 3 +- fs/overlayfs/params.c | 31 +++++++++++-- fs/overlayfs/super.c | 29 ++++++++++-- fs/overlayfs/util.c | 61 +++++++++++++++++++++++++ 10 files changed, 186 insertions(+), 20 deletions(-)--
2.34.1
Miklos Szeredi
maintainerQuoted 38 lines · Amir Goldstein
Quoted 30 lines · Amir Goldstein
Miklos,
This is the second part of the work to support fanotify reporting of events with file ids on overlayfs. The
fanotify_event_info_fidstruct reported with fanotify events has an object file handle and an fsid, so fanotify requires that filesystems can encode file handles and have a non-zerof_fsid.The first part [1] that was merged to v6.5-rc1, relaxed the fanotify requirements for filesystems to support reporting events with fid to require only the ->encode_fh() export operation.
Patch 1 changes overlayfs
export_operationsto meet the new requirements with the default overlay configurations (i.e. no need for nfs_export=on), thus, allowing an fanotify watch withFAN_REPORT_FIDon overlayfs. There are LTS tests [2] forfanotify(FAN_REPORT_FID)+ overlayfs.Patches 2-4 are not strcitly needed to support reporting fanotify events with fid, because overlayfs already reports a non-zero
f_fsid, it's just not a unique fsid. So before allowing to report events with overlayfs fids, I wanted to fix overlayfs fsid to be more unique.I wanted to implement a persistent and unique fsid for overlayfs. I wanted it to be the default behavior, but needed to avoid breaking applications that rely on an existing overlayfs fsid to persist. I came up with a solution that is described in patch 4 (uuid=auto) that meets all the requirement above.
There is an xfstest to test the persistent-unique fsid feature [3].Quoted 5 lines
This patch set has been in overlayfs-next for soaking since v6.5-rc1. If you have any reservations that we will not be able to resolve in time for 6.6, especially regarding the on-disk format and backward compat, we could also merge only patch 1 and leave the fsid patches for later.
Quoted 3 lines
Hi Miklos,
Any comments on this series (which is currently on overlayfs-next)?
No issues from me on the design. Will review the patches.
Quoted 2 lines
Please let me know if you want me to take care of the 6.6 PR or if you intend to prepare it yourself.
I'd be very thankful If you could take care of the 6.6 PR.
Quoted 6 lines
Other candidates for 6.6 include:
- ovl lock re-ordering
https://lore.kernel.org/linux-unionfs/[email protected]/
-CONFIG_OVERLAY_FS_DEBUG
https://lore.kernel.org/linux-unionfs/[email protected]/
Yep, those look good, with the exception of the lockdep silencing. If it's a false positive, then we should teach lockdep about why. If it's a real deadlock, then let's try to fix it properly (only holding sb_writers over operations that need it).
Thanks,
Miklos
Amir Goldstein
maintainerQuoted 42 lines · Miklos Szeredi
Quoted 40 lines · Amir Goldstein
Quoted 30 lines · Amir Goldstein
Miklos,
This is the second part of the work to support fanotify reporting of events with file ids on overlayfs. The
fanotify_event_info_fidstruct reported with fanotify events has an object file handle and an fsid, so fanotify requires that filesystems can encode file handles and have a non-zerof_fsid.The first part [1] that was merged to v6.5-rc1, relaxed the fanotify requirements for filesystems to support reporting events with fid to require only the ->encode_fh() export operation.
Patch 1 changes overlayfs
export_operationsto meet the new requirements with the default overlay configurations (i.e. no need for nfs_export=on), thus, allowing an fanotify watch withFAN_REPORT_FIDon overlayfs. There are LTS tests [2] forfanotify(FAN_REPORT_FID)+ overlayfs.Patches 2-4 are not strcitly needed to support reporting fanotify events with fid, because overlayfs already reports a non-zero
f_fsid, it's just not a unique fsid. So before allowing to report events with overlayfs fids, I wanted to fix overlayfs fsid to be more unique.I wanted to implement a persistent and unique fsid for overlayfs. I wanted it to be the default behavior, but needed to avoid breaking applications that rely on an existing overlayfs fsid to persist. I came up with a solution that is described in patch 4 (uuid=auto) that meets all the requirement above.
There is an xfstest to test the persistent-unique fsid feature [3].Quoted 5 lines
This patch set has been in overlayfs-next for soaking since v6.5-rc1. If you have any reservations that we will not be able to resolve in time for 6.6, especially regarding the on-disk format and backward compat, we could also merge only patch 1 and leave the fsid patches for later.
Hi Miklos,
Quoted 8 lines
Quoted 1 line
Any comments on this series (which is currently on overlayfs-next)?
No issues from me on the design. Will review the patches.
Quoted 2 lines
Please let me know if you want me to take care of the 6.6 PR or if you intend to prepare it yourself.
I'd be very thankful If you could take care of the 6.6 PR.
No problem.
Quoted 7 lines
Quoted 6 lines
Other candidates for 6.6 include:
- ovl lock re-ordering
https://lore.kernel.org/linux-unionfs/[email protected]/
-CONFIG_OVERLAY_FS_DEBUG
https://lore.kernel.org/linux-unionfs/[email protected]/
Quoted 5 lines
Yep, those look good, with the exception of the lockdep silencing. If it's a false positive, then we should teach lockdep about why. If it's a real deadlock, then let's try to fix it properly (only holding
sb_writersover operations that need it).
Although it's a corner case, I think the deadlock is real. All it takes is two different ovl mount with the same upper fs, one is the lower of the other doing
- upper sb_writers
- lower ovl_inode_lock
- upper sb_writers
during copy up.
Technically, to avoid the hack, we can remove ovl_want_write() from ovl_copy_up_start() and do it individually for every low level helper that needs to modify upper fs.
I can try to see how noisy this will be.
Thanks,
Amir.