ceph: new mount device syntax
Subsystems (3)
v2 cover ceph: new mount device syntax
v2:
- doc suggestions/fixes by Jeff
- parse_fsid -> ceph_parse_fsid
- avoid kstrdup whereever possible, also fixes a memleak
- fail mount when mon_addr is unavailable (for new syntax)
- use dout() instead of invalfc() during new syntax check
This series introduces changes Ceph File System mount device string. Old mount device syntax (source) has the following problems:
mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.
Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.
New device syntax is as follows:
[email protected]=/path
Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
Venky Shankar (4):
ceph: new device mount syntax
ceph: validate cluster FSID for new device syntax
ceph: record updated mon_addr on remount
doc: document new CephFS mount device syntax
Documentation/filesystems/ceph.rst | 25 +++++- fs/ceph/super.c | 137 ++++++++++++++++++++++++++--- fs/ceph/super.h | 4 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +- 5 files changed, 157 insertions(+), 15 deletions(-)
Signature · 1 line
2.27.0
PATCH v2 1/4 ceph: new device mount syntax
Old mount device syntax (source) has the following problems:
- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.
- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.
New device syntax is as follows:
[email protected]=/path
Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
--- fs/ceph/super.c | 122 ++++++++++++++++++++++++++++++++++++++++++++---- fs/ceph/super.h | 3 ++ 2 files changed, 115 insertions(+), 10 deletions(-)
fs/ceph/super.c +112 -10
@@ -145,6 +145,7 @@ enum { Opt_mds_namespace, Opt_recover_session, Opt_source, + Opt_mon_addr,/* string args above */ Opt_dirstat, Opt_rbytes,
@@ -196,6 +197,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {fsparam_u32 ("rsize" , Opt_rsize),fsparam_string ("snapdirname" , Opt_snapdirname),fsparam_string ("source" , Opt_source), +fsparam_string ("mon_addr" , Opt_mon_addr),fsparam_u32 ("wsize" , Opt_wsize), fsparam_flag_no ("wsync", Opt_wsync), {}
@@ -226,10 +228,70 @@ static void canonicalize_path(char *path) path[j] ='\0' ; } +static int ceph_parse_old_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; + +if (*dev_name_end !=':' ) +return invalfc (fc,"separator ':' missing in source" ); + + r =ceph_parse_mon_ips (dev_name, dev_name_end - dev_name, + pctx->copts, fc->log.log); +if (r) +return r; + + fsopt->new_dev_syntax = false; +return 0 ; +} + +static int ceph_parse_new_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; +char *fsid_start, *fs_name_start; + +if (*dev_name_end !='=' ) { +dout ("separator '=' missing in source" ); +return -EINVAL ; + } + + fsid_start =strchr (dev_name,'@' ); +if (!fsid_start) +return invalfc (fc,"missing cluster fsid" ); + ++fsid_start;/* start of cluster fsid */ + + fs_name_start =strchr (fsid_start,'.' ); +if (!fs_name_start) +return invalfc (fc,"missing file system name" ); + + ++fs_name_start;/* start of file system name */ + fsopt->mds_namespace =kstrndup (fs_name_start, + dev_name_end - fs_name_start,GFP_KERNEL ); +if (!fsopt->mds_namespace) +return -ENOMEM ; +dout ("file system (mds namespace) '%s'\n" , fsopt->mds_namespace); + + fsopt->new_dev_syntax = true; +return 0 ; +} +/* -* Parse the source parameter. Distinguish the server list from the path. +* Parse the source parameter for new device format. Distinguish the device +* spec from the path. Try parsing new device format and fallback to old +* format if needed. * -* The source will look like: +* New device syntax will looks like: +* <device_spec>=/<path> +* where +* <device_spec> is [email protected] +* <path> is optional, but if present must begin with '/' +* (monitor addresses are passed via mount option) +* +* Old device syntax is: * <server_spec>[,<server_spec>...]:[<path>] * where * <server_spec> is <ip>[:<port>]
@@ -262,22 +324,48 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) dev_name_end = dev_name +strlen (dev_name); } - dev_name_end--;/* back up to ':' separator */ -if (dev_name_end < dev_name || *dev_name_end !=':' ) -return invalfc (fc,"No path or : separator in source" ); + dev_name_end--;/* back up to separator */ +if (dev_name_end < dev_name) +return invalfc (fc,"path missing in source" );dout ("device name '%.*s'\n" , (int )(dev_name_end - dev_name), dev_name);if (fsopt->server_path)dout ("server path '%s'\n" , fsopt->server_path); - ret =ceph_parse_mon_ips (param->string, dev_name_end - dev_name, - pctx->copts, fc->log.log); +dout ("trying new device syntax" ); + ret =ceph_parse_new_source (dev_name, dev_name_end, fc); +if (ret ==0 ) + goto done; + +dout ("trying old device syntax" ); + ret =ceph_parse_old_source (dev_name, dev_name_end, fc);if (ret) -return ret; + goto out; + done: fc->source = param->string; param->string =NULL ; -return 0 ; + out: +return ret; +} + +static int ceph_parse_mon_addr (struct fs_parameter *param, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; + +kfree (fsopt->mon_addr); + fsopt->mon_addr = param->string; + param->string =NULL ; + +strreplace (fsopt->mon_addr,'/' ,',' ); + r =ceph_parse_mon_ips (fsopt->mon_addr,strlen (fsopt->mon_addr), + pctx->copts, fc->log.log); +// since its used in ceph_show_options() +strreplace (fsopt->mon_addr,',' ,'/' ); +return r; } static int ceph_parse_mount_param(struct fs_context *fc,
@@ -322,6 +410,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,if (fc->source)return invalfc (fc,"Multiple sources specified" );return ceph_parse_source (param, fc); +case Opt_mon_addr: +return ceph_parse_mon_addr (param, fc);case Opt_wsize:if (result.uint_32 <PAGE_SIZE || result.uint_32 >CEPH_MAX_WRITE_SIZE )
@@ -473,6 +563,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)kfree (args->mds_namespace);kfree (args->server_path);kfree (args->fscache_uniq); +kfree (args->mon_addr);kfree (args); }
@@ -516,6 +607,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,if (ret)return ret; + ret =strcmp_null (fsopt1->mon_addr, fsopt2->mon_addr); +if (ret) +return ret; +return ceph_compare_options (new_opt, fsc->client); }
@@ -571,9 +666,13 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)if ((fsopt->flags &CEPH_MOUNT_OPT_NOCOPYFROM ) ==0 )seq_puts (m,",copyfrom" ); -if (fsopt->mds_namespace) +/* dump mds_namespace when old device syntax is in use */ +if (fsopt->mds_namespace && !fsopt->new_dev_syntax)seq_show_option (m,"mds_namespace" , fsopt->mds_namespace); +if (fsopt->mon_addr) +seq_printf (m,",mon_addr=%s" , fsopt->mon_addr); +if (fsopt->flags &CEPH_MOUNT_OPT_CLEANRECOVER )seq_show_option (m,"recover_session" ,"clean" );
@@ -1048,6 +1147,7 @@ static int ceph_setup_bdi(struct super_block *sb, struct ceph_fs_client *fsc)static int ceph_get_tree (struct fs_context *fc) {struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts;struct super_block *sb;struct ceph_fs_client *fsc;struct dentry *res;
@@ -1059,6 +1159,8 @@ static int ceph_get_tree(struct fs_context *fc)if (!fc->source)return invalfc (fc,"No source" ); +if (fsopt->new_dev_syntax && !fsopt->mon_addr) +return invalfc (fc,"No monitor address" );/* create client (which we may/may not use) */ fsc =create_fs_client (pctx->opts, pctx->copts);
fs/ceph/super.h +3 -0
@@ -87,6 +87,8 @@ struct ceph_mount_options {unsigned int max_readdir;/* max readdir result (entries) */ unsigned int max_readdir_bytes;/* max readdir result (bytes) */ +bool new_dev_syntax; +/* * everything above this point can be memcmp'd; everything below * is handled in compare_mount_options()
@@ -96,6 +98,7 @@ struct ceph_mount_options {char *mds_namespace;/* default NULL */ char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ +char *mon_addr; };struct ceph_fs_client {
Signature · 1 line
2.27.0
Re: [PATCH v2 1/4] ceph: new device mount syntax
Quoted 139 lines · Venky Shankar
Old mount device syntax (source) has the following problems:
- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.
- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in
/proc/mountsis different than what was used for mounting.New device syntax is as follows:
[email protected]=/path
Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 122 ++++++++++++++++++++++++++++++++++++++++++++---- fs/ceph/super.h | 3 ++ 2 files changed, 115 insertions(+), 10 deletions(-)fs/ceph/super.c +67
-7@@ -145,6 +145,7 @@ enum { Opt_mds_namespace, Opt_recover_session, Opt_source, + Opt_mon_addr,/* string args above */ Opt_dirstat, Opt_rbytes,@@ -196,6 +197,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {fsparam_u32 ("rsize" , Opt_rsize),fsparam_string ("snapdirname" , Opt_snapdirname),fsparam_string ("source" , Opt_source), +fsparam_string ("mon_addr" , Opt_mon_addr),fsparam_u32 ("wsize" , Opt_wsize), fsparam_flag_no ("wsync", Opt_wsync), {}@@ -226,10 +228,70 @@ static void canonicalize_path(char *path) path[j] ='\0' ; } +static int ceph_parse_old_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; + +if (*dev_name_end !=':' ) +return invalfc (fc,"separator ':' missing in source" ); + + r =ceph_parse_mon_ips (dev_name, dev_name_end - dev_name, + pctx->copts, fc->log.log); +if (r) +return r; + + fsopt->new_dev_syntax = false; +return 0 ; +} + +static int ceph_parse_new_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; +char *fsid_start, *fs_name_start; + +if (*dev_name_end !='=' ) { +dout ("separator '=' missing in source" ); +return -EINVAL ; + } + + fsid_start =strchr (dev_name,'@' ); +if (!fsid_start) +return invalfc (fc,"missing cluster fsid" ); + ++fsid_start;/* start of cluster fsid */ + + fs_name_start =strchr (fsid_start,'.' ); +if (!fs_name_start) +return invalfc (fc,"missing file system name" ); + + ++fs_name_start;/* start of file system name */ + fsopt->mds_namespace =kstrndup (fs_name_start, + dev_name_end - fs_name_start,GFP_KERNEL ); +if (!fsopt->mds_namespace) +return -ENOMEM ; +dout ("file system (mds namespace) '%s'\n" , fsopt->mds_namespace); + + fsopt->new_dev_syntax = true; +return 0 ; +} +/* -* Parse the source parameter. Distinguish the server list from the path. +* Parse the source parameter for new device format. Distinguish the device +* spec from the path. Try parsing new device format and fallback to old +* format if needed. * -* The source will look like: +* New device syntax will looks like: +* <device_spec>=/<path> +* where +* <device_spec> is [email protected] +* <path> is optional, but if present must begin with '/' +* (monitor addresses are passed via mount option) +* +* Old device syntax is: * <server_spec>[,<server_spec>...]:[<path>] * where * <server_spec> is <ip>[:<port>] @@ -262,22 +324,48 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) dev_name_end = dev_name +strlen (dev_name); } - dev_name_end--;/* back up to ':' separator */ -if (dev_name_end < dev_name || *dev_name_end !=':' ) -return invalfc (fc,"No path or : separator in source" ); + dev_name_end--;/* back up to separator */ +if (dev_name_end < dev_name) +return invalfc (fc,"path missing in source" );dout ("device name '%.*s'\n" , (int )(dev_name_end - dev_name), dev_name);if (fsopt->server_path)dout ("server path '%s'\n" , fsopt->server_path); - ret =ceph_parse_mon_ips (param->string, dev_name_end - dev_name, - pctx->copts, fc->log.log);
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 343
Quoted 10 lines
fs/ceph/super.c +8
-1+dout ("trying new device syntax" ); + ret =ceph_parse_new_source (dev_name, dev_name_end, fc); +if (ret ==0 ) + goto done; + +dout ("trying old device syntax" ); + ret =ceph_parse_old_source (dev_name, dev_name_end, fc);if (ret) -return ret; + goto out;
Do we really need this goto? My (personal) preference would be to drop
this one and simply return. And maybe it's even possible to drop the
previous 'goto done' too.
Quoted 13 lines
fs/ceph/super.c +10
-1+ done: fc->source = param->string; param->string =NULL ; -return 0 ; + out: +return ret; +} + +static int ceph_parse_mon_addr (struct fs_parameter *param, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private;
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 366
Quoted 10 lines
fs/ceph/super.c +10
-0+struct ceph_mount_options *fsopt = pctx->opts; + +kfree (fsopt->mon_addr); + fsopt->mon_addr = param->string; + param->string =NULL ; + +strreplace (fsopt->mon_addr,'/' ,',' ); + r =ceph_parse_mon_ips (fsopt->mon_addr,strlen (fsopt->mon_addr), + pctx->copts, fc->log.log); +// since its used in ceph_show_options()
(nit: use c-style comments...? yeah, again personal preference :-)
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 367
Quoted 1 line
fs/ceph/super.c +1
-0+strreplace (fsopt->mon_addr,',' ,'/' );
This is ugly. Have you considered modifying ceph_parse_mon_ips() (and ceph_parse_ips()) to receive a delimiter as a parameter?
Quoted 33 lines
fs/ceph/super.c +8
-0+return r; } static int ceph_parse_mount_param(struct fs_context *fc,@@ -322,6 +410,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,if (fc->source)return invalfc (fc,"Multiple sources specified" );return ceph_parse_source (param, fc); +case Opt_mon_addr: +return ceph_parse_mon_addr (param, fc);case Opt_wsize:if (result.uint_32 <PAGE_SIZE || result.uint_32 >CEPH_MAX_WRITE_SIZE )@@ -473,6 +563,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)kfree (args->mds_namespace);kfree (args->server_path);kfree (args->fscache_uniq); +kfree (args->mon_addr);kfree (args); }@@ -516,6 +607,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,if (ret)return ret; + ret =strcmp_null (fsopt1->mon_addr, fsopt2->mon_addr); +if (ret) +return ret; +return ceph_compare_options (new_opt, fsc->client); }
fs/ceph/super.c static int ceph_show_options(struct seq_file *m, struct dentry *root) · line 671
Quoted 7 lines
fs/ceph/super.c +2
-1@@ -571,9 +666,13 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)if ((fsopt->flags &CEPH_MOUNT_OPT_NOCOPYFROM ) ==0 )seq_puts (m,",copyfrom" ); -if (fsopt->mds_namespace) +/* dump mds_namespace when old device syntax is in use */ +if (fsopt->mds_namespace && !fsopt->new_dev_syntax)seq_show_option (m,"mds_namespace" , fsopt->mds_namespace);
I haven't really tested it, but... what happens if we set mds_namespace *and* we're using the new syntax? Or, in another words, what should happen?
Cheers,
--
Luís
Quoted 48 lines
fs/ceph/super.c +6
-0+if (fsopt->mon_addr) +seq_printf (m,",mon_addr=%s" , fsopt->mon_addr); +if (fsopt->flags &CEPH_MOUNT_OPT_CLEANRECOVER )seq_show_option (m,"recover_session" ,"clean" );@@ -1048,6 +1147,7 @@ static int ceph_setup_bdi(struct super_block *sb, struct ceph_fs_client *fsc)static int ceph_get_tree (struct fs_context *fc) {struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts;struct super_block *sb;struct ceph_fs_client *fsc;struct dentry *res;@@ -1059,6 +1159,8 @@ static int ceph_get_tree(struct fs_context *fc)if (!fc->source)return invalfc (fc,"No source" ); +if (fsopt->new_dev_syntax && !fsopt->mon_addr) +return invalfc (fc,"No monitor address" );/* create client (which we may/may not use) */ fsc =create_fs_client (pctx->opts, pctx->copts);fs/ceph/super.h +3
-0@@ -87,6 +87,8 @@ struct ceph_mount_options {unsigned int max_readdir;/* max readdir result (entries) */ unsigned int max_readdir_bytes;/* max readdir result (bytes) */ +bool new_dev_syntax; +/* * everything above this point can be memcmp'd; everything below * is handled in compare_mount_options() @@ -96,6 +98,7 @@ struct ceph_mount_options {char *mds_namespace;/* default NULL */ char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ +char *mon_addr; };struct ceph_fs_client {Signature · 1 line
2.27.0
Re: [PATCH v2 1/4] ceph: new device mount syntax
Quoted 139 lines · Luis Henriques
Quoted 137 lines · Venky Shankar
Old mount device syntax (source) has the following problems:
- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.
- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in
/proc/mountsis different than what was used for mounting.New device syntax is as follows:
[email protected]=/path
Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 122 ++++++++++++++++++++++++++++++++++++++++++++---- fs/ceph/super.h | 3 ++ 2 files changed, 115 insertions(+), 10 deletions(-)fs/ceph/super.c +67
-5@@ -145,6 +145,7 @@ enum { Opt_mds_namespace, Opt_recover_session, Opt_source, + Opt_mon_addr,/* string args above */ Opt_dirstat, Opt_rbytes,@@ -196,6 +197,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {fsparam_u32 ("rsize" , Opt_rsize),fsparam_string ("snapdirname" , Opt_snapdirname),fsparam_string ("source" , Opt_source), +fsparam_string ("mon_addr" , Opt_mon_addr),fsparam_u32 ("wsize" , Opt_wsize), fsparam_flag_no ("wsync", Opt_wsync), {}@@ -226,10 +228,70 @@ static void canonicalize_path(char *path) path[j] ='\0' ; } +static int ceph_parse_old_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; + +if (*dev_name_end !=':' ) +return invalfc (fc,"separator ':' missing in source" ); + + r =ceph_parse_mon_ips (dev_name, dev_name_end - dev_name, + pctx->copts, fc->log.log); +if (r) +return r; + + fsopt->new_dev_syntax = false; +return 0 ; +} + +static int ceph_parse_new_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; +char *fsid_start, *fs_name_start; + +if (*dev_name_end !='=' ) { +dout ("separator '=' missing in source" ); +return -EINVAL ; + } + + fsid_start =strchr (dev_name,'@' ); +if (!fsid_start) +return invalfc (fc,"missing cluster fsid" ); + ++fsid_start;/* start of cluster fsid */ + + fs_name_start =strchr (fsid_start,'.' ); +if (!fs_name_start) +return invalfc (fc,"missing file system name" ); + + ++fs_name_start;/* start of file system name */ + fsopt->mds_namespace =kstrndup (fs_name_start, + dev_name_end - fs_name_start,GFP_KERNEL ); +if (!fsopt->mds_namespace) +return -ENOMEM ; +dout ("file system (mds namespace) '%s'\n" , fsopt->mds_namespace); + + fsopt->new_dev_syntax = true; +return 0 ; +} +/* -* Parse the source parameter. Distinguish the server list from the path. +* Parse the source parameter for new device format. Distinguish the device +* spec from the path. Try parsing new device format and fallback to old +* format if needed. * -* The source will look like: +* New device syntax will looks like: +* <device_spec>=/<path> +* where +* <device_spec> is [email protected] +* <path> is optional, but if present must begin with '/' +* (monitor addresses are passed via mount option) +* +* Old device syntax is: * <server_spec>[,<server_spec>...]:[<path>] * where * <server_spec> is <ip>[:<port>] @@ -262,22 +324,48 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) dev_name_end = dev_name +strlen (dev_name); } - dev_name_end--;/* back up to ':' separator */ -if (dev_name_end < dev_name || *dev_name_end !=':' ) -return invalfc (fc,"No path or : separator in source" ); + dev_name_end--;/* back up to separator */ +if (dev_name_end < dev_name) +return invalfc (fc,"path missing in source" );dout ("device name '%.*s'\n" , (int )(dev_name_end - dev_name), dev_name);if (fsopt->server_path)dout ("server path '%s'\n" , fsopt->server_path);
Quoted 16 lines
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 343
Quoted 12 lines
fs/ceph/super.c +8
-3- ret =ceph_parse_mon_ips (param->string, dev_name_end - dev_name, - pctx->copts, fc->log.log); +dout ("trying new device syntax" ); + ret =ceph_parse_new_source (dev_name, dev_name_end, fc); +if (ret ==0 ) + goto done; + +dout ("trying old device syntax" ); + ret =ceph_parse_old_source (dev_name, dev_name_end, fc);if (ret) -return ret; + goto out;Do we really need this goto? My (personal) preference would be to drop
this one and simply return. And maybe it's even possible to drop the
previous 'goto done' too.
Sure. Just a preference I had.
Quoted 27 lines
Quoted 14 lines
fs/ceph/super.c +10
-1+ done: fc->source = param->string; param->string =NULL ; -return 0 ; + out: +return ret; +} + +static int ceph_parse_mon_addr (struct fs_parameter *param, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private;fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 366
Quoted 10 lines
fs/ceph/super.c +10
-0+struct ceph_mount_options *fsopt = pctx->opts; + +kfree (fsopt->mon_addr); + fsopt->mon_addr = param->string; + param->string =NULL ; + +strreplace (fsopt->mon_addr,'/' ,',' ); + r =ceph_parse_mon_ips (fsopt->mon_addr,strlen (fsopt->mon_addr), + pctx->copts, fc->log.log); +// since its used in ceph_show_options() (nit: use c-style comments...? yeah, again personal preference :-)
Quoted 4 lines
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 367
Quoted 1 line
fs/ceph/super.c +1
-0+strreplace (fsopt->mon_addr,',' ,'/' );This is ugly. Have you considered modifying
ceph_parse_mon_ips()(andceph_parse_ips()) to receive a delimiter as a parameter?''
I did. However, for most cases (all possibly), "," is the addr delimiter. So, I didn't take the approach to make these helpers really generic for parsing addrs.
Quoted 37 lines
Quoted 37 lines
fs/ceph/super.c +8
-0+return r; } static int ceph_parse_mount_param(struct fs_context *fc,@@ -322,6 +410,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,if (fc->source)return invalfc (fc,"Multiple sources specified" );return ceph_parse_source (param, fc); +case Opt_mon_addr: +return ceph_parse_mon_addr (param, fc);case Opt_wsize:if (result.uint_32 <PAGE_SIZE || result.uint_32 >CEPH_MAX_WRITE_SIZE )@@ -473,6 +563,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)kfree (args->mds_namespace);kfree (args->server_path);kfree (args->fscache_uniq); +kfree (args->mon_addr);kfree (args); }@@ -516,6 +607,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,if (ret)return ret; + ret =strcmp_null (fsopt1->mon_addr, fsopt2->mon_addr); +if (ret) +return ret; +return ceph_compare_options (new_opt, fsc->client); }@@ -571,9 +666,13 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)if ((fsopt->flags &CEPH_MOUNT_OPT_NOCOPYFROM ) ==0 )seq_puts (m,",copyfrom" );
Quoted 8 lines
fs/ceph/super.c static int ceph_show_options(struct seq_file *m, struct dentry *root) · line 671
Quoted 4 lines
fs/ceph/super.c +2
-1-if (fsopt->mds_namespace) +/* dump mds_namespace when old device syntax is in use */ +if (fsopt->mds_namespace && !fsopt->new_dev_syntax)seq_show_option (m,"mds_namespace" , fsopt->mds_namespace);I haven't really tested it, but... what happens if we set
mds_namespace*and* we're using the new syntax? Or, in another words, what should happen?
Depends on the order of token parsing in ceph_parse_mount_param() which mds_namespace gets used.
(however, that means ceph_parse_new_source() should additionally kfree(fsopt->mds_namespace)).
Quoted 54 lines
Cheers,
--
LuísQuoted 48 lines
fs/ceph/super.c +6
-0+if (fsopt->mon_addr) +seq_printf (m,",mon_addr=%s" , fsopt->mon_addr); +if (fsopt->flags &CEPH_MOUNT_OPT_CLEANRECOVER )seq_show_option (m,"recover_session" ,"clean" );@@ -1048,6 +1147,7 @@ static int ceph_setup_bdi(struct super_block *sb, struct ceph_fs_client *fsc)static int ceph_get_tree (struct fs_context *fc) {struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts;struct super_block *sb;struct ceph_fs_client *fsc;struct dentry *res;@@ -1059,6 +1159,8 @@ static int ceph_get_tree(struct fs_context *fc)if (!fc->source)return invalfc (fc,"No source" ); +if (fsopt->new_dev_syntax && !fsopt->mon_addr) +return invalfc (fc,"No monitor address" );/* create client (which we may/may not use) */ fsc =create_fs_client (pctx->opts, pctx->copts);fs/ceph/super.h +3
-1@@ -87,6 +87,8 @@ struct ceph_mount_options {unsigned int max_readdir;/* max readdir result (entries) */ unsigned int max_readdir_bytes;/* max readdir result (bytes) */ +bool new_dev_syntax; +/* * everything above this point can be memcmp'd; everything below * is handled in compare_mount_options() @@ -96,6 +98,7 @@ struct ceph_mount_options {char *mds_namespace;/* default NULL */ char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ +char *mon_addr; };struct ceph_fs_client { --2.27.0
Signature · 2 lines
Cheers, Venky
Re: [PATCH v2 1/4] ceph: new device mount syntax
Quoted 186 lines · Venky Shankar
Quoted 139 lines · Luis Henriques
Quoted 137 lines · Venky Shankar
Old mount device syntax (source) has the following problems:
- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.
- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in
/proc/mountsis different than what was used for mounting.New device syntax is as follows:
[email protected]=/path
Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 122 ++++++++++++++++++++++++++++++++++++++++++++---- fs/ceph/super.h | 3 ++ 2 files changed, 115 insertions(+), 10 deletions(-)fs/ceph/super.c +67
-5@@ -145,6 +145,7 @@ enum { Opt_mds_namespace, Opt_recover_session, Opt_source, + Opt_mon_addr,/* string args above */ Opt_dirstat, Opt_rbytes,@@ -196,6 +197,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {fsparam_u32 ("rsize" , Opt_rsize),fsparam_string ("snapdirname" , Opt_snapdirname),fsparam_string ("source" , Opt_source), +fsparam_string ("mon_addr" , Opt_mon_addr),fsparam_u32 ("wsize" , Opt_wsize), fsparam_flag_no ("wsync", Opt_wsync), {}@@ -226,10 +228,70 @@ static void canonicalize_path(char *path) path[j] ='\0' ; } +static int ceph_parse_old_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; + +if (*dev_name_end !=':' ) +return invalfc (fc,"separator ':' missing in source" ); + + r =ceph_parse_mon_ips (dev_name, dev_name_end - dev_name, + pctx->copts, fc->log.log); +if (r) +return r; + + fsopt->new_dev_syntax = false; +return 0 ; +} + +static int ceph_parse_new_source (const char *dev_name,const char *dev_name_end, +struct fs_context *fc) +{ +struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts; +char *fsid_start, *fs_name_start; + +if (*dev_name_end !='=' ) { +dout ("separator '=' missing in source" ); +return -EINVAL ; + } + + fsid_start =strchr (dev_name,'@' ); +if (!fsid_start) +return invalfc (fc,"missing cluster fsid" ); + ++fsid_start;/* start of cluster fsid */ + + fs_name_start =strchr (fsid_start,'.' ); +if (!fs_name_start) +return invalfc (fc,"missing file system name" ); + + ++fs_name_start;/* start of file system name */ + fsopt->mds_namespace =kstrndup (fs_name_start, + dev_name_end - fs_name_start,GFP_KERNEL ); +if (!fsopt->mds_namespace) +return -ENOMEM ; +dout ("file system (mds namespace) '%s'\n" , fsopt->mds_namespace); + + fsopt->new_dev_syntax = true; +return 0 ; +} +/* -* Parse the source parameter. Distinguish the server list from the path. +* Parse the source parameter for new device format. Distinguish the device +* spec from the path. Try parsing new device format and fallback to old +* format if needed. * -* The source will look like: +* New device syntax will looks like: +* <device_spec>=/<path> +* where +* <device_spec> is [email protected] +* <path> is optional, but if present must begin with '/' +* (monitor addresses are passed via mount option) +* +* Old device syntax is: * <server_spec>[,<server_spec>...]:[<path>] * where * <server_spec> is <ip>[:<port>] @@ -262,22 +324,48 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) dev_name_end = dev_name +strlen (dev_name); } - dev_name_end--;/* back up to ':' separator */ -if (dev_name_end < dev_name || *dev_name_end !=':' ) -return invalfc (fc,"No path or : separator in source" ); + dev_name_end--;/* back up to separator */ +if (dev_name_end < dev_name) +return invalfc (fc,"path missing in source" );dout ("device name '%.*s'\n" , (int )(dev_name_end - dev_name), dev_name);if (fsopt->server_path)dout ("server path '%s'\n" , fsopt->server_path);Quoted 16 lines
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 343
Quoted 12 lines
fs/ceph/super.c +8
-3- ret =ceph_parse_mon_ips (param->string, dev_name_end - dev_name, - pctx->copts, fc->log.log); +dout ("trying new device syntax" ); + ret =ceph_parse_new_source (dev_name, dev_name_end, fc); +if (ret ==0 ) + goto done; + +dout ("trying old device syntax" ); + ret =ceph_parse_old_source (dev_name, dev_name_end, fc);if (ret) -return ret; + goto out;Do we really need this goto? My (personal) preference would be to drop
this one and simply return. And maybe it's even possible to drop the
previous 'goto done' too.Sure. Just a preference I had.
Quoted 27 lines
Quoted 14 lines
fs/ceph/super.c +10
-1+ done: fc->source = param->string; param->string =NULL ; -return 0 ; + out: +return ret; +} + +static int ceph_parse_mon_addr (struct fs_parameter *param, +struct fs_context *fc) +{ +int r; +struct ceph_parse_opts_ctx *pctx = fc->fs_private;fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 366
Quoted 10 lines
fs/ceph/super.c +10
-0+struct ceph_mount_options *fsopt = pctx->opts; + +kfree (fsopt->mon_addr); + fsopt->mon_addr = param->string; + param->string =NULL ; + +strreplace (fsopt->mon_addr,'/' ,',' ); + r =ceph_parse_mon_ips (fsopt->mon_addr,strlen (fsopt->mon_addr), + pctx->copts, fc->log.log); +// since its used in ceph_show_options() (nit: use c-style comments...? yeah, again personal preference :-)
Quoted 9 lines
Quoted 4 lines
fs/ceph/super.c static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc) · line 367
Quoted 1 line
fs/ceph/super.c +1
-0+strreplace (fsopt->mon_addr,',' ,'/' );This is ugly. Have you considered modifying
ceph_parse_mon_ips()(andceph_parse_ips()) to receive a delimiter as a parameter?''I did. However, for most cases (all possibly), "," is the addr delimiter. So, I didn't take the approach to make these helpers really generic for parsing addrs.
Yeah, I'm not a fan of this either. Why not just add an extra char argument to to ceph_parse_mon_ips and use that to pass in a delimiter char? Looks like you'll have to plumb it through a couple of functions but that seems doable.
Quoted 37 lines
Quoted 37 lines
Quoted 37 lines
fs/ceph/super.c +8
-0+return r; } static int ceph_parse_mount_param(struct fs_context *fc,@@ -322,6 +410,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,if (fc->source)return invalfc (fc,"Multiple sources specified" );return ceph_parse_source (param, fc); +case Opt_mon_addr: +return ceph_parse_mon_addr (param, fc);case Opt_wsize:if (result.uint_32 <PAGE_SIZE || result.uint_32 >CEPH_MAX_WRITE_SIZE )@@ -473,6 +563,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)kfree (args->mds_namespace);kfree (args->server_path);kfree (args->fscache_uniq); +kfree (args->mon_addr);kfree (args); }@@ -516,6 +607,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,if (ret)return ret; + ret =strcmp_null (fsopt1->mon_addr, fsopt2->mon_addr); +if (ret) +return ret; +return ceph_compare_options (new_opt, fsc->client); }@@ -571,9 +666,13 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)if ((fsopt->flags &CEPH_MOUNT_OPT_NOCOPYFROM ) ==0 )seq_puts (m,",copyfrom" );
Quoted 8 lines
Quoted 8 lines
fs/ceph/super.c static int ceph_show_options(struct seq_file *m, struct dentry *root) · line 671
Quoted 4 lines
fs/ceph/super.c +2
-1-if (fsopt->mds_namespace) +/* dump mds_namespace when old device syntax is in use */ +if (fsopt->mds_namespace && !fsopt->new_dev_syntax)seq_show_option (m,"mds_namespace" , fsopt->mds_namespace);I haven't really tested it, but... what happens if we set
mds_namespace*and* we're using the new syntax? Or, in another words, what should happen?
Probably best to just throw an error at mount time if they don't match.
Quoted 63 lines
Depends on the order of token parsing in
ceph_parse_mount_param()whichmds_namespacegets used.(however, that means
ceph_parse_new_source()should additionallykfree(fsopt->mds_namespace)).Quoted 53 lines
Cheers,
--
LuísQuoted 48 lines
fs/ceph/super.c +6
-0+if (fsopt->mon_addr) +seq_printf (m,",mon_addr=%s" , fsopt->mon_addr); +if (fsopt->flags &CEPH_MOUNT_OPT_CLEANRECOVER )seq_show_option (m,"recover_session" ,"clean" );@@ -1048,6 +1147,7 @@ static int ceph_setup_bdi(struct super_block *sb, struct ceph_fs_client *fsc)static int ceph_get_tree (struct fs_context *fc) {struct ceph_parse_opts_ctx *pctx = fc->fs_private; +struct ceph_mount_options *fsopt = pctx->opts;struct super_block *sb;struct ceph_fs_client *fsc;struct dentry *res;@@ -1059,6 +1159,8 @@ static int ceph_get_tree(struct fs_context *fc)if (!fc->source)return invalfc (fc,"No source" ); +if (fsopt->new_dev_syntax && !fsopt->mon_addr) +return invalfc (fc,"No monitor address" );/* create client (which we may/may not use) */ fsc =create_fs_client (pctx->opts, pctx->copts);fs/ceph/super.h +3
-1@@ -87,6 +87,8 @@ struct ceph_mount_options {unsigned int max_readdir;/* max readdir result (entries) */ unsigned int max_readdir_bytes;/* max readdir result (bytes) */ +bool new_dev_syntax; +/* * everything above this point can be memcmp'd; everything below * is handled in compare_mount_options() @@ -96,6 +98,7 @@ struct ceph_mount_options {char *mds_namespace;/* default NULL */ char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ +char *mon_addr; };struct ceph_fs_client { --2.27.0
Signature · 1 line
Jeff Layton <[email protected]>
PATCH v2 2/4 ceph: validate cluster FSID for new device syntax
In reply to Venky ShankarThe new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename parse_fsid() to ceph_parse_fsid() as it is too generic.
--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)
fs/ceph/super.c +9 -0
@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );
@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid); + fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );
fs/ceph/super.h +1 -0
@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {
include/linux/ceph/libceph.h +1 -0
@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;
net/ceph/ceph_common.c +3 -2
@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];
@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options
@@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err;
Signature · 1 line
2.27.0
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );
fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);
This call to ceph_check_fsid() made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:
[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msg
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Cheers,
--
Luís
Quoted 63 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +3
-2@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err;Signature · 1 line
2.27.0
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 48 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msg
Quoted 5 lines
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
I forgot to mention that the above errors are, obviously, not due to this call to ceph_check_fsid() but rather from calling it from mon_dispatch().
Cheers,
--
Luís
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
In reply to Luis HenriquesQuoted 49 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msg
Quoted 5 lines
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Yes, this results in a big dump of messages. I haven't looked at gracefully handling these.
I'm not sure if it needs to be done in these set of patches though.
Quoted 69 lines
Cheers,
--
LuísQuoted 63 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +3
-3@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err; --2.27.0
Signature · 2 lines
Cheers, Venky
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 52 lines · Venky Shankar
Quoted 51 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msg... followed by a msg dump.
Quoted 8 lines
Quoted 3 lines
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Yes, this results in a big dump of messages. I haven't looked at gracefully handling these.
I'm not sure if it needs to be done in these set of patches though.
Ah, sure! I didn't meant you'd need to change the messenger to handle it (as I'm not even sure it's the messenger or the mons client that require changes). But I also don't think that this patchset can be merged without making sure we can handle a bad fsid correctly and without all this noise.
Cheers,
--
Luís
Quoted 76 lines
Quoted 70 lines
Cheers,
--
LuísQuoted 63 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +3
-3@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err; --2.27.0
Signature · 2 lines
Cheers, Venky
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 61 lines · Luis Henriques
Quoted 59 lines · Venky Shankar
Quoted 49 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msgQuoted 5 lines
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Yes, this results in a big dump of messages. I haven't looked at gracefully handling these.
Quoted 6 lines
Quoted 1 line
I'm not sure if it needs to be done in these set of patches though.
Ah, sure! I didn't meant you'd need to change the messenger to handle it (as I'm not even sure it's the messenger or the mons client that require changes). But I also don't think that this patchset can be merged without making sure we can handle a bad fsid correctly and without all this noise.
True. However, for most cases users really won't be filling in the fsid and the mount helper would fill the "correct" one automatically.
Quoted 82 lines
Cheers,
--
LuísQuoted 76 lines
Quoted 60 lines
Cheers,
--
LuísQuoted 54 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +2
-1@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;Quoted 10 lines
Quoted 9 lines
net/ceph/ceph_common.c +1
-2case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err; --2.27.0
--
Cheers,
Venky
Signature · 2 lines
Cheers, Venky
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 64 lines · Venky Shankar
Quoted 63 lines · Luis Henriques
Quoted 60 lines · Venky Shankar
Quoted 49 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msgQuoted 5 lines
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Yes, this results in a big dump of messages. I haven't looked at gracefully handling these.
I'm not sure if it needs to be done in these set of patches though.
Quoted 8 lines
Quoted 4 lines
Ah, sure! I didn't meant you'd need to change the messenger to handle it (as I'm not even sure it's the messenger or the mons client that require changes). But I also don't think that this patchset can be merged without making sure we can handle a bad fsid correctly and without all this noise.
True. However, for most cases users really won't be filling in the fsid and the mount helper would fill the "correct" one automatically.
Yes, but some of them may and I think we do need to handle this gracefully. Let's step back a moment and consider:
AIUI, the fsid is only here to disambiguate when you have multiple clusters. The kernel doesn't really care about this value at all. All it cares about is whether it's talking to the right mons (as evidenced by the fact that we don't pass the fsid in at mount time today).
So probably the right thing to do is to just return an error (-EINVAL?) to mount() when there is a mismatch between the fsid and the one in the maps. Is that possible?
Quoted 84 lines
Quoted 82 lines
Cheers,
--
LuísQuoted 76 lines
Quoted 60 lines
Cheers,
--
LuísQuoted 54 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +2
-1@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;Quoted 10 lines
Quoted 9 lines
net/ceph/ceph_common.c +1
-2case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err; --2.27.0
--
Cheers,
Venky
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v2 2/4] ceph: validate cluster FSID for new device syntax
Quoted 78 lines · Jeff Layton
Quoted 64 lines · Venky Shankar
Quoted 63 lines · Luis Henriques
Quoted 60 lines · Venky Shankar
Quoted 49 lines · Luis Henriques
Quoted 31 lines · Venky Shankar
The new device syntax requires the cluster FSID as part of the device string. Use this FSID to verify if it matches the cluster FSID we get back from the monitor, failing the mount on mismatch.
Also, rename
parse_fsid()toceph_parse_fsid()as it is too generic.Signed-off-by Venky Shankar <[email protected]>--- fs/ceph/super.c | 9 +++++++++ fs/ceph/super.h | 1 + include/linux/ceph/libceph.h | 1 + net/ceph/ceph_common.c | 5 +++-- 4 files changed, 14 insertions(+), 2 deletions(-)fs/ceph/super.c +3
-0@@ -268,6 +268,9 @@ static int ceph_parse_new_source(const char *dev_name, const char *dev_name_end,if (!fs_name_start)return invalfc (fc,"missing file system name" ); +if (ceph_parse_fsid (fsid_start, &fsopt->fsid)) +return invalfc (fc,"invalid fsid format" ); + ++fs_name_start;/* start of file system name */ fsopt->mds_namespace =kstrndup (fs_name_start, dev_name_end - fs_name_start,GFP_KERNEL );fs/ceph/super.c static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, · line 758
Quoted 6 lines
fs/ceph/super.c +3
-0@@ -750,6 +753,12 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, } opt =NULL ;/* fsc->client now owns this */ +/* help learn fsid */ +if (fsopt->new_dev_syntax) { +ceph_check_fsid (fsc->client, &fsopt->fsid);This call to
ceph_check_fsid()made me wonder what would happen if I use the wrong fsid with the new syntax. And the result is:[ 41.882334] libceph: mon0 (1)192.168.155.1:40594 session established [ 41.884537] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.885955] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.889313] libceph: bad fsid, had d52783e6-efc2-4dce-ad01-aa3272fa5f66 got 90bdb539-9d95-402e-8f23-b0e26cba8b1b [ 41.892578] libceph: osdc handle_map corrupt msgQuoted 5 lines
... followed by a msg dump.
I guess this means that manually setting the fsid requires changes to the messenger (I've only tested with v1) so that it gracefully handles this scenario.
Yes, this results in a big dump of messages. I haven't looked at gracefully handling these.
I'm not sure if it needs to be done in these set of patches though.
Quoted 8 lines
Quoted 4 lines
Ah, sure! I didn't meant you'd need to change the messenger to handle it (as I'm not even sure it's the messenger or the mons client that require changes). But I also don't think that this patchset can be merged without making sure we can handle a bad fsid correctly and without all this noise.
True. However, for most cases users really won't be filling in the fsid and the mount helper would fill the "correct" one automatically.
Yes, but some of them may and I think we do need to handle this gracefully. Let's step back a moment and consider:
Quoted 8 lines
AIUI, the fsid is only here to disambiguate when you have multiple clusters. The kernel doesn't really care about this value at all. All it cares about is whether it's talking to the right mons (as evidenced by the fact that we don't pass the fsid in at mount time today).
So probably the right thing to do is to just return an error (
-EINVAL?) tomount()when there is a mismatch between the fsid and the one in the maps. Is that possible?
Gracefully handling this is the correct way forward. With these changes, a fsid mismatch results in a big splat of message dumps and mount failure.
I haven't yet figured where (all) to plumb in the work to handle this, but looks doable.
Quoted 89 lines
Quoted 78 lines
Quoted 78 lines
Cheers,
--
LuísQuoted 72 lines
Quoted 70 lines
Cheers,
--
LuísQuoted 63 lines
fs/ceph/super.c +3
-0+ fsc->client->have_fsid = true; + } + fsc->client->extra_mon_dispatch = extra_mon_dispatch;ceph_set_opt (fsc->client,ABORT_ON_FULL );fs/ceph/super.h +1
-0@@ -99,6 +99,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +struct ceph_fsid fsid; };struct ceph_fs_client {include/linux/ceph/libceph.h +1
-0@@ -296,6 +296,7 @@ extern bool libceph_compatible(void *data);extern const char *ceph_msg_type_name (int type);extern int ceph_check_fsid (struct ceph_client *client,struct ceph_fsid *fsid);extern void *ceph_kvmalloc (size_t size,gfp_t flags); +extern int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid);struct fs_parameter ;struct fc_log ;net/ceph/ceph_common.c +3
-3@@ -217,7 +217,7 @@ void *ceph_kvmalloc(size_t size, gfp_t flags)return p; } -static int parse_fsid (const char *str,struct ceph_fsid *fsid) +int ceph_parse_fsid (const char *str,struct ceph_fsid *fsid) {int i =0 ;char tmp[3 ];@@ -247,6 +247,7 @@ static int parse_fsid(const char *str, struct ceph_fsid *fsid)dout ("parse_fsid ret %d got fsid %pU\n" , err, fsid);return err; } +EXPORT_SYMBOL (ceph_parse_fsid);/* * ceph options @@ -465,7 +466,7 @@ int ceph_parse_param(struct fs_parameter *param, struct ceph_options *opt,break ;case Opt_fsid: - err =parse_fsid (param->string, &opt->fsid); + err =ceph_parse_fsid (param->string, &opt->fsid);if (err) {error_plog (&log,"Failed to parse fsid: %d" , err);return err; --2.27.0
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
--
Cheers,
Venky--
Jeff Layton <[email protected]>
Signature · 2 lines
Cheers, Venky
PATCH v2 3/4 ceph: record updated mon_addr on remount
In reply to Venky ShankarNote that the new monitors are just shown in /proc/mounts. Ceph does not (re)connect to new monitors yet.
--- fs/ceph/super.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/ceph/super.c +6 -0
@@ -1255,6 +1255,12 @@ static int ceph_reconfigure_fc(struct fs_context *fc)else ceph_clear_mount_opt (fsc,ASYNC_DIROPS ); +if (strcmp (fsc->mount_options->mon_addr, fsopt->mon_addr)) { +kfree (fsc->mount_options->mon_addr); + fsc->mount_options->mon_addr = fsopt->mon_addr; + fsopt->mon_addr =NULL ; + } +sync_filesystem (fc->root->d_sb);return 0 ; }
Signature · 1 line
2.27.0
PATCH v2 4/4 doc: document new CephFS mount device syntax
In reply to Venky Shankar--- Documentation/filesystems/ceph.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)
Documentation/filesystems/ceph.rst +22 -3
@@ -82,7 +82,7 @@ Mount Syntax The basic mount syntax is:: - # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt + # mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]] You only need to specify a single monitor, as the client will get the full list when it connects. (However, if the monitor you specify
@@ -90,16 +90,35 @@ happens to be down, the mount won't succeed.) The port can be left off if the monitor is using the default. So if the monitor is at 1.2.3.4:: - # mount -t ceph 1.2.3.4:/ /mnt/ceph + # mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4 is sufficient. If /sbin/mount.ceph is installed, a hostname can be -used instead of an IP address. +used instead of an IP address and the cluster FSID can be left out +(as the mount helper will fill it in by reading the ceph configuration +file):: + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr +Multiple monitor addresses can be passed by separating each address with a slash (`/`):: + + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101 + +When using the mount helper, monitor address can be read from ceph +configuration file if available. Note that, the cluster FSID (passed as part +of the device string) is validated by checking it with the FSID reported by +the monitor. Mount Options ============= + mon_addr=ip_address[:port][/ip_address[:port]] + Monitor address to the cluster. This is used to bootstrap the + connection to the cluster. Once connection is established, the + monitor addresses in the monitor map are followed. + + fsid=cluster-id + FSID of the cluster + ip=A.B.C.D[:N] Specify the IP and/or port the client should bind to locally. There is normally not much reason to do this. If the IP is not
Signature · 1 line
2.27.0
Re: [PATCH v2 4/4] doc: document new CephFS mount device syntax
Quoted 11 lines · Venky Shankar
Signed-off-by Venky Shankar <[email protected]>--- Documentation/filesystems/ceph.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/filesystems/ceph.rst +0
-0
Documentation/filesystems/ceph.rst Mount Syntax · line 85
Quoted 5 lines
Documentation/filesystems/ceph.rst +1
-1@@ -82,7 +82,7 @@ Mount Syntax The basic mount syntax is:: - # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt + # mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]
Somewhat unrelated question to this patchset: can you specify the mons in the ceph.conf format? i.e. with v2/v1 syntax?
Quoted 26 lines
Documentation/filesystems/ceph.rst +13
-2You only need to specify a single monitor, as the client will get the full list when it connects. (However, if the monitor you specify@@ -90,16 +90,35 @@ happens to be down, the mount won't succeed.) The port can be left off if the monitor is using the default. So if the monitor is at 1.2.3.4:: - # mount -t ceph 1.2.3.4:/ /mnt/ceph + # mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4 is sufficient. If /sbin/mount.ceph is installed, a hostname can be -used instead of an IP address. +used instead of an IP address and the cluster FSID can be left out +(as the mount helper will fill it in by reading the ceph configuration +file):: + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr +Multiple monitor addresses can be passed by separating each address with a slash (`/`):: + + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101 + +When using the mount helper, monitor address can be read from ceph +configuration file if available. Note that, the cluster FSID (passed as part +of the device string) is validated by checking it with the FSID reported by +the monitor.
Documentation/filesystems/ceph.rst happens to be down, the mount won't succeed.) The port can be left · line 120
Quoted 10 lines
Documentation/filesystems/ceph.rst +7
-0Mount Options ============= + mon_addr=ip_address[:port][/ip_address[:port]] + Monitor address to the cluster. This is used to bootstrap the + connection to the cluster. Once connection is established, the + monitor addresses in the monitor map are followed. + + fsid=cluster-id + FSID of the cluster
Let's note it's the output of ceph fsid.
Quoted 6 lines
Documentation/filesystems/ceph.rst +0
-1ip=A.B.C.D[:N] Specify the IP and/or port the client should bind to locally. There is normally not much reason to do this. If the IP is not --2.27.0
Signature · 5 lines
Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Re: [PATCH v2 4/4] doc: document new CephFS mount device syntax
Quoted 14 lines · Patrick Donnelly
Quoted 12 lines · Venky Shankar
Signed-off-by Venky Shankar <[email protected]>--- Documentation/filesystems/ceph.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/filesystems/ceph.rst +0
-0@@ -82,7 +82,7 @@ Mount Syntax
Quoted 7 lines
Documentation/filesystems/ceph.rst Mount Syntax · line 85
Quoted 4 lines
Documentation/filesystems/ceph.rst +1
-1The basic mount syntax is:: - # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt + # mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]Somewhat unrelated question to this patchset: can you specify the mons in the ceph.conf format? i.e. with v2/v1 syntax?
The problem with that is the delimiter used is comma (",") which restricts passing it through the mount option.
Quoted 55 lines
Quoted 27 lines
Documentation/filesystems/ceph.rst +13
-2You only need to specify a single monitor, as the client will get the full list when it connects. (However, if the monitor you specify@@ -90,16 +90,35 @@ happens to be down, the mount won't succeed.) The port can be left off if the monitor is using the default. So if the monitor is at 1.2.3.4:: - # mount -t ceph 1.2.3.4:/ /mnt/ceph + # mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4 is sufficient. If /sbin/mount.ceph is installed, a hostname can be -used instead of an IP address. +used instead of an IP address and the cluster FSID can be left out +(as the mount helper will fill it in by reading the ceph configuration +file):: + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr +Multiple monitor addresses can be passed by separating each address with a slash (`/`):: + + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101 + +When using the mount helper, monitor address can be read from ceph +configuration file if available. Note that, the cluster FSID (passed as part +of the device string) is validated by checking it with the FSID reported by +the monitor.Documentation/filesystems/ceph.rst happens to be down, the mount won't succeed.) The port can be left · line 120
Quoted 10 lines
Documentation/filesystems/ceph.rst +7
-0Mount Options ============= + mon_addr=ip_address[:port][/ip_address[:port]] + Monitor address to the cluster. This is used to bootstrap the + connection to the cluster. Once connection is established, the + monitor addresses in the monitor map are followed. + + fsid=cluster-id + FSID of the clusterLet's note it's the output of
ceph fsid.Quoted 6 lines
Documentation/filesystems/ceph.rst +0
-1ip=A.B.C.D[:N] Specify the IP and/or port the client should bind to locally. There is normally not much reason to do this. If the IP is not --2.27.0
--
Patrick Donnelly, Ph.D.
He / Him / His
Principal Software Engineer
Red Hat Sunnyvale, CA
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Signature · 2 lines
Cheers, Venky
Re: [PATCH v2 4/4] doc: document new CephFS mount device syntax
Quoted 17 lines · Venky Shankar
Quoted 16 lines · Patrick Donnelly
Quoted 14 lines · Venky Shankar
Signed-off-by Venky Shankar <[email protected]>--- Documentation/filesystems/ceph.rst | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-)Documentation/filesystems/ceph.rst +0
-0@@ -82,7 +82,7 @@ Mount Syntax The basic mount syntax is::
Quoted 9 lines
Quoted 5 lines
Documentation/filesystems/ceph.rst Mount Syntax · line 85
Quoted 2 lines
Documentation/filesystems/ceph.rst +1
-1- # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt + # mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]Somewhat unrelated question to this patchset: can you specify the mons in the ceph.conf format? i.e. with v2/v1 syntax?
The problem with that is the delimiter used is comma (",") which restricts passing it through the mount option.
Quoted 1 line
Yeah. I don't see an alternative here. You'd have to escape the comma somehow. You'd also have to build an in-kernel parser for that format.
Doing this would be a project in and of itself, and it doesn't seem valuable. mount.ceph is mostly what's going to populate this anyway and for that we don't really care.
Quoted 56 lines
Quoted 54 lines
Quoted 26 lines
Documentation/filesystems/ceph.rst +13
-2You only need to specify a single monitor, as the client will get the full list when it connects. (However, if the monitor you specify@@ -90,16 +90,35 @@ happens to be down, the mount won't succeed.) The port can be left off if the monitor is using the default. So if the monitor is at 1.2.3.4:: - # mount -t ceph 1.2.3.4:/ /mnt/ceph + # mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4 is sufficient. If /sbin/mount.ceph is installed, a hostname can be -used instead of an IP address. +used instead of an IP address and the cluster FSID can be left out +(as the mount helper will fill it in by reading the ceph configuration +file):: + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr +Multiple monitor addresses can be passed by separating each address with a slash (`/`):: + + # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101 + +When using the mount helper, monitor address can be read from ceph +configuration file if available. Note that, the cluster FSID (passed as part +of the device string) is validated by checking it with the FSID reported by +the monitor.Documentation/filesystems/ceph.rst happens to be down, the mount won't succeed.) The port can be left · line 120
Quoted 10 lines
Documentation/filesystems/ceph.rst +7
-0Mount Options ============= + mon_addr=ip_address[:port][/ip_address[:port]] + Monitor address to the cluster. This is used to bootstrap the + connection to the cluster. Once connection is established, the + monitor addresses in the monitor map are followed. + + fsid=cluster-id + FSID of the clusterLet's note it's the output of
ceph fsid.Quoted 6 lines
Documentation/filesystems/ceph.rst +0
-1ip=A.B.C.D[:N] Specify the IP and/or port the client should bind to locally. There is normally not much reason to do this. If the IP is not --2.27.0
--
Patrick Donnelly, Ph.D.
He / Him / His
Principal Software Engineer
Red Hat Sunnyvale, CA
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 3 lines · Venky Shankar
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
The kernel is also backwards compatible too, right?
Signature · 5 lines
Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Quoted 7 lines · Patrick Donnelly
Quoted 3 lines · Venky Shankar
Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.
The kernel is also backwards compatible too, right?
Yes.
Quoted 8 lines
--
Patrick Donnelly, Ph.D.
He / Him / His
Principal Software Engineer
Red Hat Sunnyvale, CA
GPG: 19F28A586F808C2402351B93C3301A3E258DD79D
Signature · 2 lines
Cheers, Venky