Skip to contentSkip to conversation

ceph: use new mount device syntax

PATCH In review Venky Shankar 2 participants 8 messages Jun 4, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover ceph: use new mount device syntax

This series introduces changes Ceph File System mount device string. Old mount device syntax (source) has the following problems:

mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

The user space mount helper changes are here:

    http://github.com/ceph/ceph/pull/41334

Venky Shankar (3):
ceph: new device mount syntax
ceph: record updated mon_addr on remount
doc: document new CephFS mount device syntax

 Documentation/filesystems/ceph.rst | 16 +++++--
 fs/ceph/super.c                    | 75 +++++++++++++++++++++---------
 fs/ceph/super.h                    |  1 +
 3 files changed, 67 insertions(+), 25 deletions(-)
Signature · 1 line
2.27.0
#1 lore raw

PATCH 1/3 ceph: new device mount syntax

Old mount device syntax (source) has the following problems:

- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

---
 fs/ceph/super.c | 69 +++++++++++++++++++++++++++++++++----------------
 fs/ceph/super.h |  1 +
 2 files changed, 48 insertions(+), 22 deletions(-)
fs/ceph/super.c +47 -22
index 9b1b7f4cfdd4..e273eabb0397 100644
@@ -142,9 +142,9 @@ enum {
 	Opt_congestion_kb,
 	/* int args above */
 	Opt_snapdirname,
-	Opt_mds_namespace,
 	Opt_recover_session,
 	Opt_source,
+	Opt_mon_addr,
 	/* string args above */
 	Opt_dirstat,
 	Opt_rbytes,
@@ -184,7 +184,6 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
 	fsparam_flag_no	("fsc",				Opt_fscache), // fsc|nofsc
 	fsparam_string	("fsc",				Opt_fscache), // fsc=...
 	fsparam_flag_no ("ino32",			Opt_ino32),
-	fsparam_string	("mds_namespace",		Opt_mds_namespace),
 	fsparam_flag_no ("poolperm",			Opt_poolperm),
 	fsparam_flag_no ("quotadf",			Opt_quotadf),
 	fsparam_u32	("rasize",			Opt_rasize),
@@ -196,6 +195,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
 	fsparam_u32	("rsize",			Opt_rsize),
 	fsparam_string	("snapdirname",			Opt_snapdirname),
 	fsparam_string	("source",			Opt_source),
+	fsparam_string	("mon_addr",			Opt_mon_addr),
 	fsparam_u32	("wsize",			Opt_wsize),
 	fsparam_flag_no	("wsync",			Opt_wsync),
 	{}
@@ -227,12 +227,12 @@ static void canonicalize_path(char *path)
 }
 
 /*
- * Parse the source parameter.  Distinguish the server list from the path.
+ * Parse the source parameter.  Distinguish the device spec from the path.
  *
  * The source will look like:
- *     <server_spec>[,<server_spec>...]:[<path>]
+ *     <device_spec>=/<path>
  * where
- *     <server_spec> is <ip>[:<port>]
+ *     <device_spec> is name@fsname
  *     <path> is optional, but if present must begin with '/'
  */
 static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
@@ -240,12 +240,17 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
 	struct ceph_parse_opts_ctx *pctx = fc->fs_private;
 	struct ceph_mount_options *fsopt = pctx->opts;
 	char *dev_name = param->string, *dev_name_end;
-	int ret;
+	char *fs_name_start;
 
 	dout("%s '%s'\n", __func__, dev_name);
 	if (!dev_name || !*dev_name)
 		return invalfc(fc, "Empty source");
 
+	fs_name_start = strchr(dev_name, '@');
+	if (!fs_name_start)
+		return invalfc(fc, "Missing file system name");
+	++fs_name_start; /* start of file system name */
+
 	dev_name_end = strchr(dev_name, '/');
 	if (dev_name_end) {
 		/*
@@ -262,24 +267,42 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
 		dev_name_end = dev_name + strlen(dev_name);
 	}
 
-	dev_name_end--;		/* back up to ':' separator */
-	if (dev_name_end < dev_name || *dev_name_end != ':')
-		return invalfc(fc, "No path or : separator in source");
+	dev_name_end--;		/* back up to '=' separator */
+	if (dev_name_end < dev_name || *dev_name_end != '=')
+		return invalfc(fc, "No path or = separator in source");
 
 	dout("device name '%.*s'\n", (int)(dev_name_end - dev_name), dev_name);
-	if (fsopt->server_path)
-		dout("server path '%s'\n", fsopt->server_path);
 
-	ret = ceph_parse_mon_ips(param->string, dev_name_end - dev_name,
-				 pctx->copts, fc->log.log);
-	if (ret)
-		return ret;
+	fsopt->mds_namespace = kstrndup(fs_name_start,
+					dev_name_end - fs_name_start, GFP_KERNEL);
+	dout("file system (mds namespace) '%s'\n", fsopt->mds_namespace);
 
+	if (fsopt->server_path)
+		dout("server path '%s'\n", fsopt->server_path);
 	fc->source = param->string;
 	param->string = NULL;
 	return 0;
 }
 
+static int ceph_parse_mon_addr(struct fs_parameter *param,
+			       struct fs_context *fc)
+{
+	int r;
+	struct ceph_parse_opts_ctx *pctx = fc->fs_private;
+	struct ceph_mount_options *fsopt = pctx->opts;
+
+	kfree(fsopt->mon_addr);
+	fsopt->mon_addr = kstrdup(param->string, GFP_KERNEL);
+	if (!fsopt->mon_addr)
+		return -ENOMEM;
+
+	strreplace(param->string, '/', ',');
+	r = ceph_parse_mon_ips(param->string, strlen(param->string),
+			       pctx->copts, fc->log.log);
+	param->string = NULL;
+	return r;
+}
+
 static int ceph_parse_mount_param(struct fs_context *fc,
 				  struct fs_parameter *param)
 {
@@ -304,11 +327,6 @@ static int ceph_parse_mount_param(struct fs_context *fc,
 		fsopt->snapdir_name = param->string;
 		param->string = NULL;
 		break;
-	case Opt_mds_namespace:
-		kfree(fsopt->mds_namespace);
-		fsopt->mds_namespace = param->string;
-		param->string = NULL;
-		break;
 	case Opt_recover_session:
 		mode = result.uint_32;
 		if (mode == ceph_recover_session_no)
@@ -322,6 +340,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,
 		if (fc->source)
 			return invalfc(fc, "Multiple sources specified");
 		return ceph_parse_source(param, fc);
+	case Opt_mon_addr:
+		return ceph_parse_mon_addr(param, fc);
 	case Opt_wsize:
 		if (result.uint_32 < PAGE_SIZE ||
 		    result.uint_32 > CEPH_MAX_WRITE_SIZE)
@@ -473,6 +493,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)
 	kfree(args->mds_namespace);
 	kfree(args->server_path);
 	kfree(args->fscache_uniq);
+	kfree(args->mon_addr);
 	kfree(args);
 }
 
@@ -516,6 +537,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,
 	if (ret)
 		return ret;
 
+	ret = strcmp_null(fsopt1->mon_addr, fsopt2->mon_addr);
+	if (ret)
+		return ret;
+
 	return ceph_compare_options(new_opt, fsc->client);
 }
 
@@ -571,8 +596,8 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)
 	if ((fsopt->flags & CEPH_MOUNT_OPT_NOCOPYFROM) == 0)
 		seq_puts(m, ",copyfrom");
 
-	if (fsopt->mds_namespace)
-		seq_show_option(m, "mds_namespace", fsopt->mds_namespace);
+	if (fsopt->mon_addr)
+		seq_printf(m, ",mon_addr=%s", fsopt->mon_addr);
 
 	if (fsopt->flags & CEPH_MOUNT_OPT_CLEANRECOVER)
 		seq_show_option(m, "recover_session", "clean");
fs/ceph/super.h +1 -0
index db80d89556b1..ead73dfb8804 100644
@@ -97,6 +97,7 @@ struct ceph_mount_options {
 	char *mds_namespace;  /* default NULL */
 	char *server_path;    /* default NULL (means "/") */
 	char *fscache_uniq;   /* default NULL */
+	char *mon_addr;
 };
 
 struct ceph_fs_client {
Signature · 1 line
2.27.0
#2 lore raw

Re: [PATCH 1/3] ceph: new device mount syntax

Quoted 34 lines · Venky Shankar

Old mount device syntax (source) has the following problems:

- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

---
 fs/ceph/super.c | 69 +++++++++++++++++++++++++++++++++----------------
 fs/ceph/super.h |  1 +
 2 files changed, 48 insertions(+), 22 deletions(-)
fs/ceph/super.c +0 -0
index 9b1b7f4cfdd4..e273eabb0397 100644

fs/ceph/super.c enum { · line 145

Quoted 4 lines
fs/ceph/super.c +0 -1
@@ -142,9 +142,9 @@ enum {
 	Opt_congestion_kb,
 	/* int args above */
 	Opt_snapdirname,
-	Opt_mds_namespace,

I don't think we can just remove the old mds_namespace option. What if we have an old mount helper with the new kernel? It'll pass that in as an option and it wouldn't be recognized.

Quoted 172 lines
fs/ceph/super.c +47 -21
 	Opt_recover_session,
 	Opt_source,
+	Opt_mon_addr,
 	/* string args above */
 	Opt_dirstat,
 	Opt_rbytes,
@@ -184,7 +184,6 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
 	fsparam_flag_no	("fsc",				Opt_fscache), // fsc|nofsc
 	fsparam_string	("fsc",				Opt_fscache), // fsc=...
 	fsparam_flag_no ("ino32",			Opt_ino32),
-	fsparam_string	("mds_namespace",		Opt_mds_namespace),
 	fsparam_flag_no ("poolperm",			Opt_poolperm),
 	fsparam_flag_no ("quotadf",			Opt_quotadf),
 	fsparam_u32	("rasize",			Opt_rasize),
@@ -196,6 +195,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
 	fsparam_u32	("rsize",			Opt_rsize),
 	fsparam_string	("snapdirname",			Opt_snapdirname),
 	fsparam_string	("source",			Opt_source),
+	fsparam_string	("mon_addr",			Opt_mon_addr),
 	fsparam_u32	("wsize",			Opt_wsize),
 	fsparam_flag_no	("wsync",			Opt_wsync),
 	{}
@@ -227,12 +227,12 @@ static void canonicalize_path(char *path)
 }
 
 /*
- * Parse the source parameter.  Distinguish the server list from the path.
+ * Parse the source parameter.  Distinguish the device spec from the path.
  *
  * The source will look like:
- *     <server_spec>[,<server_spec>...]:[<path>]
+ *     <device_spec>=/<path>
  * where
- *     <server_spec> is <ip>[:<port>]
+ *     <device_spec> is name@fsname
  *     <path> is optional, but if present must begin with '/'
  */
 static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
@@ -240,12 +240,17 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
 	struct ceph_parse_opts_ctx *pctx = fc->fs_private;
 	struct ceph_mount_options *fsopt = pctx->opts;
 	char *dev_name = param->string, *dev_name_end;
-	int ret;
+	char *fs_name_start;
 
 	dout("%s '%s'\n", __func__, dev_name);
 	if (!dev_name || !*dev_name)
 		return invalfc(fc, "Empty source");
 
+	fs_name_start = strchr(dev_name, '@');
+	if (!fs_name_start)
+		return invalfc(fc, "Missing file system name");
+	++fs_name_start; /* start of file system name */
+
 	dev_name_end = strchr(dev_name, '/');
 	if (dev_name_end) {
 		/*
@@ -262,24 +267,42 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
 		dev_name_end = dev_name + strlen(dev_name);
 	}
 
-	dev_name_end--;		/* back up to ':' separator */
-	if (dev_name_end < dev_name || *dev_name_end != ':')
-		return invalfc(fc, "No path or : separator in source");
+	dev_name_end--;		/* back up to '=' separator */
+	if (dev_name_end < dev_name || *dev_name_end != '=')
+		return invalfc(fc, "No path or = separator in source");
 
 	dout("device name '%.*s'\n", (int)(dev_name_end - dev_name), dev_name);
-	if (fsopt->server_path)
-		dout("server path '%s'\n", fsopt->server_path);
 
-	ret = ceph_parse_mon_ips(param->string, dev_name_end - dev_name,
-				 pctx->copts, fc->log.log);
-	if (ret)
-		return ret;
+	fsopt->mds_namespace = kstrndup(fs_name_start,
+					dev_name_end - fs_name_start, GFP_KERNEL);
+	dout("file system (mds namespace) '%s'\n", fsopt->mds_namespace);
 
+	if (fsopt->server_path)
+		dout("server path '%s'\n", fsopt->server_path);
 	fc->source = param->string;
 	param->string = NULL;
 	return 0;
 }
 
+static int ceph_parse_mon_addr(struct fs_parameter *param,
+			       struct fs_context *fc)
+{
+	int r;
+	struct ceph_parse_opts_ctx *pctx = fc->fs_private;
+	struct ceph_mount_options *fsopt = pctx->opts;
+
+	kfree(fsopt->mon_addr);
+	fsopt->mon_addr = kstrdup(param->string, GFP_KERNEL);
+	if (!fsopt->mon_addr)
+		return -ENOMEM;
+
+	strreplace(param->string, '/', ',');
+	r = ceph_parse_mon_ips(param->string, strlen(param->string),
+			       pctx->copts, fc->log.log);
+	param->string = NULL;
+	return r;
+}
+
 static int ceph_parse_mount_param(struct fs_context *fc,
 				  struct fs_parameter *param)
 {
@@ -304,11 +327,6 @@ static int ceph_parse_mount_param(struct fs_context *fc,
 		fsopt->snapdir_name = param->string;
 		param->string = NULL;
 		break;
-	case Opt_mds_namespace:
-		kfree(fsopt->mds_namespace);
-		fsopt->mds_namespace = param->string;
-		param->string = NULL;
-		break;
 	case Opt_recover_session:
 		mode = result.uint_32;
 		if (mode == ceph_recover_session_no)
@@ -322,6 +340,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,
 		if (fc->source)
 			return invalfc(fc, "Multiple sources specified");
 		return ceph_parse_source(param, fc);
+	case Opt_mon_addr:
+		return ceph_parse_mon_addr(param, fc);
 	case Opt_wsize:
 		if (result.uint_32 < PAGE_SIZE ||
 		    result.uint_32 > CEPH_MAX_WRITE_SIZE)
@@ -473,6 +493,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)
 	kfree(args->mds_namespace);
 	kfree(args->server_path);
 	kfree(args->fscache_uniq);
+	kfree(args->mon_addr);
 	kfree(args);
 }
 
@@ -516,6 +537,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,
 	if (ret)
 		return ret;
 
+	ret = strcmp_null(fsopt1->mon_addr, fsopt2->mon_addr);
+	if (ret)
+		return ret;
+
 	return ceph_compare_options(new_opt, fsc->client);
 }
 
@@ -571,8 +596,8 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)
 	if ((fsopt->flags & CEPH_MOUNT_OPT_NOCOPYFROM) == 0)
 		seq_puts(m, ",copyfrom");
 
-	if (fsopt->mds_namespace)
-		seq_show_option(m, "mds_namespace", fsopt->mds_namespace);
+	if (fsopt->mon_addr)
+		seq_printf(m, ",mon_addr=%s", fsopt->mon_addr);
 
 	if (fsopt->flags & CEPH_MOUNT_OPT_CLEANRECOVER)
 		seq_show_option(m, "recover_session", "clean");
fs/ceph/super.h +1 -0
index db80d89556b1..ead73dfb8804 100644
@@ -97,6 +97,7 @@ struct ceph_mount_options {
 	char *mds_namespace;  /* default NULL */
 	char *server_path;    /* default NULL (means "/") */
 	char *fscache_uniq;   /* default NULL */
+	char *mon_addr;
 };
 
 struct ceph_fs_client {
Signature · 1 line
Jeff Layton <[email protected]>
#3 lore raw

Re: [PATCH 1/3] ceph: new device mount syntax

Quoted 31 lines · Jeff Layton
Quoted 29 lines · Venky Shankar

Old mount device syntax (source) has the following problems:

- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

---
 fs/ceph/super.c | 69 +++++++++++++++++++++++++++++++++----------------
 fs/ceph/super.h |  1 +
 2 files changed, 48 insertions(+), 22 deletions(-)
Quoted 13 lines

fs/ceph/super.c enum { · line 145

Quoted 9 lines
fs/ceph/super.c +0 -1
index 9b1b7f4cfdd4..e273eabb0397 100644
@@ -142,9 +142,9 @@ enum {
      Opt_congestion_kb,
      /* int args above */
      Opt_snapdirname,
-     Opt_mds_namespace,

I don't think we can just remove the old mds_namespace option. What if we have an old mount helper with the new kernel? It'll pass that in as an option and it wouldn't be recognized.

Ah, ok. I kind of assumed that old user space + new kernel need not be taken care of.

That means the new kernel should preserve old mount syntax for old user space helper.

Quoted 177 lines
Quoted 166 lines
fs/ceph/super.c +47 -21
      Opt_recover_session,
      Opt_source,
+     Opt_mon_addr,
      /* string args above */
      Opt_dirstat,
      Opt_rbytes,
@@ -184,7 +184,6 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
      fsparam_flag_no ("fsc",                         Opt_fscache), // fsc|nofsc
      fsparam_string  ("fsc",                         Opt_fscache), // fsc=...
      fsparam_flag_no ("ino32",                       Opt_ino32),
-     fsparam_string  ("mds_namespace",               Opt_mds_namespace),
      fsparam_flag_no ("poolperm",                    Opt_poolperm),
      fsparam_flag_no ("quotadf",                     Opt_quotadf),
      fsparam_u32     ("rasize",                      Opt_rasize),
@@ -196,6 +195,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
      fsparam_u32     ("rsize",                       Opt_rsize),
      fsparam_string  ("snapdirname",                 Opt_snapdirname),
      fsparam_string  ("source",                      Opt_source),
+     fsparam_string  ("mon_addr",                    Opt_mon_addr),
      fsparam_u32     ("wsize",                       Opt_wsize),
      fsparam_flag_no ("wsync",                       Opt_wsync),
      {}
@@ -227,12 +227,12 @@ static void canonicalize_path(char *path)
 }
 
 /*
- * Parse the source parameter.  Distinguish the server list from the path.
+ * Parse the source parameter.  Distinguish the device spec from the path.
  *
  * The source will look like:
- *     <server_spec>[,<server_spec>...]:[<path>]
+ *     <device_spec>=/<path>
  * where
- *     <server_spec> is <ip>[:<port>]
+ *     <device_spec> is name@fsname
  *     <path> is optional, but if present must begin with '/'
  */
 static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
@@ -240,12 +240,17 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
      struct ceph_parse_opts_ctx *pctx = fc->fs_private;
      struct ceph_mount_options *fsopt = pctx->opts;
      char *dev_name = param->string, *dev_name_end;
-     int ret;
+     char *fs_name_start;
 
      dout("%s '%s'\n", __func__, dev_name);
      if (!dev_name || !*dev_name)
              return invalfc(fc, "Empty source");
 
+     fs_name_start = strchr(dev_name, '@');
+     if (!fs_name_start)
+             return invalfc(fc, "Missing file system name");
+     ++fs_name_start; /* start of file system name */
+
      dev_name_end = strchr(dev_name, '/');
      if (dev_name_end) {
              /*
@@ -262,24 +267,42 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
              dev_name_end = dev_name + strlen(dev_name);
      }
 
-     dev_name_end--;         /* back up to ':' separator */
-     if (dev_name_end < dev_name || *dev_name_end != ':')
-             return invalfc(fc, "No path or : separator in source");
+     dev_name_end--;         /* back up to '=' separator */
+     if (dev_name_end < dev_name || *dev_name_end != '=')
+             return invalfc(fc, "No path or = separator in source");
 
      dout("device name '%.*s'\n", (int)(dev_name_end - dev_name), dev_name);
-     if (fsopt->server_path)
-             dout("server path '%s'\n", fsopt->server_path);
 
-     ret = ceph_parse_mon_ips(param->string, dev_name_end - dev_name,
-                              pctx->copts, fc->log.log);
-     if (ret)
-             return ret;
+     fsopt->mds_namespace = kstrndup(fs_name_start,
+                                     dev_name_end - fs_name_start, GFP_KERNEL);
+     dout("file system (mds namespace) '%s'\n", fsopt->mds_namespace);
 
+     if (fsopt->server_path)
+             dout("server path '%s'\n", fsopt->server_path);
      fc->source = param->string;
      param->string = NULL;
      return 0;
 }
 
+static int ceph_parse_mon_addr(struct fs_parameter *param,
+                            struct fs_context *fc)
+{
+     int r;
+     struct ceph_parse_opts_ctx *pctx = fc->fs_private;
+     struct ceph_mount_options *fsopt = pctx->opts;
+
+     kfree(fsopt->mon_addr);
+     fsopt->mon_addr = kstrdup(param->string, GFP_KERNEL);
+     if (!fsopt->mon_addr)
+             return -ENOMEM;
+
+     strreplace(param->string, '/', ',');
+     r = ceph_parse_mon_ips(param->string, strlen(param->string),
+                            pctx->copts, fc->log.log);
+     param->string = NULL;
+     return r;
+}
+
 static int ceph_parse_mount_param(struct fs_context *fc,
                                struct fs_parameter *param)
 {
@@ -304,11 +327,6 @@ static int ceph_parse_mount_param(struct fs_context *fc,
              fsopt->snapdir_name = param->string;
              param->string = NULL;
              break;
-     case Opt_mds_namespace:
-             kfree(fsopt->mds_namespace);
-             fsopt->mds_namespace = param->string;
-             param->string = NULL;
-             break;
      case Opt_recover_session:
              mode = result.uint_32;
              if (mode == ceph_recover_session_no)
@@ -322,6 +340,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,
              if (fc->source)
                      return invalfc(fc, "Multiple sources specified");
              return ceph_parse_source(param, fc);
+     case Opt_mon_addr:
+             return ceph_parse_mon_addr(param, fc);
      case Opt_wsize:
              if (result.uint_32 < PAGE_SIZE ||
                  result.uint_32 > CEPH_MAX_WRITE_SIZE)
@@ -473,6 +493,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)
      kfree(args->mds_namespace);
      kfree(args->server_path);
      kfree(args->fscache_uniq);
+     kfree(args->mon_addr);
      kfree(args);
 }
 
@@ -516,6 +537,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,
      if (ret)
              return ret;
 
+     ret = strcmp_null(fsopt1->mon_addr, fsopt2->mon_addr);
+     if (ret)
+             return ret;
+
      return ceph_compare_options(new_opt, fsc->client);
 }
 
@@ -571,8 +596,8 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)
      if ((fsopt->flags & CEPH_MOUNT_OPT_NOCOPYFROM) == 0)
              seq_puts(m, ",copyfrom");
 
-     if (fsopt->mds_namespace)
-             seq_show_option(m, "mds_namespace", fsopt->mds_namespace);
+     if (fsopt->mon_addr)
+             seq_printf(m, ",mon_addr=%s", fsopt->mon_addr);
 
      if (fsopt->flags & CEPH_MOUNT_OPT_CLEANRECOVER)
              seq_show_option(m, "recover_session", "clean");
fs/ceph/super.h +0 -0
index db80d89556b1..ead73dfb8804 100644

fs/ceph/super.h struct ceph_mount_options { · line 103

Quoted 7 lines
fs/ceph/super.h +1 -0
@@ -97,6 +97,7 @@ struct ceph_mount_options {
      char *mds_namespace;  /* default NULL */
      char *server_path;    /* default NULL (means "/") */
      char *fscache_uniq;   /* default NULL */
+     char *mon_addr;
 };
 
 struct ceph_fs_client {

--
Jeff Layton <[email protected]>

Signature · 2 lines
Cheers,
Venky
#4 lore raw

Re: [PATCH 1/3] ceph: new device mount syntax

Quoted 46 lines · Venky Shankar
Quoted 31 lines · Jeff Layton
Quoted 29 lines · Venky Shankar

Old mount device syntax (source) has the following problems:

- mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

- Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

---
 fs/ceph/super.c | 69 +++++++++++++++++++++++++++++++++----------------
 fs/ceph/super.h |  1 +
 2 files changed, 48 insertions(+), 22 deletions(-)
Quoted 13 lines

fs/ceph/super.c enum { · line 145

Quoted 9 lines
fs/ceph/super.c +0 -1
index 9b1b7f4cfdd4..e273eabb0397 100644
@@ -142,9 +142,9 @@ enum {
      Opt_congestion_kb,
      /* int args above */
      Opt_snapdirname,
-     Opt_mds_namespace,

I don't think we can just remove the old mds_namespace option. What if we have an old mount helper with the new kernel? It'll pass that in as an option and it wouldn't be recognized.

Quoted 7 lines

Ah, ok. I kind of assumed that old user space + new kernel need not be taken care of.

That means the new kernel should preserve old mount syntax for old user space helper.

Yeah. We have to allow for the entire matrix of old/new kernel with old/new mount helper.

Note that I don't think we need to worry about displaying the old syntax in /proc/mounts if someone mounted with it, but we do want kernels to accept both mount syntaxes, at least for now. Eventually, we should be able to deprecate the old syntax once we've dropped support pre-Quincy versions of the mount helper (2-3 years out or so).

Quoted 178 lines
Quoted 176 lines
Quoted 165 lines
fs/ceph/super.c +47 -21
      Opt_recover_session,
      Opt_source,
+     Opt_mon_addr,
      /* string args above */
      Opt_dirstat,
      Opt_rbytes,
@@ -184,7 +184,6 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
      fsparam_flag_no ("fsc",                         Opt_fscache), // fsc|nofsc
      fsparam_string  ("fsc",                         Opt_fscache), // fsc=...
      fsparam_flag_no ("ino32",                       Opt_ino32),
-     fsparam_string  ("mds_namespace",               Opt_mds_namespace),
      fsparam_flag_no ("poolperm",                    Opt_poolperm),
      fsparam_flag_no ("quotadf",                     Opt_quotadf),
      fsparam_u32     ("rasize",                      Opt_rasize),
@@ -196,6 +195,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {
      fsparam_u32     ("rsize",                       Opt_rsize),
      fsparam_string  ("snapdirname",                 Opt_snapdirname),
      fsparam_string  ("source",                      Opt_source),
+     fsparam_string  ("mon_addr",                    Opt_mon_addr),
      fsparam_u32     ("wsize",                       Opt_wsize),
      fsparam_flag_no ("wsync",                       Opt_wsync),
      {}
@@ -227,12 +227,12 @@ static void canonicalize_path(char *path)
 }
 
 /*
- * Parse the source parameter.  Distinguish the server list from the path.
+ * Parse the source parameter.  Distinguish the device spec from the path.
  *
  * The source will look like:
- *     <server_spec>[,<server_spec>...]:[<path>]
+ *     <device_spec>=/<path>
  * where
- *     <server_spec> is <ip>[:<port>]
+ *     <device_spec> is name@fsname
  *     <path> is optional, but if present must begin with '/'
  */
 static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
@@ -240,12 +240,17 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
      struct ceph_parse_opts_ctx *pctx = fc->fs_private;
      struct ceph_mount_options *fsopt = pctx->opts;
      char *dev_name = param->string, *dev_name_end;
-     int ret;
+     char *fs_name_start;
 
      dout("%s '%s'\n", __func__, dev_name);
      if (!dev_name || !*dev_name)
              return invalfc(fc, "Empty source");
 
+     fs_name_start = strchr(dev_name, '@');
+     if (!fs_name_start)
+             return invalfc(fc, "Missing file system name");
+     ++fs_name_start; /* start of file system name */
+
      dev_name_end = strchr(dev_name, '/');
      if (dev_name_end) {
              /*
@@ -262,24 +267,42 @@ static int ceph_parse_source(struct fs_parameter *param, struct fs_context *fc)
              dev_name_end = dev_name + strlen(dev_name);
      }
 
-     dev_name_end--;         /* back up to ':' separator */
-     if (dev_name_end < dev_name || *dev_name_end != ':')
-             return invalfc(fc, "No path or : separator in source");
+     dev_name_end--;         /* back up to '=' separator */
+     if (dev_name_end < dev_name || *dev_name_end != '=')
+             return invalfc(fc, "No path or = separator in source");
 
      dout("device name '%.*s'\n", (int)(dev_name_end - dev_name), dev_name);
-     if (fsopt->server_path)
-             dout("server path '%s'\n", fsopt->server_path);
 
-     ret = ceph_parse_mon_ips(param->string, dev_name_end - dev_name,
-                              pctx->copts, fc->log.log);
-     if (ret)
-             return ret;
+     fsopt->mds_namespace = kstrndup(fs_name_start,
+                                     dev_name_end - fs_name_start, GFP_KERNEL);
+     dout("file system (mds namespace) '%s'\n", fsopt->mds_namespace);
 
+     if (fsopt->server_path)
+             dout("server path '%s'\n", fsopt->server_path);
      fc->source = param->string;
      param->string = NULL;
      return 0;
 }
 
+static int ceph_parse_mon_addr(struct fs_parameter *param,
+                            struct fs_context *fc)
+{
+     int r;
+     struct ceph_parse_opts_ctx *pctx = fc->fs_private;
+     struct ceph_mount_options *fsopt = pctx->opts;
+
+     kfree(fsopt->mon_addr);
+     fsopt->mon_addr = kstrdup(param->string, GFP_KERNEL);
+     if (!fsopt->mon_addr)
+             return -ENOMEM;
+
+     strreplace(param->string, '/', ',');
+     r = ceph_parse_mon_ips(param->string, strlen(param->string),
+                            pctx->copts, fc->log.log);
+     param->string = NULL;
+     return r;
+}
+
 static int ceph_parse_mount_param(struct fs_context *fc,
                                struct fs_parameter *param)
 {
@@ -304,11 +327,6 @@ static int ceph_parse_mount_param(struct fs_context *fc,
              fsopt->snapdir_name = param->string;
              param->string = NULL;
              break;
-     case Opt_mds_namespace:
-             kfree(fsopt->mds_namespace);
-             fsopt->mds_namespace = param->string;
-             param->string = NULL;
-             break;
      case Opt_recover_session:
              mode = result.uint_32;
              if (mode == ceph_recover_session_no)
@@ -322,6 +340,8 @@ static int ceph_parse_mount_param(struct fs_context *fc,
              if (fc->source)
                      return invalfc(fc, "Multiple sources specified");
              return ceph_parse_source(param, fc);
+     case Opt_mon_addr:
+             return ceph_parse_mon_addr(param, fc);
      case Opt_wsize:
              if (result.uint_32 < PAGE_SIZE ||
                  result.uint_32 > CEPH_MAX_WRITE_SIZE)
@@ -473,6 +493,7 @@ static void destroy_mount_options(struct ceph_mount_options *args)
      kfree(args->mds_namespace);
      kfree(args->server_path);
      kfree(args->fscache_uniq);
+     kfree(args->mon_addr);
      kfree(args);
 }
 
@@ -516,6 +537,10 @@ static int compare_mount_options(struct ceph_mount_options *new_fsopt,
      if (ret)
              return ret;
 
+     ret = strcmp_null(fsopt1->mon_addr, fsopt2->mon_addr);
+     if (ret)
+             return ret;
+
      return ceph_compare_options(new_opt, fsc->client);
 }
 
@@ -571,8 +596,8 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)
      if ((fsopt->flags & CEPH_MOUNT_OPT_NOCOPYFROM) == 0)
              seq_puts(m, ",copyfrom");
 
-     if (fsopt->mds_namespace)
-             seq_show_option(m, "mds_namespace", fsopt->mds_namespace);
+     if (fsopt->mon_addr)
+             seq_printf(m, ",mon_addr=%s", fsopt->mon_addr);
 
      if (fsopt->flags & CEPH_MOUNT_OPT_CLEANRECOVER)
              seq_show_option(m, "recover_session", "clean");
fs/ceph/super.h +0 -0
index db80d89556b1..ead73dfb8804 100644

fs/ceph/super.h struct ceph_mount_options { · line 103

Quoted 7 lines
fs/ceph/super.h +1 -0
@@ -97,6 +97,7 @@ struct ceph_mount_options {
      char *mds_namespace;  /* default NULL */
      char *server_path;    /* default NULL (means "/") */
      char *fscache_uniq;   /* default NULL */
+     char *mon_addr;
 };
 
 struct ceph_fs_client {

--
Jeff Layton <[email protected]>

Signature · 1 line
Jeff Layton <[email protected]>
#5 lore raw

PATCH 2/3 ceph: record updated mon_addr on remount

In reply to Venky Shankar

Note that the new monitors are just shown in /proc/mounts. Ceph does not (re)connect to new monitors yet.

---
 fs/ceph/super.c | 6 ++++++
 1 file changed, 6 insertions(+)
fs/ceph/super.c +6 -0
index e273eabb0397..ccbd72e53998 100644
@@ -1169,6 +1169,12 @@ static int ceph_reconfigure_fc(struct fs_context *fc)
 	else
 		ceph_clear_mount_opt(fsc, ASYNC_DIROPS);
 
+	if (strcmp(fsc->mount_options->mon_addr, fsopt->mon_addr)) {
+		kfree(fsc->mount_options->mon_addr);
+		fsc->mount_options->mon_addr = kstrdup(fsopt->mon_addr,
+						       GFP_KERNEL);
+	}
+
 	sync_filesystem(fc->root->d_sb);
 	return 0;
 }
Signature · 1 line
2.27.0
#6 lore raw

PATCH 3/3 doc: document new CephFS mount device syntax

In reply to Venky Shankar
---
 Documentation/filesystems/ceph.rst | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
Documentation/filesystems/ceph.rst +13 -3
index 7d2ef4e27273..0e4ccefb7aea 100644
@@ -82,7 +82,7 @@ Mount Syntax
 
 The basic mount syntax is::
 
- # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt
+ # mount -t ceph user@fs_name=/[subdir] mnt -o mon_host=monip1[:port][/monip2[:port]]
 
 You only need to specify a single monitor, as the client will get the
 full list when it connects.  (However, if the monitor you specify
@@ -90,16 +90,26 @@ happens to be down, the mount won't succeed.)  The port can be left
 off if the monitor is using the default.  So if the monitor is at
 1.2.3.4::
 
- # mount -t ceph 1.2.3.4:/ /mnt/ceph
+ # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_host=1.2.3.4
 
 is sufficient.  If /sbin/mount.ceph is installed, a hostname can be
-used instead of an IP address.
+used instead of an IP address. Multiple monitor addresses can be
+passed by separating each address with a slash (`/`)::
 
+  # mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_host=192.168.1.100/192.168.1.101
 
+When using the mount helper, monitor address can be read from ceph
+configuration file if available.
 
 Mount Options
 =============
 
+  mon_host=ip_address[:port][/ip_address[:port]]
+	Monitor address to the cluster
+
+  fsid=cluster-id
+	FSID of the cluster
+
   ip=A.B.C.D[:N]
 	Specify the IP and/or port the client should bind to locally.
 	There is normally not much reason to do this.  If the IP is not
Signature · 1 line
2.27.0
#7 lore raw
In reply to Venky Shankar
Quoted 34 lines · Venky Shankar

This series introduces changes Ceph File System mount device string. Old mount device syntax (source) has the following problems:

mounts to the same cluster but with different fsnames and/or creds have identical device string which can confuse xfstests.

Userspace mount helper tool resolves monitor addresses and fill in mon addrs automatically, but that means the device shown in /proc/mounts is different than what was used for mounting.

New device syntax is as follows:

cephuser@mycephfs2=/path

Note, there is no "monitor address" in the device string. That gets passed in as mount option. This keeps the device string same when monitor addresses change (on remounts).

Also note that the userspace mount helper tool is backward compatible. I.e., the mount helper will fallback to using old syntax after trying to mount with the new syntax.

The user space mount helper changes are here:

    http://github.com/ceph/ceph/pull/41334

Venky Shankar (3):
ceph: new device mount syntax
ceph: record updated mon_addr on remount
doc: document new CephFS mount device syntax

Quoted 5 lines
 Documentation/filesystems/ceph.rst | 16 +++++--
 fs/ceph/super.c                    | 75 +++++++++++++++++++++---------
 fs/ceph/super.h                    |  1 +
 3 files changed, 67 insertions(+), 25 deletions(-)

Nice work, Venky!

This looks good at first glance. The only real issue I see is that we can't just deprecate the mds_namespace option. I think we have to keep that around for now for the case of a legacy mount helper.

I'll plan to pull down this and your userland patches and give them a spin later today.

Cheers,

Signature · 1 line
Jeff Layton <[email protected]>