Skip to contentSkip to conversation

possible NFSv2/v3 lock recovery enhancements

RFC PATCH In review Chuck Lever 2 participants 15 messages Apr 1, 2010 3 messages not in the archive

Subsystems (2)
NFS client NFS server
lore
Share & tools
lore

Chuck Lever

author
#0 lore raw

cover possible NFSv2/v3 lock recovery enhancements

For the purpose of discussion, here are 11 WIP patches. These have seen very little testing, and are presented here as a proof of concept only.

There are three main goals of this work:

1) Provide an API for an NFSv2/v3 server administrator to release stale locks

2) Provide an alternate API that doesn't depend on the existance of a particular network loopback interface to allow statd to perform NLM callbacks (ie statd won't have to care whether to use IPv4 or IPv6 loopback to make NLM downcalls)

3) Allow clients to send correct caller_name strings in NLMPROC_LOCK requests.

1 and 2 are met by implementing a set of directories under /sys that expose the kernel's nlm_host and nsm_handle cache. Each entry in the nsm_handle cache has a reboot attribute that would allow a user space process to trigger reboot recovery on the remote peer represented by that entry.

3 is met by adding a new NFSv2/v3 mount option, similar to clientaddr= for NFSv4 mounts, that allows the mount command to specify the caller_name string to use for lock requests on this mount point.

There is still a chicken-and-egg type problem for statd. These new /sys files are owned and readable by only root. The reboot attribute is owned and writable by only root. statd runs as "rpcuser" usually, so unless something special is done, it wouldn't be able to use a /sys file to trigger reboot recovery.

A possible way to address this is to add support for AF_LOCAL to the kernel. The kernel can create an RPC socket that statd can use to perform downcalls. AF_LOCAL can also be used in other cases, like when the kernel has to contact rpcbind, so kernel AF_LOCAL support wouldn't be a single-use type of thing.

---

Chuck Lever (11):
lockd: Allow mount option to specify caller_name
lockd: use sm_my_name for nsm_handle lookups
lockd: Keep my_name in nsm_handle
lockd: Add "reboot" attribute to nsm_handles
lockd: Refactor nlm_host_rebooted()
lockd: Add /sys/fs/lockd/nsm_handle/*
lockd: Add /sys/fs/lockd/mon
lockd: Add attributes to /sys/fs/lockd/hosts/*/
lockd: Add /sys/fs/lockd/hosts/*
lockd: Add /sys/fs/lockd/hosts/
lockd: Add /sys/fs/lockd

 fs/lockd/clntlock.c         |    3 
 fs/lockd/clntproc.c         |    6 -
 fs/lockd/host.c             |  319 +++++++++++++++++++++++++++++++++++++++++--
 fs/lockd/mon.c              |  253 ++++++++++++++++++++++++++++++++--
 fs/lockd/svc.c              |   31 ++++
 fs/lockd/svclock.c          |    2 
 fs/nfs/client.c             |   11 +
 fs/nfs/internal.h           |    1 
 fs/nfs/super.c              |    9 +
 include/linux/lockd/bind.h  |    1 
 include/linux/lockd/lockd.h |   19 ++-
 include/linux/nfs_fs_sb.h   |    5 +
 12 files changed, 620 insertions(+), 40 deletions(-)
Signature · 1 line
Chuck Lever

A message that is not in the archive (sent off-list or not yet received): 20100401183724.6395.60353.stgit-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]

author
#2 lore raw

PATCH 1/11 lockd: Add /sys/fs/lockd

Create the parent directory for all lockd-related sysfs entries. The directory is created when lockd.ko is loaded, and removed when it is unloaded.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/svc.c              |   16 +++++++++++++---
 include/linux/lockd/lockd.h |    1 +
 2 files changed, 14 insertions(+), 3 deletions(-)
fs/lockd/svc.c +13 -3
index 7d15051..94e0d4b 100644
@@ -76,6 +76,8 @@ static const int		nlm_port_min = 0, nlm_port_max = 65535;
 static struct ctl_table_header * nlm_sysctl_table;
 #endif
 
+struct kobject			*nlm_kobj;
+
 static unsigned long get_lockd_grace_period(void)
 {
 	/* Note: nlm_timeout should always be nonzero */

@@ -515,12 +517,19 @@ module_param(nlm_max_connections, uint, 0644);
 
 static int __init init_nlm(void)
 {
+	nlm_kobj = kobject_create_and_add("lockd", fs_kobj);
+	if (!nlm_kobj)
+		return -ENOMEM;
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
-	return nlm_sysctl_table ? 0 : -ENOMEM;
-#else
-	return 0;
+	if (!nlm_sysctl_table) {
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
 #endif
+
+	return 0;
 }
 
 static void __exit exit_nlm(void)

@@ -530,6 +539,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	kobject_put(nlm_kobj);
 }
 
 module_init(init_nlm);
include/linux/lockd/lockd.h +1 -0
index a34dea4..56c749e 100644
@@ -196,6 +196,7 @@ extern int			nlmsvc_grace_period;
 extern unsigned long		nlmsvc_timeout;
 extern int			nsm_use_hostnames;
 extern u32			nsm_local_state;
+extern struct kobject		*nlm_kobj;
 
 /*
  * Lockd client functions
author
#3 lore raw

PATCH 2/11 lockd: Add /sys/fs/lockd/hosts/

In reply to a message not in the archive

Add parent directory to contain directories representing entries in lockd's host cache.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c             |   46 +++++++++++++++++++++++++++++++++++++++++++
 fs/lockd/svc.c              |    7 +++++++
 include/linux/lockd/lockd.h |    2 ++
 3 files changed, 55 insertions(+), 0 deletions(-)
fs/lockd/host.c +46 -0
index 4600c20..3fbb0a5 100644
@@ -45,6 +45,10 @@ struct nlm_lookup_host_info {
 	const int		noresvport;	/* use non-priv port */
 };
 
+static struct kobject		*nlm_hosts_kobj;
+static struct kobject		*nlm_hosts_client_kobj;
+static struct kobject		*nlm_hosts_server_kobj;
+
 /*
  * Hash function must work well on big- and little-endian platforms
  */

@@ -567,3 +571,45 @@ nlm_gc_hosts(void)
 
 	next_gc = jiffies + NLM_HOST_COLLECT;
 }
+
+/**
+ * nlm_host_init - Called when lockd module is loaded
+ *
+ */
+int nlm_host_init(void)
+{
+	nlm_hosts_kobj = kobject_create_and_add("hosts", nlm_kobj);
+	if (nlm_hosts_kobj == NULL)
+		return -ENOMEM;
+
+	nlm_hosts_server_kobj = kobject_create_and_add("server", nlm_hosts_kobj);
+	if (nlm_hosts_server_kobj == NULL) {
+		kobject_put(nlm_hosts_kobj);
+		return -ENOMEM;
+	}
+
+	nlm_hosts_client_kobj = kobject_create_and_add("client", nlm_hosts_kobj);
+	if (nlm_hosts_client_kobj == NULL) {
+		kobject_put(nlm_hosts_server_kobj);
+		kobject_put(nlm_hosts_kobj);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+/**
+ * nlm_host_shutdown - Called when lockd module is about to be unloaded
+ *
+ */
+void nlm_host_shutdown(void)
+{
+	if (nrhosts != 0) {
+		printk(KERN_WARNING "NLM hosts remain at shutdown");
+		return;
+ 	}
+
+	kobject_put(nlm_hosts_client_kobj);
+	kobject_put(nlm_hosts_server_kobj);
+	kobject_put(nlm_hosts_kobj);
+}
fs/lockd/svc.c +7 -0
index 94e0d4b..acfb6d7 100644
@@ -521,9 +521,15 @@ static int __init init_nlm(void)
 	if (!nlm_kobj)
 		return -ENOMEM;
 
+	if (nlm_host_init()) {
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
 	if (!nlm_sysctl_table) {
+		nlm_host_shutdown();
 		kobject_put(nlm_kobj);
 		return -ENOMEM;
 	}

@@ -539,6 +545,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	nlm_host_shutdown();
 	kobject_put(nlm_kobj);
 }
include/linux/lockd/lockd.h +2 -0
index 56c749e..2d89cdb 100644
@@ -217,6 +217,8 @@ void		  nlmclnt_next_cookie(struct nlm_cookie *);
 /*
  * Host cache
  */
+int		  nlm_host_init(void);
+void		  nlm_host_shutdown(void);
 struct nlm_host  *nlmclnt_lookup_host(const struct sockaddr *sap,
 					const size_t salen,
 					const unsigned short protocol,
author
#4 lore raw

PATCH 3/11 lockd: Add /sys/fs/lockd/hosts/*

In reply to a message not in the archive

Populate the new hosts/ directory as nlm_hosts come and go. The new subdirectories are named after the IP address, transport protocol, and NLM protocol version of each entry in the nlm_hosts cache.

Introduce a simple-minded approach to managing the kref in the nlm_host's kobject. Unfortunately the existing nlm_host reference counting is more complex than other typical kref usage in the kernel, so we retain the h_count field. The kref is always 1 until the nlm_host is ready to be destroyed.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c             |   89 ++++++++++++++++++++++++++++++++++++++++++-
 include/linux/lockd/lockd.h |    1 
 2 files changed, 87 insertions(+), 3 deletions(-)
fs/lockd/host.c +86 -3
index 3fbb0a5..2414ef7 100644
@@ -30,6 +30,7 @@ static unsigned long		next_gc;
 static int			nrhosts;
 static DEFINE_MUTEX(nlm_host_mutex);
 
+static void			nlm_kobj_release(struct kobject *kobj);
 static void			nlm_gc_hosts(void);
 
 struct nlm_lookup_host_info {

@@ -49,6 +50,48 @@ static struct kobject		*nlm_hosts_kobj;
 static struct kobject		*nlm_hosts_client_kobj;
 static struct kobject		*nlm_hosts_server_kobj;
 
+struct nlm_host_attr {
+	struct attribute	attr;
+	ssize_t (*show)(struct nlm_host *host, char *buf);
+	ssize_t (*store)(struct nlm_host *host, const char *buf, size_t count);
+};
+
+static ssize_t nlm_kobj_show(struct kobject *kobj, struct attribute *attr,
+			     char *buf)
+{
+	struct nlm_host_attr *hattr = container_of(attr,
+						struct nlm_host_attr, attr);
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
+	if (hattr->show)
+		return hattr->show(host, buf);
+	return 0;
+}
+
+static ssize_t nlm_kobj_store(struct kobject *kobj, struct attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct nlm_host_attr *hattr = container_of(attr,
+						struct nlm_host_attr, attr);
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
+	if (hattr->store)
+		return hattr->store(host, buf, count);
+	return 0;
+}
+
+static struct sysfs_ops nlm_kobj_sysfs_ops = {
+	.show		= nlm_kobj_show,
+	.store		= nlm_kobj_store,
+};
+
+static struct attribute *nlm_kobj_attrs[] = {
+};
+
+static struct kobj_type nlm_host_ktype = {
+	.release		= nlm_kobj_release,
+	.sysfs_ops		= &nlm_kobj_sysfs_ops,
+	.default_attrs		= nlm_kobj_attrs,
+};
+
 /*
  * Hash function must work well on big- and little-endian platforms
  */

@@ -92,6 +135,35 @@ static unsigned int nlm_hash_address(const struct sockaddr *sap)
 }
 
 /*
+ * New kobjects are placed in either client/ or server/
+ * subdirectory and are named "ip-proto-vers".
+ */
+int nlm_add_kobject(struct nlm_host *host)
+{
+	struct kobject *parent;
+	char *proto_name;
+
+	parent = nlm_hosts_server_kobj;
+	if (host->h_server)
+		parent = nlm_hosts_client_kobj;
+
+	switch (host->h_proto) {
+	case IPPROTO_UDP:
+		proto_name = "udp";
+		break;
+	case IPPROTO_TCP:
+		proto_name = "tcp";
+		break;
+	default:
+		return -EPROTONOSUPPORT;
+	}
+
+	return kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
+				parent, "%s-%s-%u", host->h_addrbuf,
+				proto_name, host->h_version);
+}
+
+/*
  * Common host lookup routine for server & client
  */
 static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)

@@ -100,6 +172,7 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	struct hlist_node *pos;
 	struct nlm_host	*host;
 	struct nsm_handle *nsm = NULL;
+	int ret;
 
 	mutex_lock(&nlm_host_mutex);
 

@@ -191,6 +264,16 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	INIT_LIST_HEAD(&host->h_granted);
 	INIT_LIST_HEAD(&host->h_reclaim);
 
+	ret = nlm_add_kobject(host);
+	if (unlikely(ret != 0)) {
+		atomic_set(&host->h_count, 0);
+		kobject_put(&host->h_kobj);	/* releases nsm too */
+		dprintk("lockd: nlm_lookup_host "
+				"could not add nlm_host kobject\n");
+		host = NULL;
+		goto out;
+	}
+
 	nrhosts++;
 
 	dprintk("lockd: nlm_lookup_host created host %s\n",

@@ -204,9 +287,9 @@ out:
 /*
  * Destroy a host
  */
-static void
-nlm_destroy_host(struct nlm_host *host)
+static void nlm_kobj_release(struct kobject *kobj)
 {
+	struct nlm_host *host = container_of(kobj, struct nlm_host, h_kobj);
 	struct rpc_clnt	*clnt;
 
 	BUG_ON(!list_empty(&host->h_lockowners));

@@ -564,7 +647,7 @@ nlm_gc_hosts(void)
 			dprintk("lockd: delete host %s\n", host->h_name);
 			hlist_del_init(&host->h_hash);
 
-			nlm_destroy_host(host);
+			kobject_put(&host->h_kobj);
 			nrhosts--;
 		}
 	}
include/linux/lockd/lockd.h +1 -0
index 2d89cdb..bdb87a6 100644
@@ -39,6 +39,7 @@
  * Lockd host handle (used both by the client and server personality).
  */
 struct nlm_host {
+	struct kobject		h_kobj;
 	struct hlist_node	h_hash;		/* doubly linked list */
 	struct sockaddr_storage	h_addr;		/* peer address */
 	size_t			h_addrlen;
author
#5 lore raw

PATCH 4/11 lockd: Add attributes to /sys/fs/lockd/hosts/*/

In reply to a message not in the archive

Expose fields in each nlm_host cache entry via files under /sys/fs/lockd/hosts/{client,server}/*/.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c |  137 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 137 insertions(+), 0 deletions(-)
fs/lockd/host.c +137 -0
index 2414ef7..b15579b 100644
@@ -83,7 +83,144 @@ static struct sysfs_ops nlm_kobj_sysfs_ops = {
 	.store		= nlm_kobj_store,
 };
 
+static ssize_t address_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", host->h_addrbuf);
+}
+
+static ssize_t source_address_show(struct nlm_host *host, char *buf)
+{
+	struct sockaddr *sap = (struct sockaddr *)(char *)&host->h_srcaddr;
+	switch (sap->sa_family) {
+	case AF_UNSPEC:
+		return snprintf(buf, PAGE_SIZE, "unspecified address\n");
+	case AF_INET:
+		return snprintf(buf, PAGE_SIZE, "%pI4\n", sap);
+	case AF_INET6:
+		return snprintf(buf, PAGE_SIZE, "%pI6c\n", sap);
+	}
+	return snprintf(buf, PAGE_SIZE, "unrecognized address\n");
+}
+
+static ssize_t name_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n", host->h_name);
+}
+
+static ssize_t nlm_version_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_version);
+}
+
+static ssize_t transport_show(struct nlm_host *host, char *buf)
+{
+	switch (host->h_proto) {
+	case IPPROTO_UDP:
+		return snprintf(buf, PAGE_SIZE, "udp\n");
+	case IPPROTO_TCP:
+		return snprintf(buf, PAGE_SIZE, "tcp\n");
+	}
+	return snprintf(buf, PAGE_SIZE, "unknown\n");
+}
+
+static ssize_t reclaiming_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_reclaiming ? "in progress" : "no");
+}
+
+static ssize_t peer_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_server ? "client" : "server");
+}
+
+static ssize_t resvport_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_noresvport ? "no" : "yes");
+}
+
+static ssize_t in_use_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%s\n",
+				host->h_inuse ? "yes" : "no");
+}
+
+static ssize_t pseudo_state_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_state);
+}
+
+static ssize_t nsm_state_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_nsmstate);
+}
+
+static ssize_t pidcount_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%u\n", host->h_pidcount);
+}
+
+static ssize_t refcount_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%d\n", atomic_read(&host->h_count));
+}
+
+static ssize_t nextrebind_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%lu seconds \n",
+				(host->h_nextrebind - jiffies) / HZ);
+}
+
+static ssize_t expires_show(struct nlm_host *host, char *buf)
+{
+	return snprintf(buf, PAGE_SIZE, "%lu seconds\n",
+				(host->h_expires - jiffies) / HZ);
+}
+
+#ifdef __ATTR_RO
+#undef __ATTR_RO
+#endif
+
+#define __ATTR_RO(_name) { \
+	.attr	= { .name = __stringify(_name), .mode = S_IRUSR, }, \
+ 	.show	= _name##_show,                                 \
+}
+
+static struct nlm_host_attr address_attr	= __ATTR_RO(address);
+static struct nlm_host_attr source_address_attr	= __ATTR_RO(source_address);
+static struct nlm_host_attr name_attr		= __ATTR_RO(name);
+static struct nlm_host_attr nlm_version_attr	= __ATTR_RO(nlm_version);
+static struct nlm_host_attr transport_attr	= __ATTR_RO(transport);
+static struct nlm_host_attr reclaiming_attr	= __ATTR_RO(reclaiming);
+static struct nlm_host_attr peer_attr		= __ATTR_RO(peer);
+static struct nlm_host_attr resvport_attr	= __ATTR_RO(resvport);
+static struct nlm_host_attr in_use_attr		= __ATTR_RO(in_use);
+static struct nlm_host_attr pseudo_state_attr	= __ATTR_RO(pseudo_state);
+static struct nlm_host_attr nsm_state_attr	= __ATTR_RO(nsm_state);
+static struct nlm_host_attr pidcount_attr	= __ATTR_RO(pidcount);
+static struct nlm_host_attr refcount_attr	= __ATTR_RO(refcount);
+static struct nlm_host_attr nextrebind_attr	= __ATTR_RO(nextrebind);
+static struct nlm_host_attr expires_attr	= __ATTR_RO(expires);
+
 static struct attribute *nlm_kobj_attrs[] = {
+	&address_attr.attr,
+	&source_address_attr.attr,
+	&name_attr.attr,
+	&nlm_version_attr.attr,
+	&transport_attr.attr,
+	&reclaiming_attr.attr,
+	&peer_attr.attr,
+	&resvport_attr.attr,
+	&in_use_attr.attr,
+	&pseudo_state_attr.attr,
+	&nsm_state_attr.attr,
+	&pidcount_attr.attr,
+	&refcount_attr.attr,
+	&nextrebind_attr.attr,
+	&expires_attr.attr,
+	NULL,
 };
 
 static struct kobj_type nlm_host_ktype = {
author
#6 lore raw

PATCH 5/11 lockd: Add /sys/fs/lockd/mon

In reply to a message not in the archive

Add parent directory to contain entries representing each entry in lockd's nsm_handles cache.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/mon.c              |   27 +++++++++++++++++++++++++++
 fs/lockd/svc.c              |    8 ++++++++
 include/linux/lockd/lockd.h |    2 ++
 3 files changed, 37 insertions(+), 0 deletions(-)
fs/lockd/mon.c +27 -0
index f956651..77e4927 100644
@@ -56,6 +56,8 @@ static				DEFINE_SPINLOCK(nsm_lock);
 u32	__read_mostly		nsm_local_state;
 int	__read_mostly		nsm_use_hostnames;
 
+static struct kobject		*nsm_handles_kobj;
+
 static inline struct sockaddr *nsm_addr(const struct nsm_handle *nsm)
 {
 	return (struct sockaddr *)&nsm->sm_addr;

@@ -398,6 +400,31 @@ void nsm_release(struct nsm_handle *nsm)
 	}
 }
 
+/**
+ * nsm_init - Called when lockd module is loaded
+ *
+ */
+int nsm_init(void)
+{
+	nsm_handles_kobj = kobject_create_and_add("monitor", nlm_kobj);
+	if (nsm_handles_kobj == NULL)
+		return -ENOMEM;
+	return 0;
+}
+
+/**
+ * nsm_shutdown - Called when lockd module is about to be unloaded
+ *
+ */
+void nsm_shutdown(void)
+{
+	if (!list_empty(&nsm_handles)) {
+		printk(KERN_WARNING "NSM handles remain at shutdown");
+		return;
+	}
+	kobject_put(nsm_handles_kobj);
+}
+
 /*
  * XDR functions for NSM.
  *
fs/lockd/svc.c +8 -0
index acfb6d7..e715ed7 100644
@@ -526,9 +526,16 @@ static int __init init_nlm(void)
 		return -ENOMEM;
 	}
 
+	if (nsm_init()) {
+		nlm_host_shutdown();
+		kobject_put(nlm_kobj);
+		return -ENOMEM;
+	}
+
 #ifdef CONFIG_SYSCTL
 	nlm_sysctl_table = register_sysctl_table(nlm_sysctl_root);
 	if (!nlm_sysctl_table) {
+		nsm_shutdown();
 		nlm_host_shutdown();
 		kobject_put(nlm_kobj);
 		return -ENOMEM;

@@ -545,6 +552,7 @@ static void __exit exit_nlm(void)
 #ifdef CONFIG_SYSCTL
 	unregister_sysctl_table(nlm_sysctl_table);
 #endif
+	nsm_shutdown();
 	nlm_host_shutdown();
 	kobject_put(nlm_kobj);
 }
include/linux/lockd/lockd.h +2 -0
index bdb87a6..8622a28 100644
@@ -239,6 +239,8 @@ void		  nlm_host_rebooted(const struct nlm_reboot *);
 /*
  * Host monitoring
  */
+int		  nsm_init(void);
+void		  nsm_shutdown(void);
 int		  nsm_monitor(const struct nlm_host *host);
 void		  nsm_unmonitor(const struct nlm_host *host);
author
#7 lore raw

PATCH 6/11 lockd: Add /sys/fs/lockd/nsm_handle/*

In reply to a message not in the archive

Populate the new nsm_handle/ directory as nsm_handles come and go. The entries are named by their priv cookie, which is always guaranteed to be unique.

Like the nlm_hosts cache, the kref is fixed at one when the handle is created, and goes to zero only when the handle is ready to be destroyed.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c             |    7 ++
 fs/lockd/mon.c              |  166 +++++++++++++++++++++++++++++++++++++++++--
 include/linux/lockd/lockd.h |    1 
 3 files changed, 166 insertions(+), 8 deletions(-)
fs/lockd/host.c +6 -1
index b15579b..0a62d8d 100644
@@ -279,6 +279,7 @@ int nlm_add_kobject(struct nlm_host *host)
 {
 	struct kobject *parent;
 	char *proto_name;
+	int ret;
 
 	parent = nlm_hosts_server_kobj;
 	if (host->h_server)

@@ -295,9 +296,13 @@ int nlm_add_kobject(struct nlm_host *host)
 		return -EPROTONOSUPPORT;
 	}
 
-	return kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
+	ret = kobject_init_and_add(&host->h_kobj, &nlm_host_ktype,
 				parent, "%s-%s-%u", host->h_addrbuf,
 				proto_name, host->h_version);
+	if (likely(ret == 0))
+		ret = sysfs_create_link(&host->h_kobj,
+				&host->h_nsmhandle->sm_kobj, "nsm_handle");
+	return ret;
 }
 
 /*
fs/lockd/mon.c +159 -7

212 lines not on this page · raw message

include/linux/lockd/lockd.h +1 -0
index 8622a28..3d1c594 100644
@@ -78,6 +78,7 @@ struct nlm_host {
 #define NSM_ADDRBUF		((8 * 4 + 7) + (1 + 10) + 1)
 
 struct nsm_handle {
+	struct kobject		sm_kobj;
 	struct list_head	sm_link;
 	atomic_t		sm_count;
 	char			*sm_mon_name;
author
#8 lore raw

PATCH 7/11 lockd: Refactor nlm_host_rebooted()

In reply to a message not in the archive

We want to be able to call the lock recovery logic directly with an nsm_handle. Factor that piece out of nlm_host_rebooted().

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c             |   35 ++++++++++++++++++++++++-----------
 include/linux/lockd/lockd.h |    2 ++
 2 files changed, 26 insertions(+), 11 deletions(-)
fs/lockd/host.c +24 -11
index 0a62d8d..001d05b 100644
@@ -669,26 +669,39 @@ void nlm_release_host(struct nlm_host *host)
  */
 void nlm_host_rebooted(const struct nlm_reboot *info)
 {
-	struct hlist_head *chain;
-	struct hlist_node *pos;
 	struct nsm_handle *nsm;
-	struct nlm_host	*host;
 
 	nsm = nsm_reboot_lookup(info);
 	if (unlikely(nsm == NULL))
 		return;
 
-	/* Mark all hosts tied to this NSM state as having rebooted.
-	 * We run the loop repeatedly, because we drop the host table
-	 * lock for this.
-	 * To avoid processing a host several times, we match the nsmstate.
-	 */
-again:	mutex_lock(&nlm_host_mutex);
+	nlm_host_rebooted_nsm(nsm, info->state);
+}
+
+/**
+ * nlm_host_rebooted_nsm - start reboot recovery on remote peer
+ * @nsm: pointer to NSM handle for rebooting peer
+ * @nsmstate: new NSM state number of rebooting peer
+ *
+ * Mark all hosts tied to this NSM state as having rebooted.  We run
+ * the loop repeatedly, because we drop the host table lock for this.
+ *
+ * To avoid processing a host several times, we match the nsmstate.
+ */
+void nlm_host_rebooted_nsm(const struct nsm_handle *nsm, const int nsmstate)
+{
+	struct hlist_head *chain;
+	struct hlist_node *pos;
+	struct nlm_host	*host;
+
+again:
+	mutex_lock(&nlm_host_mutex);
+
 	for (chain = nlm_hosts; chain < nlm_hosts + NLM_HOST_NRHASH; ++chain) {
 		hlist_for_each_entry(host, pos, chain, h_hash) {
 			if (host->h_nsmhandle == nsm
-			 && host->h_nsmstate != info->state) {
-				host->h_nsmstate = info->state;
+			 && host->h_nsmstate != nsmstate) {
+				host->h_nsmstate = nsmstate;
 				host->h_state++;
 
 				nlm_get_host(host);
include/linux/lockd/lockd.h +2 -0
index 3d1c594..3c20114 100644
@@ -235,6 +235,8 @@ void		  nlm_rebind_host(struct nlm_host *);
 struct nlm_host * nlm_get_host(struct nlm_host *);
 void		  nlm_release_host(struct nlm_host *);
 void		  nlm_shutdown_hosts(void);
+void		  nlm_host_rebooted_nsm(const struct nsm_handle *nsm,
+					const int nsmstate);
 void		  nlm_host_rebooted(const struct nlm_reboot *);
 
 /*
author
#9 lore raw

PATCH 8/11 lockd: Add "reboot" attribute to nsm_handles

In reply to a message not in the archive

There can be multiple nlm_host cache entries associated with one remote peer, but there is only one nsm_handle per peer. So the right place to add an API to trigger lock recovery for a particular remote peer is with the nsm_handle cache entry for that remote.

Add a "reboot" attribute to each subdirectory in nsm_handles/.

Writing the new NSM state into the peer's nsm_handles "reboot" attribute invokes the kernel's lock recovery for that peer. A user space tool, for example, could be constructed so an administrator can remove stale locks on an NFS server.

Note that this new API can replace statd's NLM downcalls as well. Instead of calling a specified RPC program/version/procedure with the new state number and the priv cookie, statd can walk through /sys/fs/lockd looking for a directory named after the priv cookie, and write the new state number into its "reboot" attribute.

Using this API instead of an RPC downcall would eliminate one lockd dependency on IPv4 loopback.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/mon.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)
fs/lockd/mon.c +20 -0
index dc185e1..690c236 100644
@@ -147,6 +147,23 @@ static ssize_t refcount_show(struct nsm_handle *nsm, char *buf)
 			atomic_read(&nsm->sm_count));
 }
 
+static ssize_t reboot_show(struct nsm_handle *nsm, char *buf)
+{
+	return 0;
+}
+
+static ssize_t reboot_store(struct nsm_handle *nsm, 
+				const char *buf, size_t count)
+{
+	int state;
+
+	if (sscanf(buf, "%d", &state) != 1)
+		return -EINVAL;
+
+	nlm_host_rebooted_nsm(nsm, state);
+	return count;
+}
+
 #ifdef __ATTR_RO
 #undef __ATTR_RO
 #endif

@@ -163,6 +180,8 @@ static struct nsm_handle_attr rpcparms_attr	= __ATTR_RO(rpcparms);
 static struct nsm_handle_attr status_attr	= __ATTR_RO(status);
 static struct nsm_handle_attr sticky_attr	= __ATTR_RO(sticky);
 static struct nsm_handle_attr refcount_attr	= __ATTR_RO(refcount);
+static struct nsm_handle_attr reboot_attr	= __ATTR(reboot,
+			S_IWUSR | S_IRUSR, reboot_show, reboot_store);
 
 static struct attribute *nsm_kobj_attrs[] = {
 	&mon_name_attr.attr,

@@ -172,6 +191,7 @@ static struct attribute *nsm_kobj_attrs[] = {
 	&status_attr.attr,
 	&sticky_attr.attr,
 	&refcount_attr.attr,
+	&reboot_attr.attr,
 	NULL,
 };
author
#10 lore raw

PATCH 9/11 lockd: Keep my_name in nsm_handle

In reply to a message not in the archive

lockd assumes everywhere that my_name is always utsname->nodename. If we want to use multiple my_names (say, for multi-homed support) then we need to get the my_name value from one place, and ensure that value is also communicated consistently to statd.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/clntproc.c         |    6 +++---
 fs/lockd/mon.c              |   20 ++++++++++++++++----
 fs/lockd/svclock.c          |    2 +-
 include/linux/lockd/lockd.h |    6 ++++++
 4 files changed, 26 insertions(+), 8 deletions(-)
fs/lockd/clntproc.c +3 -3
index c81249f..2051515 100644
@@ -12,7 +12,6 @@
 #include <linux/errno.h>
 #include <linux/fs.h>
 #include <linux/nfs_fs.h>
-#include <linux/utsname.h>
 #include <linux/freezer.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/svc.h>

@@ -125,14 +124,15 @@ static void nlmclnt_setlockargs(struct nlm_rqst *req, struct file_lock *fl)
 {
 	struct nlm_args	*argp = &req->a_args;
 	struct nlm_lock	*lock = &argp->lock;
+	char *my_name = nsm_my_name(req);
 
 	nlmclnt_next_cookie(&argp->cookie);
 	memcpy(&lock->fh, NFS_FH(fl->fl_file->f_path.dentry->d_inode), sizeof(struct nfs_fh));
-	lock->caller  = utsname()->nodename;
+	lock->caller  = my_name;
 	lock->oh.data = req->a_owner;
 	lock->oh.len  = snprintf(req->a_owner, sizeof(req->a_owner), "%u@%s",
 				(unsigned int)fl->fl_u.nfs_fl.owner->pid,
-				utsname()->nodename);
+				my_name);
 	lock->svid = fl->fl_u.nfs_fl.owner->pid;
 	lock->fl.fl_start = fl->fl_start;
 	lock->fl.fl_end = fl->fl_end;
fs/lockd/mon.c +16 -4
index 690c236..e470038 100644
@@ -39,6 +39,7 @@ struct nsm_args {
 	u32			proc;
 
 	char			*mon_name;
+	char			*my_name;
 };
 
 struct nsm_res {

@@ -93,6 +94,8 @@ static void nsm_kobj_release(struct kobject *kobj)
 
 	dprintk("lockd: destroyed nsm_handle for %s (%s)\n",
 			nsm->sm_name, nsm->sm_addrbuf);
+
+	kfree(nsm->sm_my_name);
 	kfree(nsm);
 }
 

@@ -115,7 +118,7 @@ static ssize_t mon_name_show(struct nsm_handle *nsm, char *buf)
 
 static ssize_t my_name_show(struct nsm_handle *nsm, char *buf)
 {
-	return snprintf(buf, PAGE_SIZE, "%s\n", utsname()->nodename);
+	return snprintf(buf, PAGE_SIZE, "%s\n", nsm->sm_my_name);
 }
 
 static ssize_t address_show(struct nsm_handle *nsm, char *buf)

@@ -236,6 +239,7 @@ static int nsm_mon_unmon(struct nsm_handle *nsm, u32 proc, struct nsm_res *res)
 		.vers		= 3,
 		.proc		= NLMPROC_NSM_NOTIFY,
 		.mon_name	= nsm->sm_mon_name,
+		.my_name	= nsm->sm_my_name,
 	};
 	struct rpc_message msg = {
 		.rpc_argp	= &args,

@@ -404,7 +408,8 @@ static void nsm_init_private(struct nsm_handle *nsm)
 static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 					    const size_t salen,
 					    const char *hostname,
-					    const size_t hostname_len)
+					    const size_t hostname_len,
+					    const char *my_name)
 {
 	char *p, buf[SM_PRIV_SIZE * 2 + 1];
 	struct nsm_handle *new;

@@ -415,6 +420,12 @@ static struct nsm_handle *nsm_create_handle(const struct sockaddr *sap,
 	if (unlikely(new == NULL))
 		return NULL;
 
+	new->sm_my_name = kstrdup(my_name, GFP_KERNEL);
+	if (unlikely(new->sm_my_name == NULL)) {
+		kfree(new);
+		return NULL;
+	}
+
 	atomic_set(&new->sm_count, 1);
 	new->sm_name = (char *)(new + 1);
 	memcpy(nsm_addr(new), sap, salen);

@@ -515,7 +526,8 @@ retry:
 
 	spin_unlock(&nsm_lock);
 
-	new = nsm_create_handle(sap, salen, hostname, hostname_len);
+	new = nsm_create_handle(sap, salen, hostname, hostname_len,
+						init_utsname()->nodename);
 	if (unlikely(new == NULL))
 		return NULL;
 	goto retry;

@@ -637,7 +649,7 @@ static int encode_my_id(struct xdr_stream *xdr, const struct nsm_args *argp)
 	int status;
 	__be32 *p;
 
-	status = encode_nsm_string(xdr, utsname()->nodename);
+	status = encode_nsm_string(xdr, argp->my_name);
 	if (unlikely(status != 0))
 		return status;
 	p = xdr_reserve_space(xdr, 3 * sizeof(u32));
fs/lockd/svclock.c +1 -1
index d100179..938a9e4 100644
@@ -304,7 +304,7 @@ static int nlmsvc_setgrantargs(struct nlm_rqst *call, struct nlm_lock *lock)
 {
 	locks_copy_lock(&call->a_args.lock.fl, &lock->fl);
 	memcpy(&call->a_args.lock.fh, &lock->fh, sizeof(call->a_args.lock.fh));
-	call->a_args.lock.caller = utsname()->nodename;
+	call->a_args.lock.caller = nsm_my_name(call);
 	call->a_args.lock.oh.len = lock->oh.len;
 
 	/* set default data area */
include/linux/lockd/lockd.h +6 -0
index 3c20114..9e5da26 100644
@@ -82,6 +82,7 @@ struct nsm_handle {
 	struct list_head	sm_link;
 	atomic_t		sm_count;
 	char			*sm_mon_name;
+	char			*sm_my_name;
 	char			*sm_name;
 	struct sockaddr_storage	sm_addr;
 	size_t			sm_addrlen;

@@ -254,6 +255,11 @@ struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 struct nsm_handle *nsm_reboot_lookup(const struct nlm_reboot *info);
 void		  nsm_release(struct nsm_handle *nsm);
 
+static inline char *nsm_my_name(const struct nlm_rqst *req)
+{
+	return (char *)req->a_host->h_nsmhandle->sm_my_name;
+}
+
 /*
  * This is used in garbage collection and resource reclaim
  * A return value != 0 means destroy the lock/block/share
author
#11 lore raw

PATCH 10/11 lockd: use sm_my_name for nsm_handle lookups

In reply to a message not in the archive

Have nsm_get_handle() distinguish between two nsm_handles that are the same in every way except their sm_my_name field is different.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/host.c             |    2 +-
 fs/lockd/mon.c              |   28 +++++++++++++++++++---------
 include/linux/lockd/lockd.h |    3 ++-
 3 files changed, 22 insertions(+), 11 deletions(-)
fs/lockd/host.c +1 -1
index 001d05b..7d3cd2e 100644
@@ -366,7 +366,7 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	else {
 		host = NULL;
 		nsm = nsm_get_handle(ni->sap, ni->salen,
-					ni->hostname, ni->hostname_len);
+				ni->hostname, ni->hostname_len, NULL);
 		if (!nsm) {
 			dprintk("lockd: nlm_lookup_host failed; "
 				"no nsm handle\n");
fs/lockd/mon.c +19 -9
index e470038..d4b9cb3 100644
@@ -16,6 +16,7 @@
 #include <linux/sunrpc/svc.h>
 #include <linux/lockd/lockd.h>
 
+#include <asm/string.h>
 #include <asm/unaligned.h>
 
 #define NLMDBG_FACILITY		NLMDBG_MONITOR

@@ -344,23 +345,27 @@ void nsm_unmonitor(const struct nlm_host *host)
 }
 
 static struct nsm_handle *nsm_lookup_hostname(const char *hostname,
-					      const size_t len)
+					      const size_t len,
+					      const char *my_name)
 {
 	struct nsm_handle *nsm;
 
 	list_for_each_entry(nsm, &nsm_handles, sm_link)
 		if (strlen(nsm->sm_name) == len &&
-		    memcmp(nsm->sm_name, hostname, len) == 0)
+		    memcmp(nsm->sm_name, hostname, len) == 0 &&
+		    strcmp(my_name, nsm->sm_my_name) == 0)
 			return nsm;
 	return NULL;
 }
 
-static struct nsm_handle *nsm_lookup_addr(const struct sockaddr *sap)
+static struct nsm_handle *nsm_lookup_addr(const struct sockaddr *sap,
+					  const char *my_name)
 {
 	struct nsm_handle *nsm;
 
 	list_for_each_entry(nsm, &nsm_handles, sm_link)
-		if (rpc_cmp_addr(nsm_addr(nsm), sap))
+		if (rpc_cmp_addr(nsm_addr(nsm), sap) &&
+		    strcmp(my_name, nsm->sm_my_name) == 0)
 			return nsm;
 	return NULL;
 }

@@ -474,6 +479,8 @@ out:
  * @salen: length of socket address
  * @hostname: pointer to C string containing hostname to find
  * @hostname_len: length of C string
+ * @my_name: pointer to '\0'-terminated C string containing
+ *			hostname of local system, or NULL
  *
  * Behavior is modulated by the global nsm_use_hostnames variable.
  *

@@ -484,7 +491,8 @@ out:
  */
 struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 				  const size_t salen, const char *hostname,
-				  const size_t hostname_len)
+				  const size_t hostname_len,
+				  const char *my_name)
 {
 	struct nsm_handle *cached, *new = NULL;
 

@@ -497,13 +505,16 @@ struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 		return NULL;
 	}
 
+	if (my_name == NULL)
+		my_name = (const char *)init_utsname()->nodename;
+
 retry:
 	spin_lock(&nsm_lock);
 
 	if (nsm_use_hostnames && hostname != NULL)
-		cached = nsm_lookup_hostname(hostname, hostname_len);
+		cached = nsm_lookup_hostname(hostname, hostname_len, my_name);
 	else
-		cached = nsm_lookup_addr(sap);
+		cached = nsm_lookup_addr(sap, my_name);
 
 	if (cached != NULL) {
 		atomic_inc(&cached->sm_count);

@@ -526,8 +537,7 @@ retry:
 
 	spin_unlock(&nsm_lock);
 
-	new = nsm_create_handle(sap, salen, hostname, hostname_len,
-						init_utsname()->nodename);
+	new = nsm_create_handle(sap, salen, hostname, hostname_len, my_name);
 	if (unlikely(new == NULL))
 		return NULL;
 	goto retry;
include/linux/lockd/lockd.h +2 -1
index 9e5da26..a19bdde 100644
@@ -251,7 +251,8 @@ void		  nsm_unmonitor(const struct nlm_host *host);
 struct nsm_handle *nsm_get_handle(const struct sockaddr *sap,
 					const size_t salen,
 					const char *hostname,
-					const size_t hostname_len);
+					const size_t hostname_len,
+					const char *my_name);
 struct nsm_handle *nsm_reboot_lookup(const struct nlm_reboot *info);
 void		  nsm_release(struct nsm_handle *nsm);
author
#12 lore raw

PATCH 11/11 lockd: Allow mount option to specify caller_name

In reply to a message not in the archive

NLMPROC_LOCK requests have a "caller_name" argument which is supposed to contain the hostname the server uses to call the client back. Linux simply stuffs the system's utsname in this field, but this is not always the correct choice. For example:

o If an unqualified hostname is used for the client's utsname, it could be ambiguous when the server tries to resolve it
o If the client's actual hostname is determined by DHCP, it may not match its utsname
o If the NFS mount was done in a network namespace, the namespace name won't match the client's utsname
o If the client has multiple network interfaces, it should provide a hostname that matches the source address used to contact the server

In all of these cases, user space can determine the correct value of the caller_name argument at mount time.

So, add a mount option that allows user space to specify the value of the caller_name argument of NLMPROC_LOCK requests. If not specified, the kernel continues to use the init utsname, as before.

Signed-off-by Chuck Lever <[email protected]>

---

 fs/lockd/clntlock.c         |    3 ++-
 fs/lockd/host.c             |    7 ++++++-
 fs/nfs/client.c             |   11 +++++++++++
 fs/nfs/internal.h           |    1 +
 fs/nfs/super.c              |    9 +++++++++
 include/linux/lockd/bind.h  |    1 +
 include/linux/lockd/lockd.h |    1 +
 include/linux/nfs_fs_sb.h   |    5 +++++
 8 files changed, 36 insertions(+), 2 deletions(-)
fs/lockd/clntlock.c +2 -1
index fc9032d..0a0d98a 100644
@@ -61,7 +61,8 @@ struct nlm_host *nlmclnt_init(const struct nlmclnt_initdata *nlm_init)
 
 	host = nlmclnt_lookup_host(nlm_init->address, nlm_init->addrlen,
 				   nlm_init->protocol, nlm_version,
-				   nlm_init->hostname, nlm_init->noresvport);
+				   nlm_init->myname, nlm_init->hostname,
+				   nlm_init->noresvport);
 	if (host == NULL) {
 		lockd_down();
 		return ERR_PTR(-ENOLCK);
fs/lockd/host.c +6 -1
index 7d3cd2e..c23bf0d 100644
@@ -39,6 +39,7 @@ struct nlm_lookup_host_info {
 	const size_t		salen;		/* it's length */
 	const unsigned short	protocol;	/* transport to search for*/
 	const u32		version;	/* NLM version to search for */
+	const char		*my_name;	/* local hostname */
 	const char		*hostname;	/* remote's hostname */
 	const size_t		hostname_len;	/* it's length */
 	const struct sockaddr	*src_sap;	/* our address (optional) */

@@ -366,7 +367,8 @@ static struct nlm_host *nlm_lookup_host(struct nlm_lookup_host_info *ni)
 	else {
 		host = NULL;
 		nsm = nsm_get_handle(ni->sap, ni->salen,
-				ni->hostname, ni->hostname_len, NULL);
+				ni->hostname, ni->hostname_len,
+				ni->my_name);
 		if (!nsm) {
 			dprintk("lockd: nlm_lookup_host failed; "
 				"no nsm handle\n");

@@ -453,6 +455,7 @@ static void nlm_kobj_release(struct kobject *kobj)
  * @protocol: transport protocol to use
  * @version: NLM protocol version
  * @hostname: '\0'-terminated hostname of server
+ * @myname: '\0'-terminated hostname of local system
  * @noresvport: 1 if non-privileged port should be used
  *
  * Returns an nlm_host structure that matches the passed-in

@@ -465,6 +468,7 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 				     const unsigned short protocol,
 				     const u32 version,
 				     const char *hostname,
+				     const char *myname,
 				     int noresvport)
 {
 	const struct sockaddr source = {

@@ -476,6 +480,7 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 		.salen		= salen,
 		.protocol	= protocol,
 		.version	= version,
+		.my_name	= myname,
 		.hostname	= hostname,
 		.hostname_len	= strlen(hostname),
 		.src_sap	= &source,
fs/nfs/client.c +11 -0
index ee77713..920b842 100644
@@ -98,6 +98,7 @@ struct rpc_program		nfsacl_program = {
 #endif  /* CONFIG_NFS_V3_ACL */
 
 struct nfs_client_initdata {
+	const char *myname;
 	const char *hostname;
 	const struct sockaddr *addr;
 	size_t addrlen;

@@ -129,6 +130,13 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 	memcpy(&clp->cl_addr, cl_init->addr, cl_init->addrlen);
 	clp->cl_addrlen = cl_init->addrlen;
 
+	if (cl_init->myname) {
+		err = -ENOMEM;
+		clp->cl_myname = kstrdup(cl_init->myname, GFP_KERNEL);
+		if (!clp->cl_myname)
+			goto error_cleanup;
+	}
+
 	if (cl_init->hostname) {
 		err = -ENOMEM;
 		clp->cl_hostname = kstrdup(cl_init->hostname, GFP_KERNEL);

@@ -159,6 +167,7 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 	return clp;
 
 error_cleanup:
+	kfree(clp->cl_myname);
 	kfree(clp);
 error_0:
 	return ERR_PTR(err);

@@ -645,6 +654,7 @@ static int nfs_start_lockd(struct nfs_server *server)
 	struct nlm_host *host;
 	struct nfs_client *clp = server->nfs_client;
 	struct nlmclnt_initdata nlm_init = {
+		.myname		= clp->cl_myname,
 		.hostname	= clp->cl_hostname,
 		.address	= (struct sockaddr *)&clp->cl_addr,
 		.addrlen	= clp->cl_addrlen,

@@ -780,6 +790,7 @@ static int nfs_init_server(struct nfs_server *server,
 			   const struct nfs_parsed_mount_data *data)
 {
 	struct nfs_client_initdata cl_init = {
+		.myname = data->myname,
 		.hostname = data->nfs_server.hostname,
 		.addr = (const struct sockaddr *)&data->nfs_server.address,
 		.addrlen = data->nfs_server.addrlen,
fs/nfs/internal.h +1 -0
index 37c37f8..ff0c906 100644
@@ -80,6 +80,7 @@ struct nfs_parsed_mount_data {
 	unsigned int		version;
 	unsigned int		minorversion;
 	char			*fscache_uniq;
+	char			*myname;
 
 	struct {
 		struct sockaddr_storage	address;
fs/nfs/super.c +9 -0
index 21a0dcb..50a6d82 100644
@@ -99,6 +99,7 @@ enum {
 	Opt_addr, Opt_mountaddr, Opt_clientaddr,
 	Opt_lookupcache,
 	Opt_fscache_uniq,
+	Opt_caller_name,
 
 	/* Special mount options */
 	Opt_userspace, Opt_deprecated, Opt_sloppy,

@@ -170,6 +171,7 @@ static const match_table_t nfs_mount_option_tokens = {
 
 	{ Opt_lookupcache, "lookupcache=%s" },
 	{ Opt_fscache_uniq, "fsc=%s" },
+	{ Opt_caller_name, "callername=%s" },
 
 	{ Opt_err, NULL }
 };

@@ -1383,6 +1385,13 @@ static int nfs_parse_mount_options(char *raw,
 			mnt->fscache_uniq = string;
 			mnt->options |= NFS_OPTION_FSCACHE;
 			break;
+		case Opt_caller_name:
+			string = match_strdup(args);
+			if (string == NULL)
+				goto out_nomem;
+			kfree(mnt->myname);
+			mnt->myname = string;
+			break;
 
 		/*
 		 * Special options
include/linux/lockd/bind.h +1 -0
index fbc48f8..5d51b87 100644
@@ -36,6 +36,7 @@ extern struct nlmsvc_binding *	nlmsvc_ops;
  * rpc_ops field.
  */
 struct nlmclnt_initdata {
+	const char		*myname;
 	const char		*hostname;
 	const struct sockaddr	*address;
 	size_t			addrlen;
include/linux/lockd/lockd.h +1 -0
index a19bdde..e83b62d 100644
@@ -226,6 +226,7 @@ struct nlm_host  *nlmclnt_lookup_host(const struct sockaddr *sap,
 					const size_t salen,
 					const unsigned short protocol,
 					const u32 version,
+					const char *myname,
 					const char *hostname,
 					int noresvport);
 struct nlm_host  *nlmsvc_lookup_host(const struct svc_rqst *rqstp,
include/linux/nfs_fs_sb.h +5 -0
index 34fc6be..7c86c2f 100644
@@ -66,6 +66,11 @@ struct nfs_client {
 	/* idmapper */
 	struct idmap *		cl_idmap;
 
+	/* Our local hostname, as a null-terminated string.
+	 * This is used to generate the caller_name for v2/v3 locking.
+	 */
+	char			*cl_myname;
+
 	/* Our own IP address, as a null-terminated string.
 	 * This is used to generate the clientid, and the callback address.
 	 */

A message that is not in the archive (sent off-list or not yet received): 20100401190400.6395.52787.stgit-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]

#14 lore raw

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

Quoted 15 lines · Chuck Lever

NLMPROC_LOCK requests have a "caller_name" argument which is supposed to contain the hostname the server uses to call the client back. Linux simply stuffs the system's utsname in this field, but this is not always the correct choice. For example:

o If an unqualified hostname is used for the client's utsname, it could be ambiguous when the server tries to resolve it
o If the client's actual hostname is determined by DHCP, it may not match its utsname
o If the NFS mount was done in a network namespace, the namespace name won't match the client's utsname
o If the client has multiple network interfaces, it should provide a hostname that matches the source address used to contact the server

Quoted 6 lines

In all of these cases, user space can determine the correct value of the caller_name argument at mount time.

So, add a mount option that allows user space to specify the value of the caller_name argument of NLMPROC_LOCK requests. If not specified, the kernel continues to use the init utsname, as before.

This argument makes no sense. Mount points do _not_ follow network namespace boundaries, so making this hostname of yours a mount option will make matters worse, not better.

Furthermore, even if we were to accept your argument, you are not matching nfs_clients to your "namespace name", so if you do have more than 1 mount to the same server but from different namespaces, then the namespace name of the first to mount will automatically become the default for the second mount too.

Trond

A message that is not in the archive (sent off-list or not yet received): 1270151136.13329.8.camel-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]

#16 lore raw

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

Quoted 27 lines · Trond Myklebust
Quoted 15 lines · Chuck Lever

NLMPROC_LOCK requests have a "caller_name" argument which is supposed to contain the hostname the server uses to call the client back. Linux simply stuffs the system's utsname in this field, but this is not always the correct choice. For example:

o If an unqualified hostname is used for the client's utsname, it could be ambiguous when the server tries to resolve it
o If the client's actual hostname is determined by DHCP, it may not match its utsname
o If the NFS mount was done in a network namespace, the namespace name won't match the client's utsname
o If the client has multiple network interfaces, it should provide a hostname that matches the source address used to contact the server

Quoted 6 lines

In all of these cases, user space can determine the correct value of the caller_name argument at mount time.

So, add a mount option that allows user space to specify the value of the caller_name argument of NLMPROC_LOCK requests. If not specified, the kernel continues to use the init utsname, as before.

This argument makes no sense. Mount points do _not_ follow network namespace boundaries, so making this hostname of yours a mount option will make matters worse, not better.

Quoted 5 lines

Furthermore, even if we were to accept your argument, you are not matching nfs_clients to your "namespace name", so if you do have more than 1 mount to the same server but from different namespaces, then the namespace name of the first to mount will automatically become the default for the second mount too.

Basically, it boils down to: what does all this extra stuff give me that the current (per-thread) utsname group namespace does not allow?

Trond

author
#17 lore raw

Re: [PATCH 11/11] lockd: Allow mount option to specify caller_name

Quoted 20 lines · Trond Myklebust
Quoted 19 lines · Trond Myklebust
Quoted 18 lines · Chuck Lever

NLMPROC_LOCK requests have a "caller_name" argument which is supposed to contain the hostname the server uses to call the client back. Linux simply stuffs the system's utsname in this field, but this is not always the correct choice. For example:

o If an unqualified hostname is used for the client's utsname, it could be ambiguous when the server tries to resolve it
o If the client's actual hostname is determined by DHCP, it may not match its utsname
o If the NFS mount was done in a network namespace, the namespace name won't match the client's utsname
o If the client has multiple network interfaces, it should provide a hostname that matches the source address used to contact the server

In all of these cases, user space can determine the correct value of the caller_name argument at mount time.

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

So, add a mount option that allows user space to specify the value of the caller_name argument of NLMPROC_LOCK requests. If not specified, the kernel continues to use the init utsname, as before.

This argument makes no sense. Mount points do _not_ follow network namespace boundaries, so making this hostname of yours a mount option will make matters worse, not better.

Um, "this hostname of yours" is snide and unnecessary. It's the caller_name string, and it's been an argument of NLMPROC_LOCK and used for lock recovery ever since NFSv2 was invented. Let's keep it civil, please.

So, ignore the network namespace example, then, and consider the majority of the examples above.

The server's statd stores the client's caller_name string on the monitor list, and uses it as part of a heuristic to match incoming SM_NOTIFY requests. If we send an accurate caller_name string in our NLMPROC_LOCK requests, there's a better chance that the remote statd will recognize us when we reboot. Refer to Talpey's Cthon slides or _NFS_Illustrated_ for visual aids.

This applies to three of the four examples I provided above:

1) It's been a best practice for a long time to ensure that your Linux client's nodename (ie its utsname) matches it's fully-qualified domain name, and for exactly this purpose (see NetApp TR-3183). With this mount option, the correct caller_name can be determined automatically.

What happens if the client's utsname is unqualified, and then it contacts a server that is already talking to a client with the same unqualified hostname in a different domain? The result is that the server will have to choose between these two clients when one of them reboots.

2) If a client's address is assigned automatically, it won't necessarily match its utsname. That's true of my laptops on wireless, for example. In that case, my Dell laptop would send "SM_NOTIFY ellison.1015granger.net" from, say, anon-dhcp-108.1015granger.net. statd's DNS monname matching heuristic might fail.

Note that most contemporary Linux servers store the client's address rather than the caller_name string, but that just means our server won't recognize a client's reboot if the client is assigned a different address after it reboots, and that DNS configuration is especially critical to get lock recovery right.

If our client is operating with an automatically assigned IPv6 address, where a router gives an IPv6 address prefix, and the rest of the address is constructed from the NIC's MAC address, or, if our IPv4 address is DHCP-assigned by MAC address, what happens if we shut down the client, and then replace the NIC? What if our client switches from wireless to wired?

In other words, we can't rely solely on source IP address to identify rebooting hosts.

3) If the client is talking to a server on a private area network, there's no guarantee the server will recognize the client's caller_name string if it's the hostname of the client on the public side network. It may even attempt to contact the client via it's public side name, which might fail, depending on how the network is set up.

Therefore, I assert that this feature is needed to support multi-homed locking adequately, and to provide better lock recovery in the face of dynamically assigned IP addresses.

Signature · 1 line
chuck[dot]lever[at]oracle[dot]com