Skip to contentSkip to conversation

NFSD: Guard admin state-revocation walks with NFSD_NET_UP

PATCH In review Chuck Lever 2 participants 4 messages Jun 21 – Jun 22

Subsystems (2)
lore
Share & tools
lore

Chuck Lever

author maintainer
#0 lore raw

PATCH 1/1 NFSD: Guard admin state-revocation walks with NFSD_NET_UP

Writing to /proc/fs/nfsd/unlock_filesystem, or sending the NFSD_CMD_UNLOCK_FILESYSTEM or NFSD_CMD_UNLOCK_EXPORT netlink command, walks the NFSv4 client hash tables to revoke open state and cancel async COPY operations. All three handlers gate that walk on nn->nfsd_serv, but a listener added via portlist or netlink listener_set sets nn->nfsd_serv before any nfsd thread starts. nfsd_startup_net() has not yet allocated nn->conf_id_hashtbl, so the walkers dereference a NULL table. A local administrator with CAP_SYS_ADMIN can crash the kernel this way without ever starting the server.

nn->nfsd_serv is set when the service is created, which precedes table allocation. NFSD_NET_UP instead brackets the window where the tables are live: set at the end of nfsd_startup_net() and cleared in nfsd_shutdown_net() after they are freed, both under nfsd_mutex. Gating the three unlock paths on NFSD_NET_UP fixes the startup-time NULL dereference while preserving the earlier post-shutdown use-after-free fix.

Fixes 1ac3629bf012 ("nfsd: prepare for supporting admin-revocation of state")
---
 fs/nfsd/nfs4proc.c  |  7 +++----
 fs/nfsd/nfs4state.c | 14 ++++++--------
 fs/nfsd/nfsctl.c    |  6 +++---
 3 files changed, 12 insertions(+), 15 deletions(-)
fs/nfsd/nfs4proc.c +3 -4
index c413ed0810b9..8351ccaae59c 100644
@@ -1588,10 +1588,9 @@ static bool nfsd4_copy_on_sb(const struct nfsd4_copy *copy,
  * @net: net namespace containing the copy operations
  * @sb: targeted superblock
  *
- * Context: Caller must hold nfsd_mutex with nn->nfsd_serv confirmed
- *          non-NULL.  nfs4_state_destroy_net() frees conf_id_hashtbl
- *          at server shutdown without clearing the pointer, so a
- *          walk without these guarantees iterates freed slab memory.
+ * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ *          that window nn->conf_id_hashtbl is unallocated or freed,
+ *          so the walk would dereference a NULL or dangling pointer.
  */
 void nfsd4_cancel_copy_by_sb(struct net *net, struct super_block *sb)
 {
fs/nfsd/nfs4state.c +6 -8
index 4ae5d65c056a..a4398dc861a5 100644
@@ -1946,10 +1946,9 @@ static void revoke_one_stid(struct nfsd_net *nn, struct nfs4_client *clp,
  * The clients which own the states will subsequently be notified that the
  * states have been "admin-revoked".
  *
- * Context: Caller must hold nfsd_mutex with nn->nfsd_serv confirmed
- *          non-NULL.  nfs4_state_destroy_net() frees conf_id_hashtbl
- *          at server shutdown without clearing the pointer, so a
- *          walk without these guarantees iterates freed slab memory.
+ * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ *          that window nn->conf_id_hashtbl is unallocated or freed,
+ *          so the walk would dereference a NULL or dangling pointer.
  */
 void nfsd4_revoke_states(struct nfsd_net *nn, struct super_block *sb)
 {
@@ -2024,10 +2023,9 @@ static struct nfs4_stid *find_one_export_stid(struct nfs4_client *clp,
  * Userspace (exportfs -u) sends this after removing the last client
  * for a path, enabling the underlying filesystem to be unmounted.
  *
- * Context: Caller must hold nfsd_mutex with nn->nfsd_serv confirmed
- *          non-NULL.  nfs4_state_destroy_net() frees conf_id_hashtbl
- *          at server shutdown without clearing the pointer, so a
- *          walk without these guarantees iterates freed slab memory.
+ * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ *          that window nn->conf_id_hashtbl is unallocated or freed,
+ *          so the walk would dereference a NULL or dangling pointer.
  */
 void nfsd4_revoke_export_states(struct nfsd_net *nn, const struct path *path)
 {
fs/nfsd/nfsctl.c +3 -3
index caf59421f8f4..bc16fc7ca24f 100644
@@ -299,7 +299,7 @@ static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size)
 	error = nlmsvc_unlock_all_by_sb(path.dentry->d_sb);
 	mutex_lock(&nfsd_mutex);
 	nn = net_generic(netns(file), nfsd_net_id);
-	if (nn->nfsd_serv) {
+	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd4_cancel_copy_by_sb(netns(file), path.dentry->d_sb);
 		nfsd4_revoke_states(nn, path.dentry->d_sb);
 	} else {
@@ -2424,7 +2424,7 @@ int nfsd_nl_unlock_filesystem_doit(struct sk_buff *skb,
 	error = nlmsvc_unlock_all_by_sb(path.dentry->d_sb);
 
 	mutex_lock(&nfsd_mutex);
-	if (nn->nfsd_serv) {
+	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd4_cancel_copy_by_sb(net, path.dentry->d_sb);
 		nfsd4_revoke_states(nn, path.dentry->d_sb);
 	} else {
@@ -2471,7 +2471,7 @@ int nfsd_nl_unlock_export_doit(struct sk_buff *skb, struct genl_info *info)
 		return error;
 
 	mutex_lock(&nfsd_mutex);
-	if (nn->nfsd_serv) {
+	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd_file_close_export(net, &path);
 		nfsd4_revoke_export_states(nn, &path);
 	} else
Signature · 1 line
2.54.0
maintainer
#1 lore raw
Quoted 11 lines · Chuck Lever
fs/nfsd/nfsctl.c +0 -1
index 11bbc7e8210c..29d68abfa5c8 100644
@@ -296,14 +296,15 @@ static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size)
 	 * 2.  Is that directory a mount point, or
 	 * 3.  Is that directory the root of an exported file system?
 	 */
-	nfsd4_cancel_copy_by_sb(netns(file), path.dentry->d_sb);
 	error = nlmsvc_unlock_all_by_sb(path.dentry->d_sb);
 	mutex_lock(&nfsd_mutex);

fs/nfsd/nfsctl.c static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size) · line 308

Quoted 10 lines
fs/nfsd/nfsctl.c +4 -2
 	nn = net_generic(netns(file), nfsd_net_id);
-	if (nn->nfsd_serv)
+	if (nn->nfsd_serv) {
+		nfsd4_cancel_copy_by_sb(netns(file), path.dentry->d_sb);
 		nfsd4_revoke_states(nn, path.dentry->d_sb);
-	else
+	} else {
 		error = -EINVAL;
+	}
 	mutex_unlock(&nfsd_mutex);

Can nn->nfsd_serv be non-NULL while the NFSv4 state tables are still NULL? Looking at nfsd_create_serv() in nfssvc.c, it sets nn->nfsd_serv when a listener is added via portlist or netlink, and that path does not run nfsd_startup_net().

nfsd_startup_net() is what calls nfs4_state_start_net() which allocates nn->conf_id_hashtbl in nfs4_state_create_net(). Until threads are started, nn->nfsd_net_up stays false and the hashtables stay NULL.

In that startup window, write_unlock_fs() would pass the nn->nfsd_serv check and then call nfsd4_cancel_copy_by_sb():

fs/nfsd/nfs4proc.c:nfsd4_cancel_copy_by_sb() {
    ...
    spin_lock(&nn->client_lock);
    for (idhashval = 0; idhashval < CLIENT_HASH_SIZE; idhashval++) {
        struct list_head *head = &nn->conf_id_hashtbl[idhashval];
        ...

Would this dereference a NULL conf_id_hashtbl and trigger the null-ptr-deref reported in the thread? The KASAN trace shows RIP in nfsd4_cancel_copy_by_sb at offset 0x184 with RAX 0, which matches the head pointer load.

The same pattern applies to nfsd4_revoke_states() which walks the same table, and to nfsd_nl_unlock_filesystem_doit() in the netlink path.

Would checking nn->nfsd_net_up instead of nn->nfsd_serv be more accurate here, since nfsd_net_up is set at the tail of nfsd_startup_net() exactly when the state tables become valid?

Signature · 1 line
Jeff Layton <[email protected]>
author maintainer
#2 lore raw
Quoted 6 lines · Jeff Layton

Can nn->nfsd_serv be non-NULL while the NFSv4 state tables are still NULL?
...
Would checking nn->nfsd_net_up instead of nn->nfsd_serv be more accurate here, since nfsd_net_up is set at the tail of nfsd_startup_net() exactly when the state tables become valid?

Yep, that's exactly the gap. The hunk you've quoted is 1/3 of the earlier "post-shutdown use-after-free" series, which only guarded the shutdown side of the window. nn->nfsd_serv is also set at service creation, before nfsd_startup_net() allocates conf_id_hashtbl, so the startup side was still exposed.

The thread you're replying to is the follow-on that closes it: it switches all three unlock paths (write_unlock_fs, nfsd_nl_unlock_filesystem_doit, nfsd_nl_unlock_export_doit) from nn->nfsd_serv to test_bit(NFSD_NET_UP, &nn->flags) and rewrites the walkers' Context: notes to match. NFSD_NET_UP is set at the tail of nfsd_startup_net() and cleared in nfsd_shutdown_net() after the tables are freed, so it brackets precisely the window you describe.

I assume your comment was meant for 1/3 of that series rather than this thread... but we've reached the same conclusion, so no action needed beyond this patch, correct?

Signature · 1 line
Chuck Lever
maintainer Reviewed-by
#3 lore raw
Quoted 22 lines · Chuck Lever
Quoted 6 lines · Jeff Layton

Can nn->nfsd_serv be non-NULL while the NFSv4 state tables are still NULL?
...
Would checking nn->nfsd_net_up instead of nn->nfsd_serv be more accurate here, since nfsd_net_up is set at the tail of nfsd_startup_net() exactly when the state tables become valid?

Yep, that's exactly the gap. The hunk you've quoted is 1/3 of the earlier "post-shutdown use-after-free" series, which only guarded the shutdown side of the window. nn->nfsd_serv is also set at service creation, before nfsd_startup_net() allocates conf_id_hashtbl, so the startup side was still exposed.

The thread you're replying to is the follow-on that closes it: it switches all three unlock paths (write_unlock_fs, nfsd_nl_unlock_filesystem_doit, nfsd_nl_unlock_export_doit) from nn->nfsd_serv to test_bit(NFSD_NET_UP, &nn->flags) and rewrites the walkers' Context: notes to match. NFSD_NET_UP is set at the tail of nfsd_startup_net() and cleared in nfsd_shutdown_net() after the tables are freed, so it brackets precisely the window you describe.

Quoted 3 lines

I assume your comment was meant for 1/3 of that series rather than this thread... but we've reached the same conclusion, so no action needed beyond this patch, correct?

Ahh right. I missed that that had been fixed in the other series. This is fine then.