NFS: Fix interaction between fs_context and user namespaces
Subsystems (3)
v5 cover NFS: Fix interaction between fs_context and user namespaces
Right now, it is possible to mount NFS with an non-matching super block user ns, and NFS sunrpc user ns. This (for the user) results in an awkward set of interactions if using anything other than auth_null, where the UIDs being sent to the server are different than the local UIDs being checked. This can cause "breakage", where if you try to communicate with the NFS server with any other set of mappings, it breaks.
The reason for this is that you can call fsopen("nfs4") in the unprivileged namespace, and that configures fs_context with all the right information for that user namespace. In addition, it also keeps a gets a cred object associated with the caller -- which should match the user namespace. Unfortunately, the mount has to be finished in the init_user_ns because we currently require CAP_SYS_ADMIN in the init user namespace to call fsmount. This means that the superblock's user namespace is set "correctly" to the container, but there's absolutely no way nfs4idmap to consume an unprivileged user namespace because the cred / user_ns that's passed down to nfs4idmap is the one at fsmount.
How this actually exhibits is let's say that the UID 0 in the user namespace is mapped to UID 1000 in the init user ns (and kuid space). What will happen is that nfs4idmap will translate the UID 1000 into UID 0 on the wire, even if the mount is in entirely in the mount / user namespace of the container.
So, it looks something like this
Client in unprivileged User NS (UID: 0, KUID: 0)
->Perform open() ...VFS / NFS bits... nfs_map_uid_to_name -> from_kuid_munged(init_user_ns, uid) (returns 0) RPC with UID 0
This behaviour happens "the other way" as well, where the UID in the container may be 0, but the corresponding kuid is 1000. When a response from an NFS server comes in we decode it according to the idmap userns. The way this exhibits is even more odd.
Server responds with file attribute (UID: 0, GID: 0)
->nfs_map_name_to_uid(..., 0) ->make_kuid(init_user_ns, id) (returns 0) ....VFS / NFS Bits... ->from_kuid(container_ns, 0) -> invalid uid -> EOVERFLOW
This changes the nfs server to use the cred / userns from fs_context, which is how idmap is constructed. This subsequently is used in the above described flow of converting uids back-and-forth.
Trond gave the feedback that this behaviour [implemented by this patch] is how the legacy sys_mount() behaviour worked[1], and that the intended behaviour is for UIDs to be plumbed through entirely, where the user namespaces UIDs are what is sent over the wire, and not the init user ns.
[1]: https://lore.kernel.org/linux-nfs/[email protected]/
Sargun Dhillon (2):
NFS: NFSv2/NFSv3: Use cred from fs_context during mount
NFSv4: Refactor to use user namespaces for nfs4idmap
fs/nfs/client.c | 4 ++-- fs/nfs/nfs4client.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
base-commit: 8c39076c276be0b31982e44654e2c2357473258a
Signature · 1 line
2.25.1
PATCH v5 1/2 NFS: NFSv2/NFSv3: Use cred from fs_context during mount
There was refactoring done to use the fs_context for mounting done in:62a55d088cd87: NFS: Additional refactoring for fs_context conversion
This made it so that the net_ns is fetched from the fs_context (the netns that fsopen is called in). This change also makes it so that the credential fetched during fsopen is used as well as the net_ns.
NFS has already had a number of changes to prepare it for user namespaces:1a58e8a0e5c1: NFS: Store the credential of the mount process in the nfs_server 264d948ce7d0: NFS: Convert NFSv3 to use the container user namespacec207db2f5da5: NFS: Convert NFSv2 to use the container user namespace
Previously, different credentials could be used for creation of the fs_context versus creation of the nfs_server, as FSCONFIG_CMD_CREATE did the actual credential check, and that's where current_creds() were fetched. This meant that the user namespace which fsopen was called in could be a non-init user namespace. This still requires that the user that calls FSCONFIG_CMD_CREATE has CAP_SYS_ADMIN in the init user ns.
This roughly allows a privileged user to mount on behalf of an unprivileged usernamespace, by forking off and calling fsopen in the unprivileged user namespace. It can then pass back that fsfd to the privileged process which can configure the NFS mount, and then it can call FSCONFIG_CMD_CREATE before switching back into the mount namespace of the container, and finish up the mounting process and call fsmount and move_mount.
--- fs/nfs/client.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/nfs/client.c +2 -2
@@ -571,7 +571,7 @@ static int nfs_start_lockd(struct nfs_server *server) 1 : 0, .net = clp->cl_net, .nlmclnt_ops = clp->cl_nfs_mod->rpc_ops->nlmclnt_ops, - .cred = current_cred(), + .cred = server->cred, };if (nlm_init.nfs_version >3 )
@@ -985,7 +985,7 @@ struct nfs_server *nfs_create_server(struct fs_context *fc)if (!server)return ERR_PTR (-ENOMEM ); - server->cred =get_cred (current_cred ()); + server->cred =get_cred (fc->cred); error = -ENOMEM ; fattr =nfs_alloc_fattr ();
Signature · 1 line
2.25.1
PATCH v5 2/2 NFSv4: Refactor to use user namespaces for nfs4idmap
In reply to Sargun DhillonIn several patches work has been done to enable NFSv4 to use user namespaces:58002399da65: NFSv4: Convert the NFS client idmapper to use the container user namespace 3b7eb5e35d0f: NFS: When mounting, don't share filesystems between different user namespaces
Unfortunately, the userspace APIs were only such that the userspace facing side of the filesystem (superblock s_user_ns) could be set to a non init user namespace. This furthers the fs_context related refactoring, and piggybacks on top of that logic, so the superblock user namespace, and the NFS user namespace are the same.
Users can still use rpc.idmapd if they choose to, but there are complexities with user namespaces and request-key that have yet to be addresssed.
Eventually, we will need to at least:
* Separate out the keyring cache by namespace
* Come up with an upcall mechanism that can be triggered inside of the container, or safely triggered outside, with the requisite context to do the right mapping. * Handle whatever refactoring needs to be done in net/sunrpc.
--- fs/nfs/nfs4client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/nfs/nfs4client.c +1 -1
@@ -1153,7 +1153,7 @@ struct nfs_server *nfs4_create_server(struct fs_context *fc)if (!server)return ERR_PTR (-ENOMEM ); - server->cred =get_cred (current_cred ()); + server->cred =get_cred (fc->cred); auth_probe = ctx->auth_info.flavor_len <1 ;
Signature · 1 line
2.25.1
Re: [PATCH v5 2/2] NFSv4: Refactor to use user namespaces for nfs4idmap
Sorry for chiming in, but Sargun, Alban and I had a chat about this yesterday and so I took a closer at this patch and NFS today.
Quoted 10 lines · Sargun Dhillon
In several patches work has been done to enable NFSv4 to use user namespaces:
58002399da65: NFSv4: Convert the NFS client idmapper to use the container user namespace3b7eb5e35d0f: NFS: When mounting, don't share filesystems between different user namespacesUnfortunately, the userspace APIs were only such that the userspace facing side of the filesystem (superblock
s_user_ns) could be set to a non init user namespace. This furthers thefs_contextrelated refactoring, and piggybacks on top of that logic, so the superblock user namespace, and the NFS user namespace are the same.
The s_user_ns aka the user namespace of the superblock is taken from the struct fs_context->user_ns. This struct is created during fsopen() and it records the caller's credentials in fs_context->cred and the caller's user namespace in fs_context->cred->user_ns.
Actual creation of a new superblock happens during fsconfig(FSCONFIG_CMD_CREATE) which checks whether the caller is privileged wrt to the fs_context->user_ns if the filesystem is FS_USERNS_MOUNT and otherwise requires capabilities in the initial user namespace. The latter applies to NFS.
From fsconfig(FSCONFIG_CMD_CREATE() the vfs calls into the filesystem specific ->get_tree() methods which for NFS lands us in nfs_get_tree_common() which calls into sget_fc(). This sets sb->s_user_ns to fc->user_ns and thus to the user_ns of the creator of the fs_context from the original fsopen() call.
However, when a new nfs server is created the initial user namespace is used to stash the current clients credentials and this indeed feels like a bug. So it seems like the change here makes sense.
Quoted 9 lines
Users can still use rpc.idmapd if they choose to, but there are complexities with user namespaces and request-key that have yet to be addresssed.
Eventually, we will need to at least:
* Separate out the keyring cache by namespace
* Come up with an upcall mechanism that can be triggered inside of the container, or safely triggered outside, with the requisite context to do the right mapping. * Handle whatever refactoring needs to be done in net/sunrpc.
I think this whole section is orthogonal to the problem solved here, especially since I think the patch fixes a bug while enabling a use-case that should've likely worked before. There's no need tie these things together. Maybe the paragraph I have below makes this a little more obvious.
Quoted 12 lines
Signed-off-by Sargun Dhillon <[email protected]>Tested-by Alban Crequy <[email protected]>--- fs/nfs/nfs4client.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/nfs/nfs4client.c +0
-0
fs/nfs/nfs4client.c struct nfs_server *nfs4_create_server(struct fs_context *fc) · line 1158
Quoted 7 lines
fs/nfs/nfs4client.c +1
-1@@ -1153,7 +1153,7 @@ struct nfs_server *nfs4_create_server(struct fs_context *fc)if (!server)return ERR_PTR (-ENOMEM ); - server->cred =get_cred (current_cred ()); + server->cred =get_cred (fc->cred); auth_probe = ctx->auth_info.flavor_len <1 ;
So I looked at what file creation does involve and I'm not an nfs expert so I may be off and I trust people will yell at me if I am.
When a new file is created and the vfs part is done, the name and group is retrieved which is supposed to be used for file creation. The retrieved name and group is then translated in a first step via the upcall mechanism. (This first translation is technically equivalent to reading raw device ids from disk for other filesystems.) But this translation part is not really relevant to the problem here.
The interesting part for this fix is the step that happens when the raw ids are translated into their kernel representations aka kuid and kgid which is ultimately used to initialize i_uid and i_gid.
The raw device ids are translated based on server->nfs_client->cl_idmap. Afaict, the cl_idmap is setup based on server->creds and server->creds->user_ns. And that's when things get weird because server->creds are the callers credentials not the fsopen() credentials. Similarly the "wrong" creds are used when translating from the in-kernel to the userspace representation later on.
If it is intentional that server->cred uses the caller's creds by default and there's a fear of regressions then one option could be to add a mount option to NFS that indicates that the creator's credentials are supposed to be used. This might be interesting in so far, as it makes the intent to delegate the new client to another user more explicit. But whether that makes sense really depends on whether or not this used to work before or not, I guess.
In general, this solves a mapping problem specific to the nfs client it seems to me and looks fine, I think.
This is the rough callchain I based my analysis on:
i_op->create()::nfs_create()
-> NFS_PROTO(dir)->create()::nfs_v4_clientops->create()::nfs4_proc_create()
-> alloc_nfs_open_context()
-> if (filp)
ctx->cred = get_cred(filp->f_cred);
else
ctx->cred = get_current_cred();
-> nfs4_do_open()
-> _nfs4_do_open()
-> nfs4_get_state_owner()
/* The state is owned by ctx->cred and it is compared against server->cred. */
-> _nfs4_open_and_get_state()
/* Set owner_name. */
-> _nfs4_proc_open()
-> nfs4_run_open_task()
-> nfs4_procedures[NFSPROC4_CLNT_OPEN]::
-> nfs4_xdr_enc_open()
-> encode_getfattr_open()
-> encode_getattr()
/*
* Lookup the uid based on the name and store in fattr->uid.
* fattr->uid is used to initialize i_uid later after having been
* mapped is then used to initialize i_uid.
*/
-> nfs_fattr_map_and_free_names()
-> nfs_fattr_map_owner_name()
-> nfs_map_name_to_uid()
-> nfs_map_string_to_numeric() /* Try to parse the name as a uid. */
-> nfs_idmap_lookup_id() /* Lookup ids upcall mechanism. */
-> nfs_idmap_get_key()
-> nfs_idmap_request_key()
-> nfs_idmap_get_desc()
-> request_key_with_auxdata()/request_key()
-> fattr->uid = make_kuid(idmap_userns(idmap), id); Turn raw device id into kuids. */
-> _nfs4_opendata_to_nfs4_state()
-> nfs4_opendata_find_nfs4_state()
-> nfs4_opendata_get_inode()
-> nfs_fhget()
-> iget5_locked()
/* Initialize i_uid to INVALID_UID. */
-> inode->i_uid = make_kuid(&init_user_ns, -2);
/* Initialize i_uid according to fattr. */
if (fattr->valid & NFS_ATTR_FATTR_OWNER)
inode->i_uid = fattr->uid;
-> nfs4_do_setattr()
-> _nfs4_do_setattr()
-> nfs4_procedures[NFSPROC4_CLNT_SETATTR]::nfs4_xdr_enc_setattr()
-> encode_setattr()
-> encode_attrs()
-> nfs_map_uid_to_name()
-> struct idmap *idmap = server->nfs_client->cl_idmap;
id = from_kuid_munged(idmap_userns(idmap), uid);
-> nfs_setattr_update_inode()
-> if ((attr->ia_valid & ATTR_UID) != 0)
inode->i_uid = attr->ia_uid;
-> nfs_update_inode()
-> inode->i_uid = fattr->uid;Christian
Quoted 59 lines · Sargun Dhillon
Right now, it is possible to mount NFS with an non-matching super block user ns, and NFS sunrpc user ns. This (for the user) results in an awkward set of interactions if using anything other than
auth_null, where the UIDs being sent to the server are different than the local UIDs being checked. This can cause "breakage", where if you try to communicate with the NFS server with any other set of mappings, it breaks.The reason for this is that you can call
fsopen("nfs4")in the unprivileged namespace, and that configuresfs_contextwith all the right information for that user namespace. In addition, it also keeps a gets a cred object associated with the caller -- which should match the user namespace. Unfortunately, the mount has to be finished in theinit_user_nsbecause we currently requireCAP_SYS_ADMINin the init user namespace to call fsmount. This means that the superblock's user namespace is set "correctly" to the container, but there's absolutely no way nfs4idmap to consume an unprivileged user namespace because the cred /user_nsthat's passed down to nfs4idmap is the one at fsmount.How this actually exhibits is let's say that the UID 0 in the user namespace is mapped to UID 1000 in the init user ns (and kuid space). What will happen is that nfs4idmap will translate the UID 1000 into UID 0 on the wire, even if the mount is in entirely in the mount / user namespace of the container.
So, it looks something like this
Client in unprivileged User NS (UID: 0, KUID: 0)->Perform open() ...VFS / NFS bits... nfs_map_uid_to_name -> from_kuid_munged(init_user_ns, uid) (returns 0) RPC with UID 0This behaviour happens "the other way" as well, where the UID in the container may be 0, but the corresponding kuid is 1000. When a response from an NFS server comes in we decode it according to the idmap userns. The way this exhibits is even more odd.
Server responds with file attribute (UID: 0, GID: 0)
->nfs_map_name_to_uid(..., 0) ->make_kuid(init_user_ns, id) (returns 0) ....VFS / NFS Bits... ->from_kuid(container_ns, 0) -> invalid uid -> EOVERFLOWThis changes the nfs server to use the cred / userns from
fs_context, which is how idmap is constructed. This subsequently is used in the above described flow of converting uids back-and-forth.Trond gave the feedback that this behaviour [implemented by this patch] is how the legacy
sys_mount()behaviour worked[1], and that the intended behaviour is for UIDs to be plumbed through entirely, where the user namespaces UIDs are what is sent over the wire, and not the init user ns.[1]: https://lore.kernel.org/linux-nfs/[email protected]/
Sargun Dhillon (2):
NFS: NFSv2/NFSv3: Use cred fromfs_contextduring mount
NFSv4: Refactor to use user namespaces for nfs4idmap
Quoted 9 lines
fs/nfs/client.c | 4 ++-- fs/nfs/nfs4client.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)base-commit:
8c39076c276be0b31982e44654e2c2357473258aSignature · 1 line
2.25.1
Trond,
I was just thinking, since you said that this is the behaviour of the sys_mount API, would this be considered a regression? Should it go to stable (v5.9)?
Quoted 59 lines · Sargun Dhillon
Right now, it is possible to mount NFS with an non-matching super block user ns, and NFS sunrpc user ns. This (for the user) results in an awkward set of interactions if using anything other than
auth_null, where the UIDs being sent to the server are different than the local UIDs being checked. This can cause "breakage", where if you try to communicate with the NFS server with any other set of mappings, it breaks.The reason for this is that you can call
fsopen("nfs4")in the unprivileged namespace, and that configuresfs_contextwith all the right information for that user namespace. In addition, it also keeps a gets a cred object associated with the caller -- which should match the user namespace. Unfortunately, the mount has to be finished in theinit_user_nsbecause we currently requireCAP_SYS_ADMINin the init user namespace to call fsmount. This means that the superblock's user namespace is set "correctly" to the container, but there's absolutely no way nfs4idmap to consume an unprivileged user namespace because the cred /user_nsthat's passed down to nfs4idmap is the one at fsmount.How this actually exhibits is let's say that the UID 0 in the user namespace is mapped to UID 1000 in the init user ns (and kuid space). What will happen is that nfs4idmap will translate the UID 1000 into UID 0 on the wire, even if the mount is in entirely in the mount / user namespace of the container.
So, it looks something like this
Client in unprivileged User NS (UID: 0, KUID: 0)->Perform open() ...VFS / NFS bits... nfs_map_uid_to_name -> from_kuid_munged(init_user_ns, uid) (returns 0) RPC with UID 0This behaviour happens "the other way" as well, where the UID in the container may be 0, but the corresponding kuid is 1000. When a response from an NFS server comes in we decode it according to the idmap userns. The way this exhibits is even more odd.
Server responds with file attribute (UID: 0, GID: 0)
->nfs_map_name_to_uid(..., 0) ->make_kuid(init_user_ns, id) (returns 0) ....VFS / NFS Bits... ->from_kuid(container_ns, 0) -> invalid uid -> EOVERFLOWThis changes the nfs server to use the cred / userns from
fs_context, which is how idmap is constructed. This subsequently is used in the above described flow of converting uids back-and-forth.Trond gave the feedback that this behaviour [implemented by this patch] is how the legacy
sys_mount()behaviour worked[1], and that the intended behaviour is for UIDs to be plumbed through entirely, where the user namespaces UIDs are what is sent over the wire, and not the init user ns.[1]: https://lore.kernel.org/linux-nfs/[email protected]/
Sargun Dhillon (2):
NFS: NFSv2/NFSv3: Use cred fromfs_contextduring mount
NFSv4: Refactor to use user namespaces for nfs4idmap
Quoted 9 lines
fs/nfs/client.c | 4 ++-- fs/nfs/nfs4client.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)base-commit:
8c39076c276be0b31982e44654e2c2357473258aSignature · 1 line
2.25.1
Trond,
Are there any other concerns you have before landing this, or do you want to wait until the v5.11 merge window?