Skip to contentSkip to conversation

why does each NFS mount have (at least) 2 rpc_clnt's ?

DISCUSSION Jeff Layton 3 participants 7 messages Apr 1, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I'm working on an issue in an older kernel where we see occasional panics when trying to refresh credentials. Here's the bug in case anyone is interested:

    https://bugzilla.redhat.com/show_bug.cgi?id=572870

...I think I understand the problem well enough now. The problem is pretty complex, but the issue is that some operations are done using credentials from a stateowner associated with a nfs_client, but using the rpc_clnt in nfs_server->client. The two can have different authtypes if there are a mix of mounts with different authtypes to the same server. This problem seems to have been fixed in mainline with the introduction of the auth_generic code.

It leaves me wondering though...what exactly is the reason for having two rpc_clients per NFS mount? To clarify, I'm talking about these two, which seem to be somewhat redundant:

nfs_server->client
nfs_server->nfs_client->cl_rpcclient

On mount, the nfs4_set_client calls nfs_get_client to search the list of nfs_client structs until it finds one that matches the address, port, etc of the NFS server. If one isn't found, the kernel creates one using whatever authtype was requested for the mount.

Later, nfs_init_server_rpcclient looks at the rpc_clnt in the nfs_client and copies it. If the auth pseudoflavor doesn't match however, it creates a new rpc_auth for it.

What exactly is the point of having two rpc_clnt's? Why not just get always use nfs_client->cl_rpcclient instead of nfs_server->client and simply have nfs_get_client filter by authtype?

Signature · 5 lines
Jeff Layton <[email protected]>
_______________________________________________
NFSv4 mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4
#1 lore raw
Quoted 26 lines · Jeff Layton

I'm working on an issue in an older kernel where we see occasional panics when trying to refresh credentials. Here's the bug in case anyone is interested:

    https://bugzilla.redhat.com/show_bug.cgi?id=572870

...I think I understand the problem well enough now. The problem is pretty complex, but the issue is that some operations are done using credentials from a stateowner associated with a nfs_client, but using the rpc_clnt in nfs_server->client. The two can have different authtypes if there are a mix of mounts with different authtypes to the same server. This problem seems to have been fixed in mainline with the introduction of the auth_generic code.

It leaves me wondering though...what exactly is the reason for having two rpc_clients per NFS mount? To clarify, I'm talking about these two, which seem to be somewhat redundant:

nfs_server->client
nfs_server->nfs_client->cl_rpcclient

On mount, the nfs4_set_client calls nfs_get_client to search the list of nfs_client structs until it finds one that matches the address, port, etc of the NFS server. If one isn't found, the kernel creates one using whatever authtype was requested for the mount.

Quoted 8 lines

Later, nfs_init_server_rpcclient looks at the rpc_clnt in the nfs_client and copies it. If the auth pseudoflavor doesn't match however, it creates a new rpc_auth for it.

What exactly is the point of having two rpc_clnt's? Why not just get always use nfs_client->cl_rpcclient instead of nfs_server->client and simply have nfs_get_client filter by authtype?

Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the only thing that is changed. We also change the soft flag and the timeout properties of the server->client.

The point is that users sometimes want to specify per-mountpoint transport properties, and so we try to give them that possibility, while at the same time sharing sockets/rdma connections.

Cheers
Trond

_______________________________________________
NFSv4 mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

#2 lore raw

Hi Trond,

Does it have anything to do with the situation when we want an the same filesystem to be mounted twice somewhere else with a different flags? (i.e. RO/RW for example) If yes, I remember some discussion about it in the past (the above was not possible).
Thanks,

Ondrej

Quoted 16 lines

Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the only thing that is changed. We also change the soft flag and the timeout properties of the server->client.

The point is that users sometimes want to specify per-mountpoint transport properties, and so we try to give them that possibility, while at the same time sharing sockets/rdma connections.

Cheers
Trond

_______________________________________________
NFSv4 mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

#3 lore raw
Quoted 8 lines · Ondrej Valousek

Hi Trond,

Does it have anything to do with the situation when we want an the same filesystem to be mounted twice somewhere else with a different flags? (i.e. RO/RW for example)
If yes, I remember some discussion about it in the past (the above was not possible).
Thanks,

Yes. That's exactly what it is supposed to allow (and btw, mounting filesystems both ro and rw should be possible now).

Cheers
Trond

Quoted 22 lines

Ondrej

Quoted 8 lines

Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the only thing that is changed. We also change the soft flag and the timeout properties of the server->client.

The point is that users sometimes want to specify per-mountpoint transport properties, and so we try to give them that possibility, while at the same time sharing sockets/rdma connections.

Quoted 8 lines

Cheers
Trond

_______________________________________________
NFSv4 mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
In reply to Trond Myklebust
Quoted 40 lines · On Thu, 01 Apr 2010 09:17:49 -0400 Trond Myklebust…
Quoted 26 lines · Jeff Layton

I'm working on an issue in an older kernel where we see occasional panics when trying to refresh credentials. Here's the bug in case anyone is interested:

    https://bugzilla.redhat.com/show_bug.cgi?id=572870

...I think I understand the problem well enough now. The problem is pretty complex, but the issue is that some operations are done using credentials from a stateowner associated with a nfs_client, but using the rpc_clnt in nfs_server->client. The two can have different authtypes if there are a mix of mounts with different authtypes to the same server. This problem seems to have been fixed in mainline with the introduction of the auth_generic code.

It leaves me wondering though...what exactly is the reason for having two rpc_clients per NFS mount? To clarify, I'm talking about these two, which seem to be somewhat redundant:

nfs_server->client
nfs_server->nfs_client->cl_rpcclient

On mount, the nfs4_set_client calls nfs_get_client to search the list of nfs_client structs until it finds one that matches the address, port, etc of the NFS server. If one isn't found, the kernel creates one using whatever authtype was requested for the mount.

Quoted 8 lines

Later, nfs_init_server_rpcclient looks at the rpc_clnt in the nfs_client and copies it. If the auth pseudoflavor doesn't match however, it creates a new rpc_auth for it.

What exactly is the point of having two rpc_clnt's? Why not just get always use nfs_client->cl_rpcclient instead of nfs_server->client and simply have nfs_get_client filter by authtype?

Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the only thing that is changed. We also change the soft flag and the timeout properties of the server->client.

Quoted 7 lines

The point is that users sometimes want to specify per-mountpoint transport properties, and so we try to give them that possibility, while at the same time sharing sockets/rdma connections.

Cheers
Trond

Ok, I think I sort of understand now. The nfs_client holds some global state info that's common to all nfs mounts on the server so that info has to be shared between NFS mounts. The nfs_server struct holds stuff that's specific to a single superblock.

That said, I still am not certain I see why we want to have an rpc_clnt that's shared between the mounts though clearly there's benefit to sharing the xprt between them.

Signature · 5 lines
Jeff Layton <[email protected]>
_______________________________________________
NFSv4 mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4
#6 lore raw
Quoted 3 lines · Jeff Layton

That said, I still am not certain I see why we want to have an rpc_clnt that's shared between the mounts though clearly there's benefit to sharing the xprt between them.

Sharing the xprt is by far the main (and most important) benefit. Without it, we would have the same mess with running out of privileged port numbers that we had with the 2.4.x series.

Cheers
Trond