NFS client virtualization plan

DISCUSSION Stanislav Kinsbursky 5 participants 11 messages Aug 16, 2011 – Aug 22, 2011

Subsystems (2)

NFS client NFS server

lore

Share & tools

lore

Stanislav Kinsbursky

author

2011-08-16 10:59 UTC #0 lore raw

Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.

==============================================================================

"NFS client" virtualization plan:

_RPC layer_:

1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.

_NFS layer_:

1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.

2) Make Lockd kthread able to handle requests from different net namespaces.

3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.

4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.

5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.

6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.

Signature · 2 lines

Best regards,
Stanislav Kinsbursky

Stanislav Kinsbursky

author

2011-08-16 11:07 UTC #1 lore raw

Added maintainer to mail recipients.
Trond, please, have a look.
Sorry for inconvenience.

16.08.2011 15:00, Stanislav Kinsbursky пишет:

Quoted 36 lines

Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.
==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.

Signature · 2 lines

Best regards,
Stanislav Kinsbursky

Ben Greear

2011-08-16 12:00 UTC #2 lore raw

Quoted 3 lines · Stanislav Kinsbursky

Added maintainer to mail recipients.
Trond, please, have a look.
Sorry for inconvenience.

I have a set of patches that let NFS clients bind to a specific IP address, but I haven't gotten any feedback in the month since I posted them.

But, likely you will want to touch similar paths to what I touched.

Here's my patch series:

https://lkml.org/lkml/2011/7/15/382

You need some patches to mount.nfs
for full functionality, in case you want to try it out:

https://github.com/greearb/nfs-utils-ct

Thanks,
Ben

Quoted 40 lines

16.08.2011 15:00, Stanislav Kinsbursky пишет:
Quoted 36 lines
Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.
==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.

Signature · 2 lines

Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

Stanislav Kinsbursky

author

2011-08-16 12:19 UTC #3 lore raw

Thanks. Will have a look.

16.08.2011 15:59, Ben Greear =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

Quoted 16 lines · Stanislav Kinsbursky

Quoted 3 lines
Added maintainer to mail recipients.
Trond, please, have a look.
Sorry for inconvenience.
I have a set of patches that let NFS clients bind to a specific IP address, but I haven't gotten any feedback in the month since I posted them.
But, likely you will want to touch similar paths to what I touched.
Here's my patch series:
https://lkml.org/lkml/2011/7/15/382

Quoted 11 lines

You need some patches to mount.nfs
for full functionality, in case you want to try it out:
https://github.com/greearb/nfs-utils-ct
Thanks,
Ben
Quoted 2 lines
16.08.2011 15:00, Stanislav Kinsbursky =D0=BF=D0=B8=D1=88=D0=B5=D1=82=

Quoted 2 lines

Quoted 2 lines
Quoted 2 lines
Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS=

client part.

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
Any suggestions or criticism are welcome.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D

Quoted 6 lines

Quoted 6 lines
Quoted 6 lines
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to b=

Quoted 5 lines

Quoted 5 lines
Quoted 5 lines
per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used=

instead

Quoted 2 lines

Quoted 2 lines
Quoted 2 lines
of init_net).
Pointer to net namespace can be stored on nfs_client structure, whi=

ch will

Quoted 1 line

Quoted 1 line
Quoted 1 line
give easy access to proper net namespace during RPC transports crea=

tion.

Quoted 2 lines

Quoted 2 lines
Quoted 2 lines
2) Make Lockd kthread able to handle requests from different net na=

mespaces.

Quoted 2 lines

Quoted 2 lines
Quoted 2 lines
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callba=

ck_svc) able

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be =

per

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to b=

e per

Quoted 3 lines

Quoted 3 lines
Quoted 3 lines
net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to=

be per

Quoted 8 lines

Quoted 6 lines
Quoted 4 lines
net namespace in addition to global ones.

--=20
Best regards,
Stanislav Kinsbursky

Chuck Lever

2011-08-18 15:54 UTC #4 lore raw

In reply to Stanislav Kinsbursky

Quoted 3 lines · Stanislav Kinsbursky

Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.

What are you going to do about statd?

Quoted 39 lines

==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.
Signature · 6 lines
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Stanislav Kinsbursky

author

2011-08-19 09:46 UTC #5 lore raw

18.08.2011 19:54, Chuck Lever пишет:

Quoted 9 lines · Stanislav Kinsbursky

Quoted 4 lines
Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.
What are you going to do about statd?

Do you mean user-space statd?
If so, what are the problems with it do you see?

Quoted 40 lines

Quoted 39 lines
==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.
--
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Best regards,
Stanislav Kinsbursky

Chuck Lever

2011-08-19 16:12 UTC #6 lore raw

Quoted 8 lines · Stanislav Kinsbursky

18.08.2011 19:54, Chuck Lever пишет:
Quoted 7 lines · Stanislav Kinsbursky
Quoted 4 lines
Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.

Quoted 5 lines

Quoted 2 lines
What are you going to do about statd?
Do you mean user-space statd?
If so, what are the problems with it do you see?

You go to the trouble of setting up multiple transports for rpcbind upcalls. This suggests you plan to have an rpcbind in each net namespace.

On behalf of NFS, the kernel also performs idmapd, gssd, and statd (NSM) upcalls. Do you intend to have separate instances of these daemons for each net namespace?

Quoted 50 lines

Quoted 41 lines
Quoted 40 lines
==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.
--
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 6 lines
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Stanislav Kinsbursky

author

2011-08-19 17:21 UTC #7 lore raw

19.08.2011 20:12, Chuck Lever пишет:

Quoted 14 lines · Stanislav Kinsbursky

Quoted 12 lines
18.08.2011 19:54, Chuck Lever пишет:
Quoted 9 lines · Stanislav Kinsbursky
Quoted 4 lines
Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part. Any suggestions or criticism are welcome.
What are you going to do about statd?

Quoted 7 lines

Quoted 2 lines
Do you mean user-space statd?
If so, what are the problems with it do you see?
You go to the trouble of setting up multiple transports for rpcbind upcalls. This suggests you plan to have an rpcbind in each net namespace.
On behalf of NFS, the kernel also performs idmapd, gssd, and statd (NSM) upcalls. Do you intend to have separate instances of these daemons for each net namespace?

Actually, yes, I do. Thanks for notice.
Kernel NSM monitor routines will be virtualized.
For idmapd and gssd rpc_pipefs virtualization is required.
I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.

Quoted 51 lines

Quoted 50 lines
Quoted 33 lines
Quoted 33 lines
==============================================================================
"NFS client" virtualization plan:
_RPC layer_:
1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be per net namespace.
_NFS layer_:
1) Net namespace inheritance (current->nsproxy->net have to be used instead of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will give easy access to proper net namespace during RPC transports creation.
2) Make Lockd kthread able to handle requests from different net namespaces.
3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able to handle requests from different net namespaces.
4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per net namespace in addition to global ones.
5) Make NFS proc variables and it's internal content variables to be per net namespace in addition to global ones.
6) Make NFS sysctl variables and it's internal content variables to be per net namespace in addition to global ones.
Quoted 8 lines
Quoted 7 lines
--
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Best regards,
Stanislav Kinsbursky
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Best regards,
Stanislav Kinsbursky

J. Bruce Fields

2011-08-19 19:33 UTC #8 lore raw

Quoted 4 lines · Stanislav Kinsbursky

Actually, yes, I do. Thanks for notice.
Kernel NSM monitor routines will be virtualized.
For idmapd and gssd rpc_pipefs virtualization is required.
I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.

You'll need rpc_pipefs for gssd and the old idmapd.

Also the new idmapper uses request_key. I think there's a little discussion about this in the archives.

--b.

Stanislav Kinsbursky

author

2011-08-22 17:30 UTC #9 lore raw

19.08.2011 23:32, J. Bruce Fields =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

Quoted 5 lines · Stanislav Kinsbursky

Quoted 4 lines
Actually, yes, I do. Thanks for notice.
Kernel NSM monitor routines will be virtualized.
For idmapd and gssd rpc_pipefs virtualization is required. I wasn't going to virtualize rpc_pipefs for now - it was planned to =

be a part of NFS server virtualization, which is planned to be done aft= er NFS client virtualization.

Quoted 6 lines

You'll need rpc_pipefs for gssd and the old idmapd.
Also the new idmapper uses request_key. I think there's a little discussion about this in the archives.

Thanks, Bruce.
Nice to know, that you noticed this discussion.
I'll rpc_pipefs to the todo list for NFS client.

--=20
Best regards,
Stanislav Kinsbursky

Jim Rees

2011-08-22 18:54 UTC #10 lore raw

Stanislav Kinsbursky пишет:

19.08.2011 23:32, J. Bruce Fields пишет:

Quoted 11 lines · Stanislav Kinsbursky

Quoted 4 lines
Actually, yes, I do. Thanks for notice.
Kernel NSM monitor routines will be virtualized.
For idmapd and gssd rpc_pipefs virtualization is required.
I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.
You'll need rpc_pipefs for gssd and the old idmapd.
Also the new idmapper uses request_key. I think there's a little discussion about this in the archives.

Thanks, Bruce.
Nice to know, that you noticed this discussion. I'll rpc_pipefs to the todo list for NFS client.

As of 3.1, rpc_pipefs is also used by the block layout driver and blkmapd.