Skip to contentSkip to conversation

Support for VRF in NFS?

DISCUSSION Ben Greear 4 participants 10 messages Nov 6, 2018 – Nov 12, 2018 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

Ben Greear

author
#0 lore raw

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#1 lore raw
Quoted 9 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

--b.

author
#2 lore raw
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#3 lore raw
Quoted 16 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Quoted 8 lines

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

--b.

author
#4 lore raw
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations. These bind-local-IP patches are not upstream and were rejected in the past as un-wanted. I'm hoping VRF support would be more acceptable.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#5 lore raw

Hi,

Quoted 40 lines · Ben Greear
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic.  It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Quoted 7 lines

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations.  These bind-local-IP patches are not upstream and were rejected in the past as un-wanted.  I'm hoping VRF support would be more acceptable.

Thanks,
Ben

For similar reasons David Windsor has been looking at some extensions for DLM along these lines. Improving our ability to test seems to me like it should be a good thing to do - in both cases. Likewise VRF support seems also like it should be useful in a number of contexts.

Do you have a reference to your past work? I think it would be interesting to get some discussion going here - maybe it would be possible to have some common approach between kernel-side socket users, and/or bounce some ideas around,

Steve.

author
#6 lore raw
Quoted 52 lines · Steven Whitehouse

Hi,

Quoted 40 lines · Ben Greear
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Quoted 7 lines

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations. These bind-local-IP patches are not upstream and were rejected in the past as un-wanted. I'm hoping VRF support would be more acceptable.

Thanks,
Ben

Quoted 5 lines

For similar reasons David Windsor has been looking at some extensions for DLM along these lines. Improving our ability to test seems to me like it should be a good thing to do - in both cases. Likewise VRF support seems also like it should be useful in a number of contexts.

Do you have a reference to your past work? I think it would be interesting to get some discussion going here - maybe it would be possible to have some common approach between kernel-side socket users, and/or bounce some ideas around,

Steve.

Here is an old thread on the topic.

https://www.spinics.net/lists/linux-nfs/msg34811.html

My patches are in all my 'ct' kernels, and the needed patches are also in my nfs-utils repo.

My 4.19 tree has just been ported, so no idea if it works or not.

https://github.com/greearb?tab=repositories

My patch work fine using routing table rules without VRF, but they will not work with VRF.

Thanks,
Ben

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com

A message that is not in the archive (sent off-list or not yet received): CAGKpdE9YMQNisieoOUbenwA8a7PhTngLTX=v8Bkznfqc2816dQ@mail.gmail.com

#8 lore raw

Forwarding to linux-nfs because I forgot to disable HTML formatting when originally replying.

---------- Forwarded message ---------
From: David Windsor <[email protected]>
Date: Mon, Nov 12, 2018 at 9:13 AM
Subject: Re: Support for VRF in NFS?
To: <[email protected]>
Cc: swhiteho <[email protected]>, <[email protected]>, <[email protected]>

Hi,

Quoted 52 lines · Ben Greear
Quoted 48 lines · Steven Whitehouse

Hi,

Quoted 44 lines · Ben Greear
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations. These bind-local-IP patches are not upstream and were rejected in the past as un-wanted. I'm hoping VRF support would be more acceptable.

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

Thanks,
Ben

For similar reasons David Windsor has been looking at some extensions for DLM along these lines. Improving our ability to test seems to me like it should be a good thing to do - in both cases. Likewise VRF support seems also like it should be useful in a number of contexts.

Do you have a reference to your past work? I think it would be interesting to get some discussion going here - maybe it would be possible to have some common approach between kernel-side socket users, and/or bounce some ideas around,

Did you have anything specific in mind here? AFAICT, the separation provided by VRF wouldn't buy us much over using SO_MARK/iptables with DLM.

With respect to a common approach between kernel-side socket users, I would be interested in seeing if we could come up with something here. In the case of DLM, the changes needed to support *just* multihoming could perhaps be abstracted into some sort of higher layer, but the bits required to support failover to other interfaces are fairly specific to the DLM protocol itself (i.e. adding sequence numbers to DLM messages, etc.).

Thanks,
David

Quoted 22 lines
Quoted 2 lines

Steve.

Here is an old thread on the topic.

https://www.spinics.net/lists/linux-nfs/msg34811.html

My patches are in all my 'ct' kernels, and the needed patches are also in my nfs-utils repo.

My 4.19 tree has just been ported, so no idea if it works or not.

https://github.com/greearb?tab=repositories

My patch work fine using routing table rules without VRF, but they will not work with VRF.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

author
#9 lore raw
Quoted 73 lines · David Windsor

Forwarding to linux-nfs because I forgot to disable HTML formatting when originally replying.

---------- Forwarded message ---------
From: David Windsor <[email protected]>
Date: Mon, Nov 12, 2018 at 9:13 AM
Subject: Re: Support for VRF in NFS?
To: <[email protected]>
Cc: swhiteho <[email protected]>, <[email protected]>, <[email protected]>

Hi,

Quoted 58 lines · Ben Greear
Quoted 54 lines · Steven Whitehouse

Hi,

Quoted 40 lines · Ben Greear
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Quoted 7 lines

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations. These bind-local-IP patches are not upstream and were rejected in the past as un-wanted. I'm hoping VRF support would be more acceptable.

Thanks,
Ben

For similar reasons David Windsor has been looking at some extensions for DLM along these lines. Improving our ability to test seems to me like it should be a good thing to do - in both cases. Likewise VRF support seems also like it should be useful in a number of contexts.

Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

Do you have a reference to your past work? I think it would be interesting to get some discussion going here - maybe it would be possible to have some common approach between kernel-side socket users, and/or bounce some ideas around,

Did you have anything specific in mind here? AFAICT, the separation provided by VRF wouldn't buy us much over using SO_MARK/iptables with DLM.

I don't know anything in particular about DLM (is this Distributed Lock Manager?).

In general, having enough ip tables rules to support 2000 virtual interfaces/mounts appears less efficient than using VRF, and VRF gives some additional abilities such as allowing duplicated IP addresses, so that is why we are interested in VRF.

My existing patches do support multi-homing using iptables rules though.

Thanks,
Ben

Quoted 35 lines

With respect to a common approach between kernel-side socket users, I would be interested in seeing if we could come up with something here. In the case of DLM, the changes needed to support *just* multihoming could perhaps be abstracted into some sort of higher layer, but the bits required to support failover to other interfaces are fairly specific to the DLM protocol itself (i.e. adding sequence numbers to DLM messages, etc.).

Thanks,
David

Quoted 22 lines
Quoted 2 lines

Steve.

Here is an old thread on the topic.

https://www.spinics.net/lists/linux-nfs/msg34811.html

My patches are in all my 'ct' kernels, and the needed patches are also in my nfs-utils repo.

My 4.19 tree has just been ported, so no idea if it works or not.

https://github.com/greearb?tab=repositories

My patch work fine using routing table rules without VRF, but they will not work with VRF.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

Signature · 2 lines
Ben Greear <[email protected]>
Candela Technologies Inc  http://www.candelatech.com
#10 lore raw
Quoted 81 lines · Ben Greear
Quoted 77 lines · David Windsor

Forwarding to linux-nfs because I forgot to disable HTML formatting when originally replying.

---------- Forwarded message ---------
From: David Windsor <[email protected]>
Date: Mon, Nov 12, 2018 at 9:13 AM
Subject: Re: Support for VRF in NFS?
To: <[email protected]>
Cc: swhiteho <[email protected]>, <[email protected]>, <[email protected]>

Hi,

Quoted 60 lines · Ben Greear
Quoted 56 lines · Steven Whitehouse

Hi,

Quoted 40 lines · Ben Greear
Quoted 23 lines · J. Bruce Fields
Quoted 22 lines · Ben Greear
Quoted 6 lines · J. Bruce Fields
Quoted 5 lines · Ben Greear

Hello,

I made a stab at implementing VRF support in NFS, but it appears fairly complicated and I ended up reverting my changes....

Quoted 6 lines
Quoted 4 lines

Is anyone working on this?

And, if not, if anyone would like to be sponsored to work on this, please let me know.

Um, sorry--what's VRF?

Virtual Router logic. It is sort of like network stack containers, and has been solid and fully featured in the kernel since 4.16 or so.

In the end, you effectively need to call the logic that SO_BINDTODEVICE calls on the socket before binding to an IP.

Quoted 6 lines
Quoted 2 lines

The NFS and RPC logic is a giant tangled mess to my eyes, so hoping I could bribe someone else to do it :)

So it's not enough to support network namespaces?

What's your motivation for this?

Network namespaces are difficult to uses for lots of use cases, and thus VRF was born.

My own motivation is that it allows me to make hundreds or thousands of individual NFS mounts from local mac-vlan (or other virtual/physical interfaces), for testing purposes.

Quoted 7 lines

Similar to my patch set that binds to local IP address, which gives similar feature set for non-VRF configurations. These bind-local-IP patches are not upstream and were rejected in the past as un-wanted. I'm hoping VRF support would be more acceptable.

Thanks,
Ben

For similar reasons David Windsor has been looking at some extensions for DLM along these lines. Improving our ability to test seems to me like it should be a good thing to do - in both cases. Likewise VRF support seems also like it should be useful in a number of contexts.

Do you have a reference to your past work? I think it would be interesting to get some discussion going here - maybe it would be possible to have some common approach between kernel-side socket users, and/or bounce some ideas around,

Quoted 6 lines
Quoted 3 lines

Did you have anything specific in mind here? AFAICT, the separation provided by VRF wouldn't buy us much over using SO_MARK/iptables with DLM.

I don't know anything in particular about DLM (is this Distributed Lock Manager?).

Yes. DLM is used in clustered environments to provide access to resources. The DLM runs on every node in the cluster and normally uses TCP as its L4 transport (SCTP is also supported, but not widely used). Currently, DLM over TCP is restricted to only supporting one IP address per node.

I'm working on extensions to DLM to allow it to support multihoming at the TCP layer, as well as adding extensions to make it more robust in the face of network failures (retransmitting lost messages, automatic failover to the next available interface, etc.).

Quoted 7 lines

In general, having enough ip tables rules to support 2000 virtual interfaces/mounts appears less efficient than using VRF, and VRF gives some additional abilities such as allowing duplicated IP addresses, so that is why we are interested in VRF.

My existing patches do support multi-homing using iptables rules though.

Thanks. I'm currently looking at these and seeing if any common bits can be extracted and made into a more generic interface.

Quoted 42 lines

Thanks,
Ben

Quoted 9 lines

With respect to a common approach between kernel-side socket users, I would be interested in seeing if we could come up with something here. In the case of DLM, the changes needed to support *just* multihoming could perhaps be abstracted into some sort of higher layer, but the bits required to support failover to other interfaces are fairly specific to the DLM protocol itself (i.e. adding sequence numbers to DLM messages, etc.).

Quoted 26 lines

Thanks,
David

Quoted 22 lines
Quoted 2 lines

Steve.

Here is an old thread on the topic.

https://www.spinics.net/lists/linux-nfs/msg34811.html

My patches are in all my 'ct' kernels, and the needed patches are also in my nfs-utils repo.

My 4.19 tree has just been ported, so no idea if it works or not.

https://github.com/greearb?tab=repositories

My patch work fine using routing table rules without VRF, but they will not work with VRF.

Thanks,
Ben

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com

--
Ben Greear <[email protected]>
Candela Technologies Inc http://www.candelatech.com