"VRF support on Linux" / --bind-dev support
Hi,
this set reintroduces the patch for "VRF support on Linux" by implementing --bind-dev <dev> option from 2016 as well as some real live documentation as requested by Gert :)
I wrote a .rst file in the style of the "man-page overhaul project" in the hope that this format is useful and can easily be included.
Best
Max
PATCH 1/2 Add --bind-dev option.
This options allows the user to specify a network device the OpenVPN process should use when making a connection or binding to an address. This translates in setting the SO_BINDTODEVICE option to the corresponding socket (on Linux).
When for example using VRFs on Linux [0] this allows making connections using the non-default VRF and having the tun/tap interface in the default VRF.
Thanks to David Ahern (Cumulus Networks) for insights on this.
--- src/openvpn/init.c | 1 + src/openvpn/options.c | 10 ++++++++++ src/openvpn/options.h | 1 + src/openvpn/socket.c | 10 ++++++++++ src/openvpn/socket.h | 2 ++ 5 files changed, 24 insertions(+)
src/openvpn/init.c +1 -0
@@ -3445,6 +3445,7 @@ do_init_socket_1(struct context *c, const int mode) c->options.rcvbuf, c->options.sndbuf, c->options.mark, + c->options.bind_dev, &c->c2.server_poll_interval, sockflags); }
src/openvpn/options.c +10 -0
@@ -294,6 +294,9 @@ static const char usage_message[] =#if defined(TARGET_LINUX ) &&HAVE_DECL_SO_MARK "--mark value : Mark encrypted packets being sent with value. The mark value\n" " can be matched in policy routing and packetfilter rules.\n" +"--bind-dev dev : Bind to the given device when making connection to a peer or\n" +" listening for connections. This allows sending encrypted packets\n" + " via a VRF present on the system.\n"#endif "--txqueuelen n : Set the tun/tap TX queue length to n (Linux only).\n" #ifdef ENABLE_MEMSTATS
@@ -6064,6 +6067,13 @@ add_option(struct options *options, } } } +#ifdef TARGET_LINUX + else if (streq (p[0], "bind-dev") && p[1]) + { +VERIFY_PERMISSION (OPT_P_SOCKFLAGS ); + options->bind_dev = p[1 ]; + } +#endif else if (streq(p[0], "txqueuelen") && p[1] && !p[2]) {VERIFY_PERMISSION (OPT_P_GENERAL );
src/openvpn/options.h +1 -0
@@ -352,6 +352,7 @@ struct options/* mark value */ int mark; +char *bind_dev;/* socket flags */ unsigned int sockflags;
src/openvpn/socket.c +10 -0
@@ -1138,6 +1138,14 @@ create_socket(struct link_socket *sock, struct addrinfo *addr)/* set socket to --mark packets with given value */ socket_set_mark (sock->sd, sock->mark); +#if defined(TARGET_LINUX ) +if (sock->bind_dev) + { +msg (M_INFO ,"Using bind-dev %s" , sock->bind_dev); +setsockopt (sock->sd,SOL_SOCKET ,SO_BINDTODEVICE , sock->bind_dev,strlen (sock->bind_dev) +1 ); + } +#endif +bind_local (sock, addr->ai_family); }
@@ -1880,6 +1888,7 @@ link_socket_init_phase1(struct link_socket *sock,int rcvbuf, int sndbuf, int mark, + const char *bind_dev, struct event_timeout *server_poll_timeout, unsigned int sockflags) {
@@ -1906,6 +1915,7 @@ link_socket_init_phase1(struct link_socket *sock, sock->sockflags = sockflags; sock->mark = mark; + sock->bind_dev = bind_dev; sock->info.proto = proto; sock->info.af = af;
src/openvpn/socket.h +2 -0
@@ -212,6 +212,7 @@ struct link_socket#define SF_GETADDRINFO_DGRAM (1<<4)unsigned int sockflags;int mark; +const char *bind_dev;/* for stream sockets */ struct stream_buf stream_buf;
@@ -326,6 +327,7 @@ link_socket_init_phase1(struct link_socket *sock,int rcvbuf, int sndbuf, int mark, + const char *bind_dev, struct event_timeout *server_poll_timeout, unsigned int sockflags);/* Reenable uncrustify *INDENT-ON* */
Signature · 1 line
2.1.4
Re: [Openvpn-devel] [PATCH 1/2] Add --bind-dev option.
Hi,
reading this more closely at merging/testing time, I do have a change request...
src/openvpn/options.c add_option(struct options *options, · line 6076
Quoted 7 lines · Maximilian Wilhelm
src/openvpn/options.c +7
-0+#ifdef TARGET_LINUX + else if (streq (p[0], "bind-dev") && p[1]) + { +VERIFY_PERMISSION (OPT_P_SOCKFLAGS ); + options->bind_dev = p[1 ]; + } +#endif
One could argue whether the argument should be changed for IFNAMSIZ here (so we can error-out right away if it's too long). But this is just something to consider.
src/openvpn/socket.c create_socket(struct link_socket *sock, struct addrinfo *addr) · line 1147
Quoted 13 lines
src/openvpn/socket.c +7
-0@@ -1138,6 +1138,14 @@ create_socket(struct link_socket *sock, struct addrinfo *addr)/* set socket to --mark packets with given value */ socket_set_mark (sock->sd, sock->mark); +#if defined(TARGET_LINUX ) +if (sock->bind_dev) + { +msg (M_INFO ,"Using bind-dev %s" , sock->bind_dev); +setsockopt (sock->sd,SOL_SOCKET ,SO_BINDTODEVICE , sock->bind_dev,strlen (sock->bind_dev) +1 ); + } +#endif
Here, we *must* have a return code check, and logging of an error message if setsocktopt() fails.
Imagine someone calling "openvpn --bind-dev eht0" (because he has fat fingers). The current code will silently fail the setsockopt() - because there is no such interface name - but nothing in the logs will show a hint *why* openvpn is just not doing what requested.
Sorry for not spotting this in the first review.
gert
Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress
Gert Doering - Munich, Germany [email protected]...Re: [Openvpn-devel] [PATCH 1/2] Add --bind-dev option.
Anno domini 2020 Gert Doering scripsit:
Hi,
Quoted 4 lines
reading this more closely at merging/testing time, I do have a change request...
On Fri, Jun 26, 2020 at 08:49:44PM +0200, Maximilian Wilhelm wrote:
Quoted 11 lines
src/openvpn/options.c add_option(struct options *options, · line 6076
Quoted 7 lines
src/openvpn/options.c +7
-0+#ifdef TARGET_LINUX + else if (streq (p[0], "bind-dev") && p[1]) + { +VERIFY_PERMISSION (OPT_P_SOCKFLAGS ); + options->bind_dev = p[1 ]; + } +#endif One could argue whether the argument should be changed for IFNAMSIZ here (so we can error-out right away if it's too long). But this is just something to consider.
That might be a bad idea as of upcoming altnames, see
Quoted 17 lines
src/openvpn/socket.c create_socket(struct link_socket *sock, struct addrinfo *addr) · line 1147
Quoted 13 lines
src/openvpn/socket.c +7
-0@@ -1138,6 +1138,14 @@ create_socket(struct link_socket *sock, struct addrinfo *addr)/* set socket to --mark packets with given value */ socket_set_mark (sock->sd, sock->mark); +#if defined(TARGET_LINUX ) +if (sock->bind_dev) + { +msg (M_INFO ,"Using bind-dev %s" , sock->bind_dev); +setsockopt (sock->sd,SOL_SOCKET ,SO_BINDTODEVICE , sock->bind_dev,strlen (sock->bind_dev) +1 ); + } +#endif Here, we *must* have a return code check, and logging of an error message if
setsocktopt()fails.
Quoted 4 lines
Imagine someone calling "openvpn --bind-dev eht0" (because he has fat fingers). The current code will silently fail the
setsockopt()- because there is no such interface name - but nothing in the logs will show a hint *why* openvpn is just not doing what requested.
I'll look into that and add a patch for that.
Best
Max
Signature · 3 lines
"I have to admit I've always suspected that MTBWTF would be a more useful metric of real-world performance." -- Valdis Kletnieks on NANOG
PATCH v2 1/1 "VRF support on Linux" / --bind-dev support
In reply to Maximilian WilhelmHi,
once again, this time with added error handling in the setsockopt(). Sorry for missing this before.
Best
Max
PATCH v2 1/2 Add --bind-dev option.
From: Maximilian Wilhelm <[email protected]...>
This options allows the user to specify a network device the OpenVPN process should use when making a connection or binding to an address. This translates in setting the SO_BINDTODEVICE option to the corresponding socket (on Linux).
When for example using VRFs on Linux [0] this allows making connections using the non-default VRF and having the tun/tap interface in the default VRF.
Thanks to David Ahern (Cumulus Networks) for insights on this.
--- src/openvpn/init.c | 1 + src/openvpn/options.c | 10 ++++++++++ src/openvpn/options.h | 1 + src/openvpn/socket.c | 14 ++++++++++++++ src/openvpn/socket.h | 2 ++ 5 files changed, 28 insertions(+)
src/openvpn/init.c +1 -0
@@ -3445,6 +3445,7 @@ do_init_socket_1(struct context *c, const int mode) c->options.rcvbuf, c->options.sndbuf, c->options.mark, + c->options.bind_dev, &c->c2.server_poll_interval, sockflags); }
src/openvpn/options.c +10 -0
@@ -294,6 +294,9 @@ static const char usage_message[] =#if defined(TARGET_LINUX ) &&HAVE_DECL_SO_MARK "--mark value : Mark encrypted packets being sent with value. The mark value\n" " can be matched in policy routing and packetfilter rules.\n" +"--bind-dev dev : Bind to the given device when making connection to a peer or\n" +" listening for connections. This allows sending encrypted packets\n" + " via a VRF present on the system.\n"#endif "--txqueuelen n : Set the tun/tap TX queue length to n (Linux only).\n" #ifdef ENABLE_MEMSTATS
@@ -6084,6 +6087,13 @@ add_option(struct options *options, } } } +#ifdef TARGET_LINUX + else if (streq (p[0], "bind-dev") && p[1]) + { +VERIFY_PERMISSION (OPT_P_SOCKFLAGS ); + options->bind_dev = p[1 ]; + } +#endif else if (streq(p[0], "txqueuelen") && p[1] && !p[2]) {VERIFY_PERMISSION (OPT_P_GENERAL );
src/openvpn/options.h +1 -0
@@ -352,6 +352,7 @@ struct options/* mark value */ int mark; +char *bind_dev;/* socket flags */ unsigned int sockflags;
src/openvpn/socket.c +14 -0
@@ -1138,6 +1138,18 @@ create_socket(struct link_socket *sock, struct addrinfo *addr)/* set socket to --mark packets with given value */ socket_set_mark (sock->sd, sock->mark); +#if defined(TARGET_LINUX ) +if (sock->bind_dev) + { +msg (M_INFO ,"Using bind-dev %s" , sock->bind_dev); +if (setsockopt (sock->sd,SOL_SOCKET ,SO_BINDTODEVICE , sock->bind_dev,strlen (sock->bind_dev) +1 ) !=0 ) + { +msg (M_WARN |M_ERRNO ,"WARN: setsockopt SO_BINDTODEVICE=%s failed" , sock->bind_dev); + } + + } +#endif +bind_local (sock, addr->ai_family); }
@@ -1880,6 +1892,7 @@ link_socket_init_phase1(struct link_socket *sock,int rcvbuf, int sndbuf, int mark, + const char *bind_dev, struct event_timeout *server_poll_timeout, unsigned int sockflags) {
@@ -1906,6 +1919,7 @@ link_socket_init_phase1(struct link_socket *sock, sock->sockflags = sockflags; sock->mark = mark; + sock->bind_dev = bind_dev; sock->info.proto = proto; sock->info.af = af;
src/openvpn/socket.h +2 -0
@@ -212,6 +212,7 @@ struct link_socket#define SF_GETADDRINFO_DGRAM (1<<4)unsigned int sockflags;int mark; +const char *bind_dev;/* for stream sockets */ struct stream_buf stream_buf;
@@ -326,6 +327,7 @@ link_socket_init_phase1(struct link_socket *sock,int rcvbuf, int sndbuf, int mark, + const char *bind_dev, struct event_timeout *server_poll_timeout, unsigned int sockflags);/* Reenable uncrustify *INDENT-ON* */
Signature · 1 line
2.1.4
[Openvpn-devel] [PATCH applied] Re: Add --bind-dev option.
Your patch has been applied to the master branch.
I have stared at the code (looks reasonable) and run t_client tests on Linux and FreeBSD (pass, no major surprise).
I have not actually tested the functionality, because I do not have a test rig with VRF (or multiple ethernet links) around. From what I found so far, what SO_BINDTODEVICE does is "ensure that packets sent via that socket go out via the specified interface, and only that interface" and "packets
coming in have been received on that interface". Besides "a network
interface" you can also specify the name of a "VRF device" (which seems to be a Linux abstraction to group together "Interfaces and Routes" under one umbrella).
So you can use this for VRFs ("--bind-dev outer-vrf") or to select one particular interface (*and* routes) inside "global" or "VRF".
I have tested the latter ("--bind-to lo") which makes connection setup *fail* - obviously. Trying to bind to nonexistant devices gives a proper error message in v2 now
2020-06-29 13:27:03 WARN: setsockopt SO_BINDTODEVICE=MyTunnelIsLongerThanYours failed: No such device (errno=19)
Googling for SO_BINDTODEVICE shows lots of confusion on how this is used and what it does, but I consider the kernel documentation to be authoritative (Documentation/networking/vrf.txt) and that says "what you do is correct":
Applications
------------
Applications that are to work within a VRF need to bind their socket to the VRF device:
setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1);
(note the "strlen(dev)+1" part, and "not a struct ifreq")
Your patch has been applied to the master branch.
commit 19d3c602e7a3881cf7c2244b7c40b9958c0b7ebc
Author: Maximilian Wilhelm Date: Mon Jun 29 12:49:07 2020 +0200
Add --bind-dev option.
Signed-off-by: Maximilian Wilhelm <[email protected]...>
Acked-by: Gert Doering <[email protected]...>
Message-Id: <[email protected]...>
URL: https://www.mail-archive.com/search?l=mid&[email protected]...
Signed-off-by: Gert Doering <[email protected]...>--
kind regards,
Gert Doering
PATCH v2 2/2 Documentation for --bind-dev / VRFs on Linux
In reply to Maximilian WilhelmFrom: Maximilian Wilhelm <[email protected]...>
--- doc/man-sections/vrf.rst | 75 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 doc/man-sections/vrf.rst
doc/man-sections/vrf.rst new file +75 -0
@@ -0,0 +1,75 @@ +Virtual Routing and Forwarding +--------------------------------------- + +Options in this section relates to configuration of virtual routing and +forwarding in combination with the underlying operating system. + +As of today this is only supported on Linux, a kernel >= 4.9 is recommended. + +This could come in handy when for example the external network should be only +used as a means to connect to some VPN endpoints and all regular traffic +should only be routed through any tunnel(s). +This could be achieved by setting up a VRF and configuring the interface +connected to the external network to be part of the VRF. The examples below +will cover this setup. + +Another option would be to put the tun/tap interface into a VRF. This could +be done by an up-script which uses the `ip link set` command shown below. + + +**VRF setup with iproute2** + +Create VRF `vrf_external` and map it to routing table `1023` + +:: + + ip link add vrf_external type vrf table 1023 + +Move `eth0` into `vrf_external` + +:: + + ip link set master vrf_external dev eth0 + +Any prefixes configured on `eth0` will be moved from the `main` routing +table into routing table `1023` + + +**VRF setup with ifupdown** + +For Debian based Distributions `ifupdown2` provides an almost drop-in +replacement for `ifupdown` including VRFs and other features. +A configuration for an interface `eth0` being part of VRF `vrf_external` +could look like this: + +:: + + auto eth0 + iface eth0 + address 192.0.2.42/24 + address 2001:db8:08:15::42/64 + gateway 192.0.2.1 + gateway 2001:db8:08:15::1 + vrf vrf_external + + auto vrf_external + iface vrf_external + vrf-table 1023 + + +**OpenVPN config** + +--bind-dev device + Set the device to bind the server socket to to the VRF + + --bind-dev vrf_external + + +**Further reading** + +Wikipedia has nice page one VRFs: https://en.wikipedia.org/wiki/Virtual_routing_and_forwarding + +This talk from the Network Track of FrOSCon 2018 provides an overview about +advanced layer 2 and layer 3 features of Linux + - Slides: https://www.slideshare.net/BarbarossaTM/l2l3-fr-fortgeschrittene-helle-und-dunkle-magie-im-linuxnetzwerkstack + - Video (german): https://media.ccc.de/v/froscon2018-2247-l2_l3_fur_fortgeschrittene_-_helle_und_dunkle_magie_im_linux-netzwerkstack
Signature · 1 line
2.1.4
PATCH 2/2 Documentation for --bind-dev / VRFs on Linux
In reply to Maximilian WilhelmFrom: Maximilian Wilhelm <[email protected]...>
--- doc/man-sections/vrf.rst | 75 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 doc/man-sections/vrf.rst
doc/man-sections/vrf.rst new file +75 -0
@@ -0,0 +1,75 @@ +Virtual Routing and Forwarding +--------------------------------------- + +Options in this section relates to configuration of virtual routing and +forwarding in combination with the underlying operating system. + +As of today this is only supported on Linux, a kernel >= 4.9 is recommended. + +This could come in handy when for example the external network should be only +used as a means to connect to some VPN endpoints and all regular traffic +should only be routed through any tunnel(s). +This could be achieved by setting up a VRF and configuring the interface +connected to the external network to be part of the VRF. The examples below +will cover this setup. + +Another option would be to put the tun/tap interface into a VRF. This could +be done by an up-script which uses the `ip link set` command shown below. + + +**VRF setup with iproute2** + +Create VRF `vrf_external` and map it to routing table `1023` + +:: + + ip link add vrf_external type vrf table 1023 + +Move `eth0` into `vrf_external` + +:: + + ip link set master vrf_external dev eth0 + +Any prefixes configured on `eth0` will be moved from the `main` routing +table into routing table `1023` + + +**VRF setup with ifupdown** + +For Debian based Distributions `ifupdown2` provides an almost drop-in +replacement for `ifupdown` including VRFs and other features. +A configuration for an interface `eth0` being part of VRF `vrf_external` +could look like this: + +:: + + auto eth0 + iface eth0 + address 192.0.2.42/24 + address 2001:db8:08:15::42/64 + gateway 192.0.2.1 + gateway 2001:db8:08:15::1 + vrf vrf_external + + auto vrf_external + iface vrf_external + vrf-table 1023 + + +**OpenVPN config** + +--bind-dev device + Set the device to bind the server socket to to the VRF + + --bind-dev vrf_external + + +**Further reading** + +Wikipedia has nice page one VRFs: https://en.wikipedia.org/wiki/Virtual_routing_and_forwarding + +This talk from the Network Track of FrOSCon 2018 provides an overview about +advanced layer 2 and layer 3 features of Linux + - Slides: https://www.slideshare.net/BarbarossaTM/l2l3-fr-fortgeschrittene-helle-und-dunkle-magie-im-linuxnetzwerkstack + - Video (german): https://media.ccc.de/v/froscon2018-2247-l2_l3_fur_fortgeschrittene_-_helle_und_dunkle_magie_im_linux-netzwerkstack
Signature · 1 line
2.1.4