Repair special-casing of EEXIST for Linux/SITNL route install
PATCH 1/1 Repair special-casing of EEXIST for Linux/SITNL route install
The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".
This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.
So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to route_add(), assign "2" on "-EEXIST" (and log appropriate message).
(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)
--- src/openvpn/networking_sitnl.c | 6 +----- src/openvpn/route.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 7 deletions(-)
src/openvpn/networking_sitnl.c +1 -5
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family, } ret =sitnl_send (&req.n,0 ,0 ,NULL ,NULL ); -if (ret == -EEXIST ) - { - ret =0 ; - } err:return ret; }
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst, scope =RT_SCOPE_LINK ; } -return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE |NLM_F_REPLACE , ifindex, +return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE , ifindex, af_family, dst, prefixlen, gw, table, metric, scope,RTPROT_BOOT ,RTN_UNICAST ); }
src/openvpn/route.c +8 -2
@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r, } status =1 ; -if (net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), - &r->gateway, iface,0 , metric) <0 ) +int ret =net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), + &r->gateway, iface,0 , metric); +if (ret == -EEXIST ) + { +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists" ); + status =2 ; + } +else if (ret <0 ) {msg (M_WARN ,"ERROR: Linux route add command failed" ); status =0 ;
Signature · 1 line
2.38.2
Hi,
Netlink is antonio's realm, but fwiw, I gave it a whirl:
Quoted 77 lines · Gert Doering
The code in
sitnl_route_set()used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".This is arguably a reasonable approach, but needs to handled higher up - if the low level
add_route()function say "no error", we will try to remove that route later on indelete_route(), possibly removing someone else's "already existing" route then.So:
- remove special case insitnl_route_set()
- do not passNLM_F_REPLACEflag tositnl_route_set()call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling toroute_add(), assign "2" on
"-EEXIST"
(and log appropriate message).(Note:
sitnl_route_set()is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)Signed-off-by Gert Doering <[email protected]...>--- src/openvpn/networking_sitnl.c | 6 +----- src/openvpn/route.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 7 deletions(-)src/openvpn/networking_sitnl.c +0
-4@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex,sa_family_t af_family, } ret =sitnl_send (&req.n,0 ,0 ,NULL ,NULL ); -if (ret == -EEXIST ) - { - ret =0 ; - } err:return ret;}
src/openvpn/networking_sitnl.c +1
-1@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst, scope =RT_SCOPE_LINK ; } - return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE, ifindex, + return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex, af_family, dst, prefixlen, gw, table, metric,scope,
RTPROT_BOOT, RTN_UNICAST);}
src/openvpn/route.c +8
-2@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r, } status =1 ; -if (net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), - &r->gateway, iface,0 , metric) <0 ) + int ret = net_route_v4_add(ctx, &r->network,netmask_to_netbits2 (r->netmask), + &r->gateway, iface,0 , metric); +if (ret == -EEXIST ) + { +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists"); + status =2 ; + } +else if (ret <0 )
Quoted 4 lines
{
msg(M_WARN, "ERROR: Linux route add command failed");
status = 0;Looks good and works as expected. Got CONNECTED,ROUTE_ERROR on real errors, not if the route exists. Also, an existing route no longer gets removed.
net_route_v4_add is also used in dco.c but there the return value is not checked and should not be affected by the change as far as I can tell..
A couple of things:
- call to net_route_v6_add() in route_add_ipv6() also could benefit from the same treatment
- net_route_v?_add() using iproute2 always returns success (0). We can't distinguish between "exists" and other errors in that case, but still it should not return unconditional success.
BTW, the same functions in networking_freebsd.c returns 0 on success, +1 on error, not < 0. Not used in route.c and return value not checked in dco.c so could be cleaned up in 2.7. I notice Antonio prefers -ve for error, but that doesn't always work in cross-platform code. On Windows, some error codes are > 2^31.
One thing to note: when a route exists here we log with D_ROUTE and real errors with M_WARN. While this should be preferred, we do not do the same in other route methods like IPAPI or service where such a distinction is possible. Cases where existing route cannot be reliably detected, we have no option. Making all route methods log consistently could be a part of the planned re-write/fixup in 2.7.
Selva
Hi,
for the netlink/sitnl bits: this makes sense to me.
I agree with Selva that the v6 variant could benefit from the same treatment.
However, this patch can also be hacked on its own
Quoted 71 lines · Gert Doering
The code in
sitnl_route_set()used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".This is arguably a reasonable approach, but needs to handled higher up - if the low level
add_route()function say "no error", we will try to remove that route later on indelete_route(), possibly removing someone else's "already existing" route then.So:
- remove special case insitnl_route_set()
- do not passNLM_F_REPLACEflag tositnl_route_set()call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling toroute_add(), assign "2" on "-EEXIST" (and log appropriate message).(Note:
sitnl_route_set()is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)Signed-off-by Gert Doering <[email protected]...>--- src/openvpn/networking_sitnl.c | 6 +----- src/openvpn/route.c | 10 ++++++++-- 2 files changed, 9 insertions(+), 7 deletions(-)src/openvpn/networking_sitnl.c +1
-5@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family, } ret =sitnl_send (&req.n,0 ,0 ,NULL ,NULL ); -if (ret == -EEXIST ) - { - ret =0 ; - } err:return ret; }@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst, scope =RT_SCOPE_LINK ; } -return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE |NLM_F_REPLACE , ifindex, +return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE , ifindex, af_family, dst, prefixlen, gw, table, metric, scope,RTPROT_BOOT ,RTN_UNICAST ); }src/openvpn/route.c +8
-2@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r, } status =1 ; -if (net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), - &r->gateway, iface,0 , metric) <0 ) +int ret =net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), + &r->gateway, iface,0 , metric); +if (ret == -EEXIST ) + { +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists" ); + status =2 ; + } +else if (ret <0 ) {msg (M_WARN ,"ERROR: Linux route add command failed" ); status =0 ;
Signature · 1 line
Antonio Quartulli
Quoted 10 lines · Antonio Quartulli
Hi,
for the netlink/sitnl bits: this makes sense to me.
I agree with Selva that the v6 variant could benefit from the same treatment.However, this patch can also be hacked on its own
Not so fast.
As it stands the patch introduces a bug that was not there: existing v6 routes can cause CONENCTED,ROUTE_ERROR instead of CONNECTED,SUCCESS. That's why I did not ACK it.
Not sure I understand "can also be hacked alone".
Selva
Hi,
Quoted 1 line · Selva Nair
Not sure I understand "can also be hacked alone".
I think that was intended to be "acked" alone :-)
Anyway, thanks for spotting this, and shame on me for only testing v4 ("the v6 stuff is new and shiny and has none of these problems") - but of course, the v6 netlink stuff was written together with the v4 netlink stuff, and shares traits.
I'll send a v2 with v6 fixed and tested.
gert
Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress
Gert Doering - Munich, Germany [email protected]...Quoted 8 lines · Gert Doering
The code in
sitnl_route_set()used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".This is arguably a reasonable approach, but needs to handled higher up - if the low level
add_route()function say "no error", we will try to remove that route later on indelete_route(), possibly removing someone else's "already existing" route then.
Found one more case of route addition flag not correctly recorded -- RL_DID_LOCAL used for the direct route to the server. I will wait for the final shape and form of this one before sending a patch.
Selva
PATCH v2 1/1 Repair special-casing of EEXIST for Linux/SITNL route install
In reply to Gert DoeringThe code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".
This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.
So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to add_route(), assign "2" on "-EEXIST" (and log appropriate message).
(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)
v2: use RTA_ macros, also adjust add_route_ipv6()
--- src/openvpn/networking_sitnl.c | 6 +----- src/openvpn/route.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 11 deletions(-)
src/openvpn/networking_sitnl.c +1 -5
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family, } ret =sitnl_send (&req.n,0 ,0 ,NULL ,NULL ); -if (ret == -EEXIST ) - { - ret =0 ; - } err:return ret; }
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst, scope =RT_SCOPE_LINK ; } -return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE |NLM_F_REPLACE , ifindex, +return sitnl_route_set (RTM_NEWROUTE ,NLM_F_CREATE , ifindex, af_family, dst, prefixlen, gw, table, metric, scope,RTPROT_BOOT ,RTN_UNICAST ); }
src/openvpn/route.c +18 -6
@@ -1599,8 +1599,14 @@ add_route(struct route_ipv4 *r, } status =RTA_SUCCESS ; -if (net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), - &r->gateway, iface,0 , metric) <0 ) +int ret =net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), + &r->gateway, iface,0 , metric); +if (ret == -EEXIST ) + { +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists" ); + status =RTA_EEXIST ; + } +else if (ret <0 ) {msg (M_WARN ,"ERROR: Linux route add command failed" ); status =RTA_ERROR ;
@@ -1963,11 +1969,17 @@ add_route_ipv6(struct route_ipv6 *r6, const struct tuntap *tt, } status =RTA_SUCCESS ; -if (net_route_v6_add (ctx, &r6->network, r6->netbits, - gateway_needed ? &r6->gateway :NULL , device,0 , - metric) <0 ) +int ret =net_route_v6_add (ctx, &r6->network, r6->netbits, + gateway_needed ? &r6->gateway :NULL , + device,0 , metric); +if (ret == -EEXIST ) { -msg (M_WARN ,"ERROR: Linux IPv6 route can't be added" ); +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists" ); + status =RTA_EEXIST ; + } +else if (ret <0 ) + { +msg (M_WARN ,"ERROR: Linux route add command failed" ); status =RTA_ERROR ; }
Signature · 1 line
2.38.2
Hi,
Quoted 17 lines · Gert Doering
The code in
sitnl_route_set()used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".This is arguably a reasonable approach, but needs to handled higher up - if the low level
add_route()function say "no error", we will try to remove that route later on indelete_route(), possibly removing someone else's "already existing" route then.So:
- remove special case insitnl_route_set()
- do not passNLM_F_REPLACEflag tositnl_route_set()call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling toadd_route(), assign "2" on
"-EEXIST"
(and log appropriate message).
Quoted 6 lines
(Note:
sitnl_route_set()is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)v2: use
RTA_macros, also adjustadd_route_ipv6()
Looks good now :)
Quoted 67 lines
Signed-off-by Gert Doering <[email protected]...>--- src/openvpn/networking_sitnl.c | 6 +----- src/openvpn/route.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 11 deletions(-)src/openvpn/networking_sitnl.c +0
-4@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex,sa_family_t af_family, } ret =sitnl_send (&req.n,0 ,0 ,NULL ,NULL ); -if (ret == -EEXIST ) - { - ret =0 ; - } err:return ret;}
src/openvpn/networking_sitnl.c +1
-1@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst, scope =RT_SCOPE_LINK ; } - return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE, ifindex, + return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex, af_family, dst, prefixlen, gw, table, metric,scope,
RTPROT_BOOT, RTN_UNICAST);}
src/openvpn/route.c +8
-2@@ -1599,8 +1599,14 @@ add_route(struct route_ipv4 *r, } status =RTA_SUCCESS ; -if (net_route_v4_add (ctx, &r->network,netmask_to_netbits2 (r->netmask), - &r->gateway, iface,0 , metric) <0 ) + int ret = net_route_v4_add(ctx, &r->network,netmask_to_netbits2 (r->netmask), + &r->gateway, iface,0 , metric); +if (ret == -EEXIST ) + { +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists"); + status =RTA_EEXIST ; + } +else if (ret <0 ){ msg(M_WARN, "ERROR: Linux route add command failed"); status = RTA_ERROR;src/openvpn/route.c +0
-2@@ -1963,11 +1969,17 @@ add_route_ipv6(struct route_ipv6 *r6, const structtuntap *tt, } status =RTA_SUCCESS ; - if (net_route_v6_add(ctx, &r6->network, r6->netbits, - gateway_needed ? &r6->gateway : NULL, device, 0,
src/openvpn/route.c add_route_ipv6(struct route_ipv6 *r6, const struct · line 1980
Quoted 10 lines
src/openvpn/route.c +5
-2- metric) < 0) +int ret =net_route_v6_add (ctx, &r6->network, r6->netbits, + gateway_needed ? &r6->gateway :NULL , + device,0 , metric); +if (ret == -EEXIST ) { -msg (M_WARN ,"ERROR: Linux IPv6 route can't be added" ); +msg (D_ROUTE ,"NOTE: Linux route add command failed because route exists");
Even the word "Linux" is redundant here and elsewhere, but we need more changes to make logging consistent and avoid duplicates. Something for 2.7.
Quoted 8 lines
src/openvpn/route.c +5
-0+ status =RTA_EEXIST ; + } +else if (ret <0 ) + { +msg (M_WARN ,"ERROR: Linux route add command failed" );status = RTA_ERROR; }
For good measure, ran through all the client & server side tests on Linux (other platforms not affected). Since this is where I developed and tested the patch, no surprises :-)
Thanks to having RTA_SUCCESS/RTA_EEXIST now, doing a double-review again ("did I get all the numbers right?") was also very straightforward.
Patch has been applied to the master and release/2.6 branch.
commit adc54f483b210484ff1488e01c8aee1b2b0ea477 (master) commit eb85961e72c927256ce9267a43f3519d3ab4edcb (release/2.6)
Author: Gert Doering Date: Wed Jan 18 08:46:33 2023 +0100
Repair special-casing of EEXIST for Linux/SITNL route install
Signed-off-by: Gert Doering <[email protected]...>
Acked-by: Selva Nair <[email protected]...>
Message-Id: <[email protected]...>
URL: https://www.mail-archive.com/[email protected]/msg26046.html
Signed-off-by: Gert Doering <[email protected]...>--
kind regards,
Gert Doering