Skip to contentSkip to conversation

Repair special-casing of EEXIST for Linux/SITNL route install

PATCH Superseded Gert Doering 3 participants 9 messages Jan 11, 2023 – Jan 19, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Repair special-casing of EEXIST for Linux/SITNL route install

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to route_add(), assign "2" on "-EEXIST" (and log appropriate message).

(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)

---
 src/openvpn/networking_sitnl.c |  6 +-----
 src/openvpn/route.c            | 10 ++++++++--
 2 files changed, 9 insertions(+), 7 deletions(-)
src/openvpn/networking_sitnl.c +1 -5
index fe124616..92f30044 100644
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family,
     }
 
     ret = sitnl_send(&req.n, 0, 0, NULL, NULL);
-    if (ret == -EEXIST)
-    {
-        ret = 0;
-    }
 err:
     return ret;
 }
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst,
         scope = RT_SCOPE_LINK;
     }
 
-    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE, ifindex,
+    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex,
                            af_family, dst, prefixlen, gw, table, metric, scope,
                            RTPROT_BOOT, RTN_UNICAST);
 }
src/openvpn/route.c +8 -2
index 99f948ba..2db127bb 100644
@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r,
     }
 
     status = 1;
-    if (net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
-                         &r->gateway, iface, 0, metric) < 0)
+    int ret = net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
+                               &r->gateway, iface, 0, metric);
+    if (ret == -EEXIST)
+    {
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route exists");
+        status = 2;
+    }
+    else if (ret < 0)
     {
         msg(M_WARN, "ERROR: Linux route add command failed");
         status = 0;
Signature · 1 line
2.38.2
#1 lore raw

Hi,

Netlink is antonio's realm, but fwiw, I gave it a whirl:

Quoted 77 lines · Gert Doering

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to route_add(), assign "2" on
"-EEXIST"
(and log appropriate message).

(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)

---
 src/openvpn/networking_sitnl.c |  6 +-----
 src/openvpn/route.c            | 10 ++++++++--
 2 files changed, 9 insertions(+), 7 deletions(-)
src/openvpn/networking_sitnl.c +0 -4
index fe124616..92f30044 100644
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex,
 sa_family_t af_family,
     }
 
     ret = sitnl_send(&req.n, 0, 0, NULL, NULL);
-    if (ret == -EEXIST)
-    {
-        ret = 0;
-    }
 err:
     return ret;

}

src/openvpn/networking_sitnl.c +1 -1
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t
 af_family, const void *dst,
         scope = RT_SCOPE_LINK;
     }
 
-    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE,
 ifindex,
+    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex,
                            af_family, dst, prefixlen, gw, table, metric,

scope,

                            RTPROT_BOOT, RTN_UNICAST);

}

src/openvpn/route.c +8 -2
index 99f948ba..2db127bb 100644
@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r,
     }
 
     status = 1;
-    if (net_route_v4_add(ctx, &r->network,
 netmask_to_netbits2(r->netmask),
-                         &r->gateway, iface, 0, metric) < 0)
+    int ret = net_route_v4_add(ctx, &r->network,
 netmask_to_netbits2(r->netmask),
+                               &r->gateway, iface, 0, metric);
+    if (ret == -EEXIST)
+    {
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route
 exists");
+        status = 2;
+    }
+    else if (ret < 0)
Quoted 4 lines
     {
         msg(M_WARN, "ERROR: Linux route add command failed");
         status = 0;

Looks good and works as expected. Got CONNECTED,ROUTE_ERROR on real errors, not if the route exists. Also, an existing route no longer gets removed.

net_route_v4_add is also used in dco.c but there the return value is not checked and should not be affected by the change as far as I can tell..

A couple of things:

- call to net_route_v6_add() in route_add_ipv6() also could benefit from the same treatment
- net_route_v?_add() using iproute2 always returns success (0). We can't distinguish between "exists" and other errors in that case, but still it should not return unconditional success.

BTW, the same functions in networking_freebsd.c returns 0 on success, +1 on error, not < 0. Not used in route.c and return value not checked in dco.c so could be cleaned up in 2.7. I notice Antonio prefers -ve for error, but that doesn't always work in cross-platform code. On Windows, some error codes are > 2^31.

One thing to note: when a route exists here we log with D_ROUTE and real errors with M_WARN. While this should be preferred, we do not do the same in other route methods like IPAPI or service where such a distinction is possible. Cases where existing route cannot be reliably detected, we have no option. Making all route methods log consistently could be a part of the planned re-write/fixup in 2.7.

Selva

Acked-by on v1 1/1
#2 lore raw
In reply to Gert Doering

Hi,

for the netlink/sitnl bits: this makes sense to me.
I agree with Selva that the v6 variant could benefit from the same treatment.

However, this patch can also be hacked on its own

Quoted 71 lines · Gert Doering

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to route_add(), assign "2" on "-EEXIST" (and log appropriate message).

(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)

---
  src/openvpn/networking_sitnl.c |  6 +-----
  src/openvpn/route.c            | 10 ++++++++--
  2 files changed, 9 insertions(+), 7 deletions(-)
src/openvpn/networking_sitnl.c +1 -5
index fe124616..92f30044 100644
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family,
      }
 
      ret = sitnl_send(&req.n, 0, 0, NULL, NULL);
-    if (ret == -EEXIST)
-    {
-        ret = 0;
-    }
  err:
      return ret;
  }
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst,
          scope = RT_SCOPE_LINK;
      }
 
-    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE, ifindex,
+    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex,
                             af_family, dst, prefixlen, gw, table, metric, scope,
                             RTPROT_BOOT, RTN_UNICAST);
  }
src/openvpn/route.c +8 -2
index 99f948ba..2db127bb 100644
@@ -1594,8 +1594,14 @@ add_route(struct route_ipv4 *r,
      }
 
      status = 1;
-    if (net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
-                         &r->gateway, iface, 0, metric) < 0)
+    int ret = net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
+                               &r->gateway, iface, 0, metric);
+    if (ret == -EEXIST)
+    {
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route exists");
+        status = 2;
+    }
+    else if (ret < 0)
      {
          msg(M_WARN, "ERROR: Linux route add command failed");
          status = 0;
Signature · 1 line
Antonio Quartulli
#3 lore raw
Quoted 10 lines · Antonio Quartulli

Hi,

for the netlink/sitnl bits: this makes sense to me.
I agree with Selva that the v6 variant could benefit from the same treatment.

However, this patch can also be hacked on its own

Not so fast.
As it stands the patch introduces a bug that was not there: existing v6 routes can cause CONENCTED,ROUTE_ERROR instead of CONNECTED,SUCCESS. That's why I did not ACK it.

Not sure I understand "can also be hacked alone".

Selva

#4 lore raw

Hi,

Quoted 1 line · Selva Nair

Not sure I understand "can also be hacked alone".

I think that was intended to be "acked" alone :-)

Anyway, thanks for spotting this, and shame on me for only testing v4 ("the v6 stuff is new and shiny and has none of these problems") - but of course, the v6 netlink stuff was written together with the v4 netlink stuff, and shares traits.

I'll send a v2 with v6 fixed and tested.

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#5 lore raw
In reply to Gert Doering
Quoted 8 lines · Gert Doering

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

Found one more case of route addition flag not correctly recorded -- RL_DID_LOCAL used for the direct route to the server. I will wait for the final shape and form of this one before sending a patch.

Selva

#6 lore raw

PATCH v2 1/1 Repair special-casing of EEXIST for Linux/SITNL route install

In reply to Gert Doering

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to add_route(), assign "2" on "-EEXIST" (and log appropriate message).

(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)

v2: use RTA_ macros, also adjust add_route_ipv6()

---
 src/openvpn/networking_sitnl.c |  6 +-----
 src/openvpn/route.c            | 24 ++++++++++++++++++------
 2 files changed, 19 insertions(+), 11 deletions(-)
src/openvpn/networking_sitnl.c +1 -5
index dece83c2..cb9a47c0 100644
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex, sa_family_t af_family,
     }
 
     ret = sitnl_send(&req.n, 0, 0, NULL, NULL);
-    if (ret == -EEXIST)
-    {
-        ret = 0;
-    }
 err:
     return ret;
 }
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t af_family, const void *dst,
         scope = RT_SCOPE_LINK;
     }
 
-    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE, ifindex,
+    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex,
                            af_family, dst, prefixlen, gw, table, metric, scope,
                            RTPROT_BOOT, RTN_UNICAST);
 }
src/openvpn/route.c +18 -6
index 61a40ff1..f1257b00 100644
@@ -1599,8 +1599,14 @@ add_route(struct route_ipv4 *r,
     }
 
     status = RTA_SUCCESS;
-    if (net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
-                         &r->gateway, iface, 0, metric) < 0)
+    int ret = net_route_v4_add(ctx, &r->network, netmask_to_netbits2(r->netmask),
+                               &r->gateway, iface, 0, metric);
+    if (ret == -EEXIST)
+    {
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route exists");
+        status = RTA_EEXIST;
+    }
+    else if (ret < 0)
     {
         msg(M_WARN, "ERROR: Linux route add command failed");
         status = RTA_ERROR;
@@ -1963,11 +1969,17 @@ add_route_ipv6(struct route_ipv6 *r6, const struct tuntap *tt,
     }
 
     status = RTA_SUCCESS;
-    if (net_route_v6_add(ctx, &r6->network, r6->netbits,
-                         gateway_needed ? &r6->gateway : NULL, device, 0,
-                         metric) < 0)
+    int ret = net_route_v6_add(ctx, &r6->network, r6->netbits,
+                               gateway_needed ? &r6->gateway : NULL,
+                               device, 0, metric);
+    if (ret == -EEXIST)
     {
-        msg(M_WARN, "ERROR: Linux IPv6 route can't be added");
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route exists");
+        status = RTA_EEXIST;
+    }
+    else if (ret < 0)
+    {
+        msg(M_WARN, "ERROR: Linux route add command failed");
         status = RTA_ERROR;
     }
 
Signature · 1 line
2.38.2
Acked-by on v2 1/1
#7 lore raw

Hi,

Quoted 17 lines · Gert Doering

The code in sitnl_route_set() used to treat "route can not be installed because it already exists" (EEXIST) as "not an error".

This is arguably a reasonable approach, but needs to handled higher up - if the low level add_route() function say "no error", we will try to remove that route later on in delete_route(), possibly removing someone else's "already existing" route then.

So:
- remove special case in sitnl_route_set()
- do not pass NLM_F_REPLACE flag to sitnl_route_set() call - this would cause netlink to just replace existing routes, never return EEXIST (see "man netlink(7)")
- add detailed return code handling to add_route(), assign "2" on
"-EEXIST"
(and log appropriate message).

Quoted 6 lines

(Note: sitnl_route_set() is a common function for sitnl route add and delete, but EEXIST can not happen on delete - so this change has no impact for the "delete" case)

v2: use RTA_ macros, also adjust add_route_ipv6()

Looks good now :)

Quoted 67 lines
---
 src/openvpn/networking_sitnl.c |  6 +-----
 src/openvpn/route.c            | 24 ++++++++++++++++++------
 2 files changed, 19 insertions(+), 11 deletions(-)
src/openvpn/networking_sitnl.c +0 -4
index dece83c2..cb9a47c0 100644
@@ -944,10 +944,6 @@ sitnl_route_set(int cmd, uint32_t flags, int ifindex,
 sa_family_t af_family,
     }
 
     ret = sitnl_send(&req.n, 0, 0, NULL, NULL);
-    if (ret == -EEXIST)
-    {
-        ret = 0;
-    }
 err:
     return ret;

}

src/openvpn/networking_sitnl.c +1 -1
@@ -1177,7 +1173,7 @@ sitnl_route_add(const char *iface, sa_family_t
 af_family, const void *dst,
         scope = RT_SCOPE_LINK;
     }
 
-    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE | NLM_F_REPLACE,
 ifindex,
+    return sitnl_route_set(RTM_NEWROUTE, NLM_F_CREATE, ifindex,
                            af_family, dst, prefixlen, gw, table, metric,

scope,

                            RTPROT_BOOT, RTN_UNICAST);

}

src/openvpn/route.c +8 -2
index 61a40ff1..f1257b00 100644
@@ -1599,8 +1599,14 @@ add_route(struct route_ipv4 *r,
     }
 
     status = RTA_SUCCESS;
-    if (net_route_v4_add(ctx, &r->network,
 netmask_to_netbits2(r->netmask),
-                         &r->gateway, iface, 0, metric) < 0)
+    int ret = net_route_v4_add(ctx, &r->network,
 netmask_to_netbits2(r->netmask),
+                               &r->gateway, iface, 0, metric);
+    if (ret == -EEXIST)
+    {
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route
 exists");
+        status = RTA_EEXIST;
+    }
+    else if (ret < 0)
     {
         msg(M_WARN, "ERROR: Linux route add command failed");
         status = RTA_ERROR;
src/openvpn/route.c +0 -2
@@ -1963,11 +1969,17 @@ add_route_ipv6(struct route_ipv6 *r6, const struct
 tuntap *tt,
     }
 
     status = RTA_SUCCESS;
-    if (net_route_v6_add(ctx, &r6->network, r6->netbits,
-                         gateway_needed ? &r6->gateway : NULL, device, 0,

src/openvpn/route.c add_route_ipv6(struct route_ipv6 *r6, const struct · line 1980

Quoted 10 lines
src/openvpn/route.c +5 -2
-                         metric) < 0)
+    int ret = net_route_v6_add(ctx, &r6->network, r6->netbits,
+                               gateway_needed ? &r6->gateway : NULL,
+                               device, 0, metric);
+    if (ret == -EEXIST)
     {
-        msg(M_WARN, "ERROR: Linux IPv6 route can't be added");
+        msg(D_ROUTE, "NOTE: Linux route add command failed because route
 exists");
 

Even the word "Linux" is redundant here and elsewhere, but we need more changes to make logging consistent and avoid duplicates. Something for 2.7.

Quoted 8 lines
src/openvpn/route.c +5 -0
+        status = RTA_EEXIST;
+    }
+    else if (ret < 0)
+    {
+        msg(M_WARN, "ERROR: Linux route add command failed");
         status = RTA_ERROR;
     }
#8 lore raw
In reply to Gert Doering

For good measure, ran through all the client & server side tests on Linux (other platforms not affected). Since this is where I developed and tested the patch, no surprises :-)

Thanks to having RTA_SUCCESS/RTA_EEXIST now, doing a double-review again ("did I get all the numbers right?") was also very straightforward.

Patch has been applied to the master and release/2.6 branch.

commit adc54f483b210484ff1488e01c8aee1b2b0ea477 (master) commit eb85961e72c927256ce9267a43f3519d3ab4edcb (release/2.6)

Author: Gert Doering
Date:   Wed Jan 18 08:46:33 2023 +0100
     Repair special-casing of EEXIST for Linux/SITNL route install

     Signed-off-by: Gert Doering <[email protected]...>
     Acked-by: Selva Nair <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg26046.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering