Skip to contentSkip to conversation

Fix fatal error at switching remotes (#629)

PATCH Superseded Vladislav Grishenko 3 participants 8 messages Sep 16, 2020 – Sep 17, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Fix fatal error at switching remotes (#629)

If remote server has been resolved to multiple addresses, at least one connection attemt has been made and connection to the last address was skipped by management - resolved earlier link socket addrinfo objects will not be cleared neither on instance close nor in the next connection entry loop. This causes fatal error assert:

Quoted 1 line

REMOTE:openvpn.net,1194,udp

remote ACCEPT
SUCCESS: remote command succeeded
>REMOTE:openvpn.net,1194,udp
remote SKIP
SUCCESS: remote command succeeded
>FATAL:Assertion failed at init.c:504 (c->c1.link_socket_addr.current_remote == NULL)

Fix this behaviour by cleaning stale addinfo objects.

---
 src/openvpn/init.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
src/openvpn/init.c +10 -0
index a785934a..508270a7 100644
@@ -500,6 +500,16 @@ next_connection_entry(struct context *c)
                  */
                 if (!c->options.persist_remote_ip)
                 {
+                    if (c->c1.link_socket_addr.remote_list)
+                    {
+                        /* Connection entry addinfo objects might have been resolved
+                         * earlier but the entry itself might have been skipped by
+                         * management on the previous loop. if so, need to clear the
+                         * addrinfo objects as close_instance does */
+                        clear_remote_addrlist(&c->c1.link_socket_addr,
+                                              !c->options.resolve_in_advance);
+                    }
+
                     /* close_instance should have cleared the addrinfo objects */
                     ASSERT(c->c1.link_socket_addr.current_remote == NULL);
                     ASSERT(c->c1.link_socket_addr.remote_list == NULL);
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/1 Fix fatal error at switching remotes (#629)

If remote server has been resolved to multiple addresses, at least one connection attemt has been made and connection to the last address was skipped by management - resolved earlier link socket addrinfo objects will not be cleared neither on instance close nor in the next connection entry loop. This causes fatal error assert:

Quoted 1 line

REMOTE:openvpn.net,1194,udp

remote ACCEPT
SUCCESS: remote command succeeded
>REMOTE:openvpn.net,1194,udp
remote SKIP
SUCCESS: remote command succeeded
>FATAL:Assertion failed at init.c:504 (c->c1.link_socket_addr.current_remote == NULL)

Fix this behaviour by cleaning stale addinfo objects.

v2: better comment placement and too long length fix

---
 src/openvpn/init.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
src/openvpn/init.c +11 -0
index a785934a..8fc79638 100644
@@ -500,6 +500,17 @@ next_connection_entry(struct context *c)
                  */
                 if (!c->options.persist_remote_ip)
                 {
+                    /* Connection entry addinfo objects might have been resolved
+                     * earlier but the entry itself might have been skipped by
+                     * management on the previous loop.
+                     * If so, clear the addrinfo objects as close_instance does
+                     */
+                    if (c->c1.link_socket_addr.remote_list)
+                    {
+                        clear_remote_addrlist(&c->c1.link_socket_addr,
+                                              !c->options.resolve_in_advance);
+                    }
+
                     /* close_instance should have cleared the addrinfo objects */
                     ASSERT(c->c1.link_socket_addr.current_remote == NULL);
                     ASSERT(c->c1.link_socket_addr.remote_list == NULL);
Signature · 1 line
2.17.1
Acked-by on v2 1/1
#2 lore raw

Hi,

Quoted 2 lines

If remote server has been resolved to multiple addresses, at least one connection attemt

typ0

Quoted 1 line

Fix this behaviour by cleaning stale addinfo objects.

Stared at code and it looks fine - same cleanup logic as in do_close_link_socket().
Built and tested with MSVC - reproduced assert without patch and no assert with patch.

#3 lore raw
In reply to Vladislav Grishenko

Your patch has been applied to the master, release/2.5 and release/2.4 branch (bugfix).

I have fixed a few "addinfo" occurances and re-wrapped the comment slightly. Not checked the actual code, just ran a t_client test on 2.4 "to be sure".

As Arne wrote there is a "fix for real" dangling here... :-)

commit 3ad86c2534a92af137809b6d446d570193e6d01f (master) commit 6554025a422d3d7e5465bcbfad34fa3e196b53b0 (release/2.5) commit 7fdcd286a15fb4f64e979c4fdbf52223d4bdede0 (release/2.4)

Author: Vladislav Grishenko
Date:   Wed Sep 16 19:17:55 2020 +0500
     Fix fatal error at switching remotes (#629)

     Signed-off-by: Vladislav Grishenko <[email protected]...>
     Acked-by: Lev Stipakov <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg21019.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#4 lore raw

Thank you a lot,
That "fix for real" is about persist_remote_ip option as far as I understand, not directly related to this fatal assert fix.

--
Best Regards, Vladislav Grishenko

Quoted 8 lines

-----Original Message-----
From: Gert Doering <[email protected]...>
Sent: Thursday, September 17, 2020 1:46 PM
To: Vladislav Grishenko <[email protected]...>
Cc: [email protected]
Subject: [PATCH applied] Re: Fix fatal error at switching remotes (#629)

Your patch has been applied to the master, release/2.5 and release/2.4

branch

Quoted 3 lines

(bugfix).

I have fixed a few "addinfo" occurances and re-wrapped the comment

slightly.

Quoted 25 lines

Not checked the actual code, just ran a t_client test on 2.4 "to be sure".

As Arne wrote there is a "fix for real" dangling here... :-)

commit 3ad86c2534a92af137809b6d446d570193e6d01f (master) commit 6554025a422d3d7e5465bcbfad34fa3e196b53b0 (release/2.5) commit 7fdcd286a15fb4f64e979c4fdbf52223d4bdede0 (release/2.4)

Author: Vladislav Grishenko
Date:   Wed Sep 16 19:17:55 2020 +0500
     Fix fatal error at switching remotes (#629)

     Signed-off-by: Vladislav Grishenko <[email protected]...>
     Acked-by: Lev Stipakov <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/openvpn-
[email protected]/msg21019.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#5 lore raw

Hi,

Quoted 1 line · Vladislav Grishenko

Thank you a lot,

Not needed :-)

Quoted 2 lines

That "fix for real" is about persist_remote_ip option as far as I understand, not directly related to this fatal assert fix.

Well, the whole preresolve / connection entry "complex" is old and has been extended and updated a few times, and your SVR patch also builds on top of that. At some point, refactoring is needed...

(We have some other thing to consider which is even more intrusive - when we reconnect to a different IP address, and that new IP address is currently routed into the tunnel, we need to set up new /32 host routes before moving to a new server can work... openvpn3, as I understand, sets up "all host routes!" right at the start, but that might or might not be the best solution either)

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#6 lore raw

Hi,

Quoted 1 line

openvpn3, as I understand, sets up "all host routes!" right at the start

It depends on how openvpn3 library is used.

OpenVPN3 Linux client adds bypass route for the specific remote just before connection attempt. Same for our Connect Windows / Mac clients, which are partially closed-source, but this specific logic is part of opensource library and "agent" (windows agent is opensourced, mac agent is still closed-source) - a component similar to "iservice".

IIRC, "test" client (cli) indeed adds all routes from the beginning.

Signature · 1 line
-Lev
#7 lore raw
In reply to Gert Doering

Hi, Gert

Quoted 4 lines
Quoted 2 lines

That "fix for real" is about persist_remote_ip option as far as I understand, not directly related to this fatal assert fix.

Well, the whole preresolve / connection entry "complex" is old and has

been

Quoted 2 lines

extended and updated a few times, and your SVR patch also builds on top of that.

That's true, I hit this assert for SRV initially, 'coz same advancing logic was used, v5 version is upcoming following this commit.

Quoted 2 lines

At some point, refactoring is needed...
(We have some other thing to consider which is even more intrusive - when

we

Quoted 1 line

reconnect to a different IP address, and that new IP address is currently

routed

Quoted 1 line

into the tunnel, we need to set up new /32 host routes before moving to a

new

Quoted 1 line

server can work... openvpn3, as I understand, sets up "all host routes!"

right at

Quoted 1 line

the start, but that might or might not be the best solution either)

New address is being handled in disconnected (yet) state, so tunnel routes should not be active, since 2.x supports at most one tunnel active. While this is preserved, /32 host route can be made anytime between resolved state and connection attempt, sounds not so intrusive, if I got you right.

Meanwhile, "--persist-remote-ip" documented as "Preserve most recently authenticated remote IP address and port number across SIGUSR1 and --ping-restart".
Current implementation doesn't follow it precisely, instead it does "Preserve most recently authenticated remote host name and port...", if that remote name resolves into multiple addresses - they will be still iterated. Guess, this is what was meant by "fix by real"

Quoted 4 lines

gert
--
"If was one thing all people took for granted, was conviction that if you

feed

Quoted 3 lines

honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh

Mistress

Quoted 2 lines

Gert Doering - Munich, Germany

[email protected]...