Speedup TCP remote hosts connections
PATCH 1/1 Speedup TCP remote hosts connections
For non-blocking TCP/Unix connection, OpenVPN checks was it established in loop and if not - sleeps or handles management for next one second. Since the first check is made right after the connection attempt, it will likely be always unsuccessful, resulting in redundant wait for one or more seconds:
00:00:00.667607 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.667713 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.667832 poll([{fd=5, events=POLLOUT}], 1, 0) = 0 (Timeout)
00:00:00.667954 nanosleep({tv_sec=1, tv_nsec=0}, 0x7fff52450270) = 0
00:00:01.668608 poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])With this patch openvpn_connect() will wait for connection establishment for the first second (if possible) and will return success immediately: this speedups connection and provides real connection time that can be used for detection of the fastest remote server in subsequent patches:
00:00:00.019133 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.019265 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.019388 poll([{fd=5, events=POLLOUT}], 1, 1000) = 1 ([{fd=5, revents=POLLOUT}])If connection still can't established - this should be treated as either too slow/far or non-responding server, so imprecise connection checks every next one second in loop will be performed as usual.
--- src/openvpn/socket.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
src/openvpn/socket.c +19 -1
@@ -1464,13 +1464,26 @@ openvpn_connect(socket_descriptor_t sd,#endif ) { +/* Non-blocking connection is in the progress and can be likely +* established within the first second with no need for imprecise +* sleeping in loop. +*/ +bool connect_wait = (connect_timeout >0 );while (true) {#if POLL +int timeout =0 ;struct pollfd fds[1 ]; + fds[0 ].fd = sd; fds[0 ].events =POLLOUT ; - status =poll (fds,1 ,0 ); +if (connect_wait) + { + connect_wait = false; + timeout =1000 ; + } + + status =poll (fds,1 , timeout);#else fd_set writes;struct timeval tv;
@@ -1479,6 +1492,11 @@ openvpn_connect(socket_descriptor_t sd,openvpn_fd_set (sd, &writes); tv.tv_sec =0 ; tv.tv_usec =0 ; +if (connect_wait) + { + connect_wait = false; + tv.tv_sec =1 ; + } status =select (sd +1 ,NULL , &writes,NULL , &tv);#endif
Signature · 1 line
2.17.1
PATCH 1/1 Speedup TCP remote hosts connections
For non-blocking TCP/Unix connection, OpenVPN checks was it established in loop and if not - sleeps or handles management for next one second. Since the first check is made right after the connection attempt, it will likely be always unsuccessful, resulting in redundant wait for one or more seconds:
00:00:00.667607 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.667713 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.667832 poll([{fd=5, events=POLLOUT}], 1, 0) = 0 (Timeout)
00:00:00.667954 nanosleep({tv_sec=1, tv_nsec=0}, 0x7fff52450270) = 0
00:00:01.668608 poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])With this patch openvpn_connect() will wait for connection establishment for the first second (if possible) and will return success immediately: this speedups connection and provides real connection time that can be used for detection of the fastest remote server in subsequent patches:
00:00:00.019133 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.019265 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.019388 poll([{fd=5, events=POLLOUT}], 1, 1000) = 1 ([{fd=5, revents=POLLOUT}])If connection still can't established - this should be treated as either too slow/far or non-responding server, so imprecise connection checks every next one second in loop will be performed as usual.
v2: cosmetics, decrease connection_timeout to avoid wait more than it in total
--- src/openvpn/socket.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
src/openvpn/socket.c +21 -1
@@ -1464,13 +1464,27 @@ openvpn_connect(socket_descriptor_t sd,#endif ) { +/* Non-blocking connection is in the progress and can be likely +* established within the first second with no need for imprecise +* sleeping in loop. +*/ +bool connect_wait = (connect_timeout >0 );while (true) {#if POLL +int timeout =0 ;struct pollfd fds[1 ]; + fds[0 ].fd = sd; fds[0 ].events =POLLOUT ; - status =poll (fds,1 ,0 ); +if (connect_wait) + { + connect_wait = false; + connect_timeout--; + timeout =1000 ; + } + + status =poll (fds,1 , timeout);#else fd_set writes;struct timeval tv;
@@ -1479,6 +1493,12 @@ openvpn_connect(socket_descriptor_t sd,openvpn_fd_set (sd, &writes); tv.tv_sec =0 ; tv.tv_usec =0 ; +if (connect_wait) + { + connect_wait = false; + connect_timeout--; + tv.tv_sec =1 ; + } status =select (sd +1 ,NULL , &writes,NULL , &tv);#endif
Signature · 1 line
2.17.1
Am 28.09.20 um 01:32 schrieb Vladislav Grishenko:
Quoted 4 lines
For non-blocking TCP/Unix connection, OpenVPN checks was it established in loop and if not - sleeps or handles management for next one second. Since the first check is made right after the connection attempt, it will likely be always unsuccessful, resulting in redundant wait for one or more seconds:
Okay, I looked at the code and you are right. That code is a mess. It basically calls select/pool with a timeout of 0, then actively does select/pool in management for 1s and does this in a loop. That this totally wrong indeed.
I think we should be more radical here. Always do a 1s poll/select timeout and do a 0s management sleep/poll. That will also make the code a bit simpler.
Arne
Hi Arne,
Quoted 6 lines
Okay, I looked at the code and you are right. That code is a mess. It basically calls select/pool with a timeout of 0, then actively does select/pool in management for 1s and does this in a loop. That this totally wrong indeed.
I think we should be more radical here. Always do a 1s poll/select timeout and do a 0s management sleep/poll. That will also make the code a bit simpler.
Othe thing, management_sleep(0) results in infinite wait, so I;ve changed api in subsequent patch.
Now, management_sleep() can handle only non-negative value and managemet_event_loop_n_seconds() can take negative values to have infinite wait, if necessary. Since there were no negative or zero parameter for management_sleep() users, no side effect behavior change is expected.
Seems, a bit simpler can't be achieved :)
--
Best Regards, Vladislav Grishenko
PATCH v3 1/1 Speedup TCP remote hosts connections
In reply to Vladislav GrishenkoFor non-blocking TCP/Unix connection, OpenVPN checks was it established in loop and if not - sleeps or handles management for next one second. Since the first check is made right after the connection attempt, it will likely be always unsuccessful, causing redundant wait for one or more seconds:
00:00:00.667607 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.667713 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.667832 poll([{fd=5, events=POLLOUT}], 1, 0) = 0 (Timeout)
00:00:00.667954 nanosleep({tv_sec=1, tv_nsec=0}, 0x7fff52450270) = 0
00:00:01.668608 poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])After this patch openvpn_connect() will perform blocking wait for connection establishment (if possible) and just check for management events once in one second (if management enabled) w/o sleep. This speedups TCP/Unix connection establishment and provides almost real connection time that can be used for detection of the fastest remote server in subsequent patches:
00:00:00.790510 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.790616 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.790877 poll([{fd=5, events=POLLOUT}], 1, 1000) = 0 (Timeout)
00:00:01.792880 poll([{fd=5, events=POLLOUT}], 1, 1000) = 1 ([{fd=5, revents=POLLOUT}])Or, with management interface enabled:
00:00:00.906421 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
00:00:00.906527 connect(6, {...}, 16) = -1 EINPROGRESS (Operation now in progress)
00:00:00.906779 poll([{fd=6, events=POLLOUT}], 1, 1000) = 0 (Timeout)
00:00:01.910418 poll([{fd=3, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
00:00:01.911365 poll([{fd=6, events=POLLOUT}], 1, 1000) = 0 ([{fd=6, revents=POLLOUT}])v2: cosmetics, decrease connection_timeout to avoid wait more than it
v3: teach management_sleep() to handle zero timeout and reject negative
use 1s timeout for connection and 0s timeout for management events
--- src/openvpn/manage.c | 30 +++++++++++++++++++++++------- src/openvpn/socket.c | 6 +++--- 2 files changed, 26 insertions(+), 10 deletions(-)
src/openvpn/manage.c +23 -7
@@ -3310,12 +3310,17 @@ man_block(struct management *man, volatile int *signal_received, const time_t exif (man_standalone_ok (man)) { +/* expire time can be already overdue, for this case init zero +* timeout to avoid waiting first time and exit loop early with +* either obtained event or timeout. +*/ + tv.tv_usec =0 ; + tv.tv_sec =0 ; +while (true) {event_reset (man->connection.es);management_socket_set (man, man->connection.es,NULL ,NULL ); - tv.tv_usec =0 ; - tv.tv_sec =1 ;if (man_check_for_signals (signal_received)) { status =-1 ;
@@ -3343,6 +3348,10 @@ man_block(struct management *man, volatile int *signal_received, const time_t ex }break ; } + +/* wait one second more */ + tv.tv_sec =1 ; + tv.tv_usec =0 ; } }return status;
@@ -3444,7 +3453,7 @@ management_event_loop_n_seconds(struct management *man, int sec)/* set expire time */ update_time (); -if (sec) +if (sec >=0 ) { expire = now + sec; }
@@ -3474,7 +3483,7 @@ management_event_loop_n_seconds(struct management *man, int sec)/* revert state */ man->persist.standalone_disabled = standalone_disabled_save; } -else +else if (sec >0 ) {sleep (sec); }
@@ -4117,11 +4126,15 @@ log_history_ref(const struct log_history *h, const int index)void management_sleep (const int n) { -if (management) +if (n <0 ) + { +return ; + } +else if (management) {management_event_loop_n_seconds (management, n); } -else +else if (n >0 ) {sleep (n); }
@@ -4132,7 +4145,10 @@ management_sleep(const int n)void management_sleep (const int n) { -sleep (n); +if (n >0 ) + { +sleep (n); + } }#endif /* ENABLE_MANAGEMENT */
src/openvpn/socket.c +3 -3
@@ -1470,14 +1470,14 @@ openvpn_connect(socket_descriptor_t sd,struct pollfd fds[1 ]; fds[0 ].fd = sd; fds[0 ].events =POLLOUT ; - status =poll (fds,1 ,0 ); + status =poll (fds,1 , (connect_timeout >0 ) ?1000 :0 );#else fd_set writes;struct timeval tv;FD_ZERO (&writes);openvpn_fd_set (sd, &writes); - tv.tv_sec =0 ; + tv.tv_sec = (connect_timeout >0 ) ?1 :0 ; tv.tv_usec =0 ; status =select (sd +1 ,NULL , &writes,NULL , &tv);
@@ -1507,7 +1507,7 @@ openvpn_connect(socket_descriptor_t sd,#endif break ; } -management_sleep (1 ); +management_sleep (0 );continue ; }
Signature · 1 line
2.17.1
Am 02.10.20 um 00:53 schrieb Vladislav Grishenko:
Quoted 30 lines
For non-blocking TCP/Unix connection, OpenVPN checks was it established in loop and if not - sleeps or handles management for next one second. Since the first check is made right after the connection attempt, it will likely be always unsuccessful, causing redundant wait for one or more seconds:
00:00:00.667607 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 00:00:00.667713 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress) 00:00:00.667832 poll([{fd=5, events=POLLOUT}], 1, 0) = 0 (Timeout) 00:00:00.667954 nanosleep({tv_sec=1, tv_nsec=0}, 0x7fff52450270) = 0 00:00:01.668608 poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])After this patch
openvpn_connect()will perform blocking wait for connection establishment (if possible) and just check for management events once in one second (if management enabled) w/o sleep. This speedups TCP/Unix connection establishment and provides almost real connection time that can be used for detection of the fastest remote server in subsequent patches:00:00:00.790510 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 00:00:00.790616 connect(5, {...}, 16) = -1 EINPROGRESS (Operation now in progress) 00:00:00.790877 poll([{fd=5, events=POLLOUT}], 1, 1000) = 0 (Timeout) 00:00:01.792880 poll([{fd=5, events=POLLOUT}], 1, 1000) = 1 ([{fd=5, revents=POLLOUT}])Or, with management interface enabled:
00:00:00.906421 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 00:00:00.906527 connect(6, {...}, 16) = -1 EINPROGRESS (Operation now in progress) 00:00:00.906779 poll([{fd=6, events=POLLOUT}], 1, 1000) = 0 (Timeout) 00:00:01.910418 poll([{fd=3, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout) 00:00:01.911365 poll([{fd=6, events=POLLOUT}], 1, 1000) = 0 ([{fd=6, revents=POLLOUT}])
Quoted 3 lines
v2: cosmetics, decrease
connection_timeoutto avoid wait more than it
v3: teachmanagement_sleep()to handle zero timeout and reject negative
use 1s timeout for connection and 0s timeout for management events
I like this version much more. It doesn't add extra complexity.
Your patch has been applied to the master branch.
I have run a number of TCP client connect tests, to a very close server (speedup very noticeable!), to distant server (still some difference), and to a non-reachable IP address (because I was worried that this would block out management, but it doesn't - it just shifts the "delay 1 s" to openvpn_connect(), and makes it a "max 1s" instead of a hard sleep).
I have not looked into manage.c as closely, but from a quick glance, this also looks reasonable.
Since we need a RC3 anyway (due to the redirect-gateway bug), and Arne's motto for 2.5 is "MAKE THINGS FAST!", I think this still fits into 2.5.
commit b68aa00603332357e6c711e91c5b4ba04d78294b (master) commit d08b66f39cf80733a51e1607386dc4993faef09f (release/2.5)
Author: Vladislav Grishenko Date: Fri Oct 2 03:53:19 2020 +0500
Speedup TCP remote hosts connections
Signed-off-by: Vladislav Grishenko <[email protected]...>
Acked-by: Arne Schwabe <[email protected]...>
Message-Id: <[email protected]...>
URL: https://www.mail-archive.com/[email protected]/msg21139.html
Signed-off-by: Gert Doering <[email protected]...>--
kind regards,
Gert Doering
Hi Gert,
Thanks for that.
Perhaps same approach can be applied to server's tcp listening, would require testing of more management cases.
--
Best Regards, Vladislav Grishenko
Quoted 10 lines
-----Original Message-----
From: Gert Doering <[email protected]...>
Sent: Sunday, October 4, 2020 5:19 PM
To: Vladislav Grishenko <[email protected]...>
Cc: [email protected]
Subject: [PATCH applied] Re: Speedup TCP remote hosts connectionsYour patch has been applied to the master branch.
I have run a number of TCP client connect tests, to a very close server
(speedup
Quoted 1 line
very noticeable!), to distant server (still some difference), and to a
non-
Quoted 2 lines
reachable IP address (because I was worried that this would block out management, but it doesn't - it just shifts the "delay 1 s" to
openvpn_connect(),
Quoted 3 lines
and makes it a "max 1s" instead of a hard sleep).
I have not looked into
manage.cas closely, but from a quick glance, this
also
Quoted 24 lines
looks reasonable.
Since we need a RC3 anyway (due to the redirect-gateway bug), and Arne's motto for 2.5 is "MAKE THINGS FAST!", I think this still fits into 2.5.
commit
b68aa00603332357e6c711e91c5b4ba04d78294b(master) commitd08b66f39cf80733a51e1607386dc4993faef09f(release/2.5)Author: Vladislav Grishenko Date: Fri Oct 2 03:53:19 2020 +0500Speedup TCP remote hosts connections Signed-off-by: Vladislav Grishenko <[email protected]...> Acked-by: Arne Schwabe <[email protected]...> Message-Id: <[email protected]...> URL: https://www.mail-archive.com/openvpn- [email protected]/msg21139.html Signed-off-by: Gert Doering <[email protected]...>--
kind regards,Gert Doering
Hi,
Quoted 2 lines · Vladislav Grishenko
Perhaps same approach can be applied to server's tcp listening, would require testing of more management cases.
There's two code paths here
--tcp-server
and
--mode server + --tcp
I think the "mode server" code path is already very quick (= if I test with your fast TCP client, against a "git master" server, I get 0.16s connection setup time). This is what people use on "more than one client" servers, so it's already fine.
The "--tcp-server" code path is "point to point". Not sure we currently test this at all (I have UDP p2p instance, but no TCP yet... seems I need to add one). This might indeed be slow, I had a look at the accept() path in socket.c recently and was wondering "how can this work at all?" - but that's "socket.c" vs. "mtcp.c", I think.
Since this is somewhat of a niche case, I'd put that into the "2.6" bin :-)
- the other one, TCP on the client, is much more heavily used, so 2.5 for that was appropriate.
gert
Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress
Gert Doering - Munich, Germany [email protected]...Hi Gert,
Quoted 1 line
"--tcp-server"
Yep, mean it, even poll doesn't used there. Have no any prio about it tho, just related thoughts.
--
Best Regards, Vladislav Grishenko
Quoted 16 lines
-----Original Message-----
From: Gert Doering <[email protected]...>
Sent: Monday, October 5, 2020 10:28 PM
To: Vladislav Grishenko <[email protected]...>
Cc: 'Gert Doering' <[email protected]...>; openvpn- [email protected]
Subject: Re: [PATCH applied] Re: Speedup TCP remote hosts connectionsHi,
Quoted 2 lines · Vladislav Grishenko
Perhaps same approach can be applied to server's tcp listening, would require testing of more management cases.
There's two code paths here
Quoted 7 lines
--tcp-server
and
--mode server + --tcp
I think the "mode server" code path is already very quick (= if I test
with your fast
Quoted 1 line
TCP client, against a "git master" server, I get 0.16s connection setup
time). This
Quoted 1 line
is what people use on "more than one client" servers, so it's already
fine.
Quoted 2 lines
The "--tcp-server" code path is "point to point". Not sure we currently
test this
Quoted 1 line
at all (I have UDP p2p instance, but no TCP yet... seems I need to add
one). This
Quoted 2 lines
might indeed be slow, I had a look at the
accept()path insocket.crecently and was wondering "how can this work at
all?"
Quoted 4 lines
- but that's "
socket.c" vs. "mtcp.c", I think.Since this is somewhat of a niche case, I'd put that into the "2.6" bin
:-)
Quoted 7 lines
- the other one, TCP on the client, is much more heavily used, so 2.5 for that was appropriate.
gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never
doubted
Quoted 2 lines
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh
Mistress
Quoted 2 lines
Gert Doering - Munich, Germany