Skip to contentSkip to conversation

package/libwebsockets: add an option to enable lws async dns

PATCH Applied Thomas Devoogdt 2 participants 8 messages Sep 20, 2024 – Feb 4

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 package/libwebsockets: add an option to enable lws async dns

From: Bart Van Severen <[email protected]>

Lws now features optional asynchronous, ie, nonblocking recursive DNS resolution done on the event loop, enable -DLWS_WITH_SYS_ASYNC_DNS=1 at cmake to build it in.

The default libc name resolution is via libc getaddrinfo(), which is blocking, possibly for quite long periods (seconds). If you are taking care about latency, but want to create outgoing connections, you can't tolerate this exception from the rule that everything in lws is nonblocking.

Lws' asynchronous DNS resolver creates a caching name resolver that directly queries the configured nameserver itself over UDP, from the event loop.

https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html

---
 package/libwebsockets/Config.in        | 6 ++++++
 package/libwebsockets/libwebsockets.mk | 4 ++++
 2 files changed, 10 insertions(+)
package/libwebsockets/Config.in +6 -0
index 05c132641e..6116cfecb5 100644
@@ -19,6 +19,12 @@ config BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL
 	  unconditionally compiled in for library versions
 	  prior to 3.2.0.
 
+config BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS
+	bool "enable async dns support"
+	help
+	  Enable asynchronous DNS resolver that directly queries
+	  the configured nameserver over UDP, from the event loop.
+
 endif
 
 comment "libwebsockets needs a toolchain w/ dynamic library"
package/libwebsockets/libwebsockets.mk +4 -0
index c00c44cac3..e53febfce4 100644
@@ -98,4 +98,8 @@ ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL),y)
 LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_EXTERNAL_POLL=ON
 endif
 
+ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS),y)
+LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_SYS_ASYNC_DNS=ON
+endif
+
 $(eval $(cmake-package))
Signature · 6 lines
2.43.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 2/2 package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

From: Bart Van Severen <[email protected]>

"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset, LWS_MAX_SMP defaults to 32 and a small amount of pthread mutex code is built into the library."

This is a misconception, when unset, LWS_MAX_SMP is actually set to 1, so mutexes aren't built in.

To fix, set it to 32 explicitly when threads are enabled.

https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html

---
 package/libwebsockets/libwebsockets.mk | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
package/libwebsockets/libwebsockets.mk +2 -3
index e53febfce4..90ab5da0c9 100644
@@ -18,15 +18,14 @@ LIBWEBSOCKETS_CONF_OPTS = \
 	-DLWS_WITHOUT_EXTENSIONS=OFF
 
 # If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
-# in the library. If unset, LWS_MAX_SMP defaults to 32 and a small
-# amount of pthread mutex code is built into the library.
+# in the library. If unset, LWS_MAX_SMP defaults to 1.
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),)
 LIBWEBSOCKETS_CONF_OPTS += \
 	-DLWS_MAX_SMP=1 \
 	-DLWS_WITH_SYS_SMD=OFF
 else
 LIBWEBSOCKETS_CONF_OPTS += \
-	-DLWS_MAX_SMP= \
+	-DLWS_MAX_SMP=32 \
 	-DLWS_WITH_SYS_SMD=ON
 endif
 
Signature · 6 lines
2.43.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#7 lore raw
In reply to Thomas Devoogdt
Quoted 16 lines · On Fri, 20 Sep 2024 08:20:13 +0200 Thomas Devoogdt <[email protected]> wrote:

From: Bart Van Severen <[email protected]>

Lws now features optional asynchronous, ie, nonblocking recursive DNS resolution done on the event loop, enable -DLWS_WITH_SYS_ASYNC_DNS=1 at cmake to build it in.

The default libc name resolution is via libc getaddrinfo(), which is blocking, possibly for quite long periods (seconds). If you are taking care about latency, but want to create outgoing connections, you can't tolerate this exception from the rule that everything in lws is nonblocking.

Lws' asynchronous DNS resolver creates a caching name resolver that directly queries the configured nameserver itself over UDP, from the event loop.

Quoted 8 lines

https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html

---
 package/libwebsockets/Config.in        | 6 ++++++
 package/libwebsockets/libwebsockets.mk | 4 ++++
 2 files changed, 10 insertions(+)

Applied to master, thanks.

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

In reply to Thomas Devoogdt

Hello Thomas/Bart,

Quoted 11 lines · On Fri, 20 Sep 2024 08:20:14 +0200 Thomas Devoogdt <[email protected]> wrote:

From: Bart Van Severen <[email protected]>

"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset, LWS_MAX_SMP defaults to 32 and a small amount of pthread mutex code is built into the library."

This is a misconception, when unset, LWS_MAX_SMP is actually set to 1, so mutexes aren't built in.

To fix, set it to 32 explicitly when threads are enabled.

Quoted 7 lines

https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html

---
 package/libwebsockets/libwebsockets.mk | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

The question is then why 32, and not 8, 16, or 64 ?

Thanks!

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

Hi Thomas

Op za 26 okt 2024 om 16:43 schreef Thomas Petazzoni <[email protected]>:

Quoted 19 lines

Hello Thomas/Bart,

Quoted 13 lines · On Fri, 20 Sep 2024 08:20:14 +0200 Thomas Devoogdt <[email protected]> wrote:

From: Bart Van Severen <[email protected]>

"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset, LWS_MAX_SMP defaults to 32 and a small amount of pthread mutex code is built into the library."

This is a misconception, when unset, LWS_MAX_SMP is actually set to 1, so mutexes aren't built in.

To fix, set it to 32 explicitly when threads are enabled.

https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html

Quoted 7 lines
Quoted 5 lines
---
 package/libwebsockets/libwebsockets.mk | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

The question is then why 32, and not 8, 16, or 64 ?

The original comment mentioned the number 32, so it makes sense to keep that 32.

Also, see this older manual
https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."

And this newer manual
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "

So, given that the original manual talks about 32, it makes sense to use it, even though it is no longer valid.
Other values will also work, as long the minimum is 2. But I guess that this 32 is a good trade-off.

Quoted 9 lines

Thanks!

Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Kr,

Thomas Devoogdt
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

Hello Thomas,

Quoted 14 lines · On Sat, 26 Oct 2024 20:25:26 +0200 Thomas Devoogdt <[email protected]> wrote:
Quoted 1 line

The question is then why 32, and not 8, 16, or 64 ?

The original comment mentioned the number 32, so it makes sense to keep that 32.

Also, see this older manual
https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."

And this newer manual
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "

Quoted 4 lines

So, given that the original manual talks about 32, it makes sense to use it, even though it is no longer valid.
Other values will also work, as long the minimum is 2. But I guess that this 32 is a good trade-off.

It felt a bit random, but OK, I guess there's no "good" value there. Thanks for the feedback!

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#5 lore raw

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

Hi Thomas, all,

This thread has been a bit abandoned for some time now, any concerns still to be addressed?
Would be nice to have it for the upcoming 2025.02 release.

Kind regards,

Thomas Devoogdt
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#6 lore raw

Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set

In reply to Thomas Devoogdt

Hello Thomas,

Quoted 9 lines · Thomas Devoogdt

From: Bart Van Severen <[email protected]>

"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset, LWS_MAX_SMP defaults to 32 and a small amount of pthread mutex code is built into the library."

This is a misconception, when unset, LWS_MAX_SMP is actually set to 1, so mutexes aren't built in.

Quoted 6 lines

To fix, set it to 32 explicitly when threads are enabled.

https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html

Thanks for the patch and additional details. I have improved a bit the commit log with some of the details provided as part of the discussion, and finally applied your patch... after 1.5 years.

Thanks!

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot