package/libwebsockets: add an option to enable lws async dns
PATCH 1/2 package/libwebsockets: add an option to enable lws async dns
From: Bart Van Severen <[email protected]>
Lws now features optional asynchronous, ie, nonblocking recursive DNS resolution done on the event loop, enable -DLWS_WITH_SYS_ASYNC_DNS=1 at cmake to build it in.
The default libc name resolution is via libc getaddrinfo(), which is blocking, possibly for quite long periods (seconds). If you are taking care about latency, but want to create outgoing connections, you can't tolerate this exception from the rule that everything in lws is nonblocking.
Lws' asynchronous DNS resolver creates a caching name resolver that directly queries the configured nameserver itself over UDP, from the event loop.
https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html
--- package/libwebsockets/Config.in | 6 ++++++ package/libwebsockets/libwebsockets.mk | 4 ++++ 2 files changed, 10 insertions(+)
package/libwebsockets/Config.in +6 -0
@@ -19,6 +19,12 @@ config BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL unconditionally compiled in for library versions prior to 3.2.0. +config BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS + bool "enable async dns support" + help + Enable asynchronous DNS resolver that directly queries + the configured nameserver over UDP, from the event loop. + endif comment "libwebsockets needs a toolchain w/ dynamic library"
package/libwebsockets/libwebsockets.mk +4 -0
@@ -98,4 +98,8 @@ ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_EXT_POLL),y) LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_EXTERNAL_POLL=ON endif +ifeq ($(BR2_PACKAGE_LIBWEBSOCKETS_ASYNC_DNS),y) +LIBWEBSOCKETS_CONF_OPTS += -DLWS_WITH_SYS_ASYNC_DNS=ON +endif + $(eval $(cmake-package))
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 2/2 package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
From: Bart Van Severen <[email protected]>
"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset, LWS_MAX_SMP defaults to 32 and a small amount of pthread mutex code is built into the library."
This is a misconception, when unset, LWS_MAX_SMP is actually set to 1, so mutexes aren't built in.
To fix, set it to 32 explicitly when threads are enabled.
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
--- package/libwebsockets/libwebsockets.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
package/libwebsockets/libwebsockets.mk +2 -3
@@ -18,15 +18,14 @@ LIBWEBSOCKETS_CONF_OPTS = \ -DLWS_WITHOUT_EXTENSIONS=OFF # If LWS_MAX_SMP=1, then there is no code related to pthreads compiled -# in the library. If unset, LWS_MAX_SMP defaults to 32 and a small -# amount of pthread mutex code is built into the library. +# in the library. If unset, LWS_MAX_SMP defaults to 1. ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),) LIBWEBSOCKETS_CONF_OPTS += \ -DLWS_MAX_SMP=1 \ -DLWS_WITH_SYS_SMD=OFF else LIBWEBSOCKETS_CONF_OPTS += \ - -DLWS_MAX_SMP= \ + -DLWS_MAX_SMP=32 \ -DLWS_WITH_SYS_SMD=ON endif
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
Hello Thomas/Bart,
Quoted 11 lines · On Fri, 20 Sep 2024 08:20:14 +0200 Thomas Devoogdt <[email protected]> wrote:
From: Bart Van Severen <[email protected]>
"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset,LWS_MAX_SMPdefaults to 32 and a small amount of pthread mutex code is built into the library."This is a misconception, when unset,
LWS_MAX_SMPis actually set to 1, so mutexes aren't built in.To fix, set it to 32 explicitly when threads are enabled.
Quoted 7 lines
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
Signed-off-by Bart Van Severen <[email protected]>Signed-off-by Thomas Devoogdt <[email protected]>--- package/libwebsockets/libwebsockets.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
The question is then why 32, and not 8, 16, or 64 ?
Thanks!
Thomas
Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
Hi Thomas
Op za 26 okt 2024 om 16:43 schreef Thomas Petazzoni <[email protected]>:
Quoted 19 lines
Hello Thomas/Bart,
Quoted 13 lines · On Fri, 20 Sep 2024 08:20:14 +0200 Thomas Devoogdt <[email protected]> wrote:
From: Bart Van Severen <[email protected]>
"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset,LWS_MAX_SMPdefaults to 32 and a small amount of pthread mutex code is built into the library."This is a misconception, when unset,
LWS_MAX_SMPis actually set to 1, so mutexes aren't built in.To fix, set it to 32 explicitly when threads are enabled.
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
Quoted 7 lines
Quoted 5 lines
Signed-off-by Bart Van Severen <[email protected]>Signed-off-by Thomas Devoogdt <[email protected]>--- package/libwebsockets/libwebsockets.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)The question is then why 32, and not 8, 16, or 64 ?
The original comment mentioned the number 32, so it makes sense to keep that 32.
Also, see this older manual
https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."
And this newer manual
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "
So, given that the original manual talks about 32, it makes sense to use it, even though it is no longer valid.
Other values will also work, as long the minimum is 2. But I guess that this 32 is a good trade-off.
Quoted 9 lines
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Kr,
Thomas Devoogdt
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
Hello Thomas,
Quoted 14 lines · On Sat, 26 Oct 2024 20:25:26 +0200 Thomas Devoogdt <[email protected]> wrote:
Quoted 1 line
The question is then why 32, and not 8, 16, or 64 ?
The original comment mentioned the number 32, so it makes sense to keep that 32.
Also, see this older manual
https://libwebsockets.org/lws-api-doc-master/html/md_README.coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 32."And this newer manual
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html:
"You can control the context basic data allocation for multithreading from Cmake using -DLWS_MAX_SMP=, if not given it's set to 1. "
Quoted 4 lines
So, given that the original manual talks about 32, it makes sense to use it, even though it is no longer valid.
Other values will also work, as long the minimum is 2. But I guess that this 32 is a good trade-off.
It felt a bit random, but OK, I guess there's no "good" value there. Thanks for the feedback!
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering and training https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
Hi Thomas, all,
This thread has been a bit abandoned for some time now, any concerns still to be addressed?
Would be nice to have it for the upcoming 2025.02 release.
Kind regards,
Thomas Devoogdt
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Re: [Buildroot] [PATCH v1 2/2] package/libwebsockets: fix LWS_MAX_SMP when BR2_TOOLCHAIN_HAS_THREADS is set
In reply to Thomas DevoogdtHello Thomas,
Quoted 9 lines · Thomas Devoogdt
From: Bart Van Severen <[email protected]>
"If LWS_MAX_SMP=1, then there is no code related to pthreads compiled
in the library. If unset,LWS_MAX_SMPdefaults to 32 and a small amount of pthread mutex code is built into the library."This is a misconception, when unset,
LWS_MAX_SMPis actually set to 1, so mutexes aren't built in.
Quoted 6 lines
To fix, set it to 32 explicitly when threads are enabled.
https://libwebsockets.org/lws-api-doc-master/html/md_READMEs_README_8coding.html
Signed-off-by Bart Van Severen <[email protected]>Signed-off-by Thomas Devoogdt <[email protected]>
Thanks for the patch and additional details. I have improved a bit the commit log with some of the details provided as part of the discussion, and finally applied your patch... after 1.5 years.
Thanks!
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering and training https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Thomas Petazzoni via buildroot
AppliedQuoted 16 lines · On Fri, 20 Sep 2024 08:20:13 +0200 Thomas Devoogdt <[email protected]> wrote:
From: Bart Van Severen <[email protected]>
Lws now features optional asynchronous, ie, nonblocking recursive DNS resolution done on the event loop, enable
-DLWS_WITH_SYS_ASYNC_DNS=1at cmake to build it in.The default libc name resolution is via libc
getaddrinfo(), which is blocking, possibly for quite long periods (seconds). If you are taking care about latency, but want to create outgoing connections, you can't tolerate this exception from the rule that everything in lws is nonblocking.Lws' asynchronous DNS resolver creates a caching name resolver that directly queries the configured nameserver itself over UDP, from the event loop.
Quoted 8 lines
https://libwebsockets.org/lws-api-doc-main/html/md_READMEs_2README_8async-dns.html
Signed-off-by Bart Van Severen <[email protected]>Signed-off-by Thomas Devoogdt <[email protected]>--- package/libwebsockets/Config.in | 6 ++++++ package/libwebsockets/libwebsockets.mk | 4 ++++ 2 files changed, 10 insertions(+)
Applied to master, thanks.
Thomas
Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot