package/librdkafka: add new package
PATCH 1/3 package/librdkafka: add new package
Will be a dependency of fluent-bit.
Signed-off-by Thomas Devoogdt <[email protected]>
---
$ ./utils/test-pkg -p librdkafka -a
arm-aarch64 [ 1/32]: OK
bootlin-aarch64-glibc [ 2/32]: OK
bootlin-aarch64-glibc-old [ 3/32]: OK
bootlin-arcle-hs38-uclibc [ 4/32]: OK
bootlin-armv5-uclibc [ 5/32]: OK
bootlin-armv7-glibc [ 6/32]: OK
bootlin-armv7m-uclibc [ 7/32]: OK
bootlin-armv7-musl [ 8/32]: OK
bootlin-m68k-5208-uclibc [ 9/32]: SKIPPED
bootlin-m68k-68040-uclibc [10/32]: OK
bootlin-microblazeel-uclibc [11/32]: OK
bootlin-mips64el-glibc [12/32]: OK
bootlin-mipsel32r6-glibc [13/32]: OK
bootlin-mipsel-uclibc [14/32]: OK
bootlin-openrisc-uclibc [15/32]: OK
bootlin-powerpc64le-power8-glibc [16/32]: OK
bootlin-powerpc-e500mc-uclibc [17/32]: OK
bootlin-riscv32-glibc [18/32]: OK
bootlin-riscv64-glibc [19/32]: OK
bootlin-riscv64-musl [20/32]: OK
bootlin-s390x-z13-glibc [21/32]: OK
bootlin-sh4-uclibc [22/32]: OK
bootlin-sparc64-glibc [23/32]: OK
bootlin-sparc-uclibc [24/32]: OK
bootlin-x86-64-glibc [25/32]: OK
bootlin-x86-64-musl [26/32]: OK
bootlin-x86-64-uclibc [27/32]: OK
bootlin-x86-i686-musl [28/32]: OK
bootlin-xtensa-uclibc [29/32]: OK
br-arm-basic [30/32]: OK
br-arm-full-nothread [31/32]: SKIPPED
br-arm-full-static [32/32]: OK
32 builds, 2 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
---
$ ./utils/test-pkg -p librdkafka -c librdkafka.config -a
arm-aarch64 [ 1/32]: OK
bootlin-aarch64-glibc [ 2/32]: OK
bootlin-aarch64-glibc-old [ 3/32]: OK
bootlin-arcle-hs38-uclibc [ 4/32]: OK
bootlin-armv5-uclibc [ 5/32]: OK
bootlin-armv7-glibc [ 6/32]: OK
bootlin-armv7m-uclibc [ 7/32]: OK
bootlin-armv7-musl [ 8/32]: OK
bootlin-m68k-5208-uclibc [ 9/32]: SKIPPED
bootlin-m68k-68040-uclibc [10/32]: OK
bootlin-microblazeel-uclibc [11/32]: OK
bootlin-mips64el-glibc [12/32]: OK
bootlin-mipsel32r6-glibc [13/32]: OK
bootlin-mipsel-uclibc [14/32]: OK
bootlin-openrisc-uclibc [15/32]: OK
bootlin-powerpc64le-power8-glibc [16/32]: OK
bootlin-powerpc-e500mc-uclibc [17/32]: OK
bootlin-riscv32-glibc [18/32]: OK
bootlin-riscv64-glibc [19/32]: OK
bootlin-riscv64-musl [20/32]: OK
bootlin-s390x-z13-glibc [21/32]: OK
bootlin-sh4-uclibc [22/32]: OK
bootlin-sparc64-glibc [23/32]: OK
bootlin-sparc-uclibc [24/32]: OK
bootlin-x86-64-glibc [25/32]: OK
bootlin-x86-64-musl [26/32]: OK
bootlin-x86-64-uclibc [27/32]: OK
bootlin-x86-i686-musl [28/32]: OK
bootlin-xtensa-uclibc [29/32]: OK
br-arm-basic [30/32]: OK
br-arm-full-nothread [31/32]: SKIPPED
br-arm-full-static [32/32]: OK
32 builds, 2 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
$ cat librdkafka.config
BR2_PACKAGE_ZLIB=y
BR2_PACKAGE_LIBCURL=y
BR2_PACKAGE_ZSTD=y
BR2_PACKAGE_OPENSSL=y
BR2_PACKAGE_LIBRDKAFKA=y
---
DEVELOPERS | 1 +
package/Config.in | 1 +
...-allow-compilation-without-C-support.patch | 127 ++++++++++++++++++
....c-fix-curl-curl.h-include-when-WITH.patch | 32 +++++
...d.c-add-proper-HAVE_GETENTROPY-check.patch | 37 +++++
package/librdkafka/Config.in | 14 ++
package/librdkafka/librdkafka.hash | 4 +
package/librdkafka/librdkafka.mk | 51 +++++++
8 files changed, 267 insertions(+)
create mode 100644 package/librdkafka/0001-CMakeLists.txt-allow-compilation-without-C-support.patch
create mode 100644 package/librdkafka/0002-src-rdkafka_conf.c-fix-curl-curl.h-include-when-WITH.patch
create mode 100644 package/librdkafka/0003-src-rdrand.c-add-proper-HAVE_GETENTROPY-check.patch
create mode 100644 package/librdkafka/Config.in
create mode 100644 package/librdkafka/librdkafka.hash
create mode 100644 package/librdkafka/librdkafka.mkDEVELOPERS +1 -0
@@ -3191,6 +3191,7 @@ F: package/fcgiwrap/ N: Thomas Devoogdt <[email protected]> F: package/fluent-bit/ +F: package/librdkafka/ F: package/libsoup3/ F: package/yq/
package/Config.in +1 -0
@@ -2498,6 +2498,7 @@ endif source "package/leafnode2/Config.in" source "package/lft/Config.in" source "package/lftp/Config.in" + source "package/librdkafka/Config.in" source "package/lighttpd/Config.in" source "package/linknx/Config.in" source "package/links/Config.in"
package/librdkafka/0001-CMakeLists.txt-allow-compilation-without-C-support.patch new file +127 -0
127 lines not on this page · raw message
package/librdkafka/0002-src-rdkafka_conf.c-fix-curl-curl.h-include-when-WITH.patch new file +32 -0
@@ -0,0 +1,32 @@ +From 5061a5b33dce11ada740e2a34137dce6934b731a Mon Sep 17 00:00:00 2001 +From: Thomas Devoogdt <[email protected]> +Date: Sun, 15 Feb 2026 18:37:16 +0100 +Subject: [PATCH] src/rdkafka_conf.c: fix curl/curl.h include when + WITH_CURL=OFF + +Change #ifdef to #if for WITH_OAUTHBEARER_OIDC preprocessor check. +When using CMake's cmakedefine01, the macro is always defined (as 0 or 1), +so #ifdef always evaluates to true. Using #if correctly checks the value. + +Upstream: https://github.com/confluentinc/librdkafka/pull/5309 +Signed-off-by: Thomas Devoogdt <[email protected]> +--- + src/rdkafka_conf.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/rdkafka_conf.c b/src/rdkafka_conf.c +index 0e346e74..f8f0d0ca 100644 +--- a/src/rdkafka_conf.c ++++ b/src/rdkafka_conf.c +@@ -56,7 +56,7 @@ + #include <windows.h> + #endif + +-#ifdef WITH_OAUTHBEARER_OIDC ++#if WITH_OAUTHBEARER_OIDC + #include <curl/curl.h> + #endif + +-- +2.43.0 +
package/librdkafka/0003-src-rdrand.c-add-proper-HAVE_GETENTROPY-check.patch new file +37 -0
@@ -0,0 +1,37 @@ +From 692d1dc27b58231af9f0b96816f3445a66bfdbf3 Mon Sep 17 00:00:00 2001 +From: Thomas Devoogdt <[email protected]> +Date: Sun, 15 Feb 2026 19:08:27 +0100 +Subject: [PATCH] src/rdrand.c: add proper HAVE_GETENTROPY check + +/home/thomas/br-test-pkg/bootlin-aarch64-glibc-old/build/librdkafka-2.13.0/src/rdrand.c:37:24: fatal error: sys/random.h: No such file or directory + #include <sys/random.h> + ^ + +This should be covered by packaging/cmake/try_compile/getentropy_test.c. + +Upstream: https://github.com/confluentinc/librdkafka/pull/5310 +Signed-off-by: Thomas Devoogdt <[email protected]> +--- + src/rdrand.c | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/src/rdrand.c b/src/rdrand.c +index 77d02cfc..4bdc7885 100644 +--- a/src/rdrand.c ++++ b/src/rdrand.c +@@ -32,10 +32,12 @@ + #include "tinycthread.h" + #include "rdmurmur2.h" + #ifndef _WIN32 ++#if HAVE_GETENTROPY + /* getentropy() can be present in one of these two */ + #include <unistd.h> + #include <sys/random.h> + #endif ++#endif + + #ifdef HAVE_OSSL_SECURE_RAND_BYTES + #include <openssl/rand.h> +-- +2.43.0 +
package/librdkafka/Config.in new file +14 -0
@@ -0,0 +1,14 @@ +config BR2_PACKAGE_LIBRDKAFKA + bool "librdkafka" + depends on !BR2_m68k_cf # assembler "out of range" error + depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_LZ4 + help + librdkafka is a C library implementation of the + Apache Kafka protocol, providing Producer, Consumer + and Admin clients. + + https://github.com/confluentinc/librdkafka + +comment "librdkafka needs a toolchain w/ threads" + depends on !BR2_TOOLCHAIN_HAS_THREADS
package/librdkafka/librdkafka.hash new file +4 -0
@@ -0,0 +1,4 @@ +# From https://github.com/confluentinc/librdkafka/releases/tag/v2.13.0 +sha256 3bd351601d8ebcbc99b9a1316cae1b83b00edbcf9411c34287edf1791c507600 librdkafka-2.13.0.tar.gz +# Locally computed +sha256 b9e4616bb9dd0c9ce3fa52e81c7bfc596e60b8855a1df5dce5fa50732b1ccdec LICENSES.txt
package/librdkafka/librdkafka.mk new file +51 -0
51 lines not on this page · raw message
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 2/3 package/fluent-bit: link against the new librdkafka package
By this, we can drop 0001-lib-librdkafka-only-require-a-C-compiler.patch.
Signed-off-by Thomas Devoogdt <[email protected]>
--- ...librdkafka-only-require-a-C-compiler.patch | 118 ------------------ package/fluent-bit/Config.in | 2 +- package/fluent-bit/fluent-bit.mk | 5 +- 3 files changed, 4 insertions(+), 121 deletions(-) delete mode 100644 package/fluent-bit/0001-lib-librdkafka-only-require-a-C-compiler.patch
package/fluent-bit/0001-lib-librdkafka-only-require-a-C-compiler.patch deleted +0 -118
@@ -1,118 +0,0 @@ -From 7329316bdcc72aceb6bbbdf8711f5bf8894c2a15 Mon Sep 17 00:00:00 2001 -From: Thomas Devoogdt <[email protected]> -Date: Tue, 25 Jul 2023 09:10:41 +0200 -Subject: [PATCH] lib: librdkafka: only require a C compiler - -Upstream: https://github.com/confluentinc/librdkafka/pull/4366 -Signed-off-by: Thomas Devoogdt <[email protected]> ---- - lib/librdkafka-2.10.1/CMakeLists.txt | 22 +++++++++++-- - lib/librdkafka-2.10.1/examples/CMakeLists.txt | 31 ++++++++++++------- - 2 files changed, 39 insertions(+), 14 deletions(-) - -diff --git a/lib/librdkafka-2.10.1/CMakeLists.txt b/lib/librdkafka-2.10.1/CMakeLists.txt -index f3d05bad7..8ef47053a 100644 ---- a/lib/librdkafka-2.10.1/CMakeLists.txt -+++ b/lib/librdkafka-2.10.1/CMakeLists.txt -@@ -3,7 +3,16 @@ cmake_minimum_required(VERSION 3.5) - include("packaging/cmake/parseversion.cmake") - parseversion("src/rdkafka.h") - --project(RdKafka VERSION ${RDKAFKA_VERSION}) -+project(RdKafka VERSION ${RDKAFKA_VERSION} LANGUAGES C) -+ -+# Check for CXX support -+include(CheckLanguage) -+check_language(CXX) -+if(CMAKE_CXX_COMPILER) -+ enable_language(CXX) -+else() -+ message(STATUS "C++ compiler not found, skipping C++ support") -+endif() - - set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/packaging/cmake/Modules/") - -@@ -181,7 +190,11 @@ endif() - - option(RDKAFKA_BUILD_STATIC "Build static rdkafka library" OFF) - option(RDKAFKA_BUILD_EXAMPLES "Build examples" ON) --option(RDKAFKA_BUILD_TESTS "Build tests" ON) -+ -+if(CMAKE_CXX_COMPILER) -+ option(RDKAFKA_BUILD_TESTS "Build tests" ON) -+endif() -+ - if(WIN32) - option(WITHOUT_WIN32_CONFIG "Avoid including win32_config.h on cmake builds" ON) - endif(WIN32) -@@ -279,7 +292,10 @@ install( - ) - - add_subdirectory(src) --add_subdirectory(src-cpp) -+ -+if(CMAKE_CXX_COMPILER) -+ add_subdirectory(src-cpp) -+endif() - - if(RDKAFKA_BUILD_EXAMPLES) - add_subdirectory(examples) -diff --git a/lib/librdkafka-2.10.1/examples/CMakeLists.txt b/lib/librdkafka-2.10.1/examples/CMakeLists.txt -index 91851d2cb..6874195ce 100644 ---- a/lib/librdkafka-2.10.1/examples/CMakeLists.txt -+++ b/lib/librdkafka-2.10.1/examples/CMakeLists.txt -@@ -5,8 +5,10 @@ endif(WIN32) - add_executable(producer producer.c ${win32_sources}) - target_link_libraries(producer PUBLIC rdkafka) - --add_executable(producer_cpp producer.cpp ${win32_sources}) --target_link_libraries(producer_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(producer_cpp producer.cpp ${win32_sources}) -+ target_link_libraries(producer_cpp PUBLIC rdkafka++) -+endif() - - add_executable(consumer consumer.c ${win32_sources}) - target_link_libraries(consumer PUBLIC rdkafka) -@@ -14,14 +16,20 @@ target_link_libraries(consumer PUBLIC rdkafka) - add_executable(rdkafka_performance rdkafka_performance.c ${win32_sources}) - target_link_libraries(rdkafka_performance PUBLIC rdkafka) - --add_executable(rdkafka_example_cpp rdkafka_example.cpp ${win32_sources}) --target_link_libraries(rdkafka_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(rdkafka_example_cpp rdkafka_example.cpp ${win32_sources}) -+ target_link_libraries(rdkafka_example_cpp PUBLIC rdkafka++) -+endif() - --add_executable(rdkafka_complex_consumer_example_cpp rdkafka_complex_consumer_example.cpp ${win32_sources}) --target_link_libraries(rdkafka_complex_consumer_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(rdkafka_complex_consumer_example_cpp rdkafka_complex_consumer_example.cpp ${win32_sources}) -+ target_link_libraries(rdkafka_complex_consumer_example_cpp PUBLIC rdkafka++) -+endif() - --add_executable(openssl_engine_example_cpp openssl_engine_example.cpp ${win32_sources}) --target_link_libraries(openssl_engine_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(openssl_engine_example_cpp openssl_engine_example.cpp ${win32_sources}) -+ target_link_libraries(openssl_engine_example_cpp PUBLIC rdkafka++) -+endif() - - add_executable(misc misc.c ${win32_sources}) - target_link_libraries(misc PUBLIC rdkafka) -@@ -73,7 +81,8 @@ if(NOT WIN32) - add_executable(rdkafka_complex_consumer_example rdkafka_complex_consumer_example.c) - target_link_libraries(rdkafka_complex_consumer_example PUBLIC rdkafka) - -- add_executable(kafkatest_verifiable_client kafkatest_verifiable_client.cpp) -- target_link_libraries(kafkatest_verifiable_client PUBLIC rdkafka++) -- -+ if(CMAKE_CXX_COMPILER) -+ add_executable(kafkatest_verifiable_client kafkatest_verifiable_client.cpp) -+ target_link_libraries(kafkatest_verifiable_client PUBLIC rdkafka++) -+ endif() - endif(NOT WIN32) --- -2.43.0 -
package/fluent-bit/Config.in +1 -1
@@ -4,7 +4,7 @@ config BR2_PACKAGE_FLUENT_BIT depends on BR2_TOOLCHAIN_HAS_SYNC_4 depends on !BR2_STATIC_LIBS # dlfcn.h select BR2_PACKAGE_C_ARES - select BR2_PACKAGE_LIBOPENSSL_ENGINES # needed by bundled librdkafka + select BR2_PACKAGE_LIBRDKAFKA select BR2_PACKAGE_LIBYAML select BR2_PACKAGE_MSGPACK_C select BR2_PACKAGE_MUSL_FTS if !BR2_TOOLCHAIN_USES_GLIBC
package/fluent-bit/fluent-bit.mk +3 -2
@@ -10,8 +10,8 @@ FLUENT_BIT_LICENSE = Apache-2.0 FLUENT_BIT_LICENSE_FILES = LICENSE FLUENT_BIT_CPE_ID_VENDOR = treasuredata FLUENT_BIT_CPE_ID_PRODUCT = fluent_bit -FLUENT_BIT_DEPENDENCIES = c-ares host-bison host-flex libyaml msgpack-c \ - nghttp2 openssl sqlite zstd +FLUENT_BIT_DEPENDENCIES = c-ares host-bison host-flex librdkafka libyaml \ + msgpack-c nghttp2 openssl sqlite zstd FLUENT_BIT_CMAKE_BACKEND = ninja
@@ -22,6 +22,7 @@ FLUENT_BIT_CONF_OPTS += \ -DFLB_EXAMPLES=No \ -DFLB_CHUNK_TRACE=No \ -DFLB_PREFER_SYSTEM_LIB_CARES=Yes \ + -DFLB_PREFER_SYSTEM_LIB_KAFKA=Yes \ -DFLB_PREFER_SYSTEM_LIB_MSGPACK=Yes \ -DFLB_PREFER_SYSTEM_LIB_NGHTTP2=Yes \ -DFLB_PREFER_SYSTEM_LIB_SQLITE=Yes \
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 3/3 support/testing/tests/package/test_fluent_bit.py: new runtime test
In reply to Thomas DevoogdtSigned-off-by Thomas Devoogdt <[email protected]>
--- DEVELOPERS | 1 + .../testing/tests/package/test_fluent_bit.py | 46 +++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 support/testing/tests/package/test_fluent_bit.py
DEVELOPERS +1 -0
@@ -3194,6 +3194,7 @@ F: package/fluent-bit/ F: package/librdkafka/ F: package/libsoup3/ F: package/yq/ +F: support/testing/tests/package/test_fluent_bit.py N: Thomas Huth <[email protected]> F: package/ascii-invaders/
support/testing/tests/package/test_fluent_bit.py new file +46 -0
46 lines not on this page · raw message
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 1/3 package/librdkafka: add new package
In reply to Thomas DevoogdtWill be a dependency of fluent-bit.
Signed-off-by Thomas Devoogdt <[email protected]>
---
v2: bump to 2.13.2
---
$ ./utils/test-pkg -p librdkafka -a
arm-aarch64 [ 1/32]: OK
bootlin-aarch64-glibc [ 2/32]: OK
bootlin-aarch64-glibc-old [ 3/32]: OK
bootlin-arcle-hs38-uclibc [ 4/32]: OK
bootlin-armv5-uclibc [ 5/32]: OK
bootlin-armv7-glibc [ 6/32]: OK
bootlin-armv7m-uclibc [ 7/32]: OK
bootlin-armv7-musl [ 8/32]: OK
bootlin-m68k-5208-uclibc [ 9/32]: SKIPPED
bootlin-m68k-68040-uclibc [10/32]: OK
bootlin-microblazeel-uclibc [11/32]: OK
bootlin-mips64el-glibc [12/32]: OK
bootlin-mipsel32r6-glibc [13/32]: OK
bootlin-mipsel-uclibc [14/32]: OK
bootlin-openrisc-uclibc [15/32]: OK
bootlin-powerpc64le-power8-glibc [16/32]: OK
bootlin-powerpc-e500mc-uclibc [17/32]: OK
bootlin-riscv32-glibc [18/32]: OK
bootlin-riscv64-glibc [19/32]: OK
bootlin-riscv64-musl [20/32]: OK
bootlin-s390x-z13-glibc [21/32]: OK
bootlin-sh4-uclibc [22/32]: OK
bootlin-sparc64-glibc [23/32]: OK
bootlin-sparc-uclibc [24/32]: OK
bootlin-x86-64-glibc [25/32]: OK
bootlin-x86-64-musl [26/32]: OK
bootlin-x86-64-uclibc [27/32]: OK
bootlin-x86-i686-musl [28/32]: OK
bootlin-xtensa-uclibc [29/32]: OK
br-arm-basic [30/32]: OK
br-arm-full-nothread [31/32]: SKIPPED
br-arm-full-static [32/32]: OK
32 builds, 2 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
---
$ ./utils/test-pkg -p librdkafka -c librdkafka.config -a
arm-aarch64 [ 1/32]: OK
bootlin-aarch64-glibc [ 2/32]: OK
bootlin-aarch64-glibc-old [ 3/32]: OK
bootlin-arcle-hs38-uclibc [ 4/32]: OK
bootlin-armv5-uclibc [ 5/32]: OK
bootlin-armv7-glibc [ 6/32]: OK
bootlin-armv7m-uclibc [ 7/32]: OK
bootlin-armv7-musl [ 8/32]: OK
bootlin-m68k-5208-uclibc [ 9/32]: SKIPPED
bootlin-m68k-68040-uclibc [10/32]: OK
bootlin-microblazeel-uclibc [11/32]: OK
bootlin-mips64el-glibc [12/32]: OK
bootlin-mipsel32r6-glibc [13/32]: OK
bootlin-mipsel-uclibc [14/32]: OK
bootlin-openrisc-uclibc [15/32]: OK
bootlin-powerpc64le-power8-glibc [16/32]: OK
bootlin-powerpc-e500mc-uclibc [17/32]: OK
bootlin-riscv32-glibc [18/32]: OK
bootlin-riscv64-glibc [19/32]: OK
bootlin-riscv64-musl [20/32]: OK
bootlin-s390x-z13-glibc [21/32]: OK
bootlin-sh4-uclibc [22/32]: OK
bootlin-sparc64-glibc [23/32]: OK
bootlin-sparc-uclibc [24/32]: OK
bootlin-x86-64-glibc [25/32]: OK
bootlin-x86-64-musl [26/32]: OK
bootlin-x86-64-uclibc [27/32]: OK
bootlin-x86-i686-musl [28/32]: OK
bootlin-xtensa-uclibc [29/32]: OK
br-arm-basic [30/32]: OK
br-arm-full-nothread [31/32]: SKIPPED
br-arm-full-static [32/32]: OK
32 builds, 2 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
$ cat librdkafka.config
BR2_PACKAGE_ZLIB=y
BR2_PACKAGE_LIBCURL=y
BR2_PACKAGE_ZSTD=y
BR2_PACKAGE_OPENSSL=y
BR2_PACKAGE_LIBRDKAFKA=y
---
DEVELOPERS | 1 +
package/Config.in | 1 +
...-allow-compilation-without-C-support.patch | 127 ++++++++++++++++++
....c-fix-curl-curl.h-include-when-WITH.patch | 32 +++++
package/librdkafka/Config.in | 14 ++
package/librdkafka/librdkafka.hash | 4 +
package/librdkafka/librdkafka.mk | 51 +++++++
7 files changed, 230 insertions(+)
create mode 100644 package/librdkafka/0001-CMakeLists.txt-allow-compilation-without-C-support.patch
create mode 100644 package/librdkafka/0002-src-rdkafka_conf.c-fix-curl-curl.h-include-when-WITH.patch
create mode 100644 package/librdkafka/Config.in
create mode 100644 package/librdkafka/librdkafka.hash
create mode 100644 package/librdkafka/librdkafka.mkDEVELOPERS +1 -0
@@ -3194,6 +3194,7 @@ F: package/fcgiwrap/ N: Thomas Devoogdt <[email protected]> F: package/fluent-bit/ +F: package/librdkafka/ F: package/libsoup3/ F: package/yq/
package/Config.in +1 -0
@@ -2499,6 +2499,7 @@ endif source "package/leafnode2/Config.in" source "package/lft/Config.in" source "package/lftp/Config.in" + source "package/librdkafka/Config.in" source "package/lighttpd/Config.in" source "package/linknx/Config.in" source "package/links/Config.in"
package/librdkafka/0001-CMakeLists.txt-allow-compilation-without-C-support.patch new file +127 -0
127 lines not on this page · raw message
package/librdkafka/0002-src-rdkafka_conf.c-fix-curl-curl.h-include-when-WITH.patch new file +32 -0
@@ -0,0 +1,32 @@ +From 5061a5b33dce11ada740e2a34137dce6934b731a Mon Sep 17 00:00:00 2001 +From: Thomas Devoogdt <[email protected]> +Date: Sun, 15 Feb 2026 18:37:16 +0100 +Subject: [PATCH] src/rdkafka_conf.c: fix curl/curl.h include when + WITH_CURL=OFF + +Change #ifdef to #if for WITH_OAUTHBEARER_OIDC preprocessor check. +When using CMake's cmakedefine01, the macro is always defined (as 0 or 1), +so #ifdef always evaluates to true. Using #if correctly checks the value. + +Upstream: https://github.com/confluentinc/librdkafka/pull/5309 +Signed-off-by: Thomas Devoogdt <[email protected]> +--- + src/rdkafka_conf.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/rdkafka_conf.c b/src/rdkafka_conf.c +index 0e346e74..f8f0d0ca 100644 +--- a/src/rdkafka_conf.c ++++ b/src/rdkafka_conf.c +@@ -56,7 +56,7 @@ + #include <windows.h> + #endif + +-#ifdef WITH_OAUTHBEARER_OIDC ++#if WITH_OAUTHBEARER_OIDC + #include <curl/curl.h> + #endif + +-- +2.43.0 +
package/librdkafka/Config.in new file +14 -0
@@ -0,0 +1,14 @@ +config BR2_PACKAGE_LIBRDKAFKA + bool "librdkafka" + depends on !BR2_m68k_cf # assembler "out of range" error + depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_LZ4 + help + librdkafka is a C library implementation of the + Apache Kafka protocol, providing Producer, Consumer + and Admin clients. + + https://github.com/confluentinc/librdkafka + +comment "librdkafka needs a toolchain w/ threads" + depends on !BR2_TOOLCHAIN_HAS_THREADS
package/librdkafka/librdkafka.hash new file +4 -0
@@ -0,0 +1,4 @@ +# From https://github.com/confluentinc/librdkafka/releases/tag/v2.13.2 +sha256 14972092e4115f6e99f798a7cb420cbf6daa0c73502b3c52ae42fb5b418eea8f librdkafka-2.13.2.tar.gz +# Locally computed +sha256 b9e4616bb9dd0c9ce3fa52e81c7bfc596e60b8855a1df5dce5fa50732b1ccdec LICENSES.txt
package/librdkafka/librdkafka.mk new file +51 -0
51 lines not on this page · raw message
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 2/3 package/fluent-bit: link against the new librdkafka package
By this, we can drop 0001-lib-librdkafka-only-require-a-C-compiler.patch.
Signed-off-by Thomas Devoogdt <[email protected]>
---
v2: no change
--- ...librdkafka-only-require-a-C-compiler.patch | 118 ------------------ package/fluent-bit/Config.in | 2 +- package/fluent-bit/fluent-bit.mk | 5 +- 3 files changed, 4 insertions(+), 121 deletions(-) delete mode 100644 package/fluent-bit/0001-lib-librdkafka-only-require-a-C-compiler.patch
package/fluent-bit/0001-lib-librdkafka-only-require-a-C-compiler.patch deleted +0 -118
@@ -1,118 +0,0 @@ -From 7329316bdcc72aceb6bbbdf8711f5bf8894c2a15 Mon Sep 17 00:00:00 2001 -From: Thomas Devoogdt <[email protected]> -Date: Tue, 25 Jul 2023 09:10:41 +0200 -Subject: [PATCH] lib: librdkafka: only require a C compiler - -Upstream: https://github.com/confluentinc/librdkafka/pull/4366 -Signed-off-by: Thomas Devoogdt <[email protected]> ---- - lib/librdkafka-2.10.1/CMakeLists.txt | 22 +++++++++++-- - lib/librdkafka-2.10.1/examples/CMakeLists.txt | 31 ++++++++++++------- - 2 files changed, 39 insertions(+), 14 deletions(-) - -diff --git a/lib/librdkafka-2.10.1/CMakeLists.txt b/lib/librdkafka-2.10.1/CMakeLists.txt -index f3d05bad7..8ef47053a 100644 ---- a/lib/librdkafka-2.10.1/CMakeLists.txt -+++ b/lib/librdkafka-2.10.1/CMakeLists.txt -@@ -3,7 +3,16 @@ cmake_minimum_required(VERSION 3.5) - include("packaging/cmake/parseversion.cmake") - parseversion("src/rdkafka.h") - --project(RdKafka VERSION ${RDKAFKA_VERSION}) -+project(RdKafka VERSION ${RDKAFKA_VERSION} LANGUAGES C) -+ -+# Check for CXX support -+include(CheckLanguage) -+check_language(CXX) -+if(CMAKE_CXX_COMPILER) -+ enable_language(CXX) -+else() -+ message(STATUS "C++ compiler not found, skipping C++ support") -+endif() - - set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/packaging/cmake/Modules/") - -@@ -181,7 +190,11 @@ endif() - - option(RDKAFKA_BUILD_STATIC "Build static rdkafka library" OFF) - option(RDKAFKA_BUILD_EXAMPLES "Build examples" ON) --option(RDKAFKA_BUILD_TESTS "Build tests" ON) -+ -+if(CMAKE_CXX_COMPILER) -+ option(RDKAFKA_BUILD_TESTS "Build tests" ON) -+endif() -+ - if(WIN32) - option(WITHOUT_WIN32_CONFIG "Avoid including win32_config.h on cmake builds" ON) - endif(WIN32) -@@ -279,7 +292,10 @@ install( - ) - - add_subdirectory(src) --add_subdirectory(src-cpp) -+ -+if(CMAKE_CXX_COMPILER) -+ add_subdirectory(src-cpp) -+endif() - - if(RDKAFKA_BUILD_EXAMPLES) - add_subdirectory(examples) -diff --git a/lib/librdkafka-2.10.1/examples/CMakeLists.txt b/lib/librdkafka-2.10.1/examples/CMakeLists.txt -index 91851d2cb..6874195ce 100644 ---- a/lib/librdkafka-2.10.1/examples/CMakeLists.txt -+++ b/lib/librdkafka-2.10.1/examples/CMakeLists.txt -@@ -5,8 +5,10 @@ endif(WIN32) - add_executable(producer producer.c ${win32_sources}) - target_link_libraries(producer PUBLIC rdkafka) - --add_executable(producer_cpp producer.cpp ${win32_sources}) --target_link_libraries(producer_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(producer_cpp producer.cpp ${win32_sources}) -+ target_link_libraries(producer_cpp PUBLIC rdkafka++) -+endif() - - add_executable(consumer consumer.c ${win32_sources}) - target_link_libraries(consumer PUBLIC rdkafka) -@@ -14,14 +16,20 @@ target_link_libraries(consumer PUBLIC rdkafka) - add_executable(rdkafka_performance rdkafka_performance.c ${win32_sources}) - target_link_libraries(rdkafka_performance PUBLIC rdkafka) - --add_executable(rdkafka_example_cpp rdkafka_example.cpp ${win32_sources}) --target_link_libraries(rdkafka_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(rdkafka_example_cpp rdkafka_example.cpp ${win32_sources}) -+ target_link_libraries(rdkafka_example_cpp PUBLIC rdkafka++) -+endif() - --add_executable(rdkafka_complex_consumer_example_cpp rdkafka_complex_consumer_example.cpp ${win32_sources}) --target_link_libraries(rdkafka_complex_consumer_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(rdkafka_complex_consumer_example_cpp rdkafka_complex_consumer_example.cpp ${win32_sources}) -+ target_link_libraries(rdkafka_complex_consumer_example_cpp PUBLIC rdkafka++) -+endif() - --add_executable(openssl_engine_example_cpp openssl_engine_example.cpp ${win32_sources}) --target_link_libraries(openssl_engine_example_cpp PUBLIC rdkafka++) -+if(CMAKE_CXX_COMPILER) -+ add_executable(openssl_engine_example_cpp openssl_engine_example.cpp ${win32_sources}) -+ target_link_libraries(openssl_engine_example_cpp PUBLIC rdkafka++) -+endif() - - add_executable(misc misc.c ${win32_sources}) - target_link_libraries(misc PUBLIC rdkafka) -@@ -73,7 +81,8 @@ if(NOT WIN32) - add_executable(rdkafka_complex_consumer_example rdkafka_complex_consumer_example.c) - target_link_libraries(rdkafka_complex_consumer_example PUBLIC rdkafka) - -- add_executable(kafkatest_verifiable_client kafkatest_verifiable_client.cpp) -- target_link_libraries(kafkatest_verifiable_client PUBLIC rdkafka++) -- -+ if(CMAKE_CXX_COMPILER) -+ add_executable(kafkatest_verifiable_client kafkatest_verifiable_client.cpp) -+ target_link_libraries(kafkatest_verifiable_client PUBLIC rdkafka++) -+ endif() - endif(NOT WIN32) --- -2.43.0 -
package/fluent-bit/Config.in +1 -1
@@ -4,7 +4,7 @@ config BR2_PACKAGE_FLUENT_BIT depends on BR2_TOOLCHAIN_HAS_SYNC_4 depends on !BR2_STATIC_LIBS # dlfcn.h select BR2_PACKAGE_C_ARES - select BR2_PACKAGE_LIBOPENSSL_ENGINES # needed by bundled librdkafka + select BR2_PACKAGE_LIBRDKAFKA select BR2_PACKAGE_LIBYAML select BR2_PACKAGE_MSGPACK_C select BR2_PACKAGE_MUSL_FTS if !BR2_TOOLCHAIN_USES_GLIBC
package/fluent-bit/fluent-bit.mk +3 -2
@@ -10,8 +10,8 @@ FLUENT_BIT_LICENSE = Apache-2.0 FLUENT_BIT_LICENSE_FILES = LICENSE FLUENT_BIT_CPE_ID_VENDOR = treasuredata FLUENT_BIT_CPE_ID_PRODUCT = fluent_bit -FLUENT_BIT_DEPENDENCIES = c-ares host-bison host-flex libyaml msgpack-c \ - nghttp2 openssl sqlite zstd +FLUENT_BIT_DEPENDENCIES = c-ares host-bison host-flex librdkafka libyaml \ + msgpack-c nghttp2 openssl sqlite zstd FLUENT_BIT_CMAKE_BACKEND = ninja
@@ -22,6 +22,7 @@ FLUENT_BIT_CONF_OPTS += \ -DFLB_EXAMPLES=No \ -DFLB_CHUNK_TRACE=No \ -DFLB_PREFER_SYSTEM_LIB_CARES=Yes \ + -DFLB_PREFER_SYSTEM_LIB_KAFKA=Yes \ -DFLB_PREFER_SYSTEM_LIB_MSGPACK=Yes \ -DFLB_PREFER_SYSTEM_LIB_NGHTTP2=Yes \ -DFLB_PREFER_SYSTEM_LIB_SQLITE=Yes \
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH v2 3/3 support/testing/tests/package/test_fluent_bit.py: new runtime test
In reply to Thomas DevoogdtSigned-off-by Thomas Devoogdt <[email protected]>
---
v2: no change
--- DEVELOPERS | 1 + .../testing/tests/package/test_fluent_bit.py | 46 +++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 support/testing/tests/package/test_fluent_bit.py
DEVELOPERS +1 -0
@@ -3197,6 +3197,7 @@ F: package/fluent-bit/ F: package/librdkafka/ F: package/libsoup3/ F: package/yq/ +F: support/testing/tests/package/test_fluent_bit.py N: Thomas Huth <[email protected]> F: package/ascii-invaders/
support/testing/tests/package/test_fluent_bit.py new file +46 -0
46 lines not on this page · raw message
Signature · 6 lines
2.43.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Raphaël Mélotte via buildroot
AppliedRe: [Buildroot] [PATCH v2 3/3] support/testing/tests/package/test_fluent_bit.py: new runtime test
In reply of:
Quoted 1 line
Signed-off-by Thomas Devoogdt <[email protected]>
Applied to 2025.02.x, 2026.08.x. Thanks
Quoted 62 lines
---
v2: no change--- DEVELOPERS | 1 + .../testing/tests/package/test_fluent_bit.py | 46 +++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100644 support/testing/tests/package/test_fluent_bit.pyDEVELOPERS +1
-0@@ -3197,6 +3197,7 @@ F: package/fluent-bit/ F: package/librdkafka/ F: package/libsoup3/ F: package/yq/ +F: support/testing/tests/package/test_fluent_bit.py N: Thomas Huth <[email protected]> F: package/ascii-invaders/support/testing/tests/package/test_fluent_bit.py new file +36
-0@@ -0,0 +1,46 @@ +import os +import json +import time + +import infra.basetest + + +class TestFluentBit(infra.basetest.BRTest): + config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \ + """ + BR2_TARGET_ROOTFS_CPIO=y + # BR2_TARGET_ROOTFS_TAR is not set + BR2_PACKAGE_FLUENT_BIT=y + """ + + def test_run(self): + cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio") + self.emulator.boot(arch="armv5", + kernel="builtin", + options=["-initrd", cpio_file]) + self.emulator.login() + + # Enable the http server + cmd = ("sed -i 's/http_server Off/http_server On/g' " + "/etc/fluent-bit/fluent-bit.conf && " + "/etc/init.d/S99fluent-bit restart") + _, exit_code = self.emulator.run(cmd) + self.assertEqual(exit_code, 0) + + # Give some runtime + time.sleep(5) + + # Check the uptime + cmd = "wget -qO- http://127.0.0.1:2020/api/v1/uptime" + output, exit_code = self.emulator.run(cmd) + self.assertEqual(exit_code, 0)
support/testing/tests/package/test_fluent_bit.py line 46
Quoted 10 lines
support/testing/tests/package/test_fluent_bit.py new file +10
-0+ uptime = json.loads(output[0]) + self.assertGreater(uptime["uptime_sec"], 3) + + # Check the metrics + cmd = "wget -qO- http://127.0.0.1:2020/api/v1/metrics" + output, exit_code = self.emulator.run(cmd) + self.assertEqual(exit_code, 0) + metrics = json.loads(output[0]) + self.assertGreater(metrics["input"]["cpu.0"]["records"], 0) + self.assertGreater(metrics["output"]["stdout.0"]["proc_records"], 0)
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Hello,
package/librdkafka/Config.in line 3
Quoted 9 lines · Thomas Devoogdt
package/librdkafka/Config.in new file +3
-0@@ -0,0 +1,14 @@ +config BR2_PACKAGE_LIBRDKAFKA + bool "librdkafka" + depends on !BR2_m68k_cf # assembler "out of range" error
Minor nit: this architecture dependency...
package/librdkafka/Config.in line 13
Quoted 10 lines
package/librdkafka/Config.in new file +10
-0+ depends on BR2_TOOLCHAIN_HAS_THREADS + select BR2_PACKAGE_LZ4 + help + librdkafka is a C library implementation of the + Apache Kafka protocol, providing Producer, Consumer + and Admin clients. + + https://github.com/confluentinc/librdkafka + +comment "librdkafka needs a toolchain w/ threads"
... should have been replicated here.
package/librdkafka/Config.in line 14
Quoted 1 line
package/librdkafka/Config.in new file +1
-0+ depends on !BR2_TOOLCHAIN_HAS_THREADS
Other than that, the series looked good to me, so: whole series applied to master. Thanks, and sorry for the delay!
Perhaps one concerning issue though is that none of your patches submitted upstream have been reviewed/merged. But oh well, maybe they are as slow as we are in Buildroot :/
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering and training https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot