Skip to contentSkip to conversation

package/llvm: add BPF backend

PATCH New Jugurtha BELKALEM 3 participants 6 messages Aug 24, 2020 – Nov 24, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 package/llvm: add BPF backend

bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.

BCC requires enabling LLVM backend BPF support.

---
 package/llvm/Config.in | 6 ++++++
 package/llvm/llvm.mk   | 5 +++++
 2 files changed, 11 insertions(+)
package/llvm/Config.in +6 -0
index 1d21d87..6ec2ffc 100644
@@ -47,6 +47,12 @@ config BR2_PACKAGE_LLVM_RTTI
 
 	  https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html
 
+config BR2_PACKAGE_LLVM_BPF
+	bool "BPF backend"
+	help
+	  Build BPF target. Select this option if you are going
+	  to install bcc on the target.
+
 endif
 
 comment "llvm needs a toolchain w/ wchar, threads, C++, gcc >= 4.8, dynamic library"
package/llvm/llvm.mk +5 -0
index 24d033d..48a2981 100644
@@ -61,6 +61,11 @@ ifeq ($(BR2_PACKAGE_LLVM_AMDGPU),y)
 LLVM_TARGETS_TO_BUILD += AMDGPU
 endif
 
+# Build BPF backend
+ifeq ($(BR2_PACKAGE_LLVM_BPF),y)
+LLVM_TARGETS_TO_BUILD += BPF
+endif
+
 # Use native llvm-tblgen from host-llvm (needed for cross-compilation)
 LLVM_CONF_OPTS += -DLLVM_TABLEGEN=$(HOST_DIR)/bin/llvm-tblgen
 
Signature · 1 line
2.7.4
#1 lore raw

PATCH 2/2 package/bcc: new package

bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.

bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).

bcc can also make it easy to create observabilty tools, SDN configuration, ddos mitigation, intrusion detection and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.

BCC can be tested on the target :

$ mount -t debugfs none /sys/kernel/debug
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py

V3
- Bump to version 0.16.0.
- Add required python dependency.
- Remove unnecessary and duplicated
kernel flags (#Testing section) in bcc.mk.
-------------
V2
- Add eBPF's required Kernel flags.
- Fix submodule source fetch problem.
- Add toolchain dependency.

---
 DEVELOPERS                                       |  1 +
 package/Config.in                                |  1 +
 package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
 package/bcc/Config.in                            | 49 ++++++++++++++++++
 package/bcc/bcc.hash                             |  3 ++
 package/bcc/bcc.mk                               | 57 +++++++++++++++++++++
 6 files changed, 176 insertions(+)
 create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
 create mode 100644 package/bcc/Config.in
 create mode 100644 package/bcc/bcc.hash
 create mode 100644 package/bcc/bcc.mk
DEVELOPERS +1 -0
index 42fa5a4..bd836a6 100644
@@ -1396,6 +1396,7 @@ N:	Joshua Henderson <[email protected]>
 F:	package/qt5/qt5wayland/
 
 N:	Jugurtha BELKALEM <[email protected]>
+F:	package/bcc/
 F:	package/python-cycler/
 F:	package/python-matplotlib/
 
package/Config.in +1 -0
index d7e79f4..8ec3932 100644
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
 endmenu
 
 menu "Debugging, profiling and benchmark"
+	source "package/bcc/Config.in"
 	source "package/blktrace/Config.in"
 	source "package/bonnie/Config.in"
 	source "package/cache-calibrator/Config.in"
package/bcc/0001-fix-aarch64-cross-compile.patch new file +65 -0

65 lines not on this page · raw message

package/bcc/Config.in new file +49 -0

49 lines not on this page · raw message

package/bcc/bcc.hash new file +3 -0
new file mode 100644
index 0000000..e3fa662
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d  bcc-v0.16.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1  LICENSE.txt
package/bcc/bcc.mk new file +57 -0

57 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw

PATCH 2/2 package/bcc: new package

Hi Jugurtha

I am very late to the party and missed earlier reviews. So apologies in advance if any of my comments was already brought up/addressed.

Can you keep me on CC on future posting and any other related discussion please?

Quoted 5 lines · Jugurtha BELKALEM

bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.

Quoted 8 lines

bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).

bcc can also make it easy to create observabilty tools, SDN configuration, ddos mitigation, intrusion detection and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.

There's the brand new ebpf website too

	http://www.ebpf.io
Quoted 3 lines

BCC can be tested on the target :

$ mount -t debugfs none /sys/kernel/debug

I wonder if this something we can automatically append to /etc/fstab when this package is enabled..

Quoted 5 lines
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar

All the above is unnecessary. BCC automatically does this if it detects /sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC to handle this correctly. I guess busybox::tar is too limited to work with BCC commands.

Quoted 2 lines
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py

/usr/share/bcc/tools contains actual tools. I recommend picking up an example from there.

Quoted 11 lines

V3
- Bump to version 0.16.0.
- Add required python dependency.
- Remove unnecessary and duplicated
kernel flags (#Testing section) in bcc.mk.
-------------
V2
- Add eBPF's required Kernel flags.
- Fix submodule source fetch problem.
- Add toolchain dependency.

Did you intend to include these in the commit message?

Quoted 45 lines
---
 DEVELOPERS                                       |  1 +
 package/Config.in                                |  1 +
 package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
 package/bcc/Config.in                            | 49 ++++++++++++++++++
 package/bcc/bcc.hash                             |  3 ++
 package/bcc/bcc.mk                               | 57 +++++++++++++++++++++
 6 files changed, 176 insertions(+)
 create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
 create mode 100644 package/bcc/Config.in
 create mode 100644 package/bcc/bcc.hash
 create mode 100644 package/bcc/bcc.mk
DEVELOPERS +1 -0
index 42fa5a4..bd836a6 100644
@@ -1396,6 +1396,7 @@ N:	Joshua Henderson <[email protected]>
 F:	package/qt5/qt5wayland/
 
 N:	Jugurtha BELKALEM <[email protected]>
+F:	package/bcc/
 F:	package/python-cycler/
 F:	package/python-matplotlib/
 
package/Config.in +1 -0
index d7e79f4..8ec3932 100644
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
 endmenu
 
 menu "Debugging, profiling and benchmark"
+	source "package/bcc/Config.in"
 	source "package/blktrace/Config.in"
 	source "package/bonnie/Config.in"
 	source "package/cache-calibrator/Config.in"
package/bcc/0001-fix-aarch64-cross-compile.patch new file +0 -0
new file mode 100644
index 0000000..6b42797

package/bcc/0001-fix-aarch64-cross-compile.patch line 5

Quoted 5 lines
package/bcc/0001-fix-aarch64-cross-compile.patch new file +5 -0
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <[email protected]>
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling

Is this luajit required to enable Lua front-end to BCC? Given python is the most common front-end, I'd argue to make this an extra option that is only enabled if the user selects it in the menu. Same way BPF backend in LLVM is only compiled in if selected. IMHO that's a better default behavior.

Quoted 66 lines
package/bcc/0001-fix-aarch64-cross-compile.patch new file +60 -0
+
+Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
+as a valid architecture, luajit does not recognize it.
+luajit defines aarch64 as arm64.
+
+LuaJIT doesn't use usual arch naming, so we have to convert
+between CMake and Luajit for each architectures while
+cross-compiling.
+
+Signed-off-by: Jugurtha BELKALEM <[email protected]>
+Signed-off-by: Romain Naour <[email protected]>
+---
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ 		DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ 	)
+ 
++	# LuaJIT doesn't use usual arch naming, so we have to convert
++	# between CMake and Luajit while cross-compiling.
++	if (CMAKE_CROSSCOMPILING)
++		SET (LUAJIT_TARGET_ARCH "-a")
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++		if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++		else()
++			MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++		endif()
++	endif()
++
+ 	ADD_CUSTOM_COMMAND(
+ 		OUTPUT bcc.o
+-		COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++		COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ 		DEPENDS bcc.lua
+ 	)
+ 
package/bcc/Config.in new file +0 -0
new file mode 100644
index 0000000..cf03927

package/bcc/Config.in line 6

Quoted 6 lines
package/bcc/Config.in new file +6 -0
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+	bool "bcc"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+	depends on BR2_LINUX_KERNEL # needs kernel sources on the target

This dependency on building the kernel in buildroot is not user friendly. For example I never build the kernel in buildroot, and certainly would like to be able to enable this package without having to build the kernel in buildroot.

package/bcc/Config.in line 10

Quoted 5 lines
package/bcc/Config.in new file +4 -0
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+	depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+	depends on BR2_INSTALL_LIBSTDCPP # clang
 
+	select BR2_PACKAGE_AUDIT # runtime

This..

package/bcc/Config.in line 14

Quoted 5 lines
package/bcc/Config.in new file +4 -0
+	select BR2_PACKAGE_CLANG
+	select BR2_PACKAGE_ELFUTILS
+	select BR2_PACKAGE_FLEX # needs FlexLexer.h
 
+	select BR2_PACKAGE_IPERF3 # runtime

.. this..

package/bcc/Config.in line 17

Quoted 4 lines
package/bcc/Config.in new file +3 -0
+	select BR2_PACKAGE_LLVM_BPF
+	select BR2_PACKAGE_LUAJIT
 
+	select BR2_PACKAGE_NETPERF # runtime

.. and this look odd to me. Can you expand which runtime requires them?

package/bcc/Config.in line 19

Quoted 2 lines
package/bcc/Config.in new file +2 -0
+	select BR2_PACKAGE_PYTHON3
+	select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC

If you replace XZ with TAR, you should find that BCC automatically handles the ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel that could replace the dependency on the headers. Haven't played much with that yet though. BTF requires a tool called pahole. We can handle this support later I guess.

Quoted 13 lines
package/bcc/Config.in new file +13 -0
+	help
+	  BPF Compiler Collection (BCC)
+
+	  BCC is a toolkit for creating efficient kernel tracing and
+	  manipulation programs, and includes several useful tools and
+	  examples. It makes use of extended BPF (Berkeley Packet
+	  Filters), formally known as eBPF, a new feature that was
+	  first added to Linux 3.15. Much of what BCC uses requires
+	  Linux 4.1 and above.
+
+	  Note: Before using bcc, you need either need to :
+	  - For kernel_ver = [4.1, 5.2) : Copy kernel source code
+	  to target folder /lib/module/<kernel_ver>/build.

package/bcc/Config.in line 42

Quoted 10 lines
package/bcc/Config.in new file +10 -0
+	  - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
+	  and use generated headers under /sys/kernel/kheaders.tar.xz
+	  to populate /lib/module/<kernel_ver>/build.
+
+	  That's because the clang frontend build eBPF code at runtime.
+
+	  https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+	depends on !BR2_LINUX_KERNEL

I must remove this to compile this patch. This hard dependency is not required IMHO. You can still auto fix up the kernel if you like if the user is building the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so the set of configs might require a bit of maintenance. Given it's all documented in BCC website, I personally think it's better to leave correct kernel configuration to the user to follow the right instructions from BCC website rather than automatically handle it for them.

Quoted 45 lines
package/bcc/Config.in new file +7 -0
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+	depends on BR2_LINUX_KERNEL
+	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+		|| !BR2_INSTALL_LIBSTDCPP
package/bcc/bcc.hash new file +3 -0
new file mode 100644
index 0000000..e3fa662
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d  bcc-v0.16.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1  LICENSE.txt
package/bcc/bcc.mk new file +23 -0
new file mode 100644
index 0000000..4fe69cb
@@ -0,0 +1,57 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.16.0
+BCC_SITE = https://github.com/iovisor/bcc.git
+BCC_SITE_METHOD = git
+BCC_GIT_SUBMODULES = YES
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
+		   luajit python3
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+	-DREVISION=$(BCC_VERSION) \
+	-DENABLE_CLANG_JIT=ON \
+	-DENABLE_MAN=OFF

package/bcc/bcc.mk line 33

Quoted 10 lines
package/bcc/bcc.mk new file +10 -0
+
+# BCC requires python-bcc (shiped under bcc/src/python) to work.
+# However, BCC compiles bcc/src/python/setup.py.in into bcc/src/python.
+# The generated compiled folder (bcc/src/python/bcc-python/bcc) should
+# be copied to target python's site-packages directory.
+define BCC_INSTALL_PYTHON_BCC
+	cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
+	$(TARGET_DIR)/usr/lib/python3.*/site-packages/
+endef
+BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC

I think you're hitting a bug in BCC build system which causes the package to be installed in the wrong location on the target.

Does the attached patch help?

Quoted 12 lines
package/bcc/bcc.mk new file +12 -0
+
+define BCC_LINUX_CONFIG_FIXUPS
+	# Enable kernel support for eBPF
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+	# [for Linux kernel versions 4.1 through 4.6]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+	# [for Linux kernel versions 4.7 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)

package/bcc/bcc.mk line 55

Quoted 10 lines
package/bcc/bcc.mk new file +10 -0
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+	# [for Linux kernel versions 5.2 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+	# For running bcc networking examples on vanilla kernel
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+endef

This will be a big maintenance headache. New dependency can come up and configs like CONFIG_NET_* are not essential and seem to be specific to your usage.

I'm not in favour of them personally, but if you want them the list need to shrink to the bare minimum and depend on user building the kernel in buildroot IMO.

Thanks!

--
Qais Yousef

package/bcc/bcc.mk line 57

Quoted 2 lines
package/bcc/bcc.mk new file +2 -0
+
+$(eval $(cmake-package))

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-fix-python-installation-path.patch
Type: text/x-diff
Size: 531 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20201004/d99fa0bb/attachment.patch>

#3 lore raw

PATCH 2/2 package/bcc: new package

Hi Jugurtha

On 10/04/20 19:12, Qais Yousef wrote:

[...]

Quoted 41 lines
Quoted 41 lines

+---

package/bcc/0001-fix-aarch64-cross-compile.patch new file +40 -0
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ 		DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ 	)
+ 
++	# LuaJIT doesn't use usual arch naming, so we have to convert
++	# between CMake and Luajit while cross-compiling.
++	if (CMAKE_CROSSCOMPILING)
++		SET (LUAJIT_TARGET_ARCH "-a")
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++		if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++		else()
++			MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++		endif()
++	endif()
Quoted 8 lines

package/bcc/0001-fix-aarch64-cross-compile.patch line 65

Quoted 8 lines

++

package/bcc/0001-fix-aarch64-cross-compile.patch new file +7 -0
+ 	ADD_CUSTOM_COMMAND(
+ 		OUTPUT bcc.o
+-		COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++		COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ 		DEPENDS bcc.lua
+ 	)
+ 

With the attached patch where I remove some of the dependencies I could compile and use bcc and apply my bpftrace patch on top and use it.

Maybe I missed trying something that you do that could make this dependency obvious. As it stands, it seems this lua front-end support requires this functionality + a specific use case. iperf3 dependency particularly looks strange to me as I can't think why bcc would need that.

I suggest wrapping wrapping the lua front-end support with its own config option and keep it in a separate patch.

We can remove PYTHON_BCC copying if we apply a fixup patch (included in the attached patch). I'll try to find a proper solution that can be upstreamed back. If you know the right solution, please go ahead and do it of course :-)

Thanks!

--
Qais Yousef
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcc.fixup.patch
Type: text/x-diff
Size: 3757 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20201008/650be8ba/attachment.patch>

#4 lore raw

PATCH 2/2 package/bcc: new package

In reply to Qais Yousef

Hello Qais,

Sorry for my late reply.

Le 04/10/2020 ? 20:12, Qais Yousef a ?crit?:

Quoted 7 lines

Hi Jugurtha

I am very late to the party and missed earlier reviews. So apologies in advance if any of my comments was already brought up/addressed.

Can you keep me on CC on future posting and any other related discussion please?

I'm reworking the series and I'm adding you in Cc for each patch. Jugurtha is not available at the moment, but he is still interested by BCC.

Quoted 11 lines · Jugurtha BELKALEM
Quoted 9 lines

bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.

bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).

Quoted 8 lines
Quoted 4 lines

bcc can also make it easy to create observabilty tools, SDN configuration, ddos mitigation, intrusion detection and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.

There's the brand new ebpf website too

	http://www.ebpf.io

Link updated, thanks!

Quoted 7 lines
Quoted 4 lines

BCC can be tested on the target :

$ mount -t debugfs none /sys/kernel/debug

I wonder if this something we can automatically append to /etc/fstab when this package is enabled..

Sure, but this is a fstab customization. I don't think the bcc package should change the /etc/fstab.

Quoted 11 lines
Quoted 6 lines
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar

All the above is unnecessary. BCC automatically does this if it detects /sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC to handle this correctly. I guess busybox::tar is too limited to work with BCC commands.

Indeed, thanks for the info.
Now I see:
https://github.com/iovisor/bcc/blob/v0.17.0/src/cc/frontends/clang/kbuild_helper.cc#L172

What was the error you get with busybox's tar ?
bcc doesn't use advenced tar option that is not supported by busybox's tar.

# tar --help
BusyBox v1.31.1 (2020-05-01 00:37:37 CEST) multi-call binary.

Usage: tar c|x|t [-hvokO] [-f TARFILE] [-C DIR] [-T FILE] [-X FILE] [--exclude PATTERN]... [FILE]...

Create, extract, or list files from a tar file

c Create
x Extract
t List
-f FILE Name of TARFILE ('-' for stdin/out)
-C DIR Change to DIR before operation
-v Verbose
-O Extract to stdout
-o Don't restore user:group
-k Don't replace existing files
-h Follow symlinks
-T FILE File with names to include
-X FILE File with glob patterns to exclude
--exclude PATTERN Glob pattern to exclude

Quoted 6 lines
Quoted 3 lines
$ cd /usr/share/bcc/examples/tracing
$ ./disksnoop.py

/usr/share/bcc/tools contains actual tools. I recommend picking up an example from there.

Fixed.

Quoted 13 lines
Quoted 12 lines

V3
- Bump to version 0.16.0.
- Add required python dependency.
- Remove unnecessary and duplicated
kernel flags (#Testing section) in bcc.mk.
-------------
V2
- Add eBPF's required Kernel flags.
- Fix submodule source fetch problem.
- Add toolchain dependency.

Quoted 1 line

Did you intend to include these in the commit message?

No, it's a mistake. Fixed.

Quoted 48 lines
Quoted 48 lines
---
 DEVELOPERS                                       |  1 +
 package/Config.in                                |  1 +
 package/bcc/0001-fix-aarch64-cross-compile.patch | 65 ++++++++++++++++++++++++
 package/bcc/Config.in                            | 49 ++++++++++++++++++
 package/bcc/bcc.hash                             |  3 ++
 package/bcc/bcc.mk                               | 57 +++++++++++++++++++++
 6 files changed, 176 insertions(+)
 create mode 100644 package/bcc/0001-fix-aarch64-cross-compile.patch
 create mode 100644 package/bcc/Config.in
 create mode 100644 package/bcc/bcc.hash
 create mode 100644 package/bcc/bcc.mk
DEVELOPERS +1 -0
index 42fa5a4..bd836a6 100644
@@ -1396,6 +1396,7 @@ N:	Joshua Henderson <[email protected]>
 F:	package/qt5/qt5wayland/
 
 N:	Jugurtha BELKALEM <[email protected]>
+F:	package/bcc/
 F:	package/python-cycler/
 F:	package/python-matplotlib/
 
package/Config.in +1 -0
index d7e79f4..8ec3932 100644
@@ -83,6 +83,7 @@ menu "Compressors and decompressors"
 endmenu
 
 menu "Debugging, profiling and benchmark"
+	source "package/bcc/Config.in"
 	source "package/blktrace/Config.in"
 	source "package/bonnie/Config.in"
 	source "package/cache-calibrator/Config.in"
package/bcc/0001-fix-aarch64-cross-compile.patch new file +2 -0
new file mode 100644
index 0000000..6b42797
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <[email protected]>
Quoted 8 lines

package/bcc/0001-fix-aarch64-cross-compile.patch line 5

Quoted 3 lines
package/bcc/0001-fix-aarch64-cross-compile.patch new file +3 -0
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling

Is this luajit required to enable Lua front-end to BCC? Given python is the most common front-end, I'd argue to make this an extra option that is only enabled if the user selects it in the menu. Same way BPF backend in LLVM is only compiled in if selected. IMHO that's a better default behavior.

As far I can see in other packaging, BCC is using luajit as lua interpreter
https://src.fedoraproject.org/rpms/bcc/blob/master/f/bcc.spec

I'm agree that python is the most common front-end. I'll try to add a sub option to enable or disable it.

Quoted 69 lines
Quoted 69 lines
package/bcc/0001-fix-aarch64-cross-compile.patch new file +60 -0
+
+Unlike CMAKE_SYSTEM_PROCESSOR which identifies aarch64
+as a valid architecture, luajit does not recognize it.
+luajit defines aarch64 as arm64.
+
+LuaJIT doesn't use usual arch naming, so we have to convert
+between CMake and Luajit for each architectures while
+cross-compiling.
+
+Signed-off-by: Jugurtha BELKALEM <[email protected]>
+Signed-off-by: Romain Naour <[email protected]>
+---
+v2: Do the same for other architecture supported by LuaJIT.
+https://github.com/iovisor/bcc/pull/2480
+---
+ src/lua/CMakeLists.txt | 29 ++++++++++++++++++++++++++++-
+ 1 file changed, 28 insertions(+), 1 deletion(-)
+
+diff --git a/src/lua/CMakeLists.txt b/src/lua/CMakeLists.txt
+index 7541d48df..226e1b1d2 100644
+--- a/src/lua/CMakeLists.txt
++++ b/src/lua/CMakeLists.txt
+@@ -13,9 +13,36 @@ if (LUAJIT_LIBRARIES AND LUAJIT)
+ 		DEPENDS ${SRC_LUA} ${CMAKE_CURRENT_SOURCE_DIR}/squishy
+ 	)
+ 
++	# LuaJIT doesn't use usual arch naming, so we have to convert
++	# between CMake and Luajit while cross-compiling.
++	if (CMAKE_CROSSCOMPILING)
++		SET (LUAJIT_TARGET_ARCH "-a")
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L30
++		# https://github.com/LuaJIT/LuaJIT/blob/f0e865dd4861520258299d0f2a56491bd9d602e1/src/jit/bcsave.lua#L65
++		if (CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64_be")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm64be")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "arm")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "arm")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86)$")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x86")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mips")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mips")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "mipsel")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "mipsel")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "powerpc")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "ppc")
++		elseif (CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
++			SET (LUAJIT_TARGET_ARCH ${LUAJIT_TARGET_ARCH} "x64")
++		else()
++			MESSAGE(FATAL_ERROR "${CMAKE_SYSTEM_PROCESSOR} is not supported by LuaJIT")
++		endif()
++	endif()
++
+ 	ADD_CUSTOM_COMMAND(
+ 		OUTPUT bcc.o
+-		COMMAND ${LUAJIT} -bg bcc.lua bcc.o
++		COMMAND ${LUAJIT} -bg bcc.lua ${LUAJIT_TARGET_ARCH} bcc.o
+ 		DEPENDS bcc.lua
+ 	)
+ 
package/bcc/Config.in new file +2 -0
new file mode 100644
index 0000000..cf03927
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+	bool "bcc"
Quoted 8 lines

package/bcc/Config.in line 6

Quoted 4 lines
package/bcc/Config.in new file +4 -0
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+	depends on BR2_LINUX_KERNEL # needs kernel sources on the target

This dependency on building the kernel in buildroot is not user friendly. For example I never build the kernel in buildroot, and certainly would like to be able to enable this package without having to build the kernel in buildroot.

IIRC this option was adde because we used a previous version of BCC (0.8) with a kernel 4.x (at the time CONFIG_IKHEADERS didn't exist).
But still, if we don't enforce building a kernel to select BCC it's the user responsibility to install the kernel sources (manually, rootfsoverlay or by a post build script).

I'll remove this constrain.

Quoted 17 lines

package/bcc/Config.in line 10

Quoted 6 lines
package/bcc/Config.in new file +4 -0
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+	depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+	depends on BR2_INSTALL_LIBSTDCPP # clang
 
+	select BR2_PACKAGE_AUDIT # runtime

This..

package/bcc/Config.in line 14

Quoted 5 lines
package/bcc/Config.in new file +4 -0
+	select BR2_PACKAGE_CLANG
+	select BR2_PACKAGE_ELFUTILS
+	select BR2_PACKAGE_FLEX # needs FlexLexer.h
 
+	select BR2_PACKAGE_IPERF3 # runtime

.. this..

Quoted 6 lines

package/bcc/Config.in line 17

Quoted 4 lines
package/bcc/Config.in new file +3 -0
+	select BR2_PACKAGE_LLVM_BPF
+	select BR2_PACKAGE_LUAJIT
 
+	select BR2_PACKAGE_NETPERF # runtime

.. and this look odd to me. Can you expand which runtime requires them?

for example, netperf is one of recommanded program for BCC. They are listed in debian/control
https://github.com/iovisor/bcc/blob/v0.17.0/debian/control#L13

I don't remenber about audit, Jugurtha ?

Quoted 9 lines

package/bcc/Config.in line 19

Quoted 3 lines
package/bcc/Config.in new file +2 -0
+	select BR2_PACKAGE_PYTHON3
+	select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC

If you replace XZ with TAR, you should find that BCC automatically handles the ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel that could replace the dependency on the headers. Haven't played much with that yet though. BTF requires a tool called pahole. We can handle this support later I guess.

There is no package that provide pahole tool. Indeed this can be done later.

Quoted 25 lines
Quoted 14 lines
package/bcc/Config.in new file +13 -0
+	help
+	  BPF Compiler Collection (BCC)
+
+	  BCC is a toolkit for creating efficient kernel tracing and
+	  manipulation programs, and includes several useful tools and
+	  examples. It makes use of extended BPF (Berkeley Packet
+	  Filters), formally known as eBPF, a new feature that was
+	  first added to Linux 3.15. Much of what BCC uses requires
+	  Linux 4.1 and above.
+
+	  Note: Before using bcc, you need either need to :
+	  - For kernel_ver = [4.1, 5.2) : Copy kernel source code
+	  to target folder /lib/module/<kernel_ver>/build.

package/bcc/Config.in line 42

Quoted 10 lines
package/bcc/Config.in new file +10 -0
+	  - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
+	  and use generated headers under /sys/kernel/kheaders.tar.xz
+	  to populate /lib/module/<kernel_ver>/build.
+
+	  That's because the clang frontend build eBPF code at runtime.
+
+	  https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+	depends on !BR2_LINUX_KERNEL
Quoted 7 lines

I must remove this to compile this patch. This hard dependency is not required IMHO. You can still auto fix up the kernel if you like if the user is building the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so the set of configs might require a bit of maintenance. Given it's all documented in BCC website, I personally think it's better to leave correct kernel configuration to the user to follow the right instructions from BCC website rather than automatically handle it for them.

You mean the bcc.mk shouldn't handler kernel options for BCC ?

Ok to remove this constraint on the Linux kernel.

Quoted 51 lines
Quoted 51 lines
package/bcc/Config.in new file +7 -0
+
+comment "bcc needs a glibc toolchain w/ wchar, threads, C++, gcc >= 4.8, host gcc >= 4.8"
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+	depends on BR2_LINUX_KERNEL
+	depends on !BR2_TOOLCHAIN_USES_GLIBC || !BR2_TOOLCHAIN_HAS_THREADS \
+		|| !BR2_INSTALL_LIBSTDCPP
package/bcc/bcc.hash new file +3 -0
new file mode 100644
index 0000000..e3fa662
@@ -0,0 +1,3 @@
+# locally calculated
+sha256 38105c6c7e53d6a8d4c2be5c2e6dd46ebeaa1489600501711d85cfe9bc065e9d  bcc-v0.16.0.tar.gz
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1  LICENSE.txt
package/bcc/bcc.mk new file +28 -0
new file mode 100644
index 0000000..4fe69cb
@@ -0,0 +1,57 @@
+################################################################################
+#
+# bcc
+#
+################################################################################
+
+BCC_VERSION = v0.16.0
+BCC_SITE = https://github.com/iovisor/bcc.git
+BCC_SITE_METHOD = git
+BCC_GIT_SUBMODULES = YES
+BCC_LICENSE = Apache-2.0
+BCC_LICENSE_FILES = LICENSE.txt
+# libbcc.so and libbpf.so
+BCC_INSTALL_STAGING = YES
+BCC_DEPENDENCIES = host-bison host-flex host-luajit clang elfutils flex llvm \
+		   luajit python3
+
+# ENABLE_LLVM_SHARED=ON to use llvm.so.
+# Force REVISION otherwise bcc will use git describe to generate a version number.
+BCC_CONF_OPTS = -DENABLE_LLVM_SHARED=ON \
+	-DREVISION=$(BCC_VERSION) \
+	-DENABLE_CLANG_JIT=ON \
+	-DENABLE_MAN=OFF
+
+# BCC requires python-bcc (shiped under bcc/src/python) to work.
+# However, BCC compiles bcc/src/python/setup.py.in into bcc/src/python.
+# The generated compiled folder (bcc/src/python/bcc-python/bcc) should
+# be copied to target python's site-packages directory.
Quoted 8 lines

package/bcc/bcc.mk line 33

Quoted 5 lines
package/bcc/bcc.mk new file +5 -0
+define BCC_INSTALL_PYTHON_BCC
+	cp -r $(BUILD_DIR)/bcc-$(BCC_VERSION)/src/python/bcc-python/bcc \
+	$(TARGET_DIR)/usr/lib/python3.*/site-packages/
+endef
+BCC_POST_INSTALL_TARGET_HOOKS += BCC_INSTALL_PYTHON_BCC

I think you're hitting a bug in BCC build system which causes the package to be installed in the wrong location on the target.

I already told to Jugurtha that was a weird hack.

Quoted 2 lines

Does the attached patch help?

Thanks I'll take a look.

Quoted 27 lines
Quoted 13 lines
package/bcc/bcc.mk new file +12 -0
+
+define BCC_LINUX_CONFIG_FIXUPS
+	# Enable kernel support for eBPF
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+	# [for Linux kernel versions 4.1 through 4.6]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+	# [for Linux kernel versions 4.7 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)

package/bcc/bcc.mk line 55

Quoted 10 lines
package/bcc/bcc.mk new file +10 -0
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+	# [for Linux kernel versions 5.2 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+	# For running bcc networking examples on vanilla kernel
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+endef

This will be a big maintenance headache. New dependency can come up and configs like CONFIG_NET_* are not essential and seem to be specific to your usage.

Quoted 3 lines

I'm not in favour of them personally, but if you want them the list need to shrink to the bare minimum and depend on user building the kernel in buildroot IMO.

As you said, bcc is moving fast and requires different kernel option depending on the kernel version or tool/example that will be running on the target. Agree it's may be a big maintenance headache.

But on otherhand I find userfriendly to enable CONFIG_BPF to CONFIG_IKHEADERS option. OK to drop kernel options for networking examples.

Thanks for your review!

Best regards,
Romain

Quoted 13 lines

Thanks!

--
Qais Yousef

Quoted 7 lines
package/bcc/bcc.mk new file +2 -0
+
+$(eval $(cmake-package))

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#5 lore raw

PATCH 2/2 package/bcc: new package

Hi Romain

Quoted 3 lines · Romain Naour

Hello Qais,

Sorry for my late reply.

And sorry for my late reply too. I was off for 2 weeks.

Quoted 4 lines

Le 04/10/2020 ??? 20:12, Qais Yousef a ???crit???:

Quoted 2 lines

Hi Jugurtha

Quoted 8 lines
Quoted 5 lines

I am very late to the party and missed earlier reviews. So apologies in advance if any of my comments was already brought up/addressed.

Can you keep me on CC on future posting and any other related discussion please?

I'm reworking the series and I'm adding you in Cc for each patch. Jugurtha is not available at the moment, but he is still interested by BCC.

Great. Thanks for that. I will allocate time to review it soon.

Quoted 46 lines
Quoted 12 lines · Jugurtha BELKALEM
Quoted 9 lines

bcc is a front-end tool for eBPF :
https://github.com/iovisor/bcc/blob/master/README.md.
eBPF is the most powerful Linux tracer, and bcc
allows to write eBPF scripts in C and PYTHON3.

bcc can help to troubleshoot issues quickly on
embedded systems (as long as Linux kernel
version >= 4.1).

Quoted 8 lines
Quoted 4 lines

bcc can also make it easy to create observabilty tools, SDN configuration, ddos mitigation, intrusion detection and secure containers. More information is available at:
http://www.brendangregg.com/ebpf.html.

There's the brand new ebpf website too

	http://www.ebpf.io

Link updated, thanks!

Quoted 7 lines
Quoted 4 lines

BCC can be tested on the target :

$ mount -t debugfs none /sys/kernel/debug

I wonder if this something we can automatically append to /etc/fstab when this package is enabled..

Sure, but this is a fstab customization. I don't think the bcc package should change the /etc/fstab.

Quoted 11 lines
Quoted 6 lines
$ mkdir -p /lib/modules/KERNEL_VERSION/build
$ cd /lib/modules/KERNEL_VERSION//build
$ cp /sys/kernel/kheaders.tar.xz .
$ unxz kheaders.tar.xz
$ tar xf kheaders.tar

All the above is unnecessary. BCC automatically does this if it detects /sys/kernel/kheaders.tar.xz is present. I had to enable BR2_PACKAGE_TAR for BCC to handle this correctly. I guess busybox::tar is too limited to work with BCC commands.

Quoted 5 lines

Indeed, thanks for the info.
Now I see:
https://github.com/iovisor/bcc/blob/v0.17.0/src/cc/frontends/clang/kbuild_helper.cc#L172

What was the error you get with busybox's tar ?

Sadly I completely forgot. I can try to reproduce it. If you managed to resolve this in a different way, it works for me.

Quoted 24 lines

bcc doesn't use advenced tar option that is not supported by busybox's tar.

# tar --help
BusyBox v1.31.1 (2020-05-01 00:37:37 CEST) multi-call binary.

Usage: tar c|x|t [-hvokO] [-f TARFILE] [-C DIR] [-T FILE] [-X FILE] [--exclude PATTERN]... [FILE]...

Create, extract, or list files from a tar file

c Create
x Extract
t List
-f FILE Name of TARFILE ('-' for stdin/out)
-C DIR Change to DIR before operation
-v Verbose
-O Extract to stdout
-o Don't restore user:group
-k Don't replace existing files
-h Follow symlinks
-T FILE File with names to include
-X FILE File with glob patterns to exclude
--exclude PATTERN Glob pattern to exclude

[...]

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines

menu "Debugging, profiling and benchmark"

package/Config.in +1 -0
+	source "package/bcc/Config.in"
 	source "package/blktrace/Config.in"
 	source "package/bonnie/Config.in"
 	source "package/cache-calibrator/Config.in"
package/bcc/0001-fix-aarch64-cross-compile.patch new file +2 -0
new file mode 100644
index 0000000..6b42797
@@ -0,0 +1,65 @@
+From 5a5b0f04484e00c88e7be902101367d6d591fb96 Mon Sep 17 00:00:00 2001
+From: Jugurtha BELKALEM <[email protected]>
Quoted 8 lines

package/bcc/0001-fix-aarch64-cross-compile.patch line 5

Quoted 3 lines
package/bcc/0001-fix-aarch64-cross-compile.patch new file +3 -0
+Date: Thu, 2 May 2019 11:06:23 +0200
+Subject: [PATCH] cmake/luajit: Provide the target architecture to luaJIT while
+ cross-compiling

Is this luajit required to enable Lua front-end to BCC? Given python is the most common front-end, I'd argue to make this an extra option that is only enabled if the user selects it in the menu. Same way BPF backend in LLVM is only compiled in if selected. IMHO that's a better default behavior.

Quoted 5 lines

As far I can see in other packaging, BCC is using luajit as lua interpreter
https://src.fedoraproject.org/rpms/bcc/blob/master/f/bcc.spec

I'm agree that python is the most common front-end. I'll try to add a sub option to enable or disable it.

I think a sub option is the better way to go. Thanks!

[...]

Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
package/bcc/Config.in new file +2 -0
new file mode 100644
index 0000000..cf03927
@@ -0,0 +1,49 @@
+config BR2_PACKAGE_BCC
+	bool "bcc"
Quoted 8 lines

package/bcc/Config.in line 6

Quoted 4 lines
package/bcc/Config.in new file +4 -0
+	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
+	depends on BR2_TOOLCHAIN_USES_GLIBC # hardcode GNU tuple (x86_64-unknown-linux-gnu)
+	depends on BR2_LINUX_KERNEL # needs kernel sources on the target

This dependency on building the kernel in buildroot is not user friendly. For example I never build the kernel in buildroot, and certainly would like to be able to enable this package without having to build the kernel in buildroot.

Quoted 7 lines

IIRC this option was adde because we used a previous version of BCC (0.8) with a kernel 4.x (at the time CONFIG_IKHEADERS didn't exist).
But still, if we don't enforce building a kernel to select BCC it's the user responsibility to install the kernel sources (manually, rootfsoverlay or by a post build script).

I'll remove this constrain.

Thanks. IMO relying on users reading the documentation to enable kernel support is a better option.

Quoted 25 lines
Quoted 18 lines

package/bcc/Config.in line 10

Quoted 7 lines
package/bcc/Config.in new file +4 -0
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # clang
+	depends on BR2_TOOLCHAIN_HAS_THREADS # clang
+	depends on BR2_INSTALL_LIBSTDCPP # clang
 
+	select BR2_PACKAGE_AUDIT # runtime

This..

package/bcc/Config.in line 14

Quoted 5 lines
package/bcc/Config.in new file +4 -0
+	select BR2_PACKAGE_CLANG
+	select BR2_PACKAGE_ELFUTILS
+	select BR2_PACKAGE_FLEX # needs FlexLexer.h
 
+	select BR2_PACKAGE_IPERF3 # runtime

.. this..

Quoted 6 lines

package/bcc/Config.in line 17

Quoted 4 lines
package/bcc/Config.in new file +3 -0
+	select BR2_PACKAGE_LLVM_BPF
+	select BR2_PACKAGE_LUAJIT
 
+	select BR2_PACKAGE_NETPERF # runtime

.. and this look odd to me. Can you expand which runtime requires them?

Quoted 5 lines

for example, netperf is one of recommanded program for BCC. They are listed in debian/control
https://github.com/iovisor/bcc/blob/v0.17.0/debian/control#L13

I don't remenber about audit, Jugurtha ?

I think this is a specific use case for a class of people who would like to use BCC. I don't think we should impose netperf on them as a requirement. They can enable it independently if they want it, no?

If you see a big value in enabling these packages automatically, can we hide them behind another sub option at least?

Image bloat is a problem I faced in the past, and being able to have the bare minimum only is always good to have.

Quoted 11 lines
Quoted 10 lines

package/bcc/Config.in line 19

Quoted 4 lines
package/bcc/Config.in new file +2 -0
+	select BR2_PACKAGE_PYTHON3
+	select BR2_PACKAGE_XZ # Decompress kernel headers required by BCC

If you replace XZ with TAR, you should find that BCC automatically handles the ikheader.tar.xz for you. There's now BPF Type Format (BTF) option in the kernel that could replace the dependency on the headers. Haven't played much with that yet though. BTF requires a tool called pahole. We can handle this support later I guess.

Quoted 1 line

There is no package that provide pahole tool. Indeed this can be done later.

+1

Quoted 34 lines
Quoted 26 lines
Quoted 15 lines
package/bcc/Config.in new file +13 -0
+	help
+	  BPF Compiler Collection (BCC)
+
+	  BCC is a toolkit for creating efficient kernel tracing and
+	  manipulation programs, and includes several useful tools and
+	  examples. It makes use of extended BPF (Berkeley Packet
+	  Filters), formally known as eBPF, a new feature that was
+	  first added to Linux 3.15. Much of what BCC uses requires
+	  Linux 4.1 and above.
+
+	  Note: Before using bcc, you need either need to :
+	  - For kernel_ver = [4.1, 5.2) : Copy kernel source code
+	  to target folder /lib/module/<kernel_ver>/build.

package/bcc/Config.in line 42

Quoted 10 lines
package/bcc/Config.in new file +10 -0
+	  - Or kernel_ver >= 5.2 : Compile kernel with CONFIG_IKHEADERS
+	  and use generated headers under /sys/kernel/kheaders.tar.xz
+	  to populate /lib/module/<kernel_ver>/build.
+
+	  That's because the clang frontend build eBPF code at runtime.
+
+	  https://github.com/iovisor/bcc
+
+comment "bcc needs a Linux kernel to be built"
+	depends on !BR2_LINUX_KERNEL
Quoted 7 lines

I must remove this to compile this patch. This hard dependency is not required IMHO. You can still auto fix up the kernel if you like if the user is building the kernel and has enabled BCC. Just keep in mind it's a fast moving target, so the set of configs might require a bit of maintenance. Given it's all documented in BCC website, I personally think it's better to leave correct kernel configuration to the user to follow the right instructions from BCC website rather than automatically handle it for them.

Quoted 1 line

You mean the bcc.mk shouldn't handler kernel options for BCC ?

Yep.

Quoted 2 lines

Ok to remove this constraint on the Linux kernel.

Thanks!

[...]

Quoted 31 lines
Quoted 27 lines
Quoted 13 lines
package/bcc/Config.in new file +12 -0
+
+define BCC_LINUX_CONFIG_FIXUPS
+	# Enable kernel support for eBPF
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_SYSCALL)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_CLS_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_BPF)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_JIT)
+	# [for Linux kernel versions 4.1 through 4.6]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_BPF_JIT)
+	# [for Linux kernel versions 4.7 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_HAVE_EBPF_JIT)

package/bcc/Config.in

Quoted 10 lines
package/bcc/Config.in new file +10 -0
+	$(call KCONFIG_ENABLE_OPT,CONFIG_BPF_EVENTS)
+	# [for Linux kernel versions 5.2 and later]
+	$(call KCONFIG_ENABLE_OPT,CONFIG_IKHEADERS)
+	# For running bcc networking examples on vanilla kernel
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_SCH_SFQ)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_POLICE)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_NET_ACT_GACT)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_DUMMY)
+	$(call KCONFIG_ENABLE_OPT,CONFIG_VXLAN)
+endef

This will be a big maintenance headache. New dependency can come up and configs like CONFIG_NET_* are not essential and seem to be specific to your usage.

Quoted 3 lines

I'm not in favour of them personally, but if you want them the list need to shrink to the bare minimum and depend on user building the kernel in buildroot IMO.

Quoted 8 lines

As you said, bcc is moving fast and requires different kernel option depending on the kernel version or tool/example that will be running on the target. Agree it's may be a big maintenance headache.

But on otherhand I find userfriendly to enable CONFIG_BPF to CONFIG_IKHEADERS option. OK to drop kernel options for networking examples.

Thanks for your review!

Thanks for taking care of the issues and sending a new version too! I'll review as soon as I can.

Cheers

--
Qais Yousef